Method, medium and system for detecting biological activity in tumor section preservation process
By combining high-definition images and microscopic images, using multi-scale feature fusion structures, the problem that multi-level structure and functional changes in tumor section preservation is difficult to reflect, and the accurate evaluation of the biological activity of the section is achieved, and the accuracy of detection is improved. and reliability.
Patent Information
- Application Number
- CN202410993894.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-24
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2044-07-24
AI Technical Summary
The existing tumor section preservation methods are difficult to fully reflect the multi-level structure and functional changes of the section during the preservation process, and it is difficult to accurately predict the level of biological activity in a single-scale feature analysis.
Using a combination of high-definition images and microscopic images, key points are obtained through scale-invariant feature transformation to form a microscopic image set, and multi-scale feature fusion structures are used, including high-definition feature subnetwork, micromacro feature subnetwork, and microscopic Micro feature subnetwork and microtextured feature subnetworks fusion features through a multi-layer perceptron network to output the current biological activity vector of the slice.
A comprehensive evaluation of multi-level structure and functional changes during tumor section preservation is achieved, and the accuracy and reliability of biological activity detection is improved.
Smart Images

Figure CN118982828A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of tumor slice preservation, and in particular, relates to a method, medium and system for detecting biological activity in a tumor slice preservation process. Background Art
[0002] In recent years, tumor slice specimens have played an increasingly important role in pathological diagnosis, drug screening and other fields. These tumor slice samples usually need to be stored for a long time under specific storage conditions to ensure that their biological activity does not change significantly during subsequent testing and analysis. Existing common tumor slice preservation methods include cryopreservation, chemical fixation and other methods, but these methods often have different degrees of impact on the microstructure and physiological function of the slices. Therefore, how to accurately evaluate the changes in the biological activity of tumor slices during the preservation process has become one of the key issues that need to be solved in this field.
[0003] At present, some studies have attempted to use high-definition imaging and microscopic imaging technology to quantitatively analyze the structural and functional characteristics of tumor slices, and to evaluate the changes in their biological activity during the preservation process. For example, some scholars have proposed to reflect the changes in the overall structure of the slice by analyzing the texture characteristics and shape characteristics of high-definition images of tumor slices. There are also studies that use microscopic imaging technology to collect dynamic change data on the microscopic cell structure of slices in order to reveal the evolution of their functional characteristics. However, these methods usually only focus on the characteristics of a single scale, and it is difficult to fully reflect the multi-level structural and functional changes of slices during the preservation process. In addition, due to the large differences between different slice samples in the initial state and during the preservation process, a single feature analysis method is often difficult to accurately predict their current level of biological activity. Summary of the invention
[0004] In view of this, the present invention provides a method, medium and system for detecting biological activity during the preservation process of tumor slices, which can solve the technical problem that existing image analysis-based methods are difficult to fully reflect the multi-level structural and functional changes of tumor slices during the preservation process.
[0005] The present invention is achieved in that:
[0006] The first aspect of the present invention provides a method for detecting biological activity during the preservation process of tumor slices, which comprises the following steps:
[0007] S10, obtaining a high-definition image of the tumor slice just placed in the storage device, and obtaining a first high-definition image after preprocessing
[0008] S20, according to the first high-definition image, using a scale-invariant feature transformation method to obtain multiple key points, and obtain a microscopic image of each key point to form a first microscopic image set, including multiple first microscopic images;
[0009] S30, obtaining a high-definition image of the tumor slice at the time to be detected, obtaining a second high-definition image after preprocessing, and obtaining corresponding microscopic images according to the multiple key points to form a second microscopic image set, including multiple second microscopic images;
[0010] S40, aligning the first high-definition image with the second high-definition image, and aligning the first microscopic image with the second microscopic image of each key point;
[0011] S50, respectively extracting features of the aligned first high-definition image and the second high-definition image, recorded as first high-definition features and second high-definition features; respectively extracting features of the first microscopic image and the second microscopic image of each key point, recorded as first microscopic features and second microscopic features;
[0012] S60, inputting the first high-definition feature and the second high-definition feature, and the first microscopic feature and the second microscopic feature of each key point into a pre-trained tumor slice preservation process biological activity detection model to obtain and output a current activity vector of the tumor slice.
[0013] Among them, the biological activity detection model of the tumor slice preservation process adopts a multi-scale feature fusion structure, including a high-definition feature sub-network, a microscopic-macroscopic feature sub-network, a microscopic-microscopic feature sub-network, a microscopic-texture feature sub-network and a fusion sub-network.
[0014] Furthermore, the high-definition feature subnetwork is used to extract the overall features of the high-definition image, its input is the first high-definition feature and the second high-definition feature, and its output is the high-definition image feature change vector; its structure is a convolutional neural network based on ResNet50.
[0015] Furthermore, the microscopic and macroscopic feature subnetwork is used to extract the macroscopic structural features of the microscopic image, and its input is the first microscopic feature and the second microscopic feature, and its output is the microscopic and macroscopic feature change vector; its structure is a convolutional neural network based on VGG16.
[0016] Furthermore, the microscopic feature subnetwork is used to extract microscopic cell features of the microscopic image, and its input is the first microscopic feature and the second microscopic feature, and its output is the microscopic feature change vector; its structure is a convolutional neural network based on DenseNet121.
[0017] Furthermore, the micro-texture feature sub-network is used to extract texture features of microscopic images, its input is the first microscopic feature and the second microscopic feature, and its output is the microscopic texture feature change vector; its structure is a feature extraction network based on gray-level co-occurrence matrix and local binary pattern.
[0018] Furthermore, the fusion subnetwork is used to fuse the features extracted by each subnetwork, and its input is the high-definition image feature change vector, the microscopic-macroscopic feature change vector, the microscopic-microscopic feature change vector and the microscopic-texture feature change vector, and its output is the current activity vector of the tumor slice; its structure is a multi-layer perceptron network.
[0019] The following are the steps to establish the training data set for the biological activity detection model during tumor slice preservation:
[0020] 1. Sample collection: Collect tumor slice samples of different types and stages, and take high-definition images and microscopic images of each sample at different storage time points (such as 0h, 2h, 4h, 6h...72h, etc.);
[0021] 2. Data preprocessing: standardize and denoise the high-definition images, extract key points using the SIFT algorithm, and obtain microscopic images of the key points;
[0022] 3. Feature extraction: extract high-definition image features, extract macroscopic structural features, microscopic cell features and texture features of microscopic images;
[0023] 4. Activity annotation: Use experimental methods to evaluate the activity of slice samples at each time point and convert the evaluation results into numerical activity vectors;
[0024] 5. Data enhancement: Rotate, flip, and scale the image to expand the data set, and use GAN and other technologies to generate synthetic samples to further increase data diversity;
[0025] 6. Dataset division: The processed data is divided into training set, validation set and test set in the ratio of 7:2:1.
[0026] Furthermore, the preprocessing includes image normalization, image enhancement, and denoising.
[0027] Specifically, the step S10 specifically includes:
[0028] Step 101: Collect a tumor slice sample and place it in a dedicated storage device. Use a high-definition camera to immediately capture a high-definition image of the tumor slice to obtain a first high-definition image.
[0029] Step 102: Perform preprocessing operations such as image standardization, image enhancement, and denoising on the first high-definition image to improve the quality and usability of the image and obtain a final first high-definition image.
[0030] Wherein, the step S20 specifically includes:
[0031] Step 201: Use a scale-invariant feature transform (SIFT) algorithm to perform feature point detection on the first high-definition image to obtain multiple significant key points.
[0032] Step 202: For each of the key points, use a high-power microscopic imaging device to capture a local microscopic image of the tumor slice tissue at the position to form a first microscopic image set.
[0033] Wherein, the step S30 specifically includes:
[0034] Step 301: When it is necessary to detect the biological activity change of the tumor slice during the preservation process, a high-definition image of the slice is taken again using a high-definition camera to obtain a second high-definition image.
[0035] Step 302: Use the SIFT algorithm to detect key points on the second high-definition image, and use a microscopic imaging device to obtain local microscopic images of these key points to form a second microscopic image set.
[0036] Wherein, the step S40 specifically includes:
[0037] Step 401: Use the feature point-based RANSAC image registration algorithm to perform geometric correction and registration on the first high-definition image and the second high-definition image to eliminate the difference during shooting and align the two high-definition images in spatial position.
[0038] Step 402: For each key point, the corresponding first microscopic image and second microscopic image are also locally registered based on SIFT features to ensure that the microscopic images of the same key point taken at two time points are completely aligned in spatial position.
[0039] Wherein, the step S50 specifically includes:
[0040] Step 501: Use the pre-trained ResNet50 convolutional neural network to extract the overall features of the first high-definition image and the second high-definition image, respectively, and record them as the first high-definition features and the second high-definition features.
[0041] Step 502: extract the macroscopic structural features (based on VGG16), microscopic cell features (based on DenseNet121), and texture features (based on GLCM and LBP) of the first microscopic image and the second microscopic image corresponding to each key point, respectively, and record them as the first microscopic features and the second microscopic features.
[0042] Wherein, the step S60 specifically includes:
[0043] Step 601: The biological activity detection model of the tumor slice preservation process adopts a multi-scale feature fusion structure, including a high-definition feature sub-network, a microscopic and macroscopic feature sub-network, a microscopic and microscopic feature sub-network, a microscopic and texture feature sub-network, and a fusion sub-network.
[0044] Step 602, the high-definition feature sub-network is based on the ResNet50 structure and is used to extract the overall features of the high-definition image. Its input is the first high-definition feature and the second high-definition feature, and its output is the high-definition image feature change vector.
[0045] Step 603, the microscopic and macroscopic feature subnetwork is based on the VGG16 structure and is used to extract the macroscopic structural features of the microscopic image. Its input is the first microscopic feature and the second microscopic feature, and its output is the microscopic and macroscopic feature change vector.
[0046] Step 604, the microscopic feature subnetwork is based on the DenseNet121 structure and is used to extract the microscopic cell features of the microscopic image. Its input is the first microscopic feature and the second microscopic feature, and its output is the microscopic feature change vector.
[0047] Step 605: The micro-texture feature sub-network is based on GLCM and LBP and is used to extract texture features of the microscopic image. Its input is the first microscopic feature and the second microscopic feature, and its output is the microscopic texture feature change vector.
[0048] Step 606, the fusion sub-network adopts an MLP structure to fuse the features extracted by each sub-network, and its input is the high-definition image feature change vector, the microscopic macroscopic feature change vector, the microscopic microscopic feature change vector and the microscopic texture feature change vector, and the output is the current activity vector of the tumor slice.
[0049] Optionally, the high-definition camera used in step 101 includes a digital camera or a microscopic imaging system, and the image preprocessing operation in step 102 includes image normalization, histogram equalization, edge sharpening, median filtering or bilateral filtering, etc.
[0050] Optionally, the SIFT algorithm used in step 201 can stably detect key points under changing conditions such as image scale, rotation, and illumination.
[0051] Optionally, the high-definition camera device used in step 301 is of the same type as the high-definition camera device used in step 101.
[0052] Optionally, the RANSAC image registration algorithm used in step 401 can eliminate differences in translation, rotation, scaling, etc. during shooting, so that the two high-definition images are accurately aligned in spatial position.
[0053] Optionally, the microscopic image features extracted in step 502 include macroscopic structural features, microscopic cell features and texture features, which can fully reflect the multi-scale information of the sliced tissue.
[0054] Optionally, the output feature change vectors of the high-definition feature subnetwork, the microscopic-macroscopic feature subnetwork, the microscopic-microscopic feature subnetwork, and the microscopic-texture feature subnetwork can comprehensively reflect the multi-scale structural and texture change characteristics of tumor slices during the preservation process.
[0055] Optionally, the fusion sub-network adopts a multi-layer perceptron (MLP) structure, which can effectively fuse the features extracted by each sub-network and output the current biological activity vector of the tumor slice.
[0056] Optionally, steps S10 to S60 constitute a complete biological activity detection process for the tumor slice preservation process, which can comprehensively evaluate the structural and functional changes of the slices during the preservation process.
[0057] Optionally, the method can be applied to tumor slice samples of different types and stages to detect and monitor changes in their biological activity during the preservation process.
[0058] Optionally, the training data collected by the method include high-definition images and microscopic images of tumor slices of different types and at different preservation time points, and the activity at each time point is experimentally evaluated and labeled to establish a representative training data set; the training data set is further increased in data diversity and generalization capability by data enhancement operations such as rotation, flipping, and scaling, as well as by using technologies such as generative adversarial networks (GANs) to generate synthetic samples.
[0059] A second aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores program instructions, and when the program instructions are executed, they are used to execute the above-mentioned method for detecting biological activity in a tumor slice preservation process.
[0060] A third aspect of the present invention provides a biological activity detection system for a tumor slice preservation process, which includes the above-mentioned computer-readable storage medium.
[0061] Compared with the prior art, the beneficial effects of the method, medium and system for detecting biological activity during the preservation process of tumor slices provided by the present invention are: 1) The method makes full use of information at two different scales, namely high-definition images and microscopic images, and can comprehensively reflect the multi-level structural and functional change characteristics of the slices during the preservation process, which is more accurate and reliable than feature analysis at a single scale.
[0062] 2) This method uses key point detection and local microscopic image acquisition technology based on the SIFT algorithm, which can accurately align high-definition images and microscopic images taken at different time points, laying the foundation for subsequent feature comparison and analysis.
[0063] 3) This method constructs a multi-scale feature fusion model that includes high-definition feature sub-network, microscopic-macroscopic feature sub-network, microscopic-microscopic feature sub-network and microscopic-texture feature sub-network. It can effectively extract and fuse various features and output the current biological activity level of the slice, providing a reliable basis for slice preservation and management.
[0064] 4) This method collected a wealth of tumor slice sample data, including high-definition images and microscopic images of different types and different preservation time points, as well as corresponding experimental activity evaluation results, providing sufficient support for model training. At the same time, the generalization ability of the model was further improved through data enhancement and other means.
[0065] In general, the method for detecting biological activity during the preservation of tumor slices proposed in the present invention can comprehensively evaluate the multi-scale structural and functional changes of slices during the preservation process, and has achieved significant improvements in accuracy and reliability compared to the existing technology. It solves the technical problem that the existing image analysis-based methods are difficult to fully reflect the multi-level structural and functional changes of tumor slices during the preservation process. BRIEF DESCRIPTION OF THE DRAWINGS
[0066] Figure 1 A flow chart of the method provided by the present invention. DETAILED DESCRIPTION
[0067] In order to make the purpose, technical solution and advantages of the embodiments of the present invention more clear, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention.
[0068] like Figure 1 FIG. 1 is a flow chart of a method for detecting biological activity during tumor slice preservation provided by the present invention. The method comprises the following steps:
[0069] S10, obtaining a high-definition image of the tumor slice just placed in the storage device, and obtaining a first high-definition image after preprocessing
[0070] S20, according to the first high-definition image, using a scale-invariant feature transformation method to obtain multiple key points, and obtain a microscopic image of each key point to form a first microscopic image set, including multiple first microscopic images;
[0071] S30, obtaining a high-definition image of the tumor slice at the time to be detected, obtaining a second high-definition image after preprocessing, and obtaining corresponding microscopic images according to multiple key points to form a second microscopic image set, including multiple second microscopic images;
[0072] S40, aligning the first high-definition image with the second high-definition image, and aligning the first microscopic image with the second microscopic image of each key point;
[0073] S50, respectively extracting features of the aligned first high-definition image and the second high-definition image, recorded as first high-definition features and second high-definition features; respectively extracting features of the first microscopic image and the second microscopic image of each key point, recorded as first microscopic features and second microscopic features;
[0074] S60, inputting the first high-definition feature and the second high-definition feature, and the first microscopic feature and the second microscopic feature of each key point into a pre-trained tumor slice preservation process biological activity detection model, obtaining a current activity vector of the tumor slice and outputting it.
[0075] The specific implementation methods of the above steps are described in detail below:
[0076] Step S10: Obtain a high-definition image of the tumor slice just after it is placed in the storage device, and obtain the first high-definition image after preprocessing
[0077] The specific implementation is as follows:
[0078] 1) Collect tumor slice samples and place them in a dedicated storage device. Use a high-definition camera (such as a high-resolution digital camera or a microscopic imaging system) to immediately capture a high-definition image of the tumor slice. The image obtained in this step is recorded as the first high-definition image.
[0079] 2) Perform a series of preprocessing operations on the first high-definition image, including: image standardization (such as adjusting to a uniform size and color channel), image enhancement (such as histogram equalization, edge sharpening), denoising (such as median filtering, bilateral filtering), etc., to improve the quality and usability of the image. After preprocessing, the final first high-definition image is obtained.
[0080] Step S20: Based on the first high-definition image, a plurality of key points are obtained by using a scale-invariant feature transformation method, and a microscopic image of each key point is obtained to form a first microscopic image set, including a plurality of first microscopic images.
[0081] The specific implementation is as follows:
[0082] 1) Use the scale-invariant feature transform (SIFT) algorithm to detect feature points on the first high-definition image and obtain multiple significant key points. The SIFT algorithm can stably detect key points under changing conditions such as image scale, rotation, and lighting. These key points often correspond to important structural features in the image.
[0083] 2) For each detected key point, a high-power microscopic imaging device (such as an optical microscope, an electron microscope, etc.) is used to capture a local microscopic image of the tumor slice tissue at that location. These microscopic images constitute a first microscopic image set.
[0084] Step S30: Obtain a high-definition image of the tumor slice at the time of detection, obtain a second high-definition image after preprocessing, and obtain corresponding microscopic images according to the multiple key points to form a second microscopic image set, including multiple second microscopic images
[0085] The specific implementation is as follows:
[0086] 1) When it is necessary to detect the biological activity changes of the tumor slice during the preservation process, use the high-definition camera again to take a high-definition image of the slice. The image obtained in this step is recorded as the second high-definition image.
[0087] 2) Similar to step S20, the SIFT algorithm is also used to detect key points on the second high-definition image, and the local microscopic images at these key points are obtained using a microscopic imaging device. These microscopic images constitute the second microscopic image set. It should be noted that the key point positions in the second microscopic image set should be consistent with the key point positions in step S20 to facilitate subsequent comparative analysis.
[0088] Step S40: aligning the first high-definition image with the second high-definition image, and aligning the first microscopic image with the second microscopic image of each key point
[0089] The specific implementation is as follows:
[0090] 1) Using a feature point-based image registration algorithm, such as the RANSAC algorithm, the first high-definition image and the second high-definition image are geometrically corrected and registered to eliminate differences in translation, rotation, scaling, etc. during shooting, so that the two high-definition images are aligned in space.
[0091] 2) For each key point, the corresponding first microscopic image and second microscopic image also need to be registered. This can be done by using a local registration algorithm based on image features, such as a local feature matching method based on scale-invariant feature transform (SIFT). Through this step, it is ensured that the microscopic images of the same key point taken at two time points are completely aligned in spatial position.
[0092] Step S50: extract the features of the aligned first high-definition image and the second high-definition image respectively, and record them as the first high-definition feature and the second high-definition feature; extract the features of the first microscopic image and the second microscopic image of each key point respectively, and record them as the first microscopic feature and the second microscopic feature
[0093] The specific implementation is as follows:
[0094] 1) For the first high-definition image and the second high-definition image, use a pre-trained convolutional neural network (such as ResNet50) to extract their overall features. These features are recorded as the first high-definition feature and the second high-definition feature.
[0095] 2) For each key point corresponding to the first microscopic image and the second microscopic image, extract its macroscopic structural features (such as convolutional features based on VGG16), microscopic cell features (such as features based on DenseNet121), and texture features (such as features based on gray-level co-occurrence matrix and local binary pattern). These features are recorded as the first microscopic features and the second microscopic features.
[0096] Step S60: Input the first high-definition feature and the second high-definition feature, and the first microscopic feature and the second microscopic feature of each key point into a pre-trained tumor slice preservation process biological activity detection model to obtain the current activity vector of the tumor slice and output it.
[0097] The specific implementation is as follows:
[0098] 1) The biological activity detection model of the tumor slice preservation process adopts a multi-scale feature fusion structure, including a high-definition feature sub-network, a microscopic-macroscopic feature sub-network, a microscopic-microscopic feature sub-network, a microscopic-texture feature sub-network and a fusion sub-network.
[0099] 2) The high-definition feature sub-network is based on the ResNet50 structure and is used to extract the overall features of the high-definition image. Its input is the first high-definition feature and the second high-definition feature, and its output is the high-definition image feature change vector.
[0100] 3) The microscopic and macroscopic feature subnetwork is based on the VGG16 structure and is used to extract the macroscopic structural features of microscopic images. Its input is the first microscopic feature and the second microscopic feature, and its output is the microscopic and macroscopic feature change vector.
[0101] 4) The microscopic feature subnetwork is based on the DenseNet121 structure and is used to extract the microscopic cell features of microscopic images. Its input is the first microscopic feature and the second microscopic feature, and its output is the microscopic feature change vector.
[0102] 5) The micro-texture feature sub-network is a feature extraction network based on gray-level co-occurrence matrix (GLCM) and local binary pattern (LBP), which is used to extract the texture features of microscopic images. Its input is the first microscopic feature and the second microscopic feature, and its output is the microscopic texture feature change vector.
[0103] 6) The fusion sub-network adopts a multi-layer perceptron (MLP) structure to fuse the features extracted by each sub-network. Its input is the high-definition image feature change vector, microscopic-macroscopic feature change vector, microscopic-microscopic feature change vector and microscopic-texture feature change vector, and its output is the current activity vector of the tumor slice.
[0104] A second aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores program instructions, and when the program instructions are executed, they are used to execute the above-mentioned method for detecting biological activity in a tumor slice preservation process.
[0105] A third aspect of the present invention provides a biological activity detection system for a tumor slice preservation process, which includes the above-mentioned computer-readable storage medium.
[0106] Specifically, the principle of the present invention is:
[0107] First, the method uses a high-definition camera to obtain a high-definition image of the slice at the initial moment when the slice is placed in the preservation device, and uses the SIFT algorithm to detect multiple significant key points on the high-definition image. For each key point, a microscopic imaging device is also used to capture a local microscopic image of the slice tissue at that position to form an initial high-definition image set and microscopic image set. These image data in the initial state record the overall structural characteristics and microscopic cell characteristics of the slice.
[0108] When it is necessary to detect the changes in the biological activity of the slice during the preservation process, this method collects the high-definition image and microscopic image of the slice at the time to be detected again. By aligning the high-definition image and microscopic image at these two time points, it can be ensured that the corresponding key points and local areas are completely consistent in spatial position.
[0109] Then, the method extracts high-definition image features and microscopic image features in the initial state and the state to be tested, respectively. Among them, the high-definition image features include overall structural features, and the microscopic image features include multi-scale information such as macroscopic structural features, microscopic cell features, and texture features. The changes in these features reflect the multi-level structural and functional changes of the slices during the preservation process.
[0110] Finally, the method inputs the extracted high-definition image feature changes and microscopic image feature changes into a pre-trained multi-scale feature fusion model. The model learns the mapping relationship between feature changes and biological activity through deep learning. Therefore, the output of the model is the current biological activity level of the slice, which provides an important reference for subsequent slice preservation management.
[0111] The above technical effects can be achieved mainly due to the innovations of this method in several key aspects: 1) It makes full use of information at both high-definition image and microscopic image scales to achieve a comprehensive characterization of the slice structure and function; 2) It adopts precise image registration technology to ensure the temporal correspondence of features; 3) It constructs an analysis model that integrates multi-scale feature extraction and deep feature fusion to improve the accuracy of activity evaluation.
[0112] In summary, the method for detecting biological activity during the preservation process of tumor slices proposed in the present invention can make full use of image analysis and deep learning technology to comprehensively evaluate the multi-scale structural and functional changes of slices during the preservation process.
[0113] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.
Claims
1. A method for detecting biological activity during the preservation of tumor slices, characterized in that: The following steps are involved: S10, obtaining a high-definition image of the tumor slice just placed in the storage device, and obtaining a first high-definition image after preprocessing S20, according to the first high-definition image, using a scale-invariant feature transformation method to obtain multiple key points, and obtain a microscopic image of each key point to form a first microscopic image set, including multiple first microscopic images; S30, obtaining a high-definition image of the tumor slice at the time to be detected, obtaining a second high-definition image after preprocessing, and obtaining corresponding microscopic images according to the multiple key points to form a second microscopic image set, including multiple second microscopic images; S40, aligning the first high-definition image with the second high-definition image, and aligning the first microscopic image with the second microscopic image of each key point; S50, extracting features of the aligned first high-definition image and the second high-definition image respectively, and recording them as first high-definition features and second high-definition features; Extract the features of the first microscopic image and the second microscopic image of each key point respectively, and record them as the first microscopic feature and the second microscopic feature; S60, inputting the first high-definition feature and the second high-definition feature, and the first microscopic feature and the second microscopic feature of each key point into a pre-trained tumor slice preservation process biological activity detection model to obtain and output a current activity vector of the tumor slice.
2. A method for detecting biological activity during tumor slice preservation according to claim 1, characterized in that: The tumor slice preservation process biological activity detection model adopts a multi-scale feature fusion structure, including a high-definition feature sub-network, a microscopic and macroscopic feature sub-network, a microscopic and microscopic feature sub-network, a microscopic and texture feature sub-network and a fusion sub-network.
3. A method for detecting biological activity during tumor slice preservation according to claim 2, characterized in that: The high-definition feature sub-network is used to extract the overall features of the high-definition image, the input of which is the first high-definition feature and the second high-definition feature, and the output is the high-definition image feature change vector; Its structure is a convolutional neural network based on ResNet50.
4. A method for detecting biological activity during tumor slice preservation according to claim 3, characterized in that: The microscopic and macroscopic feature subnetwork is used to extract the macroscopic structural features of the microscopic image, and its input is the first microscopic feature and the second microscopic feature, and its output is the microscopic and macroscopic feature change vector; Its structure is a convolutional neural network based on VGG16.
5. A method for detecting biological activity during tumor slice preservation according to claim 4, characterized in that: The microscopic feature subnetwork is used to extract microscopic cell features of a microscopic image, the input of which is the first microscopic feature and the second microscopic feature, and the output is a microscopic feature change vector; Its structure is a convolutional neural network based on DenseNet121.
6. A method for detecting biological activity during tumor slice preservation according to claim 5, characterized in that: The micro-texture feature sub-network is used to extract texture features of microscopic images, and its input is the first microscopic feature and the second microscopic feature, and its output is the micro-texture feature change vector; its structure is a feature extraction network based on gray-level co-occurrence matrix and local binary pattern.
7. A method for detecting biological activity during tumor slice preservation according to claim 6, characterized in that: The fusion subnetwork is used to fuse the features extracted by each subnetwork, and its input is the high-definition image feature change vector, the microscopic macroscopic feature change vector, the microscopic microscopic feature change vector and the microscopic texture feature change vector, and its output is the current activity vector of the tumor slice; Its structure is a multi-layer perceptron network.
8. The method for detecting biological activity during tumor slice preservation according to claim 7, characterized in that: The preprocessing includes image normalization, image enhancement and denoising.
9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program instructions, and when the program instructions are executed, they are used to execute the method for detecting biological activity in a tumor slice preservation process according to any one of claims 1 to 8.
10. A biological activity detection system for tumor slice preservation process, characterized in that: A computer-readable storage medium comprising the computer-readable storage medium of claim 9.
Citation Information
Patent Citations
Intelligent cancer diagnosis method based on multi-modal microscopic imaging and deep learning
CN113935964A
Focus detection method, device and equipment based on visual angle decoupling Transform model and medium
CN114549462A
Single-stage cell nucleus instance segmentation method for medical microscopic image
CN116309545A
Image fusion method for operating microscope
CN117474777A
Pathological image segmentation model and method based on multi-scale hybrid analysis
CN117853502A
