Pathological analysis method, device and equipment for thyroid cancer lymph node metastasis and medium

Through the Inception-v4 neural network model and weakly supervised multi-example learning module, the pathological images of thyroid cancer tissue are automatically analyzed, solving the problem that traditional methods are difficult to identify lymph node metastasis, and achieving efficient and accurate pathological diagnosis.

CN118918091APending Publication Date: 2024-11-08THE FIRST AFFILIATED HOSPITAL OF SUN YAT SEN UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411057012.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-02
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

Traditional pathological methods are difficult to accurately identify lymph node metastasis in thyroid cancer tissues, resulting in inaccurate diagnosis and poor patient prognosis.

Method used

Using deep learning method based on Inception-v4 neural network model, preprocessing, cutting and analyzing the HE-stained frozen pathological section images of thyroid cancer tissues, combined with weakly supervised multi-example learning modules, automatically extracting and identifying lymph node metastasis features in the image. , generate pathological analysis results.

Benefits of technology

It improves the efficiency and accuracy of pathological analysis, reduces artificial deviations, provides more reliable diagnostic results, and helps clinical treatment make scientific decisions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118918091A_ABST
    Figure CN118918091A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field related to biological medicine, in particular to a pathological analysis method, device and equipment for thyroid cancer lymph node metastasis and a medium. The method comprises the following steps: acquiring a scanning image of an HE dyed frozen pathological section of a thyroid cancer tissue, and preprocessing the scanning image; cutting the scanned image into a plurality of sub-pictures; for each sub-picture, inputting the sub-picture into a preset neural network model to obtain a sub-pathology analysis result; wherein the neural network model takes a product neural network Inception-v4 as a trunk, is embedded with a weak supervision multi-example learning module, and is used for carrying out pathological analysis on the sub-pictures to obtain a sub-pathological analysis result for representing whether the thyroid cancer lymph nodes are metastatic or not; and summarizing the sub-pathology analysis results to obtain a target pathology analysis result.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of biomedicine-related technologies, and specifically to a pathological analysis method, device, equipment and medium for thyroid cancer lymph node metastasis. Background Art

[0002] Thyroid cancer is the malignant tumor with the fastest growing incidence rate in the world in the past three decades. The incidence rate of thyroid cancer in my country is also showing a rapid upward trend. More than 50% of thyroid cancer patients have lymph node metastasis at the time of diagnosis. Patients with lymph node metastasis have a high postoperative recurrence rate, resulting in poor prognosis. Accurately identifying thyroid cancer patients with lymph node metastasis in clinical practice, so as to carry out reasonable and thorough lymph node dissection and scientific postoperative stratified management, is of great significance to improving patient prognosis.

[0003] Traditional pathological analysis is performed by pathologists using microscopes or digital pathology analyzers based on prior knowledge. The image information obtained by the naked eye is limited, and the biological information contained in the pathological images cannot be fully identified. It is difficult to analyze whether there is lymph node metastasis from the pathological images of thyroid cancer tissue. Summary of the invention

[0004] In view of this, the embodiments of the present application are committed to providing a pathological analysis method, device, equipment and medium for thyroid cancer lymph node metastasis, so as to perform data security analysis more comprehensively and effectively.

[0005] The present application provides a pathological analysis method for thyroid cancer lymph node metastasis, comprising:

[0006] Obtaining a scanned image of a HE-stained frozen pathological section of a thyroid cancer tissue, and preprocessing the scanned image;

[0007] Cut the scanned image into multiple sub-images

[0008] For each sub-image, input the sub-image into a preset neural network model to obtain a sub-pathology analysis result;

[0009] The neural network model is based on the inception neural network Inception-v4 as the main body, and is embedded with a weakly supervised multi-instance learning module, which is used to perform pathological analysis on sub-images to obtain sub-pathological analysis results for characterizing whether thyroid cancer lymph nodes have metastasized;

[0010] The sub-pathology analysis results are summarized to obtain the target pathology analysis result.

[0011] In some embodiments, the preprocessing of the scanned image includes:

[0012] Eliminate the background area in the scanned image; or,

[0013] removing spots from the scanned image; or,

[0014] The scanned image is subjected to color standardization processing.

[0015] In some embodiments, the aggregating the sub-pathology analysis results to obtain a target pathology analysis result includes:

[0016] Calculating the mean of the sub-pathological analysis;

[0017] The mean value is the target pathology analysis result.

[0018] In some embodiments, the scanned images include: slice pathology images at different magnifications.

[0019] In some embodiments, the neural network model includes: a feature extraction network and a pathology analysis network;

[0020] The feature extraction network is used to extract features of the input sub-image and send the features to the pathology analysis network;

[0021] The pathological analysis network is used to obtain a sub-pathological analysis result for characterizing whether thyroid cancer lymph nodes have metastasized based on the characteristics;

[0022] The feature extraction network is a neural network with Inception-v4 as the backbone and a weakly supervised multi-instance learning module embedded in it.

[0023] In some embodiments, the training process of the neural network model includes:

[0024] Obtain a scanned image sample;

[0025] Generate training samples based on the scanned image samples and the transfer results corresponding to each scanned image sample;

[0026] Based on the training samples, weakly supervised multi-example training is performed on the pre-built feature extraction network and pathology analysis network to obtain a neural network model.

[0027] In some embodiments, the scanned image samples include: slice pathology images at different magnifications:

[0028] The generating of training samples comprises:

[0029] For each scanned image sample; preprocess the slice pathology images at different magnifications respectively; perform image segmentation on the preprocessed slice pathology images at different magnifications respectively to obtain image block samples; extract the image block samples corresponding to different magnifications respectively to obtain an image package composed of the image block samples corresponding to different magnifications; set a label for the image package based on the transfer result corresponding to the scanned image sample to obtain a training sample.

[0030] The present application also provides a pathological analysis method and device for thyroid cancer lymph node metastasis, comprising:

[0031] An acquisition module is used to acquire a scanned image of a HE-stained frozen pathological section of a thyroid cancer tissue and pre-process the scanned image;

[0032] A cutting module, used for cutting the scanned image into a plurality of sub-images;

[0033] The analysis module is used to input each sub-image into a preset neural network model to obtain a sub-pathology analysis result; wherein the neural network model is based on the Inception-v4 multi-instance neural network and is embedded with a weakly supervised multi-instance learning module, which is used to perform pathological analysis on the sub-image to obtain a sub-pathology analysis result for characterizing whether thyroid cancer lymph nodes have metastasized; and the sub-pathology analysis results are summarized to obtain a target pathology analysis result.

[0034] The present application also provides an electronic device, including:

[0035] A processor, and a memory for storing a program executable by the processor;

[0036] The processor is used to implement the above-mentioned pathological analysis method of thyroid cancer lymph node metastasis by running the program in the memory.

[0037] The present application also proposes a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the processor executes the pathological analysis method of thyroid cancer lymph node metastasis as described above.

[0038] The present application provides a pathological analysis method for thyroid cancer lymph node metastasis, which obtains a scanned image of a HE-stained frozen pathological section of thyroid cancer tissue and preprocesses the scanned image; cuts the scanned image into multiple sub-images; for each sub-image, inputs the sub-image into a preset neural network model to obtain a sub-pathological analysis result; wherein the neural network model is based on the product neural network Inception-v4 as the main trunk, and embeds a weakly supervised multi-instance learning module, which is used to perform pathological analysis on the sub-image to obtain a sub-pathological analysis result for characterizing whether the thyroid cancer lymph node has metastasized; summarize the sub-pathological analysis results to obtain the target pathological analysis result. The scheme provided by the present application is set up in this way, which has the following beneficial effects: automated image scanning and preprocessing reduce the need for manual operation, improve the efficiency of pathological analysis, and allow doctors to handle more cases. By obtaining high-resolution HE-stained frozen pathological section scanned images, the microscopic details of thyroid cancer tissue can be captured, providing rich visual information for in-depth analysis. The image preprocessing step ensures the consistency of image quality and reduces analysis errors caused by factors such as uneven staining and uneven illumination. Cutting the scanned image into multiple sub-images allows independent analysis of different areas of the pathological section, which helps to more fully understand the pathological status of the entire tissue. Using a deep neural network model based on Inception-v4, complex patterns and features related to thyroid cancer lymph node metastasis in the image can be automatically learned and identified. The embedded weakly supervised multi-instance learning module allows the model to learn from a package containing multiple sub-instances, and can effectively train and extract features even if the sub-instances do not have separate labels. Inception-v4 is used as the backbone network, combined with the multi-instance learning module, it can more accurately extract key features in pathological images and perform pathological analysis of lymph node metastasis. Traditional pathological analysis relies on the subjective judgment of doctors, while the analysis method based on deep learning can reduce human bias and improve the consistency and reliability of diagnosis. Through the comprehensive analysis of the deep learning model, the lymph node metastasis of thyroid cancer can be more accurately predicted, thereby providing more accurate guidance for clinical treatment. In summary, this deep learning-based pathological image analysis method provides an efficient, accurate and reliable means of detecting thyroid cancer lymph node metastasis through automation, high-resolution imaging, standardized preprocessing, multi-region analysis and advanced neural network models. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0040] Figure 1 This is a partial flow chart of a method for pathological analysis of thyroid cancer lymph node metastasis provided in one embodiment of the present application.

[0041] Figure 2 It is a schematic diagram of the architecture of a neural network model provided by one embodiment of the present application.

[0042] Figure 3 It is a schematic diagram of the structure of a pathological analysis device for thyroid cancer lymph node metastasis provided by an embodiment of the present application.

[0043] Figure 4 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0044] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] Application Overview

[0046] Thyroid cancer is the malignant tumor with the fastest growing incidence rate in the world in the past three decades. The incidence rate of thyroid cancer in my country is also showing a rapid upward trend. More than 50% of thyroid cancer patients have lymph node metastasis at the time of diagnosis. Patients with lymph node metastasis have a high postoperative recurrence rate, resulting in poor prognosis. Accurately identifying thyroid cancer patients with lymph node metastasis in clinical practice, so as to carry out reasonable and thorough lymph node dissection and scientific postoperative stratified management, is of great significance to improving patient prognosis.

[0047] The pathological images of thyroid cancer tumor areas contain rich biological information, and the metastatic behavior of thyroid cancer is significantly correlated with the pathological characteristics of its tumor tissue. Deep learning technology has the advantages of identifying image features that cannot be distinguished by the naked eye, quantitative analysis, and strong objectivity of evaluation criteria. By analyzing pathological images, we can obtain macroscopic information on the morphology of thyroid cancer cells, and gain insight into microscopic information such as the tumor microenvironment. These massive amounts of high-dimensional information are helpful in analyzing whether thyroid cancer has lymph node metastasis. Traditional pathological methods of analysis are performed by pathologists based on prior knowledge through microscopes or digital pathology analyzers. The image information obtained by the naked eye is limited, and it is impossible to fully identify the biological information contained in the pathological images. It is difficult to analyze whether there is lymph node metastasis from the pathological images of thyroid cancer tissue.

[0048] In order to solve the above problems, the present application provides a pathological analysis method for thyroid cancer lymph node metastasis, which automatically analyzes the frozen pathological images of scanned intraoperative thyroid cancer tissues, extracts the most relevant features of lymph node metastasis in the pathological images of cancer tissues based on neural networks, and thus detects lymph node metastasis, which has the advantages of quantitative analysis and objective judgment criteria. The existing technology is that pathologists manually read pathological sections for judgment, which has a certain degree of subjectivity.

[0049] After introducing the basic principles of the present application, various non-limiting embodiments of the present application will be described in detail with reference to the accompanying drawings.

[0050] Exemplary Methods

[0051] Figure 1 FIG. 1 is a flow chart of a pathological analysis method for thyroid cancer lymph node metastasis provided by an embodiment of the present application. Figure 1 As shown, the method includes the following contents.

[0052] Step S110, obtaining a scanned image of a HE-stained frozen pathological section of a thyroid cancer tissue, and preprocessing the scanned image;

[0053] Specifically, the scanned images include: slice pathology images at different magnifications, for example: HE-stained frozen pathology sections of thyroid cancer tissue removed during surgery are scanned using a digital pathology scanner, the scanning magnification is 40×, and the scanned pathology images are output in SVS format to obtain digital panoramic slice pathology images. A panoramic slice pathology image is saved at 4 levels: the image resolution of level 0 saves a full HD image at a magnification of 40×, and the image resolutions of levels 1-4 are reduced by 2 times in sequence (i.e., the magnifications are 40×, 20×, 10×, and 5×).

[0054] Use a high-resolution scanner to obtain digital images of HE-stained frozen pathological sections of thyroid cancer tissue. By digitally scanning HE-stained frozen pathological sections, high-resolution images can be obtained to provide a clear microscopic view for pathological analysis. The scanned images are preprocessed, including adjusting brightness and contrast, denoising, color correction, etc., to improve image quality and prepare for subsequent analysis. Preprocessing steps (such as denoising, contrast enhancement, and color correction) can improve image quality and lay a good foundation for subsequent analysis.

[0055] Step S120, cutting the scanned image into a plurality of sub-images;

[0056] Specifically, image processing algorithms (such as threshold segmentation, edge detection, etc.) are used to segment the large scanned image into multiple small sub-images, each of which contains different areas of the pathological slice. By segmenting the large scanned image into multiple sub-images, we can focus on specific areas of the image for more detailed analysis, which helps to identify local features and increase the precision of the analysis.

[0057] Step S130, for each sub-image, input the sub-image into a preset neural network model to obtain a sub-pathology analysis result;

[0058] The neural network model is based on the inception neural network Inception-v4 as the main body, and is embedded with a weakly supervised multi-instance learning module, which is used to perform pathological analysis on sub-images to obtain sub-pathological analysis results for characterizing whether thyroid cancer lymph nodes have metastasized;

[0059] Specifically, using Inception-v4 as the backbone network, we can use its powerful feature extraction capabilities to capture microscopic features related to thyroid cancer lymph node metastasis in sub-images. Embedding a weakly supervised multi-instance learning module allows the model to learn from unlabeled sub-instances, handle the common label missing problem in actual pathological image analysis, and improve the generalization ability of the model.

[0060] Step S140, summarizing the sub-pathology analysis results to obtain a target pathology analysis result.

[0061] Specifically, the analysis results of all sub-images are summarized, and the results may be integrated by averaging, weighted averaging, or more complex statistical methods. By summarizing the analysis results from each sub-image, a comprehensive pathology analysis report can be obtained, which helps doctors obtain more comprehensive diagnostic information. By integrating the analysis of multiple sub-instances, the accuracy and reliability of the judgment of thyroid cancer lymph node metastasis can be improved.

[0062] In some embodiments, the preprocessing of the scanned image includes:

[0063] Eliminate the background area in the scanned image; or, remove the stains in the scanned image; or, perform color standardization processing on the scanned image.

[0064] Specifically, the goal of removing the background area is to remove the blank or irrelevant background area in the image that does not contain pathological tissue information. The implementation method is as follows: Use image segmentation technology, such as threshold segmentation or edge-based segmentation, to identify and remove the background.

[0065] The goal of spot removal is to remove spots from the image, which may come from dust, stains or other contaminants during the scanning process. The specific implementation methods are as follows: image filtering techniques such as median filtering or Gaussian blurring are used to smooth the image and remove small spots; Otsu threshold segmentation and other methods are used to identify and eliminate larger spots.

[0066] The goals of color standardization include: ensuring the consistency of color distribution in images and improving color comparability. The specific implementation methods are as follows: Color space conversion: converting images from RGB color space to other color spaces, such as HSV or Lab, to facilitate color standardization. Histogram equalization: applying histogram equalization on selected color channels to enhance the contrast of the image and make the color distribution more uniform. White balance adjustment: adjusting the image to match a specific white reference point to ensure that white objects appear pure white in the image.

[0067] It should be noted that, in practical applications, different specific methods can be selected based on different actual situations.

[0068] In some embodiments, aggregating the sub-pathology analysis results to obtain a target pathology analysis result includes: obtaining a mean of the sub-pathology analyses; wherein the mean is the target pathology analysis result.

[0069] Specifically, the pathological analysis results of all sub-images are collected and their mean is calculated. This mean reflects the comprehensive analysis result of the entire pathological section. The calculated mean is used as the target pathological analysis result. This result can be expressed as a continuous score or classification label to characterize the possibility of lymph node metastasis of thyroid cancer. Further, based on the mean result, a threshold or other decision rules are applied to generate a final diagnosis report.

[0070] It should be noted that the neural network model is the focus of the solution of this application. Figure 2 The neural network model includes: a feature extraction network and a pathology analysis network;

[0071] The feature extraction network is used to extract features of the input sub-image and send the features to the pathology analysis network; the pathology analysis network is used to obtain sub-pathology analysis results for characterizing whether thyroid cancer lymph nodes have metastasized based on the features; wherein the feature extraction network is a neural network with Inception-v4 as the backbone and embedded in a weakly supervised multi-instance learning module.

[0072] With this setting, the role of the Weakly Supervised Multi-Instance Learning (WS-MIL) module in the deep learning model is mainly reflected in the following aspects:

[0073] In the WS-MIL framework, each bag contains multiple sub-instances, but there are only bag-level annotations, not per-sub-instance annotations. This module can handle such incompletely annotated data, enabling the model to learn even when sub-instance annotations are missing or ambiguous.

[0074] By analyzing all sub-instances within a package, the WS-MIL module can help the model learn which features are helpful in identifying the overall label of the package, even though these features may only appear in a small number of sub-instances of the package.

[0075] Since the model is trained at the package level, it needs to abstract general features from multiple sub-instances, which helps improve the model's generalization ability to new data.

[0076] In real-world applications such as medical image analysis, obtaining accurate instance-level annotations may be costly or infeasible. The WS-MIL module allows the model to learn effectively even with imprecise annotations.

[0077] By using the WS-MIL module in the model, the reliance on professional annotators can be reduced, thereby reducing the cost and time of data preparation.

[0078] Since the model does not rely on the precise annotation of each sub-instance, it is more robust to annotation errors and noise.

[0079] The model can learn to identify which sub-instance features are important, thereby performing effective feature selection. The WS-MIL module can adapt to different data distributions and task requirements, and optimize performance by adjusting the model structure and learning strategy.

[0080] Through these efforts, the weakly supervised multi-instance learning module provides a flexible and powerful method for processing datasets with complex annotation structures, especially in medical image analysis and other fields that require processing large amounts of unlabeled or weakly labeled data.

[0081] In some embodiments, the training process of the neural network model includes: obtaining scanned image samples; generating training samples based on the scanned image samples and the transfer results corresponding to each scanned image sample; performing weakly supervised multi-example training on a pre-built feature extraction network and pathology analysis network based on the training samples to obtain a neural network model.

[0082] The training process of a neural network model is a key step in building an effective deep learning system. The following is a detailed description of how to perform training, especially in the scenario of using weakly supervised multi-instance learning:

[0083] Get a sample scanned image:

[0084] Scanned images of HE-stained frozen pathological sections of thyroid cancer tissues were collected and these images will serve as the basis for training data.

[0085] Generate training samples:

[0086] For each scanned image, the metastasis results of the image sample are annotated according to whether it shows the clinical result of lymph node metastasis. These results will serve as the labels of the training samples.

[0087] Image preprocessing:

[0088] The collected scanned images are preprocessed, including background area removal, stain removal, color standardization, etc., to improve image quality.

[0089] Image Segmentation:

[0090] The preprocessed image is cut into multiple sub-images, each of which contains a part of the pathological section.

[0091] Construct the training set:

[0092] Each bag (consisting of multiple sub-images) is combined with the corresponding transfer results to form a training sample. In weakly supervised multi-instance learning, each bag has only one label, but each sub-instance in the bag does not have a separate label.

[0093] Build a neural network model:

[0094] A neural network model is designed that includes a feature extraction network and a pathology analysis network. The feature extraction network is responsible for extracting the features of the sub-images, and the pathology analysis network performs pathology analysis based on these features.

[0095] Weakly supervised multi-instance training:

[0096] The neural network model is trained using the training samples. During the training process, the model learns how to extract useful features from multiple sub-instances in a bag and predict the overall label of the bag based on these features.

[0097] Loss function definition:

[0098] Define a suitable loss function to quantify the difference between the model prediction and the actual label. In weakly supervised learning, a specially designed loss function may be needed to handle the uncertainty of the label.

[0099] Optimization algorithm selection:

[0100] Choose an optimization algorithm (such as SGD, Adam, etc.) to update the weights of the network and minimize the loss function.

[0101] Model tuning:

[0102] Adjust the model's hyperparameters, such as learning rate, batch size, number of network layers, etc., to optimize the training process and model performance.

[0103] Cross Validation:

[0104] Use cross-validation to evaluate the stability and generalization ability of the model and ensure that the model performs well on different subsets of data.

[0105] Model Evaluation:

[0106] Use appropriate evaluation metrics (such as precision, recall, F1 score, etc.) to evaluate the performance of the model.

[0107] Model fine-tuning:

[0108] Based on the evaluation results, the model is fine-tuned to further improve its accuracy and robustness.

[0109] Clinically validated:

[0110] Collaborate with pathologists to validate the clinical validity of the model and make adjustments based on feedback.

[0111] Model saving and deployment:

[0112] After training is complete, the model parameters are saved and deployed to real-world applications, such as pathology diagnosis systems.

[0113] Through the above steps, a neural network model capable of handling weakly supervised multi-instance learning tasks can be constructed, which can automatically learn and identify features related to thyroid cancer lymph node metastasis from pathological images and provide accurate pathological analysis results.

[0114] Specifically, the scanned image samples include: slice pathology images at different magnifications: generating training samples includes: for each scanned image sample; preprocessing the slice pathology images at different magnifications respectively; performing image segmentation on the preprocessed slice pathology images at different magnifications respectively to obtain image block samples; extracting the image block samples corresponding to different magnifications respectively to obtain image packages composed of image block samples corresponding to different magnifications; setting labels for the image packages based on the transfer results corresponding to the scanned image samples to obtain training samples.

[0115] The detailed description of the specific implementation method is as follows: Collect scanned image samples: Collect HE-stained frozen pathological section images at different magnifications from the surgery of thyroid cancer patients. Preprocess images at different magnifications: Preprocess the pathological section images of each magnification, including denoising, color correction, contrast enhancement, etc., to improve image quality. Image segmentation: Segment the preprocessed images and cut each magnification image into multiple image blocks (or image patches). Generate image block samples: Extract image blocks from each magnification image, which will be used as feature carriers for model training. Construct image packages: Combine image block samples at different magnifications into image packages, each of which contains image blocks from different magnifications. Label setting: Set a label for each image package according to the lymph node metastasis results corresponding to the scanned image samples. This label can be binary (metastatic or non-metastatic) or multi-classified (for example, classified according to the degree of metastasis). Generate training samples: Each labeled image package becomes a training sample, which will be used to train the neural network model.

[0116] In practical applications, the solution provided in this application specifically includes: 1. digital scanning of pathological sections; 2. preprocessing of pathological images; 3. extraction of pathological image features; 4. verification of the accuracy of this method in predicting the risk of lymph node metastasis.

[0117] Specifically, the digital scanning of pathological sections is as follows: the HE-stained frozen pathological sections of thyroid cancer tissue removed during surgery are scanned using a digital pathological scanner, with a scanning magnification of 40×, and the scanned pathological images are output in SVS format to obtain digital panoramic pathological images. One panoramic pathological image is saved at four levels: the image resolution of level 0 saves a full HD image at a magnification of 40×, and the image resolutions of levels 1-4 are reduced by 2 times in sequence (i.e., the magnifications are 40×, 20×, 10×, and 5×).

[0118] The pathological image preprocessing is as follows: The pathological image preprocessing includes the following three points: 1) Since there are many white background areas in the pathological images, and there may be some pollution in the process of making pathological sections and staining, which leads to stains in the pathological images and affects the image analysis. First, the Otsu threshold segmentation method in image processing technology is used to extract the organized area of ​​each pathological image, and then the red, green and blue three-channel values ​​of the image are judged by setting rules to remove the stains; 2) The pathological images after the above processing are color standardized, and the red, green and blue three-channel values ​​of all pathological images are normalized and mapped to the standardized parameters to ensure that the color distribution of all pathological images is consistent; 3) Image block segmentation, using the Python interface of Openslide-3.4.1, each pathological image is segmented into 512×512 pixel image blocks (ie: sub-pixel blocks or image block samples).

[0119] The pathological image feature extraction is as follows: The feature extraction network is based on the convolutional neural network Inception-v4, embedded with a weakly supervised multi-instance learning module, and 64 image patches are randomly extracted from each magnification of each panoramic slice pathological image to form a bag (Patch Bag), which is input into the convolutional neural network Inception-v4 ( Figure 2 ). At the same time, the convolutional neural network parameters are adjusted using the label of whether the patient to whom the pathological section belongs has lymph node metastasis (paraffin pathology after lymph node dissection to determine whether there is lymph node metastasis) to extract the most relevant features of lymph node metastasis in the frozen pathological image of thyroid cancer tissue. Further, in practical applications, after extracting these features based on the feature extraction network, pathological analysis can be performed based on the pathological analysis network to obtain sub-pathological analysis results for characterizing whether thyroid cancer lymph nodes have metastasized; the sub-pathological analysis results are summarized to obtain the target pathological analysis results.

[0120] To verify the accuracy of this method in predicting the risk of lymph node metastasis, the specific process is as follows: collect a batch of intraoperative frozen pathological section data of thyroid cancer patients for digital scanning and establish a pathological image test set. After the pathology of the test set is preprocessed and standardized according to the above content, it is input into the neural network module, the output results are recorded, and compared with the actual pathological results of the patient's lymph node metastasis to evaluate the accuracy of lymph node metastasis detection. If the accuracy is too low, the neural network model can be retrained.

[0121] Exemplary Devices

[0122] The device embodiments of the present application can be used to execute the method embodiments of the present application. For details not disclosed in the device embodiments of the present application, please refer to the method embodiments of the present application.

[0123] Figure 3 FIG. 1 is a block diagram of a pathological analysis device for thyroid cancer lymph node metastasis provided by an embodiment of the present application. Figure 3 As shown, the device comprises:

[0124] An acquisition module 31 is used to acquire a scanned image of a HE-stained frozen pathological section of a thyroid cancer tissue and pre-process the scanned image;

[0125] A cutting module 32, used for cutting the scanned image into a plurality of sub-images;

[0126] The analysis module 33 is used to input each sub-image into a preset neural network model to obtain a sub-pathology analysis result; wherein the neural network model is based on the convolutional neural network Inception-v4 and is embedded with a weakly supervised multi-instance learning module, which is used to perform pathological analysis on the sub-image to obtain a sub-pathology analysis result for characterizing whether thyroid cancer lymph nodes have metastasized; and the sub-pathology analysis results are summarized to obtain a target pathology analysis result.

[0127] In some embodiments, the preprocessing of the scanned image includes:

[0128] Eliminate the background area in the scanned image; or,

[0129] removing spots from the scanned image; or,

[0130] The scanned image is subjected to color standardization processing.

[0131] In some embodiments, the aggregating the sub-pathology analysis results to obtain a target pathology analysis result includes:

[0132] Calculating the mean of the sub-pathological analysis;

[0133] The mean value is the target pathology analysis result.

[0134] In some embodiments, the scanned images include: slice pathology images at different magnifications.

[0135] In some embodiments, the neural network model includes: a feature extraction network and a pathology analysis network;

[0136] The feature extraction network is used to extract features of the input sub-image and send the features to the pathology analysis network;

[0137] The pathological analysis network is used to obtain a sub-pathological analysis result for characterizing whether thyroid cancer lymph nodes have metastasized based on the characteristics;

[0138] The feature extraction network is a neural network with Inception-v4 as the backbone and a weakly supervised multi-instance learning module embedded in it.

[0139] In some embodiments, the training process of the neural network model includes:

[0140] Obtain a scanned image sample;

[0141] Generate training samples based on the scanned image samples and the transfer results corresponding to each scanned image sample;

[0142] Based on the training samples, weakly supervised multi-example training is performed on the pre-built feature extraction network and pathology analysis network to obtain a neural network model.

[0143] In some embodiments, the scanned image samples include: slice pathology images at different magnifications:

[0144] The generating of training samples comprises:

[0145] For each scanned image sample; preprocess the slice pathology images at different magnifications respectively; perform image segmentation on the preprocessed slice pathology images at different magnifications respectively to obtain image block samples; extract the image block samples corresponding to different magnifications respectively to obtain an image package composed of the image block samples corresponding to different magnifications; set a label for the image package based on the transfer result corresponding to the scanned image sample to obtain a training sample.

[0146] Exemplary Electronic Devices

[0147] Below, reference Figure 4 To describe an electronic device according to an embodiment of the present application. Figure 4 A block diagram of an electronic device according to an embodiment of the present application is illustrated.

[0148] like Figure 4 As shown, electronic device 400 includes one or more processors 410 and memory 420 .

[0149] The processor 410 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 400 to perform desired functions.

[0150] The memory 420 may include one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 410 may run the program instructions to implement the pathological analysis method of thyroid cancer lymph node metastasis of each embodiment of the present application described above and / or other desired functions. Various contents such as category correspondences may also be stored in the computer-readable storage medium.

[0151] In one example, the electronic device 400 may further include: an input device 430 and an output device 440 , and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0152] In addition, the input device 430 may also include, for example, a keyboard, a mouse, an interface, etc. The output device 440 may output various information to the outside, including analysis results, etc. The output device 440 may include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, etc.

[0153] Of course, to simplify, Figure 4 Only some of the components in the electronic device related to the present application are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, the electronic device may further include any other appropriate components according to specific application conditions.

[0154] Exemplary computer program products and computer-readable storage media

[0155] In addition to the above-mentioned methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the pathological analysis method of thyroid cancer lymph node metastasis according to various embodiments of the present application described in the above-mentioned "Exemplary Method" section of this specification.

[0156] The computer program product may be written in any combination of one or more programming languages ​​to write program codes for performing the operations of the embodiments of the present application, including object-oriented programming languages, such as Java, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as an independent software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0157] In addition, an embodiment of the present application may also be a computer-readable storage medium on which computer program instructions are stored. When the computer program instructions are executed by a processor, the processor executes the steps of the pathological analysis method of thyroid cancer lymph node metastasis according to various embodiments of the present application described in the above "Exemplary Method" section of this specification.

[0158] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0159] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present application to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. A pathological analysis method for thyroid cancer lymph node metastasis, characterized in that: include: Obtaining a scanned image of a HE-stained frozen pathological section of a thyroid cancer tissue, and preprocessing the scanned image; Cutting the scanned image into a plurality of sub-images; For each sub-image, input the sub-image into a preset neural network model to obtain a sub-pathology analysis result; The neural network model is based on the inception neural network Inception-v4 as the main body, and is embedded with a weakly supervised multi-instance learning module, which is used to perform pathological analysis on sub-images to obtain sub-pathological analysis results for characterizing whether thyroid cancer lymph nodes have metastasized; The sub-pathology analysis results are summarized to obtain the target pathology analysis result.

2. The method for pathological analysis of thyroid cancer lymph node metastasis according to claim 1, characterized in that: The preprocessing of the scanned image comprises: Eliminate the background area in the scanned image; or, removing spots from the scanned image; or, The scanned image is subjected to color standardization processing.

3. The method for pathological analysis of thyroid cancer lymph node metastasis according to claim 1, characterized in that: The step of aggregating the sub-pathology analysis results to obtain a target pathology analysis result includes: Calculating the mean of the sub-pathological analysis; The mean value is the target pathology analysis result.

4. The method for pathological analysis of thyroid cancer lymph node metastasis according to claim 1, characterized in that: The scanned images include: slice pathology images at different magnifications.

5. The method for pathological analysis of thyroid cancer lymph node metastasis according to claim 1, characterized in that: The neural network model includes: a feature extraction network and a pathology analysis network; The feature extraction network is used to extract features of the input sub-image and send the features to the pathology analysis network; The pathological analysis network is used to obtain a sub-pathological analysis result for characterizing whether thyroid cancer lymph nodes have metastasized based on the characteristics; The feature extraction network is a neural network with Inception-v4 as the backbone and a weakly supervised multi-instance learning module embedded in it.

6. The method for pathological analysis of thyroid cancer lymph node metastasis according to claim 5, characterized in that: The training process of the neural network model includes: Obtain a scanned image sample; Generate training samples based on the scanned image samples and the transfer results corresponding to each scanned image sample; Based on the training samples, weakly supervised multi-example training is performed on the pre-built feature extraction network and pathology analysis network to obtain a neural network model.

7. The method for pathological analysis of thyroid cancer lymph node metastasis according to claim 6, characterized in that: The scanned image samples include: slice pathology images at different magnifications: The generating of training samples comprises: For each scanned image sample; preprocess the slice pathology images at different magnifications respectively; perform image segmentation on the preprocessed slice pathology images at different magnifications respectively to obtain image block samples; extract the image block samples corresponding to different magnifications respectively to obtain an image package composed of the image block samples corresponding to different magnifications; set a label for the image package based on the transfer result corresponding to the scanned image sample to obtain a training sample.

8. A method and device for pathological analysis of thyroid cancer lymph node metastasis, characterized in that: include: An acquisition module is used to acquire a scanned image of a HE-stained frozen pathological section of a thyroid cancer tissue and pre-process the scanned image; A cutting module, used for cutting the scanned image into a plurality of sub-images; The analysis module is used to input each sub-image into a preset neural network model to obtain a sub-pathology analysis result; wherein the neural network model is based on the Inception-v4 multi-instance neural network and is embedded with a weakly supervised multi-instance learning module, which is used to perform pathological analysis on the sub-image to obtain a sub-pathology analysis result for characterizing whether thyroid cancer lymph nodes have metastasized; and the sub-pathology analysis results are summarized to obtain a target pathology analysis result.

9. An electronic device, characterized in that: include: A processor, and a memory for storing a program executable by the processor; The processor is used to implement the pathological analysis method of thyroid cancer lymph node metastasis according to any one of claims 1 to 7 by running the program in the memory.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, enables the processor to execute the pathological analysis method for thyroid cancer lymph node metastasis according to any one of claims 1 to 7.