Sliced image lymph node micro-change region identification model, system and method
By constructing the MetaNet model and training it with meta-learning algorithms and multi-class tissue slice datasets, the problem of identifying subtle changes in lymph node regions in paraffin and frozen sections was solved, achieving higher identification accuracy and detection capabilities across slice categories.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SUZHOU UNIV
- Filing Date
- 2024-04-01
- Publication Date
- 2026-07-21
AI Technical Summary
Existing technologies struggle to effectively identify subtle changes in lymph nodes in paraffin and frozen sections, especially when datasets are limited, resulting in insufficient generalization ability of deep learning models and making it difficult to meet clinical needs.
The MetaNet model is constructed using a meta-learning algorithm. It combines shallow and deep feature extractors with a fusion feature voter and performs meta-training using a multi-class tissue slice dataset. It is further trained using a lymph node slice dataset and is specifically designed to identify subtle changes in lymph node regions in paraffin and frozen sections.
The model improved accuracy and generalization ability in identifying areas of subtle changes in lymph nodes, significantly enhanced metrics such as Acc and Auc, and demonstrated higher recognition ability and detection performance across slice categories.
Smart Images
Figure CN118229989B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of medical imaging and image processing technology, specifically relating to a method, system, and apparatus for identifying subtle changes in lymph node regions in paraffin and frozen section images. Background Technology
[0002] Cancer is a major disease threatening human health. For example, breast cancer has an annual incidence rate exceeding 2 million cases and is the leading cause of cancer-related deaths among women in 95% of countries. Ipsilateral axillary lymph node metastasis is an important mechanism in the development and progression of breast cancer and is one of the initial sites of metastasis. Pathological examination of the axillary sentinel lymph nodes plays a crucial role in subsequent treatment decisions, surgical approach selection, staging, and prognostic assessment. With the development of whole-slide images (WSIs), recent years have seen good results in identifying areas of change (such as atypical cells and calcifications) within lymph nodes, especially in areas with large areas of change, demonstrating that artificial intelligence can greatly assist pathologists in their examinations.
[0003] However, the smaller the area of lymph node change, the more difficult it is to identify. This poses a challenge not only to pathologists but also to artificial intelligence algorithms. Subtle changes indicate that the lymph node is in the initial stage of disease development and have significant clinical implications. However, due to the insidious nature of lesions, they are more easily overlooked during detection, making it difficult to collect datasets of areas with subtle lymph node changes. Furthermore, because of the small lesion size, the number of positive marker patches in areas of subtle changes is far fewer than in large atypical areas, further limiting the expansion of datasets. This is a significant limitation for deep learning (DL) methods and convolutional neural networks (CNNs) that require large datasets.
[0004] Frozen sections are crucial for rapid intraoperative examination and determining surgical approach. Due to different tissue fixation methods, frozen sections are typically thicker to prevent tissue shrinkage, resulting in reduced clarity. Furthermore, the ice crystals produced during rapid freezing alter cell morphology, causing swelling and enlarged, distorted cell nuclei. These differences lead to poor generalization ability of CNN models trained using deep learning methods on paraffin sections to frozen sections. Typically, during rapid intraoperative frozen section examination, a portion of tissue is collected for subsequent paraffin section verification. However, not all paraffin sections undergo corresponding frozen section examinations. Therefore, the size of the frozen section dataset for areas of subtle lymph node changes is further reduced compared to the paraffin section dataset, making CNN model training even more challenging. Summary of the Invention
[0005] To address the series of problems existing in the prior art, the present invention provides a model, system, method, computer device, and readable storage medium for identifying regions of minute changes in lymph nodes in slice images.
[0006] To solve the above-mentioned technical problems, the present invention is achieved through the following technical solution:
[0007] A model for identifying subtle changes in lymph nodes in slice images is proposed. The model uses a network architecture consisting of a shallow feature extractor, a deep feature extractor, and a fusion feature voter. It constructs a meta-task using a WSIs dataset of multi-class tissue slices, performs meta-training using a meta-learning algorithm, and then further trains the model using a lymph node slice dataset. The resulting network model, known as MetaNet, is specifically designed for identifying subtle changes in lymph nodes in paraffin and frozen slices.
[0008] Furthermore, the training method for MetaNet is as follows:
[0009] Obtain routinely stained lymph node slices and tissue slices from multi-class datasets;
[0010] The acquired slices were scanned into WSI images with magnifications of 4× and 10×, respectively.
[0011] The scanned WSI images were cropped into WSI-field-of-view images of the same size as the actual microscope camera's field of view, measuring 3072×2048.
[0012] The 3072×2048 WSI-field of view image was further cropped into 224×224 WSI-patches images;
[0013] For WSI-patches images of multi-category tissue sections, labels are made according to the tissues contained within the patches;
[0014] For WSI-patches images of lymph node sections, the areas of lymphatic tissue change contained within the patches are labeled;
[0015] The labeled multi-class tissue slice WSI-patches images constitute the multi-class slice WSIs dataset, and the labeled lymph node slice WSI-patches images constitute the lymph node dataset;
[0016] A meta-task is constructed using the multi-class sliced WSIs dataset, and MetaNet is then meta-trained.
[0017] MetaNet was further trained using the lymph node dataset.
[0018] Furthermore, the shallow feature extractor consists of four 3×3 convolutional layers, activation layers, and normalization layers. The shallow feature extractor is used to extract surface features from the input MetaNet slice WSI-patches image to obtain shallow features.
[0019] The deep feature extractor consists of 12 Transformer blocks and is used to extract more complex features from the shallow features to obtain deep features.
[0020] The fusion feature voter includes a feature fusion layer, a hidden layer, and a feature voting layer, which are used to combine the shallow features with the deep features, and then use the fusion features to predict the final lymph node micro-change area and output the prediction result.
[0021] Furthermore, in the deep feature extractor, each Transformer block includes a multi-head self-attention mechanism, a feed-forward neural network, and residual connections; wherein,
[0022] The multi-head self-attention mechanism: In the Transformer block, the input sequence is divided into multiple sub-sequences simultaneously, and then the self-attention weights are calculated for each sub-sequence; this allows the model to pay attention to information at different positions simultaneously when processing the input sequence, which helps to capture long-distance dependencies;
[0023] The feedforward neural network: After the multi-head self-attention mechanism, the representation of each subsequence is processed by a fully connected feedforward neural network; the feedforward neural network is a two-layer fully connected neural network containing the ReLU activation function;
[0024] The residual connection: In the Transformer block, the representation of each subsequence is added to its input, i.e., the residual connection; the residual connection helps to alleviate the gradient vanishing problem, making it easier to optimize when training deep models.
[0025] Furthermore, the embed_dim of the deep feature extractor is set to 384, and num_heads is set to 6;
[0026] embed_dim represents the dimension of the embedding vector encoded for each word or position, i.e., the embedding dimension; num_heads represents the number of attention heads computed in parallel in each self-attention mechanism, i.e., the number of attention heads.
[0027] Furthermore, the fusion prediction method of the fusion feature voter is as follows:
[0028] The last layer output of the shallow feature extractor is selected as the shallow feature input of the feature fusion layer; and the outputs of the last 6 Transformer blocks of the deep feature extractor are selected as the deep feature input of the feature fusion layer.
[0029] The feature fusion layer applies a linear transformation to the shallow and deep features of different scales in the input, mapping them to the same dimensional space, and then concatenates them together to form fused features;
[0030] The synthesized fused features are input into a two-layer hidden neural network for further processing to help the model learn higher-order feature representations.
[0031] The fused features processed by the neural network in the hidden layer are input into the feature voting layer to predict the region of minor changes in lymph nodes, and finally the prediction result is output.
[0032] A system for identifying regions of minute changes in lymph nodes in slice images includes:
[0033] A receiving unit for acquiring paraffin or frozen WSI images;
[0034] The cropping unit is used to crop the acquired WSI image into a WSI-field-of-view image of the same size as the real microscope, and then further crop it into WSI-patches images;
[0035] The processing unit is used to identify the lymph node micro-change region in the WSI-patches image using the trained lymph node micro-change region identification model of the above slice image and obtain the identification result. It is used to generate a discrimination probability map of the WSI-field map to be identified based on the obtained identification result, and merge the discrimination probability map with the WSI-field map to be identified to generate an identified WSI-field map with lymph node micro-change region markings.
[0036] The output unit is used to display the discrimination probability map of the WSI field to be identified and the identified WSI field with markings of small changes in lymph nodes.
[0037] A method for identifying regions of minute changes in lymph nodes in slice images includes the following steps:
[0038] Step 1) Obtain frozen or paraffin whole slide images (WSI) at 4× and 10× magnification.
[0039] Step 2) Crop the acquired WSI image into a field of view to be identified with the same size as the actual microscope camera's field of view of 3072×2048.
[0040] Step 3) Further crop the WSI-view map to be identified into 224×224 WSI-patches images;
[0041] Step 4) Input the cropped WSI-patches image into the pre-trained lymph node micro-change region recognition model of the above slice image to identify whether there are lymph node micro-change regions in the WSI-patches image and obtain the recognition result;
[0042] Step 5) Based on the obtained recognition results, the WSI-patches images identified as areas of minor changes in lymph nodes are marked in red, and the WSI-patches images identified as normal areas are marked in blue, forming a discrimination probability map of the WSI-field map to be identified;
[0043] Step 6) Merge the resulting discrimination probability map with the original cropped WSI-field map to be identified to form an identified WSI-field map, and in the identified WSI-field map, the areas in the lymph node where tissue changes can be seen are highlighted in red.
[0044] A computer device includes a processor, a memory, a communication interface, and a communication bus. The processor, the memory, and the communication interface communicate with each other via the communication bus. The memory stores at least one executable instruction, which causes the processor to perform the operation corresponding to the above-described method for identifying subtle changes in lymph node regions in sliced images.
[0045] A computer-readable storage medium storing at least one executable instruction that causes a processor to perform operations corresponding to the above-described method for identifying subtle changes in lymph node regions in sliced images.
[0046] This invention, based on meta-learning theory, designs an innovative system for identifying subtle changes in paraffin and frozen section images of lymph nodes. Specifically, this invention designs a novel network structure based on a meta-learning algorithm, named MetaNet, aiming to leverage the strong generalization performance of meta-learning. This means that even with limited sample data, this invention can train a model with superior performance.
[0047] To train a model with superior performance, this invention also collected and labeled a multi-class WSI dataset specifically designed for meta-training. This dataset was constructed to fully leverage the advantages of meta-learning algorithms and improve the model's generalization ability and robustness. During training, this invention used the MetaNet architecture and employed meta-learning algorithms to train it.
[0048] Compared with network structures trained using traditional deep learning methods, such as ResNet18, the method proposed in this invention shows significant improvements in multiple metrics, including Acc and Auc. This indicates that the method of this invention has higher accuracy and discrimination ability in the task of identifying subtle changes in lymph node regions in paraffin and frozen sections.
[0049] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the invention and to implement it according to the contents of the specification, the preferred embodiments of the present invention are described in detail below with reference to the accompanying drawings. Specific embodiments of the present invention are given in detail below with reference to the accompanying drawings. Attached Figure Description
[0050] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:
[0051] Figure 1 This is an architectural diagram of the lymph node micro-change region recognition model in sliced images according to the present invention;
[0052] Figure 2 This is a schematic diagram of the identification process of the system and method for identifying minute changes in lymph nodes in sliced images according to the present invention. Detailed Implementation
[0053] The preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings to provide a clearer understanding of the invention's purpose, features, and advantages. It should be understood that the embodiments shown in the drawings are not intended to limit the scope of the invention, but are merely illustrative of the essential spirit of the invention's technical solution.
[0054] In the following description, certain specific details are set forth for the purpose of illustrating various disclosed embodiments in order to provide a thorough understanding of the various disclosed embodiments. However, those skilled in the art will recognize that embodiments may be practiced without one or more of these specific details. In other instances, well-known apparatuses, structures, and techniques associated with this application may not have been shown or described in detail to avoid unnecessarily obscuring the description of the embodiments.
[0055] Unless the context requires otherwise, throughout the specification and claims, the word “comprising” and its variations, such as “including” and “having”, shall be understood to have an open, inclusive meaning, that is, to be interpreted as “including, but not limited to”.
[0056] Throughout this specification, references to "an embodiment" or "an embodiment" indicate that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment. Therefore, the appearance of "in an embodiment" or "an embodiment" in various places throughout the specification does not necessarily refer to the same embodiment. Furthermore, a particular feature, structure, or characteristic may be combined in any manner in one or more embodiments.
[0057] The singular forms “a” and “the” used in this specification and the appended claims include plural references unless otherwise expressly stated herein. It should be noted that the term “or” is generally used to mean “and / or” unless otherwise expressly stated herein.
[0058] Furthermore, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.
[0059] This invention provides a model for identifying subtle changes in lymph nodes in paraffin and frozen section images. The model uses a network architecture consisting of a shallow feature extractor, a deep feature extractor, and a fusion feature voter. It constructs a meta-task using a WSIs dataset of multi-class tissue sections, performs meta-training using a meta-learning algorithm, and then further trains the model using a lymph node section dataset. The resulting network model is specifically designed for identifying subtle changes in lymph nodes in paraffin and frozen sections and is also known as MetaNet.
[0060] Since training deep learning algorithms typically requires a large amount of labeled training data for model parameter optimization, collecting and labeling more data is time-consuming and expensive. Therefore, this invention employs a meta-learning algorithm, which exhibits excellent generalization performance on few-shot tasks, to train the model. Simultaneously, this invention designs a novel network structure based on the meta-learning algorithm for the identification of micrometastases in breast cancer lymph nodes.
[0061] Furthermore, since meta-learning algorithms rely on diverse datasets to construct high-quality meta-tasks, this invention collects and labels a diverse slice WSIs dataset specifically for constructing meta-learning multi-tasks.
[0062] Furthermore, on the one hand, meta-learning algorithms typically choose models with simple structures and few parameters, such as four-layer convolutions, but such models are not suitable for processing complex medical images. On the other hand, Transformer models have attracted much attention due to their excellent performance; however, they are not suitable for medical tasks where the amount of data is usually limited.
[0063] In summary, medical images (such as frozen sections) typically have a small amount of data, but the images themselves are more complex, making the recognition task more challenging. To address these challenges, this invention designs a novel model based on meta-learning algorithms and Transformers, namely MetaNet, specifically for recognizing subtle variations in lymph node regions in paraffin and frozen sections.
[0064] See Figure 1 As shown, the MetaNet designed in this invention mainly consists of three components: a shallow feature extractor, a deep feature extractor, and a fusion feature voting unit.
[0065] The shallow feature extractor is used to extract surface features from the input MetaNet WSI-patches image to obtain shallow features. The deep feature extractor is used to extract more complex features from the shallow features to obtain deep features. The deep feature extractor is used to combine the shallow features and the deep features, and then use the fused features to predict the final lymph node micro-change region and output the prediction result.
[0066] 1. Shallow Feature Extractor
[0067] The shallow feature extractor consists of four 3×3 convolutional layers, activation layers, and normalization layers. Convolutional neural networks are renowned for their excellent feature extraction capabilities, especially in image processing. Meta-learning algorithms typically also tend to employ simple and efficient model structures, with four-layer convolutional networks being a common choice. Therefore, this invention selects such a simple and effective structure as the shallow feature extractor for MetaNet and trains the model using a meta-learning algorithm to quickly and accurately extract shallow features from WSI-patches images. This design not only conforms to the traditional choices of meta-learning algorithms but also meets the needs of medical image processing, providing a reliable foundation for the model of this invention.
[0068] 2. Deep Feature Extractor
[0069] The deep feature extractor consists of 12 Transformer blocks, and each Transformer block includes a multi-head self-attention mechanism, a feed-forward neural network, and a residual connection.
[0070] The following is a detailed introduction to the deep feature extractor:
[0071] Multi-head self-attention mechanism: In the Transformer block, the input sequence is divided into multiple sub-sequences simultaneously, and self-attention weights are calculated for each sub-sequence. This allows the model to pay attention to information from different locations simultaneously when processing the input sequence, which helps to capture long-distance dependencies.
[0072] Feedforward Neural Network: Following the multi-head self-attention mechanism, the representation of each subsequence is processed through a fully connected feedforward neural network. This feedforward neural network is a two-layer fully connected neural network containing the ReLU activation function.
[0073] Residual connections: In a Transformer block, the representation of each subsequence is added to its input, which is called a residual connection. Residual connections help alleviate the vanishing gradient problem, making it easier to optimize when training deep models.
[0074] Based on the complexity of the current task, the deep feature extractor of MetaNet in this invention uses 12 Transformer blocks, with each Transformer block having a embedd_dim of 384 and a num_heads of 6. embed_dim (embedding dimension) represents the dimension of the embedding vector encoded for each word or position, and num_heads (number of attention heads) represents the number of attention heads computed in parallel within each self-attention mechanism.
[0075] The 12 Transformer blocks mean that this deep feature extractor can perform 12 deep feature extractions and information interactions when processing the input sequence. This enables the model of this invention to better capture the complex structure and semantic information of the input sequence, improving the model's representation and generalization capabilities.
[0076] In summary, the deep feature extractor of this invention achieves complex feature extraction and representation learning of input sequences through multiple layers of Transformer blocks, providing the model with powerful feature representation capabilities for micro-transfer diagnosis tasks in paraffin and frozen WSI sections.
[0077] 3. Integrated Feature Voting Unit
[0078] The fused feature voter includes a feature fusion layer, a hidden layer, and a feature voting layer. The task of this fused feature voter is to effectively fuse features from shallow and deep feature extractors and use the fused features for the final prediction. To achieve this goal, the present invention adopts the following design strategy:
[0079] The last layer output of the shallow feature extractor is selected as the shallow feature input of the feature fusion layer; and the outputs of the last 6 Transformer blocks of the deep feature extractor are selected as the deep feature input of the feature fusion layer.
[0080] The feature fusion layer applies a linear transformation to the shallow and deep features of different scales in the input, mapping them to the same dimensional space, and then concatenates them together to form fused features;
[0081] The synthesized fused features are input into a two-layer hidden neural network for further processing to help the model learn higher-order feature representations.
[0082] The fused features processed by the neural network in the hidden layer are input into the feature voting layer to predict the region of minor changes in lymph nodes, and finally the prediction result is output.
[0083] 4. Multi-class slice WSIs dataset and lymph node dataset
[0084] This invention specifically collects and annotates a rich variety of slice WSIs datasets to build high-quality meta-learning multi-task datasets.
[0085] The methods for collecting and labeling the multi-class slice WSIs dataset and the lymph node dataset are as follows:
[0086] Obtain routinely stained lymph node slices and tissue slices from multi-class datasets;
[0087] The acquired slices were scanned into WSI images with magnifications of 4× and 10×, respectively.
[0088] The scanned WSI images were cropped into WSI-field-of-view images of the same size as the actual microscope camera's field of view, measuring 3072×2048.
[0089] The 3072×2048 WSI-field of view image was further cropped into 224×224 WSI-patches images;
[0090] For WSI-patches images of multi-category tissue sections, labels are made according to the tissues contained within the patches;
[0091] For WSI-patches images of lymph node sections, the areas of lymphatic tissue change contained within the patches are labeled;
[0092] The labeled multi-category tissue slice WSI-patches images constitute the multi-category slice WSIs dataset, and the labeled lymph node slice WSI-patches images constitute the lymph node dataset.
[0093] In the development and research process of this invention, 41 frozen sections containing 41 lymph nodes and 62 paraffin sections containing 70 lymph nodes from breast cancer patients at Sun Yat-sen Memorial Hospital of Sun Yat-sen University were collected. The above-mentioned cropping process was then performed at magnification of 4× and 10× to construct a lymph node dataset.
[0094] In addition, as shown in Table 1, during the development and research process of this invention, 69 slices of WSIs containing various glands, tissues and tissues undergoing basic pathological changes were collected from Sun Yat-sen Memorial Hospital of Sun Yat-sen University. The above-mentioned cropping process was then performed at 4× magnification and 10× magnification respectively to construct a meta-learning multi-class dataset.
[0095] These sections of WSIs include epidermis, intestinal glands, hepatic congestion, liver, hepatic steatosis, skeletal muscle, necrosis, parathyroid glands, pus, smooth muscle, granulation tissue, cartilage, renal infarction, renal tubular edema, gastric glands, pyloric glands, fibrous tissue, hyalinization of fibrous tissue, intestinal glands, blood, bronchi, and adipose tissue, totaling 34 categories.
[0096] Table 1 shows the dataset of multi-class sliced WSIs for meta-learning.
[0097]
[0098] 5. Training methods and strategies
[0099] After collecting and labeling a multi-class tissue slice WSIs dataset, a meta-task is constructed using the labeled multi-class tissue slice WSI-patches images to perform meta-training on MetaNet. Then, MetaNet is further trained using the labeled lymph node paraffin slice WSI-patches images from the lymph node dataset.
[0100] This invention employs three optimization-based meta-learning algorithms: MAML, iMAML, and Reptile. During the meta-training phase, both the outer and inner loops utilize the Adam optimizer, with the outer loop learning rate set to 0.001 and the inner loop learning rate set to 0.01. Furthermore, the number of gradient updates in the inner loop for MAML, iMAML, and Reptile are set to 5, 25, and 5, respectively, and the meta-training epoch is 50. In the testing task, this invention uses the Adam optimizer with a learning rate of 0.01 and an epoch of 30.
[0101] This invention utilizes a meta-learning multi-class dataset to construct 200 meta-training tasks, each with parameters of 2ways-5shot-5queries, and meta-trains the MetaNet network. For the meta-testing task, paraffin section data is used. To ensure fairness, 5-fold cross-validation is employed, guaranteeing that patches from the same lymph node only appear in the same fold of data. Furthermore, frozen section data is directly used for testing to measure the model's detection capability across different section types.
[0102] Based on MetaNet, this invention also provides a system for identifying regions of minute changes in lymph nodes in slice images, including:
[0103] A receiving unit for acquiring paraffin or frozen WSI images;
[0104] The cropping unit is used to crop the acquired WSI image into a WSI-field-of-view image of the same size as the real microscope, and then further crop it into WSI-patches images;
[0105] The processing unit is used to identify the lymph node micro-change region in the WSI-patches image using the trained MetaNet model for identifying the lymph node micro-change region in the slice image and obtain the identification result. It is used to generate a discrimination probability map of the WSI-field map to be identified based on the obtained identification result, and merge the discrimination probability map with the WSI-field map to be identified to generate an identified WSI-field map with lymph node micro-change region markings.
[0106] The output unit is used to display the discrimination probability map of the WSI field to be identified and the identified WSI field with markings of small changes in lymph nodes.
[0107] See Figure 2 As shown, based on MetaNet and the aforementioned system, this invention also provides a method for identifying regions of minute changes in lymph nodes in slice images, comprising the following steps:
[0108] Step 1) Obtain frozen or paraffin whole slide images (WSI) at 4× and 10× magnification.
[0109] Step 2) Crop the acquired WSI image into a WSI field-of-view map of the same size as the actual microscope camera field of view of 3072×2048;
[0110] Step 3) Further crop the visual field map to be identified into WSI-patches images of size 224×224;
[0111] Step 4) Input the cropped WSI-patches image into the pre-trained MetaNet model for identifying small changes in lymph nodes in the above sliced image, identify whether there are small changes in lymph nodes in the WSI-patches image, and obtain the identification results;
[0112] Step 5) Based on the obtained recognition results, the WSI-patches images identified as areas of minor changes in lymph nodes are marked in red, and the WSI-patches images identified as normal areas are marked in blue, forming a discrimination probability map of the WSI-field map to be identified;
[0113] Step 6) Merge the resulting discrimination probability map with the original cropped WSI-field map to be identified to form an identified WSI-field map, and in the identified WSI-field map, the areas in the lymph node where tissue changes can be seen are highlighted in red.
[0114] Based on the above identification method, the present invention also provides a computer device, including: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other through the communication bus, the memory is used to store at least one executable instruction, and the executable instruction causes the processor to perform the operation corresponding to the above-mentioned method for identifying small changes in lymph node regions in sliced images.
[0115] Based on the above identification method, the present invention also provides a computer-readable storage medium storing at least one executable instruction, which causes a processor to perform the operation corresponding to the above-described method for identifying small changes in lymph node regions in sliced images.
[0116] The proposed method was compared with other deep learning-based methods in terms of performance. Specifically, five-fold cross-validation was used, including commonly used meta-learning algorithms such as four-layer convolutional networks, ResNet18, AlexNet, Vgg11, and Vit. Tests were conducted on the task of identifying subtle changes in lymph node regions in paraffin and frozen sections, and the results are as follows:
[0117] In the task of identifying subtle changes in lymph node regions in paraffin sections, the method proposed in this invention achieved the highest scores on all five metrics: Acc, Auc, Precision, Recall, and F1.
[0118] As shown in Table 2, in the task of recognizing paraffin slices WSIs at 4× magnification, the method of this invention improves the performance of the second-place method by 5.62%, 4.75%, 3.56%, 5.72%, and 5.99% in five metrics, respectively; and improves the performance of the five metrics by 8.12%, 16.35%, 8.19%, 15.23%, and 12.34% compared to the simple four-layer convolutional network commonly used in meta-learning.
[0119] As shown in Table 2, in the task of identifying small changes in lymph node regions in paraffin sections (WSIs) at 10× magnification, the method of this invention improved the performance of the second-place method by 2.01%, 3.32%, 2.94%, 1.26%, and 1.6% in five metrics, respectively; improved it by 5.27%, 3.32%, 7.1%, 4.46%, and 5.7% compared to the classic segmentation network ResNet18; and improved it by 5.6%, 6.82%, 9.07%, 4.84%, and 6.76% compared to the commonly used simple four-layer convolutional network in meta-learning.
[0120] In addition, all methods achieved higher scores than 4x magnification on most metrics at a 10x magnification slice.
[0121] Table 2. Performance comparison of different networks on the task of identifying subtle changes in lymph node regions in paraffin sections using WSIs.
[0122]
[0123] Frozen section data was used for testing purposes only. The model trained on paraffin section data was directly tested on frozen section data to measure the model's detection capability across section categories. As shown in Table 3, on frozen sections at 4× magnification, the method of this invention achieved the highest scores on both Auc and F1. On frozen sections at 10× resolution, the method of this invention achieved the highest scores on all three metrics: Acc, Auc, and Recall, demonstrating that when generalizing from paraffin sections to frozen sections, the method of this invention has better performance and higher stability compared to other methods.
[0124] Table 3. Performance comparison of different networks on the task of identifying subtle changes in lymph node regions in frozen sections WSIs.
[0125]
[0126] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A model for identifying regions of minute changes in lymph nodes in slice images, characterized in that, The model is a network architecture consisting of a shallow feature extractor, a deep feature extractor, and a fusion feature voter. It constructs a meta-task using a WSIs dataset of multi-class tissue slices, performs meta-training using a meta-learning algorithm, and then further trains using a lymph node slice dataset. This results in a network model specifically designed to identify subtle changes in lymph node regions in paraffin and frozen sections, namely MetaNet. The training method for MetaNet is as follows: Obtain routinely stained lymph node slices and tissue slices from multi-class datasets; The acquired slices were scanned into WSI images with magnifications of 4× and 10×, respectively. The scanned WSI images were cropped into WSI-field-of-view images of the same size as the actual microscope camera's field of view, measuring 3072×2048. The 3072×2048 WSI-field of view image was further cropped into 224×224 WSI-patches images; For WSI-patches images of multi-category tissue sections, labels are made according to the tissues contained within the patches; For WSI-patches images of lymph node sections, the areas of lymphatic tissue change contained within the patches are labeled; The labeled multi-class tissue slice WSI-patches images constitute the multi-class slice WSIs dataset, and the labeled lymph node slice WSI-patches images constitute the lymph node dataset; A meta-task is constructed using the multi-class sliced WSIs dataset, and MetaNet is then meta-trained. MetaNet was further trained using the lymph node dataset; The shallow feature extractor consists of four 3×3 convolutional layers, activation layers, and normalization layers. The shallow feature extractor is used to extract surface features from the input MetaNet slice WSI-patches image to obtain shallow features. The deep feature extractor consists of 12 Transformer blocks and is used to extract more complex features from the shallow features to obtain deep features. The fusion feature voter includes a feature fusion layer, a hidden layer, and a feature voting layer, which are used to combine the shallow features with the deep features, and then use the fusion features to predict the final lymph node micro-change area and output the prediction result.
2. The lymph node minute change region identification model in sliced images according to claim 1, characterized in that, In the deep feature extractor, each Transformer block includes a multi-head self-attention mechanism, a feedforward neural network, and residual connections; wherein, The multi-head self-attention mechanism: In the Transformer block, the input sequence is divided into multiple sub-sequences simultaneously, and then the self-attention weights are calculated for each sub-sequence; this allows the model to pay attention to information at different positions simultaneously when processing the input sequence, which helps to capture long-distance dependencies; The feedforward neural network: After the multi-head self-attention mechanism, the representation of each subsequence is processed by a fully connected feedforward neural network; the feedforward neural network is a two-layer fully connected neural network containing the ReLU activation function; The residual connection: In the Transformer block, the representation of each subsequence is added to its input, i.e., the residual connection; the residual connection helps to alleviate the gradient vanishing problem, making it easier to optimize when training deep models.
3. The lymph node minute change region identification model in sliced images according to claim 2, characterized in that, The deep feature extractor's embed_dim is set to 384, and num_heads is set to 6; embed_dim represents the dimension of the embedding vector encoded for each word or position, i.e., the embedding dimension; num_heads represents the number of attention heads computed in parallel in each self-attention mechanism, i.e., the number of attention heads.
4. The lymph node minute change region identification model in sliced images according to claim 1, characterized in that, The fusion prediction method of the fusion feature voter is as follows: The last layer output of the shallow feature extractor is selected as the shallow feature input of the feature fusion layer; and the outputs of the last 6 Transformer blocks of the deep feature extractor are selected as the deep feature input of the feature fusion layer. The feature fusion layer applies a linear transformation to the shallow and deep features of different scales in the input, mapping them to the same dimensional space, and then concatenates them together to form fused features; The synthesized fused features are input into a two-layer hidden neural network for further processing to help the model learn higher-order feature representations. The fused features processed by the neural network in the hidden layer are input into the feature voting layer to predict the region of minor changes in lymph nodes, and finally the prediction result is output.
5. A system for identifying regions of minute changes in lymph nodes in slice images, characterized in that, include: A receiving unit for acquiring paraffin or frozen WSI images; The cropping unit is used to crop the acquired WSI image into a WSI-field-of-view image of the same size as the real microscope, and then further crop it into WSI-patches images; The processing unit is used to identify the lymph node micro-change region in the WSI-patches image using the trained slice image lymph node micro-change region identification model as described in any one of claims 1-4, and obtain the identification result. It is used to generate a discrimination probability map of the WSI-field map to be identified based on the obtained identification result, and merge the discrimination probability map with the WSI-field map to be identified to generate an identified field map with lymph node micro-change region markings. The output unit is used to display the discrimination probability map of the WSI field to be identified and the identified WSI field with markings of small changes in lymph nodes.
6. A method for identifying regions of minute changes in lymph nodes in slice images, characterized in that, Includes the following steps: Step 1) Obtain WSI images of frozen or paraffin at 4× and 10× magnification; Step 2) Crop the acquired WSI image into a WSI field-of-view map of the same size as the actual microscope camera field of view of 3072×2048; Step 3) Further crop the WSI-field map to be identified into 224×224 WSI-patches images; Step 4) Input the cropped WSI-patches image into the pre-trained slice image lymph node micro-change region recognition model as described in any one of claims 1-4, identify whether there are lymph node micro-change regions in the WSI-patches image, and obtain the recognition result; Step 5) Based on the obtained recognition results, the WSI-patches images identified as areas of minor changes in lymph nodes are marked in red, and the WSI-patches images identified as normal areas are marked in blue, forming a discrimination probability map of the WSI-field map to be identified; Step 6) Merge the resulting discrimination probability map with the original cropped WSI-field map to be identified to form an identified WSI-field map, and highlight the areas in the identified WSI-field map where changes in the tissue within the lymph nodes are visible in red.
7. A computer device, comprising: The system includes a processor, a memory, a communication interface, and a communication bus. The processor, the memory, and the communication interface communicate with each other via the communication bus. The memory stores at least one executable instruction, which causes the processor to perform the operation corresponding to the method for identifying small changes in lymph nodes in sliced images as described in claim 6.
8. A computer-readable storage medium, characterized in that, The computer storage medium stores at least one executable instruction that causes the processor to perform the operation corresponding to the method for identifying small changes in lymph node regions in sliced images as described in claim 6.