A pathological section image analysis method and device based on an image coding large model

Abstract

The application provides a pathological section image analysis method and device based on an image coding large model, and belongs to the technical field of medical images. The method comprises the following steps: cutting an original pathological section image to obtain a pathological section sub-image set; sequentially inputting the pathological section sub-images in the set into a target region segmentation large model to obtain corresponding local target region distribution maps, and then obtaining an overall target region distribution map and generating a set of positive local target region distribution maps; based on the set of positive local target region distribution maps, using an abnormal lesion detection network to generate an abnormal lesion distribution map; and based on the target region distribution map and the abnormal lesion distribution map, finally realizing statistical analysis on the original pathological section image. The application can analyze various pathological section images, automatically identify and classify abnormal lesion regions in the images, and automatically statistically analyze to obtain key clinical indicators such as lesion rates, thereby improving the accuracy and efficiency of diagnosis.

Description

Technical Field

[0001] This invention belongs to the field of medical imaging technology, specifically relating to a method and apparatus for pathological slide image analysis based on a large image coding model. Background Technology

[0002] With the rapid development of medical imaging technology, pathological slides have become an important tool for diagnosing various diseases. However, traditional pathological diagnostic methods mainly rely on pathologists manually observing and analyzing slides to identify and classify lesions. This method is not only time-consuming and labor-intensive, but also easily affected by subjective factors, leading to low diagnostic efficiency and insufficient accuracy. Especially when faced with complex and diverse pathological slides, the limitations of traditional methods become increasingly apparent.

[0003] Currently, there are numerous studies both domestically and internationally on methods for analyzing pathological slide images. For example, the pathology team at Harvard Medical School trained a self-supervised learning model called UNI, specifically designed for pathological slide image analysis, based on the DinoV2 visual model from the MetaAI team. The UNI model is trained in a self-supervised manner, using a visual transformer based on DinoV2 (ViT-L / 16 architecture), and does not rely on labeled data, enabling it to automatically extract efficient features from pathological slides. This model performs well in different types of pathological feature extraction tasks, such as tissue classification and tumor segmentation, and has strong generalization ability. Although pathological slide image analysis methods, represented by UNI, have certain capabilities in feature extraction, they have not yet achieved direct quantitative representation of pathological features. This limits the practical application value of these methods, making it difficult for physicians to efficiently extract clinically valuable information from complex and fragmented image features. Summary of the Invention

[0004] The purpose of this invention is to overcome the lack of quantitative representation of lesion severity in existing pathological slide image analysis methods. To address the shortcomings of existing technologies, this invention proposes a pathological slide image analysis method and device based on a large image coding model. Utilizing deep learning technology, this invention can analyze various pathological slide images, automatically identify and classify abnormal lesion areas in the images, and automatically perform statistical analysis to obtain key clinical indicators such as lesion rate. This solves the problems of low efficiency and poor real-time performance associated with manual slide observation for diagnosis, improving diagnostic accuracy and efficiency, and promoting the advancement and development of digital pathology.

[0005] A first aspect of this invention proposes a method for analyzing pathological slide images based on a large image coding model, comprising:

[0006] The original pathological slide image is segmented according to a preset size to obtain a set of pathological slide sub-images;

[0007] The pathological slide sub-images in the set of pathological slide sub-images are sequentially input into a preset target region segmentation large model to obtain the corresponding local target region distribution map; wherein, the target region segmentation large model consists of an encoder and a decoder, and the encoder adopts an image encoding large model;

[0008] By processing the local target region distribution map, the target region distribution map corresponding to the original pathological slide image is obtained, and a set of positive local target region distribution maps is generated.

[0009] Based on the set of positive local target area distribution maps, an abnormal lesion distribution map corresponding to the original pathological slide image is generated using a preset abnormal lesion detection network.

[0010] Based on the target region distribution map and the abnormal lesion distribution map, statistical analysis of the original pathological slide images is achieved.

[0011] In one specific embodiment of the present invention, obtaining the set of pathological slide sub-images includes:

[0012] 1) Obtain a single pathological slide image containing the complete lesion as the original pathological slide image;

[0013] 2) The original pathological slide images obtained in step 1) are segmented to obtain a set of pathological slide sub-images;

[0014] The size of the pathological slide sub-image is H×W, where H represents the length of the pathological slide sub-image and W represents the width of the pathological slide sub-image.

[0015] The segmentation expression is as follows:

[0016]

[0017] Where height and width represent the length and width of the original pathological slide sub-image, respectively; n is the number of sub-images to be segmented in the length direction, and m is the number of sub-images to be segmented in the width direction; for parts that cannot be completely segmented at the image boundary, mirror filling is used to fill them to a size of H×W pixels before segmentation, and the final number of pathological slide sub-images is n×m.

[0018] In one specific embodiment of the present invention, the encoder of the target region segmentation large model uses the DinoV2 image information encoding large model. The encoder processes the input pathological slide sub-image to generate a multi-scale feature map. The decoder of the target region segmentation large model uses a combination of the main decoder UPerNet and the auxiliary decoder FCNNet. UPerNet and FCNNet work in parallel on the multi-scale feature map generated by the encoder. Then, at the output of the decoder, the results processed by UPerNet and FCNNet are weighted and fused to finally output a local target region distribution map of size H×W corresponding to the input pathological slide sub-image.

[0019] In one specific embodiment of the present invention, obtaining the target region distribution map corresponding to the original pathological slide image and generating a set of positive local target region distribution maps includes:

[0020] Based on the local target region distribution map of each pathological slide sub-image in the pathological slide sub-image set output by the target region segmentation large model, the sub-images corresponding to the local target region distribution map are stitched together according to the positions of the sub-images in the original pathological slide images to obtain the target region distribution map corresponding to the original pathological slide image.

[0021] Traverse all local target region distribution maps, mark the local target region distribution maps that contain target regions as positive, and then number the positively marked local target region distribution maps and their corresponding pathological slide sub-images in the order they were input into the target region segmentation model, finally generating a set of positive local target region distribution maps.

[0022] In one specific embodiment of the present invention, before sequentially inputting the pathological slide sub-images from the pathological slide sub-image set into a preset target region segmentation large model, the method further includes:

[0023] Train the large-scale target region segmentation model;

[0024] The training of the target region segmentation large model includes:

[0025] 1) Construct a training set for a large target region segmentation model;

[0026] The pathological types of the images in the training set of the target region segmentation model are consistent with the pathological types of the original pathological slide images.

[0027] The target region is labeled as the training label of the image in the training set of the target region segmentation large model;

[0028] Then, each image in the training set is divided into a sub-image set of size H×W, which is used as input for training the large target region segmentation model;

[0029] 2) Construct a large-scale model for target region segmentation;

[0030] 3) Use the training set obtained in step 1) to train the target region segmentation large model constructed in step 2) to obtain the trained target region segmentation large model.

[0031] In one specific embodiment of the present invention, the abnormal lesion detection network adopts Attention-UNet, which consists of an encoder and a decoder. The encoder of Attention-UNet is used to progressively reduce the spatial dimension of the image while increasing the number of feature channels to capture more abstract features. The decoder of Attention-UNet is used to progressively restore the spatial dimension of the image and reduce the number of feature channels to achieve accurate localization. In Attention-UNet, each skip connection stage of the encoder and decoder is equipped with an attention module, which controls the features from the encoder and the features at the corresponding positions of the decoder by weighting.

[0032] In one specific embodiment of the present invention, the step of generating an abnormal lesion distribution map corresponding to the original pathological slide image based on the set of positive local target region distribution maps and using a preset abnormal lesion detection network includes:

[0033] 1) Input the pathological slide sub-images corresponding to each distribution map in the set of positive local target area distribution maps into the preset abnormal lesion detection network in sequence. The network outputs the corresponding local abnormal lesion distribution map.

[0034] 2) The local abnormal lesion distribution map obtained in step 1) is stitched together according to the position of its corresponding pathological slide sub-image in the original pathological slide image. During the stitching, any gaps are filled to form a completely empty distribution map. After the stitching is completed, a complete abnormal lesion distribution map is obtained.

[0035] In one specific embodiment of the present invention, before generating the abnormal lesion distribution map corresponding to the original pathological slide image using a preset abnormal lesion detection network, the method further includes:

[0036] Train the abnormal lesion detection network;

[0037] Training the abnormal lesion detection network includes:

[0038] 1) Construct a training set for the abnormal lesion detection network;

[0039] The pathological types of the images in the training set of the abnormal lesion detection network are consistent with the pathological types of the original pathological slide images.

[0040] The lesion regions are labeled as training labels for the images in the training set of the abnormal lesion detection network.

[0041] Then, each image in the training set is divided into a sub-image set of size H×W, which is used as input for training the abnormal lesion detection network;

[0042] 2) Construct an abnormal lesion detection network;

[0043] 3) Use the training set of the abnormal lesion detection network model obtained in step 1) to train the abnormal lesion detection network constructed in step 2) to obtain the trained abnormal lesion detection network.

[0044] In one specific embodiment of the present invention, the statistical analysis of the original pathological slide images includes:

[0045] Based on the distribution map of abnormal lesions, the number of abnormal lesions and the total area of ​​lesions are obtained by using the connected domain counting algorithm and area statistics.

[0046] In the statistical abnormal lesion distribution map, the number of closed connected regions enclosed by lesion labels is the number of abnormal lesions, denoted as n1;

[0047] The total area of ​​the lesion, denoted as S1, is the sum of the number of pixels in the closed connected region enclosed by the lesion labels in the statistical abnormal lesion distribution map. The expression is as follows:

[0048]

[0049] Among them, S i The number of pixels occupied by the i-th lesion region;

[0050] The number of target areas and the proportion of abnormal lesions are obtained from the target area distribution map;

[0051] Among them, the number of closed connected regions enclosed by the target region labels in the statistical target region distribution map is the number of target regions, denoted as n2;

[0052] The total area of ​​the target region, denoted as S2, is the sum of the number of pixels in the closed connected region enclosed by the target region label in the statistical target region distribution map. The expression is as follows:

[0053] Among them, S i This represents the number of pixels occupied by the i-th target region;

[0054] The percentage of abnormal lesions is the ratio of the number of abnormal lesions to the total number of target areas, expressed as follows:

[0055]

[0056] Where, r num This is recorded as the percentage of abnormal lesions.

[0057] The percentage of abnormal lesion area is the ratio of the total lesion area to the total target area, expressed as follows:

[0058]

[0059] Where, r area This is recorded as the percentage of abnormal lesion area.

[0060] A second aspect of the present invention provides a pathological slide image analysis device based on an image coding large model, comprising:

[0061] The image segmentation module is used to segment the original pathological slide image according to a preset size to obtain a set of pathological slide sub-images;

[0062] The local target region distribution map generation module is used to sequentially input the pathological slide sub-images in the pathological slide sub-image set into a preset target region segmentation large model to obtain the corresponding local target region distribution map; wherein, the target region segmentation large model consists of an encoder and a decoder, and the encoder adopts an image encoding large model;

[0063] The target region distribution map generation module is used to process the local target region distribution map to obtain the target region distribution map corresponding to the original pathological slide image and generate a set of positive local target region distribution maps;

[0064] An abnormal lesion distribution map generation module is used to generate an abnormal lesion distribution map corresponding to the original pathological slide image based on the set of positive local target area distribution maps and using a preset abnormal lesion detection network.

[0065] The statistical analysis module is used to perform statistical analysis on the original pathological slide images based on the target area distribution map and the abnormal lesion distribution map.

[0066] A third aspect of the present invention provides an electronic device comprising:

[0067] At least one processor; and a memory communicatively connected to said at least one processor;

[0068] The memory stores instructions that can be executed by the at least one processor, and the instructions are configured to perform the above-described method for analyzing pathological slide images based on a large image coding model.

[0069] A fourth aspect of the present invention provides a computer-readable storage medium storing computer instructions for causing the computer to execute the above-described method for pathological slide image analysis based on an image coding large model.

[0070] The features and beneficial effects of this invention are as follows:

[0071] This invention utilizes deep learning technology to analyze various pathological slide images, automatically identify and classify abnormal lesion areas in the images; specifically, it can identify various lesion features, such as changes in tissue structure and abnormal cell morphology, and provide classification result images and quantitative indicators such as the proportion of lesion areas.

[0072] The main advantages of this invention lie in its efficiency and accuracy. Through automated analysis using deep learning models, the workload of pathologists can be significantly reduced, improving diagnostic efficiency. Simultaneously, by eliminating the influence of subjective human factors, this invention can provide more objective and accurate diagnostic results. Furthermore, based on big data and image encoding technologies, this invention can effectively integrate and analyze pathology slide data from different hospitals, breaking down data silos and achieving the sharing and optimization of medical resources. Attached Figure Description

[0073] Figure 1 This is an overall flowchart of a pathological slide image analysis method based on a large image coding model according to an embodiment of the present invention. Detailed Implementation

[0074] This invention proposes a method and apparatus for pathological slide image analysis based on a large image coding model, which will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0075] A first aspect of this invention proposes a method for analyzing pathological slide images based on a large image coding model, comprising:

[0076] The original pathological slide image is segmented according to a preset size to obtain a set of pathological slide sub-images;

[0077] The pathological slide sub-images in the set of pathological slide sub-images are sequentially input into a preset target region segmentation large model to obtain the corresponding local target region distribution map; wherein, the target region segmentation large model consists of an encoder and a decoder, and the encoder adopts an image encoding large model;

[0078] By processing the local target region distribution map, the target region distribution map corresponding to the original pathological slide image is obtained, and a set of positive local target region distribution maps is generated.

[0079] Based on the set of positive local target area distribution maps, an abnormal lesion distribution map corresponding to the original pathological slide image is generated using a preset abnormal lesion detection network.

[0080] Based on the target region distribution map and the abnormal lesion distribution map, statistical analysis of the original pathological slide images is achieved.

[0081] In one specific embodiment of the present invention, the overall process of the pathological slide image analysis method based on an image coding large model is as follows: Figure 1 As shown, it includes the following steps:

[0082] 1) The acquired pathological slide images are segmented to obtain a set of pathological slide sub-images; the specific steps are as follows:

[0083] 1-1) Obtain a single pathological slide image containing a complete lesion.

[0084] This embodiment can load pathological slide images from a slide scanner, storage device, or microscope. The image needs to include the entire field of view of the slide, and the file format can be .jpg or .png. The storage device can be a portable hard drive, USB flash drive, or computer internal storage containing the pathological slide images.

[0085] In this embodiment, the pathological slide images can also be acquired by a microscope. That is, the stained pathological slides are placed on the microscope stage, the microscope is focused, and a camera is used to acquire a scan image of the pathological slides under the microscope's field of view. The pathological slides are moved, and finally all the scan images of the pathological slides are stitched together to form a complete pathological slide image.

[0086] 1-2) The pathological slide images obtained in step 1) are segmented to obtain a set of pathological slide sub-images;

[0087] In this set of sub-images, each sub-image is H×W in size, that is, the length of the sub-image is H and the width is W, and the unit is pixels (px). For different pathological slide images, H and W are not fixed.

[0088] In a specific embodiment of the present invention, the expression for image segmentation of the original pathological slide image is as follows, where H = 512 and W = 512:

[0089]

[0090] Where height and width represent the length and width of the pathological slide sub-image, respectively, in pixels (px); in a specific embodiment of the present invention, height = 512 and width = 512. n is the number of sub-images to be segmented in the height direction, and m is the number of sub-images to be segmented in the width direction. In this embodiment, the pathological slide image is segmented in steps of 512 pixels, from the top left to the bottom right. For parts that cannot be completely segmented at the image boundary, mirror filling is used to fill them to a size of 512×512 pixels before segmentation. The final number of 512×512 pixel pathological slide sub-images should be n×m. All pathological slide sub-images constitute a pathological slide sub-image set.

[0091] 2) Input the pathological slide sub-images obtained after segmentation in step 1) into the preset target region segmentation large model F-model to obtain the corresponding local target region distribution map; by processing the local target region distribution map, obtain the target region distribution map of the corresponding original pathological slide image and generate a set of positive local target region distribution maps; the specific steps are as follows:

[0092] 2-1) Input the pathological slide sub-image after step 1) into the preset target region segmentation large model F-model. The large model outputs the local target region distribution map corresponding to the input sub-image.

[0093] The target region segmentation large model F-model consists of an encoder and a decoder. In this embodiment, the encoder uses the DinoV2 image information encoding large model, which extracts features from the input image and outputs multi-scale, multi-layer feature maps. In this embodiment, for each 512×512 pixel pathological slide sub-image, the encoder outputs feature maps at three scales: 64x64 (8x downsampling), 32x32 (16x downsampling), and 16x16 (32x downsampling). The decoder uses a combination of the main decoder UPerNet and the auxiliary decoder FCNNet. That is, UPerNet and FCNNet work in parallel on the multi-scale feature maps generated by the encoder. UPerNet and FCNNet process these feature maps independently, and then the results processed by UPerNet and FCNNet are weighted and fused at the output of the decoder. This method ensures that the strengths of the two decoders are utilized simultaneously. UPerNet is responsible for processing global structural information, while FCNNet is responsible for processing detailed information.

[0094] In this embodiment, the multi-scale, multi-layer feature maps output by the encoder are input into the decoder. The main decoder and the auxiliary decoder complete the recognition of the target region, and finally, the recognition results obtained by the main decoder and the auxiliary decoder are weighted and fused. Initially, their respective weights are set (e.g., 0.5 and 0.5). The weight values ​​can be adjusted manually: the weights can be gradually adjusted experimentally to observe their impact on segmentation accuracy; alternatively, an adaptive learning mechanism can be introduced to achieve automatic weight adjustment. Specifically, two initial weights w1 and w2 are set, corresponding to the main decoder (UPerNet) and the auxiliary decoder (FCNNet) respectively. These weights can be initialized to the same value (0.5), with the constraint w1 + w2 = 1. During model training, gradient descent is used to optimize all parameters in the model, including the network weights of the decoder.

[0095] This embodiment, after processing by the target region segmentation large model F-model, finally obtains a local target region distribution map of 512×512 pixels in size for the corresponding input pathological slide sub-image;

[0096] 2-2) The distribution maps of all local target regions obtained in step 2-1) are stitched together according to the positions of their corresponding sub-images in the original pathological slide images to obtain the complete distribution map of the target regions.

[0097] 2-3) Traverse all local target region distribution maps, mark the local target region distribution maps with target region distribution as positive, and mark the local target region distribution maps without target region distribution as negative. Then, number the local target region distribution maps with positive marking and the local target region distribution maps with negative marking according to the input order of their corresponding pathological slide sub-images in step 2-1), and finally form a set of positive local target region distribution maps and a set of negative local target region distribution maps respectively.

[0098] Furthermore, prior to this step, the method described in this embodiment also includes training the large-scale target region segmentation model, specifically including:

[0099] a) Construct a training set for a large target region segmentation model;

[0100] In this embodiment, the images in the training set of the target region segmentation model come from diverse sources, including public datasets, pathological slide scanners, microscopes, and other equipment. The pathological types of the images in this training set need to be consistent with the pathological slide images in step 1), and they need to include the complete field of view of the entire slide. This embodiment does not require that the brand or type of the acquisition equipment be consistent, nor does it require that the size of the complete pathological slide images be consistent, but the magnification of the acquisition equipment and the pixel density (DPI) of the images need to be consistent.

[0101] Each complete pathological slide image must be annotated by a clinical pathologist to delineate the target region, which serves as the training label for that image. The more images in the training set of the target region segmentation model, the better the final training model will perform; generally, more than 50 images should be used.

[0102] Furthermore, the training set images need to be segmented. In this embodiment, each image in the training set is segmented into a sub-image set of size H×W (e.g., 512×512 pixels) as input for training the target region segmentation large model.

[0103] b) Construct a large-scale model for target region segmentation.

[0104] In this embodiment, the target region segmentation large model consists of an encoder and a decoder. The encoder uses a model such as DinoV2 for multi-scale feature extraction, and the decoder uses a combination of UPerNet and FCNNet to process feature maps of different scales, and outputs the combined data.

[0105] c) Train the target region segmentation model using the training set of the target region segmentation model.

[0106] Optimizer: The Adam optimizer is typically used, with a learning rate of 1e-4, but this can be adjusted based on the size of the training set and the complexity of the model.

[0107] Loss function: Cross-entropy loss is used for the segmentation task.

[0108] Batch size: Usually set to 16 or 32, the specific size depends on the hardware's memory capacity and training speed.

[0109] Learning rate scheduler: CosineAnnealing is used to gradually reduce the learning rate to avoid overfitting during training.

[0110] Regularization method: In order to prevent overfitting, L2 regularization is added during training.

[0111] Training termination conditions:

[0112] In this embodiment, training stops when the loss on the preset validation set no longer decreases, or when the loss stabilizes after several epochs. A maximum number of epochs can be set, such as 100 epochs, depending on the actual situation. Metrics for measuring the model's segmentation performance, such as the Dice coefficient or IoU (Intersection over Union), can also be set. Training stops when these metrics reach a certain threshold on the validation set, such as 90%.

[0113] Specifically, the training process is as follows:

[0114] In each epoch, the model generates predictions through forward propagation, calculates the error using a loss function, and updates the network weights using backpropagation. A validation set is used to evaluate the model's generalization performance and prevent overfitting. During training, the loss and segmentation accuracy on the validation set are monitored. Training ends when the early stopping condition is met or the maximum number of iterations is reached, and the optimal model is saved as the final trained target region segmentation model.

[0115] 3) Based on the set of positive local target area distribution maps obtained in step 2), an abnormal lesion distribution map corresponding to the original pathological slide image is generated using a pre-set abnormal lesion detection network; the specific steps are as follows.

[0116] 3-1) Input the pathological slide sub-images corresponding to each distribution map in the set of positive local target area distribution maps into the preset abnormal lesion detection network Attention-UNet in sequence. The network outputs the corresponding local abnormal lesion distribution map.

[0117] Attention-UNet is an advanced deep learning architecture based on the classic U-Net model. It enhances the model's ability to recognize key features in images by introducing an attention mechanism; specifically, an attention module is added to the skip connections between the encoder and decoder of the original U-Net. Attention-UNet consists of an encoder and a decoder. The encoder progressively reduces the spatial dimensionality of the image while increasing the number of feature channels to capture more abstract features. The decoder progressively restores the spatial dimensionality of the image and reduces the number of feature channels to achieve accurate localization. In Attention-UNet, each skip connection stage of the encoder and decoder includes an attention module. This module weights the features from the encoder and the corresponding features from the decoder, passing more important feature information to the corresponding decoding layer and suppressing features irrelevant or secondary to the segmentation task. In this embodiment, the network structure that combines the context awareness capability and the focusing capability of the attention mechanism of U-Net can accurately locate abnormal lesions. In a specific embodiment of the present invention, a pathological slide image of size 512×512 pixels is input into the network, and finally a local abnormal lesion distribution map of 512×512 pixels is obtained by processing and outputting the Attention-UNet.

[0118] In this embodiment, through this step, the pathological slide sub-images corresponding to all the positive local target area distribution maps obtain their corresponding local abnormal lesion distribution maps.

[0119] 3-2) The local abnormal lesion distribution map obtained in step 3-1) is stitched together according to the position of its corresponding pathological section sub-image in the original pathological section image to obtain a complete abnormal lesion distribution map.

[0120] Since this embodiment only processes the positive marker sub-images, some gaps will appear during the stitching process. For the gaps, it is assumed that there are no abnormal lesions, and they are directly filled into a completely empty distribution map (if there are lesions, mark them as 1; if there are no lesions, mark them as 0; then mark all gaps as 0). After stitching, a complete abnormal lesion distribution map is finally obtained.

[0121] Furthermore, prior to this step, the method described in this embodiment also includes training the abnormal lesion detection network, specifically including:

[0122] a) Construct a training set for the abnormal lesion detection network;

[0123] In this embodiment, the images in the training set of the abnormal lesion detection network come from diverse sources, including public datasets, pathological slide scanners, microscopes, and other equipment. The pathological types of the images in this training set need to be consistent with the pathological slide images in step 1), and they need to include the complete field of view of the entire slide. This embodiment does not require that the brand or type of the acquisition equipment be consistent, nor does it require that the size of the complete pathological slide images be consistent, but the magnification of the acquisition equipment and the pixel density (DPI) of the images need to be consistent.

[0124] Each complete pathological slide image must be annotated by a clinical pathologist, outlining the lesion area, which serves as the training label for that image. The more images in the training set of the abnormal lesion detection network, the better the final training model will perform; generally, more than 50 images are recommended.

[0125] Furthermore, the training set images need to be segmented. In this embodiment, each image in the training set is segmented into a sub-image set of size H×W (e.g., 512×512 pixels) as input for training the abnormal lesion detection network.

[0126] b) Construct an abnormal lesion detection network.

[0127] In this embodiment, the abnormal lesion detection network uses Attention-UNet, where the model uses an attention mechanism between the encoder and decoder to enhance the model's attention to key regions.

[0128] c) Train the abnormal lesion detection network using the training set of the abnormal lesion detection network model.

[0129] The training parameters in this embodiment are set as follows:

[0130] Optimizer: The Adam optimizer is typically used, with a learning rate of 1e-4, but this can be adjusted based on the size of the training set and the complexity of the model.

[0131] Loss function: Cross-entropy loss is used for the segmentation task.

[0132] Batch size: Usually set to 16 or 32, the specific size depends on the hardware's memory capacity and training speed.

[0133] Learning rate scheduler: The learning rate is gradually reduced using the cosine annealing algorithm to avoid overfitting during training.

[0134] Regularization method: In order to prevent overfitting, L2 regularization is added during training.

[0135] Training termination conditions:

[0136] In this embodiment, training stops when the loss on the preset validation set no longer decreases, or when the loss stabilizes after several epochs. A maximum number of epochs can be set, such as 100 epochs, depending on the actual situation. Metrics for measuring the model's segmentation performance, such as the Dice coefficient or IoU (Intersection over Union), can also be set. Training stops when these metrics reach a certain threshold on the validation set, such as 90%.

[0137] Specifically, the training process is as follows:

[0138] In each epoch, the model generates predictions through forward propagation, calculates the error using a loss function, and updates the network weights using backpropagation. A validation set is used to evaluate the model's generalization performance and prevent overfitting. During training, the loss and segmentation accuracy on the validation set are monitored. Training ends when the early stopping condition is met or the maximum number of iterations is reached, and the optimal model is saved as the final trained abnormal lesion detection network model.

[0139] 4) Based on the target area distribution map from step 2) and the abnormal lesion distribution map from step 3), the distribution of abnormal lesions in the pathological slide image is statistically analyzed, and lesion count and lesion rate analysis are performed; specifically including:

[0140] Based on the distribution map of abnormal lesions, the number of abnormal lesions and the total area of ​​lesions can be obtained using connected domain counting algorithms and area statistics. These two indicators reflect the distribution area and severity of lesions in the pathological slide image, and are important bases for physicians to make diagnoses, thus possessing significant importance.

[0141] The number of abnormal lesions can be obtained by counting the number of closed connected regions enclosed by lesion labels in the abnormal lesion distribution map. This number is defined as the number of abnormal lesions, denoted as n1.

[0142] The total lesion area can be obtained as follows: The total number of pixels in the closed connected region enclosed by the lesion labels in the abnormal lesion distribution map is defined as the total lesion area, denoted as S1. The calculation method is as follows:

[0143]

[0144] Among them, S i The number of pixels occupied by the i-th lesion region.

[0145] The target area distribution map provides the number of target areas and the percentage of abnormal lesions. The target area refers to suspected lesion areas, i.e., areas where lesions may develop. The percentage of abnormal lesions, including both the percentage by number and the percentage by area, provides physicians with a comprehensive view of the overall lesion situation in the pathological section image, serving as a crucial diagnostic basis.

[0146] The number of target regions can be obtained by counting the number of closed connected regions enclosed by the target region labels in the target region distribution map. This number is defined as the number of target regions, denoted as n2.

[0147] The total area of ​​the target region can be obtained as follows: The total number of pixels in the closed connected region enclosed by the target region label in the target region distribution map is defined as the total area of ​​the target region, denoted as S2. The calculation method is as follows:

[0148]

[0149] Among them, S i The number of pixels occupied by the i-th target region.

[0150] The percentage of abnormal lesions can be obtained as follows: The percentage of abnormal lesions is the ratio of the number of abnormal lesions to the total number of target areas, calculated as follows:

[0151]

[0152] Where, r num This is recorded as the percentage of abnormal lesions.

[0153] The percentage of abnormal lesion area can be obtained as follows: The percentage of abnormal lesion area is the ratio of the total lesion area to the total target area, and the calculation method is as follows:

[0154]

[0155] Where, r area This is recorded as the percentage of abnormal lesion area.

[0156] Furthermore, the method described in this embodiment also includes:

[0157] The target area distribution map, abnormal lesion distribution map, and generated quantitative indicators, including total lesion area, percentage of abnormal lesion number, and percentage of abnormal lesion area, are visualized and output in both image and text format to assist physicians in diagnosis.

[0158] To implement the above embodiments, a second aspect of the present invention proposes a pathological slide image analysis device based on an image coding large model, comprising:

[0159] The image segmentation module is used to segment the original pathological slide image according to a preset size to obtain a set of pathological slide sub-images;

[0160] The local target region distribution map generation module is used to sequentially input the pathological slide sub-images in the pathological slide sub-image set into a preset target region segmentation large model to obtain the corresponding local target region distribution map; wherein, the target region segmentation large model consists of an encoder and a decoder, and the encoder adopts an image encoding large model;

[0161] The target region distribution map generation module is used to process the local target region distribution map to obtain the target region distribution map corresponding to the original pathological slide image and generate a set of positive local target region distribution maps;

[0162] An abnormal lesion distribution map generation module is used to generate an abnormal lesion distribution map corresponding to the original pathological slide image based on the set of positive local target area distribution maps and using a preset abnormal lesion detection network.

[0163] The statistical analysis module is used to perform statistical analysis on the original pathological slide images based on the target area distribution map and the abnormal lesion distribution map.

[0164] It should be noted that the foregoing explanation of an embodiment of a pathological slide image analysis method based on an image coding large model also applies to a pathological slide image analysis device based on an image coding large model in this embodiment, and will not be repeated here. According to an embodiment of the present invention, a pathological slide image analysis device based on an image coding large model obtains a set of pathological slide sub-images by segmenting the original pathological slide image according to a preset size; the pathological slide sub-images in the set are sequentially input into a preset target region segmentation large model to obtain corresponding local target region distribution maps; wherein, the target region segmentation large model consists of an encoder and a decoder, and the encoder adopts an image coding large model; by processing the local target region distribution maps, a target region distribution map corresponding to the original pathological slide image is obtained, and a set of positive local target region distribution maps is generated; based on the set of positive local target region distribution maps, an abnormal lesion distribution map corresponding to the original pathological slide image is generated using a preset abnormal lesion detection network; based on the target region distribution map and the abnormal lesion distribution map, statistical analysis of the original pathological slide image is realized. This enables the analysis of various pathological slide images, automatic identification and classification of abnormal lesion areas in the images, and automatic statistical analysis to obtain key clinical indicators such as lesion rate. This solves the problems of low efficiency and poor real-time performance caused by the current manual observation of slides for diagnosis, improves the accuracy and efficiency of diagnosis, and promotes the progress and development of digital pathology.

[0165] To implement the above embodiments, a third aspect of the present invention provides an electronic device, comprising:

[0166] At least one processor; and a memory communicatively connected to said at least one processor;

[0167] The memory stores instructions that can be executed by the at least one processor, and the instructions are configured to perform the above-described method for analyzing pathological slide images based on a large image coding model.

[0168] To implement the above embodiments, a fourth aspect of the present invention provides a computer-readable storage medium storing computer instructions for causing the computer to execute the above-described method for pathological slide image analysis based on an image coding large model.

[0169] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0170] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform a pathological slide image analysis method based on an image coding large model according to the above embodiments.

[0171] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0172] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0173] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0174] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the function involved, as will be understood by those skilled in the art to which embodiments of this application pertain.

[0175] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which programs can be printed, because programs can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0176] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0177] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0178] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0179] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. A method for analyzing pathological slide images based on a large image coding model, characterized in that, include: The original pathological slide image is segmented according to a preset size to obtain a set of pathological slide sub-images; The pathological slide sub-images in the set of pathological slide sub-images are sequentially input into a preset target region segmentation large model to obtain the corresponding local target region distribution map; wherein, the target region segmentation large model consists of an encoder and a decoder, and the encoder adopts an image encoding large model; By processing the local target region distribution map, the target region distribution map corresponding to the original pathological slide image is obtained, and a set of positive local target region distribution maps is generated. Based on the set of positive local target area distribution maps, an abnormal lesion distribution map corresponding to the original pathological slide image is generated using a preset abnormal lesion detection network. Based on the target area distribution map and the abnormal lesion distribution map, statistical analysis of the original pathological slide images is achieved; The step of obtaining the target region distribution map corresponding to the original pathological slide image and generating a set of positive local target region distribution maps includes: Based on the local target region distribution map of each pathological slide sub-image in the pathological slide sub-image set output by the target region segmentation large model, the sub-images corresponding to the local target region distribution map are stitched together according to the positions of the sub-images in the original pathological slide images to obtain the target region distribution map corresponding to the original pathological slide image. Traverse all local target region distribution maps, mark the local target region distribution maps that contain target regions as positive, and then number the positively marked local target region distribution maps and their corresponding pathological slide sub-images in the order they were input into the target region segmentation model, finally generating a set of positive local target region distribution maps.

2. The method according to claim 1, characterized in that, The obtained set of pathological slide sub-images includes: 1) Obtain a single pathological slide image containing the complete lesion as the original pathological slide image; 2) The original pathological slide images obtained in step 1) are segmented to obtain a set of pathological slide sub-images; The size of the pathological slide sub-image is H×W, where H represents the length of the pathological slide sub-image and W represents the width of the pathological slide sub-image. The segmentation expression is as follows: Where height and width represent the length and width of the original pathological slide image, respectively; This represents the number of sub-images to be segmented along the length direction. This represents the number of sub-images to be segmented along the width direction. For portions that cannot be completely segmented at the image boundary, mirror filling is used to fill them to an H×W pixel size before segmentation. The final number of pathological slide sub-images is... .

3. The method according to claim 1, characterized in that, The encoder of the target region segmentation large model uses DinoV2 image information encoding to generate a multi-scale feature map after processing the input pathological slide sub-image. The decoder of the target region segmentation large model uses a combination of the main decoder UPerNet and the auxiliary decoder FCNNet. UPerNet and FCNNet work in parallel on the multi-scale feature map generated by the encoder. Then, at the output of the decoder, the results processed by UPerNet and FCNNet are weighted and fused to finally output a local target region distribution map of size H×W corresponding to the input pathological slide sub-image.

4. The method according to claim 3, characterized in that, Before sequentially inputting the pathological slide sub-images from the set of pathological slide sub-images into a preset target region segmentation large model, the method further includes: Train the large-scale target region segmentation model; The training of the target region segmentation large model includes: 1) Construct a training set for a large target region segmentation model; The pathological types of the images in the training set of the target region segmentation model are consistent with the pathological types of the original pathological slide images. The target region is labeled as the training label of the image in the training set of the target region segmentation large model; Then, each image in the training set is divided into a sub-image set of size H×W, which is used as input for training the large target region segmentation model; 2) Construct a large-scale model for target region segmentation; 3) Use the training set obtained in step 1) to train the target region segmentation large model constructed in step 2) to obtain the trained target region segmentation large model.

5. The method according to claim 3, characterized in that, The abnormal lesion detection network employs Attention-UNet, which consists of an encoder and a decoder. The encoder of Attention-UNet progressively reduces the spatial dimension of the image while increasing the number of feature channels to capture more abstract features. The decoder of Attention-UNet progressively restores the spatial dimension of the image and reduces the number of feature channels to achieve accurate localization. In Attention-UNet, each skip connection stage of the encoder and decoder is equipped with an attention module, which controls the weighted distribution of features from the encoder and features at corresponding locations in the decoder.

6. The method according to claim 5, characterized in that, The step of generating an abnormal lesion distribution map corresponding to the original pathological slide image based on the set of positive local target region distribution maps and using a preset abnormal lesion detection network includes: 1) Input the pathological slide sub-images corresponding to each distribution map in the set of positive local target area distribution maps into the preset abnormal lesion detection network in sequence. The network outputs the corresponding local abnormal lesion distribution map. 2) The local abnormal lesion distribution map obtained in step 1) is stitched together according to the position of its corresponding pathological slide sub-image in the original pathological slide image. During the stitching, any gaps are filled to form a completely empty distribution map. After the stitching is completed, a complete abnormal lesion distribution map is obtained.

7. The method according to claim 5, characterized in that, Before generating the abnormal lesion distribution map corresponding to the original pathological slide image using a preset abnormal lesion detection network, the method further includes: Train the abnormal lesion detection network; Training the abnormal lesion detection network includes: 1) Construct a training set for the abnormal lesion detection network; The pathological types of the images in the training set of the abnormal lesion detection network are consistent with the pathological types of the original pathological slide images. The lesion regions are labeled as training labels for the images in the training set of the abnormal lesion detection network. Then, each image in the training set is divided into a sub-image set of size H×W, which is used as input for training the abnormal lesion detection network; 2) Construct an abnormal lesion detection network; 3) Use the training set of the abnormal lesion detection network model obtained in step 1) to train the abnormal lesion detection network constructed in step 2) to obtain the trained abnormal lesion detection network.

8. The method according to claim 6, characterized in that, The statistical analysis of the original pathological slide images includes: Based on the distribution map of abnormal lesions, the number of abnormal lesions and the total area of ​​lesions are obtained by using the connected domain counting algorithm and area statistics. In the statistical abnormal lesion distribution map, the number of closed connected regions enclosed by lesion labels represents the number of abnormal lesions, denoted as . ; In the statistical distribution map of abnormal lesions, the total area of ​​the lesions is the sum of the pixels in the closed connected region enclosed by the lesion labels. The expression is as follows: in, For the first The number of pixels occupied by each lesion area; The number of target areas and the proportion of abnormal lesions are obtained from the target area distribution map; In the statistical target area distribution map, the number of closed connected regions enclosed by the target area labels is the number of target areas, denoted as . ; The total area of ​​the target region is the sum of the number of pixels in the closed connected region enclosed by the target region label in the statistical target region distribution map. This area is denoted as [missing information]. The expression is as follows: in, For the first The number of pixels occupied by each target region; The percentage of abnormal lesions is the ratio of the number of abnormal lesions to the total number of target areas, expressed as follows: in, This is recorded as the percentage of abnormal lesions. The percentage of abnormal lesion area is the ratio of the total lesion area to the total target area, expressed as follows: in, This is recorded as the percentage of abnormal lesion area.

9. A pathological slide image analysis device based on a large image coding model, characterized in that, include: The image segmentation module is used to segment the original pathological slide image according to a preset size to obtain a set of pathological slide sub-images; The local target region distribution map generation module is used to sequentially input the pathological slide sub-images in the pathological slide sub-image set into a preset target region segmentation large model to obtain the corresponding local target region distribution map; wherein, the target region segmentation large model consists of an encoder and a decoder, and the encoder adopts an image encoding large model; The target region distribution map generation module is used to process the local target region distribution map to obtain the target region distribution map corresponding to the original pathological slide image and generate a set of positive local target region distribution maps; An abnormal lesion distribution map generation module is used to generate an abnormal lesion distribution map corresponding to the original pathological slide image based on the set of positive local target area distribution maps and using a preset abnormal lesion detection network. The statistical analysis module is used to perform statistical analysis on the original pathological slide images based on the target area distribution map and the abnormal lesion distribution map; The step of obtaining the target region distribution map corresponding to the original pathological slide image and generating a set of positive local target region distribution maps includes: Based on the local target region distribution map of each pathological slide sub-image in the pathological slide sub-image set output by the target region segmentation large model, the sub-images corresponding to the local target region distribution map are stitched together according to the positions of the sub-images in the original pathological slide images to obtain the target region distribution map corresponding to the original pathological slide image. Traverse all local target region distribution maps, mark the local target region distribution maps that contain target regions as positive, and then number the positively marked local target region distribution maps and their corresponding pathological slide sub-images in the order they were input into the target region segmentation model, finally generating a set of positive local target region distribution maps.

Images