Rapid tumor localization method based on multi-resolution full-field pathological slice images

Abstract

The present invention provides a method for rapid tumor localization based on full-field pathology slice images with multiple resolutions, including: constructing an image block training set; using the image block training set to obtain two classification models; and using the two classification models in succession to predict full-field pathology slices to obtain positioning results. The present invention trains two models of preset resolutions respectively while ensuring that image detail information is not sufficiently lost, one classification model uses a faster full convolutional network, and the other classification model adds an efficient feature extraction module. Compared with the traditional framework that uses a single resolution for positioning, it achieves rapid positioning of tumor areas on full-field pathology slices while ensuring positioning accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of deep learning and medical image analysis, and in particular to a method for rapid tumor positioning based on multi-resolution full-field pathological slice images. Background Art

[0002] In recent years, with the continuous development of deep learning, the combination of deep learning and other fields has achieved more and more results. In the task of medical image analysis, tumor detection on pathological images is a key step in the diagnosis of clinical pathologists. The automated auxiliary pathology detection system produced by combining the two can greatly reduce the misdiagnosis rate and diagnosis time of pathologists. With the continuous development of digital pathology imaging systems, from whole slide digital scanning technology to high-throughput and fast full-field pathology slice technology (WSI: whole slide image), it is very time-consuming to locate a tumor area at the level of ultra-large digital pathology images. Often in the clinical application stage, a doctor needs 5-10 minutes to read a 150000*150000 pixel digital pathology image, and the existing methods are generally slow. Since pathological image detection is the "gold standard" for tumor diagnosis, it is necessary to be very cautious about this technology. Therefore, the main purpose of the designed system is to assist pathologists in diagnosis, which requires a fast and accurate positioning method to assist pathologists in tumor judgment, further reduce their workload, and improve the accuracy of diagnosis.

[0003] Recent studies have shown that convolutional neural networks are widely used as feature extraction methods for analyzing full-view pathology slice images in tumor localization tasks in full-view pathology slices. Existing methods pre-train tumor features at the highest magnification, and use a sliding window method to crop image blocks from slice images at the highest magnification and input them into the network to determine whether they are tumors or non-tumors during the reasoning process. However, the highest magnification represents the original resolution image. Although the image details are clear, the number of sliding windows in the reasoning process is large, resulting in a huge amount of calculation, which makes the reasoning time unable to meet clinical needs. Therefore, it is necessary to design a fast positioning method for full-view pathology slices that can ensure both positioning accuracy and positioning speed. Summary of the invention

[0004] In view of the defects in the prior art, the object of the present invention is to provide a method for rapid tumor positioning based on multi-resolution full-field pathological slice images.

[0005] According to one aspect of the present invention, a method for rapid tumor location based on multi-resolution full-field pathological slice images is provided, comprising:

[0006] Construct image patch training set;

[0007] Using the image block training set, two classification models are obtained;

[0008] Two classification models were used successively to predict the full-field pathological sections and obtain the positioning results.

[0009] Preferably, the constructing of the image block training set includes:

[0010] Reading the full-field pathology slide, and extracting the foreground and background of the full-field pathology slide by the big law method;

[0011] In the foreground and background regions, image blocks of preset sizes with different magnifications are extracted to construct image block data sets with different resolutions, and category annotations of the image block data sets are obtained. Data sets with different resolutions are constructed for training, and model feature parameters at different magnifications are obtained.

[0012] Preferably, the two classification models both have characteristic parameters of tumors and normal tissues at different magnifications; the two classification models include a low-resolution model and a high-resolution model.

[0013] The two classification models are used successively to predict the full-field pathological sections to obtain positioning results, including:

[0014] First, use a low-resolution model to predict the full-field pathological slice to obtain a rough positioning probability map;

[0015] The high-resolution model is used to judge the abnormal pixel points in the rough positioning probability map, and the probability value of the pixel position is re-obtained to obtain the final positioning probability map.

[0016] Preferably, training the low-resolution model comprises:

[0017] Use the low-resolution dataset to train a full convolution classification network, with Resnet18 as the backbone network;

[0018] Replace the fully connected layer of the network with a fully convolutional layer, and control the number of convolution channels of the last layer to the number of categories;

[0019] By weighting the positive samples during training, we can ensure that the low-resolution model has a higher response to positive samples.

[0020] The low-resolution model adopts a multi-scale feature fusion method to fuse feature maps of different levels (stages) in different convolutional networks and obtain multi-scale features;

[0021] Out = concat(s1+s2+s3),

[0022] Among them, (s1, s2, s3) are the feature output maps of different levels in the Resnet18 network, concat is the serial fusion, and Out is the multi-scale fusion feature.

[0023] The low-resolution network uses resnet18 as the backbone network, which is improved to make the network more suitable for tumor localization tasks in low-resolution images (feature pyramid is added), so that local features and global features can be integrated to enhance the discrimination ability.

[0024] Preferably, training a high-resolution model comprises:

[0025] A convolutional classification network trained with a high-resolution dataset,

[0026] The network uses Resnet18 as the backbone network. SE modules are added to each basic module in Resnet18 to calculate the weight of each stage output channel and multiply the weight with the feature map in the feature channel to filter out features that contribute most to the category. Here, the principle of the channel attention mechanism is to strengthen the channels with larger feature weights to filter out features with greater contributions.

[0027] The obtained stage output features are subjected to multi-scale feature fusion.

[0028] The high-resolution network uses resnet18 as the backbone network, which is improved to make the network more suitable for tumor localization tasks in high-resolution images (feature pyramid and channel attention mechanism are added), so that the fused local features and global features can be expressed more comprehensively and efficiently under the action of the channel attention mechanism.

[0029] Preferably, the use of a low-resolution model to predict the full-field pathological slice to obtain a rough positioning probability map includes:

[0030] The original size full field of view pathological section is S level_0 , a preset algorithm is used to obtain a low-magnification full-field pathological slice image S with a preset magnification level_i (i=1,2,3);

[0031] Through the sliding window S of preset size W and a fixed step size S S , at low magnification full field pathology slide S level_i (i=1,2,3) sliding sampling,

[0032] The sample is input into the low-resolution model, and the downsampling factor of the low-resolution model is 2 n ;

[0033] The results calculated by the sliding window samples are spliced ​​to generate 1 / 2 of the original size n+i Rough localization probability map of size.

[0034] Preferably, the method of using a high-resolution model to judge the abnormal pixel points in the rough positioning probability map, re-obtaining the probability value of the pixel position, and obtaining the final positioning probability map includes:

[0035] The probability map contains true positive and false positive pixel points, and a threshold is used to determine whether the pixel position needs further judgment;

[0036] The relationship between the pixel coordinates on the rough probability map and the pixel position of the point in the original-size full-field pathology slice is as follows:

[0037] (X l0 ,Y l0 )=(X ln ×2 n+i ,Y ln ×2 n+i )

[0038] ((X l0 ,Y l0 ) is used as the sampling center to extract the image block of preset size and input it into the high-resolution model. The downsampling factor of the high-resolution model is 2 n+i ; In particular, n is the resolution scale factor of the coarse network output graph, and i is the resolution scale factor of the coarse network input WSI. In the present invention, high resolution wsi = original size = x40 magnification; low resolution wsi = x20 magnification.

[0039] Update to get ((X ln ,Y ln ) pixel category.

[0040] According to a second aspect of the present invention, there is provided a terminal comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, can be used to execute any one of the methods for rapid tumor localization based on multi-resolution full-field pathology slice images.

[0041] According to a third aspect of the present invention, a computer-readable storage medium is provided, wherein the storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded by a processor and executes any one of the methods for rapid tumor localization in multi-resolution full-field pathology slice images.

[0042] Compared with the prior art, the present invention has the following beneficial effects:

[0043] The present invention utilizes the feature differences at different resolutions to greatly shorten the prediction time of full-field pathological sections.

[0044] While ensuring that image details are not lost, two models with preset resolutions are trained separately. The low-resolution model uses a faster full convolutional network, and the high-resolution model adds an efficient feature extraction module. Compared with the traditional framework that uses a single resolution for positioning, it can quickly locate the tumor area on the full-field pathology slice while ensuring positioning accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0046] Figure 1 This is a flow chart of a method for rapid tumor localization based on multi-resolution full-field pathology slice images according to an embodiment;

[0047] Figure 2 The specific structure of the Nc model and the Nf model in the training phase of a preferred embodiment;

[0048] Figure 3 A framework flow chart of a multi-resolution model in the prediction stage of a preferred embodiment;

[0049] Figure 4 The visualization results of two WSIs (whole-field pathology slides) in a test set in a preferred embodiment. DETAILED DESCRIPTION

[0050] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several variations and improvements may be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0051] The present invention provides an embodiment, a method for rapid tumor location based on multi-resolution full-field pathology slice images, comprising:

[0052] S10, constructing image patch training set;

[0053] S20, using the image block training set to train and obtain two classification models;

[0054] S30, using two classification models to predict the full-field pathological sections in turn to obtain positioning results.

[0055] Based on the above embodiment, an optimization is performed to provide a preferred embodiment, such as Figure 1FIG. 1 is a flow chart of a method for rapid tumor localization based on multi-resolution full-field pathology slice images of this embodiment, comprising:

[0056] S1, preprocessing the full-field pathology slice digital image using a preset algorithm to obtain the foreground and background.

[0057] In order to better obtain the foreground and background, a preferred embodiment is provided. In this embodiment, a full-view pathology slice digital image and its true value mask are obtained, and a large number of blank background areas in the full-view pathology slice image are filtered by the big law, wherein the big law (OTSU) uses an adaptive threshold method to filter the foreground and background, and obtain a mask of the entire tissue area. In the tumor WSI image, the foreground includes the tumor area and the normal tissue area (where the tumor area can be determined by the true value mask: GT_mask); in the normal WSI image, the foreground only contains the normal tissue area. Among them, the true value mask and the entire tissue mask are enlarged or reduced by different sampling magnifications.

[0058] S2, extracts fixed-size image blocks from the foreground and background areas of the original-size full-field pathological slices, and generates the image blocks into a dataset with two types of labels.

[0059] In general, the relationship between the magnification and the full-view slice resolution is: (X40, X20, X10, …, X0.315)(level0, level1, level2, …, level9). Image blocks of full-view slices with different resolutions are constructed in the foreground and background areas.

[0060] In this embodiment, the image block size of the low-resolution level 2 full-view slice is 163*163, and the image block size of the high-resolution level 0 full-view slice is 346*346. Image block datasets with different resolutions are constructed, and respective category labels of the two types of image block datasets are obtained.

[0061] Regarding the construction of image block datasets with different resolutions, specifically, after obtaining the tissue mask, the area where the tissue mask intersects with the true value mask is the tumor area, and the rest of the tissue mask is the normal area. Random sampling is performed in the two types of areas, and the coordinate points (X l0 ,Y l0) is used as the center of the image block for sampling, and the image block has a preset size (346*346) and a preset foreground-to-background ratio (1:1) of the image block, to obtain a high-resolution level0 image block data set. Among them, random sampling is used to ensure the randomness of the data. If the sampling is based on the mask order, there will be a large number of repeated images for training. These large amounts of repeated data will make the model not robust. The coordinate points are randomly sampled, and then the image blocks are extracted based on the coordinate points. The size of the image block is the preset value (low resolution, set to 163*163; high resolution, set to 346*346. Among them, the foreground-to-background ratio is set to ensure that there is enough tumor area in the positive image block (in a single image block: 346*346).

[0062] All image block data in the high-resolution level 0 image block dataset are reduced by 4 times through the bilinear interpolation method to obtain the low-resolution level 2 image block dataset, so that the image blocks of the same field of view have different resolutions in the two datasets.

[0063] Regarding obtaining the respective category labels of the two types of image block data sets, specifically, the foreground area includes the tumor area and the normal tissue area. The image blocks extracted from the tumor area have the category label of tumor, and the image blocks extracted from the normal tissue area have the label of normal tissue. The background area is a large number of white blank areas, and a small number of image blocks are taken from this area as normal tissue image blocks.

[0064] Execute S3 based on S2. S3: Input the level2 data set into the network Nc for training to obtain the model feature parameters at this magnification; similarly, input the level0 data set into the network Nf to obtain the model parameters at this magnification.

[0065] In order to obtain the two types of training models, a preferred embodiment is provided. In this example, the image block level 2 data set and its category annotation are input into the Nc network for model training to obtain tumor characteristic parameters at low magnification. The network model includes the following structure:

[0066] The Nc network model has a residual connection convolution structure and a multi-scale feature aggregation method, and ensures that the output result of the image block size in the preset level 2 dataset is a 1*1 vector after passing through Nc.

[0067] Similarly, in order to obtain high-magnification tumor features, the level 0 dataset is used as the input of the Nf network model, and the Nf model parameters at this resolution are obtained. The network model includes the following structure:

[0068] The Nf network has a residual-connected convolutional network with a multi-scale feature aggregation module and a channel attention mechanism. The network ensures that the output of the image block size in the preset level 2 dataset after passing through Nf is a 1*1 vector.

[0069] Furthermore, the Nc network and the Nf network use corresponding preset hyperparameters during training, and construct an optimization model through the cross entropy of the predicted value and the true value annotation of the image block, and use the stochastic gradient descent method for optimization.

[0070] S4 is executed based on S3. S4: Based on the network Nc trained under the level 2 dataset, a sliding window prediction is performed under the level 2 magnification of the full-field pathological slice image to obtain a rough probability map of the tumor area.

[0071] In order to improve the speed and accuracy of obtaining a rough probability map of the tumor area, a preferred embodiment is provided. In this embodiment, the Nc model parameters are used to predict the full-view pathology slice of the level 2 (×10 magnification) test, and the full-view pathology slice is sampled in an overlapping sliding window manner by presetting the step size and size of the sliding window, and the image block in each sliding window is used as the input of the Nc model. Due to the characteristics of the fully convolutional network, the Nc model generates the corresponding rough probability block at this time.

[0072] Furthermore, the probability tiles generated by each sliding window are spliced ​​in a certain order to obtain a rough probability heat map of the downsampled size of the full-field pathology slice image.

[0073] S5 is executed based on S4. In S5, the candidate pixel points located on the probability map are sampled by backtracking to the magnification of level 0 to obtain the accurate category judgment of the pixel point through the threshold.

[0074] In order to obtain accurate category judgment, a preferred embodiment is provided. In this embodiment, a preset threshold is used. In this embodiment, the threshold is set to 0.5. The pixel positions exceeding the threshold in the probability heat map of the above downsampled size are marked, and the pixel positions are calculated by the formula (X l0 ,Y l0 )=(X ln ×2 n+i ,Y ln ×2 n+i ) traces back to the position on the level0 (×40 magnification) slice. By cropping the position with a preset image block size (the pixel point obtained by tracing back is taken as the center, and the image block is selected on the slice with a magnification of X40 according to the preset size of level0 346*346), as the input to obtain Nf, the category label of the image block corresponding to the pixel point is output.

[0075] S6. Based on the above multi-resolution model structure, the tumor localization result is obtained.

[0076] In order to achieve accurate and rapid positioning of the tumor area, the present invention provides another preferred embodiment.

[0077] S100, extracting the foreground of the read full-view pathological slice through the big law. In particular, the foreground is the tissue area stained by H&E in the digital pathological image of the tumor, including normal tissue and tumor tissue. The binary mask of the tissue can be obtained through the big law. The following description refers to the binary mask as the mask.

[0078] In this embodiment, the truth mask is annotated by multiple open source software ASAP, and an XML file is obtained. The annotation file is provided by the official data set. A truth mask is generated according to the annotation file, and the area within the truth mask is identified as a tumor cell, and the non-truth mask area in the tissue mask is identified as a normal tissue cell. Image blocks are extracted from the above areas as tumor and normal classes of the training data set, and image block images and annotation information are further obtained.

[0079] At this time, the image block dataset Xlevel0 is obtained, which is the image block collected at the default ×40 magnification. For each sample in the Xlevel0 dataset, its length and width are simultaneously reduced to 4 times the original size through the bilinear interpolation method to obtain the Xlevel2 dataset, which is equivalent to the image collected at ×10 magnification.

[0080] x level0 =randomcrop(WSIs) x level0 ∈R h*w*c

[0081] x level2 = resize(x level0 ) x level2 ∈R h / 4*w / 4*c

[0082]

[0083]

[0084] Among them, randomcrop is random cropping, resize is bilinear interpolation, WSIs is the full-field pathology slice image at the highest magnification, x level0 It is a high-resolution image patch dataset. level2 It is a low-resolution image block dataset, where level represents the full-view slice resolution level at different magnifications, and level0 represents the full-view slice image at the maximum magnification. h, w represent the length and width of the image block, and c represents the number of channels of the image.

[0085] After downsampling the original maximum resolution WSI by 4 times, the obtained tumor cells do not suffer too much loss in semantics and details, and it can also be seen by observation whether the image contains tumor cells. Therefore, this embodiment is carried out while ensuring that the image detail information is not sufficiently lost.

[0086] S200, build Nc and Nf models at different resolutions, and its training model structure is as follows Figure 2 As shown in the figure, both the coarse network and the fine network use resnet18 as the backbone network. The difference from the general Resnet18 is that the features of different levels (L2-L4) are fused after passing through the M module (feature fusion in the form of feature pyramid). The last layer does not use a fully connected structure, but a convolutional layer plus a full-scale average layer, so that the output size of image blocks of different sizes is 1*C, where C is the number of categories (two categories in this case, positive and negative). The fine network Nf hopes to have stronger feature expression capabilities at high resolution, and a channel attention module (lower right module) is added to the above structure.

[0087] In this embodiment, the Nc model is based on the Resnet18 network, and the fully connected layer in the network is replaced by a convolutional layer. The input of the Nc model is Xlevel2. The Nc model is constructed by stacking a convolutional layer with a convolution kernel of 3 and a stride of 2 and a maximum pooling layer with a stride of 2.

[0088] Furthermore, the Nc model can output the original image as a 1*C vector by stacking convolutional layers and pooling layers, where C f represents the number of categories. In this embodiment, the target to be located is a tumor, i.e., C f =1, the above 1*1 eigenvalue is calculated by sigmoid function to obtain the predicted probability of each image block.

[0089] As a preferred embodiment, the sigmoid function is:

[0090]

[0091] Among them, z represents the output of the convolutional layer, Represents the predicted probability value of the image patch. The predicted value of the image patch is compared with the true value label to calculate the cross entropy loss.

[0092] As a preferred embodiment, the loss function is:

[0093]

[0094] in, represents the predicted value of the image block, Represents the true value of the image block.level2 The data and its true value label are input into the Nc model for training. The coarse network is iterated 50 times at the preset number of iterations, the learning rate is 0.001, and the Adam optimizer is used for optimization, and the parameters of the Nc model are updated.

[0095] Furthermore, the Nf model in the embodiment is constructed, and the input of the model is x level0 .

[0096] The model constructs the Nf classification model by stacking convolution layers with 3 convolution kernels and stride 2 and maximum pooling layers with stride 2. A channel attention module is added after each pooling layer to extract key features at different scales. The feature map after maximum pooling is then passed through the global average pooling layer to obtain 1*1*C e , where C e is the number of feature channels of this layer. Then, the weights of each feature channel are obtained for the two fully connected layers and the sigmoid function through the threshold mechanism. The larger the weight value, the greater the contribution of the channel to the classification result. The calculated weights of each channel are multiplied by the feature channel to obtain a feature map with weights. This step of calculation does not change the size and number of channels of the feature map. level0 The data and true value labels are input into the Nf model for training. The fine network is iterated 70 times under the preset number of iterations, the learning rate is 0.001, and the learning rate is exponentially decayed. The stochastic gradient descent method is used for optimization to train the Nf model.

[0097] S300, entering the model prediction phase of this embodiment, the test process is as follows Figure 3 After the coarse network Nf and the fine network Nc are trained, the WSI test set is predicted:

[0098] 1. First, scan the level 2 layer (low-resolution WSI) through the coarse network. At this time, the scanning sliding window size is 2883, and output small probability blocks. After scanning the entire low-resolution WSI image, all small probability blocks are spliced ​​to obtain a rough probability map.

[0099] 2. All points on the rough probability map with a value greater than 0.5 are further classified, and finally a fine positioning probability map is obtained.

[0100] The Nc and Nf models can process images with different magnifications. The full-field pathology slice image with a magnification of ×10 is obtained by downsampling, and the image is sampled by sliding window based on this magnification as the input of the Nc model.

[0101] Specifically, in the prediction stage, since the Nc model is a fully convolutional structure, when the sliding window size is S W =2883, the probability block after passing Nc is Inf out =171, the step size of the sliding window is S d =16(2 4 ),

[0102] The formula is:

[0103] S W =T In +(Inf out -T out )×S d

[0104] Where T In is the size of the model input during the training phase, T out The size of the model output during the training phase will be Inf out By stitching them in order, a rough probability heat map of the downsampled size of the full-view pathology slice image can be generated. Each pixel value on the heat map is traversed, and the pixel value size on the probability map is set by the preset threshold t. When t>0.5, the corresponding position coordinates of the pixel point ((X ln ,Y ln ), in this embodiment, n=4, i is the number of low magnifications, i=2. The coordinate position relationship between the rough probability map and the coordinate position of the full-view pathological slice of the original size is as follows:

[0105] (X l0 ,Y l0 )=(X ln ×2 n+i ,Y ln ×2 n+i )

[0106] The high-resolution image block corresponding to each pixel position that meets the requirements on the rough probability heat map is input into the Nf model for further judgment. The predicted category result is updated in the rough probability heat map, and the final fine probability heat map is generated.

[0107] This embodiment uses a multi-resolution network to reduce the resolution of the original size of the full-view pathology slice, and uses the advantages of the full convolutional network to extract tumor features in the low-resolution image by fusing multi-scale features, so that the Nc model has a higher recall rate for the tumor, and on this basis, the positive pixels output by the Nc model are further judged by the Nf model. This embodiment uses an improved full convolutional network to adopt a coarse-to-fine process, which can greatly improve the detection speed while ensuring positioning accuracy and meet certain pathology detection requirements.

[0108] In order to verify the fast positioning capability of the algorithm, it is necessary to compare the speed and positioning accuracy of the algorithm. A single 1080Ti card with 11GB of video memory is used as a test. The test is based on the public dataset Camelyon16, which contains 128 valid full-field pathology cuts.

[0109] The speed is measured by the average time it takes for the model to predict 128 full-field pathology slice images. The total number of false positives in the test set and the sensitivity within the threshold range are calculated using the prediction results. The calculation formula is as follows:

[0110] Total_FPs=FPs_thres / slide_num

[0111] Total_sensitivity=TPs_thres / all_tumor_num

[0112] Where slide_num is the number of full-view pathology slide images, all_tumor_num is the number of all tumor areas in the test slide, FPs_thres is the number of false positives in the full-view pathology slide, and TPs_thre is the number of true positives in the full-view pathology slide.

[0113] Furthermore, six thresholds [0.25, 0.5, 1, 2, 4, 8] are selected within the threshold interval, and the average value of the sensitivity (Total_sensitivity) under the six thresholds is calculated as the FROC value of the model. Figure 4 As shown, the visualization results of two WSIs in the test set of this embodiment are shown in

[0114] Table 1. Comparison of average positioning speed and FROC of different algorithms on the camelyon16 test set, with the best results in bold

[0115] Model speed FROC Human - 0.73 Scan-net 28.32mins 0.7200 NCRF 44.12mins 0.8018 C2F 7.567mins 0.7700

[0116] The comparison algorithms in this embodiment are all carried out in the same experimental environment, and the training and selection of the model are carried out by generating a training set (unified data) and a validation set. It can be seen from Table 1 that the fast positioning method (C2F) of the full-view pathological slice proposed by the present invention has obvious advantages in speed compared with other positioning algorithms, and the positioning accuracy FROC is 3% different from the NCRF algorithm, but it can surpass the positioning accuracy of human pathologists, and can provide faster and more robust auxiliary results for such labor-intensive human work.

[0117] Based on the same concept as the above embodiments, an embodiment of the present invention also provides a terminal, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it can be used to execute any one of the methods for rapid tumor localization based on multi-resolution full-field pathological slice images in the above embodiments.

[0118] Based on the same concept as the above embodiments, an embodiment of the present invention also provides a computer-readable storage medium, in which at least one instruction, at least one program, code set or instruction set is stored, and the at least one instruction, at least one program, code set or instruction set is loaded by a processor and executes any one of the methods for rapid tumor localization based on multi-resolution full-field pathological slice images in the above embodiments.

[0119] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various modifications or variations within the scope of the claims, which does not affect the essence of the present invention. The above preferred features can be used in any combination without conflicting with each other.

Claims

1. A method for rapid tumor localization based on multi-resolution full-field pathological slice images, characterized in that: include: Construct image patch training set; Using the image block training set, two classification models are obtained; Two classification models were used successively to predict the full-field pathological sections and obtain the positioning results; The constructed image block training set includes a set of low-resolution data sets and a set of high-resolution data sets; the two classification models include a low-resolution model and a high-resolution model, which are trained to have tumor and normal tissue characteristic parameters at different magnifications; Train a low-resolution model, including: Use the low-resolution data set to train a full convolution classification network, wherein the full convolution classification network uses Resnet18 as a backbone network; The fully connected layer of the fully convolutional classification network is replaced with a fully convolutional layer, and the number of convolution channels of the last layer is controlled to be the number of categories; Weighting the positive samples in the low-resolution data set and inputting them into the full convolution classification network training; The feature maps of different levels in different trained convolutional networks are fused to obtain multi-scale features; Out = concat(s1+s2+s3) Among them, (s1, s2, s3) are feature output graphs of different levels in the Resnet18 network, concat is concatenated fusion, and Out is the multi-scale fusion feature; The two classification models are used successively to predict the full-field pathological sections to obtain positioning results, including: Using the low-resolution model, predicting the full-field pathological slice to obtain a rough positioning probability map; Obtaining abnormal pixel points in the rough positioning probability map; The abnormal pixel point is judged by the high-resolution model to obtain the probability value of the pixel position again, and obtain the final positioning probability map; The low-resolution model is used to predict the full-field pathological slice to obtain a rough positioning probability map, including: The original size full field of view pathological section is S level_0 , a low-magnification full-field pathological slice image S with a preset magnification is obtained by bilinear interpolation. level_i , i=1,2,3; Through the sliding window S of preset size W and a fixed step size S S , slide sampling on low-magnification full-field pathology slides; The sliding sampling is input into the low-resolution model, and the downsampling factor of the low-resolution model is 2 n ; The results calculated by the sliding window samples are spliced ​​to generate 1 / 2 of the original size n+i Rough localization probability map of size; The obtaining of abnormal pixel points in the rough positioning probability map includes: if the probability value of the pixel point is greater than a threshold, determining that the pixel point is an abnormal pixel point; The high-resolution model is used to judge the abnormal pixel points, and the probability value of the pixel position is obtained again to obtain the final positioning probability map, including: The relationship between the pixel coordinates on the rough probability map and the pixel position of the point in the original-size full-field pathological slice is: (X l0 ,AND l0 )=(X ln ×2 n+i ,AND ln ×2 n+i ) Where (X ln ,Y ln ) is the abnormal coordinate point on the rough probability map, (X l0 ,Y l0 ) are the coordinate points on the full-view slice at maximum magnification; ((X l0 ,Y l0 ) as the sampling center to extract an image block of a preset size and input it into the high-resolution model; The high-resolution model is downsampled by a factor of 2 n+i , update to get ((X ln ,Y ln ) pixel category.

2. The method for rapid tumor location based on multi-resolution full-field pathological slice images according to claim 1, characterized in that: The constructing of the image block training set comprises: Read the full-field pathology slides and extract their foreground and background through the big law; In the foreground and background areas, image blocks of full-field pathological slices with different resolutions are constructed to obtain image block data sets with different resolutions, and category labels of the image block data sets are obtained.

3. The method for rapid tumor location based on multi-resolution full-field pathological slice images according to claim 1, characterized in that: Training high-resolution models, including: Using the high-resolution dataset to train a convolutional classification network, the convolutional classification network uses Resnet18 as a backbone network; Add SE module to each basic module in Resnet18; Calculate the weight of each stage output channel; The weight is multiplied by the feature map in the feature channel, and features with large contributions to the category are selected as stage output features based on category contributions; stage output features are selected based on category contributions; The stage output features are subjected to multi-scale feature fusion.

4. A terminal comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, it can be used to execute the method for rapid tumor positioning based on multi-resolution full-field pathological slice images according to any one of claims 1 to 3.

5. A computer-readable storage medium, characterized in that: The storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set are loaded by the processor and executed by the method for rapid tumor localization based on multi-resolution full-field pathological slice images described in any one of claims 1 to 3.

Images