Whole field pathological image tumor grading auxiliary method and system based on key lesion area screening

Abstract

The application discloses a whole-view pathological image tumor grading auxiliary method and system based on key lesion area screening, relates to the field of image processing technology, and comprises the following steps: inputting a whole-view pathological image, reading low, medium and high magnification level image information; identifying a low magnification image effective tissue area; extracting tissue features based on a medium magnification image, screening a lesion candidate area; extracting a three-dimensional diagnostic feature vector by using a pre-trained convolutional neural network model and performing splicing and fusion, outputting a lesion weight by a multilayer perceptron evaluation head, and screening a key lesion area; combining a high magnification image to extract fine cell features, fusing medium magnification tissue features and the lesion weight to generate comprehensive features; completing single-area tumor grading probability prediction according to the comprehensive features; normalizing each key lesion weight, fusing multi-area grading probabilities, and outputting an auxiliary grading result of the whole pathological section. The application realizes tumor grading assistance for a whole-view pathological image.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and specifically to a method and system for assisting in the grading of tumors in full-view pathological images based on the screening of key lesion regions. Background Technology

[0002] Tumor pathological grading is a crucial basis for determining tumor malignancy, formulating adjuvant therapy plans, and assessing patient prognosis. When grading tumors, pathologists typically need to comprehensively observe information such as the tumor region's tissue structure, cellular atypia, mitotic activity, extent of necrosis, infiltration boundaries, local heterogeneity, and tumor cell arrangement. Different grades of tumors often exhibit significant differences in tissue and cell morphology; therefore, accurately identifying lesion areas with grading value is of great importance for pathological diagnosis.

[0003] With the development of digital pathology technology, pathological slides can be scanned into high-resolution full-view digital images. Full-view pathological images can completely preserve large-scale structural information within tissue sections, providing a foundation for computer-aided pathological diagnosis. Using image processing and deep learning methods, the system can automatically analyze tissue regions, lesion regions, and cell morphology within the full slide to assist physicians in determining tumor grading. Full-view pathological images are typically large in size, and a single slide may contain numerous local areas. Of these, only a portion is truly relevant to tumor grading; the remaining areas may be normal tissue, background areas, inflammatory areas, necrotic margins, preparation artifacts, or tissue areas with low diagnostic value. If all image blocks are directly averaged, the information of key lesion areas is easily diluted by a large number of irrelevant areas, leading to inaccurate final grading results.

[0004] Current full-view pathological image grading methods mostly employ image patch classification, multi-instance learning, or simple feature aggregation. Some methods lack effective mechanisms for screening key lesion regions and are easily affected by normal tissue, irrelevant tissue, and artifact regions. While some methods can output slice-level grading results, they struggle to identify which regions contribute significantly to the final grading, making it difficult for physicians to verify the model's judgment process. Tumor tissue itself exhibits significant heterogeneity. Within the same pathological slice, different regions may display varying degrees of cellular atypia, tissue structure destruction, nuclear mitotic activity, and necrotic infiltration. The judgment result of a single region is insufficient to represent the true grading status of the entire slice, and simply averaging the prediction results of multiple regions may overlook the diagnostic role of high-value lesion regions. Full-view pathological images also involve substantial computational costs. Performing high-magnification detailed analysis on all regions of the entire slice would incur high computational costs, hindering practical system deployment and clinical workflow applications. A more reasonable approach is to first screen for candidate lesion regions with diagnostic value across the entire slice, and then perform detailed analysis and grading prediction on key regions. In actual pathological auxiliary diagnosis, doctors not only need to obtain the final tumor grade result, but also need to view the location of key lesion areas, the probability of regional grading, risk heatmaps, and areas requiring focused review. Only by providing clear regional evidence and interpretable results can the auxiliary diagnostic system better serve the pathologist's review and decision-making process. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the prior art and provide an auxiliary method and system for tumor grading of full-view pathological images based on the screening of key lesion areas.

[0006] The objective of this invention is achieved through the following technical solution:

[0007] In the first aspect, this application discloses an auxiliary method for tumor grading in full-view pathological images based on screening of key lesion regions, including the following steps:

[0008] S1. Input the full-field pathological image and read its multi-magnification level information, including low-magnification, medium-magnification, and high-magnification images;

[0009] S2. Identify the effective tissue regions in the low-magnification image and remove their invalid background and interference regions;

[0010] S3. Within the effective tissue area, based on the medium magnification image, extract the medium magnification tissue structure features and calculate the lesion candidate score to screen and obtain the lesion candidate area;

[0011] S4. A pre-trained convolutional neural network model is used to extract the cell morphology feature vector, tissue structure feature vector, and regional context feature vector of the lesion candidate region. The cell morphology feature vector, tissue structure feature vector, and regional context feature vector are then concatenated and fused. The key lesion weight of each lesion candidate region is calculated through a decision evaluation head composed of a multilayer perceptron, and key lesion regions with tumor grading diagnosis value are selected.

[0012] S5. Within the critical lesion area, based on the high-magnification image, extract the high-magnification cell morphology features and fuse them with the medium-magnification tissue structure features and the critical lesion weight to obtain the comprehensive features of the critical lesion area.

[0013] S6. Based on the comprehensive characteristics of key lesion areas, perform regional tumor grading prediction to obtain the probability of different tumor grades corresponding to each key lesion area.

[0014] S7. Using a multi-key region evidence fusion method, the key lesion weights of the key lesion regions are normalized, and the regional-level grading results of multiple key regions are integrated into slice-level auxiliary grading results.

[0015] Based on the first aspect, step S1 specifically includes:

[0016] Suppose the input is a full-field pathological image. Multiplier level information is ,in, Indicates a low-magnification image. Indicates a medium magnification image. This indicates a high-magnification image.

[0017] Based on the first aspect, step S2 specifically includes:

[0018] For each region in the low-magnification image, the formula is used. Calculate the probability that it belongs to the effective organization region. ,in, Indicates low-magnification tissue characteristics. and Indicates model parameters, This represents the Sigmoid activation function. If the probability of a certain region is higher than a preset threshold, then that region is retained as a valid organization region.

[0019] Based on the first aspect, step S3 specifically includes:

[0020] For the Image patches at medium magnification Extracting tissue structure features at medium magnification ,in, Indicates abnormal organizational structure characteristics. Represents the boundary complexity feature. Indicates cell aggregation characteristics, This represents the local cell density characteristics, and then, through the formula... Calculate candidate scores for lesions ,in, This represents the feature weight corresponding to the local cell density feature. This represents the feature weights corresponding to the abnormal features of the organizational structure. The feature weights represent the feature weights corresponding to the boundary complexity features. The feature weights represent the features corresponding to cell aggregation characteristics. Indicates the feature weights corresponding to texture anomaly features;

[0021] Based on the threshold for screening candidate lesion regions Filter candidate lesion regions to obtain a set of candidate lesion regions. .

[0022] Based on the first aspect, the training steps of the convolutional neural network model described in step S4 specifically include:

[0023] S401, preprocess the standard image block The input is fed into a convolutional neural network with ResNet-50 as its backbone. Through four residual feature extraction stages, continuous downsampling and channel mapping are performed, resulting in four feature maps of different resolutions. Then, the feature pyramid network performs upsampling and lateral connection fusion from top to bottom to output a fused feature map that has both high semantic information and high spatial resolution. ;

[0024] S402, Fuse feature maps Three independent convolutional decoupling branches are fed in parallel, specifically including:

[0025] The feature encoding and semantic segmentation branch includes two 3×3 convolutional layers with ReLU activation and one 1×1 classification convolutional layer. Finally, it outputs a pixel-level cell kernel foreground probability matrix through a sigmoid activation function. ;

[0026] The boundary enhancement and distance transform branch contains two 3×3 convolutional layers with ReLU activation and one 1×1 classification convolutional layer. The last layer uses linear activation and outputs a normalized distance transform map from the pixel to the nearest edge of its cell nucleus. ;

[0027] The kernel center prediction and centroid regression branches contain two layers of feature-sharing convolutions. The last layer outputs a two-channel two-dimensional vector field, which is a two-dimensional offset vector field pointing from each pixel to the geometric centroid of its respective cell nucleus. ,in, This indicates the offset distance along the horizontal direction. Indicates the offset distance along the vertical direction;

[0028] S403, Designed a multi-task joint loss function End-to-end training is performed to achieve synergistic optimization between feature encoding and semantic segmentation branches, boundary enhancement and distance transformation branches, and kernel center prediction and centroid regression branches. ,in, This represents the weighted cross-entropy loss. Indicates balance The hyperparameter of learning rate, Indicates the mean square error loss. Indicates balance The hyperparameter of learning rate, Indicates smoothed L1 loss, Indicates balance The hyperparameter of learning rate.

[0029] Based on the first aspect, step S4 specifically includes the following steps:

[0030] S411. Based on the individual segmentation results of the feature encoding and semantic segmentation branch and the nuclear center prediction and centroid regression branch, the total number of cell nuclei, average roundness, average eccentricity, and probability of occurrence of mitotic-like structures within the lesion candidate region are statistically analyzed to obtain the cell morphology feature vector. ;

[0031] S412. Based on high-level semantic feature FPN, extract the tissue structure destruction degree, gray-level co-occurrence matrix texture heterogeneity, and related representations of necrotic areas or infiltration boundaries in local lesion candidate regions to obtain tissue structure feature vectors. ;

[0032] S413. Calculate the feature spatial distance between the current image patch and its neighboring image patches to characterize the degree of heterogeneity between the current region and the surrounding tissue, and obtain the region context feature vector. ;

[0033] S414, Convert cell morphology feature vector Organizational structure feature vector and region context feature vector The data is spliced ​​and fused together, and the decision evaluation head, composed of a multi-layered sensing mechanism, calculates the first... Key lesion weights for each lesion candidate region , ,in, This represents the concatenation operation of feature vectors. This represents the weight matrix of the decision evaluation head. The bias parameter represents the decision evaluation head;

[0034] S415, if the... Key lesion weights for each lesion candidate region If the value exceeds a preset threshold, exclude high-interference non-critical areas and proceed with the next step. The candidate lesion areas were identified as key lesion areas.

[0035] Based on the first aspect, step S5 specifically includes:

[0036] For the key lesion areas identified, detailed analysis was performed on high-magnification images to extract high-magnification cell morphology features. , ,in, Indicates the morphological characteristics of the cell nucleus. Indicates the characteristics of nuclear staining intensity. Indicates nuclear fission-like structural features. Representation domain heterogeneity characteristics, Indicates regional image quality assessment features;

[0037] Then, high magnification cell morphology characteristics Medium-scale tissue structure characteristics and the weight of key lesions By fusing the data, comprehensive characteristics of key lesion areas can be obtained. , .

[0038] Based on the first aspect, step S6 specifically includes:

[0039] For the Key lesion areas, through formula Calculate its belonging to the first The probability of a tumor grade ,in, and This represents the parameters of the hierarchical model.

[0040] Based on the first aspect, step S7 specifically includes the following steps:

[0041] S71, through formula The weights of key lesions in the key lesion region are normalized using Softmax, where, Indicates the first The weights of key lesion regions after Softmax normalization Indicates the number of key lesion areas. Indicates the first Key lesion weights for each candidate lesion region;

[0042] S72, through formula Calculate the entire slice belonging to the i-th The probability of a tumor grade ;

[0043] S73. Finally, through the formula The highest probability level is output as the slice-level auxiliary classification result. At the same time, through the formula Calculate the first Uncertainty score for key lesion areas .

[0044] Secondly, this application discloses a full-view pathological image tumor grading assistance system based on key lesion region screening, which utilizes the aforementioned full-view pathological image tumor grading assistance method based on key lesion region screening, including:

[0045] The reading module is used to read the input full-field pathological image and obtain the corresponding multi-magnification level information, including low-magnification image, medium-magnification image and high-magnification image;

[0046] The recognition module is used to identify effective tissue regions in low-magnification images.

[0047] The candidate region filtering module is used to filter candidate lesion regions within the effective tissue area based on medium magnification images;

[0048] The key region screening module is used to extract the three-dimensional diagnostic feature vector of the lesion candidate region using a pre-trained convolutional neural network model, and to splice and fuse the three-dimensional diagnostic feature vectors, calculate the key lesion weight of each lesion candidate region, and screen the key lesion region.

[0049] The fine analysis module is used to extract high-magnification cell morphology features based on high-magnification images within key lesion areas, and then fuse them with medium-magnification tissue structure features and key lesion weights to obtain comprehensive features of key lesion areas.

[0050] The grading prediction module is used to predict regional tumor grading, obtaining the probability of different tumor grades corresponding to each key lesion area.

[0051] The fusion module is used to combine the regional-level classification results of multiple key regions into slice-level auxiliary classification results using a multi-key-region evidence fusion method.

[0052] The beneficial effects of this invention are:

[0053] 1) This application can automatically screen key lesion areas with tumor grading diagnostic value from full-view pathological images, reducing the interference of normal tissue, background areas, and irrelevant areas on the grading results. Employing a low-magnification, medium-magnification, and high-magnification stratified analysis method, it can capture both the overall tissue structure and analyze local cell morphological details, conforming to the pathologist's reading logic from low to high magnification.

[0054] 2) This application integrates grading evidence from multiple regions by weighting key lesion regions, which can better reflect tumor heterogeneity and improve the reliability of slice-level grading results; it can output the location of key lesion regions, regional grading probabilities, slice-level grading results, risk heatmaps, and review recommendation regions, and has strong interpretability.

[0055] 3) This application can reduce the computational load of full-view pathological image analysis and improve the operating efficiency of the auxiliary diagnostic system. It can assist pathologists in quickly locating high-risk areas, reducing slide reading time, and improving the efficiency of pathological diagnosis.

[0056] 4) This application, by setting up an uncertainty assessment mechanism, can recommend areas deemed unstable by the model to doctors for review, reducing the risk of misdiagnosis. It is applicable to various scenarios such as tumor grading, lesion screening, pathological quality control, remote pathological auxiliary diagnosis, and medical teaching.

[0057] 5) The system structure of this application has strong scalability and can adjust model parameters, screening thresholds and output categories according to different cancer types, different grading standards and different pathological image sources. Attached Figure Description

[0058] Figure 1 This is a schematic diagram illustrating the steps of the full-field pathological image tumor grading auxiliary method based on key lesion region screening according to an embodiment of the present invention.

[0059] Figure 2 This is a flowchart illustrating the auxiliary method for tumor grading in full-view pathological images based on screening of key lesion areas, according to an embodiment of the present invention.

[0060] Figure 3 This is a schematic diagram of the multi-magnification extraction of a full-field pathological image tumor grading auxiliary method based on key lesion region screening according to an embodiment of the present invention.

[0061] Figure 4 This is a flowchart of the lesion region screening process in the full-view pathological image tumor grading auxiliary method based on key lesion region screening according to an embodiment of the present invention.

[0062] Figure 5 This is a schematic diagram of multi-key chain region fusion for a full-view pathological image tumor grading auxiliary method based on key lesion region screening, according to an embodiment of the present invention. Detailed Implementation

[0063] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0064] See Figure 1-5 This application provides a method and system for assisting tumor grading in full-view pathological images based on the screening of key lesion regions. It can automatically screen key lesion regions from full-view pathological images and assist in tumor grading based on these key regions. This reduces interference from irrelevant regions, highlights high-value lesion regions, integrates grading evidence from multiple key regions, and outputs interpretable tumor grading results, providing pathologists with more reliable auxiliary references. The method first quickly eliminates invalid and irrelevant regions from the entire full-view pathological image, then screens key lesion regions with diagnostic value, subsequently performs high-precision feature extraction on the key regions, and finally integrates grading evidence from multiple key regions to output slice-level tumor grading results and interpretability explanations. Specifically, it includes the following steps:

[0065] S1. Input the full-view pathological image and read its multi-magnification level information, including low-magnification image (for observing the overall tissue distribution), medium-magnification image (for screening suspected lesion areas), and high-magnification image (for analyzing cell nucleus morphology and local details).

[0066] S2. Identify the effective tissue regions in the low-magnification image and remove invalid background (blank background) and interference regions (slide labels, scanning edges, bubbles, folds, stains and severely blurred areas).

[0067] S3. Within the effective tissue area, based on the medium magnification image, extract medium magnification tissue structure features (local cell density, tissue structure abnormality, boundary complexity, degree of cell aggregation, degree of texture abnormality, necrosis-related manifestations and changes in stroma ratio, etc.) and calculate lesion candidate scores to screen out lesion candidate areas.

[0068] S4. A pre-trained convolutional neural network model is used to extract the cell morphology feature vector, tissue structure feature vector, and regional context feature vector of the lesion candidate region. The cell morphology feature vector, tissue structure feature vector, and regional context feature vector are then concatenated and fused. The key lesion weight of each lesion candidate region is calculated through a decision evaluation head composed of a multilayer perceptron, and key lesion regions with tumor grading diagnosis value are selected.

[0069] S5. Within the critical lesion area, based on the high-magnification image, extract high-magnification cell morphology features (nuclear morphology, nuclear staining intensity, nuclear division-like structure, cell packing density, and local heterogeneity features, etc.), and fuse them with medium-magnification tissue structure features and critical lesion weights to obtain comprehensive features of the critical lesion area.

[0070] S6. Based on the comprehensive characteristics of key lesion areas, regional tumor grading prediction is performed to obtain the probability of different tumor grades corresponding to each key lesion area; the regional tumor grading prediction results are used to indicate the malignancy of the key area and serve as important evidence for the final slice-level grading.

[0071] S7. Using a multi-key region evidence fusion method, the key lesion weights of the key lesion regions are normalized, and the regional-level grading results of multiple key regions are integrated into slice-level auxiliary grading results.

[0072] For example, step S1 specifically includes:

[0073] Suppose the input is a full-field pathological image. Multiplier level information is ,in, Indicates a low-magnification image. Indicates a medium magnification image. This indicates a high-magnification image.

[0074] For example, step S2 specifically includes:

[0075] For each region in the low-magnification image, the formula is used. Calculate the probability that it belongs to the effective organization region. ,in, Indicates low-magnification tissue characteristics. and Indicates model parameters, This indicates that the Sigmoid activation function retains a region as a valid organization region if the probability of that region is higher than a preset threshold. This step reduces subsequent invalid calculations and improves operational efficiency.

[0076] For example, step S3 specifically includes:

[0077] For the Image patches at medium magnification Extracting tissue structure features at medium magnification ,in, Indicates abnormal organizational structure characteristics. Represents the boundary complexity feature. Indicates cell aggregation characteristics, This represents the local cell density characteristics, and then, through the formula... Calculate candidate scores for lesions ,in, This represents the feature weight corresponding to the local cell density feature. This represents the feature weights corresponding to the abnormal features of the organizational structure. The feature weights represent the feature weights corresponding to the boundary complexity features. The feature weights represent the features corresponding to cell aggregation characteristics. Indicates the feature weights corresponding to texture anomaly features;

[0078] Based on the threshold for screening candidate lesion regions Filter candidate lesion regions to obtain a set of candidate lesion regions. .

[0079] For example, the training step of the convolutional neural network model described in step S4 specifically includes:

[0080] S401, preprocess the standard image block The input is fed into a convolutional neural network with ResNet-50 as the backbone (used to extract multi-scale deep representations of pathological image patches). Through four residual feature extraction stages (composed of multiple residual blocks, used to extract features at different scales layer by layer), continuous downsampling and channel mapping are performed, outputting four feature maps of different resolutions. Then, the feature pyramid network performs upsampling and lateral connection fusion from top to bottom to output a fused feature map that has both high semantic information and high spatial resolution. The number of feature channels C is set to 256, utilizing... The feature resolution can maintain accurate cell boundary localization while significantly reducing the computational complexity of subsequent branches;

[0081] S402, Fuse feature maps Three independent convolutional decoupling branches are fed in parallel, specifically including:

[0082] The feature encoding and semantic segmentation branch includes two 3×3 convolutional layers with ReLU activation and one 1×1 classification convolutional layer. Finally, it outputs a pixel-level cell kernel foreground probability matrix through a sigmoid activation function. The output dimension changes to (Single-channel binary classification) or Nucleus foreground probability matrix Each element in the value range is [0,1], which is used to characterize the confidence that the pixel belongs to the internal region of the cell nucleus;

[0083] The boundary enhancement and distance transformation branch contains two 3×3 convolutional layers with ReLU activation and one 1×1 classification convolutional layer. The last layer uses linear activation (for regressing continuous physical quantities) and outputs a normalized distance transformation map from the pixel to the nearest edge of its cell nucleus. The output dimension changes to The pixel value near the center of the nucleus is close to 1, the pixel value near the edge decreases linearly to 0, and the outer pixels are 0. This branch can enhance the network's ability to truncate and distinguish overlapping and adhered cell nuclei.

[0084] The kernel center prediction and centroid regression branches contain two layers of feature-sharing convolutions. The last layer outputs a two-channel two-dimensional vector field, which is a two-dimensional offset vector field pointing from each pixel to the geometric centroid of its respective cell nucleus. ;in, This represents the offset distance along the horizontal direction (x-axis direction). This represents the offset distance along the vertical direction (y-axis); the output dimension changes as follows. Through a two-dimensional offset vector field All pixels within the adherent cells can "converge centripetally," thus enabling precise differentiation of different individual cell nuclei instances in the post-processing stage through mean shift or watershed algorithms.

[0085] S403, Designed a multi-task joint loss function End-to-end training is performed to achieve synergistic optimization between feature encoding and semantic segmentation branches, boundary enhancement and distance transformation branches, and kernel center prediction and centroid regression branches. ,in, This represents the weighted cross-entropy loss (which addresses the problem of extreme imbalance between foreground and background samples in cell nuclei). Indicates balance The hyperparameter of learning rate, This represents the mean squared error loss (used to constrain the accurate fitting of the distance field). Indicates balance The hyperparameter of learning rate, The smooth L1 loss (used to calculate the regression error between the two-dimensional offset vector output by the nuclear center prediction and centroid regression branches and the actual cell nuclear centroid offset vector; it uses the squared error form when the error is small and the absolute error form when the error is large to reduce the impact of outliers on model training) (robust regression of the center offset vector). Indicates balance The hyperparameter of the learning rate. In this embodiment... , , .

[0086] For example, the candidate lesion region may still contain inflammatory areas, necrotic areas, normal high-density cell areas, or other non-critical areas; cell morphology features, tissue structure features, and regional context features are obtained by extracting their probability maps or hidden layer feature vectors through a pre-trained convolutional neural network model; the convolutional neural network model uses the precise mask output by its three-branch network and the hidden layer feature vectors of the feature pyramid to map and construct a set of candidate lesion regions. The three-dimensional diagnostic feature vector: Step S4 specifically includes the following steps:

[0087] S411. Based on the individual segmentation results of the feature encoding and semantic segmentation branch and the nuclear center prediction and centroid regression branch, the total number (density), average roundness, average eccentricity, and probability of occurrence of mitotic-like structures of cell nuclei within the lesion candidate region are statistically analyzed to obtain the cell morphology feature vector. ;

[0088] S412. Based on high-level semantic feature network (FPN) (a deep feature representation that includes lesion tissue structure information and contextual semantic information generated by feature pyramid network fusion), extract the degree of tissue structure damage, gray-level co-occurrence matrix (GLCM) texture heterogeneity, and related representations of necrotic areas or infiltration boundaries in local lesion candidate regions to obtain tissue structure feature vectors. ;

[0089] S413. Calculate the feature spatial distance between the current image patch and its neighboring image patches to characterize the degree of heterogeneity between the current region and the surrounding tissue, and obtain the region context feature vector. ;

[0090] S414, Convert cell morphology feature vector Organizational structure feature vector and region context feature vector The data is spliced ​​and fused together, and the decision evaluation head, composed of a multi-layered sensing mechanism, calculates the first... Key lesion weights for each lesion candidate region (Used to measure its diagnostic value contribution to global tumor grading). ,in, This represents the concatenation operation of feature vectors. This represents the weight matrix of the decision evaluation head. The bias parameter represents the decision evaluation head;

[0091] S415, if the... Key lesion weights for each lesion candidate region If the value exceeds a preset threshold, exclude high-interference non-critical areas and proceed with the next step. The candidate lesion regions were identified as key lesion regions and then fed into a downstream high-magnification fine-grained grading prediction network.

[0092] For example, step S5 specifically includes:

[0093] For the key lesion areas identified, detailed analysis was performed on high-magnification images to extract high-magnification cell morphology features. , ,in, Indicates the morphological characteristics of the cell nucleus. Indicates the characteristics of nuclear staining intensity. Indicates nuclear fission-like structural features. Representation domain heterogeneity characteristics, Indicates regional image quality assessment features;

[0094] Then, high magnification cell morphology characteristics Medium-scale tissue structure characteristics and the weight of key lesions By fusing the data, comprehensive characteristics of key lesion areas can be obtained. , .

[0095] For example, step S6 specifically includes:

[0096] For the Key lesion areas, through formula Calculate its belonging to the first The probability of a tumor grade ,in, and This represents the parameters of the hierarchical model.

[0097] For example, a single full-view pathological image may contain multiple key lesion areas, and the malignancy and diagnostic value of different areas may vary. Step S7 specifically includes the following steps:

[0098] S71, through formula The weights of key lesions in the key lesion region are normalized using Softmax, where, Indicates the first The weights of key lesion regions after Softmax normalization Indicates the number of key lesion areas. Indicates the first The Softmax normalization enhances the distinguishability between high-importance regions through exponential mapping and normalizes the weights of all key lesion regions so that the sum of the weights of each region is 1.

[0099] S72, through formula Calculate the entire slice belonging to the i-th The probability of a tumor grade ;

[0100] S73. Finally, through the formula The highest probability level is output as the slice-level auxiliary classification result. At the same time, through the formula Calculate the first Uncertainty score for key lesion areas .like A high score indicates instability in the grading of this area, and the area can be recommended to a pathologist for manual review.

[0101] For example, this application discloses a tumor grading assistance system for full-view pathological images based on key lesion region screening. Utilizing the aforementioned method for tumor grading assistance for full-view pathological images based on key lesion region screening, it achieves tumor grading assistance diagnosis for full-view pathological images through low-magnification tissue identification, initial screening of candidate lesion regions at medium magnification, detailed analysis of cell morphology at high magnification, and fusion of evidence from multiple key regions; specifically including:

[0102] The reading module is used to read the input full-field pathological image and obtain the corresponding multi-magnification level information, including low-magnification image, medium-magnification image and high-magnification image;

[0103] The recognition module is used to identify effective tissue regions in low-magnification images.

[0104] The candidate region filtering module is used to filter candidate lesion regions within the effective tissue area based on medium magnification images;

[0105] The key region screening module is used to extract the three-dimensional diagnostic feature vectors of the candidate lesion regions using a pre-trained convolutional neural network model, and to concatenate and fuse the three-dimensional diagnostic feature vectors to calculate the key lesion weights of each candidate lesion region and screen key lesion regions. The module judges whether the candidate region truly has the diagnostic value for tumor grading based on the following aspects: the density of tumor cells, the degree of nuclear atypia, the degree of tissue structure destruction, the probability of the appearance of mitotic structures, the relevant features of necrotic areas or infiltration boundaries, the degree of difference between the region and the surrounding tissue, and whether the image quality of the region meets the diagnostic requirements.

[0106] The fine analysis module is used to extract high-magnification cell morphology features based on high-magnification images within key lesion areas, and then fuse them with medium-magnification tissue structure features and key lesion weights to obtain comprehensive features of key lesion areas.

[0107] The grading prediction module is used to predict regional tumor grading, obtaining the probability of different tumor grades corresponding to each key lesion area.

[0108] The fusion module is used to combine the regional-level classification results of multiple key regions into slice-level auxiliary classification results using a multi-key-region evidence fusion method.

[0109] For example, it also includes an uncertainty assessment module, which calculates the grading uncertainty score for each key lesion area. If the uncertainty score of a certain area is high, it indicates that the system's grading judgment of that area is unstable, and the area can be recommended to the pathologist for key review.

[0110] The above description is merely a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the forms disclosed herein and should not be construed as excluding other embodiments. It can be used in various other combinations, modifications, and environments, and can be altered within the scope of the concept described herein through the above teachings or related technologies or knowledge. Modifications and variations made by those skilled in the art that do not depart from the spirit and scope of the present invention should be within the protection scope of the appended claims.

Claims

1. A method for assisting in the grading of tumors in full-field pathological images based on the screening of key lesion regions, characterized in that, Includes the following steps: S1. Input the full-field pathological image and read its multi-magnification level information, including low-magnification, medium-magnification, and high-magnification images; S2. Identify the effective tissue regions in the low-magnification image and remove their invalid background and interference regions; S3. Within the effective tissue area, based on the medium magnification image, extract the medium magnification tissue structure features and calculate the lesion candidate score to screen out the lesion candidate area; S4. A pre-trained convolutional neural network model is used to extract the cell morphology feature vector, tissue structure feature vector, and regional context feature vector of the lesion candidate region. The cell morphology feature vector, tissue structure feature vector, and regional context feature vector are then concatenated and fused. The key lesion weight of each lesion candidate region is calculated through a decision evaluation head composed of a multilayer perceptron, and key lesion regions with tumor grading diagnosis value are selected. S5. Within the critical lesion area, based on the high-magnification image, extract the high-magnification cell morphology features and fuse them with the medium-magnification tissue structure features and the critical lesion weight to obtain the comprehensive features of the critical lesion area. S6. Based on the comprehensive characteristics of key lesion areas, perform regional tumor grading prediction to obtain the probability of different tumor grades corresponding to each key lesion area. S7. Using a multi-key region evidence fusion method, the key lesion weights of the key lesion regions are normalized, and the regional-level grading results of multiple key regions are integrated into slice-level auxiliary grading results.

2. The method for assisting in the grading of tumors in full-field pathological images based on screening of key lesion regions according to claim 1, characterized in that, Step S1 specifically includes: Suppose the input is a full-view pathological image. Multiplier level information is ,in, Indicates a low-magnification image. Indicates a medium magnification image. This indicates a high-magnification image.

3. The method for assisting in the grading of tumors in full-view pathological images based on screening of key lesion areas according to claim 2, characterized in that, Step S2 specifically includes: For each region in the low-magnification image, the formula is used. Calculate the probability that it belongs to the effective organization region. ,in, Indicates low-magnification tissue characteristics. and Indicates model parameters, This represents the Sigmoid activation function. If the probability of a certain region is higher than a preset threshold, then that region is retained as a valid organization region.

4. The method for assisting in the grading of tumors in full-view pathological images based on screening of key lesion areas according to claim 3, characterized in that, Step S3 specifically includes: For the Image patches at medium magnification Extracting tissue structure features at medium magnification ,in, Indicates abnormal organizational structure characteristics. Represents the boundary complexity feature. Indicates cell aggregation characteristics, This represents the local cell density characteristics, and then, through the formula... Calculate candidate scores for lesions ,in, This represents the feature weight corresponding to the local cell density feature. This represents the feature weights corresponding to the abnormal features of the organizational structure. The feature weights represent the feature weights corresponding to the boundary complexity features. The feature weights represent the features corresponding to cell aggregation characteristics. Indicates the feature weights corresponding to texture anomaly features; Based on the threshold for screening candidate lesion regions Filter candidate lesion regions to obtain a set of candidate lesion regions. .

5. The method for assisting in the grading of tumors in full-view pathological images based on screening of key lesion areas according to claim 4, characterized in that, The training steps for the convolutional neural network model described in step S4 specifically include: S401, preprocess the standard image block The input is fed into a convolutional neural network with ResNet-50 as its backbone. Through four residual feature extraction stages, continuous downsampling and channel mapping are performed, resulting in four feature maps of different resolutions. Then, the feature pyramid network performs upsampling and lateral connection fusion from top to bottom to output a fused feature map that has both high semantic information and high spatial resolution. ; S402, Fuse feature maps Three independent convolutional decoupling branches are fed in parallel, specifically including: The feature encoding and semantic segmentation branch includes two 3×3 convolutional layers with ReLU activation and one 1×1 classification convolutional layer. Finally, it outputs a pixel-level cell kernel foreground probability matrix through a sigmoid activation function. ; The boundary enhancement and distance transform branch contains two 3×3 convolutional layers with ReLU activation and one 1×1 classification convolutional layer. The last layer uses linear activation and outputs a normalized distance transform map from the pixel to the nearest edge of its cell nucleus. ; The kernel center prediction and centroid regression branches contain two layers of feature-sharing convolutions. The last layer outputs a two-channel two-dimensional vector field, which is a two-dimensional offset vector field pointing from each pixel to the geometric centroid of its respective cell nucleus. ,in, This indicates the offset distance along the horizontal direction. Indicates the offset distance along the vertical direction; S403, Designed a multi-task joint loss function End-to-end training is performed to achieve synergistic optimization between feature encoding and semantic segmentation branches, boundary enhancement and distance transformation branches, and kernel center prediction and centroid regression branches. ,in, This represents the weighted cross-entropy loss. Indicates balance The hyperparameter of learning rate, Indicates the mean square error loss. Indicates balance The hyperparameter of learning rate, Indicates smoothed L1 loss, Indicates balance The hyperparameter of learning rate.

6. The method for assisting in the grading of tumors in full-view pathological images based on screening of key lesion areas according to claim 5, characterized in that, Step S4 specifically includes the following steps: S411. Based on the individual segmentation results of the feature encoding and semantic segmentation branch and the nuclear center prediction and centroid regression branch, the total number of cell nuclei, average roundness, average eccentricity, and probability of occurrence of mitotic-like structures within the lesion candidate region are statistically analyzed to obtain the cell morphology feature vector. ; S412. Based on high-level semantic feature FPN, extract the tissue structure destruction degree, gray-level co-occurrence matrix texture heterogeneity, and related representations of necrotic areas or infiltration boundaries in local lesion candidate regions to obtain tissue structure feature vectors. ; S413. Calculate the feature spatial distance between the current image patch and its neighboring image patches to characterize the degree of heterogeneity between the current region and the surrounding tissue, and obtain the region context feature vector. ; S414, Convert cell morphology feature vector Organizational structure feature vector and region context feature vector The data is spliced ​​and fused together, and the decision evaluation head, composed of a multi-layered sensing mechanism, calculates the first... Key lesion weights for each lesion candidate region , ,in, This represents the concatenation operation of feature vectors. This represents the weight matrix of the decision evaluation head. The bias parameter represents the decision evaluation head; S415, if the... Key lesion weights for each lesion candidate region If the value exceeds a preset threshold, exclude high-interference non-critical areas and proceed with the next step. The candidate lesion areas were identified as key lesion areas.

7. The method for assisting in the grading of tumors in full-view pathological images based on screening of key lesion areas according to claim 6, characterized in that, Step S5 specifically includes: For the key lesion areas identified, detailed analysis was performed on high-magnification images to extract high-magnification cell morphology features. , ,in, Indicates the morphological characteristics of the cell nucleus. Indicates the characteristics of nuclear staining intensity. Indicates nuclear fission-like structural features. Representation domain heterogeneity characteristics, Indicates regional image quality assessment features; Then, high magnification cell morphology characteristics Medium-scale tissue structure characteristics and the weight of key lesions By fusing the data, comprehensive characteristics of key lesion areas can be obtained. , .

8. The method for assisting in the grading of tumors in full-view pathological images based on screening of key lesion areas according to claim 7, characterized in that, Step S6 specifically includes: For the Key lesion areas, through formula Calculate its belonging to the first The probability of a tumor grade ,in, and This represents the parameters of the hierarchical model.

9. The method for assisting in the grading of tumors in full-view pathological images based on screening of key lesion areas according to claim 8, characterized in that, Step S7 specifically includes the following steps: S71, through formula The weights of key lesions in the key lesion region are normalized using Softmax, where, Indicates the first The weights of key lesion regions after Softmax normalization Indicates the number of key lesion areas. Indicates the first Key lesion weights for each candidate lesion region; S72, through formula Calculate the entire slice belonging to the i-th The probability of a tumor grade ; S73. Finally, through the formula The highest probability level is output as the slice-level auxiliary classification result. At the same time, through the formula Calculate the first Uncertainty score for key lesion areas .

10. A full-view pathological image tumor grading assistance system based on key lesion region screening, employing the full-view pathological image tumor grading assistance method based on key lesion region screening as described in any one of claims 1-9, characterized in that, include: The reading module is used to read the input full-field pathological image and obtain the corresponding multi-magnification level information, including low-magnification image, medium-magnification image and high-magnification image; The recognition module is used to identify effective tissue regions in low-magnification images. The candidate region filtering module is used to filter candidate lesion regions within the effective tissue area based on medium magnification images; The key region screening module is used to extract the three-dimensional diagnostic feature vector of the lesion candidate region using a pre-trained convolutional neural network model, and to splice and fuse the three-dimensional diagnostic feature vectors, calculate the key lesion weight of each lesion candidate region, and screen the key lesion region. The fine analysis module is used to extract high-magnification cell morphology features based on high-magnification images within key lesion areas, and then fuse them with medium-magnification tissue structure features and key lesion weights to obtain comprehensive features of key lesion areas. The grading prediction module is used to predict regional tumor grading, obtaining the probability of different tumor grades corresponding to each key lesion area. The fusion module is used to combine the regional-level classification results of multiple key regions into slice-level auxiliary classification results using a multi-key-region evidence fusion method.

Images