Pathological image detection method and system based on fusion of nucleus atypia quantification and tissue texture features

Abstract

The application discloses a pathological image detection method and system based on quantitative analysis of cell nucleus heteromorphism and fusion of tissue texture features, and relates to the technical field of medical image processing. The method comprises the following steps: pre-processing an input RGB pathological image, and cutting an effective tissue region in the RGB pathological image into a plurality of image blocks with a fixed size; detecting and segmenting cell nuclei in each image block by using a pre-trained multi-task cell nucleus instance segmentation network based on an improved U-Net; extracting morphological features, staining intensity features and spatial distribution features of each cell nucleus, and combining the features into cell nucleus heteromorphism features; extracting tissue region texture features of the image block; fusing the cell nucleus heteromorphism features and the tissue region texture features to form comprehensive cancer detection features, and outputting corresponding cancer probabilities. The application simultaneously analyzes pathological features at the cell nucleus level and the tissue texture level, and can more comprehensively reflect morphological changes of a cancerous region than a single image classification method.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, specifically to a method and system for pathological image detection based on the fusion of cell nuclear atypia quantification and tissue texture features. Background Technology

[0002] Pathological image analysis is a crucial step in tumor screening, cancer diagnosis, and pathological grading. Traditional pathological diagnosis relies primarily on pathologists observing tissue structure and cell morphology changes under a microscope, including enlarged nuclei, deeply stained nuclei, irregular nuclei, increased nucleocytoplasmic ratio, disordered cell arrangement, glandular structure destruction, and abnormal tissue texture. These morphological features are important indicators of whether tissue has become cancerous. With the development of digital pathology technology, pathological slides can be scanned into high-resolution, full-view digital images, providing a data foundation for computer-aided diagnosis. Utilizing image processing and deep learning methods, the system can automatically analyze tissue regions, nuclear structures, and abnormal textures in pathological images, assisting physicians in locating suspicious areas and assessing cancer risk.

[0003] Current methods for detecting cancerous lesions in pathological images still have certain shortcomings. Some methods rely primarily on deep learning models for direct classification, outputting the probability of cancerous lesions, but lack clear quantification of pathological indicators such as nuclear atypia. Doctors find it difficult to determine which morphological features the model uses for its judgment, resulting in insufficient interpretability. Relying solely on nuclear detection also fails to fully reflect the pathological state of cancerous areas. Besides abnormalities in nuclear size, shape, and staining, cancerous areas often exhibit disordered tissue structure, dense cell arrangement, glandular destruction, enhanced stromal reaction, and increased local texture complexity. Analyzing only individual nuclei easily overlooks regional tissue texture and structural abnormalities. Differences in staining, scanning, and slide preparation also exist between different sections. Different hospitals, staining batches, and scanning equipment can lead to variations in image color, brightness, contrast, and texture clarity. When detection is based directly on pixel color or simple texture features, the model is easily affected by non-pathological factors, impacting detection stability. Existing methods typically struggle to simultaneously output cancerous lesion detection results, abnormal region locations, and interpretable quantitative indicators. In practical auxiliary diagnosis, doctors not only need to know whether the system determines the presence of cancer, but also need to understand the location of the suspicious area and what nuclear abnormalities and tissue texture abnormalities are present in that area. A single probability result is insufficient to meet the needs of pathological review and clinical auxiliary diagnosis.

[0004] Therefore, there is a need to propose a pathological image cancer detection method that combines quantitative indicators of nuclear atypia and tissue texture features. This method can not only identify suspected cancerous areas but also provide clearer pathological evidence, serving as a reference for pathologists to review and assist in diagnosis. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the prior art and provide a pathological image detection method and system based on the fusion of cell nuclear atypia quantification and tissue texture features.

[0006] The objective of this invention is achieved through the following technical solution:

[0007] In the first aspect, this application discloses a pathological image detection method based on the fusion of nuclear atypia quantification and tissue texture features, comprising the following steps:

[0008] S1. Preprocess the input RGB pathological image, identify the tissue region in the preprocessed RGB pathological image, remove the invalid tissue region, divide the valid tissue region into several fixed-size image blocks, and record the spatial coordinates of each image block in the RGB pathological image.

[0009] S2. A pre-trained multi-task cell nucleus instance segmentation network based on improved U-Net is used to detect and segment cell nuclei in each image block;

[0010] S3. After obtaining cell nucleus instances, extract the morphological features, staining intensity features, and spatial distribution features of each cell nucleus, combine them into cell nucleus atypia features, and calculate the comprehensive score of cell nucleus at the image block level.

[0011] S4. Further extract the tissue region texture features of the image patch to describe common tissue structure disorder, local density changes and abnormal cell arrangement in cancerous areas;

[0012] S5. The cell nuclear atypia features and tissue region texture features are fused to form a comprehensive cancer detection feature, and the cancer probability corresponding to each image block is output according to the comprehensive cancer detection feature.

[0013] Based on the first aspect, step S1 specifically includes:

[0014] Input RGB pathological images Transformed to optical density space, i.e. ,in, Represents an optical density image. Represents the pixel values ​​of the original RGB pathological image. Indicates the height of the RGB pathological image. Indicates the width of the RGB pathological image;

[0015] Optical density image The effective tissue region is divided into n fixed-size optical density image blocks. Obtain a set of optical density image patches , Finally, the optical density image block As input to a multi-task cell nucleus instance segmentation network.

[0016] Based on the first aspect, the multi-task cell nucleus instance segmentation network extracts multi-scale features of the cell nucleus region through an encoder-decoder structure, and simultaneously outputs the cell nucleus foreground region, cell nucleus boundary region and cell nucleus center position through a multi-task prediction method, thereby achieving instance-level segmentation of a single cell nucleus;

[0017] The cell nucleus instance segmentation network includes a feature encoding branch, a boundary enhancement branch, and a nucleus center prediction branch. The feature encoding branch uses a multi-layer convolutional residual module and a downsampling module to extract cell nucleus texture, edges, staining intensity, and local structural features. In the decoding stage, shallow edge information and deep semantic information are fused through an upsampling module and skip connections to restore the spatial contour of the cell nucleus. The boundary enhancement branch is used to predict the separating boundaries between cell nuclei to distinguish between mutually adhered or overlapping cell nuclei. The nucleus center prediction branch is used to predict the center heatmap of each cell nucleus to determine the center position of each cell nucleus and reduce the problems of cell nucleus merging, breakage, or missed detection.

[0018] Based on the first aspect, the training of the multi-task cell nucleus instance segmentation network specifically includes the following sub-steps:

[0019] S201. Construct a training sample set for cell nucleus instance segmentation. Use pathological image blocks, annotated by pathologists or manually corrected after semi-automatic annotation, as training data. Each training image block has a corresponding cell nucleus instance annotation mask. Let the first... The training image patches are The corresponding set of manually labeled cell nucleus instances is ,in, This indicates the number of cell nuclei labeled within the training image patch. Indicates the first The th training image patch Each cell nucleus instance is labeled with a mask;

[0020] S202. Generate multi-task supervision labels based on the cell nucleus instance annotation mask. Cell nucleus foreground labels, boundary labels, and center heatmap labels are generated from the manually annotated cell nucleus instance masks. The cell nucleus foreground labels indicate whether a pixel belongs to a cell nucleus region. The boundary labels are obtained by morphologically eroding each cell nucleus instance annotation mask and subtracting the original cell nucleus instance annotation mask from the eroded mask, used to strengthen the separation between adjacent cell nuclei. The center heatmap labels generate a two-dimensional Gaussian response centered on the centroid of each cell nucleus, used to guide the model in locating the cell nucleus center.

[0021] The first Pixels in a training image patch The nuclear foreground label is represented as , ;

[0022] The first Pixels in a training image patch The cell nuclear boundary label is represented as , for the The th training image patch Annotation mask for each cell nucleus instance Morphological etching was performed to obtain the morphological etching results. , ,in, Indicates radius or dimension as Structural elements; This indicates the preset boundary width parameter; This represents the morphological erosion operation; then, it is performed using the formula... Calculate the boundary region of cell nucleus instances Then the first The cell nucleus boundary labels of each training image patch are represented as follows: , ;

[0023] The first Pixels in a training image patch The nuclear center heatmap label is represented as , ,in, Indicates the first The center coordinates of each cell nucleus instance This represents the Gaussian kernel width parameter;

[0024] The Gaussian kernel width parameter The system adaptively adjusts based on the area of ​​the cell nucleus instance to accommodate scale variations in cell nuclei of different sizes. ,in, Indicates the first The area of ​​an individual cell nucleus instance. This represents the scaling factor;

[0025] S203. To ensure the integrity of cell nuclear structure information and the stability of network training, the input training image patches are... Perform uniform-size segmentation; input the segmented training image patches into the cell nucleus instance segmentation network, and the cell nucleus instance segmentation network outputs the predicted cell nucleus foreground probability map. Probability diagram of cell nuclear boundary Thermograph of the cell nucleus center ;

[0026] S204. During the training of the cell nucleus instance segmentation network, a multi-task joint loss function is used to optimize the network parameters, namely... ,in, This represents the total training loss of the cell nucleus instance segmentation network. This indicates loss of foreground segmentation of the cell nucleus. This indicates the loss in cell nuclear boundary prediction. This indicates the predicted loss at the center of the cell nucleus. The weight parameters representing the loss of cell nucleus foreground segmentation, The weight parameters represent the loss for predicting cell nuclear boundaries. The weighting parameter represents the predicted loss at the cell nucleus center;

[0027] S206. During the training of the cell nucleus instance segmentation network, data augmentation is performed on the training image patches, including random rotation, horizontal flipping, vertical flipping, scale scaling, brightness perturbation, contrast perturbation, color perturbation, and slight noise perturbation; to enhance the adaptability of the cell nucleus instance segmentation network.

[0028] Based on the first aspect, step S204 specifically includes

[0029] The pathological image detection method based on the fusion of nuclear atypia quantification and tissue texture features according to claim 4 is characterized in that step S204 specifically includes

[0030] The cell nucleus foreground segmentation loss and cell nucleus boundary prediction loss employ a combination of binary cross-entropy loss and Dice loss to simultaneously ensure pixel-level classification accuracy and cell nucleus region overlap; that is... , ;

[0031] The center prediction loss uses the mean squared error loss to constrain the difference between the predicted center heatmap and the actual center heatmap. ;in, Indicates the first The true cell nucleus foreground labels of each training image patch Indicates the first The true boundary labels of the training image patches Indicates the first The true center heatmap labels of each training image patch This represents the cross-entropy loss in binary classification. This indicates Dice's loss. This indicates the calculation of the squared error.

[0032] Based on the first aspect, step S2 specifically includes:

[0033] Each optical density image block The data is input into the trained cell nucleus instance segmentation network to obtain the corresponding cell nucleus foreground probability map, boundary probability map, and center heat map;

[0034] Candidate nucleus regions are obtained based on the nucleus foreground probability map, and the nucleus center point is extracted based on the center heat map. The adhered nuclei are then separated by combining the boundary probability map.

[0035] For adjacent and adherent nucleus regions, the nuclear center point is used as the watershed seed point, and the boundary probability map is used as the separation constraint. Labeling is used to control watershed segmentation or connected region splitting, thereby obtaining independent nucleus instance masks.

[0036] For optical density image patches The set of cell nucleus instance segmentation results is as follows ;in, Represents an image patch of optical density The set of cell nucleus instance segmentation results, Represents an image patch of optical density The number of cell nuclei detected inside. Represents an image patch of optical density Middle A mask for individual cell nuclei;

[0037] Post-processing is performed on the cell nucleus instance segmentation results to remove false positives, specifically including:

[0038] Candidate cell nuclei instances that do not conform to the preset range, have abnormal shapes, or are located at the edge of image blocks and are incomplete are filtered out; cell nuclei regions with small holes inside the mask are filled with holes; and cell nuclei regions with broken boundaries are corrected by morphological closing operations.

[0039] After post-processing, a stable mask of cell nuclei instances is obtained, and the outline, area, perimeter, center coordinates and pixel set of each cell nuclei instance are obtained.

[0040] Based on the first aspect, step S3 specifically includes the following steps:

[0041] S31. After obtaining cell nucleus instances, extract the morphological characteristics, staining intensity characteristics, and spatial distribution characteristics of each cell nucleus, and then use the formula... Calculate the roundness of cell nuclei ,in, Indicates the area of ​​the cell nucleus. Represents the perimeter of the cell nucleus; expressed by the formula Calculate the irregularity of the cell nuclear boundary Through formula Calculate optical density image patch Local cell nuclear density ,in, Represents an image patch of optical density The number of cell nuclei within the cell. Represents an image patch of optical density Effective tissue area;

[0042] S32. Calculate the combined nuclear atypia characteristics using a formula. , ,in, Indicates the eccentricity. Indicates the texture features within the kernel. Indicates cell nuclear density, This represents the statistical characteristics of the distance between adjacent cell nuclei;

[0043] S33, through formula Calculate the comprehensive score of image block-level nuclear atypia. ,in, Indicates the first One indicator of nuclear atypia, Indicates the first The corresponding weights of each nuclear atypia index Indicates the number of indicators.

[0044] Based on the first aspect, step S4 specifically includes:

[0045] Through formula Calculate tissue texture features This is used to describe common tissue structural disorders, local density changes, and abnormal cell arrangement in cancerous areas. Represents the features of the gray-level co-occurrence matrix. Indicates local binary pattern features, Representing gradient features, Indicates the characteristics of optical density distribution. Indicates the roughness of the material. Indicates consistency in direction. The entropy of organizational texture is represented by the formula: ,in, This indicates the grayscale value or optical density value. The pixel probability.

[0046] Based on the first aspect, step S5 specifically includes the following steps:

[0047] S51. Integrate nuclear atypia features and tissue texture features to form a comprehensive feature for cancer detection. , ;

[0048] S52. By introducing a feature attention mechanism, the importance of different features for cancer detection is automatically determined, i.e. ,in, Indicates the input number of the first... Class features, Indicates the first Importance weights of class features Indicates the first Learnable parameters corresponding to class features Indicates the number of feature categories. Indicates the input number of the first... Class features, Indicates the first Learnable parameters corresponding to class features;

[0049] S53, through formula Calculate the final fusion features ; Calculate the optical density image patch based on the final fusion features The corresponding probability of cancer , ,in, This represents the Sigmoid activation function. and Indicates model parameters;

[0050] S54. Cancer probability calculated based on step S53 Based on a preset judgment threshold, it is determined whether the area is a suspected cancerous area. ,in, Represents an image patch of optical density The detection label, This indicates a preset judgment threshold; then the cancer probability is... The coordinates are mapped back to the original RGB pathological image to generate a heatmap of cancer risk. For adjacent suspected cancerous image blocks, the regions are merged to form continuous cancerous candidate regions, and isolated noise points are removed.

[0051] Secondly, this application discloses a pathological image detection system based on the fusion of nuclear atypia quantification and tissue texture features. The pathological image detection method based on the fusion of nuclear atypia quantification and tissue texture features described above includes:

[0052] The image preprocessing and segmentation module is used to preprocess the input RGB pathological image, identify the tissue region of the preprocessed RGB pathological image, remove invalid tissue regions, and segment the valid tissue region into several image blocks of fixed size.

[0053] The cell nucleus instance segmentation module is used to detect and segment cell nuclei in each image block using a pre-trained multi-task cell nucleus instance segmentation network based on improved U-Net, to obtain cell nucleus instances;

[0054] The nuclear atypia feature extraction module is used to extract the morphological features, staining intensity features, and spatial distribution features of each nuclear instance after obtaining the nuclear instance, and combine them into nuclear atypia features.

[0055] The tissue texture feature extraction module is used to extract the tissue region texture features of each image patch;

[0056] The feature fusion and carcinogenesis output module is used to fuse cell nuclear atypia features and tissue region texture features to form a comprehensive carcinogenesis detection feature, and output the carcinogenesis probability corresponding to each image block based on the comprehensive carcinogenesis detection feature.

[0057] The beneficial effects of this invention are:

[0058] 1) This application simultaneously analyzes pathological features at both the nuclear level and the tissue texture level, which can more comprehensively reflect the morphological changes of cancerous areas than single image classification methods. At the same time, it transforms the criteria for judging nuclear atypia commonly used by pathologists into quantifiable indicators, thereby improving the interpretability of cancer detection results.

[0059] 2) This application supplements cell nuclear features with tissue texture features, which can identify cancer-related manifestations such as disordered cell arrangement, tissue structure destruction and local texture abnormalities; it can also output cancer risk heatmaps and abnormal indicator descriptions, which can help pathologists quickly locate suspected cancerous areas and conduct verification.

[0060] 3) The technical process of this application is not limited to a single cancer type. The feature indicators, model parameters, and judgment thresholds can be adjusted according to different tissue sources and different pathological tasks. It is applicable to scenarios such as initial screening of pathological images, localization of cancerous areas, auxiliary diagnosis, pathological quality control, teaching annotation, and medical research. Attached Figure Description

[0061] Figure 1 This is a schematic diagram illustrating the steps of a pathological image detection method based on the fusion of nuclear atypia quantification and tissue texture features according to an embodiment of the present invention.

[0062] Figure 2 This is a flowchart illustrating the pathological image detection method based on the fusion of nuclear atypia quantification and tissue texture features according to an embodiment of the present invention.

[0063] Figure 3 This is a schematic diagram of cell nucleus instance segmentation in a pathological image detection method based on the fusion of cell nuclear atypia quantification and tissue texture features according to an embodiment of the present invention.

[0064] Figure 4 This is a schematic diagram of the tissue texture features of the pathological image detection method based on the fusion of nuclear atypia quantification and tissue texture features according to an embodiment of the present invention. Detailed Implementation

[0065] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0066] See Figure 1-4 This application discloses a pathological image detection method based on the fusion of cell nuclear atypia quantification and tissue texture features. By performing instance segmentation and morphological quantification of cell nuclei in pathological images, and simultaneously extracting texture features of local tissue regions, the two methods are adaptively fused to achieve automatic detection, risk scoring, and interpretable output of cancerous areas in pathological images. Specifically, the method includes the following steps:

[0067] S1. Preprocess the input RGB pathological image and identify the tissue region in the preprocessed RGB pathological image. Remove invalid tissue regions (blank background, slide edges, scanning artifacts, folded regions, bubble regions, and severely blurred regions). Divide the valid tissue regions into several fixed-size image blocks and record the spatial coordinates of each image block in the RGB pathological image.

[0068] S2. A pre-trained multi-task cell nucleus instance segmentation network based on improved U-Net is used to detect and segment cell nuclei in each image block;

[0069] S3. After obtaining cell nucleus instances, extract the morphological features, staining intensity features, and spatial distribution features of each cell nucleus, combine them into cell nucleus atypia features, and calculate the comprehensive score of cell nucleus at the image block level.

[0070] S4. Further extract the tissue region texture features of each image block to describe common tissue structure disorder, local density changes and abnormal cell arrangement in cancerous areas;

[0071] S5. The cell nuclear atypia features and tissue region texture features are fused to form a comprehensive cancer detection feature, and the cancer probability corresponding to each image block is output according to the comprehensive cancer detection feature.

[0072] For example, step S1 specifically includes:

[0073] Input RGB pathological images Transformed to optical density space, i.e. ,in, Represents an optical density image. Represents the pixel values ​​of the original RGB pathological image. Indicates the height of the RGB pathological image. This indicates the width of the RGB pathological image, and 3 represents the three color channels of the RGB pathological image; by converting the optical density, the distinction between the stained area of ​​the cell nucleus and the surrounding tissue can be enhanced.

[0074] Optical density image The effective tissue region is divided into n fixed-size optical density image blocks. Obtain a set of optical density image patches , Finally, the optical density image block As input to a multi-task cell nucleus instance segmentation network.

[0075] For example, the multi-task cell nucleus instance segmentation network extracts multi-scale features of the cell nucleus region through an encoder-decoder structure, and simultaneously outputs the cell nucleus foreground region, cell nucleus boundary region and cell nucleus center position through a multi-task prediction method, thereby achieving instance-level segmentation of a single cell nucleus;

[0076] The cell nucleus instance segmentation network includes a feature encoding branch, a boundary enhancement branch, and a nucleus center prediction branch. The feature encoding branch uses a multi-layer convolutional residual module and a downsampling module to extract cell nucleus texture, edges, staining intensity, and local structural features. In the decoding stage, shallow edge information and deep semantic information are fused through an upsampling module and skip connections to restore the spatial contour of the cell nucleus. The boundary enhancement branch is used to predict the separating boundaries between cell nuclei to distinguish between mutually adhered or overlapping cell nuclei. The nucleus center prediction branch is used to predict the center heatmap of each cell nucleus to determine the center position of each cell nucleus and reduce the problems of cell nucleus merging, breakage, or missed detection.

[0077] For example, in order to enable a multi-task cell nucleus instance segmentation network to accurately identify optical density image patches The training of the multi-task nucleus instance segmentation network specifically includes the following sub-steps:

[0078] S201. Construct a training sample set for cell nucleus instance segmentation. Use pathological image blocks, annotated by pathologists or manually corrected after semi-automatic annotation, as training data. Each training image block has a corresponding cell nucleus instance annotation mask. Let the first... The training image patches are The corresponding set of manually labeled cell nucleus instances is ,in, This indicates the number of cell nuclei labeled within the training image patch. Indicates the first The th training image patch Each cell nucleus instance is labeled with a mask;

[0079] S202. Generate multi-task supervision labels based on the cell nucleus instance annotation mask. Cell nucleus foreground labels, boundary labels, and center heatmap labels are generated from the manually annotated cell nucleus instance masks. The cell nucleus foreground labels indicate whether a pixel belongs to a cell nucleus region. The boundary labels are obtained by morphologically eroding each cell nucleus instance annotation mask and subtracting the original cell nucleus instance annotation mask from the eroded mask. This strengthens the separation between adjacent cell nuclei, identifies the contour region of each cell nucleus instance, and enhances the separation information between adjacent or adhered cell nuclei. The center heatmap labels generate a two-dimensional Gaussian response centered on the centroid of each cell nucleus, guiding the model to locate the cell nucleus center.

[0080] The first Pixels in a training image patch The nuclear foreground label is represented as , If the cell nucleus foreground label A value of 1 indicates the first... Pixels in a training image patch It belongs to the nuclear region, if the nuclear foreground label A value of 0 indicates that the first... Pixels in a training image patch It does not belong to the cell nucleus region;

[0081] The first Pixels in a training image patch The cell nuclear boundary label is represented as , for the The th training image patch Annotation mask for each cell nucleus instance Morphological etching was performed to obtain the morphological etching results. , ,in, Indicates radius or dimension as Structural elements; This indicates the preset boundary width parameter; This represents the morphological erosion operation; then, it is performed using the formula... Calculate the boundary region of cell nucleus instances Then the first The cell nucleus boundary labels of each training image patch are represented as follows: , ;

[0082] The first Pixels in a training image patch The nuclear center heatmap label is represented as , ,in, Indicates the first The center coordinates of each cell nucleus instance This represents the Gaussian kernel width parameter;

[0083] The Gaussian kernel width parameter The system adaptively adjusts based on the area of ​​the cell nucleus instance to accommodate scale variations in cell nuclei of different sizes. ,in, Indicates the first The area of ​​an individual cell nucleus instance. This represents the scaling factor;

[0084] Through the above steps, a wider central response region is generated for larger cell nucleus instances, and a more concentrated central response region is generated for smaller cell nucleus instances, thereby improving the ability of the multi-task cell nucleus instance segmentation network to center cell nuclei of different scales and reducing the center localization error in the case of cell nucleus adhesion.

[0085] S203. To ensure the integrity of cell nuclear structure information and the stability of network training, the input training image patches are... Perform uniform-size segmentation; input the segmented training image patches into the cell nucleus instance segmentation network, and the cell nucleus instance segmentation network outputs the predicted cell nucleus foreground probability map. Probability diagram of cell nuclear boundary Thermograph of the cell nucleus center ;

[0086] S204. During the training of the cell nucleus instance segmentation network, a multi-task joint loss function is used to optimize the network parameters, namely... ,in, This represents the total training loss of the cell nucleus instance segmentation network. This indicates loss of foreground segmentation of the cell nucleus. This indicates the loss in cell nuclear boundary prediction. This indicates the predicted loss at the center of the cell nucleus. The weight parameters representing the loss of cell nucleus foreground segmentation, The weight parameters represent the loss for predicting cell nuclear boundaries. The weighting parameter represents the prediction loss for the cell nucleus center; in this embodiment... It can be adjusted according to the actual situation;

[0087] S206. During the training of the cell nucleus instance segmentation network, data augmentation is performed on the training image patches, including random rotation, horizontal flipping, vertical flipping, scaling, brightness perturbation, contrast perturbation, color perturbation, and slight noise perturbation; this enhances the adaptability of the cell nucleus instance segmentation network. Through these enhancement methods, the model can still stably identify cell nucleus regions even when there are differences in staining, brightness, and local texture.

[0088] For example, step S204 specifically includes:

[0089] Both the cell nucleus foreground segmentation loss and the cell nucleus boundary prediction loss employ a combination of binary cross-entropy loss and Dice loss to simultaneously ensure pixel-level classification accuracy and cell nucleus region overlap; that is... , ;

[0090] The center prediction loss uses the mean squared error loss to constrain the difference between the predicted center heatmap and the actual center heatmap. ;in, Indicates the first The true cell nucleus foreground labels of each training image patch Indicates the first The true boundary labels of the training image patches Indicates the first The true center heatmap labels of each training image patch This represents the cross-entropy loss in binary classification. This indicates Dice's loss. This indicates the calculation of the squared error;

[0091] Dice loss is a region overlap loss constructed based on the Dice similarity coefficient, used to measure the degree of overlap between the predicted region and the ground truth labeled region. Its calculation formula is ,in, Represents pixels The corresponding predicted probability, Represents pixels The corresponding real tags, This represents a smoothing parameter used to avoid a denominator of zero. The smaller the Dice loss, the higher the overlap between the predicted region and the actual labeled region.

[0092] For example, step S2 specifically includes:

[0093] Each optical density image block The data is input into the trained cell nucleus instance segmentation network to obtain the corresponding cell nucleus foreground probability map, boundary probability map, and center heat map;

[0094] Candidate nucleus regions are obtained based on the nucleus foreground probability map, and the nucleus center point is extracted based on the center heat map. The adhered nuclei are then separated by combining the boundary probability map.

[0095] For adjacent and adherent nucleus regions, the nuclear center point is used as the watershed seed point, and the boundary probability map is used as the separation constraint. Labeling is used to control watershed segmentation or connected region splitting, thereby obtaining independent nucleus instance masks.

[0096] For optical density image patches The set of cell nucleus instance segmentation results is as follows ;in, Represents an image patch of optical density The set of cell nucleus instance segmentation results, Represents an image patch of optical density The number of cell nuclei detected inside. Represents an image patch of optical density The Middle A mask for individual cell nuclei;

[0097] Post-processing is performed on the cell nucleus instance segmentation results to remove false positives, specifically including:

[0098] Candidate cell nuclei instances that do not conform to the preset range, have abnormal shapes, or are located at the edge of image blocks and are incomplete are filtered out; cell nuclei regions with small holes inside the mask are filled with holes; and cell nuclei regions with broken boundaries are corrected by morphological closing operations.

[0099] After post-processing, a stable mask of cell nuclei instances is obtained, yielding the outline, area, perimeter, center coordinates, and pixel set of each cell nuclei instance. This data is used for subsequent quantitative analysis of cell nucleus atypia.

[0100] For example, step S3 specifically includes the following steps:

[0101] S31. After obtaining cell nucleus instances, extract the morphological characteristics, staining intensity characteristics, and spatial distribution characteristics of each cell nucleus, and then use the formula... Calculate the roundness of cell nuclei Nuclear roundness The lower the value, the more irregular the shape of the cell nucleus. Indicates the area of ​​the cell nucleus. Represents the perimeter of the cell nucleus; expressed by the formula Calculate the irregularity of the cell nuclear boundary Irregularity of cell nuclear boundaries The larger the cell, the more complex the nuclear boundary; according to the formula Calculate optical density image patch Local cell nuclear density ,in, Represents an image patch of optical density The number of cell nuclei within the cell. Represents an image patch of optical density Effective tissue area;

[0102] S32. Nuclear atypia characteristics include nuclear area, nuclear perimeter, ratio of nuclear long to short axis, nuclear roundness, nuclear eccentricity, nuclear boundary irregularity, nuclear staining depth, nuclear texture entropy, local nuclear density, and changes in the distance between adjacent nuclei. In this embodiment, the combined nuclear atypia characteristics are calculated using a formula. , ,in, Indicates the eccentricity. Indicates the texture features within the kernel. Indicates cell nuclear density, This represents the statistical characteristics of the distance between adjacent cell nuclei;

[0103] S33, through formula Calculate the comprehensive score of image block-level nuclear atypia. ,in, Indicates the first One indicator of nuclear atypia, Indicates the first The corresponding weights of each nuclear atypia index Indicates the number of indicators.

[0104] For example, step S4 specifically includes:

[0105] Tissue texture features include gray-level co-occurrence matrix features, local binary pattern features, multi-directional gradient features, optical density distribution features, tissue texture entropy, local structural roughness, cell arrangement orientation consistency, and local tissue density variation; in this embodiment, the formula is used to... Calculate tissue texture features This is used to describe common tissue structural disorders, local density changes, and abnormal cell arrangement in cancerous areas. Represents the features of the gray-level co-occurrence matrix. Indicates local binary pattern features, Representing gradient features, Indicates the characteristics of optical density distribution. Indicates the roughness of the material. Indicates consistency in direction. The formula for representing organizational texture entropy is as follows: ,in, Indicates grayscale value or optical density value The higher the pixel probability and texture entropy, the more complex the local organizational structure.

[0106] The cell nucleus instance segmentation algorithm in step S3 accurately extracts and quantitatively calculates the morphological, staining, and spatial distribution features of the cell nucleus (including area, roundness, eccentricity, boundary irregularity, nuclear staining depth, texture entropy, and local density) at the microscopic level. Simultaneously, step S4 utilizes various spatial and gradient operators to describe tissue structural disorder and glandular destruction at the macroscopic level. This design, which integrates microscopic heterogeneity with macroscopic tissue texture, can more comprehensively reflect the complex morphological changes of cancerous regions than a single image classification method.

[0107] For example, step S5 specifically includes the following steps:

[0108] S51. Integrate nuclear atypia features and tissue texture features to form a comprehensive feature for cancer detection. , ;

[0109] S52. By introducing a feature attention mechanism, the importance of different features for cancer detection is automatically determined, i.e. ,in, Indicates the input number of the first... Class features, Indicates the first Importance weights of class features Indicates the first Learnable parameters corresponding to class features Indicates the number of feature categories. Indicates the input number of the first... Class features, Indicates the first Learnable parameters corresponding to class features;

[0110] The learnable feature attention mechanism introduced in step S52 can dynamically and adaptively calculate and adjust the importance weights of the cell nuclear atypia feature vector and the tissue texture feature vector according to the features of the input pathological image block; it effectively solves the problem of image color and contrast differences caused by different staining conditions, different scanning equipment and different pathological sources, and improves the robustness and generalization ability of the model to non-pathological factors from the bottom layer.

[0111] S53, through formula Calculate the final fusion features ; Calculate the optical density image patch based on the final fusion features The corresponding probability of cancer , ,in, This represents the Sigmoid activation function. and Indicates model parameters;

[0112] S54. Cancer probability calculated based on step S53 Based on a preset threshold, it is determined whether the area is a suspected cancerous region. ,in, Represents an image patch of optical density The detection label, This indicates a preset judgment threshold; then the cancer probability is... Mapping back to the original RGB pathological image coordinates generates a cancer risk heatmap. For adjacent suspected cancerous image patches, region merging is performed to form continuous cancer candidate regions, and isolated noise points are removed. This step not only outputs the cancer probability at the image patch level but also inversely maps the probability matrix back to the original high-resolution pathological image coordinate system, generating an intuitive cancer risk heatmap through region merging and denoising. Simultaneously, the system can output the corresponding comprehensive score for nuclear atypia and tissue texture abnormality score, transforming traditional AI black-box prediction into a visualized and quantitative evidence stream that aligns with the clinical reading logic of pathologists, significantly enhancing the clinical credibility of assisted diagnosis.

[0113] Step S5 integrates the quantitative indicators of nuclear atypia at the microscale with the tissue texture features at the macroscale in a multimodal manner. Through the feature attention mechanism, information complementarity and intelligent scheduling are achieved, providing a fully interpretable quantitative pathological diagnosis basis while ensuring high robustness across devices and staining conditions.

[0114] For example, this application discloses a pathological image detection system based on the fusion of nuclear atypia quantification and tissue texture features. The pathological image detection method based on the fusion of nuclear atypia quantification and tissue texture features described above includes:

[0115] The image preprocessing and segmentation module is used to preprocess the input RGB pathological image, identify the tissue region of the preprocessed RGB pathological image, remove invalid tissue regions, and segment the valid tissue region into several image blocks of fixed size.

[0116] The cell nucleus instance segmentation module is used to detect and segment cell nuclei in each image block using a pre-trained multi-task cell nucleus instance segmentation network based on improved U-Net, to obtain cell nucleus instances;

[0117] The nuclear atypia feature extraction module is used to extract the morphological features, staining intensity features, and spatial distribution features of each nuclear instance after obtaining the nuclear instance, and combine them into nuclear atypia features.

[0118] The tissue texture feature extraction module is used to extract the tissue region texture features of each image patch;

[0119] The feature fusion and carcinogenesis output module is used to fuse cell nuclear atypia features and tissue region texture features to form a comprehensive carcinogenesis detection feature, and output the carcinogenesis probability corresponding to each image block based on the comprehensive carcinogenesis detection feature.

[0120] The above description is merely a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the forms disclosed herein and should not be construed as excluding other embodiments. It can be used in various other combinations, modifications, and environments, and can be altered within the scope of the concept described herein through the above teachings or related technologies or knowledge. Modifications and variations made by those skilled in the art that do not depart from the spirit and scope of the present invention should be within the protection scope of the appended claims.

Claims

1. A pathological image detection method based on the fusion of nuclear atypia quantification and tissue texture features, characterized in that, Includes the following steps: S1. Preprocess the input RGB pathological image, identify the tissue region in the preprocessed RGB pathological image, remove the invalid tissue region, divide the valid tissue region into several fixed-size image blocks, and record the spatial coordinates of each image block in the RGB pathological image. S2. A pre-trained multi-task cell nucleus instance segmentation network based on improved U-Net is used to detect and segment cell nuclei in each image block; S3. After obtaining cell nucleus instances, extract the morphological features, staining intensity features, and spatial distribution features of each cell nucleus, combine them into cell nucleus atypia features, and calculate the comprehensive score of cell nucleus at the image block level. S4. Further extract the tissue region texture features of the image patch to describe common tissue structure disorder, local density changes and abnormal cell arrangement in cancerous areas; S5. The cell nuclear atypia features and tissue region texture features are fused to form a comprehensive cancer detection feature, and the cancer probability corresponding to each image block is output according to the comprehensive cancer detection feature.

2. The pathological image detection method based on the fusion of nuclear atypia quantification and tissue texture features according to claim 1, characterized in that, Step S1 specifically includes: Input RGB pathological images Transformed to optical density space, i.e. ,in, Represents an optical density image. Represents the pixel values ​​of the original RGB pathological image. Indicates the height of the RGB pathological image. Indicates the width of the RGB pathological image; Optical density image The effective tissue region is divided into n fixed-size optical density image blocks. Obtain a set of optical density image patches , Finally, the optical density image block As input to a multi-task cell nucleus instance segmentation network.

3. The pathological image detection method based on the fusion of nuclear atypia quantification and tissue texture features according to claim 2, characterized in that: The multi-task cell nucleus instance segmentation network extracts multi-scale features of the cell nucleus region through an encoder-decoder structure, and simultaneously outputs the cell nucleus foreground region, cell nucleus boundary region, and cell nucleus center position through a multi-task prediction method, thereby achieving instance-level segmentation of a single cell nucleus; The cell nucleus instance segmentation network includes a feature encoding branch, a boundary enhancement branch, and a nucleus center prediction branch. The feature encoding branch uses a multi-layer convolutional residual module and a downsampling module to extract cell nucleus texture, edges, staining intensity, and local structural features. In the decoding stage, shallow edge information and deep semantic information are fused through an upsampling module and skip connections to restore the spatial contour of the cell nucleus. The boundary enhancement branch is used to predict the separating boundaries between cell nuclei to distinguish between mutually adhered or overlapping cell nuclei. The nucleus center prediction branch is used to predict the center heatmap of each cell nucleus to determine the center position of each cell nucleus and reduce the problems of cell nucleus merging, breakage, or missed detection.

4. The pathological image detection method based on the fusion of nuclear atypia quantification and tissue texture features according to claim 3, characterized in that, The training of the multi-task cell nucleus instance segmentation network specifically includes the following sub-steps: S201. Construct a training sample set for cell nucleus instance segmentation. Use pathological image blocks, annotated by pathologists or manually corrected after semi-automatic annotation, as training data. Each training image block has a corresponding cell nucleus instance annotation mask. Let the first... The training image patches are The corresponding set of manually labeled cell nucleus instances is ,in, This indicates the number of cell nuclei labeled within the training image patch. Indicates the first The th training image patch Each cell nucleus instance is labeled with a mask; S202. Generate multi-task supervision labels based on the cell nucleus instance annotation mask. Cell nucleus foreground labels, boundary labels, and center heatmap labels are generated from the manually annotated cell nucleus instance masks. The cell nucleus foreground labels indicate whether a pixel belongs to a cell nucleus region. The boundary labels are obtained by morphologically eroding each cell nucleus instance annotation mask and subtracting the original cell nucleus instance annotation mask from the eroded mask, used to strengthen the separation between adjacent cell nuclei. The center heatmap labels generate a two-dimensional Gaussian response centered on the centroid of each cell nucleus, used to guide the model in locating the cell nucleus center. The first Pixels in a training image patch The nuclear foreground label is represented as , ; The first Pixels in a training image patch The cell nuclear boundary label is represented as , for the The th training image patch Annotation mask for each cell nucleus instance Morphological etching was performed to obtain the morphological etching results. , ,in, Indicates radius or dimension as Structural elements; This indicates the preset boundary width parameter; This represents the morphological erosion operation; then, it is performed using the formula... Calculate the boundary region of cell nucleus instances Then the first The cell nucleus boundary labels of each training image patch are represented as follows: , ; The first Pixels in a training image patch The nuclear center heatmap label is represented as , ,in, Indicates the first The center coordinates of each cell nucleus instance This represents the Gaussian kernel width parameter; The Gaussian kernel width parameter The system adaptively adjusts based on the area of ​​the cell nucleus instance to accommodate scale variations in cell nuclei of different sizes. ,in, Indicates the first The area of ​​an individual cell nucleus instance. This represents the scaling factor; S203. To ensure the integrity of cell nuclear structure information and the stability of network training, the input training image patches are... Perform uniform-size segmentation; input the segmented training image patches into the cell nucleus instance segmentation network, and the cell nucleus instance segmentation network outputs the predicted cell nucleus foreground probability map. Probability diagram of cell nuclear boundary Thermograph of the cell nucleus center ; S204. During the training of the cell nucleus instance segmentation network, a multi-task joint loss function is used to optimize the network parameters, namely... ,in, This represents the total training loss of the cell nucleus instance segmentation network. This indicates loss of foreground segmentation of the cell nucleus. This indicates the loss in cell nuclear boundary prediction. This indicates the predicted loss at the center of the cell nucleus. The weight parameters representing the loss of cell nucleus foreground segmentation, The weight parameters represent the loss for predicting cell nuclear boundaries. The weighting parameter represents the predicted loss at the cell nucleus center; S206. During the training of the cell nucleus instance segmentation network, data augmentation is performed on the training image patches, including random rotation, horizontal flipping, vertical flipping, scale scaling, brightness perturbation, contrast perturbation, color perturbation, and slight noise perturbation; to enhance the adaptability of the cell nucleus instance segmentation network.

5. The pathological image detection method based on the fusion of nuclear atypia quantification and tissue texture features according to claim 4, characterized in that, Step S204 specifically includes The cell nucleus foreground segmentation loss and cell nucleus boundary prediction loss employ a combination of binary cross-entropy loss and Dice loss to simultaneously ensure pixel-level classification accuracy and cell nucleus region overlap; that is... , ; The center prediction loss uses the mean squared error loss to constrain the difference between the predicted center heatmap and the actual center heatmap. ;in, Indicates the first The true cell nucleus foreground labels of each training image patch Indicates the first The true boundary labels of the training image patches Indicates the first The true center heatmap labels of each training image patch Represents the cross-entropy loss in binary classification. This indicates Dice's loss. This indicates the calculation of the squared error.

6. The pathological image detection method based on the fusion of nuclear atypia quantification and tissue texture features according to claim 4, characterized in that, Step S2 specifically includes: Each optical density image block The data is input into the trained cell nucleus instance segmentation network to obtain the corresponding cell nucleus foreground probability map, boundary probability map, and center heat map; Candidate nucleus regions are obtained based on the nucleus foreground probability map, and the nucleus center point is extracted based on the center heat map. The adhered nuclei are then separated by combining the boundary probability map. For adjacent and adherent nucleus regions, the nuclear center point is used as the watershed seed point, and the boundary probability map is used as the separation constraint. Labeling is used to control watershed segmentation or connected region splitting, thereby obtaining independent nucleus instance masks. For optical density image patches The set of cell nucleus instance segmentation results is as follows ;in, Represents an image patch of optical density The set of cell nucleus instance segmentation results, Represents an image patch of optical density The number of cell nuclei detected inside. Represents an image patch of optical density The Middle A mask for individual cell nuclei; Post-processing is performed on the cell nucleus instance segmentation results to remove false positives, specifically including: Candidate cell nuclei instances that do not conform to the preset range, have abnormal shapes, or are located at the edge of image blocks and are incomplete are filtered out; cell nuclei regions with small holes inside the mask are filled with holes; and cell nuclei regions with broken boundaries are corrected by morphological closing operations. After post-processing, a stable mask of cell nuclei instances is obtained, and the outline, area, perimeter, center coordinates and pixel set of each cell nuclei instance are obtained.

7. The pathological image detection method based on the fusion of nuclear atypia quantification and tissue texture features according to claim 6, characterized in that, Step S3 specifically includes the following steps: S31. After obtaining cell nucleus instances, extract the morphological characteristics, staining intensity characteristics, and spatial distribution characteristics of each cell nucleus, and then use the formula... Calculate the roundness of cell nuclei ,in, Indicates the area of ​​the cell nucleus. Represents the perimeter of the cell nucleus; expressed by the formula Calculate the irregularity of the cell nuclear boundary Through formula Calculate optical density image patch Local cell nuclear density ,in, Represents an image patch of optical density The number of cell nuclei within the cell. Represents an image patch of optical density Effective tissue area; S32. Calculate the combined nuclear atypia characteristics using a formula. , ,in, Indicates the eccentricity. Indicates the texture features within the kernel. Indicates cell nuclear density, This represents the statistical characteristics of the distance between adjacent cell nuclei; S33, through formula Calculate the comprehensive score of image block-level nuclear atypia. ,in, Indicates the first One indicator of nuclear atypia, Indicates the first The corresponding weights of each nuclear atypia index Indicates the number of indicators.

8. The pathological image detection method based on the fusion of nuclear atypia quantification and tissue texture features according to claim 7, characterized in that, Step S4 specifically includes: Through formula Calculate tissue texture features This is used to describe common tissue structural disorders, local density changes, and abnormal cell arrangement in cancerous areas. Represents the features of the gray-level co-occurrence matrix. Indicates local binary pattern features, Representing gradient features, Indicates the characteristics of optical density distribution. Indicates the roughness of the material. Indicates consistency in direction. The entropy of organizational texture is represented by the formula: ,in, This indicates the grayscale value or optical density value. The pixel probability.

9. The pathological image detection method based on the fusion of nuclear atypia quantification and tissue texture features according to claim 8, characterized in that, Step S5 specifically includes the following steps: S51. Integrate nuclear atypia features and tissue texture features to form a comprehensive feature for cancer detection. , ; S52. By introducing a feature attention mechanism, the importance of different features for cancer detection is automatically determined, i.e. ,in, Indicates the input number of the first... Class features, Indicates the first Importance weights of class features Indicates the first Learnable parameters corresponding to class features Indicates the number of feature categories. Indicates the input number of the first... Class features, Indicates the first Learnable parameters corresponding to class features; S53, through formula Calculate the final fusion features ; Calculate the optical density image patch based on the final fusion features The corresponding probability of cancer , ,in, This represents the Sigmoid activation function. and Indicates model parameters; S54. Cancer probability calculated based on step S53 Based on a preset judgment threshold, it is determined whether the area is a suspected cancerous area. ,in, Represents an image patch of optical density The detection label, This indicates a preset judgment threshold; then the cancer probability is... The coordinates are mapped back to the original RGB pathological image to generate a heatmap of cancer risk. For adjacent suspected cancerous image blocks, the regions are merged to form continuous cancerous candidate regions, and isolated noise points are removed.

10. A pathological image detection system based on the fusion of nuclear atypia quantification and tissue texture features, employing the pathological image detection method based on the fusion of nuclear atypia quantification and tissue texture features as described in any one of claims 1-9, characterized in that, include: The image preprocessing and segmentation module is used to preprocess the input RGB pathological image, identify the tissue region of the preprocessed RGB pathological image, remove invalid tissue regions, and segment the valid tissue region into several image blocks of fixed size. The cell nucleus instance segmentation module is used to detect and segment cell nuclei in each image block using a pre-trained multi-task cell nucleus instance segmentation network based on improved U-Net, to obtain cell nucleus instances; The nuclear atypia feature extraction module is used to extract the morphological features, staining intensity features, and spatial distribution features of each nuclear instance after obtaining the nuclear instance, and combine them into nuclear atypia features. The tissue texture feature extraction module is used to extract the tissue region texture features of image patches; The feature fusion and carcinogenesis output module is used to fuse cell nuclear atypia features and tissue region texture features to form a comprehensive carcinogenesis detection feature, and output the carcinogenesis probability corresponding to each image block based on the comprehensive carcinogenesis detection feature.

Images