A pathological image segmentation self-correction method based on uncertainty measurement

Abstract

This invention discloses a self-correcting method for pathological image segmentation based on uncertainty measurement. First, pathological image data, semantic description information, and spatial cue information are acquired and preprocessed. Then, semantic-spatial joint segmentation is performed based on the preprocessed multimodal data to generate an initial segmentation result. Subsequently, regions to be corrected are determined based on uncertainty measurement, and the segmentation result is corrected through human interaction. Based on this, the segmentation model is optimized and updated, and the updated results are fed back to the segmentation stage to achieve continuous optimization of subsequent pathological image segmentation. This invention, by fusing pathological image data, semantic description information, and spatial cue information, locates high-uncertainty regions in the initial segmentation result and combines human correction information and model optimization and update mechanisms to achieve continuous self-correction and optimization of pathological image segmentation results, thereby improving the accuracy, stability, and applicability of pathological image segmentation.

Description

Technical Field

[0001] This invention belongs to the field of medical image processing and digital pathology image analysis technology, specifically relating to a self-correcting method for pathology image segmentation based on uncertainty measurement. Background Technology

[0002] With the development of digital slide scanning equipment, pathology information systems, and intelligent medical image analysis technology, the traditional methods of reading and analyzing pathology slides are gradually evolving towards digitalization and intelligence. Pathology slide images contain rich information on tissue structure, lesion morphology, and cell distribution. Efficient and accurate automatic analysis of these images has become an important research direction in clinical auxiliary diagnosis, pathological scientific research analysis, and medical information processing.

[0003] Against this backdrop, pathological image segmentation, as a key foundational technology in digital pathology analysis, is primarily used to divide different tissue regions, lesion regions, or cell-related regions in pathological slide images at the pixel level. This provides fundamental support for pathological diagnosis, tissue morphology analysis, lesion extent assessment, and related analyses. In clinical auxiliary diagnosis, scientific research analysis, and pathological information processing, the results of pathological image segmentation are often an important prerequisite for subsequent tissue classification, tumor region identification, tumor microenvironment analysis, and prognostic assessment, thus possessing high application value.

[0004] In recent years, with the widespread application of deep learning technology in medical image processing, neural network-based pathological image segmentation methods have gradually become an important technical approach in this field. Existing methods typically employ segmentation models using convolutional neural networks, encoder-decoder structures, attention mechanisms, or Transformer structures to automatically identify and segment target regions in pathological images. For example, methods such as U-Net and UNet++ achieve pixel-level segmentation of pathological images through encoder-decoder structures; methods such as Hover-Net further optimize the segmentation and classification of targets such as cell nuclei in pathological images; and methods such as TransUNet enhance global context modeling capabilities by introducing a Transformer structure. In recent years, with the development of basic models and cue-based segmentation techniques, large-scale model methods such as SAM and MedSAM have also begun to be applied to medical image segmentation scenarios, achieving more flexible interactive segmentation through points, boxes, masks, or text prompts, which to some extent improves the level of segmentation automation and model generalization ability.

[0005] However, existing pathological image segmentation methods still have the following shortcomings in practical applications: First, most existing methods still rely primarily on the visual features of the pathological images themselves, making insufficient use of semantic description information and spatial cues related to the target region. In complex tissue structures, regions with blurred boundaries, weak contrast regions, and highly heterogeneous scenarios, problems such as unclear segmentation boundaries, omission of target regions, or confusion of adjacent tissues easily arise. Second, after outputting segmentation results, existing methods usually lack an effective mechanism for identifying unreliable predicted regions, making it difficult to actively locate regions to be corrected based on the model output results. The manual correction process often relies on experience for local search and adjustment, resulting in low targeting and efficiency. Third, existing manual correction results are mostly used only for optimizing the results of the current sample, making it difficult to further transform them into effective information for continuous model learning and parameter updates. Therefore, it is difficult to form a closed-loop improvement process of "automatic segmentation—region to be corrected location—manual correction—model optimization and update," which to some extent limits the model's continuous optimization capability in practical applications and is not conducive to further improving accuracy and stability in subsequent segmentation tasks.

[0006] Therefore, it is necessary to propose a self-correcting method for pathological image segmentation based on uncertainty measurement to overcome the above-mentioned shortcomings. Summary of the Invention

[0007] To address the aforementioned issues, this invention discloses a self-correcting method for pathological image segmentation based on uncertainty measurement. By fusing pathological image data, semantic description information, and spatial cue information, it locates high-uncertainty regions in the initial segmentation results and combines manual correction information and model optimization and update mechanisms to achieve continuous self-correction and optimization of pathological image segmentation results, thereby improving the accuracy, stability, and applicability of pathological image segmentation.

[0008] To achieve the above objectives, the technical solution of the present invention is as follows:

[0009] A self-correcting method for pathological image segmentation based on uncertainty measurement includes the following steps:

[0010] Step 1: Acquire pathological image data, corresponding semantic description information, and spatial cue information, and preprocess the pathological image data, semantic description information, and spatial cue information to generate multimodal feature representations.

[0011] Step 2: Perform joint fusion processing on the multimodal feature representations to generate pixel-level initial segmentation results.

[0012] Step 3: Construct uncertainty measurement results based on the initial pixel-level segmentation results, determine the regions to be corrected based on the uncertainty measurement results, receive manual correction information, and update the segmentation results based on the manual correction information.

[0013] Step four: Based on the results of manual correction, construct corrected samples, optimize and update the segmentation model, and use the optimized and updated segmentation model to segment subsequent pathological images.

[0014] Optionally, the multimodal feature representation in step two includes image features corresponding to pathological image data, semantic features corresponding to semantic description information, and spatial cue features corresponding to spatial cue information; the joint fusion processing includes performing unified feature space mapping on the image features, semantic features, and spatial cue features, and using weighted combination fusion or staged fusion to generate the pixel-level initial segmentation result.

[0015] Optionally, the uncertainty measurement result in step three is determined based on at least one of the predicted probability distribution, maximum category prediction probability, information entropy, and confidence threshold corresponding to the pixel-level initial segmentation result; the region to be corrected is determined based on the uncertainty measurement result through threshold filtering, region aggregation, connected component analysis, or region merging.

[0016] Optionally, the manual correction information in step three includes at least one of boundary adjustment, region relabeling, spatial hint supplementation, or semantic description correction; updating the segmentation result based on the manual correction information includes performing at least one of the following processing on the region to be corrected: region replacement, label update, boundary refinement, or weighted fusion.

[0017] Optionally, the optimization update in step four includes constructing a corrected sample set based on the manual correction results, updating the segmentation model parameters, and converting the manually corrected region results into spatial cue information and feeding it back to the segmentation model, so as to use the updated segmentation model and the spatial cue information to segment subsequent pathological images.

[0018] In an optional implementation, preset indicators can also be calculated based on the corrected segmentation results obtained in step three. The preset indicators include at least one indicator calculated based on the area ratio of the target segmented region, the intersection ratio of different regions, the consistency of region boundaries, or the region distribution relationship. The preset indicators are used to support the correlation analysis and application of pathological image segmentation results.

[0019] The beneficial effects of this invention are as follows:

[0020] This invention constructs a region-to-be-corrected localization mechanism based on uncertainty measurement, which can proactively identify unreliable predicted regions from the initial segmentation results, narrowing the scope of manual correction and improving the targeting and efficiency of the correction. By using the manually corrected results to update the current segmentation results and further constructing corrected samples for segmentation model optimization and updating, a self-correcting closed-loop mechanism of "initial segmentation—region-to-be-corrected localization—manual correction—model optimization and updating" can be formed, thereby improving the stability, applicability, and continuous optimization capability of subsequent pathological image segmentation. In addition, this invention integrates semantic description information and spatial cue information during the segmentation process, which helps to improve the accuracy of target region identification and boundary discrimination capability in complex tissue structure scenarios. Furthermore, it can also perform relevant analysis based on the updated segmentation results to expand the application value of the segmentation results. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of the overall process of the present invention. Detailed Implementation

[0022] The present invention will be further illustrated below with reference to the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are for illustrative purposes only and are not intended to limit the scope of the invention.

[0023] Example

[0024] like Figure 1 As shown in the illustration, this embodiment describes a self-correcting method for pathological image segmentation based on uncertainty measurement, using lung pathological slide image segmentation as an example scenario. This embodiment is also applicable to pathological image segmentation tasks for various organ types such as breast, colorectal, and ovary. In practice, the pathological image data, semantic description information, and spatial cue information are first acquired and preprocessed. Then, semantic-spatial joint segmentation is performed based on the preprocessed multimodal data to generate an initial segmentation result. Subsequently, the region to be corrected is determined based on the uncertainty measurement, and the segmentation result is corrected through human interaction. Based on this, the segmentation model is optimized and updated, and the updated result is fed back to the segmentation stage to achieve continuous optimization of subsequent pathological image segmentation. Furthermore, preset indicators can be calculated based on the corrected segmentation results to support related analysis and applications.

[0025] The specific method is as follows:

[0026] S1: Multimodal data acquisition and preprocessing

[0027] In step S1, the pathological image data to be processed, the corresponding semantic description information, and the spatial cue information are acquired. These are then preprocessed to generate a multimodal feature representation for subsequent semantic-spatial joint segmentation, determination of regions to be corrected, and model optimization and updating. The purpose of step S1 is to unify, standardize, and structure image information, semantic information, and spatial location information from different sources and with different expressions, thereby providing a consistent input basis for subsequent initial segmentation result generation and self-correcting closed-loop processing.

[0028] In this embodiment, the multimodal data acquisition and preprocessing stage includes four sub-processes: pathological image data acquisition and preprocessing, semantic description information organization and encoding, spatial cue information standardization and encoding, and multimodal feature representation generation, as detailed below.

[0029] (1) Acquisition and preprocessing of pathological image data

[0030] First, acquire the raw pathological image data. The pathological image data can be a whole-section pathological image or a local image block obtained by segmenting a whole-section image; it can come from pathological samples from different scanning devices, different staining batches, or different tissue sources.

[0031] Since pathological images from different sources may differ in brightness distribution, color style, resolution, and tissue morphology, preprocessing of the original pathological images is necessary to improve the stability of subsequent feature extraction and joint fusion. Preprocessing may include pixel normalization, color standardization, size unification, foreground region extraction, noise suppression, and necessary image block processing.

[0032] In one implementation, when the input is a high-resolution whole-section pathological image, non-tissue background areas can be removed first, and then the foreground area can be segmented to obtain a set of local image patches suitable for subsequent model processing; when the input is a local pathological image patch, size unification and standardization can be performed directly. After preprocessing, a standardized pathological image input is obtained. Based on the input, image features are extracted to characterize the texture, boundaries, local morphology, and tissue structure information in the pathological image.

[0033] (2) Semantic description information organization and encoding

[0034] While acquiring pathological image data, semantic description information related to the target to be segmented is also acquired. This semantic description information is used to characterize the category semantics, histological name, or pathological attributes of the target tissue to be segmented, such as tumor region, stroma region, inflammatory infiltration region, necrotic region, glandular structure, etc.

[0035] To enable semantic description information from different sources to be used for subsequent joint segmentation, the original semantic description information can first be structured and organized to obtain a standardized semantic sequence.

[0036] (1);

[0037] in, Represents the first in the normalized semantic sequence A semantic unit, Indicates the length of the semantic sequence.

[0038] Furthermore, the standardized semantic sequence is input into the semantic encoding module to obtain a semantic feature representation. , can be represented as:

[0039] (2);

[0040] in, This represents a semantic encoding function. This represents a sequence aggregation operation. and These represent the semantic projection matrix and the bias term, respectively. This represents semantic features.

[0041] The purpose of the above processing is to convert the semantic description information in the form of raw text into a continuous feature representation that can participate in subsequent joint modeling, so that the model can not only rely on visual appearance when processing pathological images, but also introduce category semantic constraints at the same time.

[0042] (3) Standardization and coding of spatial prompt information

[0043] Furthermore, spatial cue information related to the target region to be segmented is obtained. This spatial cue information is used to characterize the prior location information of the target region and can be point cue, bounding box cue, region mask cue, contour cue, or other cue formats that can characterize the spatial extent of the target region.

[0044] To standardize the input representation of different prompt formats, the original spatial prompt information can be standardized by transforming it to a unified coordinate space corresponding to the size of the pathological image. For any point prompt or bounding box corner coordinate... Its positional encoding can be constructed as follows:

[0045] (3);

[0046] in, Represents the position encoding mapping matrix, Representing coordinates The location encoding result.

[0047] When the spatial prompt information is in the form of a bounding box, let the coordinates of the two diagonal points of the bounding box be respectively... and Then the spatial cue feature representation It can be represented as:

[0048] (4);

[0049] in, This indicates the space hint encoding function. and These represent the tooltip embeddings corresponding to different corner point types. This represents the spatial cue feature representation.

[0050] When spatial cues are presented as point cues or region mask cues, corresponding spatial cue feature representations can be generated using similar coordinate encoding or region encoding methods. In an optional implementation, when explicit, manually provided spatial cues are lacking in the initial stage, empty cues, default global cues, or initial cues generated based on coarse-grained region detection results can be used as spatial cue inputs.

[0051] (4) Initial multimodal feature generation

[0052] After preprocessing and encoding the pathological image data, semantic description information, and spatial cue information, image features, semantic features, and spatial cue features are obtained respectively, and organized into a unified multimodal feature representation. The image features can be represented as:

[0053] (5);

[0054] in, Represents the image feature extraction function. This represents the image feature representation generated from the preprocessed pathological image; This indicates the input of pathological images.

[0055] Therefore, the set of multimodal feature representations output in step S1 can be represented as:

[0056] (6);

[0057] in, This represents the set of multimodal feature representations for further semantic-spatial joint fusion and initial segmentation processing in step S2.

[0058] Explanation of the function of this step

[0059] Through the aforementioned multimodal data acquisition and preprocessing process, this invention can uniformly organize, standardize, and structurally represent raw pathological image data, semantic description information, and spatial cue information, ensuring that input data from different sources and in different forms have a consistent input structure and feature expression before entering the subsequent joint segmentation stage. Specifically, the semantic encoding process introduces target tissue category constraints, the spatial cue encoding process introduces target region location constraints, and the multimodal feature representation generation process provides a unified data foundation for subsequent pixel-level initial segmentation results construction, determination of regions to be corrected based on uncertainty metrics, and model optimization and updates.

[0060] S2: Semantic-spatial joint segmentation

[0061] like Figure 1 As shown, in step S2, the multimodal feature representation output in step S1 is subjected to joint fusion processing to generate a pixel-level initial segmentation result. The purpose of step S2 is to combine the visual information of the pathological image with the semantic constraint information and spatial location constraint information of the target tissue to perform joint discrimination and fine segmentation of the target tissue, thereby obtaining a pixel-level initial segmentation result for subsequent uncertainty measurement, determination of regions to be corrected, and interactive correction.

[0062] In this embodiment, the semantic-spatial joint segmentation stage includes six sub-processes: image feature processing, semantic feature processing, spatial cue feature processing, unified feature space mapping, multimodal joint fusion, and initial segmentation result generation, as detailed below.

[0063] (1) Image feature extraction

[0064] In step S1, the preprocessed pathological image input has been obtained. and their corresponding image feature representation The input is the preprocessed pathological image obtained in step S1. The pathological image is input into the image feature extraction module to obtain the image feature matrix. , represented as:

[0065] (7);

[0066] in, Represents the image feature extraction function. Indicates the number of image patches or visual tokens. Represents the dimension of image features. A matrix representing the visual features of a pathological image.

[0067] In one embodiment, each row of the image feature matrix corresponds to a visual representation of a local image region, used to characterize the texture, color, boundary, and tissue structure information in the pathological image.

[0068] (2) Semantic feature generation

[0069] The standardized semantic description information obtained in step S1 Input the semantic encoding module to obtain the semantic feature matrix. , represented as:

[0070] (8);

[0071] in, This represents a semantic encoding function. Indicates the length of the semantic sequence. Represents the semantic feature dimension. This represents the semantic feature matrix.

[0072] Furthermore, a linear projection is performed on the semantic feature matrix to obtain a semantic constraint feature matrix under a unified dimension. , represented as:

[0073] (9);

[0074] in, Represents the semantic projection matrix. This represents the bias vector. Represents a column vector consisting entirely of 1s. It represents the dimension of the unified feature space.

[0075] The above processing enables the semantic description in the original text form to participate in subsequent joint modeling in matrix form.

[0076] (3) Spatial cue feature generation

[0077] The standardized spatial hint information obtained in step S1 Input spatial cue encoding module to obtain spatial cue feature matrix When spatial cues take the form of point cues or bounding boxes, their coordinates can first be normalized and then mapped to a higher-dimensional cue space using a positional encoding function. For any normalized coordinates... Its position code can be represented as:

[0078] (10);

[0079] in, Represents the position encoding mapping matrix, Representing coordinates The location encoding result.

[0080] When spatial cues are presented in the form of bounding boxes, the bounding box is defined by two diagonal points. and The corresponding spatial cue feature matrix is ​​represented as follows. It can be represented as:

[0081] (11);

[0082] in, and These represent the tooltip embeddings corresponding to different corner point types. The spatial hint projection matrix, This represents the spatial constraint feature matrix in the unified feature space.

[0083] (4) Unified feature space mapping

[0084] To facilitate joint modeling of image feature matrices, semantic feature matrices, and spatial cue feature matrices, a unified feature space mapping can be applied to these three types of features. The mapping result of the image feature matrix can be expressed as:

[0085] (12);

[0086] in, Represents the image projection matrix. This represents the bias vector. This represents the image feature matrix in a unified feature space.

[0087] Through the above processing, the image feature matrix in the unified feature space is obtained. semantic feature matrix and spatial cue feature matrix This lays the foundation for subsequent cross-modal joint integration.

[0088] (5) Multimodal joint fusion

[0089] After obtaining the three types of features in a unified feature space, a cross-modal attention mechanism can be used to achieve joint fusion. First, using the image feature matrix... For the query, using the semantic feature matrix Using the key and value, we obtain the image-semantic fusion matrix. , represented as:

[0090] (13);

[0091] in, These represent the query matrix, key matrix, and value matrix in the first stage of fusion, respectively.

[0092] Furthermore, using image-semantic fusion matrix For querying, use spatially suggested feature matrices Using the key and value, we obtain the image-semantic-spatial joint fusion matrix. , represented as:

[0093] (14);

[0094] in, These represent the query matrix, key matrix, and value matrix in the second-stage fusion process, respectively.

[0095] In a preferred embodiment, a Transformer encoder can be introduced after the cross-modal attention fusion described above for context modeling and feature refinement, as follows:

[0096] (15);

[0097] in, This represents the fused feature matrix after context refinement.

[0098] (6) Generation of segmentation results

[0099] The refined fusion feature matrix Rearranged into two-dimensional feature maps , represented as:

[0100] (16);

[0101] in, and These represent the spatial dimensions of the fused feature map.

[0102] Fuse feature maps Input the segmentation result generation module to obtain the pixel-level prediction score tensor. , represented as:

[0103] (17);

[0104] in, Indicates the segmentation and decoding module. This represents the pixel-level prediction score tensor. This indicates the total number of categories.

[0105] For any pixel position It belongs to the first The predicted probability of a class can be expressed as:

[0106] (18);

[0107] Furthermore, pixel-level initial segmentation results can be obtained based on the maximum probability principle. , represented as:

[0108] (19);

[0109] in, Indicates pixel position The initial predicted category label, This represents the pixel-level initial segmentation result output in step S2.

[0110] Explanation of the function of this step

[0111] Through the aforementioned semantic-spatial joint segmentation process, this invention can simultaneously utilize the visual features of pathological images, semantic constraints of target tissues, and spatial constraints during segmentation to generate pixel-level initial segmentation results. These pixel-level initial segmentation results not only characterize the distribution of target regions in the current pathological image but also provide fundamental input for the construction of uncertainty measurement results, the determination of regions to be corrected, and interactive correction in subsequent step S3. Furthermore, they provide initial results for subsequent model optimization and updates, thus enabling this invention to form a self-correcting closed-loop processing flow of "initial segmentation—determination of regions to be corrected—manual correction—model optimization and update."

[0112] S3: Determination and Interactive Correction of the Region to be Corrected Based on Uncertainty Measurement

[0113] In step S3, the initial pixel-level segmentation result output in step S2 is subjected to uncertainty measurement. Based on the uncertainty measurement result, the region to be corrected is determined, and manual correction information is received to update the current segmentation result. The purpose of step S3 is to proactively identify regions with low model prediction reliability based on the initial segmentation result, guide the user to prioritize targeted correction of high-risk regions, and further transform the correction results into spatial prompts and correction samples to provide input for subsequent model optimization and updates, thereby forming a self-correcting closed-loop processing flow for the segmentation result.

[0114] In this embodiment, the uncertainty-based determination and interactive correction stage includes seven sub-processes: pixel-level probability representation construction, multi-source uncertainty measurement, region-level determination of the region to be corrected, receipt of manual correction information, segmentation result update, correction result feedback, and re-segmentation, as detailed below.

[0115] (1) Construction of pixel-level probability representation

[0116] Let the pixel-level prediction score tensor output in step S2 be... ,in Indicates the total number of categories. and Let represent the image height and width, respectively. Flattening them along the spatial dimensions yields the prediction score matrix:

[0117] (20);

[0118] Let the first The maximum class probability corresponding to each pixel is:

[0119] (twenty one);

[0120] in, Indicates the first The pixel belongs to the first The predicted probability of a class.

[0121] Let the probability of the corresponding second major category be:

[0122] (twenty two);

[0123] in, and These are used to characterize the primary prediction confidence and the degree of competition between the second-best classes at the pixel location, respectively.

[0124] (2) Construction of multi-source uncertainty measure

[0125] To more fully reflect the model's prediction uncertainty for each pixel position, this implementation does not use a single index, but instead constructs a multi-source uncertainty measurement result composed of entropy uncertainty, boundary competition uncertainty, and feedback consistency uncertainty.

[0126] 21) Entropy uncertainty measurement

[0127] First, based on the pixel-level category probability matrix Constructing an entropy uncertainty vector , represented as:

[0128] (twenty three);

[0129] in, Used to reflect the The degree of dispersion of the predicted probability distribution of each category at a pixel location; when the probabilities of multiple categories are relatively close, the entropy value is large, indicating that the prediction uncertainty at that pixel location is high.

[0130] 22) Measurement of uncertainty in boundary competition

[0131] Furthermore, a boundary competition uncertainty vector is constructed based on the interval between the probability of the largest class and the probability of the second largest class. , represented as:

[0132] (twenty four);

[0133] in, Indicates the first The class discrimination interval at each pixel position; the closer the probability of the largest class is to the probability of the second largest class, the more intense the class competition is, and the higher the uncertainty of the corresponding boundary competition.

[0134] 23) Feedback Consistency Uncertainty Measurement

[0135] This implementation further introduces a re-segmentation consistency measure based on feedback prompts. Let the set of update space prompts generated based on the current round correction results in step S3 be... Under the feedback prompt, joint segmentation is performed again on the same input image to obtain the feedback segmentation probability matrix. .

[0136] Define the difference vector between the original probability distribution and the feedback probability distribution. for:

[0137] (25);

[0138] in, This represents the Euclidean norm. If the probability distribution of a pixel changes significantly after feedback is added, it indicates that the pixel is sensitive to the feedback information, the original prediction is less stable, and the uncertainty of feedback consistency is higher.

[0139] 24) Comprehensive uncertainty measurement matrix

[0140] The above three types of uncertainty measurement results are concatenated column-wise to form a pixel-level uncertainty feature matrix. , represented as:

[0141] (26);

[0142] Furthermore, the uncertainty feature matrix is ​​weighted and aggregated to obtain a comprehensive uncertainty vector. , represented as:

[0143] (27);

[0144] in, Let represent the uncertain weight vector, and satisfy:

[0145] (28);

[0146] In one implementation, the weight vector can be preset; in another implementation, it can be obtained through adaptive learning from sample statistical features or the training process.

[0147] To suppress the uncertainty interference of isolated noisy pixels, smoothing can also be performed on the combined uncertainty vector on the local neighborhood graph. Let the pixel adjacency graph be... Its adjacency matrix is The degree matrix is Then the normalized graph Laplace matrix can be expressed as:

[0148] (29);

[0149] Based on the aforementioned graph Laplacian matrix, a smooth update of the comprehensive uncertainty vector can be expressed as:

[0150] (30);

[0151] in, Represents the smoothing coefficient. This represents the comprehensive uncertainty vector after spatial consistency constraints.

[0152] (3) Determination of regional areas to be corrected

[0153] To avoid excessive interaction overhead due to pixel-by-pixel correction, a comprehensive uncertainty vector can be used. Generate a region-level area to be corrected. First, [the region is then...]. Remapping to a two-dimensional uncertainty graph Then, a candidate region set is obtained through threshold filtering, connected component analysis, and region aggregation. .

[0154] For any candidate region Its regional uncertainty score It can be represented as:

[0155] (31);

[0156] in, Indicates candidate region Number of pixels.

[0157] Furthermore, by jointly considering regional uncertainty scores and regional boundary complexity, a regional priority correction score is constructed. , represented as:

[0158] (32);

[0159] in, Indicates the length of the candidate region boundary. This represents the balance coefficient. This formula illustrates that not only areas with high average uncertainty require priority correction, but also areas with complex boundaries and irregular shapes are high-risk areas.

[0160] Therefore, the set of regions to be corrected It can be represented as:

[0161] (33);

[0162] in, This indicates the region filtering threshold.

[0163] In one alternative implementation, the area to be corrected can also be... Sort in descending order and select only the top few. The high-risk areas are designated as the target areas for the current round of interactive correction.

[0164] (4) Manual correction of information reception

[0165] After identifying the region to be corrected, the system receives manual correction information input by the user for that region. This manual correction information may include at least one of the following: boundary adjustment, region re-annotation, spatial hints supplementation, and semantic description correction. Let the set of manual correction information be denoted as:

[0166] (34);

[0167] in, Indicates boundary adjustment information. This indicates the region relabeling information. The space indicates that additional information is needed. This indicates semantic description correction information.

[0168] For any region to be corrected Its manually corrected local label matrix can be represented as:

[0169] (35);

[0170] in, This indicates a manually modified mapping function. Indicates the region The corrected local labeling results.

[0171] (5) Update segmentation results

[0172] After receiving manual correction information, the pixel-level initial segmentation result output in step S2 is processed. Perform local or global updates to obtain the corrected segmentation results. In one implementation, an update can be performed using a region replacement method, as shown below:

[0173] (36);

[0174] in, Indicates the region The binary mask matrix, This represents element-wise product.

[0175] In another implementation, a soft update approach can also be used to enhance the smoothness of boundary transitions:

[0176] (37);

[0177] in, This represents the corrected weight matrix.

[0178] (6) Feedback on correction results and re-segmentation

[0179] To ensure that manual corrections not only update the current segmentation result but also feed back into subsequent segmentation processes, the corrected local region results can be converted into spatial cue information. Let the updated cue set be... The result of further segmentation based on feedback can be expressed as:

[0180] (38);

[0181] in, This represents the joint partitioning mapping process in step S2. This represents the optimized segmentation result regenerated based on feedback prompts. If multiple rounds of interactive correction are performed, then the... The result of wheel segmentation can be represented as:

[0182] (39);

[0183] in, Indicates the first The set of update space hints generated after the round of correction.

[0184] Explanation of the function of this step

[0185] Through the aforementioned process of determining and interactively correcting regions based on uncertainty measurement, this invention can construct multi-source uncertainty measurement results from three dimensions—pixel-level probability distribution, category competition, and consistency of probabilities before and after feedback—based on the initial segmentation results. Furthermore, it generates regions to be corrected through region aggregation and graph structure smoothing. Compared to methods that directly filter high-risk pixels based solely on a single probability threshold, the uncertainty measurement mechanism constructed in this invention can more fully reflect the unreliability of predictions regarding blurred boundaries, mixed tissues, and sensitive areas in pathological images, thereby significantly improving the targeting of regions to be corrected. Simultaneously, the manual correction results are not only used to update the current segmentation results but can also be further fed back as spatial cues and participate in re-segmentation. They also serve as a high-value source of correction samples for subsequent model optimization and updates, thus forming a self-correcting closed-loop processing mechanism of "initial segmentation—region to be corrected determination—manual correction—feedback optimization."

[0186] S4: Preset Indicator Calculation

[0187] In step S4, based on the corrected segmentation results obtained in step S3, the area ratio, regional relationships, and spatial distribution characteristics of the target region are calculated to obtain preset indicators. The purpose of step S4 is to further transform the corrected pixel-level segmentation results into quantifiable analysis results, so as to provide data support for subsequent pathological image correlation analysis, tissue structure evaluation, and application output.

[0188] In this embodiment, the preset index calculation stage includes five sub-processes: target area definition, area ratio index calculation, regional relationship index calculation, regional distribution index calculation, and index output, as detailed below.

[0189] (1) Target area definition

[0190] Let the corrected segmentation result output in step S3 be: Different category labels correspond to different organizational regions. For any target organizational category... The corresponding target region set It can be represented as:

[0191] (40);

[0192] in, Indicates pixel position, Indicate category The corresponding set of pixels.

[0193] Furthermore, let the set of all pixels in the current pathological image or the current analysis region be... The total area can then be expressed as:

[0194] (41);

[0195] Correspondingly, target area The area can be expressed as:

[0196] (42);

[0197] in, Indicates the target organization category The area of ​​the corresponding region.

[0198] In one implementation, the corrected segmentation result can also be represented as a category indicator matrix. ,satisfy:

[0199] (43);

[0200] The area of ​​the target region can also be written as:

[0201] (44);

[0202] This matrix representation facilitates the unified calculation of subsequent area ratios, regional intersection relationships, and distribution characteristics.

[0203] (2) Calculation of area ratio indicators

[0204] Based on the ratio between the target area and the total area, the target area percentage index can be calculated. , represented as:

[0205] (45);

[0206] in, Used to characterize the target organization category Area percentage in the current image or analysis region.

[0207] When it is necessary to focus on multiple target organizational categories simultaneously, the relative area ratios between different category regions can also be calculated. Let the categories be... and categories Corresponding to two target areas respectively and Then its relative area ratio It can be represented as:

[0208] (46);

[0209] in, This represents a small positive number that prevents the denominator from being zero.

[0210] Furthermore, if we consider the organizational composition vector formed by multiple target categories, the area proportions of each category can be combined as follows:

[0211] (47);

[0212] in, This represents the area composition feature vector of the current pathological image at the category level.

[0213] In one implementation, tumor regions, stroma regions, inflammatory infiltration regions, necrotic regions, etc., can be regarded as different target categories, and their area proportions and pairwise area proportions can be calculated to form tissue composition proportion characteristics.

[0214] (3) Calculation of regional relationship indicators

[0215] To reflect the spatial contact, interaction, or overlap between different organizational regions, regional relationship indicators can be further calculated.

[0216] For two target areas and Its intersection ratio index It can be represented as:

[0217] (48);

[0218] in, This represents the area of ​​the intersection of two regions. This represents the area of ​​the union of two regions.

[0219] In pathological image scenarios, different tissue regions often appear to be adjacent rather than strictly overlapping; therefore, a region adjacency index can be calculated. Let... Indicates the region The set of boundary pixels, The region represents the set of pixels after performing neighborhood expansion on the region boundary. With the region Boundary contact ratio It can be represented as:

[0220] (49);

[0221] in, Used to characterize the degree of contact between two regions at their spatial boundaries.

[0222] In one implementation, if the focus is on the interaction between the target tissue region and the surrounding microenvironment region, a corresponding regional interaction index can be constructed based on the degree of adjacency between the target region and adjacent regions to characterize the spatial association characteristics between the target tissue and the surrounding tissue environment.

[0223] (4) Calculation of regional distribution indicators

[0224] In addition to area ratio and regional relationships, the spatial distribution characteristics of the target region in the image can be further calculated.

[0225] In one implementation, the current image can be divided into Local sub-regions And count the target categories The area percentage of each local sub-region. Then the target category. In the The proportion of local area in each local sub-region Cocoa is represented as:

[0226] (50);

[0227] in, Indicates the first The area of ​​each local sub-region.

[0228] Furthermore, the dispersion index of the target region can be calculated based on the fluctuation of the area proportion in each local sub-region. , represented as:

[0229] (51);

[0230] in,

[0231] (52);

[0232] Indicate target category The average area percentage of all local sub-regions Used to characterize the spatial uniformity or dispersion of a target region.

[0233] In another implementation, connectivity analysis can be performed on the target region to count the number of its connected regions. Average connected area Features such as these are used to characterize the degree of fragmentation or aggregation of the target organization. For example, the average connected region area can be expressed as:

[0234] (53);

[0235] in, Indicate target category The A connected region.

[0236] If we further consider the spatial clustering center of the target area, we can also define the centroid coordinates of this category of area as:

[0237] (54);

[0238] in, Indicates pixel position The centroid is a two-dimensional coordinate vector. It can serve as a holistic representation of the location distribution of the target region.

[0239] (5) Preset index output

[0240] After completing the above indicator calculations, one or more area ratio indicators, regional relationship indicators, and regional distribution indicators can be combined to form a preset indicator set for the current pathological image. , represented as:

[0241] (55);

[0242] in, This represents the set of preset metrics derived from the corrected segmentation results.

[0243] Furthermore, if a unified quantitative analysis representation is required, relevant indicators can be stacked to form an indicator vector:

[0244] (56);

[0245] in, This represents a quantitative feature vector for subsequent analysis tasks.

[0246] The preset set of indicators or quantitative feature vectors can be used as the analysis results of pathological images for subsequent tissue structure analysis, pathological auxiliary judgment, or related application output.

[0247] Explanation of the function of this step

[0248] Through the aforementioned preset index calculation process, this invention can further transform the corrected segmentation results obtained in step S3 into quantitative indicators with clear histological significance, enabling the segmentation results to not only remain at the region mask level but also support subsequent related analysis and application output. Since the preset indicators are based on the corrected segmentation results, compared to calculating indicators directly based on the initial segmentation results, it can improve the reliability and analytical value of the indicator results to a certain extent.

[0249] S5: Model Optimization Updates and Feedback

[0250] In step S5, based on the manual correction results in step S3 and the resulting correction samples, the segmentation model is optimized and updated, and the updated model and related feedback results are used for subsequent pathological image segmentation. The purpose of step S5 is to further transform the high-value supervision information generated during the manual correction process into training resources that the model can learn, enabling the model to continuously optimize based on the real correction results, thereby forming a self-correcting closed-loop processing mechanism of "initial segmentation - determination of the region to be corrected - manual correction - model optimization and update - re-segmentation".

[0251] In this embodiment, the model optimization update and feedback stage includes six sub-processes: constructing a modified sample set, selecting and weighting samples, constructing model update targets, updating model parameters, updating feedback, and determining stopping conditions, as detailed below.

[0252] (1) Correction of sample set construction

[0253] Let the first step in step S3 be... The corrected sample set obtained after round-robin interaction correction is The corrected sample set formed by the cumulative total of all rounds It can be represented as:

[0254] (57);

[0255] in, This indicates the total number of rounds for interactive correction.

[0256] For any corrected sample It can be represented as:

[0257] (58);

[0258] in, This represents the corresponding pathological image or image block. Represents semantic description information, This indicates a spatial prompt message. This indicates the manually corrected target segmentation annotation. This indicates the interactive correction record corresponding to this correction.

[0259] In one implementation, the correction sample can be either a local patch sample in a full-slice image or a local region sample that has been manually corrected; it can either retain the original image context or extract only the region to be corrected and its neighborhood as model update samples.

[0260] (2) Sample screening and weighting

[0261] To improve the stability of model updates, samples in the correction sample set can be screened, retaining only high-quality samples with clear correction justification, well-defined correction boundaries, and consistent semantic labels. Let the sample quality evaluation function be... Then a high-quality corrected sample subset It can be represented as:

[0262] (59);

[0263] in, This indicates the threshold for sample quality screening.

[0264] In one implementation, the sample quality evaluation function The following factors can be considered: the proportion of the area to be corrected, boundary consistency, semantic label integrity, number of corrections, and the degree of uncertainty reduction after correction.

[0265] To allow high-value corrected samples to play a greater role in model updates, different weights can be assigned to different samples. Let the samples... The training weights are Then it can be expressed as:

[0266] (60);

[0267] in, This represents the sample weight evaluation function, used to generate weights based on the importance, self-information, or correction strength of the samples.

[0268] In an optional implementation, if a sample corresponds to a region with a high uncertainty metric in step S3, and the uncertainty decreases significantly after manual correction, then the sample can be assigned a higher weight.

[0269] (3) Model update target construction

[0270] Let the parameters of the current segmentation model be... The corresponding segmentation model is denoted as For any sample The model prediction results can be expressed as:

[0271] (61);

[0272] Based on corrected sample supervision, the sample-level segmentation loss can be defined as:

[0273] (62);

[0274] in, Indicates sample The pixel regions involved in the update Indicates sample At pixel position Belongs to the Predicted probability of class This indicates an indicator function.

[0275] To enhance the model's sensitivity to the corrected region boundaries, a boundary consistency loss based on region masks can be introduced. Let the manually corrected boundary mapping be... The predicted boundary is mapped as Then the boundary loss can be expressed as:

[0276] (63);

[0277] Furthermore, considering that this invention emphasizes a self-correcting closed loop of "correction prompt feedback - re-segmentation," a feedback consistency loss can be constructed. Let the update space prompt generated from the current correction result be... The resegmentation result obtained based on the feedback prompts can be expressed as:

[0278] (64);

[0279] Accordingly, the feedback consistency loss can be expressed as:

[0280] (65);

[0281] Therefore, the overall optimization objective at the sample level can be written as:

[0282] (66);

[0283] in, Let represent the weight coefficients of the segmentation supervision term, boundary consistency term, and feedback consistency term, respectively, and satisfy the following:

[0284] (67);

[0285] Based on this, and using a high-quality corrected sample subset A global update objective function can be constructed:

[0286] (68);

[0287] To prevent the model from being overly biased towards a small number of corrected samples, the original training sample set can also be... Together with the modified sample set, we use them for optimization to construct a joint objective function:

[0288] (69);

[0289] in, This represents the basic loss on the original training samples. This represents the base loss weighting coefficient.

[0290] (4) Model parameter update

[0291] After obtaining the overall loss function Then, the model parameters are updated. Let the learning rate be... Then the model parameter update can be expressed as:

[0292] (70);

[0293] in, This represents the gradient of the overall loss function with respect to the current model parameters.

[0294] In one alternative implementation, only some parameters in the segmentation model can be fine-tuned, such as updating only the joint fusion module, the segmentation result generation module, or parameters related to spatial cues, while keeping the main parameters of the image feature extraction module and the semantic feature extraction module unchanged. In this case, let the subset of parameters to be updated be... The update process can then be represented as:

[0295] (71);

[0296] In another alternative implementation, the entire segmentation model can be retrained to further incorporate supervision information from interactive correction samples.

[0297] (5) Update feedback and re-segmentation

[0298] After the model parameters are updated, the updated segmentation model is obtained. For subsequently input pathological images Semantic description information and space prompts The segmentation result can be expressed as:

[0299] (72);

[0300] Furthermore, the updated spatial hint set generated from the manual correction results in step S3 can also be used in conjunction with the updated model in subsequent segmentation processes. Let the updated spatial hint set be... Then it can be expressed as:

[0301] (73);

[0302] Thus, the results of manual correction participate in model parameter updates in the form of corrected samples, and participate in re-segmentation in the form of spatial prompts, thereby forming a dual closed loop of "corrected samples updating the model + correction prompts providing feedback for segmentation".

[0303] In an optional implementation, when the input is a whole-slice pathological image with overlapping patches, the predicted probability maps of each patch can also be weighted and fused. Correspondingly, the first... The weight matrix of each patch is The predicted probability is The probability fusion result of the entire image can be expressed as:

[0304] (74);

[0305] This formula is used to reduce edge stitching errors and improve the smoothness and consistency of the overall image probability output when patch overlap exists.

[0306] (6) Determining the stopping condition

[0307] To avoid invalid iterations, stopping conditions for model optimization updates can be set. The current round of model optimization update process can be terminated when at least one of the following conditions is met: the number of newly added samples in the correction sample set is less than a preset threshold; the overall loss decrease after two or more consecutive rounds of updates is less than a preset threshold; the change in segmentation results after two or more consecutive rounds of updates is less than a preset threshold; the preset maximum number of update rounds is reached; or the user confirms that the current segmentation result meets the application requirements.

[0308] For example, if loss convergence is used as the stopping condition, it can be expressed as:

[0309] (75);

[0310] in, Indicates the first Round model optimization updates the corresponding overall loss value. Indicates the first Round model optimization updates the corresponding overall loss value. This indicates that the model is optimizing and updating the index in each round. This indicates the threshold at which the loss change stops.

[0311] If the stability of the segmentation result is used as the stopping criterion, it can be expressed as:

[0312] (76);

[0313] in, Indicates the first The segmentation result obtained after round model optimization and update. Indicates the first The segmentation result obtained after round model optimization and update. This function represents the difference measure between the results of two rounds of segmentation. This represents the threshold for stable segmentation results.

[0314] Explanation of the function of this step

[0315] Through the aforementioned model optimization, update, and feedback process, this invention can further transform the high-value manual correction results generated in step S3 into training resources that the model can learn, enabling the model to gradually absorb real correction feedback in subsequent segmentation tasks. Simultaneously, the correction results can re-participate in subsequent segmentation processing through spatial cues, thus forming a continuous optimization mechanism where "manual correction drives model updates, and model updates, in turn, improve subsequent segmentation." Compared to methods that only perform one-time corrections on the current sample, this invention can improve the stability, applicability, and continuous optimization capability of subsequent pathological image segmentation.

[0316] Implementation Results Description

[0317] This invention enables unified processing of pathological image data, semantic description information, and spatial cue information, and generates pixel-level initial segmentation results based on multimodal feature representation. Furthermore, this invention locates unreliable predicted regions in the initial segmentation results using uncertainty metrics, prioritizes the identification of regions requiring correction, and updates the boundaries, labels, or spatial cue information of local regions through interactive manual correction, thereby improving the accuracy and relevance of segmentation result correction.

[0318] Furthermore, this invention not only corrects the current segmentation result using manual correction, but also converts the correction result into spatial cue information and optimizes and updates the segmentation model based on the corrected samples, enabling the updated model to be used for subsequent pathological image segmentation. Thus, this invention forms a self-correcting closed-loop processing mechanism of "initial segmentation—determination of the region to be corrected—manual correction—model optimization and update—subsequent segmentation feedback".

[0319] Compared to existing methods that rely solely on a single automatic segmentation output, this invention can proactively identify unreliable regions predicted by the model, narrow the scope of manual correction, and improve correction efficiency. At the same time, the results of manual correction are no longer only used for local optimization of the current sample, but can be further transformed into effective information for continuous learning and updating of the model, thereby improving the stability, applicability, and continuous optimization capabilities in subsequent pathological image segmentation tasks.

[0320] Furthermore, this invention integrates visual information from pathological images, semantic description information of target regions, and spatial cue information during segmentation, which helps improve the ability to identify target regions and distinguish boundaries in complex tissue structures, regions with blurred boundaries, and highly heterogeneous scenarios. It can also further calculate relevant preset indicators based on the corrected segmentation results to support subsequent tissue structure analysis, pathological auxiliary judgment, and related application outputs.

[0321] In summary, this invention can achieve fine segmentation of target regions in pathological images, and continuously optimize the segmentation results through an uncertainty-driven interactive correction and model optimization update mechanism, thereby improving the accuracy, stability, and application value of pathological image segmentation methods in practical application scenarios.

[0322] It should be noted that the above content merely illustrates the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. For those skilled in the art, various improvements and modifications can be made without departing from the principle of the present invention, and all such improvements and modifications fall within the scope of protection of the claims of the present invention.

Claims

1. A self-correcting method for pathological image segmentation based on uncertainty measurement, characterized in that, Includes the following steps: S1. Acquire pathological image data, corresponding semantic description information and spatial cue information, and preprocess the pathological image data, semantic description information and spatial cue information to generate multimodal feature representation; S2. Perform joint fusion processing on the multimodal feature representation to generate pixel-level initial segmentation results; S3. Construct an uncertainty measurement result based on the pixel-level initial segmentation result, determine the region to be corrected according to the uncertainty measurement result, receive manual correction information, and update the segmentation result according to the manual correction information; S4. Based on the results of manual correction, construct corrected samples, optimize and update the segmentation model, and use the optimized and updated segmentation model to segment subsequent pathological images.

2. The pathological image segmentation self-correction method based on uncertainty metric according to claim 1, characterized in that: The multimodal feature representation in step S1 includes image features corresponding to pathological image data, semantic features corresponding to semantic description information, and spatial cue features corresponding to spatial cue information.

3. The pathological image segmentation self-correction method based on uncertainty metric according to claim 2, characterized in that: The joint fusion process in step S2 includes performing a unified feature space mapping on the image features, semantic features, and spatial cue features, and using weighted combination fusion or staged fusion to generate the pixel-level initial segmentation result.

4. The pathological image segmentation self-correction method based on uncertainty metric according to claim 1, characterized in that: The uncertainty measurement result in step S3 is determined based on at least one of the predicted probability distribution, maximum category prediction probability, information entropy, and confidence threshold corresponding to the pixel-level initial segmentation result; the region to be corrected is determined based on the uncertainty measurement result through threshold filtering, region aggregation, connected component analysis, or region merging.

5. The pathological image segmentation self-correction method based on uncertainty metric according to claim 1, characterized in that: The manual correction information in step S3 includes at least one of boundary adjustment, region relabeling, spatial hint supplementation, or semantic description correction; the manual correction information updates the segmentation result, including performing at least one of region replacement, label update, boundary refinement, or weighted fusion on the region to be corrected.

6. The pathological image segmentation self-correction method based on uncertainty metric according to claim 1, characterized in that: Step S4, the optimization update, includes constructing a correction sample set using manually corrected samples and the updated segmentation results, updating the segmentation model parameters, and converting the manually corrected region results into spatial cue information and feeding it back to the segmentation model, so as to segment subsequent pathological images using the updated segmentation model and the spatial cue information.

7. The pathological image segmentation self-correction method based on uncertainty metric according to claim 1, characterized in that: It also includes calculating preset indicators based on the corrected segmentation results obtained in step three. The preset indicators include at least one indicator calculated based on the area ratio of the target segmented region, the intersection ratio of different regions, the consistency of regional boundaries, or the regional distribution relationship.

Images