Medical image segmentation quality evaluation method and system based on multi-scale deep learning

Abstract

The invention discloses a medical image segmentation quality evaluation method and system based on multi-scale deep learning. According to the method, a SynthQA framework is constructed, controllable degradation is carried out on a high-quality labeling mask through a soft degradation strategy, and a probability graph for simulating output of a real model and a corresponding Dice quality label are generated and used for training an end-to-end regression model. According to the model, a three-level down-sampling three-dimensional convolution encoder is adopted to extract multi-scale context features, probability graphs are directly used as input, thresholding processing is not needed, and meanwhile, target anatomical region identifiers are introduced as condition signals, so that the model can carry out non-reference quality scoring on the probability graphs output by different regions; the method is suitable for quality evaluation of multi-part and multi-region medical image segmentation tasks of brain, lung, liver and the like.

Description

Technical Field

[0001] This application relates to the field of intelligent analysis and quality control technology for medical images, specifically, to a method and system for evaluating the quality of medical image segmentation based on multi-scale deep learning. This method can perform reference-free quality scoring on the three-dimensional probability maps output by automatic segmentation models, and is applicable to the quality evaluation of multi-region segmentation results for various anatomical structures such as brain tumors, liver lesions, and lung nodules. It can effectively support automated quality control in clinical deployment. Background Technology

[0002] In medical image analysis, deep learning models have been widely used for the automatic segmentation of organs or lesions. These models typically output a three-dimensional probability map representing the confidence level of each voxel belonging to the target structure, with values ​​ranging from [value range missing]. However, due to limitations in imaging quality, lesion morphology variations, or model generalization ability, automatic segmentation results may suffer from issues such as blurred boundaries, missed small structures, and topological errors, affecting subsequent clinical decisions.

[0003] To achieve reference-free quality assessment (i.e., without relying on expert annotation), existing technologies typically extract manually designed geometric or topological features (such as the number of connected components, surface smoothness, volume rationality, etc.) from the segmentation results and judge the quality by setting a preset threshold, or construct a synthetic dataset, generate labeled samples by artificially introducing errors, and train a regression model to predict the quality score.

[0004] However, in practical applications, directly using the binarized segmentation mask as input to the evaluation model will result in the loss of boundary uncertainties and confidence distribution information contained in the original probability map. The original output of clinical segmentation models is typically a continuous probability map, and its quality is reflected not only in the final binary result but also in the reasonableness of the probability distribution. Therefore, evaluation methods based solely on binary masks are insufficient to fully capture the true reliability of the segmentation results.

[0005] Furthermore, for tasks involving multiple anatomical subregions (such as whole tumor, tumor core, and enhancing tumor segmentation in brain tumor segmentation), different regions exhibit significant differences in morphology, scale, and clinical significance, making it difficult for a uniform assessment strategy to accommodate the unique characteristics of each region. Therefore, there is an urgent need for a segmentation quality assessment method that can directly process probabilistic graph inputs, preserve uncertainty information, and support regionally differentiated modeling. Summary of the Invention

[0006] The purpose of this application is to provide a medical image segmentation quality assessment method and system based on multi-scale deep learning, aiming to address several technical bottlenecks in existing no-reference quality assessment techniques, including: assessment bias caused by mismatch between input format and clinical model output, insufficient generalization ability due to narrow quality distribution of synthetic training samples, and lack of ability to model specificity of multiple anatomical regions. By introducing probabilistic graphical direct modeling, controllable soft degradation strategies, and region condition mechanisms, this invention can provide accurate, robust, and interpretable quality scoring for the output of automated clinical segmentation systems.

[0007] To achieve the above objectives, the present invention provides a method for evaluating the quality of medical image segmentation, the method comprising two parts: a training phase and an inference phase.

[0008] During the training phase, a set of high-quality 3D binary labeled masks is first obtained. These masks are manually drawn by medical experts on standard medical images, and each mask corresponds to a specific anatomical region or lesion type. These masks serve as a "clean" baseline for subsequent synthesis of labeled training samples.

[0009] Subsequently, a series of soft degradation operations are applied to each high-quality mask to generate a 3D segmentation probability map that simulates the output of a realistic segmentation model. This probability map is a continuous-valued tensor, where the value of each voxel is located at... The interval represents the confidence level that the location belongs to the target structure. The design principle of soft degradation operations is to simulate common error patterns in clinical segmentation models while preserving reasonable spatial structure. Specifically, this includes:

[0010] Boundary smoothing modeling: Calculate the distance from each point inside the mask to the boundary based on the Euclidean Distance Transform, and normalize it to... The range is defined to form the basic probability distribution. This operation simulates the characteristic of the segmentation model where the output confidence gradually decays in the boundary region;

[0011] Spatial correlation noise injection: Generate random noise that follows a standard normal distribution, filter it with a three-dimensional Gaussian filter, and then superimpose it onto the base probability map to simulate imaging artifacts or spatial correlation uncertainties in the model inference process;

[0012] Small structure attenuation: Identify connected regions with a volume smaller than a preset threshold (e.g., 60 voxels), and multiply their probability values ​​by a factor in... A random factor uniformly sampled within a range is used to simulate the phenomenon of missed detection of small lesions or weak response;

[0013] Global confidence shift: Applying a certain probability (e.g., 50%) to the entire probability map.

[0014] The constant offset within the range is then cropped to the desired value. Intervals are defined to cover situations where the model is overconfident or underconfident overall.

[0015] Degradation intensity grading control: By combining the parameters of the above operations (such as noise intensity, attenuation ratio, and offset amplitude), three degradation levels of mild, moderate, and severe are defined, and random sampling is performed according to a preset ratio to ensure that the generated training samples have broad coverage on the Dice coefficient.

[0016] After generating the probability map, it is binarized with a fixed threshold (usually 0.5) to obtain the corresponding binary segmentation result. This result is compared with the original high-quality mask to calculate the Dice similarity coefficient, which serves as the supervision label for the sample.

[0017] Next, a multi-scale 3D convolutional neural network is constructed as a quality assessment model. This network includes an encoder structure consisting of three levels of downsampled convolutional blocks. Each level achieves spatial dimensionality compression through a convolution operation with stride=2, while progressively increasing the number of channels to capture contextual information from local to global perspectives. The encoder's final output feature map is compressed into a fixed-dimensional (e.g., 128-dimensional or 256-dimensional) global feature vector through global adaptive average pooling. This operation supports arbitrary input sizes and avoids information loss caused by resampling.

[0018] Simultaneously, based on the type of anatomical region being processed, it is mapped to a one-hot encoded vector. This vector is concatenated with the aforementioned global feature vector along the channel dimension to form a fused feature. The fused feature is then fed into a regression head consisting of two fully connected layers, ultimately outputting a scalar value, namely the predicted Dice coefficient. The model is trained end-to-end by minimizing the L1 loss between the predicted value and the true Dice label.

[0019] During the inference phase, the system receives the raw output from the clinical automated segmentation system—a three-dimensional segmentation probability map that has not been thresholded. This probability map is typically stored in NIfTI or NumPy format and directly reflects the model's confidence in classifying each voxel.

[0020] The user or system specifies the anatomical region category corresponding to the probability map (e.g., "enhancing tumor"), which the system converts into a corresponding one-hot vector. The probability map, along with the region identifier, is then input into a trained quality assessment model. The model performs the same forward propagation process as during training, outputting a predicted Dice value between 0 and 1 as a quality score for the segmentation result.

[0021] The present invention also provides a corresponding medical image segmentation quality assessment system, comprising the following functional modules:

[0022] Mask loading module: used to read 3D segmentation probability maps in .nii.gz or .npy format, supporting different medical image modalities (such as T1ce-MRI, CT, etc.);

[0023] Region identification module: used to parse file naming rules or receive user input to determine the target region identifier;

[0024] Feature encoding module: Deploy the above-mentioned multi-scale 3D convolutional encoder and global adaptive average pooling operation to extract semantic features of the input probability map;

[0025] Quality prediction module: concatenates the region one-hot vector with global features and outputs the predicted Dice value through a fully connected regression layer;

[0026] Results output module: Returns the predicted score.

[0027] The system can operate as a standalone tool or be integrated into a medical image AI analysis platform, PACS system, or research pipeline to achieve automated quality screening of batch segmentation results, significantly reducing the cost of manual review.

[0028] The technical advantages of this application are reflected in the following aspects:

[0029] High input authenticity: The model input is directly based on the probability map, which is consistent with the original output format of mainstream segmentation models, and fully preserves the boundary uncertainty and confidence distribution information;

[0030] Good training sample diversity: Through multi-strategy soft degradation, synthetic data covering a wide range of Dice values ​​is generated, improving the model's sensitivity to extremely low-quality cases;

[0031] Strong region awareness: The introduction of region identifiers as conditional signals enables the model to perform differentiated modeling based on the learning characteristics of different anatomical structures.

[0032] Highly flexible in deployment: No expert annotation or image spatial normalization is required; it supports input of any size, making it easy to implement in clinical practice. Attached Figure Description

[0033] Figure 1The overall architecture flowchart of the automated clinical segmentation system provided for the implementation of this application is shown. The system includes the following modules: an input terminal for the original segmentation probability map, a mask loading module, a region identification module, a feature encoding module, a quality prediction module, a result output module, and an output interface integrated into an AI analysis platform or PACS system. Specifically, the mask loading module receives the original segmentation probability map and sends the data to the region identification module and the feature encoding module for parallel processing. The region identification module identifies potential low-quality regions, and the feature encoding module extracts three-dimensional spatial features. The output results from both modules are fed into the quality prediction module, and after comprehensive evaluation, the result output module generates a quality score, which is ultimately integrated into the AI ​​analysis platform or PACS system for clinical use.

[0034] Figure 2 This is a schematic diagram of the deep neural network structure used in the quality prediction module of this application. The network takes a 3D probability map (H×W×D×1) as input and passes it through three 3D convolutional layers sequentially: the first convolutional layer uses a 3×3×3 kernel with 32 channels and a stride of 2, resulting in an output size of (H / 2)×(W / 2)×(D / 2)×32; the second convolutional layer has 64 channels, also using a 3×3×3 kernel with a stride of 2, resulting in an output size of (H / 4)×(W / 4)×(D / 4)×64; and the third convolutional layer has 128 channels, resulting in an output size of (H / 8)×(W / 8)×(D / 8)×128. Subsequently, global adaptive average pooling converts the 3D feature map into a fixed-dimensional vector, which is then concatenated with a one-hot encoded class vector. Finally, a fully connected layer outputs the predicted Dice coefficients, enabling a quantitative evaluation of the segmentation quality. Detailed Implementation

[0035] The following detailed description of specific embodiments of the present invention is provided in the context of brain tumor segmentation tasks. It should be noted that the present invention is not limited to brain tumors, but can also be extended to multi-region segmentation quality assessment of other sites such as the liver, lungs, and prostate.

[0036] Data Preparation: Brain tumor cases were obtained from publicly available medical image datasets (such as the BraTS series) or hospital internal databases and annotated by consensus from at least three senior radiologists. Each case contained three independent 3D binary masks, corresponding to:

[0037] Whole Tumor (WT): Includes necrotic areas, edematous areas, and enhancing tumors; Tumor Core (TC): Includes necrotic areas and enhancing tumors; Enhancing Tumor (ET): Refers only to the significantly enhanced portion. Each mask is saved in NumPy format, and the filename includes the region identifier (e.g., case_001_WT.npy).

[0038] Generating a soft degradation probability map:

[0039] The first step is distance transformation, which calculates the Euclidean distance from interior points to the boundary, divides it by the maximum distance value, and normalizes it to... To obtain the basic probability map .

[0040] The second step is noise injection, generating shape and... The same Gaussian noise After 3D Gaussian kernel After convolution, we get ,make Then cut to .

[0041] The third step is the attenuation of small lesions. Connectivity component analysis was performed on the region, and for components with a volume less than 60, their probability values ​​were multiplied by . (Uniform sampling).

[0042] The fourth step is global offset, which generates the offset with a 50% probability. ,make .

[0043] The fifth step involves intensity grading. Based on the preset mode (light / medium / heavy), parameters such as noise amplitude and attenuation ratio are adjusted to ensure the final Dice distribution coverage. .

[0044] During training, A binary image B is obtained with a threshold of 0.5, and Dice is calculated as the label d with the original mask Y.

[0045] The encoder is constructed using three convolutional blocks with channels ranging from 32 to 64 to 128. Each block contains two 3×3×3 convolutions, ReLU, and BatchNorm. Downsampling is achieved using convolutions with a stride of 2. The final feature map is then subjected to adaptive average pooling to obtain a 128-dimensional vector.

[0046] Region identifiers (WT / TC / ET) are converted into 3D one-hot vectors, concatenated, and then input into FC (128+3→64→1). The output is activated by a Sigmoid function because... .

[0047] Using the Adam optimizer ( L1 Loss Train for 50 rounds on a GPU.

[0048] During deployment, the segmentation model's segmentation output for the new patient's MRI (e.g., pred_ET.nii.gz) is loaded, the tensor is read, the region is identified, a one-hot vector is constructed, and the vector is input into the model to obtain the predicted Dice.

Claims

1. A medical image segmentation quality evaluation method characterized by, The method comprises the following steps: Obtain a three-dimensional segmentation probability map to be evaluated, which represents the segmentation confidence distribution of a certain anatomical region or lesion, and the pixel value is in the interval [0, 1]; Determine the target region identifier corresponding to the probability map, which is used to distinguish different anatomical structures; Input the probability map into a pre-trained multi-scale three-dimensional convolutional encoder to extract multi-scale context features, and compress the feature map into a fixed-dimensional global vector through global adaptive average pooling; Encode the target region identifier into a one-hot vector, and input it into a fully connected regression layer after being spliced with the global vector; Output the predicted Dice coefficient as the quality score of the segmentation result.

2. The method of claim 1, wherein, The multi-scale three-dimensional convolutional encoder contains three levels of down-sampling convolutional blocks, each of which uses a stride = 2 max-pooling operation to realize spatial dimension reduction.

3. The method of claim 1, wherein, The global adaptive average pooling module supports input of any spatial size, avoiding loss of boundary information due to resampling.

4. The method of claim 1, wherein, The target region identifier includes but is not limited to the whole tumor, tumor core or enhanced tumor categories of brain tumors.

5. The method of claim 1, wherein, The segmentation probability map is derived from the output of the clinical automatic segmentation system, and its format is NIfTI or NumPy, and it is not thresholded, retaining the uncertainty information of the model output.

6. A medical image segmentation quality evaluation system characterized by comprising: It comprises: Mask loading module for reading.nii.gz or.npy format segmentation probability map; Region identification module for parsing or receiving target region identifier; Feature encoding module running the multi-scale three-dimensional convolutional encoder and global adaptive average pooling module according to any one of claims 1-5; Quality prediction module, which inputs the region one-hot vector and global feature into a fully connected regression layer after splicing, and outputs the predicted Dice value; Result output module returns the predicted Dice and triggers quality alarm or pass signal.

7. The system of claim 6, wherein, The system can be integrated into a medical image AI analysis platform to automatically screen low-quality segmentation results and reduce manual review costs.

Images