A brain image segmentation method based on similarity perception feature calibration
By using modal fusion calibration and similarity weighted gating modules, the semantic gap problem in pathological regions of MRI images is solved, achieving efficient boundary enhancement and improved segmentation accuracy. This addresses the issue of insufficient semantic association between low-level encoder texture and high-level decoder in existing technologies, thereby improving the accuracy of image segmentation.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- WANNAN MEDICAL COLLEGE
- Filing Date
- 2026-03-18
- Publication Date
- 2026-07-14
AI Technical Summary
In existing technologies, pathological regions in MRI images have a critical semantic gap, resulting in almost no structural correlation between the texture of the low-level encoder and the semantics of the high-level decoder during image segmentation. This makes standard skip connections a source of noise, especially in soft-boundary and low-contrast regions where the segmentation effect is poor.
A similarity-aware feature calibration method is adopted, which performs similarity-weighted calibration and gradient focusing of image features through a modality fusion calibration module, a similarity-weighted gating module and a convolutional layer, thereby enhancing boundary reinforcement and improving the structural association of features and segmentation accuracy.
It effectively solves the semantic gap problem in image segmentation, improves the segmentation accuracy of low-contrast boundary regions, reduces segmentation collapse and geometric errors, and enhances the reliability of clinical diagnosis.
Smart Images
Figure CN122391256A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of brain image segmentation, and more particularly to a brain image segmentation method based on similarity perception feature calibration. Background Technology
[0002] Automated segmentation of brain tumors from multimodal magnetic resonance imaging (MRI) is a prerequisite for quantitative diagnosis, treatment planning, and postoperative monitoring. Despite significant successes with convolutional neural networks (CNNs) and recent Transformer-based architectures, clinical application remains hampered by failures in "hard" scenarios: pathological regions with soft, low-contrast boundaries and scenarios with incomplete imaging protocols (missing modalities). While models typically achieve high mean overlap scores (Dice), they often suffer from segmentation collapse, predicting empty masks in challenging slices. This can lead to catastrophic geometric errors (e.g., Hausdorff distance > 100 mm), undermining clinical confidence. A major reason for this instability is the inherent ambiguity of tissue transition zones. Medical images are characterized by “soft boundaries,” where the contrast between lesion infiltration and healthy edema is almost negligible. Standard loss functions are typically dominated by a large number of background pixels, leading to gradient starvation in these low-contrast boundary regions. Consequently, models tend to learn spurious co-occurrence patterns, relying on high-contrast context (e.g., ventricles) rather than the lesion itself to infer segmentation. Existing solutions (such as adding auxiliary boundary branches or uncertainty quantization modules) introduce significant computational complexity without fundamentally addressing the deficiencies in boundary feature representation.
[0003] The more fundamental bottleneck lies in the architecture itself. The U-Net paradigm relies on skip connections to recover spatial details by fusing encoder features with decoder representations. This design is based on the implicit assumption that encoder features and decoder tasks are semantically aligned. However, these typically rely on intensity-dependent activations or data-intensive global dependencies, which degrade rapidly when input modalities are missing or intensity distributions shift across scanners.
[0004] Therefore, it can be seen that there is a key semantic gap in the pathological regions of MRI images in the existing technology, which results in almost no structural correlation between the texture of the low-level encoder and the semantics of the high-level decoder during image segmentation, thus making the standard skip connections a source of noise. Summary of the Invention
[0005] The purpose of this invention is to address the problem that in the prior art, there is a key semantic gap in the pathological regions of MRI images, which leads to almost no structural correlation between the texture of the low-level encoder and the semantics of the high-level decoder during image segmentation, thus making standard skip connections a source of noise. The invention proposes a brain image segmentation method based on similarity-aware feature calibration.
[0006] To achieve the above objectives, the present invention adopts the following technical solution: A brain image segmentation method based on similarity-perceptual feature calibration includes: Step 1: Input the acquired NMR images of different modes into the corresponding dedicated encoder for processing to generate image features of different resolutions, and input the image features of different resolutions into the corresponding fusion calibration module; Step 2: Input the acquired NMR images of different modes into the internal variational autoencoder to generate prior image features of different resolutions, and input the prior image features of different resolutions into the corresponding fusion calibration module; Step 3: For each fusion calibration module, the image features at the corresponding resolution are fused with the fused and calibrated image features of the previous fusion calibration module in the modal fusion module to obtain fused features. Then, the fused features are calibrated with the prior image features at the corresponding resolution of the fusion calibration module in the similarity weighting gating module for similarity weighting calibration. Finally, the calibrated features are input into the convolutional layer for convolution to obtain the fused and calibrated image features. Step 4: Repeat step 3 until the last-level feature fusion calibration module outputs the fused and calibrated image features, which gives the segmentation result.
[0007] Furthermore, the fusion calibration module is composed of a modality fusion module, a similarity weighted gating module, and a convolutional layer connected in sequence.
[0008] Furthermore, the specific steps in step 2 of inputting the acquired NMR images of different modalities into the internal variational autoencoder to generate prior image features of different resolutions include: Step 21: The encoding module in the internal variational autoencoder maps the acquired NMR images of different modes to the distribution parameters of the latent space; Step 22: Sample latent variables from the distribution parameters of the latent space using reparameterization techniques; Step 23: The decoding module in the internal variational autoencoder reconstructs the reconstructed NMR images of different modes based on latent variables; Step 24: After reconstruction, the NMR images of different modalities are subjected to feature extraction and resolution transformation by configuring 3×3 convolutional layers of different lengths to obtain prior image features of different resolutions.
[0009] Furthermore, the similarity-weighted gating module includes: The similarity suppression module is used to generate normalized cross-correlation scores based on the fused features and prior image features; The contrast enhancement region attention module is used to perform multi-rate dilated convolution on the fused features to achieve gradient focusing boundary enhancement. The Sigmoid function is used to calculate and generate soft-gated weights based on the normalized cross-correlation score. The feature weighting module is used to multiply the soft-gated weights with the features after multi-rate dilated convolution on an element-wise basis.
[0010] Furthermore, the specific steps of performing similarity weighting calibration in the similarity weighting gating module in step 3, where the fused features are compared with the prior image features at the corresponding resolution of the fusion calibration module, include: Step 31: Input the fused features and the prior image features of the corresponding resolution of the fusion calibration module into the similarity suppression module for normalized cross-correlation calculation to obtain the normalized cross-correlation score; Step 32: Input the normalized cross-correlation score into the Sigmoid function for calculation to obtain the soft-gating weights; Step 33: Input the fused features into the contrast enhancement region attention module and perform multi-rate dilation convolution to obtain the fused features after multi-rate dilation convolution; Step 34: Multiply the soft-gated weights element-wise with the features fused after multi-rate dilation convolution to obtain the calibrated features.
[0011] Furthermore, the formula for calculating the normalized cross-correlation in step 31 is:
[0012] Among them, This represents the normalized cross-correlation score. The eigenvalues represent the features after fusion. The feature values represent the features of the prior image. This represents the local mean of the fused features. This represents the local mean of the features of the prior image. The neighborhood of pixel position u This represents the smoothing factor.
[0013] Furthermore, in step 32, the normalized cross-correlation score is input into the Sigmoid function for calculation using the following formula:
[0014] in, Indicates the soft-gating weight, a l and b l This represents the learnable parameters.
[0015] Furthermore, the formula for element-wise multiplication of the soft-gated weights with the features fused after multi-rate dilated convolution is as follows:
[0016] in, Indicates the calibrated features, This represents the features after fusion following multi-rate dilated convolution.
[0017] Furthermore, step 33 includes the following specific steps: Step 331: After fusion, the features are convolved by three parallel dilation convolution branches in the contrast enhancement region attention module to obtain three image features with different dilation rates; Step 332: Concatenate the image features of three different dilation rates along the channel dimension to obtain the concatenated image features; Step 333: Perform a 1×1 convolution on the stitched image features and then follow it with a Sigmoid function to generate a scalar attention map; Step 334: Spatially weight the fused features according to the scalar attention map, and then add the spatially weighted features to the fused features element by element to obtain the fused features after multi-rate dilated convolution.
[0018] Compared with existing technologies, the advantages of this invention are: 1. This invention performs similarity weighted calibration on the fused features and prior image features through a similarity weighted gating module. First, the fused features and prior image features are normalized cross-correlation calculated through a similarity suppression module to generate a similarity coefficient, i.e., a normalized cross-correlation score, that reflects the consistency between the structures of the fused features and prior image features. Then, this coefficient is passed through a Sigmoid function to generate soft gating weights, which suppress high-similarity background information and enhance low-similarity region signals. Simultaneously, the fused features are processed by a contrast enhancement region attention module through multi-rate dilation convolution to obtain multi-rate dilation convolution fused features, achieving gradient focusing boundary enhancement. Finally, the soft gating weights and multi-rate dilation convolution fused features are multiplied element-wise to complete the similarity weighted calibration. This process transforms the semantic gap from an interference signal into a beneficial diagnostic prior. In the prior art, there is a key semantic gap in the pathological regions of MRI images, resulting in almost no structural correlation between the texture of the low-level encoder and the semantics of the high-level decoder during image segmentation, thus causing standard skip connections to become noise sources.
[0019] 2. The similarity weighted gating module of this invention introduces a contrast enhancement region attention module to strengthen the boundary of the fused features. The contrast enhancement region attention module redistributes gradient weights during backpropagation, actively drawing the optimization focus of the fused features to the interface region between the tumor and edema where the contrast is weak and easily ignored, thereby strengthening the boundary of gradient focusing and structurally preventing failure predictions of segmentation collapse in these regions during segmentation. Attached Figure Description
[0020] Figure 1The present invention presents an overall flowchart of a brain image segmentation method based on similarity perception feature calibration.
[0021] Figure 2 This invention presents a schematic diagram of the similarity-weighted gating module in a brain image segmentation method based on similarity-perceptual feature calibration.
[0022] Figure 3 This is a schematic diagram of the contrast enhancement region attention module in a brain image segmentation method based on similarity perception feature calibration proposed in this invention.
[0023] Figure 4 This is a performance comparison chart of various medical image segmentation algorithms, with BraTS2021 serving as the validation set during the implementation of this invention.
[0024] Figure 5 This is a comparison of the segmentation effect of the brain scan image generated by this invention and the segmentation image annotated by real experts in terms of lesion segmentation. Detailed Implementation
[0025] The invention will now be further explained with reference to the accompanying drawings.
[0026] In a U-shaped architecture, skip connections typically sum or concatenate the features of the encoder and decoder at a matching resolution. When the spatial support of the encoder features is inconsistent with the current decoder assumptions, this indiscriminate fusion can inject semantic noise and easily blur the boundaries of lesions.
[0027] like Figure 1 As shown, based on the above findings, a brain image segmentation method based on similarity-perceptual feature calibration is provided, including: Step 1: Input the acquired NMR images of different modes into the corresponding dedicated encoder for processing to generate image features of different resolutions, and input the image features of different resolutions into the corresponding fusion calibration module.
[0028] Step 2: Input the acquired NMR images of different modes into the internal variational autoencoder to generate prior image features of different resolutions, and input the prior image features of different resolutions into the corresponding fusion calibration module.
[0029] Step 3: For each fusion calibration module, the image features at the corresponding resolution are fused with the fused and calibrated image features of the previous fusion calibration module in the modal fusion module to obtain fused features. Then, the fused features are calibrated with the prior image features at the corresponding resolution of the fusion calibration module in the similarity weighting gating module for similarity weighting. The calibrated features are then input into the convolutional layer for convolution to obtain the fused and calibrated image features.
[0030] Step 4: Repeat step 3 until the last-level feature fusion calibration module outputs the fused and calibrated image features, which gives the segmentation result.
[0031] The fusion calibration module consists of a modality fusion module, a similarity weighted gating module, and a convolutional layer connected in sequence. Image features with the same resolution under different modalities are input into the same modality fusion module for feature fusion.
[0032] In step 1, four modal encoders independently process the magnetic resonance images of the T1, T1ce, flair, and T2 sequences. At the resolution level, each encoder generates image features corresponding to the resolution.
[0033] In step 2, IntroVAE is a variational autoencoder, comprising two core components: an encoder and a decoder. When different modal NMR images are input into the IntroVAE, the encoder maps the input NMR images to the distribution parameters of the latent space. The formula for this process is:
[0034] in, X The input NMR images represent different modalities, and μ represents the mean vector of the encoded feature distribution. The character set represents the degree of discreteness of the feature distribution. The encoder represents a neural network encoder, which is used to map high-dimensional inputs to a low-dimensional latent space.
[0035] Then, latent variables are sampled from the distribution parameters of the latent space using a reparameterization technique. The formula for this process is:
[0036] in, Representing latent variables, Indicates standard deviation, This indicates the noise distribution.
[0037] Next, the decoder reconstructs the reconstructed NMR images of different modes based on latent variables. The formula for this process is:
[0038] in, The image represents the reconstructed NMR images of different modalities. The decoder represents the neural network decoder, which is used to map the low-dimensional latent variable z back to the high-dimensional data space.
[0039] Finally, the reconstructed NMR images of different modalities were subjected to feature extraction and resolution transformation by configuring 3×3 convolutional layers of different lengths to obtain prior image features of different resolutions.
[0040] In step 3, there are four fusion calibration modules. In each fusion calibration module, the modal fusion module inputs the image features of its corresponding resolution and the fused and calibrated image features of the previous fusion calibration module. The image features of the corresponding resolution and the fused and calibrated image features of the previous fusion calibration module are first aligned with the channels through a 1×1 convolution, and then the features are concatenated along the channel dimension to obtain the fused features. The modal fusion module corresponding to the lowest resolution image features does not have the fused and calibrated image features of the previous fusion calibration module, and the modal fusion module corresponding to the highest resolution image features does not have the subsequent modal fusion module.
[0041] Next, the fused features and the prior image features at the corresponding resolution of the fusion calibration module are input into the similarity weighting gating module for similarity weighting calibration.
[0042] Finally, the calibrated features are input into a convolutional layer for convolution to obtain the fused and calibrated image features.
[0043] The fused features and the prior image features at the corresponding resolution of the fusion calibration module are input into the similarity weighting gating module for similarity weighting calibration. The specific steps include: First, the fused features and the prior image features at the corresponding resolution of the fusion calibration module are input into the similarity suppression module for normalized cross-correlation calculation to obtain the normalized cross-correlation score. The formula for normalized cross-correlation calculation is as follows:
[0044] Among them, This represents the normalized cross-correlation score. The eigenvalues represent the features after fusion. The feature values represent the features of the prior image. This represents the local mean of the fused features. This represents the local mean of the features of the prior image. The neighborhood of pixel position u This represents the smoothing factor.
[0045] The normalized cross-correlation score is then input into the Sigmoid function for calculation to obtain the soft-gating weights. The formula for calculating the normalized cross-correlation score using the Sigmoid function is as follows:
[0046] in, Indicates the soft-gating weight, a l and b l This represents the learnable parameters.
[0047] Meanwhile, the fused features are input into the contrast enhancement region attention module for multi-rate dilation convolution to obtain the fused features after multi-rate dilation convolution.
[0048] Finally, the soft-gated weights are multiplied element-wise with the fused features after multi-rate dilated convolution to obtain the calibrated features. The formula for this process is as follows:
[0049] in, Indicates the calibrated features, This represents the features after fusion following multi-rate dilated convolution.
[0050] like Figure 2 As shown, the similarity-weighted gating module includes: The similarity suppression module is used to generate normalized cross-correlation scores based on the fused features and prior image features; The contrast enhancement region attention module is used to perform multi-rate dilated convolution on the fused features to achieve gradient focusing boundary enhancement. The Sigmoid function is used to calculate and generate soft-gated weights based on the normalized cross-correlation score. The feature weighting module is used to multiply the soft-gated weights with the features after multi-rate dilated convolution on an element-wise basis.
[0051] like Figure 3 As shown, the specific steps for inputting the fused features into the contrast enhancement region attention module and performing multi-rate dilated convolution include: The process of fusing and then feature extraction involves convolution through three parallel dilation convolution branches in the contrast enhancement region attention module to obtain image features with three different dilation rates. The formula for this process is as follows:
[0052] Among them G d F represents image features with different dilation rates. in C represents the features of the input image. d This represents the dilated convolution branch.
[0053] The three image features with different dilation rates are then concatenated along the channel dimension to obtain the concatenated image features. The concatenated image features are then convolved with a 1×1 convolution followed by a Sigmoid function to generate a scalar attention map. Finally, the fused features are spatially weighted according to the scalar attention map, and then the spatially weighted features are added element-wise to the fused features to obtain the fused features after multi-rate dilation convolution.
[0054] In step 4, there are four fusion calibration modules, each of which repeats step 3. The modal fusion module corresponding to the lowest resolution image features does not have the fused and calibrated image features of the previous fusion calibration module, so it only needs to fuse the lowest resolution image features. The modal fusion module corresponding to the highest resolution image features does not have the subsequent modal fusion module, and its output is the segmentation result.
[0055] This invention utilizes the benchmark dataset BraTS2021 from the BraTS Challenge series. This dataset contains four standard modalities: T1, T1ce, T2, and flair sequences. The T1ce sequence highlights enhanced tumors, while the T2 and flair sequences delineate edema and whole tumor boundaries. The tumor region is defined as three types: tumor core, enhanced tumor, and whole tumor. It includes 1251 training samples and 219 validation samples. Performance comparisons are performed on various medical image segmentation algorithms (including the method of this invention), such as... Figure 4 The core evaluation metrics for performance comparison are Dice coefficient (%, the higher the better) and HD95 (mm, the lower the better), which are used to evaluate WT (whole tumor), TC (tumor core), ET (enhanced tumor) and average (Avg) results, respectively.
[0056] Figure 5 The invention demonstrates a comparison of the segmentation effect of brain scan images and real expert-annotated segmentation images in lesion segmentation. The lesion segmentation images of the invention are close to the real expert annotations and can effectively segment lesion regions of different shapes and sizes. In addition, the edges of the segmented lesions are basically consistent with the real labels.
[0057] As is known from common technical knowledge, this invention can be implemented through other embodiments that do not depart from its spirit or essential characteristics. Therefore, the disclosed embodiments described above are merely illustrative and not exhaustive. All modifications within the scope of this invention or its equivalents are included in this invention.
Claims
1. A brain image segmentation method based on similarity-perceptual feature calibration, characterized in that, include: Step 1: Input the acquired NMR images of different modes into the corresponding dedicated encoder for processing to generate image features of different resolutions, and input the image features of different resolutions into the corresponding fusion calibration module; Step 2: Input the acquired NMR images of different modes into the internal variational autoencoder to generate prior image features of different resolutions, and input the prior image features of different resolutions into the corresponding fusion calibration module; Step 3: For each fusion calibration module, the image features at the corresponding resolution are fused with the fused and calibrated image features of the previous fusion calibration module in the modal fusion module to obtain fused features. Then, the fused features are calibrated with the prior image features at the corresponding resolution of the fusion calibration module in the similarity weighting gating module for similarity weighting calibration. Finally, the calibrated features are input into the convolutional layer for convolution to obtain the fused and calibrated image features. Step 4: Repeat step 3 until the last-level feature fusion calibration module outputs the fused and calibrated image features, which gives the segmentation result.
2. The brain image segmentation method based on similarity-perceptual feature calibration according to claim 1, characterized in that: The fusion calibration module consists of a modality fusion module, a similarity weighted gating module, and a convolutional layer connected in sequence.
3. The brain image segmentation method based on similarity-perceptual feature calibration according to claim 1, characterized in that: The specific steps in step 2 of inputting the acquired NMR images of different modalities into the internal variational autoencoder to generate prior image features of different resolutions include: Step 21: The encoding module in the internal variational autoencoder maps the acquired NMR images of different modes to the distribution parameters of the latent space; Step 22: Sample latent variables from the distribution parameters of the latent space using reparameterization techniques; Step 23: The decoding module in the internal variational autoencoder reconstructs the reconstructed NMR images of different modes based on latent variables; Step 24: After reconstruction, the NMR images of different modalities are subjected to feature extraction and resolution transformation by configuring 3×3 convolutional layers of different lengths to obtain prior image features of different resolutions.
4. The brain image segmentation method based on similarity-perceptual feature calibration according to claim 1, characterized in that: The similarity-weighted gating module includes: The similarity suppression module is used to generate normalized cross-correlation scores based on the fused features and prior image features; The contrast enhancement region attention module is used to perform multi-rate dilated convolution on the fused features to achieve gradient focusing boundary enhancement. The Sigmoid function is used to calculate and generate soft-gated weights based on the normalized cross-correlation score. The feature weighting module is used to perform element-wise multiplication of the soft-gated weights with the features fused after multi-rate dilated convolution.
5. The brain image segmentation method based on similarity-perceptual feature calibration according to claim 4, characterized in that: In step 3, the fused features are compared with the prior image features of the corresponding resolution in the similarity weighting gating module for similarity weighting calibration. The specific steps include: Step 31: Input the fused features and the prior image features of the corresponding resolution of the fusion calibration module into the similarity suppression module for normalized cross-correlation calculation to obtain the normalized cross-correlation score; Step 32: Input the normalized cross-correlation score into the Sigmoid function for calculation to obtain the soft-gating weights; Step 33: Input the fused features into the contrast enhancement region attention module and perform multi-rate dilation convolution to obtain the fused features after multi-rate dilation convolution; Step 34: Multiply the soft-gated weights element-wise with the features fused after multi-rate dilation convolution to obtain the calibrated features.
6. The brain image segmentation method based on similarity-perceptual feature calibration according to claim 5, characterized in that: The formula for calculating the normalized cross-correlation in step 31 is: Among them, This represents the normalized cross-correlation score. The eigenvalues represent the features after fusion. The feature values represent the features of the prior image. This represents the local mean of the fused features. This represents the local mean of the features of the prior image. The neighborhood of pixel position u This represents the smoothing factor.
7. The brain image segmentation method based on similarity-perceptual feature calibration according to claim 6, characterized in that: Step 32 involves inputting the normalized cross-correlation score into the Sigmoid function for calculation, using the following formula: in, Indicates the soft-gating weight, a l and b l This represents the learnable parameters.
8. The brain image segmentation method based on similarity-perceptual feature calibration according to claim 7, characterized in that: The formula for element-wise multiplication of the features after fusing the soft-gated weights with the multi-rate dilated convolution is as follows: in, Indicates the calibrated features, This represents the features after fusion following multi-rate dilated convolution.
9. The brain image segmentation method based on similarity-perceptual feature calibration according to claim 6, characterized in that: Step 33 includes the following specific steps: Step 331: After fusion, the features are convolved by three parallel dilation convolution branches in the contrast enhancement region attention module to obtain three image features with different dilation rates; Step 332: Concatenate the image features of three different dilation rates along the channel dimension to obtain the concatenated image features; Step 333: Perform a 1×1 convolution on the stitched image features and then follow it with a Sigmoid function to generate a scalar attention map; Step 334: Spatially weight the fused features according to the scalar attention map, and then add the spatially weighted features to the fused features element by element to obtain the fused features after multi-rate dilated convolution.