Medical image segmentation method based on adaptive confrontation disturbance

Abstract

The invention relates to the technical field of medical image processing, in particular to a medical image segmentation method based on adaptive adversarial disturbance, which comprises the following steps: constructing and training a medical image segmentation model based on adaptive adversarial disturbance, and inputting an image to be processed into the trained medical image segmentation model to obtain a segmentation result; the medical image segmentation model comprises a feature extraction module, a mixed distribution sampling module, a sensitive disturbance screening module and a decoder module; according to the method, the robustness and generalization ability of the model on data in different fields are remarkably improved, and even if the target domain data are scarce, the segmentation precision can be kept at a high level.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to a medical image segmentation method based on adaptive adversarial disturbance. Background Art

[0002] Current medical image segmentation techniques mainly rely on deep convolutional neural networks, especially U-Net and its variants, which perform well on large-scale natural image data. However, due to reasons such as acquisition equipment, scanning protocols, and individual differences in patients, the data distribution of medical image data often has obvious deviations, resulting in insufficient robustness and decreased generalization performance of the trained models in cross-domain applications. To solve this problem, the existing technology has proposed a variety of domain adaptation and domain generalization methods, such as using adversarial domain adaptation strategies (such as the method of Ganin et al.) and the CORAL method to alleviate distribution differences by aligning source domain features. However, most of these methods rely on target domain data for fine-tuning or alignment, which is limited in actual clinical applications because target domain data is difficult to obtain. In addition, in recent years, technologies based on data augmentation and feature perturbation have also emerged, such as MixStyle, which expands the style space of model adaptation by applying random perturbations to intermediate feature statistics (such as mean and standard deviation), and tries to simulate a variety of potential domain shifts. However, these methods use fixed or single perturbation strategies, which are difficult to cover all possible changes in the target domain. At the same time, the semantic consistency of the original features may be destroyed in the process of introducing perturbations, resulting in insufficient segmentation accuracy and stability in extreme cases. Although the existing technology as a whole provides a certain solution to the problem of domain shift, it still has shortcomings such as insufficient capture of diversified domain features, insufficient perturbation strategy targeting, and strong dependence on target domain data. These shortcomings directly restrict the application effect of medical image segmentation technology in actual clinical auxiliary diagnosis. Summary of the invention

[0003] In order to cope with the common data redundancy problem in medical image segmentation tasks and the challenges brought by the distribution differences between domains under different acquisition environments, the present invention provides a medical image segmentation method based on adaptive adversarial perturbation, including constructing and training a medical image segmentation model based on adaptive adversarial perturbation, inputting the image to be processed into the trained medical image segmentation model to obtain a segmentation result; the medical image segmentation model includes a feature extraction module, a mixed distribution sampling module, a sensitive perturbation screening module and a decoder module;

[0004] The training process of the medical image segmentation model includes the following steps:

[0005] S1. Collect image data and preprocess them to obtain a training set;

[0006] S2. Randomly select samples from the training set and input them into the feature extraction module to obtain feature F, and calculate the original statistics; the original statistics include the mean μ and the standard deviation σ;

[0007] S3. Input the original statistics into the mixture distribution sampling module to obtain the perturbed feature F HDS ;

[0008] S4. Input feature F and the perturbed feature F HDS into the sensitive perturbation screening module to obtain the modulation matrix;

[0009] S5. Calculate the sensitive screening whitening loss through the modulation matrix and perform adversarial perturbation update;

[0010] S6. After reallocating the features and the perturbed features, input them into the decoder module to obtain the segmentation result;

[0011] S7. Repeat steps S2 - S6, and perform iterative training until the model parameters converge.

[0012] Further, step S3 specifically includes:

[0013] S31. Based on the mean μ and the standard deviation σ, sample random statistics from two different probability distributions; the random statistics include the mean μ p1 , the standard deviation σ p2 , the mean μ p2 and the standard deviation σ p2 ;

[0014] S32. Perform weighted fusion on the random statistics to obtain the enhanced statistics, the enhanced statistics include the mean μ aug , the standard deviation σ aug , where

[0015]

[0016] λ1 ∈ [0, 1] represents the mixing coefficient;

[0017] S33. Sample the probability parameter P from the Beta distribution, where P ~ Beta(α, α), and α is the sampling parameter;

[0018] S34. Generate a Bernoulli distribution based on the probability parameter P and sample to obtain the Bernoulli parameter λ;

[0019] S35. Obtain the mixed statistics based on the Bernoulli parameter λ, the mixed statistics include the mean μ mix , the standard deviation σ mix , where

[0020] μ mix = λμ + (1 - λ)μ aug

[0021] σ mix = λσ+(1 - λ)σ aug

[0022] S36. Obtain the perturbation feature F based on the mixed statistic HDS , where

[0023]

[0024] Furthermore, in step S31, the Logistic normal distribution and the uniform distribution are adopted.

[0025] Furthermore, step S4 specifically includes:

[0026] S41. Perform singular value decomposition on the feature F and the perturbation feature F HDS respectively to obtain the singular diagonal matrices Σ F and Σ F * ;

[0027] S42. Construct the covariance matrices F F and Σ F * ; cov and F cov * ;

[0028] S43. Calculate the modulation matrix through the covariance matrices F cov and F cov * ;

[0029] Furthermore, step S43 specifically includes

[0030] Calculate the absolute difference Δ = |F cov - F cov * | through the covariance matrices F cov - F cov * |;

[0031] Classify all channels using the K - means unsupervised clustering algorithm, and divide the channels into two categories: highly sensitive and low - sensitive;

[0032] Construct a one - hot mask matrix M, where the positions of the channels classified as highly sensitive are assigned 1, and the remaining positions are assigned 0;

[0033] Add the one - hot mask matrix M to the identity matrix I to obtain the modulation matrix M'.

[0034] Furthermore, the sensitive screening whitening loss LW Denoted as

[0035] L W = E[||(F cov ⊙M′)-I|| F

[0036] where E[] represents expectation, F cov represents the covariance matrix obtained through feature F; ⊙ represents element-wise multiplication, ||·|| F is the Frobenius norm, M’ represents the modulation matrix, and I represents the identity matrix.

[0037] Furthermore, step S5 includes: dynamically updating the domain noise parameters ∈1 and ∈2 during the training process, and the update strategy is based on the backpropagation of the whitening loss gradient, which is achieved through the following formula

[0038]

[0039] where represents the backpropagation gradient of the domain noise parameter ∈1 based on the whitening loss gradient

[0040] represents the backpropagation gradient of the domain noise parameter ∈2 based on the whitening loss gradient, and α’ represents the hyperparameter.

[0041] Advantages of the present invention

[0042] The domain generalization method based on mixed distribution sampling replaces the original statistics with enhanced statistics, fuses the mixed statistics and the original statistics in the channel dimension, making the feature style more diverse while maintaining the basic semantic information of the original features. This mixing method can introduce style transformation while avoiding feature loss caused by excessive perturbation, ensuring that the model can not only learn domain-invariant information but also adapt to the distribution differences of unseen data, thereby effectively improving the generalization ability of the model.

[0043] The mixed distribution sampling module can be used as a plug-and-play component and can be flexibly embedded into any CNN-based architecture without significantly modifying the structure of the backbone network. This design makes the mixed distribution sampling module have strong versatility and scalability and can be adapted to a variety of different tasks and datasets.

[0044] ​Generally speaking, the present invention significantly improves the robustness and generalization ability of the model on data in different fields. Even when the data in the target domain is scarce, the segmentation accuracy can still be maintained at a high level. The search space of feature styles is expanded through mixed distribution sampling, and the domain shift under various acquisition conditions is more comprehensively simulated, overcoming the defects of the traditional method with a single perturbation strategy and limited coverage. The adaptive adversarial perturbation mechanism can accurately identify and regulate key channels to ensure that the original semantic information is not damaged during the perturbation process, further improving the model stability and fine segmentation ability; the modular design and the training-inference separation mechanism are not only convenient for seamless integration with the existing segmentation network but also meet the requirements of clinical applications for real-time performance and efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 is a flowchart of the method of the present invention;

[0046] Figure 2 is an overall structure diagram of the medical image segmentation method based on adaptive adversarial perturbation of the present invention;

[0047] Figure 3 is a module structure diagram of the domain generalization method based on mixed distribution sampling of the present invention;

[0048] Figure 4 is a module structure diagram of the domain generalization method based on adaptive adversarial perturbation of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0049] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0050] The present invention provides a medical image segmentation method based on adaptive adversarial perturbation, including constructing and training a medical image segmentation model based on adaptive adversarial perturbation, and inputting the image to be processed into the trained medical image segmentation model to obtain a segmentation result; the medical image segmentation model includes a feature extraction module, a mixed distribution sampling module, a sensitive perturbation screening module, and a decoder module.

[0051] In an embodiment of the present invention, a segmentation network is constructed based on the U-Net structure, and ResNet-34 is used as the backbone network to make full use of its powerful feature extraction ability and efficient hierarchical feature expression ability. Among them, the downsampling part is used as the feature extraction module, and the upsampling part is used as the decoder module. The overall structure is as Figure 2As shown. During the training phase, the mixed distribution sampling module is inserted between downsampling and upsampling to effectively perturb the feature space so that it can maintain a stable learning ability under different style statistics.

[0052] Preferably, in medical image segmentation tasks, convolutional neural networks (CNNs) usually extract a large number of features containing texture and style information from the lower layers, and the statistics (such as mean and standard deviation) of these features vary significantly between different domains. Traditional methods usually directly use source domain statistics for feature normalization or slight perturbation, but this often fails to fully capture the diversity of the target domain. At the same time, traditional methods are limited by the statistical distribution of source domain data and are difficult to cover various extreme situations that may occur in the target domain. Therefore, the present invention designs a mixed distribution sampling module, as Figure 3 shown. It adopts a domain generalization method based on mixed distribution sampling. By introducing random statistic sampling of a mixed probability distribution (such as Logistic normal distribution and uniform distribution), it randomly mixes with the original feature statistics in the channel dimension, thereby constructing a wider and more uniform feature style search space to effectively simulate various domain shift situations caused by different acquisition devices, imaging protocols, and individual differences.

[0053] Specifically, the domain generalization method based on mixed distribution sampling aims to randomly perturb the style of intermediate features, thereby changing the feature statistics (i.e., mean and standard deviation) and enhancing the adaptability of the model under different data distributions. Assume that the input feature of the mixed distribution sampling module is F ∈ R B×C×H×W (where B is the batch size, C is the number of channels, H is the image height, and W is the image width), and its corresponding channel statistics (mean μ and standard deviation σ) can be expressed as:

[0054] μ = Mean(F), σ = Std(F)

[0055] To expand the distribution space of the statistics, random statistics are sampled from two different probability distributions, denoted respectively as:

[0056] (1) Sampling values from distribution P1 (such as Logistic normal distribution) and

[0057] (2) Sampling values from distribution P2 (such as uniform distribution) and

[0058] Inspired by the effectiveness of non-realistic styles in the input space, the present invention hypothesizes that non-realistic styles can also be extended to the feature space. Considering that a single uniform distribution may not fully cover all potential statistical variations, we introduce mixed distribution sampling, that is, sampling from two or more probability distributions separately and then performing weighted fusion. The enhanced statistics are fused through the mixing coefficient λ1 ∈ [0, 1]:

[0059]

[0060] To explicitly learn domain-invariant representations, the present invention designs a statistics mixing strategy, the core of which is to randomly mix the enhanced statistics and the original statistics along the channel dimension. The specific steps are as follows:

[0061] First, sample P ∈ R from the Beta distribution B×C :

[0062] P ∼ Beta(α, α)

[0063] where the parameter α is set to 0.1 according to experience.

[0064] Using P as the probability parameter, generate a Bernoulli distribution and sample to obtain the Bernoulli parameter λ ∼ Bern(P), λ ∈ R B×C .

[0065] The mixed statistics (μ mix and σ mix ) are calculated by the following formula:

[0066] μ mix = λμ + (1 - λ)μ aug

[0067] σ mix = λσ + (1 - λ)σ aug

[0068] Apply the mixed statistics to the normalized feature F:

[0069]

[0070] To ensure the computational efficiency and model stability in the inference stage, the mixed distribution sampling module is only activated during the training process and removed in the inference stage. This can fully exert the style perturbation effect of HDS during the training process, improve the domain generalization ability of the model, while maintaining the computational efficiency and stability of the model during inference, and avoiding additional computational overhead. Thanks to this training-inference separation design, the mixed distribution sampling module can improve the model's adaptability to unseen data without affecting the inference performance, making it more suitable for practical application scenarios such as medical image segmentation.

[0071] Preferably, medical image data usually has highly redundant feature information. Due to factors such as imaging devices, acquisition conditions, and patient individual differences, there are significant differences in the statistical distributions of images from different sources. This makes it easy for traditional models to experience performance degradation when applied across domains. Therefore, the present invention designs a sensitive perturbation screening module, as Figure 4 shown. It adopts a domain generalization method based on adaptive adversarial perturbation. This module is also only activated during the training process and removed during the inference stage. This method uses singular value decomposition (SVD) to decompose the original features and the perturbed features. Through SVD, we can extract the main representations from the redundant high-dimensional features, which not only compresses the information volume, but more importantly, retains the feature parts that are crucial for the segmentation task, effectively eliminating noise and redundant information. Then, a covariance relationship matrix is constructed to reflect the internal dependence relationships between the various feature channels. By comparing the covariance matrices corresponding to the original features and the perturbed features, we can capture the sensitive changes introduced by the perturbation, that is, which channels have changed significantly after being perturbed, thereby revealing their key roles in cross-domain generalization.

[0072] Specifically, the method specifically includes:

[0073] Since the original features usually have redundant information, in order to extract their main representations, the present invention performs SVD decomposition on both sets of features, namely the original features and the perturbed features, to obtain:

[0074] {U F ,Σ F ,V F}=SVD(F), {U F * ,Σ F * ,V F *}=SVD(F HDS )

[0075] where, Σ F and Σ F * are the singular diagonal matrices of the original features and the perturbed features respectively, which reflect the importance and energy distribution of each channel. In order to further characterize the statistical relationships between the channels, we use the singular diagonal matrices to construct a covariance relationship matrix, defined as:

[0076] F cov =Σ F Σ F T , F cov * =Σ F * (Σ F* ) T

[0077] These two matrices respectively describe the main statistical dependencies within the original features and the perturbed features, and are important bases for subsequent sensitivity analysis.

[0078] To reveal the impact of perturbations on the internal structure of features, calculate the absolute difference between the covariance matrices before and after perturbation:

[0079] Δ = |F cov - F cov * |

[0080] where the absolute value operation is an element-wise operation, and each value in Δ intuitively reflects the change amplitude of the statistical relationship of the corresponding channel under perturbation. Obviously, a larger difference means that the channel is more sensitive to perturbations and may play a key role in cross-domain generalization.

[0081] Considering the existence of the inherent differences between the original features and the perturbed features, directly using Δ may be interfered by noise. Therefore, the present invention uses the K-means unsupervised clustering algorithm to classify the sensitivity indexes of all channels, and divides the channels into two categories: "highly sensitive" and "low sensitive". For the channels classified into the highly sensitive category in the clustering, construct a one-hot mask matrix M, where the corresponding positions are assigned 1 and the remaining positions are assigned 0. To take into account both the original feature information and the sensitive perturbation effect, further add the one-hot mask matrix M and the identity matrix I to obtain the modulation matrix:

[0082] M' = M + I

[0083] This modulation matrix not only retains the original covariance structure but also assigns higher weights to the sensitive channels, thereby enhancing the influence of these channels in the subsequent optimization process.

[0084] Specifically, after constructing the modulation matrix M', we hope to make the features after perturbation as consistent with the ideal state (i.e., the identity matrix) as possible on the sensitive channels through whitening operation. For this purpose, we define the whitening optimization loss function:

[0085] L W = E[||(F cov ⊙ M') - I|| F

[0086] where ⊙ represents element-wise multiplication, ||·|| F ​is the Frobenius norm. The design objective of this loss function is to amplify the perturbation effect of sensitive channels. By maximizing the loss, the differences of these channels in the feature space can be fully utilized, forcing the network to continuously adjust its parameters during training to adapt to this perturbation.

[0087] Specifically, deep neural networks often have strong memory perturbation and adversarial noise capabilities. When simply using the uncertainty feature perturbation mechanism to simulate the distribution of unseen domains, the model may quickly adapt to this perturbation and fall into an optimization bottleneck. To prevent the model from quickly adapting to a fixed perturbation pattern and falling into a local optimum, the present invention introduces an adversarial perturbation training mechanism. During the training process, by continuously generating challenging perturbation samples and dynamically updating the perturbation parameters using the whitening loss gradient, the model is forced to always be in a state of being "challenged". This adversarial training mechanism not only breaks the limitations of traditional fixed perturbation methods but also continuously stimulates the potential of the model, enabling it to maintain high robustness and generalization performance when facing data from unseen target domains.

[0088] During the training process, we dynamically update the domain noise parameters ∈1 and ∈2. The update strategy is based on the backpropagation of the whitening loss gradient and is achieved through the following formula:

[0089]

[0090] where α is a hyperparameter that controls the perturbation update step size. This update method actually uses gradient information to continuously adjust the domain noise, forcing the model to be continuously "interfered" during the prediction process, thereby preventing the model from quickly adapting to a fixed perturbation pattern and achieving a robust transfer from the I.I.D. (independent and identically distributed) source domain to the O.O.D.

[0091] (out-of-distribution) target domain.

[0092] The overall algorithm provided by the present invention is shown in Table 1.

[0093] Table 1

[0094]

[0095]

[0096] In one embodiment, a segmentation network is constructed based on the U-Net structure, and ResNet-34 is used as the backbone network. As Figure 1 shown, the training process of the medical image segmentation model includes the following steps:

[0097] S1. Collect image data and perform preprocessing to obtain a training set.

[0098] S2. Randomly select samples from the training set, input them into the feature extraction module to obtain feature F, and calculate the original statistics; the original statistics include the mean μ and the standard deviation σ.

[0099] S3. Input the original statistics into the mixture distribution sampling module for feature perturbation to obtain the perturbed feature F HDS 。

[0100] S4. Input feature F and the perturbed feature F HDS into the sensitive perturbation screening module to obtain the modulation matrix;

[0101] S5. Calculate the sensitive screening whitening loss through the modulation matrix and perform adversarial perturbation update;

[0102] Steps S2 - S5 use the domain generalization method based on adaptive adversarial perturbation to provide constraints for the network.

[0103] S6. After feature reassignment of the feature and the perturbed feature, input them into the decoder module to obtain the segmentation result; the Dice metric is used for consistency constraint.

[0104] S7. Repeat steps S2 - S6 for iterative training until the model parameters converge.

[0105] In the present invention, unless otherwise clearly specified and defined, terms such as "installation", "setting", "connection", "fixation", "rotation", etc. should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or integrated; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, and it can be the internal communication of two components or the interaction relationship between two components. Unless otherwise clearly defined, for those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0106] Although the embodiments of the present invention have been shown and described, for those of ordinary skill in the art, it can be understood that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A medical image segmentation method based on adaptive adversarial perturbation, characterized in that Construct and train a medical image segmentation model based on adaptive adversarial perturbation, and input the image to be processed into the trained medical image segmentation model to obtain the segmentation result; the medical image segmentation model includes a feature extraction module, a mixed distribution sampling module, a sensitive perturbation screening module, and a decoder module; The training process of the medical image segmentation model includes the following steps: S1. Collect image data and preprocess it to obtain a training set; S2. Randomly select samples from the training set and input them into the feature extraction module to obtain features F, and calculate the original statistics; the original statistics include the mean μ and the standard deviation σ; S3. Input the original statistic into the mixed distribution sampling module to obtain the perturbed feature F HDS ; S4. Input the feature F and the perturbation feature F HDS into the sensitive perturbation screening module to obtain the modulation matrix; S5. Calculate the sensitive screening whitening loss through the modulation matrix and perform adversarial perturbation update; S6. After feature reassignment of the features and the perturbed features, input them into the decoder module to obtain the segmentation result; S7. Repeat steps S2 - S6, and iteratively train until the model parameters converge.

2. The medical image segmentation method based on adaptive adversarial perturbation according to claim 1, characterized in that, Step S3 specifically includes: S31. Sample random statistics from two different probability distributions based on the mean μ and the standard deviation σ; the random statistics include the mean μ p1 , the standard deviation σ p2 , the mean μ p2 and the standard deviation σ p2 ; S32. Perform weighted fusion on the random statistics to obtain enhanced statistics, where the enhanced statistics include the mean μ aug , standard deviation σ aug , where λ1 ∈ [0, 1] represents the mixing coefficient; S33. Sample the probability parameter P from the Beta distribution, where P ~ Beta(α, α), and α is the sampling parameter; S34. Generate a Bernoulli distribution based on the probability parameter P and sample to obtain the Bernoulli parameter λ; S35. Obtain a mixed statistic based on the Bernoulli parameter λ, where the mixed statistic includes a mean μ mix , a standard deviation σ mix , where μ mix = λμ+(1 - λ)μ aug σ mix = λσ+(1 - λ)σ aug S36. Obtain the perturbation feature F based on the hybrid statistic HDS , where 3. The medical image segmentation method based on adaptive adversarial perturbation according to claim 2, wherein Step S31 uses the Logistic normal distribution and the uniform distribution.

4. A medical image segmentation method based on adaptive adversarial perturbation according to claim 1, characterized in that, Step S4 specifically includes: S41. Respectively perform singular value decomposition on the feature F and the perturbed feature F HDS to obtain the singular diagonal matrix Σ F and Σ F * ; S42. According to the singular diagonal matrix Σ F and Σ F * Construct the covariance matrix F cov and F cov * ; S43. Calculate the modulation matrix through the covariance matrices F cov and F cov * ​ 5. The medical image segmentation method based on adaptive adversarial perturbation according to claim 4, wherein, Step S43 specifically includes Through covariance matrix F cov and F cov * Calculate the absolute difference Δ = |F cov - F cov * |; Use the K - means unsupervised clustering algorithm to classify all channels, and divide the channels into two categories: high - sensitivity and low - sensitivity; Construct a one - hot mask matrix M, where the positions of the channels classified as high - sensitivity are assigned 1, and the rest are assigned 0; Add the one - hot mask matrix M to the identity matrix I to obtain the modulation matrix M'; 6. The medical image segmentation method based on adaptive adversarial perturbation according to claim 1, characterized in that, Sensitive Screening Albino Loss L W Expressed as L W = E[||(F cov ⊙M′)-I|| F ​ Among them, E[] represents the expectation, and F cov represents the covariance matrix obtained through the feature F; ⊙ represents the element-wise product, and ||·|| F is the Frobenius norm, M’ represents the modulation matrix, and I represents the identity matrix.

7. A medical image segmentation method based on adaptive adversarial perturbation according to claim 1, characterized in that, Step S5 includes: Dynamically update the domain noise parameters ∈1 and ∈2 during the training process. The update strategy is based on the backpropagation of the whitening loss gradient and is achieved through the following formula Among them, represents that the domain noise parameter ∈1 is the backpropagation gradient based on the whitening loss gradient, represents that the domain noise parameter ∈2 is the backpropagation gradient based on the whitening loss gradient, and α’ represents the hyperparameter.

Images