Single-source domain generalization medical image segmentation method based on domain randomization and related device

Abstract

The invention belongs to the technical field of image processing, and discloses a single-source domain generalization medical image segmentation method and related device based on domain randomization, and the method comprises the steps: inputting a to-be-segmented medical image into a pre-trained segmentation network model, and obtaining a segmentation result of the to-be-segmented medical image; the segmentation network model comprises a global intensity nonlinear filter, an encoder and a decoder; a uniform low-frequency spectrum transformation filter is arranged in the encoder and is used for performing mixing operation on low-frequency components of the spectrum of the input image by using Bernoulli distribution according to statistical magnitude sampled from uniform distribution to obtain a disturbance low-frequency component and combining the disturbance low-frequency component with high-frequency components of the spectrum of the input image to obtain an augmented spectrum; and the decoder is used for fusing the input image, the feature map of the input image, which passes through the downsampling of the convolutional layer, and the multi-resolution feature map from the encoder through step-by-step upsampling, and generating a pixel-by-pixel segmentation result of the input image through the convolutional layer and an activation function, thereby improving the image segmentation capability of an unknown domain.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and relates to a single-source domain generalization medical image segmentation method and related device based on domain randomization. Background Art

[0002] Medical image segmentation plays a crucial role in modern medicine, providing accurate support for the diagnosis and treatment planning of various diseases. In recent years, significant progress has been made in the application of deep learning in medical image segmentation, especially when the training and test data come from the same domain, that is, the training and test data distributions are consistent. However, in practical applications, deep learning models often perform poorly when the test data and training data come from different distributions, that is, different domains. This distribution difference between the training and test domains is called domain shift.

[0003] In medical image segmentation, the main source of domain shift is the difference in image acquisition, such as different imaging modalities, scanning protocols, or device manufacturers. The prior art faces the following challenges in dealing with the domain shift problem in medical image segmentation: one is that unsupervised domain adaptation methods cannot guarantee the performance on unknown target domains, especially the target domains not observed during training, which results in the unavailability of target domain data in real clinical environments; the other is that existing domain randomization methods mainly enhance the robustness of the model to domain shift through perturbations in the input space or feature space, but rely on linear filtering and statistical methods to generate perturbations, which may limit the perturbation search space, making it difficult to explore this space uniformly and resulting in limited perturbation diversity, thus unable to fully explore the more complex style spaces between different domains, thereby limiting the adaptability of the model to different domains.

[0004] Therefore, there is an urgent need to propose new methods to solve these technical challenges, especially how to enhance the robustness of the model to domain shift and improve its generalization ability on unknown target domains in a more flexible and diverse way. Summary of the Invention

[0005] The purpose of the present invention is to overcome the above-mentioned shortcomings of the prior art and provide a single-source domain generalization medical image segmentation method and related device based on domain randomization.

[0006] To achieve the above purpose, the present invention adopts the following technical solutions:

[0007] In a first aspect, the present invention provides a single-source domain generalized medical image segmentation method based on domain randomization, comprising: obtaining a medical image to be segmented; inputting the medical image to be segmented into a pre-trained segmentation network model to obtain a segmentation result of the medical image to be segmented; wherein the segmentation network model comprises a global intensity nonlinear filter, an encoder and a decoder connected in sequence; the encoder comprises a 2D discrete Fourier transform layer, a uniform low-frequency spectrum transform filter, a global filter layer and a 2D inverse Fourier transform layer connected in sequence; wherein the uniform low-frequency spectrum transform filter is used to perform a mixing operation on the low-frequency components of the spectrum of the input image using a Bernoulli distribution according to a statistic sampled from a uniform distribution to obtain a perturbed low-frequency component, and the perturbed low-frequency component is combined with the high-frequency components of the spectrum of the input image to obtain an augmented spectrum; the decoder is used to fuse the input image, the feature map of the input image downsampled by the convolution layer and the multi-resolution feature map from the encoder by stepwise upsampling, and pass through the convolution layer and the activation function to generate a pixel-by-pixel segmentation result of the input image.

[0008] Optionally, before inputting the medical image to be segmented into the pre-trained segmentation network model, the method further includes: preprocessing the medical image to be segmented as follows: reformatting the medical image to be segmented from 3D format to 2D and resizing it, then filling it along the channel dimension based on the requirements of the segmentation network model, and normalizing the pixel value distribution to zero mean and unit variance with a mean of 0 and a standard deviation of 1.

[0009] Optionally, the global intensity nonlinear filter includes four enhancement units and a renormalization unit connected in sequence; wherein the enhancement unit includes a random convolution layer and a Leaky ReLU layer connected in sequence; wherein each random convolution layer uses two kernels sampled from a Gaussian distribution with a mean of 0 and a standard deviation of 1; and the renormalization unit uses a random coefficient ω sampled from a uniform distribution U(0,1) for a renormalization operation.

[0010] Optionally, the uniform low-frequency spectrum transform filter is implemented as follows: a binary mask M is introduced to extract the low-frequency component F of the spectrum of the input image. l , and calculate F l The channel-level mean μ(F l ) and standard deviation σ(F l ), and randomly extracting new enhanced channel-level statistics from a uniform distribution, including the channel-level enhanced mean μ r and the enhanced standard deviation σ r ; μ r and μ(F l ) and σ r and σ(F l)Randomly mix to obtain mixed channel-level statistics; among them, the parameter λ of the random mixing is sampled from the Bernoulli distribution Bern(P): λ ∼ Bern(P), and the probability P is sampled from the beta distribution Beta(α, β): P ∼ Beta(α, β), where α and β are the shape parameters of the beta distribution and are set according to experience; reconstruct the low-frequency spectrum of the input image based on the mixed channel-level statistics, obtain the reconstructed spectrum and combine it with the high-frequency components of the spectrum of the input image to obtain the augmented spectrum.

[0011] Optionally, the backbone network of the encoder uses the hierarchical model GFNet-H-Ti of the global filtering network GFNet, and a uniform low-frequency spectrum transformation filter is inserted between the 2D discrete Fourier transform layer and the global filtering layer of GFNet-H-Ti.

[0012] Optionally, the decoder is implemented as follows: extract six feature maps: the input image, the feature map obtained by downsampling the input image through a convolutional layer with a stride of 2, and the feature maps output from the first stage, second stage, third stage, and fourth stage of the encoder; pass the six feature maps through a 3×3 convolutional layer and normalize them through a normalization layer, and then repeat the following steps until a feature map with the same resolution as the input image is obtained: concatenate the processed feature map of each layer with the feature map of the previous layer, then process it through a 3×3 convolutional layer, and upsample it through a transposed convolutional layer; pass the feature map with the same resolution as the input image through a 1×1 convolutional layer and a softmax activation function layer in sequence to generate a pixel-by-pixel class label prediction, and obtain the pixel-by-pixel segmentation result of the input image.

[0013] Optionally, the loss function in the pre-training process of the pre-trained segmentation network model adopts a comprehensive loss function obtained by combining the image segmentation loss and the consistency loss.

[0014] In the second aspect of the present invention, a single-source domain generalization medical image segmentation system based on domain randomization is provided, including: an image acquisition module for acquiring the medical image to be segmented; an image segmentation module for inputting the medical image to be segmented into a pre-trained segmentation network model to obtain a segmentation result of the medical image to be segmented; wherein, the segmentation network model includes a global intensity non-linear filter, an encoder and a decoder connected in sequence; the encoder includes a 2D discrete Fourier transform layer, a uniform low-frequency spectrum transformation filter, a global filtering layer and a 2D inverse Fourier transform layer connected in sequence; wherein, the uniform low-frequency spectrum transformation filter is used to perform a mixing operation on the low-frequency components of the spectrum of the input image using a Bernoulli distribution according to the statistics sampled from a uniform distribution to obtain perturbed low-frequency components, and combine the perturbed low-frequency components with the high-frequency components of the spectrum of the input image to obtain an augmented spectrum; the decoder is used to generate a pixel-by-pixel segmentation result of the input image by gradually upsampling and fusing the input image, the feature map obtained by downsampling the input image through a convolutional layer, and the multi-resolution feature maps from the encoder, and passing through a convolutional layer and an activation function.

[0015] In the third aspect of the present invention, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the above-mentioned single-source domain generalization medical image segmentation method based on domain randomization are implemented.

[0016] In the fourth aspect of the present invention, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned single-source domain generalization medical image segmentation method based on domain randomization are implemented.

[0017] Compared with the prior art, the present invention has the following beneficial effects:

[0018] The single-source domain generalization medical image segmentation method based on domain randomization of the present invention realizes the segmentation of the medical image to be segmented based on a pre-trained segmentation network model. Among them, a global intensity non-linear filter is set in the segmentation network model. In the input space, by introducing a global intensity non-linear enhancement technology, a shallow convolutional network with randomly sampled convolutions is adopted in each training iteration, enhancing the image diversity. A new uniform low-frequency spectrum transformation filter is designed in the segmentation network model. In the feature space, by re-sampling a new low-frequency spectrum from a uniform distribution to perturb the original intermediate value, the ability of the filter to fully simulate potential domain shifts during the training process is further improved, solving the feature space perturbation problem. By randomizing the local textures in the input space and the feature space to expand the perturbation search space, the perturbation diversity of single-domain generalization in medical image segmentation is enhanced, making the single-source domain generalization medical image segmentation method based on domain randomization of the present invention have better generalization and robustness in the segmentation of medical images with domain shift. Description of the Drawings

[0019] Figure 1 This is a flowchart of the single-source domain generalization medical image segmentation method based on domain randomization according to an embodiment of the present invention.

[0020] Figure 2 This is a schematic structural diagram of the segmentation network model according to an embodiment of the present invention.

[0021] Figure 3 This is a structural diagram of the global intensity non-linear filter module according to an embodiment of the present invention.

[0022] Figure 4 This is a structural diagram of the uniform low-frequency spectrum transformation filter module according to an embodiment of the present invention.

[0023] Figure 5 This is a comparison diagram of the prediction results of abdominal multi-organ segmentation by different methods according to an embodiment of the present invention.

[0024] Figure 6 This is a comparison diagram of the prediction results of prostate segmentation by different methods according to an embodiment of the present invention.

[0025] Figure 7 This is a block diagram of the structure of the single-source domain generalization medical image segmentation system based on domain randomization according to an embodiment of the present invention. Detailed Embodiments

[0026] In order to enable those skilled in the art to better understand the solution of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0027] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily need to be used to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present invention described here can be implemented in an order different from those illustrated or described here. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device including a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0028] The present invention will be further described in detail below with reference to the accompanying drawings:

[0029] See Figure 1 In one embodiment of the present invention, a single-source domain generalization medical image segmentation method based on domain randomization is provided, which can have better generalization and robustness in abdominal CT-MRI multi-organ segmentation and cross-center prostate segmentation tasks.

[0030] Specifically, the single-source domain generalization medical image segmentation method based on domain randomization of the present invention includes the following steps:

[0031] S1: Obtain the medical image to be segmented.

[0032] S2: Input the medical image to be segmented into a pre-trained segmentation network model to obtain the segmentation result of the medical image to be segmented.

[0033] Among them, see Figure 2 , the segmentation network model includes a global intensity non-linear filter, an encoder, and a decoder connected in sequence; the encoder includes a 2D discrete Fourier transform layer, a uniform low-frequency spectrum transformation filter, a global filtering layer, and a 2D inverse Fourier transform layer connected in sequence; wherein, the uniform low-frequency spectrum transformation filter is used to perform a mixing operation on the low-frequency components of the spectrum of the input image using a Bernoulli distribution according to the statistics sampled from a uniform distribution to obtain perturbed low-frequency components, and combine the perturbed low-frequency components with the high-frequency components of the spectrum of the input image to obtain an augmented spectrum; the decoder is used to generate the pixel-by-pixel segmentation result of the input image by gradually upsampling and fusing the input image, the feature map obtained by downsampling the input image through a convolutional layer, and the multi-resolution feature maps from the encoder, and passing through a convolutional layer and an activation function.

[0034] The single-source domain generalization medical image segmentation method based on domain randomization of the present invention realizes the segmentation of the medical image to be segmented based on a pre-trained segmentation network model. Among them, a global intensity non-linear filter is set in the segmentation network model. In the input space, by introducing a global intensity non-linear enhancement technique and adopting a shallow convolutional network with randomly sampled convolutions in each training iteration, the image diversity is enhanced. A new uniform low-frequency spectrum transformation filter is designed in the segmentation network model. In the feature space, by resampling a new low-frequency spectrum from a uniform distribution to perturb the original intermediate values, the ability of the filter to fully simulate potential domain shifts during the training process is further improved, and the feature space perturbation problem is solved. By randomizing the local textures in the input space and the feature space to expand the perturbation search space, the perturbation diversity of single-domain generalization in medical image segmentation is enhanced, making the single-source domain generalization medical image segmentation method based on domain randomization of the present invention have better generalization and robustness in the segmentation of domain-shifted medical images.

[0035] In a possible implementation, before inputting the medical image to be segmented into the pre-trained segmentation network model, the following preprocessing is further included: preprocessing the medical image to be segmented as follows: reformatting the medical image to be segmented from 3D format to 2D and resizing it, then padding along the channel dimension based on the requirements of the segmentation network model, and performing normalization with a mean of 0 and a standard deviation of 1 for the pixel value distribution, i.e., zero-mean and unit-variance normalization.

[0036] In this implementation, the image is reformatted from 3D format to 2D and resized to 192×192, and then padded along the channel dimension to meet the requirements of the segmentation network model. For the abdominal CT dataset, the CT values within the window range of [-125, 275] are selected. For all MRI images, the first 0.5% of the histogram is intercepted in the present invention. In addition, all three-dimensional scan images are normalized so that the mean of the pixel value distribution is 0 and the standard deviation is 1, thereby achieving the normalization effect of zero-mean and unit-variance.

[0037] Exemplarily, in this implementation, the abdominal CT-MRI multi-organ segmentation dataset and the cross-center prostate segmentation dataset are used for training and testing. For the prostate segmentation task, the present invention designates one domain as the source and the remaining five domains as the targets, and conducts a 1:5 experiment for each of the six domains.

[0038] In a possible implementation, the global intensity non-linear filter includes four enhancement units and one re-normalization unit connected in sequence; wherein, each enhancement unit includes a random convolutional layer and a Leaky ReLU layer connected in sequence; wherein, each random convolutional layer utilizes two kernels sampled from a Gaussian distribution with a mean of 0 and a standard deviation of 1; the re-normalization unit performs a re-normalization operation using a random coefficient ω sampled from a uniform distribution U(0,1).

[0039] Explanatorily, in order to diversify the search space of perturbations in the input space, the present invention introduces a global intensity non-linear enhancement technique to convert the input training images into images with different appearances while keeping the anatomical structure shape unchanged. It is located at the beginning of the segmentation network model and can enhance the training images before feature extraction.

[0040] See Figure 3The implementation of the Global Intensity Non-linear (Gin filter) is as follows: Taking the training image from the source domain as input, four random convolutional layers are alternated with Leaky ReLU layers to effectively transform the image intensity and texture, where each random convolutional layer uses two kernels sampled from a Gaussian distribution of μ = 0 and σ = 1. Subsequently, in the Gin filter, a renormalization operation is performed using a random coefficient ω sampled from a uniform distribution U(0,1) to constrain the energy of the augmented image and output an image with the same shape information but different intensity and texture.

[0041] In a possible implementation, the uniform low-frequency spectrum transform filter is implemented as follows: a binary mask M is introduced to extract the low-frequency component F of the spectrum of the input image. l , and calculate F l The channel-level mean μ(F l ) and standard deviation σ(F l ), and randomly extracting new enhanced channel-level statistics from a uniform distribution, including the channel-level enhanced mean μ r and the enhanced standard deviation σ r ; μ r and μ(F l ) and σ r and σ(F l ) are randomly mixed to obtain a mixed channel-level statistic; wherein, the parameter λ of the random mixing is sampled from the Bernoulli distribution Bern(P): λ~Ber(P), and the probability P is sampled from the Beta distribution Beta(α,β) P~Beta(α,β), α and β are shape parameters of the Beta distribution, which are set according to experience; the low-frequency spectrum of the input image is reconstructed according to the mixed channel-level statistics to obtain the reconstructed spectrum and combined with the high-frequency component of the spectrum of the input image to obtain the augmented spectrum.

[0042] Explanatory, high-frequency components retain more global features (such as shape), which are domain-invariant, while low-frequency components contain more local features (such as texture), which are domain-specific. Therefore, it is reasonable to design a model that emphasizes high-frequency components by disturbing low-frequency components. The Uniform Low Frequencyspectrum Transform (Uloft filter) is designed to maximize the search space for feature perturbation in the low-frequency spectrum, thus more evenly covering the offsets of different domains. Specifically, it enhances and perturbs low-frequency features by constructing a uniform distribution of new channel-level statistics. The input of the Uloft filter is the spectrum of the input image features after 2D discrete Fourier transform, and the output is the augmented spectrum after perturbation and enhancement. The Uloft filter resamples new augmented statistics from a uniform distribution in the low-frequency spectrum to perturb the original spectrum, further improving the filter's ability to fully simulate potential domain shifts during training. The Uloft filter is the most core and innovative part of the segmentation network model.

[0043] Specifically, see Figure 4 , the implementation method of the Uniform Low Frequencyspectrum Transform (Uloft filter) is as follows: First, introduce a binary mask M to extract the low-frequency component F l and high-frequency component F h :

[0044] F l (x) = M ⊙ F(x)

[0045] F h (x) = (1 - M) ⊙ F(x)

[0046] Next, calculate the channel-level mean μ(F l ) and standard deviation σ(F l ): l

[0047]

[0048] where H and W represent the height and width of the image.

[0049] Subsequently, randomly draw new augmented channel-level statistics μ r and σ r , and randomly mix the new augmented channel-level statistics μ r and σ r with the original statistics μ(F l ) and σ(F l ):

[0050] μ mix (F l ) = λμ r +(1 - λ)μ(F l )

[0051] σ mix (F l ) = λσ r +(1 - λ)σ(F l )

[0052] Among them, the parameter λ is sampled from the Bernoulli distribution Bern(P): λ ~ Bern(P), and the probability P is sampled from the Beta distribution Beta(α, β): p ~ Beta(α, β), where α and β are the shape parameters of the Beta distribution and are empirically set to 0.1.

[0053] Finally, the low-frequency spectrum is reconstructed using the mixed channel feature statistics:

[0054]

[0055] Finally, the low-frequency spectrum is reconstructed using the mixed channel feature statistics, and this low-frequency spectrum is combined with the original high-frequency component F h to form the augmented spectrum of the input features as shown in the following formula:

[0056]

[0057] Augmented spectrum is used for the subsequent 2D inverse Fourier transform.

[0058] By introducing the Uloft filter, the segmentation network model comprehensively and randomly perturbs the low-frequency features in the entire feature space, further improving the robustness of the model for solving domain generalization tasks.

[0059] In a possible implementation, the backbone network of the encoder uses the hierarchical model GFNet-H-Ti of the global filtering network GFNet, and a uniform low-frequency spectrum transformation filter is inserted between the 2D discrete Fourier transform layer and the global filtering layer of GFNet-H-Ti.

[0060] Explanatorily, GFNet is used as the encoder in this implementation, which includes three key parts: the 2D discrete Fourier transform layer, the global filtering layer, and the 2D inverse Fourier transform layer. And the Uloft filter is inserted into the GFNet encoder, between the 2D discrete Fourier transform layer and the global filtering layer, to perturb the domain source data during training, thereby enhancing the diversity of the feature space. In the specific implementation, the hierarchical model GFNet-H-Ti of GFNet is used as the backbone network of the encoder.

[0061] Specifically, first, the image enhanced by the Gin filter is taken as the input and fed into the 2D discrete Fourier transform layer. The function of this layer is to transform the image from the spatial domain to the frequency domain, so as to process the low-frequency and high-frequency features in the frequency domain. The output of this layer is the representation of the image in the frequency domain, containing the spectral information of the image, especially the features of the low-frequency part. Then, this output enters the Uloft filter to perturb the low-frequency information in the frequency domain. Specifically, the Uloft filter will randomly mix the low-frequency features by sampling new statistics from a uniform distribution and then performing a mixing operation using a Bernoulli distribution. After the perturbed low-frequency features are combined with the original high-frequency features, they are then subjected to element-wise multiplication between the frequency-domain features and the global filter through a learnable global filtering layer to achieve spatial information interaction in the frequency domain. Finally, it is transformed back to the spatial domain through the 2D inverse Fourier transform layer to restore the perturbed spatial features for downstream tasks to use.

[0062] In a possible implementation manner, the decoder is implemented as follows: Six feature maps are extracted: the input image, the feature map obtained by downsampling the input image through a convolutional layer with a stride of 2, and the feature maps output from the first stage, second stage, third stage, and fourth stage of the encoder. The six feature maps are passed through a 3×3 convolutional layer and normalized through a normalization layer, and then the following steps are repeated until a feature map with the same resolution as the input image is obtained: The processed feature map of each layer is concatenated with the feature map of the previous layer, then processed through a 3×3 convolutional layer, and upsampled through a deconvolution layer. The feature map with the same resolution as the input image is sequentially passed through a 1×1 convolutional layer and a softmax activation function layer to generate a per-pixel class label prediction, obtaining the per-pixel segmentation result of the input image.

[0063] Explanatorily, the enhanced input image is obtained through the Gin filter, and the global filtering network GFNet is used as the encoder, combined with the Uloft filter to perturb the low-frequency features in the frequency domain to enhance the diversity of the feature space. The decoder restores the spatial resolution by gradually upsampling and concatenating multi-resolution feature maps, and finally, through the convolutional layer and the softmax activation function, generates a per-pixel semantic segmentation prediction to achieve segmentation.

[0064] The implementation method of the decoder is as follows: The design of the decoder is similar to the classic U-Net structure and is a typical "U-shaped" structure. It realizes high-resolution segmentation prediction by gradually upsampling and fusing multi-resolution feature maps from the encoder. The decoder plays a role in restoring the spatial resolution of the feature map and finally generates the segmentation result. Specifically, the present invention uses a 4×4 patch embedding method to form tokens and processes these tokens using non-overlapping convolutional layers. The present invention extracts six feature maps, including the input image, the feature map obtained by downsampling the input image through a convolutional layer with a stride of 2, and the feature maps output from the first stage, the second stage, the third stage, and the fourth stage of the encoder. These feature maps will be processed through a series of 3×3 convolutional layers to keep the size unchanged and be normalized through a normalization layer. The processed feature maps will be concatenated with the feature maps of the previous layer and then enter an additional 3×3 convolutional layer, and then be upsampled through a transposed convolutional layer. This process will be repeated until the resolution of the original input is restored. Finally, the output will pass through a 1×1 convolutional layer and use the softmax activation function to generate pixel-wise semantic predictions, outputting the class label of each pixel, thereby completing the segmentation task.

[0065] In a possible implementation manner, the loss function in the pre-training process of the pre-trained segmentation network model adopts a comprehensive loss function obtained by combining an image segmentation loss and a consistency loss.

[0066] Explanatorily, in terms of the domain generalization segmentation task, the segmentation network combines the image segmentation loss and the consistency loss to supervise the final segmentation result. The image segmentation loss includes the Dice Similarity Coefficient (DSC) loss and the cross-entropy loss, which consider the similarity between the segmentation result and the pixel level. The consistency loss is measured using the Kullback-Leibler divergence and encourages consistent network predictions for the same input image enhanced with different Gin filters and Uloft filters. The supervision of the domain generalization segmentation task is based on the features extracted from the encoder in the segmentation network because the performance of the encoder directly affects the final segmentation effect. Finally, the above two losses are combined to form a comprehensive total loss function.

[0067] Specifically, by comprehensively optimizing multiple losses during the training process, the segmentation network model can more comprehensively learn domain-invariant representations during the training process, improving the understanding ability and segmentation performance for target domain images.

[0068] Generally speaking, the single-source domain generalization medical image segmentation method based on domain randomization of the present invention first performs image preprocessing based on medical knowledge, obtains the enhanced input image through the Gin filter, and uses the global filtering network GFNet as the encoder, and combines the Uloft filter to perturb the low-frequency features in the frequency domain to enhance the diversity of the feature space. The decoder restores the spatial resolution by gradually upsampling and splicing multi-resolution feature maps, and finally generates pixel-by-pixel semantic segmentation predictions through the convolutional layer and the softmax activation function to achieve segmentation. These solutions jointly construct a segmentation network model by combining consistency constraint regularization and image segmentation loss supervision training, aiming to improve the accuracy of target domain image segmentation.

[0069] In a possible implementation manner, the single-source domain generalization medical image segmentation method based on domain randomization of the present invention can effectively enhance the robustness of unknown domain medical image segmentation. The overall framework includes the collaborative effects of modules such as the encoder based on GFNet, the Gin filter, and the Uloft filter, enabling the network to better perturb the domain features of medical images in the input space and the feature space, and improving the segmentation performance of unknown target domain images. In order to verify the domain generalization segmentation performance of the present invention and the functions of each part, the data of two datasets were preprocessed. The present invention is implemented based on the Pytorch framework, and the network is trained using the collected data. Compared with other methods in the same field, this method has improved in various performance indicators, and the functions of each module in the invention are verified through ablation experiments. The following is a detailed description in combination with cases:

[0070] First, data preparation: To verify the feasibility of the method of the present invention, the method was trained and evaluated on two datasets, namely the abdominal CT-MRI multi-organ segmentation dataset and the cross-center prostate segmentation dataset, using two different experimental settings. In the first setting, the generality of the method was tested using the abdominal CT-MRI multi-organ segmentation dataset. For the abdominal multi-organ segmentation task, abdominal CT scans were used as the source domain, which was divided into 70% for training, 10% for validation, and 20% for testing. Abdominal T2-SPIR MRI scans were used as the target domain, and all available images were specifically used for testing. In the second setting, for the prostate segmentation task, one domain was designated as the source, and the remaining five domains were used as the targets, and a 1-to-5 experiment was conducted for each of the six domains. Consistent with the method of abdominal segmentation, the source domain was divided into 70% for training, 10% for validation, and 20% for testing, while all target domain images were specifically used for testing. All datasets were reformatted into 2D, resized to 192×192, and padded along the channel dimension to meet the network requirements. For the abdominal CT dataset, the window range was [-125, 275] CT values. For all MRI images, the first 0.5% of the histogram was intercepted. In addition, all 3D scans were normalized to zero mean and unit variance.

[0071] Second, experimental implementation details and evaluation criteria: The method of the present invention was implemented on the open-source framework Pytorch. This experiment was carried out using an NVIDIA GeForce RTX A800 GPU. In the experiment, to better balance the loss function, for the abdominal CT-MRI multi-organ segmentation dataset, the coefficients α DSC , and α kl corresponding to the DSC loss, cross-entropy loss, and Kullback-Leibler divergence loss were set to 3, 1, and 5 respectively. For the cross-center prostate segmentation dataset, α DSC , α ce , and α kl were set to 2, 1, and 5 respectively. The segmentation network model adopted a dual learning rate strategy. The initial learning rate of the encoder was set to 0.0001, and the initial learning rate of the decoder was set to 0.001. The overall optimizer used was AdamW, which added parameter regularization on the basis of Adam, and the weight decay coefficient was 5e-5. After 500 times, the learning rate decayed to 0, and the optimizer parameters β 1 and β 2They are 0.9 and 0.999 respectively. For the specific implementation details of the filter, in the Uloft filter, for the abdominal CT-MRI multi-organ segmentation dataset, the ratio r of the binary mask M is set to 0.5, and for the cross-center prostate segmentation dataset, it is set to 0.25. Considering that the standard GFNet architecture consists of four core MLP blocks (denoted as Block 1 to Block 4), the Uloft filter is inserted into the first block to train this method.

[0072] Third, comparative experiments: To demonstrate the domain generalization segmentation performance of the method of the present invention, the same dataset will be used to compare the method of the present invention with other existing methods. Among them, the network is trained on the source domain and evaluated on the target domain. The method of the present invention is evaluated with the UNet benchmark model and several state-of-the-art single-source domain generalization methods. For example, the Cutout method simulates image corruption by deliberately removing some regions from the training images, thereby enhancing the robustness of the model. The RSC method identifies the features that produce the largest gradients and considers these features not to be robust, so they are removed during the training process. MixStyle generates new domains by mixing instance-level feature statistics. AdvBias enhances the anti-interference ability of the model by performing adversarial perturbations on the images. RandConv synthesizes new domains through random linear intensity transformations. Finally, CSDG proposes a data augmentation method inspired by causality, which enhances the robustness of the model by exposing the synthetic domain transfer training examples to the segmentation model. It should be noted that all the comparison methods are implemented on the UNet segmentation network. As shown in Tables 1 and 2, where Table 1 is the abdominal CT-MRI multi-organ segmentation dataset and Table 2 is the cross-center prostate segmentation dataset, it can be seen that this method achieves the highest score on the DSC.

[0073] Table 1

[0074]

[0075]

[0076] Table 2

[0077]

[0078] It can be seen that the method of the present invention is always superior to similar methods in the unseen domain. The average DSC score on the abdominal CT-MRI multi-organ segmentation dataset is 87.96%, and the average DSC score on the cross-center prostate segmentation dataset is 75.91%. Among them, the UNet method minimizes the loss of source domain training data, thus generating features that are highly dependent on the source domain. When applied to the unseen target domain, this dependency usually leads to poor performance. As shown in Tables 1 and 2, the UNet method performs poorly on the two datasets of abdominal multi-organ segmentation and prostate unseen target domain segmentation. In contrast, the Cutout method enhances model generalization by removing parts of the image. However, if the removed area contains key segmentation information, it may reduce the generalization performance of the model in other domains, resulting in poor DSC metric scores for the Cutout method in Tables 1 and 2. In addition, the RSC method aims to improve cross-domain generalization by zeroing out features with large gradient values, encouraging the model to focus on general representations. Although promoting a broader set of features, RSC still struggles with significant distribution changes between the source and target domains. For example, as shown in Table 1, it achieved the worst results on the abdominal multi-organ segmentation task using the abdominal CT-MRI multi-organ segmentation dataset. Similarly, Table 2 shows suboptimal performance on the prostate segmentation task using the cross-center prostate segmentation dataset. The MixStyle method creates new domain instances by mixing the feature statistics of two instances. However, this feature mixing may not fully capture general representations. As shown in Tables 1 and 2, the results of the MixStyle method are worse compared to the present method. For MRI-based medical image segmentation, the AdvBias method generates adversarial data by simulating the intensity inhomogeneity caused by the bias field in MR imaging. However, simply simulating the bias field to increase domain data does not address the broader distribution differences between the source and target domains. Therefore, as shown in Tables 1 and 2, the AdvBias method is ineffective for cross-domain medical image segmentation. At the same time, although the RandConv and CSDG methods enhance the source domain data by integrating different styles, these methods only randomize the styles within the input space. Compared with RandConv and CSDG, the performance improvement of the present method indicates that the method can better simulate domain transfer, thus achieving greater robustness in the unseen domain.

[0079] In addition, see Figure 5 and Figure 6, it shows the visual evaluation of the segmentation results of samples from these unknown target domains, where GT represents the ground truth label. For the abdominal CT-MRI multi-organ segmentation dataset, 3 T2-SPIR MRI slices from the unseen target domain were used to evaluate the inductive ability when transitioning from the CT modality to the MRI modality, and the abdominal multi-organ segmentation results were overlaid on the input images for clearer visualization and comparison. Similarly, for the cross-center prostate segmentation dataset, three T2-weighted MRI slices from the unseen target domain were used to evaluate the generalization ability when transferring from site B to site C, and the prostate segmentation results were overlaid on the input images to enhance transparency and comparison.

[0080] Fourth, ablation experiments: The contributions of the Gin filter, Uloft filter, and consistency constraint, as well as the different insertion positions of the Uloft filter, to the segmentation performance of the unseen target domains on the two datasets were evaluated. The Gin filter or Uloft filter was integrated into the ERM baseline method, or the Gin filter (Gin-CC) or Uloft (Uloft-CC) filter was combined with the consistency constraint.

[0081] See Tables 3 and 4, which present the segmentation results of the ablation experiments of the Gin filter, Uloft filter, and consistency constraint, illustrating the effects of the different components of the method of the present invention under the consistent settings of the unseen target domains of the two datasets. As shown in Tables 3 and 4, compared with all other methods, the performance of the ERM method is poor. In contrast, the results of the Gin-Only and Uloft-Only methods are improved compared to the ERM method. Specifically, the Gin filter aims to randomize the local texture in the input space by synthesizing different styles without losing shape information. Therefore, the Gin-Only method performs slightly better than the ERM method in the unseen target domains of the two datasets. In addition, the introduction of the Uloft filter enhances the segmentation performance, indicating that randomizing the local texture in the low-frequency spectrum of the feature space is beneficial to improving the segmentation of the two datasets in the unseen target domains. Furthermore, the Gin-CC and Uloft-CC methods that combine the consistency constraint learn representations that are not affected by texture changes in the input space or feature space, and their effects are better than those of the Gin-Only and Uloft-Only methods. This shows that the consistency constraint improves the segmentation performance of the two datasets in the unknown target domains.

[0082] Table 3

[0083]

[0084]

[0085] Table 4

[0086]

[0087] Referring to Tables 5 and 6, the experimental results of ablation experiments with different insertion positions of the Uloft filter are given. Considering that the segmentation network with the standard GFNet-H-Ti network has four core MLP blocks, namely blocks 1 to 4, in this experiment, the Uloft filter is inserted on different blocks to train different models. Specifically, the number of blocks with the Uloft filter inserted is gradually increased from block 1 to block 4 to verify the influence of different insertion positions of the Uloft filter. As shown in Tables 5 and 6, no matter where the Uloft filter is inserted, the model always obtains higher performance than the ERM baseline method in the unseen target domains of the two datasets. The results show that inserting the module of block 1 in GFNet-H-Ti has the best performance, indicating that increasing the frequency diversity of block 1 will obtain the best generalization ability. Based on the analysis, in all experiments of the present invention, the Uloft filter is inserted into block 1.

[0088] Table 5

[0089]

[0090] Table 6

[0091]

[0092] The single-source domain generalization medical image segmentation method based on domain randomization of the present invention has the advantage of proposing a random domain generalization medical image segmentation method, which perturbs the style of source domain data by randomizing local textures in the input and feature spaces to enhance the diversity of perturbations in the consistency constraint framework. In the input space, by introducing a global intensity nonlinear enhancement technique and using a shallow convolutional network, random convolutions are sampled during each training iteration, enhancing image diversity. In the feature space, a new uniform low-frequency spectrum transformation filter is designed, which perturbs the original spectrum by resampling a new low-frequency spectrum from a uniform distribution, further improving the ability of the filter to fully simulate potential domain shifts during training. At the same time, a consistency loss is used to encourage consistent network predictions for the same image with different input space and feature space perturbations, in order to learn a representation invariant to texture changes. Therefore, the present invention has better generalization and robustness in abdominal CT-MRI multi-organ segmentation and cross-center prostate segmentation tasks.

[0093] The following is an apparatus embodiment of the present invention, which can be used to execute the method embodiment of the present invention. For details not disclosed in the apparatus embodiment, please refer to the method embodiment of the present invention.

[0094] See Figure 7, in another embodiment of the present invention, a single-source domain generalization medical image segmentation system based on domain randomization is provided, which can be used to implement the above-mentioned single-source domain generalization medical image segmentation method based on domain randomization. Specifically, the single-source domain generalization medical image segmentation system based on domain randomization includes an image acquisition module and an image segmentation module.

[0095] Among them, the image acquisition module is used to acquire the medical image to be segmented; the image segmentation module is used to input the medical image to be segmented into a pre-trained segmentation network model to obtain the segmentation result of the medical image to be segmented; among them, the segmentation network model includes a global intensity non-linear filter, an encoder and a decoder connected in sequence; the encoder includes a 2D discrete Fourier transform layer, a uniform low-frequency spectrum transformation filter, a global filtering layer and a 2D inverse Fourier transform layer connected in sequence; among them, the uniform low-frequency spectrum transformation filter is used to perform a mixing operation on the low-frequency components of the spectrum of the input image using a Bernoulli distribution according to the statistics sampled from a uniform distribution to obtain perturbed low-frequency components, and combine the perturbed low-frequency components with the high-frequency components of the spectrum of the input image to obtain an augmented spectrum; the decoder is used to generate the pixel-by-pixel segmentation result of the input image by gradually upsampling and fusing the input image, the feature map obtained by downsampling the input image through a convolutional layer, and the multi-resolution feature maps from the encoder, and passing through a convolutional layer and an activation function.

[0096] All the relevant contents of each step involved in the embodiment of the above-mentioned single-source domain generalization medical image segmentation method based on domain randomization can be cited in the function description of the corresponding functional modules of the single-source domain generalization medical image segmentation system based on domain randomization in the embodiment of the present invention, and will not be elaborated here.

[0097] The division of modules in the embodiments of the present invention is illustrative, only a logical function division. In actual implementation, there may be other division methods. In addition, in each embodiment of the present invention, the functional modules can be integrated in one processor, or can exist physically separately, or two or more modules can be integrated in one module. The above-mentioned integrated modules can be implemented in the form of hardware or in the form of software functional modules.

[0098] In another embodiment of the present invention, a computer device is provided. The computer device includes a processor and a memory. The memory is used to store a computer program, and the computer program includes program instructions. The processor is used to execute the program instructions stored in the computer storage medium. The processor may be a Central Processing Unit (CPU), or may also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, and is suitable for implementing one or more instructions. Specifically, it is suitable for loading and executing one or more instructions in the computer storage medium to implement the corresponding method flow or corresponding function. The processor described in the embodiment of the present invention can be used for the operation of the single-source domain generalization medical image segmentation method based on domain randomization.

[0099] In another embodiment of the present invention, a storage medium is also provided, specifically a computer-readable storage medium (Memory). The computer-readable storage medium is a memory device in the computer device and is used to store programs and data. It can be understood that the computer-readable storage medium here can include both the built-in storage medium in the computer device and, of course, the extended storage medium supported by the computer device. The computer-readable storage medium provides a storage space, and the operating system of the terminal is stored in this storage space. And in this storage space, one or more instructions suitable for being loaded and executed by the processor are also stored. These instructions can be one or more computer programs (including program codes). It should be noted that the computer-readable storage medium here can be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory. The one or more instructions stored in the computer-readable storage medium can be loaded and executed by the processor to implement the corresponding steps of the single-source domain generalization medical image segmentation method based on domain randomization in the above embodiment.

[0100] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0101] The present invention is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the present invention. It should be understood that each flow and / or block in the flowchart and / or block diagram, as well as the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0102] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including instruction means that implement the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0103] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are performed on the computer or other programmable device to generate a computer-implemented process. Thus, the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0104] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the above embodiments, those of ordinary skill in the art should understand that: modifications or equivalent replacements can still be made to the specific embodiments of the present invention. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention shall be covered by the protection scope of the claims of the present invention.

Claims

1. A single-source domain generalized medical image segmentation method based on domain randomization, characterized in that: include: Acquire a medical image to be segmented; Inputting the medical image to be segmented into a pre-trained segmentation network model to obtain a segmentation result of the medical image to be segmented; Among them, the segmentation network model includes a global intensity nonlinear filter, an encoder and a decoder connected in sequence; the encoder includes a 2D discrete Fourier transform layer, a uniform low-frequency spectrum transform filter, a global filter layer and a 2D inverse Fourier transform layer connected in sequence; wherein the uniform low-frequency spectrum transform filter is used to mix the low-frequency components of the spectrum of the input image using Bernoulli distribution according to the statistics sampled from the uniform distribution to obtain the perturbed low-frequency components, and the perturbed low-frequency components are combined with the high-frequency components of the spectrum of the input image to obtain the augmented spectrum; the decoder is used to fuse the input image, the feature map of the input image downsampled by the convolution layer, and the multi-resolution feature map from the encoder by stepwise upsampling, and pass through the convolution layer and the activation function to generate a pixel-by-pixel segmentation result of the input image.

2. The single-source domain generalized medical image segmentation method based on domain randomization according to claim 1, characterized in that: Before inputting the medical image to be segmented into the pre-trained segmentation network model, the method further includes: pre-processing the medical image to be segmented as follows: The medical image to be segmented is reformatted from 3D format to 2D and resized, and then padded along the channel dimension based on the requirements of the segmentation network model, and the pixel value distribution is normalized to zero mean and unit variance with a mean of 0 and a standard deviation of 1.

3. The single-source domain generalized medical image segmentation method based on domain randomization according to claim 1, characterized in that: The global intensity nonlinear filter includes four enhancement units and a renormalization unit connected in sequence; wherein the enhancement unit includes a random convolution layer and a leaky ReLU layer connected in sequence; wherein each random convolution layer uses two kernels sampled from a Gaussian distribution with a mean of 0 and a standard deviation of 1; and the renormalization unit uses a random coefficient ω sampled from a uniform distribution U(0,1) for renormalization operation.

4. The single-source domain generalized medical image segmentation method based on domain randomization according to claim 1, characterized in that: The implementation method of the uniform low-frequency spectrum conversion filter is as follows: A binary mask M is introduced to extract the low-frequency component F of the spectrum of the input image l , and calculate F l The channel-level mean μ(F l ) and standard deviation σ(F l ), and randomly extracting new enhanced channel-level statistics from a uniform distribution, including the channel-level enhanced mean μ r and the enhanced standard deviation σ r ; μ r and μ(F l ) and σ r and σ(F l ) Random mixing obtains the mixed channel-level statistics; wherein, the parameter λ of the random mixing is sampled from the Bernoulli distribution Bern(P): λ~Bern(P), and the probability P is sampled from the Beta distribution Beta(α,β): P~Beta(α,β), where α and β are the shape parameters of the Beta distribution, which are set according to experience; A low-frequency spectrum of the input image is reconstructed according to the mixed channel-level statistics to obtain a reconstructed spectrum, which is combined with the high-frequency components of the spectrum of the input image to obtain an augmented spectrum.

5. The single-source domain generalized medical image segmentation method based on domain randomization according to claim 1, characterized in that: The backbone network of the encoder uses a hierarchical model GFNet-H-Ti of the global filter network GFNet, and inserts a uniform low-frequency spectrum transform filter between the 2D discrete Fourier transform layer and the global filter layer of GFNet-H-Ti.

6. The single-source domain generalized medical image segmentation method based on domain randomization according to claim 1, characterized in that: The implementation method of the decoder is as follows: Extract six feature maps: the input image, the feature map of the input image downsampled by a convolutional layer with a stride of 2, and the feature maps from the first, second, third, and fourth stage outputs of the encoder; The six feature maps are processed through a 3×3 convolution layer and normalized through a normalization layer, and then the following steps are repeated to obtain a feature map with the same resolution as the input image: the feature map after each layer is concatenated with the feature map of the previous layer, and then processed through a 3×3 convolution layer and upsampled through a deconvolution layer; The feature map with the same resolution as the input image is passed through a 1×1 convolution layer and a softmax activation function layer in sequence to generate pixel-by-pixel category label predictions and obtain the pixel-by-pixel segmentation result of the input image.

7. The single-source domain generalized medical image segmentation method based on domain randomization according to claim 1, characterized in that: The loss function of the pre-trained segmentation network model in the pre-training process adopts a comprehensive loss function obtained by combining image segmentation loss and consistency loss.

8. A single-source domain generalized medical image segmentation system based on domain randomization, characterized in that: include: An image acquisition module, used for acquiring the medical image to be segmented; An image segmentation module is used to input the medical image to be segmented into a pre-trained segmentation network model to obtain a segmentation result of the medical image to be segmented; Among them, the segmentation network model includes a global intensity nonlinear filter, an encoder and a decoder connected in sequence; the encoder includes a 2D discrete Fourier transform layer, a uniform low-frequency spectrum transform filter, a global filter layer and a 2D inverse Fourier transform layer connected in sequence; wherein the uniform low-frequency spectrum transform filter is used to mix the low-frequency components of the spectrum of the input image using Bernoulli distribution according to the statistics sampled from the uniform distribution to obtain the perturbed low-frequency components, and the perturbed low-frequency components are combined with the high-frequency components of the spectrum of the input image to obtain the augmented spectrum; the decoder is used to fuse the input image, the feature map of the input image downsampled by the convolution layer, and the multi-resolution feature map from the encoder by stepwise upsampling, and pass through the convolution layer and the activation function to generate a pixel-by-pixel segmentation result of the input image.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the single-source domain generalized medical image segmentation method based on domain randomization are implemented as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the single-source domain generalized medical image segmentation method based on domain randomization are implemented as described in any one of claims 1 to 7.

Images