Single-source-domain generalization medical image segmentation method and device based on shape dictionary

Abstract

The invention discloses a single-source-domain generalization medical image segmentation method and device based on a shape dictionary, and relates to the technical field of computers, and the method comprises a shape dictionary learning step, an overall segmentation network construction step, an overall segmentation network training step and a medical image segmentation step. According to the invention, by combining a domain invariant shape reference segmentation model and random feature enhanced domain specific generalization information learning, on the premise of single source domain data training, the learning ability of a network model to domain invariant features and simulated unknown domain specific features is improved; and the generalization ability and segmentation accuracy of the model in various unknown target domains are improved to the greatest extent. According to the method provided by the invention, only single-source-domain data is needed, and the problem of single-source-domain generalization SDG of a medical image segmentation task can be well relieved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a single-source domain generalized medical image segmentation method and device based on a shape dictionary. Background Art

[0002] Most existing medical image methods require access to target domain data. However, in the single-source domain generalization SDG problem, target domain data cannot be accessed, and only a single source domain dataset can be used. Existing research shows that it is necessary to enhance generalization capabilities by learning domain-invariant features and reducing the influence of domain-specific features. Currently, most SDG methods use generative adversarial data augmentation methods to synthesize fictitious samples from single-domain datasets to simulate the data distribution of unknown domains. However, in the field of medical image segmentation, domain differences may be large due to various complex factors, so the distribution of unknown domain data generated from source domain data is extremely difficult to predict. In addition, medical data is usually high-dimensional and has a large amount of tissue details. The process of synthesizing fictitious samples requires a very large model and consumes a lot of resources. Summary of the invention

[0003] The purpose of this application is to propose an unsupervised domain adaptation medical image segmentation method and device based on multi-scale features to address the above-mentioned technical problems. By combining the domain-invariant shape reference segmentation model and domain-specific generalization information learning enhanced by random features, under the premise of single source domain data training, the network model's learning ability for domain-invariant features and simulated unknown domain-specific features is improved, thereby maximizing the model's generalization ability and segmentation accuracy in various unknown target domains.

[0004] On the one hand, a single-source domain generalized medical image segmentation method based on shape dictionary, comprising:

[0005] S1, shape dictionary learning step, generates a shape dictionary and a corresponding true shape coefficient vector that can effectively represent the source domain segmented anatomical shape from the source domain medical image dataset through a dictionary learning method;

[0006] S2, the overall segmentation network construction step, constructs an overall segmentation network composed of a source domain data segmentation branch network with dual consistency regularization and a random enhanced feature segmentation branch network; the source domain data segmentation branch network includes a domain invariant shape reference segmentation model DIMD based on U-Net, a shape reference mask generation module SRMG is added to DIMD to predict the first predicted shape coefficient vector of each medical image in the source domain, a first shape reference mask is generated based on the first predicted shape coefficient vector and the shape dictionary, and the first shape reference mask is cascaded with the features output by the U-Net decoder to generate a first predicted mask; the random enhanced feature segmentation branch network adds a random low-level segmentation model before the first layer of DIMD The feature enhancement module RLFA obtains three versions of random enhanced features for the source domain medical image features through a random convolution layer with random weights generated by the input noise z, and inputs the three random enhanced features into the U-Net encoder, and inputs the features output by the U-Net encoder into the shape reference mask generation module SRMG to obtain the second predicted shape coefficient vector, the third predicted shape coefficient vector and the fourth predicted shape coefficient vector, which are then combined with the shape dictionary to obtain the second shape reference mask, the third shape reference mask and the fourth shape reference mask, which are cascaded with the features output by the U-Net decoder to output the predicted second enhanced feature prediction mask, the third enhanced feature prediction mask and the fourth enhanced feature prediction mask;

[0007] S3, overall segmentation network training step, based on the source domain medical image dataset, the overall segmentation network is trained through consistency loss, adversarial domain enhancement strategy loss and overall shape coefficient loss to obtain a trained overall segmentation network;

[0008] S4, the medical image segmentation step, uses the trained overall segmentation network to segment the medical image to be processed.

[0009] Preferably, the optimization function in the shape dictionary generation process is expressed as follows:

[0010] ;

[0011] in, is the square of the L2 norm, used to calculate the Euclidean distance of vectors; is the L1 norm, which is used to enforce The sparsity of , represents the shape dictionary, ; , represents the true shape coefficient vector corresponding to the shape dictionary, represents the i-th true shape coefficient vector; represents the true segmentation mask of the i-th source domain medical image data; As a balance parameter; Represents the total number of medical image samples in the source domain medical image dataset; Represents the total number of explicit shape templates in the shape dictionary; represents a real number; Indicates height; Indicates width.

[0012] Preferably, the shape dictionary generation process is optimized by alternation, including:

[0013] S11, fixed , updated by minimum angle regression LARS ,as follows:

[0014] ;

[0015] S12, fixed , updated by block coordinate descent ,as follows:

[0016] ;

[0017] S13, continuously iterate S11 and S12 until the convergence condition is met, and generate a shape dictionary that can effectively represent the segmented anatomical shape of the source domain and the corresponding true shape coefficient vector.

[0018] Preferably, the source domain data segmentation branch network specifically includes a U-Net encoder, a shape reference mask generation module SRMG, a U-Net decoder, Concat() and a 1x1 convolution; the U-Net encoder takes the source domain medical image as input and outputs features to SRMG; SRMG predicts a first predicted shape coefficient vector for each medical image, and the first predicted shape coefficient vector is used to generate a first shape reference mask in combination with the shape dictionary of each category, and the first shape reference mask is Concat()ed with the output of the U-Net decoder to obtain features containing shape reference information, which are then further refined through a 1x1 convolution to generate the final first predicted mask.

[0019] Preferably, SRMG designs a separate fully connected layer for each category c , The shape dictionary of the generated category c The atomic vector As the weight initialization of the fully connected layer; , SRMG learns a specific first prediction shape coefficient vector for each class ,pass With shape dictionary Multiply to generate the first shape reference mask ,as follows:

[0020] ;

[0021] in, represents the first predicted shape coefficient vector of the kth explicit shape template; represents the k-th explicit shape template.

[0022] Preferably, the random low-level feature enhancement module RLFA first inputs the source domain medical image into a random weight generation module RWGM before inputting the source domain medical image into a U-Net encoder; the random weight generation module RWGM has a convolutional layer with random weights, which achieves random enhancement of domain-specific low-level features of the image while keeping the domain-invariant global shape features unchanged by applying random filters.

[0023] Preferably, the enhanced feature vector obtained by the convolution layer is combined with the original feature vector to obtain the final random enhanced feature vector. ,as follows:

[0024] ;

[0025] in, Combine weights for features; represents the i-th source domain medical image; represents the original feature vector; represents the enhanced feature vector; .

[0026] Preferably, the enhanced feature vector , which is expressed as follows:

[0027] ;

[0028] in, is a random convolution kernel.

[0029] Preferably, the overall segmentation loss It is expressed as follows:

[0030] ;

[0031] ;

[0032] ;

[0033] ;

[0034] in, represents the consistency loss; represents the adversarial domain enhancement strategy loss; represents the overall shape factor loss; , and is a custom weight coefficient; represent the second enhanced feature prediction mask, the third enhanced feature prediction mask and the fourth enhanced feature prediction mask respectively; represents the first prediction mask; denote a second shape reference mask, a third shape reference mask, and a fourth shape reference mask, respectively; represents a first shape reference mask; A ground-truth segmentation mask representing the source domain medical image data; Represents the true shape coefficient vector corresponding to the shape dictionary.

[0035] On the other hand, a single-source domain generalized medical image segmentation device based on shape dictionary includes:

[0036] A shape dictionary learning module is used to generate a shape dictionary and a corresponding true shape coefficient vector that can effectively represent the source domain segmented anatomical shape from a source domain medical image dataset through a dictionary learning method;

[0037] The overall segmentation network construction module is used to construct an overall segmentation network composed of a source domain data segmentation branch network with dual consistency regularization and a random enhanced feature segmentation branch network; the source domain data segmentation branch network includes a domain invariant shape reference segmentation model DIMD based on U-Net, a shape reference mask generation module SRMG is added to DIMD to predict the first predicted shape coefficient vector of each medical image in the source domain, a first shape reference mask is generated based on the first predicted shape coefficient vector and the shape dictionary, and the first shape reference mask is cascaded with the features output by the U-Net decoder to generate the first predicted mask; the random enhanced feature segmentation branch network adds a random low-level segmentation model before the first layer of DIMD The feature enhancement module RLFA obtains three versions of random enhanced features for the source domain medical image features through a random convolution layer with random weights generated by the input noise z, and inputs the three random enhanced features into the U-Net encoder, and inputs the features output by the U-Net encoder into the shape reference mask generation module SRMG to obtain the second predicted shape coefficient vector, the third predicted shape coefficient vector and the fourth predicted shape coefficient vector, which are then combined with the shape dictionary to obtain the second shape reference mask, the third shape reference mask and the fourth shape reference mask, which are cascaded with the features output by the U-Net decoder to output the predicted second enhanced feature prediction mask, the third enhanced feature prediction mask and the fourth enhanced feature prediction mask;

[0038] The overall segmentation network training module is used to train the overall segmentation network based on the source domain medical image dataset through consistency loss, adversarial domain enhancement strategy loss and overall shape coefficient loss to obtain a trained overall segmentation network;

[0039] The medical image segmentation module is used to segment the medical image to be processed using the trained overall segmentation network.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] (1) Based on the feature that the shape of anatomical structures in medical images does not change due to domain shift, the present invention introduces dictionary learning. By extracting a universal semantic shape dictionary as prior knowledge from a single source domain dataset, this prior information can be further effectively utilized in unknown domain data, and the trained single domain model can be generalized to unknown domain data. This method is used to learn the domain-invariant features of the source domain data, which is not only lightweight and efficient, but also can greatly enhance the generalization ability of the network model.

[0042] (2) Since domain-specific features are mainly low-level features, the present invention introduces a random feature enhancement method to enhance the low-level features, thereby simulating the domain-specific information of the unknown domain. Through this method, the network model can learn domain-specific information from the simulated features of the unknown domain, so that it can effectively resist the influence of domain shift caused by the domain-specific information of the unknown domain data in the segmentation of unknown domain data, thereby improving the generalization ability of the network model. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0044] Figure 1 A schematic diagram of a flow chart of a single-source domain generalized medical image segmentation method based on a shape dictionary according to an embodiment of the present application;

[0045] Figure 2 A schematic diagram of the structure of a source domain data segmentation branch network according to an embodiment of the present application;

[0046] Figure 3 A schematic diagram of the structure of a random low-level feature enhancement module RLFA according to an embodiment of the present application;

[0047] Figure 4 This is a random convolution enhanced feature map of an embodiment of the present application, wherein the upper part is the convolution kernel size and the lower part is the convolution kernel stack;

[0048] Figure 5 A schematic diagram of the structure of a random weight generation module RWGM according to an embodiment of the present application;

[0049] Figure 6 A schematic diagram of the overall structure of the overall segmentation network SDRA of an embodiment of the present application;

[0050] Figure 7 Schematic diagram of the segmentation results of fundus images using different algorithms;

[0051] Figure 8 Schematic diagram of the segmentation results of prostate images using different algorithms;

[0052] Fig. 9 Schematic diagram of a single-source domain generalized medical image segmentation device based on shape dictionary according to an embodiment of the present application. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0054] like Figure 1 As shown, this embodiment provides a single-source domain generalization medical image segmentation method based on shape dictionary (Single-source Domain Generalization Medical Image Segmentation Method Based on Shape Dictionary and Random Augmentation, SDRA), which includes the following steps.

[0055] S1, shape dictionary learning step, generates a shape dictionary and the corresponding true shape coefficient vector that can effectively represent the source domain segmented anatomical shape from the source domain medical image dataset through a dictionary learning method.

[0056] In the field of medical image segmentation, the segmentation of medical images requires that the deep model can accurately analyze the anatomical structure in the medical image. The effect of this analysis process should not be limited by the type or performance of the scanning device, and the versatility of the deep model in different application scenarios should be demonstrated. However, due to domain offset, the data performance of the deep model in different domains will deteriorate due to the different feature distributions. Although there is domain offset in different domains, it is recognized that the shape of the anatomical structure of the medical image does not change due to domain offset. Based on this objective fact, the main purpose of this step is to capture the shape information of the anatomical structure in medical images to enhance the model's adaptability to new domain data. Through dictionary learning technology, representative shape prior knowledge is explicitly extracted from the medical image data in the source domain, and these key shape information is integrated into the deep segmentation model to obtain the Domain-Invariant Shape Reference Segmentation Model Based on Dictionary Learning (DIMD). This not only promotes the model to maintain high-precision segmentation capabilities when encountering previously unseen data, but also significantly improves the generalization ability of the model. Dictionary learning is a powerful machine learning method that aims to learn a compact and representative set of primitives (i.e., a dictionary) from a large amount of data so that any data can be approximately represented by a sparse linear combination of these primitives. In this embodiment, dictionary learning can extract a set of powerful shape templates from data in a single source domain, which is sufficient to represent the basic anatomical structure of all images in the source domain. Through this set of dictionaries, the model of this embodiment can recognize and understand these ubiquitous anatomical structure shapes, so that when faced with new and unknown image data, it can rely on the learned shape priors to perform accurate segmentation.

[0057] Specifically, dictionary learning techniques are used to construct a set of basic elements (i.e., dictionary) so that any given input can be represented by a sparse linear combination of these elements. In this framework, for a set containing The source domain dataset consists of samples , where each sample consists of data and the corresponding segmentation mask , defines a A dictionary of explicit shape templates This dictionary can be combined into a segmentation mask for each sample in the source domain dataset through linear combination coefficients .

[0058] In the process of building this shape dictionary, two principles must be followed to ensure that it can be applied to unknown fields: First, the number of templates in the dictionary should be much smaller than the number of samples in the dataset. , to avoid overfitting; secondly, in order to maximize the expressiveness of each template, the number of linear combination coefficients used to represent the shape dictionary should be minimized, that is, these coefficients should be sparse.

[0059] Since medical image segmentation involves multiple anatomical structures, for simplicity, the segmentation mask of each structure is regarded as an independent binary segmentation mask, and a shape dictionary is constructed and optimized for each anatomical structure. Since the steps of constructing and optimizing the shape dictionary for each anatomical structure are the same, the following is The segmentation mask of the shape dictionary (i.e., the true segmentation mask of the source domain medical image data) is uniformly represented and constructed. The dictionary generation process is equivalent to solving the problem of the dictionary and coefficient The optimization problem is shown in the following formula.

[0060] ;

[0061] in, is the square of the L2 norm, used to calculate the Euclidean distance of vectors; is the L1 norm, which is used to measure the number of non-zero elements in a vector and is used here to enforce The sparsity of , represents the shape dictionary, ; , represents the true shape coefficient vector corresponding to the shape dictionary, represents the i-th true shape coefficient vector; represents the true segmentation mask of the i-th source domain medical image data; As a balancing parameter, it is used to normalize the coefficients obtained during the learning process; Represents the total number of medical image samples in the source domain medical image dataset; Represents the total number of explicit shape templates in the shape dictionary; represents a real number; Indicates height; Indicates width.

[0062] Through this optimization problem, we can find the optimal shape dictionary and coefficient , can be combined with the most accurate To represent the source domain Segmentation masks for sample images While ensuring the coefficient It is kept sparse. The sparse representation of helps improve the generalization ability of the dictionary to unseen samples because it limits the model from overfitting to specific features of the training data.

[0063] In order to optimize the dictionary generation, this embodiment selectively converts each segmentation mask Updating sparse coefficients using Least Angle Regression (LARS) Then update the dictionary D using block coordinate descent (BCD). Update sparse coefficients The process is that for a given image mask , find a sparse coefficient vector Make Can be approximately represented as a dictionary The linear combination of the column vectors of . Mathematically, this can be achieved by the following optimization formula.

[0064] ;

[0065] This optimization problem is solved by the LARS algorithm, which finds the best reconstructed segmentation mask through an iterative process. The process starts by setting the residual as the target segmentation mask , and the coefficient vector is initialized to zero. The algorithm first identifies the dictionary atoms that are most relevant to the residual , and add it to the model. The algorithm then moves forward along the atoms while keeping the correlations of all selected atoms to the residuals equal. During this process, the residuals are continuously updated to reflect the current model's predictions and the actual segmentation mask. As the selection proceeds, LARS selects new atoms or updates the coefficients of existing atoms to maintain maximum correlation and reduce residuals. The process stops when the coefficient vector reaches a predetermined sparsity level or the residual is reduced to a certain extent. Once the sparse coefficients of all image masks are obtained , you need to update the dictionary , to better fit the data. This can be achieved through the following optimization formula.

[0066] ;

[0067] This optimization problem is solved by the BCD algorithm. In the update loop, the algorithm traverses each atom in the dictionary D ). For each atom, the algorithm first computes the residual segmentation mask, i.e., the reconstruction error if the current atom is excluded. This involves subtracting the contribution of all other atoms from the reconstruction of each segmentation mask, resulting in a residual matrix only related to the current updated atom , Y represents the matrix of the entire segmentation mask, and To remove the Then, using this residual, we use linear regression to solve the optimization problem. , to find the best update atom , the atom can be minimized with Once the optimal solution is found, the atomic will be updated into the dictionary and usually normalized to avoid numerical problems. This is done for each atom in turn until the entire dictionary D is updated for all signals The representation error is minimized.

[0068] The entire dictionary learning process is carried out alternately. First, fix renew , then fix renew By iterating these two steps until the convergence condition is met, a dictionary that can effectively represent the anatomical shape of the source domain segmentation can be obtained. and coefficient .

[0069] S2, the overall segmentation network construction step, constructs an overall segmentation network composed of a source domain data segmentation branch network with dual consistency regularization and a random enhanced feature segmentation branch network; the source domain data segmentation branch network includes a domain invariant shape reference segmentation model DIMD based on U-Net, a shape reference mask generation module SRMG is added to DIMD to predict the first predicted shape coefficient vector of each medical image in the source domain, a first shape reference mask is generated based on the first predicted shape coefficient vector and the shape dictionary, and the first shape reference mask is cascaded with the features output by the U-Net decoder to generate a first predicted mask; the random enhanced feature segmentation branch network adds a random low-level segmentation model before the first layer of DIMD The feature enhancement module RLFA obtains three versions of random enhanced features for the source domain medical image features through a random convolution layer with random weights generated by the input noise z, and inputs the three random enhanced features into the U-Net encoder, and inputs the features output by the U-Net encoder into the shape reference mask generation module SRMG to obtain the second predicted shape coefficient vector, the third predicted shape coefficient vector and the fourth predicted shape coefficient vector, which are then combined with the shape dictionary to obtain the second shape reference mask, the third shape reference mask and the fourth shape reference mask, which are cascaded with the features output by the U-Net decoder to output the predicted second enhanced feature prediction mask, the third enhanced feature prediction mask and the fourth enhanced feature prediction mask.

[0070] Once the dictionary D is trained, it can be integrated into the deep learning model, that is, the DIMD model proposed in this embodiment. When training the DIMD model, the shape dictionary is used as the prior of the shape, and the shape information obtained by dictionary learning is used to enhance the generalization ability of the model for unknown domain data. The structure of the DIMD model is as follows: Figure 2 shown.

[0071] Specifically, a DIMD model is designed based on U-Net, and a shape reference mask generation module (SRMG) is added to predict the shape coefficient vector of each medical image. Figure 2 The first predicted shape coefficient vector is taken as an example. SRMG takes the output of the U-Net encoder as input and predicts a shape coefficient vector for each medical image. The shape coefficient vector is used to combine the shape dictionary of each category to generate the first shape reference mask. Finally, the first shape reference mask is concat () with the output of the U-Net decoder to obtain features containing shape reference information, which are then further refined through a 1x1 convolution to generate the final first predicted mask (segmentation mask). Specifically, the final layer of the U-Net encoder converts the received deep features into a one-dimensional feature vector by capturing the global context information through a global average pooling layer (GAP) , SRMG receives , and then by and shape dictionary To predict the shape coefficient vector for each anatomical category. SRMG designs a separate fully connected layer for each category c , The learned shape dictionary The atomic vector As the weight initialization of the fully connected layer, the output size is set to Coefficients of the same size .pass , SRMG learns specific shape features for each category ,pass With shape dictionary Multiply together to generate the shape reference mask , as shown in the following formula.

[0072] ;

[0073] in, represents the first predicted shape coefficient vector of the kth explicit shape template; represents the k-th explicit shape template.

[0074] In the decoder part, U-Net converts the high-level semantic features (deep features) of the encoder back to the resolution of the original input image through step-by-step upsampling and convolution operations, while fusing the skip connection features from the corresponding encoder layer. The output features of the decoder are concat()ed with the shape reference mask predicted by SRMG and then 1x1 convolved to generate the final output. SRMG allows the network to leverage the prior information in the shape dictionary to enhance the accuracy and robustness of segmentation, especially when dealing with domain shift problems in unknown domains.

[0075] Since the purpose of the segmentation network is to adapt to the unknown target domain data rather than just to achieve better segmentation results in the source domain, the goal of the optimization network is to minimize the predicted shape coefficient vector The true benchmark value of the coefficient obtained and optimized in the dictionary learning phase The difference between and the traditional segmentation loss. In the shape dictionary learning process, the shape coefficient vector It is regularized into a sparse vector, which means that most elements are close to zero and only a few elements have large values. In order to maintain the sparse nature of the prediction coefficient vector, this embodiment introduces cosine similarity loss, which pays more attention to the directional consistency between vectors rather than the magnitude of absolute error. By emphasizing the directional consistency of the coefficient vector, the model learns to recognize the general pattern of shape features instead of overfitting the specific values ​​in the training data, which can improve the model's generalization ability to unknown domain shapes. Specific shape coefficient loss As shown below.

[0076] ;

[0077] in, Represents the total number of categories in the shape dictionary; represents the dot product of two coefficient vectors, is the product of the lengths of the two coefficient vectors, and dividing these two gives the cosine similarity, is a small constant used to avoid division by zero, set The more consistent the two vectors are, the greater the cosine similarity is. The smaller it is.

[0078] Using cross entropy loss and Dice loss The combined method is used to design the segmentation loss. The specific segmentation loss As shown below.

[0079] ;

[0080] ;

[0081] in, Indicates a custom weight coefficient; For sample In category The real label on is the model for medical image samples The predicted probability of belonging to category c, is a small constant used to avoid division by zero.

[0082] The shape reference mask can be integrated through the DIMD-based source domain data segmentation branch network to assist in solving the generalization problem in medical image segmentation. However, since the model is learned in a single source domain data distribution, the reliability of the generated shape coefficient cannot be guaranteed when used in an unknown domain with a distribution offset, thereby limiting the effective use of the shape reference mask. As follows, the unknown domain is simulated so that the segmentation model can achieve a good fitting effect in an unseen domain through the shape reference mask with the help of shape prior knowledge to achieve domain generalization. At present, some methods simulate unknown domains through image generation technology to ensure that shape priors can be effectively used to improve generalization performance under any unknown data distribution. However, these methods are usually computationally complex and expensive. Therefore, this embodiment proposes a random low-level feature augmentation module (RLFA), which aims to enhance low-level features while retaining the semantic information of deep features as much as possible, so as to simulate the data features of unknown domains.

[0083] It should be noted that the deep features involve more abstract semantic information, such as semantic structure and deformation. In the medical image segmentation task, the deep features may include the shape, size and relative position of the segmented structure, as well as the spatial relationship between them. These deep features help the network understand the correlation between different regions in the image and convert it into semantically meaningful information, so as to better complete the segmentation task.

[0084] Specifically, the data features of the unknown domain are simulated by modifying the low-level domain-specific features of the input image, such as texture and contrast, while keeping the shape of the object unchanged. The model enhances the generalization ability by learning the simulated data features of the unknown domain. Through the RLFA module, the model can more effectively handle unknown domains with distribution shifts without relying on complex and expensive image generation techniques. It is able to improve the generalization performance of the DIMD segmentation model while maintaining the effectiveness of the shape prior. The structure of the RLFA module is as follows: Figure 3 shown.

[0085] RLFA first feeds the input image into a convolutional layer with random weights before feeding it into the segmentation network. This random convolutional layer randomly enhances the domain-specific low-level features of the image, such as local texture, by applying random filters while keeping the domain-invariant global shape features unchanged. This random convolutional layer can maintain the global shape unchanged while randomly enhancing domain-specific features because in images, adjacent pixels are usually naturally clustered due to similarity in color or texture to form a basic shape representing a specific object or background. By using a convolutional layer with shared parameters, local patches of the image can be linearly projected to the corresponding positions of the output feature map, and the introduction of random filters further allows these local patches to be projected to any position on the output feature map. Although the projection is random, this method can roughly maintain the relative similarity between input patches, thereby ensuring that the output feature map is consistent in shape with the input image, which generates an image with random domain-specific features but unchanged domain-invariant features. By randomly selecting filters of different sizes, the generated image maintains a consistent global shape while presenting random local texture features. This process is shown in the following equation:

[0086] ;

[0087] in, is the enhanced feature vector output by the random convolution kernel, is the original feature vector of the input, * is the convolution operation, is a random convolution kernel. , whose size is , h and w are the height and width of the random convolution kernel, and is the number of channels of the input and output features. For the random convolution kernel 0, its weights are randomly generated, in this way, the local texture features are randomly enhanced to simulate the domain-specific features of the unknown domain. The size of is also randomly sampled, usually in Random sampling is performed in , generating random enhanced features of different scales, thereby retaining the shape features of each scale while randomizing the local texture features. Figure 4 It can be seen that when the convolution kernel The larger the size, the more distorted the shape semantics of the enhanced feature, which leads to the destruction of domain invariant features, affects the model's learning of basic shape semantics, and reduces segmentation performance. To this end, this embodiment proposes to replace the large convolution kernel with a stack of multiple small convolution kernels. Specifically, by stacking multiple small convolution kernels, it is more fine-grained than the top ten convolution kernels, reducing the impact on pixels far from the center in the theoretical receptive field, thereby alleviating the semantic distortion problem and retaining the domain invariant features such as shape semantics as much as possible. Unlike the previous random weights, the multiple small convolution kernels in this combination share weights, which can gradually increase the distortion amplitude of a single transformation of the center pixel and more effectively randomly enhance domain-specific features. Therefore, the random sampling of the size of the random convolution kernel 0 is redefined as starting from The corresponding number of 3x3 convolution kernels randomly sampled in [1,2,3,4,5,6,7] are stacked. The effect of the feature map enhanced by random convolution in this way is as follows Figure 4 shown.

[0088] Even if the semantic distortion problem is avoided as much as possible through the above methods, random convolution may still introduce noise that disturbs the domain invariant information. In order to preserve the domain invariant information as much as possible, this section combines the enhanced feature vector obtained by random convolution with the original feature vector to generate a new feature vector that expands the domain range while keeping the source domain invariant information unchanged as much as possible. The process is shown below.

[0089] ;

[0090] in, It is a feature combination weight. In order to maintain the randomness of RLFA, it is also randomly sampled. By combining the original feature vector, the semantic shape feature is further retained. Next, the generation process of random weights is described. At present, the generation of random weights directly adopts the method of random sampling from Gaussian distribution. Although this method can generate random convolution kernel weights from Gaussian distribution, it is random. This will cause the generated random enhancement features to be close to the source domain distribution, and thus cannot simulate the unseen target domain distribution, resulting in the model not only failing to improve the generalization ability of the model but also reducing the segmentation ability of the model. Based on this, this embodiment embeds a random weight generation module (RWGM) in the RLFA module, aiming to generate random weights by inputting a random noise, and then introduce enhanced features that challenge the model by selecting high-frequency noise randomization features through gradient inversion during training, thereby guiding the model to learn domain-invariant features from high-frequency noise and enhance domain generalization capabilities. The input noise z of RWGM is randomly sampled from a Gaussian distribution. For the input z, RWGM is processed by two stacked 3x3 transposed convolution layers and a 3x3 convolution layer. First, a transposed convolution layer is used to upsample the input data to increase the size of the feature map. Next, a batch normalization layer is used to normalize the upsampled feature map, and a ReLU activation layer is used to introduce nonlinearity to help the network capture more complex patterns. This process is then repeated again, and another transposed convolution layer is used to continue upsampling and increase the size of the feature map, followed by a second batch normalization layer and a ReLU activation layer to further refine the features. Finally, the features are finally processed and adjusted through a 3x3 convolution layer with a stride of 1 to produce accurate 3x3 convolution kernel random weights. The generated random weights are then input into the random convolution layer of the RLFA module, and the random convolution kernel weights are assigned to generate randomly enhanced features. The structure of the RWGM module is as follows Figure 5 shown.

[0091] Then, in order to make the generated random enhancement features effectively improve the generalization ability of the model, a domain enhancement strategy based on RWGM is proposed to improve the generalization performance of the model on unknown domain data. The core of this strategy is to use the idea of ​​adversarial training to enable the model to generate and adapt to challenging samples that simulate unknown domains during the training phase, and then maintain better performance and robustness when facing novel or unseen data in practical applications. Specifically, RWGM dynamically generates convolution kernel weights from the RWGM module in each training iteration. These dynamically generated weights adjust the parameters of the convolution layer, thereby generating data samples with new domain-specific features that are challenging and have never been seen by the model. This process continuously generates adversarial samples, allowing the model to learn how to effectively process these simulated unknown domain data. Then, a gradient reversal layer is added to adjust the processing loss of RWGM during the back propagation process. Specifically, the gradient is multiplied by a negative constant (set to -1) and propagated to RWGM. The purpose of this step is to increase rather than reduce the loss of RWGM, thereby promoting RWGM to generate enhanced features that can maximize the current loss of the model during training, that is, to expand the domain-specific features of the unknown domain. These enhanced features generated by the adversarial domain enhancement strategy, called challenging samples, represent possible domain changes or data distributions in unseen domains. In this way, the optimization problem is transformed into an adversarial problem, forcing the model to learn and find feature representations that can adapt to both normal samples and adversarial samples. After generating challenging samples, another part of the model (without RWGM) will try to optimize the processing of these samples. This process is carried out through the normal training process, but the training data contains both source domain samples and generated challenging samples. In this way, the model gradually learns how to handle a wider range of data distributions, including those in unseen domains that may be encountered in the future.

[0092] In summary, the structure diagram of the overall segmentation network of this embodiment is as follows: Figure 6 As shown in the figure. For the source domain dataset, the corresponding shape dictionary is first generated by the dictionary learning method for subsequent segmentation network use. Then, an overall segmentation network consisting of two DIMDs with double consistency regularization is proposed. For the source domain data segmentation branch network, the source domain data is segmented to obtain shape features. , shape reference mask M, prediction mask y. For the random enhanced feature segmentation branch network, a random low-level feature enhancement module RLFA is added before the first layer of DIMD. , three versions of random enhancement features are obtained through a random convolution layer with random weights generated by the input noise z: The three random enhancement features are input into the shape reference mask generation module SRMG to obtain their respective shape features. , and then with the shape dictionary Combine to generate shape reference mask , the final shape reference mask is concatenated with the decoder output feature and the final output is the predicted enhanced feature prediction mask In this process, gradient reversal is used to update the parameters of the random weight generation module RWGM.

[0093] S3, the overall segmentation network training step, is based on the source domain medical image dataset, and the overall segmentation network is trained through consistency loss, adversarial domain enhancement strategy loss and overall shape coefficient loss to obtain a trained overall segmentation network.

[0094] Specifically, the overall segmentation loss of the single-source domain generalized medical image segmentation method based on shape dictionary and random enhancement proposed in this embodiment is It consists of three parts: consistency loss , adversarial domain enhancement strategy loss and overall shape factor loss , as shown below respectively.

[0095] ;

[0096] ;

[0097] ;

[0098] ;

[0099] in, , and is a custom weight coefficient; represent the second enhanced feature prediction mask, the third enhanced feature prediction mask and the fourth enhanced feature prediction mask respectively; represents the first prediction mask; denote a second shape reference mask, a third shape reference mask, and a fourth shape reference mask, respectively; represents a first shape reference mask; A ground-truth segmentation mask representing the source domain medical image data; Represents the true shape coefficient vector corresponding to the shape dictionary.

[0100] S4, the medical image segmentation step, uses the trained overall segmentation network to segment the medical image to be processed.

[0101] The single-source domain generalized medical image segmentation method based on shape dictionary of this embodiment is verified by experiments as follows, which includes the following steps.

[0102] (1) Dataset.

[0103] In the field of medical image segmentation, medical image database is the cornerstone of medical image segmentation. Due to the difficulty of data collection and high data privacy requirements, it is impossible to provide sufficient samples for the realization of medical image segmentation research. Therefore, the research and development of medical image segmentation is challenged by the small data samples. This embodiment mainly uses public fundus image datasets and prostate image datasets for experiments. There are mainly three fundus image datasets, namely Drishti-GS, RIM-ONE_r3 and REFUGE challenge training set, and three prostate image datasets, namely NCI, I2CVB and PROMISE12. These datasets are introduced below.

[0104] Drishti-GS dataset: The dataset was developed by a team of researchers from India to provide high-quality fundus images to support automatic segmentation of the optic disc and optic cup. It provides 101 2049 color fundus images and corresponding annotations (the training set includes 50 images and the test set includes 51 images).

[0105] RIM-ONE_r3 dataset: is a public fundus image dataset focusing on dry eye research, aiming to promote the development of glaucoma detection and optic disc and optic cup segmentation technology. This dataset collects 159 images from several European hospitals. The training set includes 99 images and the test set includes 60 images. Each image has been accurately annotated by ophthalmologists, including the boundaries of the optic disc and optic cup.

[0106] REFUGE Challenge Dataset: The 2018 REFUGE Challenge provides a set of annotated fundus images for training and validating computer vision and machine learning algorithms. The training set includes 400 The REFUGE dataset contains high-quality color fundus images that are commonly used for the diagnosis and monitoring of glaucoma. These images can show the detailed structure of the retina, optic disc, and macula, which is crucial for the diagnosis of glaucoma. The images in the dataset are accompanied by expert annotations, including the boundary positioning of the optic disc and optic cup, which are key indicators for evaluating optic nerve head damage and glaucoma. By calculating the cup-to-disc ratio, it can assist in the diagnosis of glaucoma.

[0107] NCI-ISBI2013 Dataset: The NCI-ISBI2013 dataset is a prostate magnetic resonance imaging (MRI) data challenge jointly initiated by the Cancer Imaging Program of the National Cancer Institute (NCI) of the United States and the International Society of Biomedical Imaging (ISBI) in 2013. The training set of the challenge dataset includes 60 cases, half of which are from Boston Medical Center (BMC), with a magnetic field strength of 1.5T, a resolution of 0.4 mm in the transverse plane and 3 mm in the sagittal plane. An endoscopic coil is used, and imaging is performed by Philips equipment. The other half of the cases are provided by Radboud University Medical Center (RUNMC) in Nijmegen, the Netherlands, with a magnetic field strength of 3T, a resolution of 0.6-0.625 mm in the transverse plane and 3.6-4 mm in the sagittal plane. A surface coil is used, and imaging is performed by Siemens equipment.

[0108] I2CVB dataset: This dataset comes from the Initiative for Collaborative Computer Visior Benchmarking (I2CVB), provided by the HCRUDB institution, and contains 19 cases. The magnetic field strength is 3T, the resolution is 0.67-0.79 mm in the transverse plane and 1.25 mm in the sagittal plane. No endoscopic coil is used, and imaging is performed by Siemens equipment.

[0109] PROMISE1 dataset: This dataset is derived from ProstateMRImageSegmentation2012 (PROMISE12), provided by UCL, BIDMC and HK, with 13, 12 and 12 cases respectively. The dataset used in the experiment of this embodiment is the data provided by HK, which contains 12 cases, with a magnetic field strength of 1.5T, a resolution of 0.625 mm in the transverse plane and 3.6 mm in the sagittal plane, using an endoscopic coil and imaging by Siemens equipment.

[0110] (2) Data preprocessing

[0111] Specifically, if the entire medical image is used directly for training, there are the following problems: the background information of the original image, such as edge areas and non-target structures, may introduce noise, affecting the model's ability to recognize key features, resulting in a decrease in the model's resolution. To this end, this paper introduces the concept of ROI (region of interest). The ROI image focuses on the segmented area and reduces background interference, allowing the model to learn the details of these key areas more attentively, thereby improving the accuracy of segmentation.

[0112] Due to the limitation of the small amount of medical image datasets, the model trained directly with medical image datasets is prone to overfitting and poor generalization. In order to improve the model training effect, this embodiment adopts a series of data enhancement strategies to simulate different visual effects and expand the training dataset. By inverting and rotating the image, adjusting the brightness and contrast, the diversity of samples can be artificially increased, which helps the model have better generalization ability in practical applications. After these enhanced images, for fundus images, by fixing the zoom to For the prostate image dataset, each sample is resized to The size of the image is 1000×10, ensuring the consistency of the data during the training process. At the same time, the image is randomly flipped and standardized so that the pixel values ​​of each channel have a uniform mean and standard deviation, further reducing the risk of overfitting during model training. Through these preprocessing steps, the number of images in the training dataset is increased and the quality is improved, ultimately solving the limitation of the difficulty of training models caused by the small amount of data in the medical image dataset.

[0113] (3) Evaluation indicators

[0114] In order to objectively evaluate the segmentation performance of the target domain model on the medical image dataset in the experiment, this embodiment selects two widely used standard indicators in image segmentation evaluation: Dice coefficient (Dice) and average surface distance (ASD). These indicators together provide a comprehensive evaluation for the quantification of segmentation effect. The Dice coefficient (Dice) compares the similarity between the predicted segmentation result and the true segmentation label. It calculates the ratio of twice the intersection area of ​​the predicted and true segmentation to the sum of their respective areas. The value range of the Dice coefficient is from 0 to 1. The higher the value, the closer the segmentation result is to the actual situation, that is, the better the segmentation effect. The calculation formula is as follows.

[0115] ;

[0116] Among them, A is the predicted segmentation result set, B is the actual segmentation result set, Represents the number of elements in the intersection of set A and set B. represents the number of elements in set A. Represents the number of elements in set B.

[0117] The average surface distance (ASD) measures the average distance between the predicted segmentation and the true segmentation surface. It calculates the average shortest distance from the predicted segmentation to all points on the true segmentation surface, and the average shortest distance from the true segmentation to all points on the predicted segmentation surface. The smaller the ASD value, the closer the segmentation result is to the true situation, that is, the higher the segmentation quality. The calculation formula is as follows.

[0118] ;

[0119] Among them, X1 and Y1 represent the point sets on the predicted segmentation surface and the true segmentation surface respectively. Indicate point and Point ASD calculates the average shortest distance between all pairs of points from X1 to Y1 and from Y1 to X1.

[0120] (4) Experimental environment and training configuration information

[0121] The software and hardware experimental environment in this experiment is shown in Table 1.

[0122] Table 1 Experimental software and hardware environment configuration;

[0123]

[0124] The training parameters of this experiment are detailed as follows:

[0125] The number of shape dictionary elements K is set to 24 in the fundus image experiment and 48 in the prostate image experiment. The Adam optimizer is used for training with momentum of 0.9 and 0.99, and the learning rate is initialized to 1𝑒 -3 , a total of 100 epochs are trained, and the batch size is set to 5. Set to 0.5, the total loss Set to 1, Set to 1, Set to 1, because this example considers that keeping the shape and prediction unchanged is as important as constraining different enhanced outputs to be consistent. The 3x3 convolution kernel stack pool is initialized to 𝑛∈[1,2,3,4,5]. 𝛽 is randomly sampled from a uniform distribution of 𝑈(0,0.5), and the noise z is randomly sampled from a normal distribution of 𝑁(0,1.0).

[0126] (5) Experimental design

[0127] The experiment of this embodiment is mainly to explore the following questions:

[0128] First, in the SDG medical image segmentation scenario, what is the accuracy of the proposed SDRA method? Compared with other SFUDA medical image segmentation methods and SDG medical image segmentation methods, the results can be intuitively seen through visual segmentation results.

[0129] Second, in the SDG medical image segmentation scenario, the proposed DIMD and RLFA modules affect the model performance. Four sets of ablation experiments are designed to verify the effectiveness of each module by using the same dataset and training process.

[0130] Third, in the SDG medical image segmentation scenario, the proposed RLFA module is placed at different positions in the network encoder (such as the initial layer, middle layer, and back layer of the network) to verify the effectiveness of the strategy in this chapter.

[0131] Fourth, in the SDG medical image segmentation scenario, the impact of the input of the weight generation module in the proposed RLFA module on the model performance under different noise distributions (such as uniform distribution, Gaussian mixture distribution and Gaussian distribution).

[0132] (6) Experimental results and analysis

[0133] (6.1) Fundus image segmentation experiment analysis

[0134] Specifically, the experiments were mainly conducted using public fundus image datasets, including Drishti-GS, RIM-ONE_r3, and REFUGE challenge training sets. The segmentation performance of the model on the target domain dataset under the condition of single-source domain generalization was verified by three fundus image datasets. In this embodiment, the proposed single-source domain generalization model was trained from the REFUGE challenge training set, and then verified on the Drishti-GS dataset and the RIM-ONE_r3 dataset. The effectiveness of the SDRA method was verified by designing comparative experiments, and compared with other advanced SDG methods in the comparative experiments. Specifically, the M-ADA method, TTT method, BigAu method, FS method, and RandConv method were used as comparative methods. Among them, M-ADA is a method for creating adversarial domain enhancement of virtual data using Wasserstein autoencoders, the TTT method is a method for transforming into self-supervised rotation prediction when testing in the target domain, the BigAug method is a method for enhancing data through deep stacking transformation, the FSR method is a method for enhancing features through style randomization, and the RandConv method is a method for enhancing features through random convolution. At the same time, adding Baseline represents the direct migration method of predicting the target domain results from the baseline U-net model trained directly from the source domain data. In this experiment, all methods are trained 5 times and the average value of all evaluation indicators is taken.

[0135] Table 2 shows the results of different algorithms on the Drishti-GS dataset for the single-source domain generalization model trained on the REFUGE challenge training set. Compared with the baseline, various SDG methods have certain improvements, among which the SDRA method has the largest improvement in various evaluation indicators. Although the BigAug method is very close to the SDRA method in performance, BigAug simulates and synthesizes images in the target domain, while SDRA generates enhanced features through random convolution, and the overhead is much smaller than BigAug. SDRA has better Dice coefficients and ASD distances of optic cup and optic disc than other SDG methods, proving that SDRA can effectively solve the domain generalization problem on the Drishti-GS dataset. SDRA has a significant improvement over the RandConv method, which also uses random convolution to enhance features, with an increase of 3.45 and 3.1 in Dice coefficients of optic cup and optic disc, and a decrease of 1.49 and 1.01 in ASD distances of optic cup and optic disc, which proves the advancedness of the random weight generation module and convolution kernel stacking strategy proposed in this chapter.

[0136] Table 2 Comparison of results of different algorithms on REFUGE challenge training set → Drishti-GS dataset;

[0137]

[0138] Table 3 shows the results of different algorithms on the RIM-ONE_r3 dataset for the single-source domain generalization model trained with the REFUGE challenge training set. It can be seen from the table that the SDRA method still performs best among all SDG methods, which further proves the ability of the SDRA method to solve the generalization problem in the fundus image domain. Among them, the Dice coefficient and ASD distance of the SDRA method are better than those of other SDG methods in both cup segmentation and disc segmentation.

[0139] Table 3 Comparison of results of different algorithms on REFUGE challenge training set → RIM-ONE_r3 dataset;

[0140]

[0141] (6.2) Experimental analysis of prostate image segmentation

[0142] In order to prove the effectiveness of the SDRA method, three prostate image datasets, NCI-ISBI 2013, I2CVB and PROMISE12, were further verified. In this embodiment, a single-source domain generalization model was trained on the prostate I2CVB dataset, and then evaluated on the NCI-ISBI 2013 dataset and the PROMISE12 dataset. The comparison method is the same as the previous one, using the M-ADA method, TTT method, BigAug method, FSR method, RandConv method and Baseline as comparison algorithms. The average value of the evaluation indicators of the five trainings was also calculated.

[0143] Table 4 shows the results of the single-source domain generalization model trained on the prostate I2CVB dataset by different algorithms on the NCI-ISBI2013 dataset. It can be seen that the Dice coefficient of the SDRA method performs best among the SDG methods, 0.79 higher than the second-best FSR method, and 3.51 higher than the worst RandConv method. At the same time, the ASD of the SDRA method also performs best among the SDG methods, and is 0.65 lower than the second-best TTT. This proves the effectiveness of the SDRA method in generalizing to the NCI-ISBI 2013 dataset.

[0144] Table 4 Comparison of results of different algorithms on I2CVB dataset → NCI-ISBI 2013 dataset;

[0145]

[0146] Table 5 Comparison of results of different algorithms on I2CVB dataset → ROMISE12 dataset;

[0147]

[0148] Table 5 shows the experimental results of the single-source domain generalization model trained on the prostate I2CVB dataset using different algorithms on the PROMISE12 dataset. It can be seen that the SDRA method also performs best among the SDG methods, with its Dice coefficient increased by 0.13 compared to FSR and its ASD distance reduced by 0.65 compared to FSR. This further proves the effectiveness of the SDRA method.

[0149] Figure 7 and Figure 8The experimental results of different algorithms in the case of fundus image and prostate image domain adaptation are shown respectively. It can be seen from the figure that if direct migration without single-source domain generalization is used, very poor segmentation results will be obtained, and other single-source domain generalization methods are much worse in the target domain than the SFUDA method using target domain data because they are only trained with source domain data. The SDRA method of the embodiment provides domain generalization capability through domain-invariant feature and domain-specific feature learning, which performs well in segmentation shape and boundary, and the prediction result is close to the true label.

[0150] From the above, the SDRA method proposed in this embodiment can not only show good generalization ability in the SDG scenario of the fundus image dataset, but also perform well on the prostate image dataset. This proves that the SDRA method can well enhance the generalization ability of the model through shape dictionary and random feature enhancement. Among them, shape dictionary learning enables the model to fully learn domain-invariant features, and random feature enhancement enables the model to learn invisible domain-specific features of unknown domains. The two together enhance the generalization ability of the model and achieve good performance in the above experiments.

[0151] (6.3) Ablation experiment

[0152] This experiment trains the ablation model on the prostate I2CVB dataset, and then evaluates it on the NCI-ISBI 2013 dataset and PROMISE12 dataset. As can be seen from Table 6, each module has a great improvement on the two unseen target domain data. The SDRA method can not only combine DIMD to accurately segment the segmented shape, but also combine RFLA to simulate the low-level features of the unseen domain through random data enhancement, which enhances SDRA's ability to segment the boundary. Therefore, SDRA can grasp the segmented shape and boundary at the same time and achieve good domain generalization performance. Among them, DIMD and RFLA are both indispensable and provide essential support for improving model performance.

[0153] Table 6 Comparison of results in ablation experiments on the NCI-ISBI2013 dataset and PROMISE12 dataset;

[0154]

[0155] (6.4) Other experiments

[0156] The following is to verify the improvement effect of the proposed RLFA module on the model at different positions in the network architecture. The model is designed to be placed in different layers of Unet to train the model on the prostate I2CVB dataset, and then evaluated on the NCI-ISBI 2013 dataset and the PROMISE12 dataset. As shown in Table 7, the results show that using the RLFA module in the initial layer can best improve the generalization ability of the model. In contrast, placing the RLFA in the middle and later layers will reduce performance. Moreover, placing it in the middle layer will produce worse results than placing it in the later layers. This is because the features in the later layers are advanced and more resistant to random interference, so it is more difficult to simulate the feature distribution of data in the unknown domain. Therefore, this embodiment places the RLFA in the initial layer before the first layer of the encoder to maximize the generalization performance of the model.

[0157] Table 7 Comparison of results of different embedding strategies on NCI-ISBI2013 and PROMISE12 datasets;

[0158]

[0159] The influence of noise distribution on the generalization performance of RLFA is discussed as follows. Noise distribution is used to generate input noise in the weight generation module of RLFA. Three sets of comparative experiments are designed, sampling noise from uniform distribution, Gaussian distribution and Gaussian mixture distribution respectively. The model is also trained on the prostate I2CVB dataset, and then evaluated on the NCI-ISBI 2013 dataset and PROMISE12 dataset. Table 8 shows the results, from which it can be seen that Gaussian distribution can best improve the generalization ability of the model. And the simpler uniform distribution and Gaussian distribution are better than the complex Gaussian mixture distribution, which shows that complex distribution does not necessarily improve performance. Although the performance of RLFA will vary due to different noise distributions, they can all improve the generalization ability very well, which proves the effectiveness of the RLFA module. In summary, this embodiment uses a simple Gaussian distribution as the input distribution of noise, which is not only simple enough, but also can effectively improve the generalization ability of the model.

[0160] Table 8 Comparison of results of different noise distributions on NCI-ISBI2013 and PROMISE12 datasets;

[0161]

[0162] like Fig. 9 As shown, this embodiment also discloses a single-source domain generalized medical image segmentation device based on a shape dictionary, comprising:

[0163] The shape dictionary learning module 901 is used to generate a shape dictionary and a corresponding true shape coefficient vector that can effectively represent the source domain segmented anatomical shape from the source domain medical image data set through a dictionary learning method;

[0164] The overall segmentation network construction module 902 is used to construct an overall segmentation network composed of a source domain data segmentation branch network with dual consistency regularization and a random enhanced feature segmentation branch network; the source domain data segmentation branch network includes a domain invariant shape reference segmentation model DIMD based on U-Net, a shape reference mask generation module SRMG is added to DIMD to predict the first predicted shape coefficient vector of each medical image in the source domain, a first shape reference mask is generated based on the first predicted shape coefficient vector and the shape dictionary, and the first shape reference mask is cascaded with the features output by the U-Net decoder to generate the first predicted mask; the random enhanced feature segmentation branch network adds a random low-frequency segmentation model before the first layer of DIMD. The level feature enhancement module RLFA obtains three versions of random enhanced features for the source domain medical image features through a random convolution layer with random weights generated by the input noise z, and inputs the three random enhanced features into the U-Net encoder, and inputs the features output by the U-Net encoder into the shape reference mask generation module SRMG to obtain the second predicted shape coefficient vector, the third predicted shape coefficient vector and the fourth predicted shape coefficient vector, which are then combined with the shape dictionary to obtain the second shape reference mask, the third shape reference mask and the fourth shape reference mask, which are cascaded with the features output by the U-Net decoder to output the predicted second enhanced feature prediction mask, the third enhanced feature prediction mask and the fourth enhanced feature prediction mask;

[0165] The overall segmentation network training module 903 is used to train the overall segmentation network based on the source domain medical image dataset through consistency loss, adversarial domain enhancement strategy loss and overall shape coefficient loss to obtain a trained overall segmentation network;

[0166] The medical image segmentation module 904 is used to segment the medical image to be processed using the trained overall segmentation network.

[0167] The specific implementation of each module of a single-source domain generalized medical image segmentation device based on a shape dictionary is the same as the single-source domain generalized medical image segmentation method based on a shape dictionary, and the present embodiment will not repeat the description.

[0168] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. However, these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A single-source domain generalized medical image segmentation method based on shape dictionary, characterized in that: include: S1, shape dictionary learning step, generates a shape dictionary and a corresponding true shape coefficient vector that can effectively represent the source domain segmented anatomical shape from the source domain medical image dataset through a dictionary learning method; S2, an overall segmentation network construction step, constructing an overall segmentation network consisting of a source domain data segmentation branch network with dual consistency regularization and a random enhanced feature segmentation branch network; the source domain data segmentation branch network includes a domain invariant shape reference segmentation model DIMD based on U-Net, a shape reference mask generation module SRMG is added to DIMD to predict a first predicted shape coefficient vector of each medical image in the source domain, a first shape reference mask is generated based on the first predicted shape coefficient vector and a shape dictionary, and the first shape reference mask is cascaded with the features output by the U-Net decoder to generate a first prediction mask; The random enhanced feature segmentation branch network adds a random low-level feature enhancement module RLFA before the first layer of DIMD, obtains three versions of random enhanced features for source domain medical image features through a random convolution layer with random weights generated by input noise z, inputs the three random enhanced features into a U-Net encoder, and inputs the features output by the U-Net encoder into a shape reference mask generation module SRMG to obtain a second predicted shape coefficient vector, a third predicted shape coefficient vector, and a fourth predicted shape coefficient vector, which are then combined with a shape dictionary to obtain a second shape reference mask, a third shape reference mask, and a fourth shape reference mask, which are respectively cascaded with the features output by the U-Net decoder to output predicted second enhanced feature prediction masks, third enhanced feature prediction masks, and fourth enhanced feature prediction masks; S3, overall segmentation network training step, based on the source domain medical image dataset, the overall segmentation network is trained through consistency loss, adversarial domain enhancement strategy loss and overall shape coefficient loss to obtain a trained overall segmentation network; S4, the medical image segmentation step, uses the trained overall segmentation network to segment the medical image to be processed.

2. The single-source domain generalized medical image segmentation method based on shape dictionary according to claim 1, characterized in that: The optimization function in the shape dictionary generation process is expressed as follows: ； in, is the square of the L2 norm, used to calculate the Euclidean distance of vectors; is the L1 norm, which is used to enforce The sparsity of , represents the shape dictionary, ; , represents the true shape coefficient vector corresponding to the shape dictionary, represents the i-th true shape coefficient vector; represents the true segmentation mask of the i-th source domain medical image data; As a balance parameter; Represents the total number of medical image samples in the source domain medical image dataset; Represents the total number of explicit shape templates in the shape dictionary; represents a real number; Indicates height; Indicates width.

3. The single-source domain generalized medical image segmentation method based on shape dictionary according to claim 2 is characterized in that: The shape dictionary generation process is optimized by alternation, including: S11, fixed , updated by minimum angle regression LARS ,as follows: ； S12, fixed , updated by block coordinate descent ,as follows: ； S13, continuously iterate S11 and S12 until the convergence condition is met, and generate a shape dictionary that can effectively represent the segmented anatomical shape of the source domain and the corresponding true shape coefficient vector.

4. The single-source domain generalized medical image segmentation method based on shape dictionary according to claim 1, characterized in that: The source domain data segmentation branch network specifically includes a U-Net encoder, a shape reference mask generation module SRMG, a U-Net decoder, Concat() and a 1x1 convolution; the U-Net encoder takes the source domain medical image as input and outputs features to SRMG; SRMG predicts a first predicted shape coefficient vector for each medical image, the first predicted shape coefficient vector is used to generate a first shape reference mask in combination with a shape dictionary of each category, the first shape reference mask is Concat()ed with the output of the U-Net decoder to obtain features containing shape reference information, and then further refined through a 1x1 convolution to generate the final first predicted mask.

5. The single-source domain generalized medical image segmentation method based on shape dictionary according to claim 2, characterized in that: SRMG designs a separate fully connected layer for each category c , The shape dictionary of the generated category c The atomic vector As the weight initialization of the fully connected layer; , SRMG learns a specific first prediction shape coefficient vector for each class ,pass With shape dictionary Multiply to generate the first shape reference mask ,as follows: ； in, represents the first predicted shape coefficient vector of the kth explicit shape template; represents the k-th explicit shape template.

6. The single-source domain generalized medical image segmentation method based on shape dictionary according to claim 1, characterized in that: The random low-level feature enhancement module RLFA first inputs the source domain medical image into the random weight generation module RWGM before inputting the source domain medical image into the U-Net encoder; the random weight generation module RWGM has a convolutional layer with random weights, which realizes random enhancement of domain-specific low-level features of the image on the basis of keeping the domain-invariant global shape features unchanged by applying random filters.

7. The single-source domain generalized medical image segmentation method based on shape dictionary according to claim 6, characterized in that: Combine the enhanced feature vector obtained through the convolution layer with the original feature vector to obtain the final random enhanced feature ,as follows: ； in, Combine weights for features; represents the i-th source domain medical image; represents the original feature vector; represents the enhanced feature vector; .

8. The single-source domain generalized medical image segmentation method based on shape dictionary according to claim 7, characterized in that: Enhanced feature vector , which is expressed as follows: ； in, is a random convolution kernel.

9. The single-source domain generalized medical image segmentation method based on shape dictionary according to claim 1, characterized in that: Overall segmentation loss It is expressed as follows: ； ； ； ； in, represents the consistency loss; represents the adversarial domain enhancement strategy loss; represents the overall shape factor loss; , and is a custom weight coefficient; represent the second enhanced feature prediction mask, the third enhanced feature prediction mask and the fourth enhanced feature prediction mask respectively; represents the first prediction mask; denote a second shape reference mask, a third shape reference mask, and a fourth shape reference mask, respectively; represents a first shape reference mask; A ground-truth segmentation mask representing the source domain medical image data; Represents the true shape coefficient vector corresponding to the shape dictionary.

10. A single-source domain generalized medical image segmentation device based on shape dictionary, characterized in that: include: A shape dictionary learning module is used to generate a shape dictionary and a corresponding true shape coefficient vector that can effectively represent the source domain segmented anatomical shape from a source domain medical image dataset through a dictionary learning method; An overall segmentation network construction module is used to construct an overall segmentation network composed of a source domain data segmentation branch network with dual consistency regularization and a random enhanced feature segmentation branch network; the source domain data segmentation branch network includes a domain invariant shape reference segmentation model DIMD based on U-Net, a shape reference mask generation module SRMG is added to DIMD to predict a first predicted shape coefficient vector of each medical image in the source domain, a first shape reference mask is generated based on the first predicted shape coefficient vector and a shape dictionary, and the first shape reference mask is cascaded with the features output by the U-Net decoder to generate a first prediction mask; The random enhanced feature segmentation branch network adds a random low-level feature enhancement module RLFA before the first layer of DIMD, obtains three versions of random enhanced features for source domain medical image features through a random convolution layer with random weights generated by input noise z, inputs the three random enhanced features into a U-Net encoder, and inputs the features output by the U-Net encoder into a shape reference mask generation module SRMG to obtain a second predicted shape coefficient vector, a third predicted shape coefficient vector, and a fourth predicted shape coefficient vector, which are then combined with a shape dictionary to obtain a second shape reference mask, a third shape reference mask, and a fourth shape reference mask, which are respectively cascaded with the features output by the U-Net decoder to output predicted second enhanced feature prediction masks, third enhanced feature prediction masks, and fourth enhanced feature prediction masks; The overall segmentation network training module is used to train the overall segmentation network based on the source domain medical image dataset through consistency loss, adversarial domain enhancement strategy loss and overall shape coefficient loss to obtain a trained overall segmentation network; The medical image segmentation module is used to segment the medical image to be processed using the trained overall segmentation network.

Images