Image segmentation model training method, image segmentation method, device and apparatus

Abstract

The application discloses an image segmentation model training method, an image segmentation method, an image segmentation device and equipment. The model training method comprises the following steps: acquiring a sample image; inputting the sample image into a style transformation module to output a second source domain image after style transformation and a second target domain image after style transformation; inputting a first source domain image, the second source domain image, a first target domain image and the second target domain image into a segmentation network to output a predicted first source domain image segmentation map, a second source domain image segmentation map, a first target domain image segmentation map and a second target domain image segmentation map; and using semantic information of the source domain to constrain the first source domain image segmentation map and the second source domain image segmentation map and / or using semantic information of the target domain to constrain the first target domain image segmentation map and the second target domain image segmentation map to train an image segmentation model. The training method of the image segmentation model improves the model performance and avoids model overfitting.

Description

Technical Field

[0001] This application belongs to the field of image processing technology, and in particular relates to an image segmentation model training method, image segmentation method, apparatus and equipment. Background Technology

[0002] In the field of medical image segmentation, achieving more precise localization and quantification of patient body structures or lesion areas through deep learning technology is crucial for ensuring accurate diagnosis and developing personalized treatment plans. With the significant achievements of deep neural networks in medical image segmentation tasks, especially when the test set data and training set data are distributed in a consistent manner, their performance has reached professional human levels.

[0003] However, medical image data often includes multimodal data acquired from different medical centers using instruments with varying parameter settings, and even employing different imaging techniques such as X-rays, ultrasound, CT scans, and MRI. This leads to a significant drop in model performance in real-world applications when the distribution of the training set (source domain) differs from that of the test set (target domain). Currently, the most direct approach to address this issue is to annotate the target domain image data with pixel-level labels, and then fine-tune the data using images with target domain style and source domain images to adapt to the target domain's data distribution. However, this method requires considerable time and professional medical guidance to generate the labeled data, making it inefficient.

[0004] To address this issue, cross-domain medical image segmentation algorithms based on Unsupervised Domain Adaptation (UDA) have gradually attracted widespread attention. Unsupervised domain adaptation refers to training a model on a source domain with labeled data available for training, enabling the model to adapt to a target domain with limited or no labeled data. Compared to methods requiring labels, UDA can complete pixel-level annotation of the target domain image using only the source domain image, its pixel-level labels, and the target domain image, thus avoiding the tedious process of manual annotation.

[0005] Current unsupervised domain adaptation techniques primarily utilize Generative Adversarial Networks (GANs) to transform source domain images into images with target domain style while preserving anatomical information. Then, by combining these transformed images with pixel-level labels corresponding to the source domain images and training a convolutional neural network, a segmentation model adapted to the target domain is obtained. Summary of the Invention

[0006] This application provides an image segmentation model training method, image segmentation method, apparatus and device, which aims to solve the problems of poor adaptive ability of existing models affecting image segmentation results and easy overfitting due to too many model parameters.

[0007] In a first aspect, this application provides an image segmentation model training method, wherein the image segmentation model includes a style transformation module and a segmentation network, and the method includes:

[0008] Acquire sample images; the sample images include labeled first source domain images and unlabeled first target domain images in multiple modalities;

[0009] The sample image is input into the style transformation module to output the style-transformed second source domain image and the style-transformed second target domain image;

[0010] The first source domain image, the second source domain image, the first target domain image, and the second target domain image are input into the segmentation network to output the predicted first source domain image segmentation map, the second source domain image segmentation map, the first target domain image segmentation map, and the second target domain image segmentation map;

[0011] The image segmentation model is trained by constraining the first source domain image segmentation map and the second source domain image segmentation map using semantic information of the source domain and / or constraining the first target domain image segmentation map and the second target domain image segmentation map using semantic information of the target domain.

[0012] Secondly, this application provides an image segmentation method, the method comprising:

[0013] Acquire the medical image to be segmented;

[0014] The medical image is input into a trained image segmentation model for prediction to obtain a segmentation map corresponding to the medical image to be segmented; wherein, the image segmentation model is obtained based on the image segmentation model training method described above.

[0015] Thirdly, this application provides an image segmentation model training apparatus, the apparatus comprising:

[0016] The first acquisition module is used to acquire sample images; the sample images include a first source domain image with labels and a first target domain image without labels in multiple modalities;

[0017] The model training module inputs the sample images into the style transformation module to output a style-transformed second source domain image and a style-transformed second target domain image; and inputs the first source domain image, the second source domain image, the first target domain image, and the second target domain image into the segmentation network to output predicted first source domain image segmentation maps, second source domain image segmentation maps, first target domain image segmentation maps, and second target domain image segmentation maps; it is also used to constrain the first source domain image segmentation maps and the second source domain image segmentation maps using semantic information of the source domain and / or constrain the first target domain image segmentation maps and the second target domain image segmentation maps using semantic information of the target domain, so as to train the image segmentation model.

[0018] Fourthly, this application provides an image segmentation apparatus, the apparatus comprising:

[0019] The second acquisition module is used to acquire the medical image to be segmented;

[0020] The model invocation module is used to input the medical image into a trained image segmentation model for prediction;

[0021] The results output module is used to obtain the segmentation map corresponding to the medical image to be segmented.

[0022] Fifthly, this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the above-described image segmentation model training method or the above-described image segmentation method.

[0023] Compared with existing technologies, this application presents an image segmentation model based on a style transformation module and a segmentation network. This model employs style transformation to significantly perturb the style of the image samples used in training the model, thereby eliminating the influence of image style on the model and improving its adaptability. Furthermore, it utilizes semantic information from both the source and target domains to supervise and constrain the prediction results, ultimately enabling the model to achieve high-precision cross-domain segmentation. In addition, this image segmentation model contains only one segmentation network that shares parameters in both the source and target domains, resulting in a smaller number of parameters and effectively avoiding overfitting. Attached Figure Description

[0024] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a schematic diagram of the main flowchart of the image segmentation model training method provided in the embodiments of this application;

[0026] Figure 2 for Figure 1 The diagram shows a sub-process of the embodiment shown.

[0027] Figure 3 for Figure 2 The diagram shows a sub-process of the embodiment shown.

[0028] Figure 4 for Figure 1 The diagram shows a sub-process of the embodiment shown.

[0029] Figure 5 for Figure 1 The diagram shows a sub-process of the embodiment shown.

[0030] Figure 6 for Figure 1 The diagram shows a sub-process of the embodiment shown.

[0031] Figure 7 This is a schematic diagram of the main flowchart of the image segmentation method provided in the embodiments of this application;

[0032] Figure 8 This is a structural block diagram of the image segmentation model training device provided in the embodiments of this application;

[0033] Figure 9 This is a structural block diagram of the image segmentation apparatus provided in the embodiments of this application;

[0034] Figure 10 A structural block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0035] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0036] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0037] Please see Figures 1 to 6 , Figure 1This illustration shows a schematic diagram of the main flow of an image segmentation model training method provided by an embodiment of this application. This image segmentation model training method is applied in the field of image segmentation, such as... Figure 1 As shown, in one embodiment of an image segmentation model training method, the image segmentation model includes a style transformation module and a segmentation network. The training method of the model includes the following steps S100-S400:

[0038] S100. Acquire sample images; the sample images include labeled first source domain images and unlabeled first target domain images in multiple modalities;

[0039] Training an image segmentation model first requires acquiring sample images, which can be divided into a source domain and a target domain. The source domain typically refers to the area with a large amount of labeled data available for training, while the target domain may be the area where we want to apply the model but where there is little or no labeled data. In this embodiment, the purpose of the image segmentation model is to segment target domain images of different modalities. For example, in the field of medical image segmentation, the acquired medical image samples come from different medical centers or are obtained using instruments with different parameter settings, such as cross-modal cardiac substructure segmentation datasets, cross-modal brain tumor segmentation datasets, and cross-center prostate segmentation datasets.

[0040] S200. Input the sample image into the style transformation module to output the style-transformed second source domain image and the style-transformed second target domain image;

[0041] Image samples from multiple modalities are acquired, each with its own image style. The style transformation module is used to perform style transformation processing on the input image samples. Specifically, the source domain image x... s and target domain image x t Both images undergo a style transformation module to obtain the style-transformed source domain image x. ′s and the style-transformed target domain image x ′t .

[0042] In one exemplary embodiment, such as Figure 4 As shown, step S200 specifically includes steps S201 to S202:

[0043] S201. The sample image is randomly divided into several image parts;

[0044] The style transformation module employs shuffle remapping. First, the distribution of the original image x is normalized to [-1, 1]. Then, some control points are randomly generated between [-1, 1], with endpoints P0 and P1. NThe control points are fixed at -1 and 1 respectively. These control points randomly divide the image distribution into N parts. Normalization methods can be Min-Max normalization (linear normalization) and Z-score standardization (standard normalization), and the specific method can be adjusted according to actual needs. These normalization methods can convert the original data to [-1, 1] for subsequent data processing and analysis.

[0045] S202. Randomly map the plurality of image parts to each other to obtain a second source domain image after style transformation and a second target domain image after style transformation.

[0046] The transformed image x is obtained by randomly linearly mapping the N parts of step S201 to each other. ′ Including the style-transformed source domain image x ′s and the style-transformed target domain image x ′t .

[0047] Formula (1) shows that for a given pixel, the range (P) i P i+1 ) to range (P j P j+1 Remapping of )

[0048] x ′ =(xp i ) / (p i+1 -p i )*(p j+1 -p j )+p j (1);

[0049] Where x represents the original image before style transformation, x ′ This represents the image after style transformation.

[0050] S300: Input the first source domain image, the second source domain image, the first target domain image, and the second target domain image into the segmentation network to output the predicted first source domain image segmentation map, the second source domain image segmentation map, the first target domain image segmentation map, and the second target domain image segmentation map.

[0051] In this embodiment, the source domain image x s Target domain image x t The source domain image x after style transformation ′s and the style-transformed target domain image x ′t Simultaneously input into the segmentation network, the segmentation network outputs four segmentation probability maps: source domain image segmentation map p s Target domain image segmentation map p t The source domain image segmentation map p after style transformation′s and the target domain image segmentation map p after style transformation ′t Four segmentation probability maps are generated using semantic information constraints from the source and target domains. Image segmentation is the process of dividing an image into multiple regions or segments, each with similar properties. An image segmentation map represents these segmented regions; it is typically an image of the same size as the original image, but each pixel value represents its region or category.

[0052] S400. Constrain the first source domain image segmentation map and the second source domain image segmentation map using semantic information of the source domain and / or constrain the first target domain image segmentation map and the second target domain image segmentation map using semantic information of the target domain, so as to train the image segmentation model.

[0053] Semantic information typically refers to high-level information related to the content of an image, such as its label, location, or attributes. In image segmentation tasks, semantic information can be used to guide or constrain the segmentation process, ensuring that the segmentation result is consistent with the expected semantic category. This helps the model better adapt to the target domain and improves its segmentation performance in that domain.

[0054] In one exemplary embodiment, such as Figure 6 As shown, step S400 specifically includes steps S401 to S402:

[0055] S401. Constrain the authenticity of the output first source domain image segmentation map and second source domain image segmentation map based on the first loss function and the label of the first source domain image, and constrain the consistency between the first target domain image segmentation map and the second target domain image segmentation map based on the second loss function.

[0056] In this embodiment, the Dice loss function is used in the source domain in conjunction with the source domain label y. s Supervised and constrained source domain image segmentation probability map p s and style-transformed source domain image segmentation map p ′s As a result, Equation (2) shows the Dice loss function:

[0057]

[0058] Where y and p represent the one-hot format of the real image label and the predicted segmentation map, respectively.

[0059] In one specific implementation, a semantic consistency loss constraint p is designed in the target domain. t and p ′t The results are consistent, and Equation (3) shows the semantic consistency loss:

[0060] Lsec =L dice (p t ,p ′t (3);

[0061] Where, p t p represents the target domain image segmentation map. ′t This represents the target domain image segmentation map after style transformation.

[0062] Understandably, in real-world applications, model performance degrades significantly when there are substantial differences in the distribution of the training set (source domain) and the test set (target domain). By employing a style transformation function to significantly perturb the styles of these different modalities and utilizing the semantic information of both the source and target domains to supervise and constrain the prediction results, the influence of image style on model training can be effectively avoided, achieving accurate image segmentation.

[0063] In one exemplary embodiment, the segmentation network is a three-dimensional U-shaped network. The three-dimensional segmentation network includes an encoder and a decoder. The encoder includes three downsampling modules and is connected to the decoder via jump links.

[0064] Specifically, the 3D U-shaped network is a deep learning network architecture particularly well-suited for processing three-dimensional (3D) image data. It is itself a convolutional neural network (CNN), consisting of two parts: an encoder (also known as a downsampling path) and a decoder (also known as an upsampling path).

[0065] The encoder is used to extract features from the input image. It typically consists of a series of convolutional layers, pooling layers, and activation functions. As the network deepens, the resolution of the feature maps gradually decreases, but the number of features gradually increases. In the encoder part, the spatial size of the feature maps is gradually reduced through multiple convolutional and pooling layers, while the number of channels in the feature maps is increased. Each convolutional layer uses an N*N convolutional kernel, and an activation function is used for a non-linear transformation after convolution. Pooling layers are used to reduce the spatial size of the feature maps, typically employing max pooling.

[0066] The decoder maps the features extracted by the encoder back to the spatial dimensions of the original image. This is achieved through upsampling (e.g., using transposed convolutions or upsampling layers) and convolution operations, gradually restoring the spatial details of the image. In the decoder section, the spatial dimensions of the feature maps are gradually restored through upsampling and convolutional layers. Upsampling layers typically employ transposed convolutions or upsampling operations to enlarge the feature map size to the same size as the corresponding level's feature map in the encoder. Then, skip connections are used to concatenate the feature maps from the corresponding levels in the encoder with those in the decoder to fuse feature information at different scales. Finally, convolutional layers further process and transform the fused feature map.

[0067] In a specific application scenario, the image size input to the 3D U-shaped network is 128*128*128. The feature map sizes generated by the 3D U-shaped network at each scale are 64*64*64*16, 32*32*32*32 and 16*16*16*64 (length*width*height*number of channels).

[0068] Existing image segmentation models based on the UDA framework are two-dimensional, failing to extract complete three-dimensional spatial information. This results in poor consistency of segmentation results along the slice stacking direction. Furthermore, the complexity of existing two-dimensional frameworks means that extending them to three-dimensional frameworks leads to a significant increase in parameters, easily causing overfitting. Three-dimensional U-shaped networks are specifically designed for processing three-dimensional image data, particularly useful in processing medical images (such as CT and MRI scans), as these images are inherently three-dimensional and require consideration of both spatial and temporal information. They can capture more contextual information in three-dimensional data, thus providing more accurate segmentation results.

[0069] S402. The parameters of the segmentation network are optimized by minimizing the first loss function and the second loss function until the set conditions are met.

[0070] Optimization algorithms such as gradient descent are used to gradually reduce the loss function, continuously updating the parameters of the segmentation network in the image segmentation model until the training reaches the preset number of iterations or until the segmentation network converges. This image segmentation model contains only one segmentation network that shares parameters in the source and target domains, resulting in a small number of parameters and effectively avoiding overfitting.

[0071] In one exemplary embodiment, such as Figure 5 As shown, after model training, the model's performance also needs to be tested, which includes steps S510 to S530 after step S400:

[0072] S510, Obtain the test image;

[0073] S520. Input the test image into the image segmentation model for testing to obtain a segmentation map of the test image;

[0074] S530. Based on the comparison results between the segmentation map of the test image and the label of the test image, determine whether to continue training the image segmentation model.

[0075] In one exemplary embodiment, such as Figure 2 As shown, after acquiring the sample image, preprocessing can be performed on the sample image, that is, after step S100, steps S110 to S130 are also included:

[0076] S110. Adjust the sample image.

[0077] In one exemplary embodiment, such as Figure 3 As shown, step S110 may also include steps S111 and S112.

[0078] S111, The orientation of the sample image is redirected and adjusted.

[0079] In this implementation, the orientation of all images is redirected to RAI (Right to Left, Anterior to Posterior, Inferior to Superior, i.e., from right to left, from front to back, and from bottom to top).

[0080] S112. Adjust the window width and window level of the sample image.

[0081] In this implementation, the window width and window level of images of some modalities are adjusted according to the required segmentation structure; for example, the window width and window level of CT modal images are adjusted according to the required segmentation structure, while MRI modal images are not processed here.

[0082] It is known that adjusting the window width and window level can alter the contrast and brightness of an image. A wider window displays more grayscale levels, resulting in lower contrast and revealing more image details. A narrower window displays fewer grayscale levels, resulting in higher contrast, suitable for highlighting structures with higher or lower density tissues. Adjusting the window level changes the image's brightness level; increasing the window level makes the image darker, while decreasing it makes it brighter. In examinations such as X-rays, CT scans, and MRI scans, depending on the purpose of the examination and the tissues or lesions to be observed, appropriate window width and window level need to be selected to optimize image quality and improve diagnostic accuracy.

[0083] S120. The adjusted sample image is cropped to retain the key areas of the image sample.

[0084] In this embodiment, the adjusted sample image is cropped to reduce the influence of background pixels.

[0085] S130. Resample the cropped sample image to the set size.

[0086] In this embodiment, linear interpolation is used to resample the depth, height, and width of all sample images obtained after cropping in step S130 to a uniform size, for example, 128*128*128. Since the cropped images have inconsistent sizes, it is necessary to unify the image sizes to facilitate subsequent image processing.

[0087] Preprocessing the acquired sample images can help eliminate noise and redundant information, highlighting useful, real-world information. This contributes to improving the accuracy of subsequent image analysis, feature extraction, image segmentation, matching, and recognition.

[0088] Based on the application of the image segmentation model obtained by the training method of the above-mentioned image segmentation model, this application also provides an image segmentation method, such as... Figure 7 As shown, it includes steps S10 to S20:

[0089] S10. Obtain the medical image to be segmented;

[0090] S20. Input the medical image into the trained image segmentation model for prediction to obtain the segmentation map corresponding to the medical image to be segmented.

[0091] The trained image segmentation model is obtained based on the image segmentation model training method described above.

[0092] The performance of the image segmentation model in this application is illustrated below with experimental data. The dataset used is the 2017 Multi-Modality Whole Heart Segmentation Challenge (MMWHS17), which was validated on a cross-modal cardiac substructure segmentation task from CT to MR. Four-fifths of the images were used as the training dataset, and one-fifth were used as the test dataset. An unsupervised domain-adaptive image segmentation model based on a style transformation module and a 3D U-shaped network was constructed. The segmented substructures include the ascending aorta (AA), the left atrial blood cavity (LAC), the left ventricular blood cavity (LVC), and the myocardium of the left ventricle (Myo). The Dice coefficient and average surface distance (ASD) were used as indicators, and the results are shown in Table 1.

[0093] Table 1. Performance comparison of different methods on the CT-to-MR cross-modal cardiac substructure segmentation task.

[0094]

[0095]

[0096] As can be seen from Table 1, the image segmentation model obtained based on the image segmentation model training method proposed in this application can achieve higher accuracy in cross-modal cardiac substructure segmentation compared with other existing technologies.

[0097] like Figure 8 As shown, embodiments of this application also provide an image segmentation model training device 150, which includes a first acquisition module 151 and a model training module 152. The first acquisition module 151 is used to acquire sample images; the sample images include first source domain images with labels and first target domain images without labels under multiple modalities; the model training module 152 is used to input the sample images into the style transformation module to output style-transformed second source domain images and style-transformed second target domain images; and to input the first source domain image, the second source domain image, the first target domain image, and the second target domain image into the segmentation network to output predicted first source domain image segmentation maps, second source domain image segmentation maps, first target domain image segmentation maps, and second target domain image segmentation maps; the model training module 152 is also used to constrain the first source domain image segmentation maps and the second source domain image segmentation maps using semantic information of the source domain and / or constrain the first target domain image segmentation maps and the second target domain image segmentation maps using semantic information of the target domain, so as to train the image segmentation model.

[0098] In one exemplary embodiment, the apparatus further includes a preprocessing module for adjusting the sample image; cropping the adjusted sample image to retain key regions of the image sample; and resampling the cropped sample image to a set size.

[0099] In one exemplary embodiment, the preprocessing module further includes a pre-adjustment sub-module, which is used to redirect the orientation of the sample image; and / or adjust the window width and window level of the sample image.

[0100] In an exemplary embodiment, the style transformation module includes a splitting submodule and a mapping submodule. The splitting submodule is used to randomly divide the sample image into several image parts; the mapping submodule is used to randomly map the several image parts to each other to obtain a style-transformed second source domain image and a style-transformed second target domain image.

[0101] In an exemplary embodiment, the model training module further includes a loss function submodule and a parameter optimization submodule. The loss function submodule is used to constrain the authenticity of the output first source domain image segmentation map and second source domain image segmentation map based on a first loss function and the label of the first source domain image, and to constrain the consistency between the first target domain image segmentation map and the second target domain image segmentation map based on a second loss function. The parameter optimization module is used to optimize the parameters of the segmentation network by minimizing the first loss function and the second loss function until the set conditions are met.

[0102] like Figure 9 As shown, embodiments of this application also provide an image segmentation apparatus 160, which includes a second acquisition module 161, a model invocation module 162, and a result output module 163. The second acquisition module 161 is used to acquire a medical image to be segmented; the model invocation module 162 is used to input the medical image into a trained image segmentation model for prediction; and the result output module 163 is used to acquire the segmentation map corresponding to the medical image to be segmented.

[0103] like Figure 10 As shown, this application embodiment also provides an electronic device 140, including a processor 141, a memory 142, a non-volatile memory 144, and a computer program 1441 stored in the non-volatile memory 144 and executable on the processor. When the processor 141 executes the computer program 1441, it implements the image segmentation model training method or any embodiment of the image segmentation method described above. Specifically, it also includes an input / output interface 145 and an input / output device 146 connected thereto. The processor 141, memory 142, non-volatile memory 144, and input / output interface 145 are connected via an internal bus 143.

[0104] It should be understood that, in the embodiments of this application, the processor may be a Central Processing Unit (CPU), but it may also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0105] This application also proposes a storage medium, in an exemplary embodiment, which stores at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, at least one program, code set, or instruction set, when executed by the processor of a computer device, implements the image segmentation model training method of any of the above embodiments or implements the image segmentation method of any of the above embodiments.

[0106] Compared to existing technologies, this application's image segmentation model, built upon a style transformation module and a segmentation network, significantly perturbs the style of image samples used in training the model through style transformation, thereby eliminating the influence of image style on the model and improving its adaptability. Furthermore, it utilizes semantic information from both the source and target domains to supervise and constrain the prediction results, ultimately enabling the model to achieve high-precision cross-domain segmentation. Moreover, this image segmentation model contains only one segmentation network that shares parameters in both the source and target domains, resulting in a smaller number of parameters and effectively avoiding overfitting. Additionally, preprocessing the acquired sample images helps eliminate noise and redundant information, highlighting useful real-world information.

[0107] It will be understood by those skilled in the art that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program may be stored in a storage medium, which is a computer-readable storage medium. The computer program is executed by at least one processor in the computer system to implement the process steps of the embodiments of the above methods.

[0108] It is understood that this application has been described through some embodiments, and those skilled in the art will recognize that various changes or equivalent substitutions can be made to these features and embodiments without departing from the spirit and scope of this application. Furthermore, based on the teachings of this application, these features and embodiments can be modified to adapt to specific situations and materials without departing from the spirit and scope of this application. Therefore, this application is not limited to the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the protection scope of this application.

Claims

1. A method for training an image segmentation model, characterized in that, The image segmentation model includes a style transformation module and a segmentation network, and the method includes: Acquire sample images; the sample images include labeled first source domain images and unlabeled first target domain images in multiple modalities; The orientation of the sample image is redirected and adjusted; and / or the window width and window level are adjusted. The adjusted sample image is cropped to preserve the key regions of the sample image; The cropped sample image is resampled to the set size; The sample image is input into the style transformation module to output a style-transformed second source domain image and a style-transformed second target domain image. The style transformation uses shuffling and remapping, and control points are randomly generated between [-1,1]. The control points randomly divide the sample image into several image parts. The several image parts are randomly mapped to each other to obtain the style-transformed second source domain image and the style-transformed second target domain image. The first source domain image, the second source domain image, the first target domain image, and the second target domain image are input into the segmentation network to output the predicted first source domain image segmentation map, the second source domain image segmentation map, the first target domain image segmentation map, and the second target domain image segmentation map; The image segmentation model is trained by constraining the first source domain image segmentation map and the second source domain image segmentation map using semantic information from the source domain and / or constraining the first target domain image segmentation map and the second target domain image segmentation map using semantic information from the target domain. This includes: constraining the authenticity of the output first source domain image segmentation map and the second source domain image segmentation map based on a first loss function and the labels of the first source domain images; constraining the consistency between the first target domain image segmentation map and the second target domain image segmentation map based on a second loss function; and optimizing the parameters of the segmentation network by minimizing the first loss function and the second loss function until the set conditions are met.

2. The method according to claim 1, characterized in that, After training the image segmentation model by constraining the first source domain image segmentation map and the second source domain image segmentation map using semantic information of the source domain and / or constraining the first target domain image segmentation map and the second target domain image segmentation map using semantic information of the target domain, the method further includes: Obtain the test image; The test image is input into the image segmentation model for testing to obtain a segmentation map of the test image; The decision to continue training the image segmentation model is made based on the comparison between the segmentation map of the test image and the label of the test image.

3. An image segmentation method, characterized in that, The method includes: Acquire the medical image to be segmented; The medical image is input into a trained image segmentation model for prediction to obtain a segmentation map corresponding to the medical image to be segmented; wherein the image segmentation model is obtained based on the image segmentation model training method according to any one of claims 1 to 2.

4. An image segmentation model training device, characterized in that, The device includes: The first acquisition module is used to acquire sample images; the sample images include a first source domain image with labels and a first target domain image without labels in multiple modalities; The preprocessing module is used to redirect and adjust the orientation of the sample image; and / or adjust the window width and window level; crop the adjusted sample image to retain the key areas of the image sample; and resample the cropped sample image to a set size. The model training module is used to input the sample image into the style transformation module to output a style-transformed second source domain image and a style-transformed second target domain image. The style transformation employs shuffling and remapping, randomly generating control points between [-1, 1]. These control points randomly divide the sample image into several image parts. These image parts are then randomly mapped to each other to obtain the style-transformed second source domain image and the style-transformed second target domain image. The first source domain image, the second source domain image, the first target domain image, and the second target domain image are input into a segmentation network to output predicted segmentation maps for the first source domain image, the second source domain image, and the first target domain image. The second target domain image segmentation map is further used to constrain the first source domain image segmentation map and the second source domain image segmentation map using semantic information of the source domain and / or to constrain the first target domain image segmentation map and the second target domain image segmentation map using semantic information of the target domain, so as to train the image segmentation model, including: constraining the authenticity of the output first source domain image segmentation map and the second source domain image segmentation map based on a first loss function and the label of the first source domain image; constraining the consistency between the first target domain image segmentation map and the second target domain image segmentation map based on a second loss function; optimizing the parameters of the segmentation network by minimizing the first loss function and the second loss function until the set conditions are met.

5. An image segmentation apparatus, characterized in that, The apparatus is applied to the image segmentation method as described in claim 3, comprising: The second acquisition module is used to acquire the medical image to be segmented; The model invocation module is used to input the medical image into a trained image segmentation model for prediction; The results output module is used to obtain the segmentation map corresponding to the medical image to be segmented.

6. An electronic device, characterized in that, The electronic device includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, it implements the image segmentation model training method as described in any one of claims 1 to 2 or the image segmentation method as described in claim 3.

Images