Medical image segmentation method and device, equipment and storage medium

Abstract

Disclosed are a medical image segmentation method, apparatus and device, and a storage medium, which belong to the technical field of image segmentation, the method comprising: adding a data enhancement module and a feature fusion module in a first U-Net network to obtain a second U-Net network; training a second U-Net network by adopting the medical images in the training set; after the training of the second U-Net network is completed, removing the data enhancement module and the feature fusion module to obtain a trained first U-Net network, and inputting the medical image in the test set into the trained first U-Net network for medical image segmentation to obtain a second U-Net network; the medical image in the test set and the medical image in the training set belong to the medical images of the first organ in different domains. According to the method, the accuracy degree of cross-domain medical image segmentation can be improved.

Description

Technical Field

[0001] The present disclosure relates to the field of image segmentation technology, and in particular to a medical image segmentation method, device, equipment and storage medium. Background Art

[0002] Medical images are used to indicate images produced by various medical instruments. Medical image segmentation refers to the act of segmenting parts of a medical image according to a certain goal. For example, if the goal is to segment an organ with a lesion, then medical image segmentation is used to segment the organ with the lesion from the medical image.

[0003] Medical images of different categories or modalities are also medical images of different domains. Segmenting medical images of different domains is called cross-domain medical image segmentation.

[0004] In the related art, the U-Net network is usually used for cross-domain medical image segmentation. U-Net includes an encoder and a decoder. The encoder gradually reduces the spatial dimension of the input image through a series of convolutional layers and maximum pooling layers, and extracts and learns important features in the image in this process. The decoder gradually restores the spatial dimension of the image through upsampling operations and convolutional layers. There is also a jump connection between the encoder and the decoder. The jump connection directly connects the feature map in the encoder to the corresponding layer of the decoder. Through the jump connection, the decoder can refer to the early features during the upsampling process, thereby making up for the details that may be ignored by the traditional deep learning model.

[0005] However, the U-Net network trained using traditional methods has low accuracy when performing cross-domain medical image segmentation. Summary of the invention

[0006] The present disclosure provides a medical image segmentation method, apparatus, device and storage medium, which can improve the accuracy of cross-domain medical image segmentation. The technical solution at least includes the following solutions:

[0007] In a first aspect, a medical image segmentation method is provided, comprising: adding a data enhancement module and a feature fusion module to a first U-Net network to obtain a second U-Net network; training the second U-Net network using medical images in a training set; after the training of the second U-Net network is completed, removing the data enhancement module and the feature fusion module to obtain the trained first U-Net network, and inputting the medical images in a test set into the trained first U-Net network for medical image segmentation, wherein the medical images in the test set and the medical images in the training set are medical images of a first organ in different domains; wherein, in the second U-Net network, an encoder and a decoder in the same layer are connected through a feature fusion module, the data enhancement module is connected to the encoder of the first layer, the data enhancement module is used to perform data enhancement on the first image input into the medical image segmentation model to obtain a second image, and the first image and the second image are respectively input into the encoder of the first layer to obtain two encoding features of the first layer, and the feature fusion module is used to fuse the two encoding features input into the feature fusion module to obtain a fused feature.

[0008] Optionally, in the second U-Net network, the encoder in the lth layer is used to output two encoding features of the lth layer, and the two encoding features of the lth layer are respectively input into the feature fusion module in the lth layer and the encoder in the l+1th layer; the feature fusion module in the lth layer is used to perform feature fusion on the two encoding features of the lth layer to obtain the fused features of the lth layer, and input the fused features of the lth layer into the decoder in the lth layer; the decoder in the lth layer is used to generate the decoding features of the lth layer based on the fused features of the lth layer; wherein l is an integer, l is greater than 0 and l is less than L, and L is the number of layers of the second U-Net network.

[0009] Optionally, the coding feature corresponding to the first image is the first coding feature, and the coding feature corresponding to the second image is the second coding feature. The feature fusion module in the lth layer is used to implement feature fusion of the two coding features of the lth layer in the following manner to obtain the fused feature of the lth layer: obtain the positive attention feature of the first coding feature of the lth layer on the second coding feature of the lth layer; obtain the reverse attention feature of the second coding feature of the lth layer on the first coding feature of the lth layer; and splice the positive attention feature and the reverse attention feature to obtain the fused feature of the lth layer.

[0010] Optionally, the positive attention feature is calculated using the following formula:

[0011]

[0012] The reverse attention feature is calculated using the following formula:

[0013]

[0014] in, is the positive attention feature of layer l, The reverse attention feature of layer l, is the first coding feature of the lth layer, is the second encoding feature of the lth layer, Softmax() is the normalized exponential function, d l is the feature dimension of the lth layer.

[0015] Optionally, the data enhancement module is used to perform data enhancement on the first image input into the medical image segmentation model to obtain a second image, including: the data enhancement module is used to implement data enhancement on the first image by using any one of random cropping, random horizontal flipping, and random vertical flipping.

[0016] In a second aspect, a medical image segmentation device is also provided, including: an acquisition module, used to add a data enhancement module and a feature fusion module to a first U-Net network to obtain a second U-Net network; a training module, used to train the second U-Net network using medical images in a training set; a testing module, used to remove the data enhancement module and the feature fusion module after the training of the second U-Net network is completed, to obtain the trained first U-Net network, and input the medical images in the test set into the trained first U-Net network for medical image segmentation, wherein the medical images in the test set and the medical images in the training set are medical images of the first organ in different domains;

[0017] Among them, in the second U-Net network, the encoder and decoder at the same layer are connected through a feature fusion module, the data enhancement module is connected to the encoder of the first layer, the data enhancement module is used to perform data enhancement on the first image input into the medical image segmentation model to obtain the second image, and the first image and the second image are respectively input into the encoder of the first layer to obtain two encoding features of the first layer, and the feature fusion module is used to fuse the two encoding features input into the feature fusion module to obtain a fused feature.

[0018] Optionally, the coding feature corresponding to the first image is the first coding feature, and the coding feature corresponding to the second image is the second coding feature, and the acquisition module is further used to obtain the positive attention feature of the first coding feature of the lth layer on the second coding feature of the lth layer; obtain the reverse attention feature of the second coding feature of the lth layer on the first coding feature of the lth layer; and splice the positive attention feature and the reverse attention feature to obtain the fusion feature of the lth layer.

[0019] Optionally, the acquisition module is further used to implement data enhancement on the first image by adopting any one of random cropping, random horizontal flipping and random vertical flipping.

[0020] In a third aspect, a computer device is also provided, comprising: a memory and a processor, wherein at least one computer program is stored in the memory, and the at least one computer program is loaded and executed by the processor, thereby executing the medical image segmentation method described in the above embodiment.

[0021] In a fourth aspect, a computer-readable storage medium is also provided, wherein at least one computer program is stored in the computer-readable storage medium, and the at least one computer program is loaded and executed by a processor to execute the medical image segmentation method described in the above embodiment.

[0022] In a fifth aspect, a computer program product is provided, comprising a computer program / instruction, wherein when the computer program / instruction is executed by a processor, the method described in the first aspect is implemented.

[0023] The beneficial effects brought by the technical solution provided by the embodiments of the present disclosure include at least:

[0024] In the disclosed embodiment, data enhancement is performed on the first image through a data enhancement module to obtain a second image. Since the second image has been data enhanced, it can be regarded as an image belonging to a different domain of the first organ than the first image. In this way, the first image and the second image are simultaneously input into the second U-Net network and trained during the training process, which can enhance the sensitivity of the second U-Net network when segmenting medical images of the first organ in different domains, and improve the accuracy of the second U-Net network when segmenting medical images of the first organ in different domains. Thereby, the first U-Net network that is finally trained accurately segments the medical images of the first organ in different domains. Compared with the traditional method of training U-Net networks, the U-Net network trained by the method in the disclosed embodiment has a higher accuracy rate in the cross-domain medical image segmentation of the same organ. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0026] Figure 1 A flowchart of a medical image segmentation method provided by an exemplary embodiment of the present disclosure is shown;

[0027] Figure 2 A flowchart of a medical image segmentation method provided by another exemplary embodiment of the present disclosure is shown;

[0028] Figure 3 It is a schematic diagram of the structure of the second U-Net network;

[0029] Figure 4 It is a structural diagram of the feature fusion module;

[0030] Figure 5 A schematic diagram of the structure of a medical image segmentation device provided by an exemplary embodiment of the present disclosure is shown;

[0031] Figure 6 It is a schematic diagram of the structure of a computer device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0032] Unless otherwise defined, the technical or scientific terms used herein shall have the usual meanings understood by persons of ordinary skill in the field to which the present disclosure belongs. The words "first", "second", "third" and similar words used in the patent application specification and claims of the present disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. Similarly, words such as "one" or "one" do not indicate a quantitative limitation, but rather indicate the presence of at least one. Words such as "include" or "comprise" and similar words mean that the elements or objects appearing before "include" or "comprise" cover the elements or objects listed after "include" or "comprise" and their equivalents, and do not exclude other elements or objects. Words such as "connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect.

[0033] In order to make the objectives, technical solutions and advantages of the present disclosure more clear, the embodiments of the present disclosure will be further described in detail below with reference to the accompanying drawings.

[0034] Figure 1A flowchart of a medical image segmentation method provided by an exemplary embodiment of the present disclosure is shown, and the method can be executed by a computer device. Figure 1 , the method comprising:

[0035] In step 101, a data enhancement module and a feature fusion module are added to a first U-Net network to obtain a second U-Net network.

[0036] Among them, in the second U-Net network, the encoder and decoder at the same layer are connected through a feature fusion module, and the data enhancement module is connected to the encoder of the first layer. The data enhancement module is used to perform data enhancement on the first image input into the medical image segmentation model to obtain the second image, and the first image and the second image are respectively input into the encoder of the first layer to obtain two encoding features of the first layer. The feature fusion module is used to fuse the two encoding features input into the feature fusion module to obtain a fused feature.

[0037] Here, the first U-Net network is a traditional U-Net network structure, and the first U-Net network in step 101 is an untrained initial U-Net network.

[0038] In step 102, a second U-Net network is trained using medical images in a training set.

[0039] In step 103, after the second U-Net network is trained, the data enhancement module and the feature fusion module are removed to obtain a trained first U-Net network, and the medical images in the test set are input into the trained first U-Net network for medical image segmentation.

[0040] The medical images in the test set and the medical images in the training set are medical images of the first organ in different domains.

[0041] Here, the first organ may be any organ in a living body.

[0042] Domain of medical images: Medical images of the same category and modality can be considered to be in the same domain. Here, the category can be an organ category. For example, if multiple medical images are medical images of the same organ, then these medical images are of the same category and in the same domain.

[0043] A modality can be an instrument that produces medical images. For example, medical images produced by the same medical instrument are in the same domain.

[0044] Therefore, in the embodiment of the present disclosure, the medical images in the test set and the medical images in the training set are medical images of the first organ in different domains, that is, the medical images in the training set and the test set are medical images of different modalities of the first organ.

[0045] Here, the above-mentioned domain types are only examples. In actual application, other types of domains are also included. The embodiments of the present disclosure do not limit the types of domains.

[0046] Before executing steps 102 - 103 , it is necessary to first obtain a medical image dataset, where the medical image dataset includes medical images of the first organ in different domains.

[0047] The medical image dataset can then be divided into a training set and test set The medical images in the training set are medical images of the organ in a certain domain, and the medical images in the test set are medical images of the organ in another domain. train There are N medical images in total in the test set. test N medical images of the second category. train 、N test All are positive integers.

[0048] Indicates that the i-th medical image in the training set has 3 channels, with a length of H and a width of W; Indicates that the true segmentation mask of the i-th medical image in the training set has K categories, with a length of H and a width of W. In the true segmentation mask, the value of the mask at the position where the object to be segmented is 1, and the value of the mask at the position where there is no object to be segmented is 0. i is an integer, and the value range of i is 1 to N train .

[0049] Similarly, x k ∈R 3×H×W , indicating that the k-th second-category medical image in the test set has three channels, with a length of H and a width of W; y k ∈R K×H×W , indicating that the true segmentation mask of the kth second-category medical image in the test set has K categories, with a length of H and a width of W. k is an integer, and the value range of k is 1 to N test .

[0050] In the disclosed embodiment, data enhancement is performed on the first image through a data enhancement module to obtain a second image. Since the second image has been data enhanced, it can be regarded as an image belonging to a different domain of the first organ than the first image. In this way, the first image and the second image are simultaneously input into the second U-Net network and trained during the training process, which can enhance the sensitivity of the second U-Net network when segmenting medical images of the first organ in different domains, and improve the accuracy of the second U-Net network when segmenting medical images of the first organ in different domains. Thereby, the first U-Net network that is finally trained accurately segments the medical images of the first organ in different domains. Compared with the traditional method of training U-Net networks, the U-Net network trained by the method in the disclosed embodiment has a higher accuracy rate in the cross-domain medical image segmentation of the same organ.

[0051] Figure 2 FIG. 1 is a flowchart of a medical image segmentation method provided by another exemplary embodiment of the present disclosure, and the method can be executed by a computer device. Figure 2 , the method comprising:

[0052] In step 201, a data enhancement module and a feature fusion module are added to the first U-Net network to obtain a second U-Net network.

[0053] Among them, in the second U-Net network, the encoder and decoder at the same layer are connected through a feature fusion module, and the data enhancement module is connected to the encoder of the first layer. The data enhancement module is used to perform data enhancement on the first image input into the medical image segmentation model to obtain the second image, and the first image and the second image are respectively input into the encoder of the first layer to obtain two encoding features of the first layer. The feature fusion module is used to fuse the two encoding features input into the feature fusion module to obtain a fused feature.

[0054] Here, the first U-Net network is a traditional U-Net network structure, and the first U-Net network in step 101 is an untrained initial U-Net network.

[0055] The traditional U-Net network is a symmetrical U-shaped structure, with the encoder on the left and the decoder on the right. The U-shaped structure consists of multiple layers, each of which includes an encoder and a decoder. The output of the encoder is the encoding feature, and the output of the decoder is the decoding feature.

[0056] After a medical image is input into the U-Net network, it is first input into the encoder of the first layer. The encoder downsamples the medical image to obtain the encoding features of the first layer, and then inputs the encoding features of the first layer into the encoder of the second layer and the decoder of the first layer respectively. This process is repeated until the encoding features of the last layer are obtained. The encoding features of the last layer are directly input into the decoder of the last layer, and then the decoder of the last layer upsamples the encoding features of the last layer to obtain the decoding features of the last layer, and then inputs the decoding features of the last layer into the decoder of the penultimate layer; the decoder of the penultimate layer upsamples the decoding features of the last layer and the encoding features of the penultimate layer, and so on until the decoding features of the first layer are obtained, and then the decoding features of the first layer are input into the segmentation head, which can output the segmentation result of the medical image.

[0057] There are many related technologies for implementing the traditional U-Net network, so detailed description is omitted here.

[0058] In the disclosed embodiment, since the data enhancement module in the second U-Net network is connected to the encoder of the first layer, the data enhancement module is used to perform data enhancement on the first image input to the medical image segmentation model to obtain the second image, and the first image and the second image are respectively input to the encoder of the first layer to obtain two encoding features of the first layer. In this way, the encoder of each layer of the second U-Net network can output two encoding features, which correspond to the first image and the second image, respectively. Optionally, the encoding feature corresponding to the first image is the first encoding feature, and the encoding feature corresponding to the second image is the second encoding feature.

[0059] By connecting a feature fusion module after each layer of the encoder, the two encoding features output by the encoder can be fused to obtain a fused feature, which can be input into the decoder.

[0060] Optionally, in the second U-Net network, the encoder in the lth layer is used to output two encoding features of the lth layer, and the two encoding features of the lth layer are respectively input into the feature fusion module in the lth layer and the encoder in the +1th layer; the feature fusion module in the lth layer is used to perform feature fusion on the two encoding features of the lth layer to obtain the fused features of the lth layer, and input the fused features of the lth layer into the decoder in the lth layer; the decoder in the lth layer is used to generate decoding features of the lth layer based on the fused features of the lth layer; wherein, is an integer, l is greater than 0 and l is less than L, L is the number of layers of the second U-Net network, and L is a positive integer.

[0061] Figure 3 It is a schematic diagram of the structure of the second U-Net network. Figure 3As shown, the second U-Net network has a total of 4 layers, that is, L is 4, the data enhancement module is connected to the encoder of the first layer, and each layer includes an encoder, a feature fusion module and a decoder connected in sequence.

[0062] When l is 1, after the first image is input into the medical image segmentation model, the data enhancement module performs data enhancement on the first image to obtain the second image. Then, the data enhancement module inputs the first image and the second image into the encoder of the first layer respectively.

[0063] Encoder at layer 1 Downsampling the first image can obtain the first encoded feature of the first layer. The encoder of the first layer Downsampling the second image can obtain the second encoding feature of the first layer, thus obtaining two encoding features of the first layer.

[0064] Then, the encoder at layer 1 These two encoded features are input into the encoder of the second layer respectively The feature fusion module in the first layer fuses the two encoded features to obtain a fused feature, and inputs the fused feature into the decoder in the first layer. Downsample the two encoded features of the first layer... and so on, until the encoder of the fourth layer After outputting the two coding features, the two coding features of the 4th layer are directly input into the feature fusion module of the 4th layer. Then the feature fusion module of the 4th layer fuses the two coding features of the 4th layer to obtain the fusion features of the 4th layer, and inputs the fusion features of the 4th layer into the decoder of the 4th layer. middle.

[0065] Layer 4 decoder The fused features of the 4th layer are upsampled to obtain the decoded features of the 4th layer, and then the decoded features of the 4th layer are concatenated with the decoded features of the 3rd layer, and the concatenated features are input into the decoder of the 4th layer. ... and so on, until the decoder in layer 1 After outputting the decoded features of the first layer, the decoded features of the first layer are input into the segmentation head to obtain the segmentation mask of the first image predicted by the second U-Net network.

[0066] Optionally, the data enhancement module is used to implement data enhancement on the first image by using any one of random cropping, random horizontal flipping, and random vertical flipping. In some embodiments, other data enhancement methods may also be used to implement data enhancement on the first image.

[0067] Optionally, the data enhancement module can be expressed using formula (1).

[0068]

[0069] In formula (1), is the i-th medical image in the training set, that is, the first image, and Aug() represents the data enhancement operation. is the image obtained after data enhancement of the i-th medical image, that is, the second image obtained after the data enhancement module performs data enhancement on the first image. i is an integer, and the value range of i is 1 to N train , N train is the number of medical images in the training set.

[0070] Optionally, the two encoding features of the output layer 1 in the layer 1 are expressed using formula (2) and formula (3).

[0071]

[0072] In formula (2), is the first encoded feature output by the encoder of the i-th medical image in the training set in the l-1th layer of the second U-Net network, is the first encoded feature output by the encoder of the i-th medical image in the training set in the l-th layer of the second U-Net network, is the encoder in the lth layer. When l is 1, the first encoding feature output by the encoder in the first layer is in That is, the first image input to the second U-Net network. The meanings of other parameters in formula (2) are the same as those in formula (1), and detailed description is omitted here.

[0073]

[0074] In formula (3), is the second encoded feature output by the encoder of the i-th medical image in the training set in the l-1th layer of the second U-Net network, It is the second encoded feature output by the encoder in the lth layer of the second U-Net network for the i-th medical image in the training set. d l Represents the dimensions of the encoded features and decoded features in the lth layer; r l =h l × l represents the resolution of the encoded and decoded features in layer l, h l is the height of the encoding feature and the decoding feature in the lth layer, w lis the width of the encoding feature and decoding feature in the lth layer. When l is 1, the second encoding feature output by the encoder of the first layer is The meanings of other parameters in formula (3) are the same as those in formula (1) and formula (2), and detailed description is omitted here.

[0075] Figure 4 This is a schematic diagram of the structure of the feature fusion module. Figure 4 The feature fusion module is described below. In the feature fusion module, two encoding features are first

[0076] Optionally, the feature fusion module in the lth layer is used to implement feature fusion of two encoding features output by the encoder in the lth layer using a cross attention mechanism to obtain a fused feature of the lth layer. The method comprises the following steps ac.

[0077] Step a, obtaining the positive attention feature of the first encoding feature of the lth layer on the second encoding feature of the lth layer.

[0078] Optionally, step a can be expressed using formula (4).

[0079]

[0080] In formula (4), is the positive attention feature of the lth layer, Softmax() is the normalized exponential function, express The meanings of other parameters in formula (4) are the same as those in formula (2) and formula (3), and detailed description is omitted here.

[0081] Step b, obtaining the reverse attention feature of the second encoding feature of the lth layer on the first encoding feature of the lth layer.

[0082] Optionally, step b can be expressed using formula (5).

[0083]

[0084] In formula (5), The reverse attention feature of the lth layer. The meanings of other parameters in formula (5) are the same as those in formula (4), and their detailed description is omitted here.

[0085] Step c, concatenate the positive attention features and the reverse attention features to obtain the fused features of the lth layer.

[0086] Optionally, step c can be expressed using formula (6).

[0087]

[0088] In formula (6), That is, it is the fusion feature of the lth layer. The meanings of other parameters in formula (6) are the same as those in formula (4) and formula (5), and the detailed description is omitted here.

[0089] When l is L, the decoding features output by the decoder in the lth layer are expressed using formula (7).

[0090]

[0091] In formula (7), o i，L is the decoding feature output by the decoder of the Lth layer of the second U-Net network for the i-th medical image in the training set, is the fusion feature of the i-th medical image in the training set in the L-th layer of the second U-Net network, is the decoder in the Lth layer. The meanings of other parameters in formula (7) are the same as those in formula (1), and their detailed description is omitted here.

[0092] When l takes any value from 1 to L-1, the decoding feature output by the decoder in the lth layer is expressed by formula (8).

[0093]

[0094] In formula (8), o i，l is the decoding feature output by the decoder of the lth layer of the second U-Net network for the i-th medical image in the training set, is the fusion feature of the i-th medical image in the training set in the l-th U-Net network, is the decoder in the lth layer, o i，l+1 is the decoded feature output by the decoder of the i-th medical image in the training set in the l+1th layer of the second U-Net network. The meanings of other parameters in formula (8) are the same as those in formula (1), and detailed description is omitted here.

[0095] After the i-th medical image passes through the decoder in the first layer of the second U-Net network, the decoder of the first layer will output the decoding feature o of the first layer. i，1 , and then the decoded features of the first layer are input into the segmentation head to obtain the predicted segmentation mask of the i-th medical image. This process can be expressed by formula (9).

[0096]

[0097] In formula (9), is the predicted segmentation mask of the i-th first-category medical image, f s For the split head, oi，1 Decoded features output by the decoder in layer 1 of the second U-Net network.

[0098] Here, the encoding module in the second U-Net network can be used To express, Including encoders located in L layers, the encoder in the lth layer is represented as The decoding module in the second U-Net network can be used To express, Including decoders located in L layers, the encoder in the lth layer is represented as θ, These are the parameters that need to be learned in the first U-Net network.

[0099] In step 202, a second U-Net network is trained using medical images in a training set.

[0100] Optionally, step 202 includes: inputting the medical images in the training set into the second U-Net network in sequence, and obtaining multiple segmentation masks output by the second U-Net network. For any segmentation mask, the segmentation loss of the segmentation mask is calculated, and then the calculated segmentation loss can be used to optimize the parameters in the encoding module and the decoding module in the second U-Net network.

[0101] Exemplarily, the segmentation loss is the mean square error. For example, the segmentation loss of the segmentation mask of the i-th medical image in the training set is calculated using formula (10).

[0102]

[0103] In formula (10), loss i is the segmentation loss of the segmentation mask of the i-th medical image in the training set, Express and Calculate the mean square error. is the true segmentation mask of the i-th medical image in the training set. The meanings of other parameters in formula (10) are the same as those in formula (9), and their detailed description is omitted here.

[0104] Optionally, the SGD (Stochastic Gradient Descent) optimization algorithm is used to optimize the parameters in the first U-Net network. This process can be expressed by formula (11).

[0105]

[0106] In formula (11), η is the learning rate, which can be set to 0.01 in the initial state; is θ, The gradient of are the parameters that need to be learned in the first U-Net network; is the optimized θ, The meanings of other parameters in formula (11) are the same as those in formula (10), and their detailed description is omitted here.

[0107] After optimizing the parameters in the second U-Net network through the segmentation loss of the i-th medical image in the training set, the medical images in the training set can be continuously input to optimize the parameters in the second U-Net network until N train Each medical image is input into the second U-Net network to complete an iterative training.

[0108] Repeat multiple iterations until the maximum number of iterations of the second U-Net network is reached. The model with the smallest segmentation loss among multiple iterations is the first U-Net network that has been trained. The parameter in the model with the smallest segmentation loss is θ * , That is, the encoding module in the first U-Net network after training is After training, the decoding module in the first U-Net network is

[0109] Exemplarily, the maximum number of iterations of the second U-Net network is 50, that is, epoch1=50.

[0110] In step 203, after the second U-Net network is trained, the data enhancement module and the feature fusion module are removed to obtain a trained first U-Net network, and the medical images in the test set are input into the trained first U-Net network for medical image segmentation.

[0111] The medical images in the test set are input into the first trained U-Net network for medical image segmentation, that is, the test set is used to verify the performance of the first trained U-Net network.

[0112] There are many related technologies on how to use a test set to verify model performance, so detailed description is omitted here.

[0113] After the trained first U-Net network is verified to be qualified, the trained first U-Net network can be used for medical image segmentation. And the trained first U-Net network has good accuracy when segmenting medical images belonging to a different domain of the first organ from the medical images in the training set and the test set.

[0114] In the disclosed embodiment, data enhancement is performed on the first image through a data enhancement module to obtain a second image. Since the second image has been data enhanced, it can be regarded as an image belonging to a different domain of the first organ than the first image. In this way, the first image and the second image are simultaneously input into the second U-Net network and trained during the training process, which can enhance the sensitivity of the second U-Net network when segmenting medical images of the first organ in different domains, and improve the accuracy of the second U-Net network when segmenting medical images of the first organ in different domains. Thereby, the first U-Net network that is finally trained accurately segments the medical images of the first organ in different domains. Compared with the traditional method of training U-Net networks, the U-Net network trained by the method in the disclosed embodiment has a higher accuracy rate in the cross-domain medical image segmentation of the same organ.

[0115] The following are device embodiments of the present application. For details not described in detail in the device embodiments, reference may be made to the above method embodiments.

[0116] Figure 5 FIG. 1 shows a schematic diagram of a medical image segmentation device provided by an exemplary embodiment of the present disclosure. Figure 5 The medical image segmentation device 500 includes: an acquisition module 501, a training module 502 and a testing module 503.

[0117] The acquisition module 501 is used to add a data enhancement module and a feature fusion module to the first U-Net network to obtain a second U-Net network.

[0118] The training module 502 is used to train the second U-Net network using the medical images in the training set.

[0119] The testing module 503 is used to remove the data enhancement module and the feature fusion module after the second U-Net network training is completed, to obtain the trained first U-Net network, and to input the medical images in the test set into the trained first U-Net network for medical image segmentation, wherein the medical images in the test set and the medical images in the training set are medical images of the first organ in different domains;

[0120] Among them, in the second U-Net network, the encoder and decoder at the same layer are connected through a feature fusion module, and the data enhancement module is connected to the encoder of the first layer. The data enhancement module is used to perform data enhancement on the first image input into the medical image segmentation model to obtain the second image, and the first image and the second image are respectively input into the encoder of the first layer to obtain two encoding features of the first layer. The feature fusion module is used to fuse the two encoding features input into the feature fusion module to obtain a fused feature.

[0121] Optionally, the coding feature corresponding to the first image is the first coding feature, the coding feature corresponding to the second image is the second coding feature, and the acquisition module 501 is further used to acquire the second coding feature. l The first encoded feature of the layer is l The positive attention feature on the second encoding feature of the layer; get the first l The second encoded feature of the layer is l The reverse attention feature is based on the first encoding feature of the layer; the forward attention feature and the reverse attention feature are concatenated to obtain the fused feature of the lth layer.

[0122] Optionally, the acquisition module 501 is further configured to perform data enhancement on the first image by using any one of random cropping, random horizontal flipping, and random vertical flipping.

[0123] It should be noted that: when the medical image segmentation device provided in the above embodiment performs medical image segmentation, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the medical image segmentation device provided in the above embodiment and the medical image segmentation method embodiment belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0124] The division of modules in the embodiments of the present disclosure is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, each functional module in each embodiment of the present disclosure may be integrated into a processor, or may exist physically separately, or two or more modules may be integrated into one module. The above-mentioned integrated modules may be implemented in the form of hardware or in the form of software functional modules.

[0125] If the integrated module is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present disclosure is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a terminal device (which can be a personal computer, a mobile phone, or a communication device, etc.) or a processor (processor) to perform all or part of the steps of the method of each embodiment of the present disclosure. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0126] Figure 6 Schematic diagram of the structure of the computer device provided by the embodiment of the present disclosure. Figure 6 As shown, the computer device 600 includes: a processor 601 and a memory 602 .

[0127] The processor 601 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 601 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 601 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 601 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 601 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0128] The memory 602 may include one or more computer-readable storage media, which may be non-transitory. The memory 602 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 602 is used to store at least one instruction, which is used to be executed by the processor 601 to implement the medical image segmentation method provided in the embodiments of the present disclosure.

[0129] Those skilled in the art will understand that Figure 6 The structure shown in the figure does not constitute a limitation on the computer device 600, and the computer device 600 may include more or less components than those shown in the figure, or combine some components, or adopt a different arrangement of components.

[0130] The embodiments of the present disclosure also provide a non-temporary computer-readable storage medium. When the instructions in the storage medium are executed by a processor of a computer device, the computer device is enabled to execute the medical image segmentation method provided in the embodiments of the present disclosure.

[0131] The present disclosure also provides a computer program product, including a computer program / instruction, which implements the medical image segmentation method provided in the present disclosure when the computer program / instruction is executed by a processor.

[0132] The above description is only an optional embodiment of the present disclosure and is not intended to limit the present disclosure. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A medical image segmentation method, characterized in that: The method comprises: Adding a data enhancement module and a feature fusion module to the first U-Net network to obtain a second U-Net network; Using the medical images in the training set to train the second U-Net network; After the training of the second U-Net network is completed, the data enhancement module and the feature fusion module are removed to obtain the trained first U-Net network, and the medical images in the test set are input into the trained first U-Net network for medical image segmentation, wherein the medical images in the test set and the medical images in the training set are medical images of the first organ in different domains; Among them, in the second U-Net network, the encoder and decoder at the same layer are connected through a feature fusion module, the data enhancement module is connected to the encoder of the first layer, the data enhancement module is used to perform data enhancement on the first image input into the medical image segmentation model to obtain the second image, and the first image and the second image are respectively input into the encoder of the first layer to obtain two encoding features of the first layer, and the feature fusion module is used to fuse the two encoding features input into the feature fusion module to obtain a fused feature.

2. The method according to claim 1, characterized in that In the second U-Net network, the encoder in the lth layer is used to output two encoding features of the lth layer, and the two encoding features of the lth layer are respectively input into the feature fusion module in the lth layer and the encoder in the l+1th layer; The said l The feature fusion module in the layer is used to l The two encoding features of the layer are fused to obtain the l The fusion features of the first layer l The fusion features of the layer are input to the l Decoder in layer; The said l The decoder in the layer is used based on the l The fusion features of the layer are generated l Decoded features of the layer; in, l is an integer, l Greater than 0 and l Less than L , L is the number of layers of the second U-Net network.

3. The method according to claim 2, characterized in that The coding feature corresponding to the first image is the first coding feature, the coding feature corresponding to the second image is the second coding feature, and the feature fusion module in the first layer is used to implement feature fusion of the two coding features of the first layer in the following manner to obtain the fusion feature of the first layer: Obtain a positive attention feature of the first encoding feature of the lth layer on the second encoding feature of the lth layer; Obtain the reverse attention feature of the second encoding feature of the lth layer on the first encoding feature of the lth layer; The positive attention feature and the reverse attention feature are concatenated to obtain the fused feature of the lth layer.

4. The method according to claim 3, characterized in that The positive attention feature is calculated using the following formula: The reverse attention feature is calculated using the following formula: in, is the positive attention feature of the lth layer, The reverse attention feature of layer l, is the first coding feature of the lth layer, is the second encoding feature of the lth layer, Softmax( ) is the normalized exponential function, d l is the feature dimension of the lth layer.

5. The method according to any one of claims 1 to 4, characterized in that: The data enhancement module is used to perform data enhancement on the first image input to the medical image segmentation model to obtain a second image, including: The data enhancement module is used to implement data enhancement on the first image by using any one of random cropping, random horizontal flipping, and random vertical flipping.

6. A medical image segmentation device, characterized in that: The device comprises An acquisition module is used to add a data enhancement module and a feature fusion module to the first U-Net network to obtain a second U-Net network; A training module, used for training the second U-Net network using medical images in a training set; A testing module, used for removing the data enhancement module and the feature fusion module after the training of the second U-Net network is completed, to obtain the trained first U-Net network, and inputting the medical images in the test set into the trained first U-Net network for medical image segmentation, wherein the medical images in the test set and the medical images in the training set are medical images of the first organ in different domains; Among them, in the second U-Net network, the encoder and decoder at the same layer are connected through a feature fusion module, the data enhancement module is connected to the encoder of the first layer, the data enhancement module is used to perform data enhancement on the first image input into the medical image segmentation model to obtain the second image, and the first image and the second image are respectively input into the encoder of the first layer to obtain two encoding features of the first layer, and the feature fusion module is used to fuse the two encoding features input into the feature fusion module to obtain a fused feature.

7. A computer device, characterized in that: The computer device comprises: a memory and a processor, wherein at least one computer program is stored in the memory, and the at least one computer program is loaded and executed by the processor to implement the method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one computer program, and the at least one computer program is loaded and executed by a processor to implement the method according to any one of claims 1 to 5.

9. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images