Medical image segmentation method, device, equipment and storage medium

Abstract

Disclosed are a medical image segmentation method, apparatus, device, and storage medium, belonging to the field of image segmentation technology. The method includes: adding a data augmentation module and a feature fusion module to a first U-Net network to obtain a second U-Net network; training the second U-Net network using medical images in a training set; after the second U-Net network is trained, removing the data augmentation module and the feature fusion module to obtain a trained first U-Net network; and inputting medical images in a test set into the trained first U-Net network for medical image segmentation, wherein the medical images in the test set and the medical images in the training set belong to different domains for a first organ. This method can improve the accuracy of cross-domain medical image segmentation.

Description

Technical Field

[0001] The present disclosure relates to the field of image segmentation technology, and in particular to a medical image segmentation method, apparatus, device and storage medium. Background Art

[0002] Medical images are used to represent images produced by various medical instruments. Medical image segmentation refers to the act of segmenting parts of a medical image according to a specific goal. For example, if the goal is to segment an organ with a lesion, medical image segmentation is used to segment the organ with the lesion from the medical image.

[0003] Medical images of different categories or modalities are also medical images of different domains. Segmenting medical images of different domains is cross-domain medical image segmentation.

[0004] In related technologies, U-Net networks are commonly used for cross-domain medical image segmentation. U-Net consists of an encoder and a decoder. The encoder gradually reduces the spatial dimensions of the input image through a series of convolutional and max-pooling layers, extracting and learning important features from the image in the process. The decoder gradually restores the spatial dimensions of the image through upsampling operations and convolutional layers. Skip connections are also included between the encoder and decoder. Skip connections directly connect the feature maps in the encoder to the corresponding layers of the decoder. These skip connections allow the decoder to reference earlier features during the upsampling process, thereby compensating for details that traditional deep learning models may overlook.

[0005] However, the U-Net network trained using traditional methods has low accuracy when performing cross-domain medical image segmentation. Summary of the Invention

[0006] The present disclosure provides a medical image segmentation method, apparatus, device, and storage medium that can improve the accuracy of cross-domain medical image segmentation. The technical solution includes at least the following solutions:

[0007] In a first aspect, a medical image segmentation method is provided, including: adding a data enhancement module and a feature fusion module to a first U-Net network to obtain a second U-Net network; using medical images in a training set to train the second U-Net network; after the training of the second U-Net network is completed, removing the data enhancement module and the feature fusion module to obtain the trained first U-Net network, and inputting the medical images in the test set into the trained first U-Net network for medical image segmentation, wherein the medical images in the test set and the medical images in the training set belong to medical images of the first organ in different domains; wherein, in the second U-Net network, the encoder and decoder in the same layer are connected through a feature fusion module, the data enhancement module is connected to the encoder of the first layer, the data enhancement module is used to perform data enhancement on the first image input into the medical image segmentation model to obtain a second image, and the first image and the second image are respectively input into the encoder of the first layer to obtain two encoding features of the first layer, and the feature fusion module is used to fuse the two encoding features input into the feature fusion module to obtain a fused feature.

[0008] Optionally, in the second U-Net network, the encoder in the lth layer is used to output two encoding features of the lth layer, and the two encoding features of the lth layer are respectively input into the feature fusion module in the lth layer and the encoder in the l+1th layer; the feature fusion module in the lth layer is used to perform feature fusion on the two encoding features of the lth layer to obtain the fusion features of the lth layer, and input the fusion features of the lth layer into the decoder in the lth layer; the decoder in the lth layer is used to generate the decoding features of the lth layer based on the fusion features of the lth layer; wherein l is an integer, l is greater than 0 and l is less than L, and L is the number of layers of the second U-Net network.

[0009] Optionally, the coding feature corresponding to the first image is the first coding feature, and the coding feature corresponding to the second image is the second coding feature. The feature fusion module in the lth layer is used to implement feature fusion of the two coding features of the lth layer in the following manner to obtain the fused feature of the lth layer: obtain the positive attention feature of the first coding feature of the lth layer on the second coding feature of the lth layer; obtain the reverse attention feature of the second coding feature of the lth layer on the first coding feature of the lth layer; splice the positive attention feature and the reverse attention feature to obtain the fused feature of the lth layer.

[0010] Optionally, the positive attention feature is calculated using the following formula:

[0011]

[0012] The reverse attention feature is calculated using the following formula:

[0013]

[0014] in, is the positive attention feature of layer l, The reverse attention feature of layer l, is the first coding feature of the lth layer, is the second encoding feature of the lth layer, Softmax() is the normalized exponential function, d l is the feature dimension of the lth layer.

[0015] Optionally, the data enhancement module is used to perform data enhancement on the first image input into the medical image segmentation model to obtain a second image, including: the data enhancement module is used to implement data enhancement on the first image by using any one of random cropping, random horizontal flipping, and random vertical flipping.

[0016] In a second aspect, a medical image segmentation device is also provided, including: an acquisition module for adding a data enhancement module and a feature fusion module to a first U-Net network to obtain a second U-Net network; a training module for training the second U-Net network using medical images in a training set; a testing module for removing the data enhancement module and the feature fusion module after the second U-Net network training is completed to obtain the trained first U-Net network, and inputting the medical images in the test set into the trained first U-Net network for medical image segmentation, wherein the medical images in the test set and the medical images in the training set are medical images of the first organ in different domains;

[0017] In which, in the second U-Net network, the encoder and decoder at the same layer are connected through a feature fusion module, the data enhancement module is connected to the encoder of the first layer, the data enhancement module is used to perform data enhancement on the first image input into the medical image segmentation model to obtain the second image, and the first image and the second image are respectively input into the encoder of the first layer to obtain two encoding features of the first layer, and the feature fusion module is used to fuse the two encoding features input into the feature fusion module to obtain a fused feature.

[0018] Optionally, the coding feature corresponding to the first image is the first coding feature, and the coding feature corresponding to the second image is the second coding feature. The acquisition module is also used to obtain the positive attention feature of the first coding feature of the lth layer on the second coding feature of the lth layer; obtain the reverse attention feature of the second coding feature of the lth layer on the first coding feature of the lth layer; splice the positive attention feature and the reverse attention feature to obtain the fusion feature of the lth layer.

[0019] Optionally, the acquisition module is further configured to perform data enhancement on the first image by adopting any one of random cropping, random horizontal flipping, and random vertical flipping.

[0020] In a third aspect, a computer device is also provided, comprising: a memory and a processor, wherein the memory stores at least one computer program, and the at least one computer program is loaded and executed by the processor, thereby executing the medical image segmentation method described in the above embodiment.

[0021] In a fourth aspect, a computer-readable storage medium is further provided, wherein at least one computer program is stored in the computer-readable storage medium, and the at least one computer program is loaded and executed by a processor, thereby executing the medical image segmentation method described in the above embodiment.

[0022] In a fifth aspect, a computer program product is provided, comprising a computer program / instruction, which implements the method described in the first aspect when executed by a processor.

[0023] The beneficial effects of the technical solutions provided by the embodiments of the present disclosure include at least:

[0024] In the embodiment of the present disclosure, data enhancement is performed on the first image through a data enhancement module to obtain a second image. Since the second image has been data enhanced, it can be regarded as an image belonging to a different domain of the first organ than the first image. In this way, the first image and the second image are simultaneously input into the second U-Net network and trained during the training process, which can enhance the sensitivity of the second U-Net network when segmenting medical images of the first organ in different domains, and improve the accuracy of the second U-Net network when segmenting medical images of the first organ in different domains. Thereby, the first U-Net network that is finally trained accurately segments the medical images of the first organ in different domains. Compared with the traditional method of training U-Net networks, the U-Net network trained by the method in the embodiment of the present disclosure has a higher accuracy rate in cross-domain medical image segmentation of the same organ. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0026] Figure 1 A flowchart of a medical image segmentation method provided by an exemplary embodiment of the present disclosure is shown;

[0027] Figure 2 A flowchart of a medical image segmentation method provided by another exemplary embodiment of the present disclosure is shown;

[0028] Figure 3 It is a structural diagram of the second U-Net network;

[0029] Figure 4 It is a structural diagram of the feature fusion module;

[0030] Figure 5 A schematic structural diagram of a medical image segmentation device provided by an exemplary embodiment of the present disclosure is shown;

[0031] Figure 6 It is a structural diagram of a computer device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0032] Unless otherwise defined, the technical or scientific terms used herein shall have the ordinary meaning understood by persons of ordinary skill in the field to which the present disclosure belongs. The words "first", "second", "third" and similar terms used in the patent application specification and claims of the present disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. Similarly, words such as "one" or "a" do not indicate a quantity limitation, but rather indicate the existence of at least one. Words such as "include" or "comprising" mean that the elements or objects appearing before "include" or "comprising" cover the elements or objects listed after "include" or "comprising" and their equivalents, and do not exclude other elements or objects. Words such as "connected" or "connected" and similar terms are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect.

[0033] In order to make the objectives, technical solutions and advantages of the present disclosure more clear, the embodiments of the present disclosure will be further described in detail below with reference to the accompanying drawings.

[0034] Figure 1FIG1 shows a flowchart of a medical image segmentation method provided by an exemplary embodiment of the present disclosure, which can be executed by a computer device. Figure 1 , the method comprising:

[0035] In step 101, a data enhancement module and a feature fusion module are added to a first U-Net network to obtain a second U-Net network.

[0036] Among them, in the second U-Net network, the encoder and decoder at the same layer are connected through a feature fusion module, and the data enhancement module is connected to the encoder of the first layer. The data enhancement module is used to perform data enhancement on the first image input to the medical image segmentation model to obtain the second image, and input the first image and the second image into the encoder of the first layer respectively to obtain two encoding features of the first layer. The feature fusion module is used to fuse the two encoding features input to the feature fusion module to obtain a fused feature.

[0037] Here, the first U-Net network is a traditional U-Net network structure, and the first U-Net network in step 101 is an untrained initial U-Net network.

[0038] In step 102, a second U-Net network is trained using medical images in a training set.

[0039] In step 103, after the second U-Net network training is completed, the data enhancement module and the feature fusion module are removed to obtain a trained first U-Net network, and the medical images in the test set are input into the trained first U-Net network for medical image segmentation.

[0040] The medical images in the test set and the medical images in the training set are medical images of the first organ in different domains.

[0041] Here, the first organ may be any organ in a living body.

[0042] Medical image domain: Medical images of the same category and modality are considered to be in the same domain. Here, the category can be an organ category. For example, if multiple medical images are of the same organ, then these images are of the same category and in the same domain.

[0043] A modality can be an instrument that generates medical images. For example, medical images generated by the same medical instrument are in the same domain.

[0044] Therefore, in the embodiment of the present disclosure, the medical images in the test set and the medical images in the training set are medical images of the first organ in different domains, that is, the medical images in the training set and the test set are medical images of different modalities of the first organ.

[0045] Here, the above-mentioned domain types are only examples. In actual application, other types of domains are also included. The embodiments of the present disclosure do not limit the types of domains.

[0046] Before executing steps 102 - 103 , it is necessary to first obtain a medical image dataset, where the medical image dataset includes medical images of the first organ in different domains.

[0047] The medical image dataset can then be divided into a training set and test set The medical images in the training set are medical images of the organ in a certain domain, and the medical images in the test set are medical images of the organ in another domain. train There are N medical images in the test set. test N medical images of the second category. train 、N test All are positive integers.

[0048] Indicates that the i-th medical image in the training set has 3 channels, with a length of H and a width of W; Indicates that the true segmentation mask of the i-th medical image in the training set has K categories, with a length of H and a width of W. In the true segmentation mask, the value of the mask at the position where the object to be segmented is 1, and the value of the mask at the position where there is no object to be segmented is 0. i is an integer, and the value range of i is 1 to N train .

[0049] Similarly, x k ∈R 3×H×W , indicating that the k-th second-category medical image in the test set has 3 channels, with a length of H and a width of W; y k ∈R K×H×W , which means that the true segmentation mask of the kth second-category medical image in the test set has K categories, with a length of H and a width of W. k is an integer, and the value range of k is 1 to N test .

[0050] In the embodiment of the present disclosure, data enhancement is performed on the first image through a data enhancement module to obtain a second image. Since the second image has been data enhanced, it can be regarded as an image belonging to a different domain of the first organ than the first image. In this way, the first image and the second image are simultaneously input into the second U-Net network and trained during the training process, which can enhance the sensitivity of the second U-Net network when segmenting medical images of the first organ in different domains, and improve the accuracy of the second U-Net network when segmenting medical images of the first organ in different domains. Thereby, the first U-Net network that is finally trained accurately segments the medical images of the first organ in different domains. Compared with the traditional method of training U-Net networks, the U-Net network trained by the method in the embodiment of the present disclosure has a higher accuracy rate in cross-domain medical image segmentation of the same organ.

[0051] Figure 2 FIG2 shows a flowchart of a medical image segmentation method provided by another exemplary embodiment of the present disclosure, which can be executed by a computer device. Figure 2 , the method comprising:

[0052] In step 201, a data enhancement module and a feature fusion module are added to the first U-Net network to obtain a second U-Net network.

[0053] Among them, in the second U-Net network, the encoder and decoder at the same layer are connected through a feature fusion module, and the data enhancement module is connected to the encoder of the first layer. The data enhancement module is used to perform data enhancement on the first image input to the medical image segmentation model to obtain the second image, and input the first image and the second image into the encoder of the first layer respectively to obtain two encoding features of the first layer. The feature fusion module is used to fuse the two encoding features input to the feature fusion module to obtain a fused feature.

[0054] Here, the first U-Net network is a traditional U-Net network structure, and the first U-Net network in step 101 is an untrained initial U-Net network.

[0055] The traditional U-Net network has a symmetrical U-shaped structure, with the encoder on the left and the decoder on the right. The U-shaped structure consists of multiple layers, each of which includes an encoder and decoder. The encoder outputs the encoded features, and the decoder outputs the decoded features.

[0056] After a medical image is input into the U-Net network, it is first input into the encoder of the first layer. The encoder downsamples the medical image to obtain the encoding features of the first layer. The encoding features of the first layer are then input into the encoder of the second layer and the decoder of the first layer. This process is repeated until the encoding features of the last layer are obtained. The encoding features of the last layer are directly input into the decoder of the last layer. The decoder of the last layer then upsamples the encoding features of the last layer to obtain the decoding features of the last layer. The decoding features of the last layer are then input into the decoder of the second-to-last layer. The decoder of the second-to-last layer upsamples the decoding features of the last layer and the encoding features of the second-to-last layer. This process is repeated until the decoding features of the first layer are obtained. The decoding features of the first layer are then input into the segmentation head, which then outputs the segmentation result of the medical image.

[0057] There are many related technologies for implementing the traditional U-Net network, so detailed description is omitted here.

[0058] In the disclosed embodiment, since the data enhancement module in the second U-Net network is connected to the encoder of the first layer, the data enhancement module is used to perform data enhancement on the first image input to the medical image segmentation model to obtain the second image, and the first image and the second image are respectively input to the encoder of the first layer to obtain two encoding features of the first layer. In this way, the encoder of each layer of the second U-Net network can output two encoding features, which correspond to the first image and the second image, respectively. Optionally, the encoding feature corresponding to the first image is the first encoding feature, and the encoding feature corresponding to the second image is the second encoding feature.

[0059] By connecting a feature fusion module after each layer of encoder, the two encoding features output by the encoder can be fused to obtain a fused feature, which can be input into the decoder.

[0060] Optionally, in the second U-Net network, the encoder in the lth layer is used to output the two encoding features of the lth layer, and the two encoding features of the lth layer are respectively input into the feature fusion module in the lth layer and the encoder in the +1th layer; the feature fusion module in the lth layer is used to perform feature fusion on the two encoding features of the lth layer to obtain the fused features of the lth layer, and input the fused features of the lth layer into the decoder in the lth layer; the decoder in the lth layer is used to generate the decoding features of the lth layer based on the fused features of the lth layer; wherein, is an integer, l is greater than 0 and l is less than L, L is the number of layers of the second U-Net network, and L is a positive integer.

[0061] Figure 3 It is a schematic diagram of the structure of the second U-Net network. Figure 3As shown, the second U-Net network has a total of 4 layers, that is, L is 4, the data enhancement module is connected to the encoder of the first layer, and each layer includes an encoder, a feature fusion module and a decoder connected in sequence.

[0062] When l is 1, after the first image is input into the medical image segmentation model, the data enhancement module performs data enhancement on the first image to obtain the second image. Then, the data enhancement module inputs the first image and the second image into the encoder of the first layer respectively.

[0063] Encoder at layer 1 Downsampling the first image can obtain the first encoded feature of the first layer, and the encoder of the first layer Downsampling the second image can obtain the second coding feature of the first layer, thus obtaining two coding features of the first layer.

[0064] Then, the encoder of layer 1 These two encoded features are input to the encoder of the second layer respectively The feature fusion module in the first layer fuses the two encoding features to obtain fused features, and inputs the fused features into the decoder in the first layer. Downsample the two encoded features of the first layer... and so on, until the encoder of the fourth layer After outputting the two coding features, the two coding features of the 4th layer are directly input to the feature fusion module of the 4th layer. Then the feature fusion module of the 4th layer fuses the two coding features of the 4th layer to obtain the fusion feature of the 4th layer, and inputs the fusion feature of the 4th layer to the decoder of the 4th layer. middle.

[0065] Layer 4 decoder The fusion features of the 4th layer are upsampled to obtain the decoding features of the 4th layer, and then the decoding features of the 4th layer and the decoding features of the 3rd layer are spliced ​​and the spliced ​​features are input into the decoder of the 4th layer. ...and so on, until the decoder in layer 1 After outputting the decoded features of the first layer, the decoded features of the first layer are input into the segmentation head to obtain the segmentation mask of the first image predicted by the second U-Net network.

[0066] Optionally, the data enhancement module is used to implement data enhancement on the first image by using any one of random cropping, random horizontal flipping, and random vertical flipping. In some embodiments, other data enhancement methods may also be used to implement data enhancement on the first image.

[0067] Optionally, the data enhancement module can be expressed using formula (1).

[0068]

[0069] In formula (1), is the i-th medical image in the training set, that is, the first image, and Aug() represents the data enhancement operation. The image obtained after data enhancement of the i-th medical image, that is, the second image obtained after the data enhancement module performs data enhancement on the first image. i is an integer, and the value range of i is 1 to N train , N train is the number of medical images in the training set.

[0070] Optionally, the two encoding features of the output layer 1 are expressed using formula (2) and formula (3).

[0071]

[0072] In formula (2), is the first encoded feature output by the encoder of the l-1th layer of the second U-Net network for the i-th medical image in the training set, is the first encoded feature output by the encoder of the lth layer of the second U-Net network for the i-th medical image in the training set, is the encoder in the lth layer. When l is 1, the first encoding feature output by the encoder in the first layer is in This is the first image input to the second U-Net network. The meanings of other parameters in formula (2) are the same as those in formula (1), and their detailed description is omitted here.

[0073]

[0074] In formula (3), is the second encoded feature output by the encoder of the l-1 layer of the second U-Net network for the i-th medical image in the training set, It is the second encoded feature output by the encoder in the lth layer of the second U-Net network for the i-th medical image in the training set. d l Represents the dimensions of the encoding features and decoding features in the lth layer; r l =h l ×w l represents the resolution of the encoding and decoding features in the lth layer, h l is the height of the encoding feature and the decoding feature in the lth layer, w lis the width of the encoding feature and decoding feature in the lth layer. When l is 1, the second encoding feature output by the encoder of the first layer is The meanings of other parameters in formula (3) are the same as those in formula (1) and formula (2), and their detailed description is omitted here.

[0075] Figure 4 This is a structural diagram of the feature fusion module. Figure 4 The feature fusion module is described. In the feature fusion module, the two encoding features are first

[0076] Optionally, the feature fusion module in the first layer is used to implement feature fusion of two encoded features output by the encoder in the first layer using a cross attention mechanism to obtain a fused feature of the first layer. The method includes the following steps ac.

[0077] Step a: obtain the positive attention feature of the first encoding feature of the lth layer on the second encoding feature of the lth layer.

[0078] Optionally, step a can be expressed using formula (4).

[0079]

[0080] In formula (4), is the positive attention feature of the lth layer, Softmax() is the normalized exponential function, express The meanings of other parameters in formula (4) are the same as those in formula (2) and formula (3), and their detailed description is omitted here.

[0081] Step b: obtain the reverse attention feature of the second encoding feature of the lth layer on the first encoding feature of the lth layer.

[0082] Optionally, step b can be expressed using formula (5).

[0083]

[0084] In formula (5), The reverse attention feature of the lth layer. The meanings of other parameters in formula (5) are the same as those in formula (4), and their detailed description is omitted here.

[0085] Step c: concatenate the forward attention features and the reverse attention features to obtain the fused features of the lth layer.

[0086] Optionally, step c can be expressed using formula (6).

[0087]

[0088] In formula (6), The meanings of other parameters in formula (6) are the same as those in formula (4) and formula (5), and their detailed description is omitted here.

[0089] When l is L, the decoding features output by the decoder in the lth layer are expressed using formula (7).

[0090]

[0091] In formula (7), o i，L is the decoding feature output by the decoder of the Lth layer of the second U-Net network for the i-th medical image in the training set, is the fusion feature of the i-th medical image in the training set in the L-th layer of the second U-Net network, is the decoder in the Lth layer. The meanings of other parameters in formula (7) are the same as those in formula (1), and their detailed description is omitted here.

[0092] When l takes any value from 1 to L-1, the decoding feature output by the decoder in the lth layer is expressed using formula (8).

[0093]

[0094] In formula (8), o i，l is the decoding feature output by the decoder of the lth layer of the second U-Net network for the i-th medical image in the training set, is the fusion feature of the i-th medical image in the training set in the l-th U-Net network, is the decoder in the lth layer, o i，l+1 is the decoded feature output by the decoder in the l+1th layer of the second U-Net network for the i-th medical image in the training set. The meanings of the other parameters in formula (8) are the same as those in formula (1) and are not described in detail here.

[0095] After the i-th medical image passes through the decoder in the first layer of the second U-Net network, the decoder of the first layer will output the decoding feature o of the first layer. i，1 , and then the decoded features of the first layer are input to the segmentation head to obtain the predicted segmentation mask of the i-th medical image. This process can be expressed by formula (9).

[0096]

[0097] In formula (9), is the predicted segmentation mask of the first-category medical image, f s For the split head, oi，1 The decoded features output by the decoder in the first layer of the second U-Net network.

[0098] Here, the encoding module in the second U-Net network can be used To express, Including encoders located in L layers, the encoder in the lth layer is represented as The decoding module in the second U-Net network can be used To express, Including decoders located in L layers, the encoder in the lth layer is represented as θ、 These are the parameters that need to be learned in the first U-Net network.

[0099] In step 202, a second U-Net network is trained using medical images in a training set.

[0100] Optionally, step 202 includes sequentially inputting the medical images in the training set into a second U-Net network to obtain multiple segmentation masks output by the second U-Net network. For each segmentation mask, a segmentation loss is calculated for the segmentation mask, and the calculated segmentation loss can be used to optimize parameters in the encoding module and decoding module of the second U-Net network.

[0101] For example, the segmentation loss is the mean square error. For example, the segmentation loss of the segmentation mask of the i-th medical image in the training set is calculated using formula (10).

[0102]

[0103] In formula (10), loss i is the segmentation loss of the segmentation mask of the i-th medical image in the training set, Express and Calculate the mean square error, is the true segmentation mask of the i-th medical image in the training set. The meanings of other parameters in formula (10) are the same as those in formula (9), and their detailed description is omitted here.

[0104] Optionally, the SGD (Stochastic Gradient Descent) optimization algorithm is used to optimize the parameters in the first U-Net network. This process can be expressed by formula (11).

[0105]

[0106] In formula (11), η is the learning rate, which can be set to 0.01 in the initial state; is θ, The gradient of θ, are the parameters that need to be learned in the first U-Net network; is the optimized θ, The meanings of other parameters in formula (11) are the same as those in formula (10), and their detailed description is omitted here.

[0107] After optimizing the parameters of the second U-Net network through the segmentation loss of the i-th medical image in the training set, the medical images in the training set can be continuously input to optimize the parameters of the second U-Net network until N train Each medical image is input into the second U-Net network to complete one iterative training.

[0108] Repeat multiple iterations until the maximum number of iterations of the second U-Net network is reached. The model with the smallest segmentation loss among multiple iterations is the first U-Net network that has been trained. The parameter of the model with the smallest segmentation loss is θ * 、 That is, the encoding module in the first U-Net network after training is After training, the decoding module in the first U-Net network is

[0109] Exemplarily, the maximum number of iterations of the second U-Net network is 50, that is, epoch1=50.

[0110] In step 203, after the second U-Net network training is completed, the data enhancement module and the feature fusion module are removed to obtain a trained first U-Net network, and the medical images in the test set are input into the trained first U-Net network for medical image segmentation.

[0111] The medical images in the test set are input into the first trained U-Net network for medical image segmentation, that is, the performance of the first trained U-Net network is verified using the test set.

[0112] There are many related technologies on how to use test sets to verify model performance, so detailed description is omitted here.

[0113] After the trained first U-Net network is verified as qualified, the trained first U-Net network can be used for medical image segmentation. Furthermore, the trained first U-Net network has good accuracy when segmenting medical images belonging to a different domain than the first organ in the training set and the test set.

[0114] In the embodiment of the present disclosure, data enhancement is performed on the first image through a data enhancement module to obtain a second image. Since the second image has been data enhanced, it can be regarded as an image belonging to a different domain of the first organ than the first image. In this way, the first image and the second image are simultaneously input into the second U-Net network and trained during the training process, which can enhance the sensitivity of the second U-Net network when segmenting medical images of the first organ in different domains, and improve the accuracy of the second U-Net network when segmenting medical images of the first organ in different domains. Thereby, the first U-Net network that is finally trained accurately segments the medical images of the first organ in different domains. Compared with the traditional method of training U-Net networks, the U-Net network trained by the method in the embodiment of the present disclosure has a higher accuracy rate in cross-domain medical image segmentation of the same organ.

[0115] The following are device embodiments of the present application. For details not described in detail in the device embodiments, reference may be made to the above method embodiments.

[0116] Figure 5 A schematic diagram of the structure of a medical image segmentation device provided by an exemplary embodiment of the present disclosure is shown. Figure 5 The medical image segmentation device 500 includes: an acquisition module 501, a training module 502 and a testing module 503.

[0117] The acquisition module 501 is used to add a data enhancement module and a feature fusion module to the first U-Net network to obtain a second U-Net network.

[0118] The training module 502 is used to train the second U-Net network using the medical images in the training set.

[0119] The testing module 503 is configured to remove the data enhancement module and the feature fusion module after the second U-Net network training is completed to obtain a trained first U-Net network, and input the medical images in the test set into the trained first U-Net network for medical image segmentation, where the medical images in the test set and the medical images in the training set are medical images of the first organ in different domains;

[0120] Among them, in the second U-Net network, the encoder and decoder at the same layer are connected through a feature fusion module, and the data enhancement module is connected to the encoder of the first layer. The data enhancement module is used to perform data enhancement on the first image input to the medical image segmentation model to obtain the second image, and input the first image and the second image into the encoder of the first layer respectively to obtain two encoding features of the first layer. The feature fusion module is used to fuse the two encoding features input to the feature fusion module to obtain a fused feature.

[0121] Optionally, the coding feature corresponding to the first image is the first coding feature, the coding feature corresponding to the second image is the second coding feature, and the acquisition module 501 is further used to acquire the second coding feature. l The first encoded feature of the layer is l The positive attention feature on the second encoding feature of the layer; get the first l The second encoded feature of the layer is l The reverse attention feature is based on the first encoding feature of the layer; the forward attention feature and the reverse attention feature are concatenated to obtain the fusion feature of the lth layer.

[0122] Optionally, the acquisition module 501 is further configured to perform data enhancement on the first image by adopting any one of random cropping, random horizontal flipping, and random vertical flipping.

[0123] It should be noted that the medical image segmentation device provided in the above embodiment only uses the division of the above functional modules as an example to illustrate medical image segmentation. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the medical image segmentation device provided in the above embodiment and the medical image segmentation method embodiment are based on the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0124] The division of modules in the embodiments of the present disclosure is illustrative and represents only a logical functional division. In actual implementation, other division methods may be used. Furthermore, the functional modules in the various embodiments of the present disclosure may be integrated into a single processor, exist physically as separate modules, or be integrated into a single module. The integrated modules may be implemented in either hardware or software functional modules.

[0125] If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present disclosure is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a number of instructions to enable a terminal device (which can be a personal computer, mobile phone, or communication device, etc.) or a processor (processor) to execute all or part of the steps of the method of each embodiment of the present disclosure. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, etc., various media that can store program code.

[0126] Figure 6 Schematic diagram of the structure of the computer device provided by the embodiment of the present disclosure. Figure 6 As shown, the computer device 600 includes a processor 601 and a memory 602 .

[0127] The processor 601 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 601 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), or PLA (Programmable Logic Array). The processor 601 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 601 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 601 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0128] The memory 602 may include one or more computer-readable storage media, which may be non-transitory. The memory 602 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices and flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 602 is used to store at least one instruction, which is executed by the processor 601 to implement the medical image segmentation method provided in the embodiments of the present disclosure.

[0129] Those skilled in the art will understand that Figure 6 The structure shown in the figure does not constitute a limitation on the computer device 600, and the computer device 600 may include more or fewer components than shown in the figure, or combine some components, or adopt a different arrangement of components.

[0130] The embodiments of the present disclosure also provide a non-transitory computer-readable storage medium. When the instructions in the storage medium are executed by a processor of a computer device, the computer device is enabled to perform the medical image segmentation method provided in the embodiments of the present disclosure.

[0131] The embodiments of the present disclosure further provide a computer program product, including a computer program / instruction, which implements the medical image segmentation method provided in the embodiments of the present disclosure when the computer program / instruction is executed by a processor.

[0132] The above description is merely an optional embodiment of the present disclosure and is not intended to limit the present disclosure. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present disclosure shall be included in the scope of protection of the present disclosure.

Claims

1. A medical image segmentation method, characterized in that: The method comprises: Add a data enhancement module and a feature fusion module to the first U-Net network to obtain a second U-Net network; Using the medical images in the training set to train the second U-Net network; After the second U-Net network is trained, the data enhancement module and the feature fusion module are removed to obtain the trained first U-Net network, and the medical images in the test set are input into the trained first U-Net network for medical image segmentation, where the medical images in the test set and the medical images in the training set are medical images of the first organ in different domains; In the second U-Net network, the encoder and decoder at the same layer are connected via a feature fusion module, the data enhancement module is connected to the encoder of the first layer, the data enhancement module is used to perform data enhancement on the first image input to the medical image segmentation model to obtain a second image, and the first image and the second image are respectively input to the encoder of the first layer to obtain two encoding features of the first layer, and the feature fusion module is used to fuse the two encoding features input to the feature fusion module to obtain a fused feature; In the second U-Net network, The encoder in the layer is used to output the The two encoded features of the layer, and the The two encoding features of the layer are input to the The feature fusion module in the layer and the Encoder in layer +1; The said The feature fusion module in the layer is used to The two encoding features of the layer are fused to obtain the The fusion features of the layer The fusion features of the layer are input to the Decoder in layer; The said The decoder in the layer is used based on the The fusion features of the layer are used to generate the Decoded features of the layer; in, is an integer, Greater than 0 and Less than , is the number of layers of the second U-Net network.

2. The method according to claim 1, characterized in that The coding feature corresponding to the first image is a first coding feature, the coding feature corresponding to the second image is a second coding feature, and the first coding feature is a second coding feature. The feature fusion module in the layer is used to implement the following method for the first The two encoding features of the layer are fused to obtain the Layer fusion features: Get the The first coding feature of the layer is positive attention features on the second encoded features of the layer; Get the The second coding feature of the layer is a reverse attention feature on the first encoding feature of a layer; Concatenate the forward attention feature and the reverse attention feature to obtain the first Layer fusion features.

3. The method according to claim 2, characterized in that The positive attention feature is calculated using the following formula: The reverse attention feature is calculated using the following formula: in, For the The forward attention features of the layer, No. The reverse attention feature of the layer, For the the first coded feature of the layer, For the the second encoded feature of the layer, is the normalized exponential function, For the The feature dimension of the layer.

4. The method according to any one of claims 1 to 3, characterized in that The data enhancement module is used to perform data enhancement on the first image input to the medical image segmentation model to obtain a second image, including: The data enhancement module is used to implement data enhancement on the first image by using any one of random cropping, random horizontal flipping, and random vertical flipping.

5. A medical image segmentation device, characterized in that: The device includes An acquisition module is used to add a data enhancement module and a feature fusion module to the first U-Net network to obtain a second U-Net network; A training module, configured to train the second U-Net network using medical images in a training set; a testing module, configured to, after the training of the second U-Net network is completed, remove the data enhancement module and the feature fusion module to obtain the trained first U-Net network, and input the medical images in the test set into the trained first U-Net network for medical image segmentation, wherein the medical images in the test set and the medical images in the training set are medical images of the first organ in different domains; In the second U-Net network, the encoder and decoder at the same layer are connected via a feature fusion module, the data enhancement module is connected to the encoder of the first layer, the data enhancement module is used to perform data enhancement on the first image input to the medical image segmentation model to obtain a second image, and the first image and the second image are respectively input to the encoder of the first layer to obtain two encoding features of the first layer, and the feature fusion module is used to fuse the two encoding features input to the feature fusion module to obtain a fused feature; In the second U-Net network, The encoder in the layer is used to output the The two encoded features of the layer, and the The two encoding features of the layer are input to the The feature fusion module in the layer and the Encoder in layer +1; The said The feature fusion module in the layer is used to The two encoding features of the layer are fused to obtain the The fusion features of the layer The fusion features of the layer are input to the Decoder in layer; The said The decoder in the layer is used based on the The fusion features of the layer are used to generate the Decoded features of the layer; in, is an integer, Greater than 0 and Less than , is the number of layers of the second U-Net network.

6. A computer device, characterized in that: The computer device includes: a memory and a processor, wherein at least one computer program is stored in the memory, and the at least one computer program is loaded and executed by the processor to implement the method according to any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that The computer-readable storage medium stores at least one computer program, and the at least one computer program is loaded and executed by a processor to implement the method according to any one of claims 1 to 4.

8. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the method according to any one of claims 1 to 4 is implemented.

Images