A medical image segmentation method based on data augmentation

Abstract

The present invention discloses a medical image segmentation method based on data augmentation. Aiming at the current situation that medical images lack labeled datasets, this method expands the dataset through data augmentation and then uses it to train the segmentation network. This method uses the method of image registration and draws on the cyclic consistency idea of the cyclic generative adversarial network. Through the spatial structure registration network based on cyclic consistency and the appearance result registration network based on cyclic consistency, a spatial transformation registration domain and an appearance transformation registration domain are generated. At the same time, the cyclic consistency loss is used to further improve and optimize the accuracy of image registration. Then, image registration is performed through the generated spatial transformation registration domain and appearance transformation registration domain, so as to achieve the purpose of expanding the labeled dataset, and finally it is used to train the image segmentation network. The present invention solves the problem of the scarcity of labeled medical image datasets and further improves the quality of the generated images, and has a wide range of applications in the field of medical image segmentation.

Description

Technical Field

[0001] The present invention belongs to the field of medical image processing, and relates to a method for segmenting medical image graphics, specifically a medical image segmentation method based on data augmentation. Background Art

[0002] With the vigorous development of medical imaging technology, medical images have extensive and in-depth applications in clinical medicine. According to statistics, tens of millions of cases are assisted in diagnosis and treatment through medical images globally every year. In traditional methods of medical imaging diagnosis and treatment, physicians read and identify medical image data, and make judgments on disease diagnosis and treatment. This diagnosis and treatment method is very inefficient, with large individual differences. Long-time film reading can cause doctor fatigue and a decline in the accuracy of film reading. With the rise of artificial intelligence, by pre-screening and judging image data with machines, marking key suspicious areas, and then handing them over to doctors for diagnosis and treatment, the workload of doctors can be greatly reduced, and the results are comprehensive, stable, and efficient. Therefore, artificial intelligence has important application prospects in the field of medical imaging.

[0003] In traditional medical image segmentation tasks, in order to fully train a neural network to achieve high-accuracy results, a large amount of relevant medical image data needs to be prepared, and these medical image data need to be manually labeled at the pixel level. There are various medical diseases, and the corresponding medical images are also diverse. When using deep learning for medical image segmentation, the medical images corresponding to each disease need to be manually labeled, consuming a large amount of manpower and material resources. Even the largest public medical image dataset can only provide pixel-level annotation samples for a limited number of semantic categories. High-quality data is scarce in medical image datasets, severely limiting the accuracy of semantic segmentation models.

[0004] Therefore, in medical image segmentation tasks, the dataset is usually augmented through data augmentation, and then a medical image segmentation model is trained based on the augmented dataset. Currently, the mainstream data augmentation methods include affine transformation (Affine Transformation), image registration (Image Registration), and generative adversarial networks (GAN, Generative Adversarial Networks). The disadvantage of affine transformation is that the generated image quality is poor; the disadvantage of image registration is that reliable feature extraction and robust feature consistency are required; while the disadvantages of generative adversarial networks are insufficient medical image feature extraction, large computational volume for adversarial training, etc. Therefore, there is a need for a new medical image segmentation method based on data augmentation to solve the above problems. Summary of the Invention

[0005] The problems to be solved by the present invention are as follows: In the field of medical image segmentation technology, due to the scarcity of labeled datasets, and the medical image segmentation technologies based on affine transformation, image registration, and generative adversarial networks still need to be improved in terms of image imaging quality, feature extraction, and computational complexity.

[0006] To solve the above problems, the present invention provides a medical image segmentation method based on data augmentation. This method incorporates the cycle-consistency idea of the CycleGAN (Cycle-Consistent Generative Adversarial Networks) into image registration to achieve the purpose of data augmentation, making the quality of the generated images higher for training the image segmentation network. At the same time, the present invention adopts an image registration method with a fixed source image, that is, the same image is used as the source image during both the training process and the testing process. The method includes the following steps:

[0007] 1) Collect a large number of nuclear magnetic resonance medical images of a specified size, including a small number of labeled images and a large number of unlabeled images, assuming there are a total of n images;

[0008] 2) First, select a labeled image as the source image, and the remaining n - 1 images except the source image as the target images. Each time, select one target image and pair it with the source image as inputs and send them into the cycle-consistency-based spatial structure registration network for training. After multiple iterations of optimization, obtain the cycle-consistency-based spatial structure registration network corresponding to the source image;

[0009] 3) Sequentially take the remaining n - 1 images except the source image as the target images and pair them with the selected source image and input them into the trained cycle-consistency-based spatial structure registration network to obtain n - 1 spatial transformation registration domains;

[0010] 4) Then still use the image selected in 2) as the source image, and the remaining n - 1 images except the source image as the target images. Each time, select one target image and pair it with the source image as inputs and send them into the cycle-consistency-based appearance structure registration network for training. After multiple iterations of optimization, obtain the cycle-consistency-based appearance structure registration network corresponding to the source image;

[0011] 5) Sequentially take the remaining n - 1 images except the source image as the target images and pair them with the selected source image and input them into the trained cycle-consistency-based appearance structure registration network to obtain n - 1 appearance transformation registration domains;

[0012] 6) Perform operations on the selected source image and the n - 1 spatial transformation registration domains obtained in 3) in sequence, that is, obtain n - 1 generated images with different spatial structures;

[0013] 7) Perform operations on the segmentation labels of the selected source image and the n - 1 spatial transformation registration domains obtained in 3) in sequence, that is, obtain the segmentation labels of these n - 1 generated images with different spatial structures;

[0014] 8) Perform operations on these n - 1 generated images with different spatial structures and the n - 1 appearance transformation registration domains obtained in 5) in sequence, then (n - 1)×(n - 1) final generated images with different spatial structures and different appearance structures can be obtained;

[0015] 9) Train an image segmentation network with the (n - 1)×(n - 1) labeled images generated in 8) and 7);

[0016] 10) Input the image to be segmented and output the corresponding segmentation label of this image.

[0017] Furthermore, in step 2), training the spatial structure registration network based on cycle consistency is specifically:

[0018] 2.1) The spatial structure registration network based on cycle consistency includes two VoxelMorph subnets;

[0019] 2.2) First, select a labeled image as the source image x s , and at the same time, arbitrarily select an image different from the source image x s as the target image y s , then input the source image x s and the target image y s into the first VoxelMorph network to obtain the spatial transformation registration domain τ s ;

[0020] 2.3) Perform operations on the source image x s and the spatial transformation registration domain τ s to obtain a generated image with a spatial structure similar to the target image y s ;

[0021] 2.4) Again, take as the source image, x s as the target image, and input both into the second VoxelMorph network to obtain the spatial transformation registration domain

[0022] 2.5) Perform operations on the source image and the spatial transformation registration domain to obtain a generated image with a spatial structure similar to the target image x s ;

[0023] 2.6) As shown in the following consistency loss function, x s As the source image of the spatial structure registration network based on cycle consistency, is the image generated after registration by the spatial structure registration network based on cycle consistency. Through the constraint of the consistency loss function, the generated image and the source image x s have a more similar spatial structure. The consistency loss function is:

[0024]

[0025] 2.7) During the training process, in order to maximize the performance of the spatial structure registration network based on cycle consistency, the network often generates a discontinuous spatial transformation registration domain. Therefore, a spatial smoothness constraint needs to be imposed on the predicted spatial transformation registration domain, that is, the spatial gradient of the spatial transformation registration domain is penalized. The smoothness loss function is:

[0026]

[0027] where τ s and respectively represent the spatial transformation registration domains obtained in 2.2) and 2.4), Ω represents the set of all points in the spatial transformation registration domain, and p represents a point at the same position in the two spatial transformation registration domains.

[0028] 2.8) The spatial structure registration network based on cycle consistency is constrained by the total loss function. The loss function is as follows:

[0029]

[0030] where x s is the source image, is the operator of the image and the registration domain, and λ s is the hyperparameter.

[0031] 2.9) After multiple iterative optimizations, the spatial structure registration network based on cycle consistency corresponding to the source image is obtained.

[0032] Furthermore, the n - 1 spatial transformation registration domains obtained in step 3) are specifically:

[0033] 3.1) Using the selected x s in 2) as the source image, except for the source image x s , the remaining n - 1 images y 2 , y 3 , …, y i , …, y nRespectively as the target images, they are sequentially input into the cycle-consistent spatial structure registration network trained in 2), and then n-1 spatial transformation registration domains can be obtained:

[0034] Furthermore, in step 4), training the cycle-consistent appearance structure registration network specifically includes:

[0035] 4.1) The cycle-consistent appearance structure registration network includes two Active Appearance subnets;

[0036] 4.2) Select the same images as in 2.2) as the source image x a , and arbitrarily select an image different from the source image x a as the target image y a , and then use the source image x a and the target image y a as inputs and feed them into the first Active Appearance network to obtain the appearance transformation registration domain τ a ;

[0037] 4.3) Perform operations on the source image x a and the appearance transformation registration domain τ a to obtain a generated image that is similar in appearance structure to the target image y a

[0038] 4.4) Again, use as the source image, x a as the target image, and use the two as inputs and feed them into the second Active Appearance network to obtain the appearance transformation registration domain

[0039] 4.5) Perform operations on the source image and the appearance transformation registration domain to obtain a generated image that is similar in appearance structure to the target image x a

[0040] 4.6) As shown in the following consistency loss function, x a is used as the source image of the cycle-consistent appearance structure registration network, and is the image generated after registration by the cycle-consistent appearance structure registration network. Through the constraint of the consistency loss function, the generated image and the source image x a are made more similar in appearance structure. The consistency loss function is:

[0041]

[0042] 4.7) During the training process, in order to maximize the performance of the cyclic-consistency based appearance-structure registration network, the network often produces a discontinuous appearance transformation registration domain. Therefore, a spatial smoothness constraint needs to be imposed on the predicted appearance transformation registration domain, that is, the spatial gradient of the appearance transformation registration domain is penalized. The smoothness loss function is as follows:

[0043]

[0044] where τ a and respectively represent the appearance transformation registration domains obtained in 4.2) and 4.4), Ω represents the set of all points in the appearance transformation registration domain, and p represents a point at the same position in the two appearance transformation registration domains.

[0045] 4.8) The cyclic-consistency based appearance-structure registration network is constrained by the total loss function, and the loss function is as follows:

[0046]

[0047] where x a is the source image, is the operator of the image and the registration domain, and λ is the hyperparameter.

[0048] 4.9) After multiple iterative optimizations, the cyclic-consistency based appearance-structure registration network corresponding to the source image is obtained.

[0049] Furthermore, the n - 1 appearance transformation registration domains obtained in step 5) are specifically as follows:

[0050] 5.1) Taking the image x s in 2) as the source image x a , and taking the remaining n - 1 images y a except the source image x 2 , y 3 , …, y i , …, y n as the target images respectively, and inputting them into the cyclic-consistency based appearance-structure registration network trained in 4) in sequence, n - 1 appearance transformation registration domains can be obtained:

[0051] Furthermore, the n - 1 generated images with different spatial structures obtained in step 6) are specifically as follows:

[0052] 6.1) Operating the selected source image x s sequentially with the n - 1 spatial transformation registration domains: , n - 1 generated images with spatial structures respectively the same as those of the target images y2 , y 3 , …, y i , …, y n Similar generated images

[0053] Furthermore, obtaining the segmentation labels of the n - 1 generated images with different spatial structures in step 6) in step 7) is specifically as follows:

[0054] 7.1) Sequentially perform operations on the segmentation label l s of the selected source image x s with n - 1 spatial transformation registration domains: to obtain the segmentation label corresponding to the generated image :

[0055] Furthermore, obtaining the (n - 1)×(n - 1) final generated images with different spatial structures and different appearance structures in step 8) is specifically as follows:

[0056] 8.1) Respectively use the generated image as the source image and the appearance transformation registration domain to perform operations, and the generated images with appearance structures similar to the target image y 2 , y 3 , …, y i , …, y n can be obtained:

[0057] 8.2) Since only the appearance structure of the source image is changed through the appearance transformation registration domain, and the spatial structure of the source image is not changed, the (n - 1)×(n - 1) images with different spatial structures and different appearance structures generated: The corresponding segmentation labels are:

[0058] The present invention provides a medical image segmentation method based on data augmentation. Based on the current situation that medical images lack labeled datasets, this method expands the dataset through data augmentation and then uses it to train the segmentation network. This method uses the method of image registration and draws on the cyclic consistency idea of the cyclic generative adversarial network. Through the spatial structure registration network based on cyclic consistency and the appearance structure registration network based on cyclic consistency, a spatial transformation registration domain and an appearance transformation registration domain are generated. At the same time, the accuracy of image registration is further improved and optimized through the cyclic consistency loss. Then, image registration is performed through the generated spatial transformation registration domain and appearance transformation registration domain, so as to achieve the purpose of expanding the labeled dataset, and finally it is used to train the image segmentation network. Applying the present invention solves the problem of the scarcity of labeled medical image datasets and further improves the quality of the generated images. The present invention is applicable to the medical image segmentation of multiple parts, has a low computational complexity, accurate segmentation results, and good algorithm robustness. The present invention has a wide range of applications in the field of medical image segmentation.

[0059] Advantages of the present invention: First, based on the current situation that medical image segmentation lacks labeled datasets, the present invention uses data augmentation to improve the performance of the medical image segmentation network, and at the same time generates a large number of labeled medical images for other tasks. Second, the present invention draws on the idea of the cyclic generative adversarial network to ensure the quality of the generated images. Brief Description of the Drawings

[0060] Figure 1 is a flowchart of the medical image segmentation method based on data augmentation of the present invention;

[0061] Figure 2 Spatial structure and appearance structure registration network model based on cyclic consistency. Detailed Embodiments

[0062] The present invention provides a medical image segmentation method based on data augmentation. This method uses nuclear magnetic resonance images as input, and there are a small number of labeled images and a large number of unlabeled images in the input images. Among them, the labeled images are used as source images, and the unlabeled images are used as target images. Both are used as inputs to train the spatial structure registration network based on cyclic consistency and the appearance structure registration network based on cyclic consistency respectively. After obtaining the spatial transformation registration domain and the appearance transformation registration domain, the labeled images and the registration domain are calculated to obtain the enhanced data, and then these data are used to train the segmentation network. As Figure 1 shown, the present invention includes the following steps:

[0063] 1) Collect a large number of nuclear magnetic resonance medical images of a specified size, including a small number of labeled images and a large number of unlabeled images. Assume there are a total of n images;

[0064] 2) First, select a labeled image as the source image, and the remaining n - 1 images as the target images. Each time, select a target image and the source image as a pair and input them into the cyclic-consistency based spatial structure registration network for training. After multiple iterations of optimization, the cyclic-consistency based spatial structure registration network corresponding to the source image is obtained;

[0065] 2.1) The cyclic-consistency based spatial structure registration network contains two VoxelMorph subnets;

[0066] 2.2) First, select a labeled image as the source image x s , and at the same time, arbitrarily select an image different from the source image x s as the target image y s . Then, input the source image x s and the target image y s into the first VoxelMorph network to obtain the spatial transformation registration field τ s ;

[0067] 2.3) Perform an operation on the source image x s and the spatial transformation registration field τ s to obtain a generated image with a spatial structure similar to that of the target image y s ;

[0068] 2.4) Again, use as the source image, x s as the target image, and input the two into the second VoxelMorph network to obtain the spatial transformation registration field

[0069] 2.5) Perform an operation on the source image and the spatial transformation registration field to obtain a generated image with a spatial structure similar to that of the target image x s ;

[0070] 2.6) As shown in the following consistency loss function, x s is used as the source image of the cyclic-consistency based spatial structure registration network, and is the image generated after registration by the cyclic-consistency based spatial structure registration network. Through the constraint of the consistency loss function, the spatial structures of the generated image and the source image x s become more similar. The consistency loss function is:

[0071]

[0072] 2.7) During the training process, in order to maximize the performance of the cyclic-consistency based spatial structure registration network, the network often produces a discontinuous spatial transformation registration domain. Therefore, a spatial smoothness constraint needs to be imposed on the predicted spatial transformation registration domain, that is, the spatial gradient of the spatial transformation registration domain is penalized. The smoothness loss function is as follows:

[0073]

[0074] where τ s and represent the spatial transformation registration domains obtained in 2.2) and 2.4) respectively, Ω represents the set of all points in the spatial transformation registration domain, and p represents a point at the same position in the two spatial transformation registration domains.

[0075] 2.8) The cyclic-consistency based spatial structure registration network is constrained by the total loss function, and the loss function is as follows:

[0076]

[0077] where x s is used as the source image, is used as the operator for the image and the registration domain, and λ s is used as the hyperparameter.

[0078] 2.9) After multiple iterative optimizations, the cyclic-consistency based spatial structure registration network corresponding to the source image is obtained.

[0079] 3) Using the selected x s in 2) as the source image, and taking the remaining n - 1 images y s except the source image x 2 , y 3 , …, y i , …, y n as the target images respectively, and inputting them into the cyclic-consistency based spatial structure registration network trained in 2) in sequence, n - 1 spatial transformation registration domains can be obtained:

[0080] 4) Then still using the selected image in 2) as the source image, and taking the remaining n - 1 images except the source image as the target images. Each time, one target image and the source image are selected as a pair of inputs and sent into the cyclic-consistency based appearance structure registration network for training. After multiple iterative optimizations, the cyclic-consistency based appearance structure registration network corresponding to the source image is obtained;

[0081] 4.1) The cyclic-consistency based appearance structure registration network includes two Active Appearance subnets;

[0082] 4.2) Select the same image as in 2.2) as the source image x a , and optionally select an image different from the source image x a as the target image y a , then take the source image x a and the target image y a as inputs and feed them into the first Active Appearance Network to obtain the appearance transformation registration field τ a ;

[0083] 4.3) Perform an operation on the source image x a and the appearance transformation registration field τ a to obtain a generated image with an appearance structure similar to that of the target image y a

[0084] 4.4) Again, take as the source image, x a as the target image, and use the two as inputs to feed into the second Active Appearance Network to obtain the appearance transformation registration field

[0085] 4.5) Perform an operation on the source image and the appearance transformation registration field to obtain a generated image with an appearance structure similar to that of the target image x a

[0086] 4.6) As shown in the following consistency loss function, x a is used as the source image of the appearance structure registration network based on cycle consistency, is the image generated after being registered by the appearance structure registration network based on cycle consistency. Through the constraint of the consistency loss function, the appearance structure of the generated image and the source image x a becomes more similar. The consistency loss function is:

[0087]

[0088] 4.7) During the training process, in order to maximize the performance of the appearance structure registration network based on cycle consistency, the network often produces a discontinuous appearance transformation registration field. Therefore, a spatial smoothness constraint is imposed on the predicted appearance transformation registration field, that is, the spatial gradient of the appearance transformation registration field is penalized. The smoothness loss function is:

[0089]

[0090] where τ a and​​ respectively represent the appearance transformation registration domains obtained in 4.2) and 4.4), Ω represents the set of all points in the appearance transformation registration domain, and p represents a point at the same position in the two appearance transformation registration domains.

[0091] 4.8) Constrain the cyclic-consistency based appearance-structure registration network through the total loss function, and the loss function is as follows:

[0092]

[0093] where x a is the source image, is the operator of the image and the registration domain, and λ a is the hyperparameter.

[0094] 4.9) After multiple iterative optimizations, obtain the cyclic-consistency based appearance-structure registration network corresponding to the source image.

[0095] 5) Take the image x in 2) s as the source image x a , and except for the source image x a , the remaining n - 1 images y 2 , y 3 , …, y i , …, y n are used as the target images respectively, and input them into the cyclic-consistency based appearance-structure registration network trained in 4) in sequence, then n - 1 appearance transformation registration domains can be obtained:

[0096] 6) Take the selected source image x s , and perform operations with the n - 1 spatial transformation registration domains: in sequence, then n - 1 generated images with spatial structures similar to the target images y 2 , y 3 , …, y i , …, y n can be obtained.

[0097] 7) Take the segmentation label l s of the selected source image x s , and perform operations with the n - 1 spatial transformation registration domains: in sequence, then the segmentation label corresponding to the generated image can be obtained:

[0098] 8) The n-1 generated images with different spatial structures are successively and separately operated on with the n-1 appearance transformation registration domains obtained in 5), and (n-1)×(n-1) final generated images with different spatial structures and different appearance structures can be obtained;

[0099] 8.1) Respectively take the generated images as the source images and the appearance transformation registration domains for operation, and generated images with appearance structures similar to the target images y 2 , y 3 , …, y i , …, y n can be obtained:

[0100] 8.2) Since only the appearance structure of the source image is changed through the appearance transformation registration domain, and the spatial structure of the source image is not changed, therefore, the n-1×n-1 images with different spatial structures and different appearance structures generated: corresponding segmentation labels are:

[0101] 9) Train the image segmentation network with the (n-1)×(n-1) labeled images generated in 8) and 7);

[0102] 10) Input the image to be segmented, and output the corresponding segmentation label of the image.

[0103] The present invention has wide applications in the field of medical image segmentation technology. The present invention will be described in detail below with reference to the accompanying drawings.

[0104] (1) In the embodiment of the present invention, a large number of nuclear magnetic resonance medical images of a specified size are collected, including a small number of labeled images and a large number of unlabeled images. Assume there are n images in total;

[0105] (2) First, select a labeled image as the source image, and the remaining n-1 images except the source image as the target images. Each time, select a target image and the source image as a pair and input them into the cyclic consistency-based spatial structure registration network for training. After multiple iterative optimizations, the cyclic consistency-based spatial structure registration network corresponding to the source image is obtained;

[0106] (2.1) First, select a labeled image as the source image x s , and at the same time, arbitrarily select an image different from the source image x s as the target image y s , then the source image x s and the target image y sIt is sent as input into the first VoxelMorph network to obtain the spatial transformation registration domain τ s ;

[0107] (2.2) Perform an operation on the source image x s and the spatial transformation registration domain τ s to obtain a generated image with a spatial structure similar to the target image y s

[0108] (2.3) Again, use as the source image and x s as the target image, and send the two as inputs into the second VoxelMorph network to obtain the spatial transformation registration domain

[0109] (2.4) Perform an operation on the source image and the spatial transformation registration domain to obtain a generated image with a spatial structure similar to the target image x s

[0110] (2.5) Use the consistency loss and the smooth loss to train the model simultaneously, and use the sum of the two as the final loss of the model to obtain the cyclic consistency-based spatial structure registration network corresponding to the source image. Set the parameter λ s = 0.02.

[0111] 3) Sequentially take the remaining n - 1 images except the source image as target images and pair them with the selected source image to input into the trained cyclic consistency-based spatial structure registration network to obtain n - 1 spatial transformation registration domains;

[0112] 4) Then still use the image selected in 2) as the source image and the remaining n - 1 images except the source image as target images. Each time, select one target image and pair it with the source image and input them into the cyclic consistency-based appearance structure registration network for training. After multiple iterations of optimization, obtain the cyclic consistency-based appearance structure registration network corresponding to the source image;

[0113] 4.1) Select the same image as in 2.1) as the source image x a , and randomly select an image different from the source image x a as the target image y a , and then send the source image x a and the target image y a as inputs into the first Active Appearance network to obtain the appearance transformation registration domain τ;

[0114] ​​4.2) Perform an operation on the source image x a and the appearance transformation registration domain τ a to obtain a generated image with an appearance structure similar to the target image y a

[0115] 4.3) Again, use as the source image and x a as the target image, and feed the two as inputs into the second Active Appearance Network to obtain the appearance transformation registration domain

[0116] 4.4) Perform an operation on the source image and the appearance transformation registration domain to obtain a generated image with an appearance structure similar to the target image x a

[0117] 4.5) Use the consistency loss and the smoothing loss to train the model simultaneously, and use the sum of the two as the final loss of the model to obtain the cyclic consistency-based appearance structure registration network corresponding to the source image. Set the parameter λ a = 0.02.

[0118] 5) Sequentially use the remaining n - 1 images except the source image as target images and pair them with the selected source image to input into the trained cyclic consistency-based appearance structure registration network to obtain n - 1 appearance transformation registration domains;

[0119] 6) Perform an operation on the selected source image and the n - 1 spatial transformation registration domains obtained in 3) in sequence, i.e., obtain n - 1 generated images with different spatial structures;

[0120] 7) Perform an operation on the segmentation labels of the selected source image and the n - 1 spatial transformation registration domains obtained in 3) in sequence, i.e., obtain the segmentation labels of these n - 1 generated images with different spatial structures;

[0121] 8) Perform an operation on these n - 1 generated images with different spatial structures and the n - 1 appearance transformation registration domains obtained in 5) in sequence respectively, and then (n - 1)×(n - 1) final generated images with different spatial structures and different appearance structures can be obtained;

[0122] 9) Use the (n - 1)×(n - 1) labeled images generated in 8) and 7) to train the image segmentation network;

[0123] 10) Input the image to be segmented and output the segmentation label corresponding to the image.

[0124] ​​Under the Intel Core i9-10980 and Ubuntu 20.04-bit operating system, this method is implemented by programming with Python 3.6.

[0125] The present invention provides a medical image segmentation method based on data augmentation, which is applicable to medical image segmentation tasks, has a high segmentation accuracy, and good algorithm robustness. Experiments show that this method can perform medical image segmentation quickly and effectively.

Claims

1. A medical image segmentation method based on data augmentation, characterized in that, For a given small number of labeled images and a large number of unlabeled images, the following operations are performed: 1) Collect nuclear magnetic resonance medical images, including labeled images and unlabeled images, assuming there are a total of n images; 2) First, select a labeled image as the source image, and the remaining n - 1 images except the source image as the target images; Each time, select a target image and the source image as a pair and input them into the cycle-consistent spatial structure registration network for training. After multiple iterations of optimization, the cycle-consistent spatial structure registration network corresponding to the source image is obtained; 3) Sequentially take the remaining n - 1 images except the source image as target images and pair them with the selected source image and input them into the trained cycle-consistent spatial structure registration network to obtain n - 1 spatial transformation registration domains; 4) Then still use the image selected in 2) as the source image, and the remaining n - 1 images except the source image as the target images; Each time, select a target image and the source image as a pair and input them into the cycle-consistent appearance structure registration network for training. After multiple iterations of optimization, the cycle-consistent appearance structure registration network corresponding to the source image is obtained; 5) Sequentially take the remaining n - 1 images except the source image as target images and pair them with the selected source image and input them into the trained cycle-consistent appearance structure registration network to obtain n - 1 appearance transformation registration domains; 6) Perform operations on the selected source image and the n - 1 spatial transformation registration domains obtained in 3) in sequence, that is, obtain n - 1 generated images with different spatial structures; 7) Perform operations on the segmentation labels of the selected source image and the n - 1 spatial transformation registration domains obtained in 3) in sequence, that is, obtain the segmentation labels of these n - 1 generated images with different spatial structures; 8) Perform operations on these n - 1 generated images with different spatial structures and the n - 1 appearance transformation registration domains obtained in 5) in sequence respectively, and then (n - 1)×(n - 1) final generated images with different spatial structures and different appearance structures can be obtained; 9) Use the (n - 1)×(n - 1) labeled images generated in 8) and 7) to train the image segmentation network; 10) Input the image to be segmented and output the corresponding segmentation label of the image.

2. The medical image segmentation method based on data augmentation according to claim 1, wherein, In step 2), when training the cycle-consistent spatial structure registration network, specifically: 2.1) The cycle-consistent spatial structure registration network contains two VoxelMorph subnets; 2.2) First, select a labeled image as the source image x s , and at the same time, arbitrarily select an image different from the source image x s as the target image y s . Then, take the source image x s and the target image y s as inputs and send them into the first VoxelMorph network to obtain the spatial transformation registration field τ s ; 2.3) Perform an operation on the source image x s and the spatial transformation registration domain τ s to obtain a generated image with a spatial structure similar to the target image y s ​ 2.4) Again, take as the source image, and x s as the target image. Feed both of them into the second VoxelMorph network as inputs to obtain the spatial transformation registration field 2.5) Perform an operation on the source image and the spatial transformation registration domain to obtain a generated image that is spatially structurally similar to the target image x s ​ 2.6) In the consistency loss function, x s serves as the source image of the cyclic-consistency based spatial structure registration network, is the image generated after being registered by the cyclic-consistency based spatial structure registration network; through the constraint of the consistency loss function, the generated image and the source image x s have a more similar spatial structure; the consistency loss function is: 2.7) During the training process, impose a spatial smoothness constraint on the predicted spatial transformation registration domain, that is, punish the spatial gradient of the spatial transformation registration domain. The smoothness loss function is: where τ s and represent the spatial transformation registration fields obtained in 2.2) and 2.4) respectively, Ω represents the set of all points in the spatial transformation registration field, and p represents a point at the same position in the two spatial transformation registration fields; 2.8) Constrain the cycle-consistent spatial structure registration network through the total loss function. The loss function is as follows: where x s serves as the source image, serves as the operator for the image and the registration domain, λ s serves as a hyperparameter; 2.9) After multiple iterations of optimization, the cycle-consistent spatial structure registration network corresponding to the source image is obtained.

3. The medical image segmentation method based on data augmentation according to claim 1, wherein, In step 3), when obtaining n - 1 spatial transformation registration domains, specifically: 3.1) Take the selected x in 2) s as the source image, and for the remaining n - 1 images y s except the source image x 2 , y 3 , …, y i , …, y n take them as the target images respectively, and input them into the cycle-consistent spatial structure registration network trained in 2) in sequence, then n - 1 spatial transformation registration domains can be obtained:

4. The medical image segmentation method based on data augmentation according to claim 1, wherein In step 4), when training the cycle-consistent appearance structure registration network, specifically: 4.1) The appearance structure registration network based on cycle consistency contains two Active Appearance subnets; 4.2) Select the same image as in 2.2) as the source image x a , and optionally select an image different from the source image x a as the target image y a , and then use the source image x a and the target image y a as inputs and feed them into the first Active Appearance Network to obtain the appearance transformation registration field τ a ; 4.3) Perform an operation on the source image x a and the appearance transformation registration domain τ a to obtain a generated image with an appearance structure similar to the target image y a ​ 4.4) Again, take as the source image, and x a as the target image. Feed both of them into the second Active Appearance Network as inputs to obtain the appearance transformation registration domain 4.5) Perform an operation on the source image and the appearance transformation registration domain to obtain a generated image that is similar in appearance structure to the target image x a ​ 4.6) In the consistency loss function, x a serves as the source image of the cyclic consistency-based appearance structure registration network, is the image generated after registration by the cyclic consistency-based appearance structure registration network; through the constraint of the consistency loss function, the generated image and the source image x a are more similar in appearance structure; the consistency loss function is: 4.7) During the training process, a constraint on the spatial smoothness is imposed on the predicted appearance transformation registration field, that is, the spatial gradient of the appearance transformation registration field is penalized, and the smoothness loss function is: where τ a and represent the appearance transformation registration fields obtained in 4.2) and 4.4) respectively, Ω represents the set of all points in the appearance transformation registration field, and p represents a point at the same position in the two appearance transformation registration fields; 4.8) The appearance structure registration network based on cycle consistency is constrained by the total loss function, and the loss function is as follows: where x a serves as the source image, serves as the operator for the image and the registration field, and λ a serves as the hyperparameter; 4.9) After multiple iterative optimizations, the appearance structure registration network based on cycle consistency corresponding to the source image is obtained.

5. The medical image segmentation method based on data augmentation according to claim 1, wherein In step 5), n - 1 appearance transformation registration fields are obtained, specifically: 5.1) Take the image x in 2) s as the source image x a , except for the source image x a the remaining n - 1 images y 2 , y 3 , …, y i , …, y n are respectively used as the target images and input into the cycle - consistency - based appearance structure registration network trained in 4) in sequence, and then n - 1 appearance transformation registration domains can be obtained:

6. The medical image segmentation method based on data augmentation according to claim 1, wherein, In step 6), n - 1 generated images with different spatial structures are obtained, specifically: 6.1) The selected source image x s , is successively operated with n - 1 spatial transformation registration domains: to obtain n - 1 generated images whose spatial structures are respectively similar to the target images y 2 , y 3 , …, y i , …, y n ​ 7. The method for medical image segmentation based on data augmentation according to claim 1, wherein In step 7), the segmentation labels of the n - 1 generated images with different spatial structures in 6) are obtained, specifically: 7.1) The segmentation label l of the selected source image x s is successively operated with n - 1 spatial transformation registration domains: s to obtain the segmentation label corresponding to the generated image :

8. A method for medical image segmentation based on data augmentation according to claim 1, characterized in that In step 8), (n - 1)×(n - 1) final generated images with different spatial structures and different appearance structures are obtained, specifically: 8.1) Respectively take the generated images as the source image and the appearance transformation registration domain for operations, where j = 2, 3, …, n, to obtain generated images whose appearance structures are respectively similar to the target images y 2 , y 3 , …, y i , …, y n : 8.2) Since only the appearance structure of the source image is changed through appearance transformation registration in the registration domain, without changing the spatial structure of the source image, (n - 1)×(n - 1) images with different spatial structures and different appearance structures are generated: The corresponding segmentation labels are:

Images