Medical image registration and segmentation method and system, and computer equipment

Abstract

The invention provides a medical registration and segmentation method and system and computer equipment, and the method comprises a segmentation model training stage and a medical image registration and segmentation stage, and comprises the steps: carrying out the data expansion and fusion of a to-be-segmented medical image, and obtaining a group of training medical images; the method comprises the steps of obtaining a training medical image, performing feature extraction and registration on the training medical image to obtain a set of registered training medical image data, training a segmentation model based on the set of data to obtain a trained segmentation model, and segmenting a medical image to be segmented according to a registration model and the segmentation model to obtain a segmented medical image. Compared with a pure segmentation method, registration and segmentation are combined, and adaptive registration-segmentation training is performed on the segmentation model in advance, so that key information can be extracted more accurately after the image to be segmented is registered and segmented by the trained segmentation model, and the segmentation effect is remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and particularly to the field of methods and systems for medical image registration and segmentation. Background Art

[0002] Medical image segmentation is a task of identifying pixel points of organs or lesion regions from medical images such as computed tomography (CT) or magnetic resonance imaging (MRI), and extracting the shape and volume information of these structures. This is one of the most challenging tasks in medical image analysis, playing a key role in computer-aided diagnosis and intelligent healthcare, and greatly improving the efficiency and accuracy of diagnosis. Currently, popular medical image segmentation tasks include liver and liver tumor segmentation, brain and brain tumor segmentation, cell segmentation, lung segmentation, etc.

[0003] Medical image registration aligns corresponding anatomical structures in two images precisely in space by finding a suitable spatial transformation, ensuring that the positions of the same anatomical points are consistent in the two images, so as to achieve image alignment and information complementation, and help doctors make better diagnoses. Currently, medical image registration has wide applications in clinics, such as image guidance, image fusion, tumor growth monitoring, etc.

[0004] With the continuous progress of computer hardware devices, deep learning methods have emerged in the field of image processing tasks, demonstrating powerful capabilities. Especially in the field of medical image processing, they have gradually become an important part of image segmentation. In practical applications, due to environmental conditions and equipment limitations, the medical images collected often have missing information. Traditional registration methods are difficult to achieve sufficient accuracy when dealing with multi-modal images and cannot effectively capture the subtle differences between complex anatomical structures and lesion sites.

[0005] In terms of image segmentation, traditional methods rely on manually designed features and are difficult to handle the blurred boundaries and complex morphological changes in medical images. Especially in the segmentation of three-dimensional medical data, they face huge challenges and are difficult to fully meet the diagnostic needs of doctors.

[0006] Traditional convolutional neural networks have made significant progress in the field of medical image processing. However, their ability to capture long-range spatial relationships in images is limited, which affects the understanding of global context. Transformer (ViT) performs well in dealing with long-range relationships, but due to the loss of fine-grained localization information during its continuous downsampling process, it is not suitable for precise image registration tasks. In addition, many existing registration methods mainly target two-dimensional images, while medical imaging usually generates three-dimensional volume images, and two-dimensional methods are difficult to fully utilize three-dimensional spatial information. These defects indicate that developing more advanced three-dimensional registration methods is crucial for improving the accuracy of medical image registration and segmentation. Summary of the Invention

[0007] Aiming at the problem of low accuracy in existing medical image registration and medical image segmentation methods, the present invention uses the output of the registration model as the training data for the segmentation model of medical images, enabling the segmentation model to learn the registration features corresponding to the registration model. After training, the segmentation model can segment specific anatomical structures, thereby extracting the required key medical information, ensuring the high quality and consistency of medical image registration and segmentation, and thus improving the quality of medical image segmentation.

[0008] On the one hand, the present invention proposes a medical registration and segmentation method, including: a segmentation model training stage and a medical image registration and segmentation stage;

[0009] In the segmentation model training stage, the following steps are executed:

[0010] D10: Perform data augmentation on the medical images to be segmented to obtain a set of training medical images;

[0011] D21: Extract features from the set of training medical images to obtain the features of the set of training medical images;

[0012] D22: Extract features from a standard image to obtain the standard image features;

[0013] D30: Perform convolution processing on each training medical image feature in the set respectively with the standard image features to obtain a set of training deformation fields; where each training deformation field is the deformation field between a training medical image in the set and the standard image;

[0014] D40: Interpolate and deform each training medical image according to its corresponding training deformation field to obtain a set of registered training medical images;

[0015] D50: Train a segmentation model according to the registered training medical images to obtain a trained segmentation model;

[0016] In the medical image registration and segmentation stage, the following steps are executed:

[0017] S10: Extract features from the medical image to be segmented to obtain the features of the medical image to be segmented;

[0018] S20: Perform convolution processing on the features of the medical image to be segmented and the above standard image features to obtain a deformation field to be segmented;

[0019] S30: Interpolate and deform the medical image to be segmented according to the deformation field to be segmented to obtain a registered medical image to be segmented;

[0020] S40: Input the registered medical image to be segmented into the trained segmentation model for segmentation processing to obtain the segmented medical image.

[0021] Furthermore, the medical image to be segmented includes MRI images and CT images, and the training medical images in step D10 are obtained by the following method:

[0022] Perform variational mode decomposition on MRI images and CT images respectively to obtain high-frequency MRI components and low-frequency MRI components, as well as high-frequency CT components and low-frequency CT components;

[0023] Perform weighted average fusion on the low-frequency MRI component and the low-frequency CT component to obtain a low-frequency fusion image;

[0024] Perform adversarial generation fusion on the high-frequency MRI component and the high-frequency CT component to obtain a group of high-frequency fusion images;

[0025] Perform inverse variational mode decomposition on the low-frequency fusion image and each high-frequency fusion image to obtain a group of training medical images.

[0026] Furthermore, the weighted average fusion formula is:

[0027] Low_Fusion = λ 1 ×Low_CT + λ 2 ×Low_MRI

[0028] where Low_Fusion is the low-frequency fusion image, λ 1 is the weighted coefficient of the low-frequency CT component, and λ 2 is the weighted coefficient of the low-frequency MRI component.

[0029] Furthermore, in step D40, after obtaining a group of registered training medical images, the following steps are further included:

[0030] Calculate the reconstruction loss between the group of registered training medical images and the standard image, and calculate the regularization loss between the group of registered training medical images and the standard image;

[0031] If the reconstruction loss is greater than a first threshold, or the regularization loss is greater than a second threshold, optimize the training deformation field by backpropagation according to the reconstruction loss and the regularization loss;

[0032] Interpolate and deform each training medical image according to the optimized training deformation field until the reconstruction loss is less than the first threshold and the regularization loss is less than the second threshold;

[0033] and / or

[0034] In step S30, after obtaining a registered medical image to be segmented, the following steps are further included:

[0035] Calculate the reconstruction loss between the group of registered medical images to be segmented and the standard image, and calculate the regularization loss between the registered medical image to be segmented and the standard image;

[0036] If the reconstruction loss is greater than a first threshold, or the regularization loss is greater than a second threshold, optimize the deformation field to be segmented by backpropagation according to the reconstruction loss and the regularization loss;

[0037] Interpolate and deform the medical image to be segmented according to the optimized deformation field to be segmented until the reconstruction loss is less than the first threshold and the regularization loss is less than the second threshold.

[0038] Furthermore, the segmentation model is a Cascade-Unet model with a Unet model as the basic network architecture.

[0039] On the other hand, the present invention also provides a medical image registration and segmentation system, which includes an image augmentation unit, a feature extraction unit, a deformation field extraction unit, an image registration unit, a segmentation model, and a segmentation model training unit; first, perform the segmentation model training steps, including:

[0040] The image augmentation unit performs data augmentation on the medical image to be segmented to obtain a group of training medical images;

[0041] The feature extraction unit respectively extracts features from the group of training medical images and a standard image to obtain the group of training medical image features and the standard image features;

[0042] The deformation field extraction unit performs convolution processing on each training medical image feature in the group with the standard image feature to obtain a group of training deformation fields; each training deformation field is the deformation field between a training medical image in the group and the standard image;

[0043] The image registration unit interpolates and deforms each training medical image according to its corresponding training deformation field to obtain a group of registered training medical images;

[0044] The segmentation model training unit inputs the set of registered training medical images into the segmentation model, trains the segmentation model, and obtains a trained segmentation model;

[0045] Then, perform the medical image registration and segmentation steps, including:

[0046] The feature extraction unit extracts features from the medical image to be segmented, and obtains the features of the medical image to be segmented;

[0047] The deformation field extraction unit performs convolution processing on the features of the medical image to be segmented and the standard image features, and obtains a deformation field to be segmented;

[0048] The image registration unit interpolates and deforms the medical image to be segmented according to the deformation field to be segmented, and obtains a registered medical image to be segmented;

[0049] The registered medical image to be segmented is input into the trained segmentation model for segmentation processing, and a segmented medical image is obtained.

[0050] Furthermore, the medical image to be segmented includes MRI images and CT images, and the image augmentation unit is an image fusion model, which includes:

[0051] The variational mode decomposition sub-module: used to perform variational mode decomposition on MRI images and CT images respectively, and obtain high-frequency MRI components and low-frequency MRI components, as well as high-frequency CT components and low-frequency CT components;

[0052] The low-frequency component weighted fusion sub-module: used to perform weighted average fusion on the low-frequency MRI component and the low-frequency CT component, and obtain a low-frequency fusion image;

[0053] The high-frequency component adversarial fusion sub-module: used to perform adversarial generation fusion on the high-frequency MRI component and the high-frequency CT component, and obtain a set of high-frequency fusion images;

[0054] The inverse variational mode decomposition sub-module: used to perform inverse variational mode decomposition on the low-frequency fusion image and each high-frequency fusion image, and obtain a set of training medical images.

[0055] Furthermore, the weighted average fusion formula is:

[0056] Low_Fusion = λ 1 ×Low_CT + λ 2 ×Low_MRI

[0057] where, Low_Fusion is the low-frequency fusion image, λ 1 is the low-frequency CT component weighting coefficient, λ 2is the weighting coefficient of the low-frequency MRI component.

[0058] Furthermore, any of the above systems further includes a deformation field optimization unit, which includes:

[0059] A loss calculation sub-module: used to calculate the reconstruction loss between the registered training medical images and the standard image, and calculate the regularization loss between the registered training medical images and the standard image;

[0060] A backpropagation optimization sub-module, used to judge: if the reconstruction loss is greater than a first threshold, or the regularization loss is greater than a second threshold, then perform backpropagation optimization on the training deformation field according to the reconstruction loss and the regularization loss;

[0061] An image update sub-module, used to perform interpolation deformation on each training medical image according to the optimized training deformation field until the reconstruction loss is less than the first threshold and the regularization loss is less than the second threshold.

[0062] On the other hand, the present invention also provides a computer device, including:

[0063] At least one memory and at least one processor;

[0064] The memory is used to store one or more programs;

[0065] When the one or more programs are executed by the at least one processor, the at least one processor implements the steps of a method for registering and segmenting a medical image as described in any one of the above.

[0066] A medical registration and segmentation method proposed by the present invention combines registration and segmentation, which can effectively improve the spatial alignment accuracy of three-dimensional medical images, thereby enhancing the accuracy of subsequent segmentation. Compared with a simple segmentation method, the present invention pre-trains a segmentation model for adaptive registration-segmentation. After the image to be segmented is registered, the trained segmentation model can more accurately extract key information during segmentation, and the segmentation effect is significantly improved. In addition, the medical data used for initializing registration-segmentation is obtained by fusing MRI and CT images. With the help of a generative adversarial network, on the one hand, a large amount of effective medical data is obtained, solving the problem of insufficient training data. On the other hand, the high-frequency and low-frequency components of MRI and CT images are decomposed by variational mode decomposition, and then the low-frequency components are fused by weighted average to obtain a low-frequency image. The high-frequency part is input into the generative adversarial network for processing to output a high-frequency image. Finally, the fused image is obtained by inverse variational mode decomposition as the medical image for model training. This set of medical images is fused in the high-frequency and low-frequency domains respectively, highly restoring the spatial features and texture features of MRI and CT images, greatly improving the data quality during model training, and effectively improving the accuracy of registration and segmentation of the trained model.

[0067] For better understanding and implementation, the present invention will be described in detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Figure 1 is a structural block diagram of the medical image registration and segmentation system of the present invention;

[0069] Figure 2 For execution Figure 1 is a flowchart of the medical registration and segmentation method of the medical registration and segmentation system shown;

[0070] Figure 3 is a structural block diagram of an exemplary image fusion unit of the present invention;

[0071] Figure 4 is an execution flowchart of an exemplary image fusion unit of the present invention;

[0072] Figure 5 is an execution flowchart of an exemplary deformation field optimization unit of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0073] Traditional image segmentation methods rely on manually designed features and are difficult to handle the blurred boundaries and complex morphological changes in medical images. Especially in the segmentation of three-dimensional medical data, medical images with different morphologies and angles emerge in an endless stream. Even after registering the medical images to be segmented through a deformation field and then performing segmentation, there are significant differences in the anatomical structures and structural features of each standard image. Therefore, existing segmentation models are difficult to meet the requirements of blurred boundaries and complex morphological changes in complex medical images, that is, the segmentation accuracy is difficult to achieve the expected level.

[0074] Based on this, the medical registration and segmentation system constructed by the present invention combines the registration model and the segmentation model. When registering and segmenting medical images each time, first, the medical images to be segmented are expanded to form a training data set. Then, after registering the training data set through the registration model, it is input into the segmentation model to train the segmentation model, forming a trained segmentation model. Then, after registering the medical images to be segmented, they are input into the trained segmentation model to obtain the segmented anatomical structure diagram.

[0075] Specifically, please refer to Figure 1 , the medical registration and segmentation system of the present invention includes an image expansion unit 10, a feature extraction unit 20, a deformation field extraction unit 30, an image registration unit 40, a segmentation model A, and a segmentation model training unit 50.

[0076] The execution of a task by this medical registration and segmentation system on the medical images to be segmented includes two stages: the segmentation model training stage and the medical image registration and segmentation stage.

[0077] Please also refer to Figure 2 , in the segmentation model training stage: first, the image expansion unit 10 expands the medical images to be segmented to obtain a set of training medical images generated from the medical images to be segmented; the feature extraction unit 20 extracts features from this set of training medical images and a standard image respectively to obtain the features of this set of training medical images and the features of the standard image; the deformation field extraction unit 30 performs convolution processing on the features of each training medical image in this set with the features of the standard image respectively to obtain a set of training deformation fields; the image registration unit 40 interpolates and deforms each training medical image according to its corresponding training deformation field to obtain a set of registered training medical images; the segmentation model training unit 50 inputs the set of registered training medical images into the segmentation model A to train the segmentation model A to obtain a trained segmentation model B.

[0078] In the medical image registration stage: the feature extraction unit 20 extracts features from the medical image to be segmented to obtain the features of the medical image to be segmented; the deformation field extraction unit 30 performs convolution processing on the features of the medical image to be segmented and the standard image features to obtain a deformation field to be segmented; the image registration unit interpolates and deforms the medical image to be segmented according to the deformation field to be segmented to obtain a registered medical image to be segmented; the registered medical image to be segmented is input into the trained segmentation model B for segmentation processing to obtain a segmented medical image.

[0079] The following details the specific execution steps of each unit and model in the medical registration and segmentation system of the present invention:

[0080]

Segmentation model training stage

[0081] First, the image augmentation unit executes step D10: performing data augmentation on the medical image to be segmented to obtain a set of training medical images; wherein, the medical image to be segmented includes MRI images and CT images.

[0082] Since there is a lack of a large number of labeled data sets in medical image segmentation, this is not only because the number of data sets is small, but also because it is difficult to define the labels in the data set. However, training a deep neural network with a small number of data sets will inevitably result in overfitting, and the extracted medical information will not be good. Therefore, the present invention creatively combines a convolutional neural network with variational mode decomposition to perform feature extraction and fusion on MRI images and CTs step by step to obtain a medical image with richer high-frequency and low-frequency features, enabling it to be used for medical registration and segmentation and improving the quality of the training data of the medical registration model and the medical segmentation model. Specifically, please refer to Figure 3 and Figure 4 , at this time the image augmentation unit is an image fusion model, and the training medical images in step D10 are generated by this image fusion model. The model includes a variational mode decomposition sub-module, a low-frequency component weighted fusion sub-module, a high-frequency component adversarial fusion sub-module, and a variational mode decomposition inverse transform sub-module. The execution flows of each sub-module are as follows:

[0083] Variational mode decomposition sub-module: used to perform variational mode decomposition on MRI images and CT images respectively to obtain high-frequency MRI components and low-frequency MRI components, as well as high-frequency CT components and low-frequency CT components;

[0084] Low-frequency component weighted fusion sub-module: used to perform weighted average fusion on the low-frequency MRI component and the low-frequency CT component to obtain a low-frequency fusion image;

[0085] Specifically, the weighted average fusion formula is:

[0086] Low_Fusion = λ 1 ×Low_CT + λ 2 ×Low_MRI

[0087] Among them, Low_Fusion is the low-frequency fusion image, and λ 1 is the weighting coefficient of the low-frequency CT component, and λ 2 is the weighting coefficient of the low-frequency MRI component. In the embodiment of the present invention, take λ 1 = λ 2 = 0.5.

[0088] High-frequency component adversarial fusion sub-module: used to perform adversarial generation fusion on the high-frequency MRI component and the high-frequency CT component to obtain a set of high-frequency fusion images;

[0089] Inverse variational mode decomposition sub-module: used to perform inverse variational mode decomposition on the low-frequency fusion image and each high-frequency fusion image to obtain a set of training medical images.

[0090] By means of the adversarial generation network, on the one hand, a large amount of effective medical data is obtained, solving the problem of less training data. On the other hand, the high-frequency and low-frequency components of MRI and CT images are decomposed by variational mode decomposition, and then the low-frequency components are fused by weighted average to obtain a low-frequency image. The high-frequency part is input into the adversarial generation network for processing, and a high-frequency image is output. Finally, the inverse variational mode decomposition is performed to obtain a fused image as the medical image for model training. This set of medical images is fused in the high-frequency and low-frequency domains respectively, highly restoring the spatial features and texture features of MRI and CT images, greatly improving the data quality during model training, and effectively improving the accuracy of registration and segmentation of the trained model.

[0091] Then, the feature extraction unit 10 executes step D21: extract features from a set of training medical images to obtain the feature data of this set of training medical images;

[0092] At the same time, the feature extraction unit 10 also executes step D22: extract features from a standard image to obtain the feature data of the standard image.

[0093] The selection of the standard image is based on the specific medical scenario, and a certain image with high quality, clear morphology and meeting the requirements of a specific scenario is manually selected as the standard image by a human. Such a standard image can ensure that the medical image to be registered can be accurately aligned with it during the subsequent registration process, providing a reliable reference standard. The image features of the medical image and the standard image are extracted through a convolutional neural network.

[0094] Then, the deformation field extraction unit 20 executes step D30: performing convolution processing on each training medical image feature in the group with the standard image feature respectively to obtain a group of training deformation fields; where each training deformation field is the deformation field between a training medical image in the group and the standard image.

[0095] While extracting image features through a convolutional neural network, a deformation field is generated through learning to describe the displacement deviation in space of each pixel or voxel of each medical image compared to the standard image.

[0096] Next, the image registration unit 30 executes D40: performing interpolation deformation on each training medical image according to its corresponding training deformation field to obtain a group of registered training medical images.

[0097] Performing interpolation deformation on the corresponding medical image according to the solved deformation field, so as to match with the standard image, enabling all medical images to be precisely aligned anatomically. Maintaining the anatomical consistency is beneficial for the segmentation model to learn the segmentation positions, improving the relevance of registration and segmentation, and thus improving the segmentation accuracy.

[0098] Finally, the segmentation model training unit executes step D50: inputting the group of registered training medical images into the segmentation model A, training the segmentation model A to obtain a trained segmentation model B.

[0099] The segmentation model is an algorithm model used to segment specific regions or structures in medical images (such as CT, MRI, etc.). It segments the input medical image to achieve the segmentation of the lesion area and extract the information of the lesion body. In the present invention, the Unet structure is used as the basic framework to initialize the segmentation model, and then it is trained. The specific steps are as follows:

[0100] The registered medical image data is first divided into a training set and a validation set according to 9:1. The nnunetv2 network generates data fingerprints and pipeline fingerprints based on the data (that is, different data preprocessing strategies are performed on different data). These fingerprints provide the best preprocessing strategy for a specific dataset; the preprocessed registered medical image data is input into a Cascade-Unet model for training. The Cascade-Unet model integrates the features of low-resolution images and full-resolution images. The principle of its image segmentation is that the cascaded Unet model will learn the features of the dataset during the training process, and corresponding weights will be obtained through training. Finally, the weights can be loaded for inference to view the quality of the segmentation effect.

[0101] During the training process, first, operations such as rotation and flipping are performed on the training set data for data augmentation to increase the original data volume. Then, a Cascade-Unet model is generated based on the original 3D-Unet architecture, and the augmented training set is put into the Cascade-Unet model for training to extract the required feature information. After training, inference is performed according to the validation data set to view the inference effect, and the segmentation effect is judged by comparing the accuracy. According to the segmentation effect, post-processing strategies are applied to the model. By discarding part of the largest connected component of the medical image, it can be checked whether the accuracy is improved after discarding. If there is an improvement, this post-processing strategy is adopted. Finally, the model is deployed. The trained pth file is converted into onnx and then into engine. Compared with the original inference method using python, tensorRT inference can accelerate the model inference time, and finally complete the model training. Implement adaptive registration-segmentation training for the segmentation model in advance, so that after the image to be segmented is registered, the trained segmentation model can more accurately extract key information during segmentation, and the segmentation effect is significantly improved.

[0102]

Medical Image Registration and Segmentation Stage

[0103] First, the feature extraction unit 20 executes step S10: extract features from the medical image to be segmented to obtain the features of the medical image to be segmented.

[0104] Then, the deformation field extraction unit 30 executes step S20: perform convolution processing on the features of the medical image to be segmented and the above-mentioned standard image features to obtain a deformation field to be segmented.

[0105] Next, the image registration unit 40 executes step S30: perform interpolation deformation on the medical image to be segmented according to the deformation field to obtain a registered medical image data.

[0106] Furthermore, in order to improve the accuracy between the medical image and the standard image, the present invention also includes an optimization process during the registration process. After obtaining a set of registered training medical images by the deformation field optimization unit in step D40, and / or, after obtaining a registered medical image to be segmented in step S30; optimize the deformation field. Please refer to Figure 5 , taking the deformation field optimization after obtaining a registered medical image to be segmented in step S30 as an exemplary illustration. Specifically, it includes:

[0107] Calculate the reconstruction loss between the set of registered medical images to be segmented and the standard image, and calculate the regularization loss between the registered medical image to be segmented and the standard image;

[0108] If the reconstruction loss is greater than a first threshold or the regularization loss is greater than a second threshold, backpropagation optimization is performed on the deformation field to be segmented according to the reconstruction loss and the regularization loss;

[0109] Interpolate and deform the medical image to be segmented according to the optimized deformation field to be segmented until the reconstruction loss is less than the first threshold and the regularization loss is less than the second threshold.

[0110] The reconstruction loss is used to evaluate the similarity between the registered moving image and the fixed image, and the regularization loss constrains the smoothness of the deformation field to ensure the rationality and coherence of the registration result in spatial transformation. By backpropagating to optimize the parameters of the neural network, a more accurate deformation field is generated. Finally, the model outputs the registered moving image and the corresponding deformation field so that all images are accurately aligned anatomically.

[0111] Finally, the trained segmentation model B executes step S40: perform segmentation processing on the registered medical image data to be segmented to obtain the segmented medical image.

[0112] A medical registration and segmentation method proposed by the present invention combines registration and segmentation, which can effectively improve the spatial alignment accuracy of three-dimensional medical images, thereby improving the accuracy of subsequent segmentation. Compared with a simple segmentation method, the present invention pre-trains the segmentation model for adaptive registration-segmentation, so that after the image to be segmented is registered, the trained segmentation model can more accurately extract key information during segmentation, and the segmentation effect is significantly improved. In addition, the medical data used for initializing registration-segmentation is obtained by fusing MRI and CT images. With the help of a generative adversarial network, on the one hand, a large amount of effective medical data is obtained, solving the problem of less training data. On the other hand, the high-frequency and low-frequency components of MRI and CT images are decomposed by variational mode decomposition, and then the low-frequency components are fused by weighted average to obtain a low-frequency image. The high-frequency part is input into the generative adversarial network for processing to output a high-frequency image. Finally, the fused image is obtained by inverse variational mode decomposition as the medical image for model training. This set of medical images is fused in the high-frequency and low-frequency domains respectively, highly restoring the spatial features and texture features of MRI and CT images, greatly improving the data quality during model training, and effectively improving the registration and segmentation accuracy of the trained model.

[0113] The embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the medical registration and segmentation method described in any one of the above embodiments.

[0114] The present invention may be implemented in the form of a computer program product on one or more storage media (including but not limited to disk memories, CD-ROMs, optical memories, etc.) containing program code. The computer-readable storage medium includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. The information may be computer-readable instructions, data structures, program modules or other data. Examples of the computer's storage medium include but are not limited to: phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette tapes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission medium that can be used to store information accessible by a computing device.

[0115] The above-described embodiments merely represent several implementation manners of the present invention. The description is relatively specific and detailed, but it should not be construed as a limitation on the scope of the invention patent. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present invention, several modifications and improvements can still be made, and the present invention also intends to include these changes and modifications.

Claims

1. A medical image registration and segmentation method, characterized in that: include: Segmentation model training stage and medical image registration and segmentation stage; During the segmentation model training phase, the following steps are performed: D10: perform data expansion on the medical images to be segmented to obtain a set of training medical images; D21: extracting features from the set of training medical images to obtain features of the set of training medical images; D22: extract features from a standard image to obtain standard image features; D30: performing convolution processing on each training medical image feature in the group and the standard image feature respectively to obtain a group of training deformation fields; wherein each training deformation field is a deformation field of a training medical image in the group and the standard image; D40: performing interpolation deformation on each training medical image according to its corresponding training deformation field to obtain a set of registered training medical images; D50: training a segmentation model according to the registered training medical image to obtain a trained segmentation model; The following steps are performed in the medical image registration and segmentation stage: S10: extracting features of the medical image to be segmented to obtain features of the medical image to be segmented; S20: performing convolution processing on the feature of the medical image to be segmented and the feature of the standard image to obtain a deformation field to be segmented; S30: performing interpolation deformation on the medical image to be segmented according to the deformation field to be segmented to obtain a registered medical image to be segmented; S40: Inputting the registered medical image to be segmented into the trained segmentation model for segmentation processing to obtain a segmented medical image.

2. The medical image registration and segmentation method according to claim 1, characterized in that: The medical images to be segmented include MRI images and CT images. The training medical images in step D10 are obtained by the following method: Performing variational modal decomposition on the MRI image and the CT image respectively to obtain high-frequency MRI components and low-frequency MRI components, as well as high-frequency CT components and low-frequency CT components; Performing weighted average fusion on the low-frequency MRI component and the low-frequency CT component to obtain a low-frequency fused image; Performing adversarial generation fusion on the high-frequency MRI component and the high-frequency CT component to obtain a set of high-frequency fused images; The low-frequency fused image and each high-frequency fused image are subjected to variational modal decomposition inverse transformation to obtain a set of training medical images.

3. The medical image registration and segmentation method according to claim 2, characterized in that: The weighted average fusion formula is: Low_Fusion=λ1×Low_CT+λ2×Low_MRI Among them, Low_Fusion is the low-frequency fusion image, λ1 is the weighting coefficient of the low-frequency CT component, and λ2 is the weighting coefficient of the low-frequency MRI component.

4. The medical image registration and segmentation method according to any one of claims 1 to 3, characterized in that: In step D40, after obtaining a set of registered training medical images, the following steps are further included: Calculating the reconstruction loss of the set of registered training medical images and the standard image, and calculating the regularization loss of the set of registered training medical images and the standard image; If the reconstruction loss is greater than a first threshold, or the regularization loss is greater than a second threshold, performing back-propagation optimization on the training deformation field according to the reconstruction loss and the regularization loss; Interpolating and deforming each training medical image according to the optimized training deformation field until the reconstruction loss is less than the first threshold and the regularization loss is less than the second threshold; and / or, In step S30, after obtaining a registered medical image to be segmented, the following steps are further included: Calculating the reconstruction loss of the group of registered medical images to be segmented and the standard image, and calculating the regularization loss of the registered medical images to be segmented and the standard image; If the reconstruction loss is greater than a first threshold, or the regularization loss is greater than a second threshold, performing back-propagation optimization on the deformation field to be segmented according to the reconstruction loss and the regularization loss; The medical image to be segmented is interpolated and deformed according to the optimized deformation field to be segmented until the reconstruction loss is less than the first threshold and the regularization loss is less than the second threshold.

5. The medical image registration and segmentation method according to claim 4, characterized in that: The segmentation model is a Cascade-Unet model using the Unet model as the basic network architecture.

6. A medical image registration and segmentation system, characterized in that: It includes an image expansion unit, a feature extraction unit, a deformation field extraction unit, an image registration unit, a segmentation model and a segmentation model training unit; firstly, the segmentation model training step is performed, including: The image expansion unit performs data expansion on the medical image to be segmented to obtain a set of training medical images; The feature extraction unit extracts features from the group of training medical images and a standard image respectively to obtain features of the group of training medical images and features of the standard image; The deformation field extraction unit performs convolution processing on each training medical image feature in the group and the standard image feature respectively to obtain a group of training deformation fields; wherein each training deformation field is a deformation field of a training medical image in the group and the standard image; The image registration unit performs interpolation deformation on each training medical image according to its corresponding training deformation field to obtain a set of registered training medical images; The segmentation model training unit inputs the set of registered training medical images into the segmentation model, trains the segmentation model, and obtains a trained segmentation model; Then the medical image registration and segmentation steps are performed, including: The feature extraction unit extracts features from the medical image to be segmented to obtain features of the medical image to be segmented; The deformation field extraction unit performs convolution processing on the features of the medical image to be segmented and the features of the standard image to obtain a deformation field to be segmented; The image registration unit performs interpolation deformation on the medical image to be segmented according to the deformation field to be segmented to obtain a registered medical image to be segmented; The registered medical image to be segmented is input into the trained segmentation model for segmentation processing to obtain a segmented medical image.

7. The medical image registration and segmentation system according to claim 6, characterized in that: The medical image to be segmented includes an MRI image and a CT image, and the image expansion unit is an image fusion model, which includes: Variational modal decomposition submodule: used to perform variational modal decomposition on MRI images and CT images respectively to obtain high-frequency MRI components and low-frequency MRI components, as well as high-frequency CT components and low-frequency CT components; Low-frequency component weighted fusion submodule: used for performing weighted average fusion of the low-frequency MRI component and the low-frequency CT component to obtain a low-frequency fused image; High-frequency component adversarial fusion submodule: used for adversarially generating and fusing the high-frequency MRI component and the high-frequency CT component to obtain a set of high-frequency fused images; Variational modal decomposition inverse transformation submodule: used to perform variational modal decomposition inverse transformation on the low-frequency fusion image and each high-frequency fusion image to obtain a set of training medical images.

8. The medical image registration and segmentation system according to claim 7, characterized in that: The weighted average fusion formula is: Low_Fusion=λ1×Low_CT+λ2×Low_MRI Among them, Low_Fusion is the low-frequency fusion image, λ1 is the weighting coefficient of the low-frequency CT component, and λ2 is the weighting coefficient of the low-frequency MRI component.

9. The medical image registration and segmentation system according to any one of claims 6 to 8, characterized in that: Also included is a deformation field optimization unit, which includes: A loss calculation submodule: used to calculate the reconstruction loss of the group of registered training medical images and the standard image, and to calculate the regularization loss of the group of registered training medical images and the standard image; A back propagation optimization submodule, used to determine: if the reconstruction loss is greater than a first threshold, or the regularization loss is greater than a second threshold, then back propagation optimization is performed on the training deformation field according to the reconstruction loss and the regularization loss; The image updating submodule is used to perform interpolation deformation on each training medical image according to the optimized training deformation field until the reconstruction loss is less than the first threshold and the regularization loss is less than the second threshold.

10. A computer device, characterized in that: include: at least one memory and at least one processor; The memory is used to store one or more programs; When the one or more programs are executed by the at least one processor, the at least one processor implements the steps of the medical image registration and segmentation method as described in any one of claims 1 to 5.

Images