Image generation method and device, electronic equipment and storage medium

Abstract

The invention is suitable for the technical field of image processing, and provides an image generation method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring a target CT image; performing shallow feature extraction on the target CT image by using a first generator in the trained generative model to obtain a shallow feature map; using a first generator to perform multiple serial deep feature extraction on the shallow feature map to obtain a deep feature map, performing multiple up-sampling processing on the deep feature map to obtain target MRI image features, and generating a target MRI image based on the target MRI image features; wherein the generation model comprises a first generator, and the first generator inputs a CT image to obtain an MRI image and a corresponding MRI image feature. According to the method, high-quality conversion from the CT image to the MRI image can be realized, the high-quality image which contains rich details and has semantic consistency is generated, and the overall quality of the generated MRI image is improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image generation method, device, electronic device and storage medium. Background Art

[0002] Image Guided Radiation Therapy (IGRT) is a cancer treatment method that helps improve the survival rate of many cancers. It uses medical imaging technology to improve the accuracy of radiotherapy, collect relevant image information before and during treatment, and guide radiotherapy based on image information.

[0003] With the development of technology, three-dimensional brachytherapy has made great progress. Compared with traditional two-dimensional afterloading therapy, three-dimensional afterloading therapy uses clear volume dose parameters to evaluate the radiation dose of the target area and organs at risk, which can improve the survival rate while minimizing the radiation damage of surrounding tissues and organs. Three-dimensional image-guided therapy based on computed tomography (CT) imaging technology may affect the determination of organs at risk and target area boundaries in the treatment plan due to the insufficient soft tissue resolution and radiation risk of CT images. Magnetic resonance imaging (MRI) technology has high soft tissue resolution and multiple imaging modes, providing rich diagnostic information, which can well solve the problems existing in image-guided radiotherapy based on CT technology. However, MRI scanning requires longer scanning time and higher scanning costs, and is not suitable for patients with epilepsy, neuroticism and claustrophobia. Due to the above limitations of MRI technology, in medical imaging applications, CT technology is applicable to a wider range of people than MRI technology.

[0004] Therefore, the researchers plan to use CT images to synthesize MRI images, so that they can take advantage of the fast imaging and wide universality of CT technology, and obtain richer diagnostic information through the generated MRI images. However, there is a relative lack of research on generating MRI images based on CT images. There are many difficulties in directly converting CT images into MRI images using linear models, such as the difficulty in generating image domains with rich details from image domains with less details, which leads to the poor quality of the final generated MRI images, which cannot accurately and clearly reflect the organ and tissue information in the CT images. Summary of the invention

[0005] In view of this, the embodiments of the present application provide an image generation method, device, electronic device and storage medium to solve the technical problem that when converting CT images into MRI images using a linear model, the final generated MRI image has poor quality and cannot accurately and clearly reflect the organ and tissue information in the CT image.

[0006] In a first aspect, an embodiment of the present application provides an image generation method, comprising:

[0007] Acquire a target CT image;

[0008] Using the first generator in the trained generative model to extract shallow features from the target CT image to obtain a shallow feature map;

[0009] Using the first generator, performing multiple serial deep feature extractions on the shallow feature map to obtain a deep feature map, performing multiple upsampling processes on the deep feature map to obtain target MRI image features, and generating a target MRI image based on the target MRI image features;

[0010] The generation model includes a first generator, which inputs a CT image to obtain an MRI image and corresponding MRI image features.

[0011] In a possible implementation of the first aspect, the step of extracting shallow features from the target CT image using the first generator in the trained generation model to obtain a shallow feature map includes:

[0012] Using the first generator to perform a preset number of convolution and feature extraction cross processing on the target CT image to obtain a shallow feature map;

[0013] Wherein, the feature extraction is performed based on the residual convolution block.

[0014] In a possible implementation of the first aspect, before using the first generator in the trained generation model to extract shallow features from the target CT image, the method further includes:

[0015] The acquired target CT image is subjected to denoising processing, and the processed target CT image is used as the target CT image.

[0016] In a possible implementation of the first aspect, before using the first generator in the trained generation model to extract shallow features from the target CT image, the method further includes:

[0017] Obtain different original CT images and corresponding original MRI images;

[0018] Based on the original CT image, the corresponding original MRI image is registered, and the registered original MRI image is used as the original MRI image, and the original CT image and the corresponding original MRI image constitute a training set;

[0019] The preset generation model is trained based on the training set to obtain a trained generation model.

[0020] In a possible implementation of the first aspect, the generation model further includes a second generator, a first discriminator, and a second discriminator;

[0021] The step of training a preset generation model based on the training set to obtain a trained generation model includes:

[0022] Selecting a set of original CT images and corresponding original MRI images from the training set;

[0023] Inputting the original CT image into the first generator to obtain a first pseudo MRI image and a corresponding first pseudo MRI image feature;

[0024] The first pseudo MRI image is input into the second generator to obtain a first pseudo CT image and a corresponding first pseudo CT image feature; the second generator inputs the MRI image and outputs a CT image and a corresponding CT image feature;

[0025] Inputting the first pseudo MRI image and the corresponding original MRI image into the first discriminator, using the first discriminator to perform discriminative processing on the first pseudo MRI image and output a first discriminative probability, and using the first discriminator to perform discriminative processing on the original MRI image and output a second discriminative probability;

[0026] Inputting the first pseudo CT image and the corresponding original CT image into the second discriminator, using the second discriminator to perform discriminative processing on the first pseudo CT image and output a third discriminative probability, and using the second discriminator to perform discriminative processing on the original CT image and output a fourth discriminative probability;

[0027] The first generator and the second generator are adjusted according to the features corresponding to the original CT image, the first pseudo CT image features and the cycle consistency loss function, the first generator and the first discriminant are adjusted according to the first discriminant probability, the second discriminant probability and the first generative adversarial loss function, and the first generator, the second generator and the second discriminator are adjusted according to the third discriminant probability, the fourth discriminant probability and the second generative adversarial loss function until the training is completed.

[0028] In a possible implementation of the first aspect, inputting the first pseudo MRI image and the corresponding original MRI image into the first discriminator, using the first discriminator to perform discriminant processing on the first pseudo MRI image and outputting a first discriminant probability, and using the first discriminator to perform discriminant processing on the original MRI image and outputting a second discriminant probability includes:

[0029] Using the first discriminator, convolution processing is performed on the first pseudo MRI image to obtain a first primary feature map, and the importance of each channel in the first primary feature map relative to each channel other than the channel is learned based on the channel attention mechanism to obtain a first weighted feature map with weights, the first primary feature map and the first weighted feature map are multiplied, and upsampling and feature concatenation processing are performed on the obtained first secondary feature map to obtain a first output feature map, and a first discriminant probability is obtained based on the first output feature map;

[0030] Using the first discriminator, convolution processing is performed on the original MRI image to obtain a second primary feature map, and the importance of each channel in the second primary feature map relative to each channel other than the channel is learned based on the channel attention mechanism to obtain a second weighted feature map with weights, the second primary feature map and the second weighted feature map are multiplied, and the obtained second secondary feature map is upsampled and feature concatenated to obtain a second output feature map, and a second discriminant probability is obtained based on the second output feature map;

[0031] Among them, the discrimination probability indicates the probability value of whether the current image is true or false.

[0032] In a possible implementation of the first aspect, the first generator and the second generator are adjusted according to the features corresponding to the original CT image, the first pseudo CT image features, and the cycle consistency loss function, the first generator and the first discriminant are adjusted according to the first discriminant probability, the second discriminant probability, and the first generation adversarial loss function, and the first generator, the second generator, and the second discriminator are adjusted according to the third discriminant probability, the fourth discriminant probability, and the second generation adversarial loss function until the training is completed, including:

[0033] Calculating a first loss value according to the features corresponding to the original CT image, the features of the first pseudo CT image, and a cycle consistency loss function, and determining whether the first loss value is less than a first preset threshold;

[0034] Calculate a second loss value according to the first discrimination probability, the second discrimination probability and the first generative adversarial loss function, and determine whether the second loss value is less than a second preset threshold;

[0035] Calculate a third loss value according to the third discriminant probability, the fourth discriminant probability and the second generative adversarial loss function, and determine whether the third loss value is less than a third preset threshold;

[0036] If the first loss value is less than the first preset threshold, the second loss value is less than the second preset threshold, and the third loss value is less than the third preset threshold, the training is completed and a trained generation model is obtained; otherwise, the first generator and the second generator are adjusted according to the first loss value, the first generator and the first discriminator are adjusted according to the second loss value, and the first generator, the second generator and the second discriminator are adjusted according to the third loss value to obtain a generation model after parameter adjustment; and the next group of original CT images and the corresponding original MRI images are selected from the training set, and the generation model after parameter adjustment is trained until the training is completed to obtain a trained generation model.

[0037] In a second aspect, an embodiment of the present application provides an image generating device, including:

[0038] An acquisition module, used for acquiring a target CT image;

[0039] A processing module, configured to extract shallow features from the target CT image using the first generator in the trained generation model to obtain a shallow feature map;

[0040] A generating module, configured to use the first generator to perform multiple serial deep feature extractions on the shallow feature map to obtain a deep feature map, perform multiple upsampling processes on the deep feature map to obtain target MRI image features, and generate a target MRI image based on the target MRI image features;

[0041] The generation model includes a first generator, which inputs a CT image to obtain an MRI image and corresponding MRI image features.

[0042] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the computer program, the image generation method as described in any one of the first aspects is implemented.

[0043] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the image generation method as described in any one of the first aspects is implemented.

[0044] In a fifth aspect, an embodiment of the present application provides a computer program product. When the computer program product is run on an electronic device, the electronic device executes the image generation method described in any one of the first aspects above.

[0045] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here.

[0046] The image generation method, device, electronic device and storage medium provided in the embodiments of the present application use the first generator in the trained generation model to perform shallow feature extraction on the acquired target CT image to capture features of different levels of the image in multiple stages to obtain a shallow feature map. Thereafter, the first generator is used to perform multiple serial deep feature extractions on the shallow feature map to better capture details and contextual information in the image to obtain a deep feature map. The deep feature map is upsampled multiple times to obtain target MRI image features, and a target MRI image is generated based on the target MRI image features. In this way, high-quality conversion from CT images to MRI images can be achieved, and high-quality images that are rich in details and have semantic consistency can be generated, thereby improving the overall quality of the generated MRI images.

[0047] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present specification. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0049] Figure 1 is a flowchart of an image generation method provided by an embodiment of the present application;

[0050] Figure 2 It is a structural diagram of a generation model provided in an embodiment of the present application;

[0051] Figure 3 is a structural diagram of a first generator provided in an embodiment of the present application;

[0052] Figure 4 is a schematic diagram of the structure of a first discriminator provided in an embodiment of the present application;

[0053] Figure 5 is a schematic diagram of the structure of a generation model provided by another embodiment of the present application;

[0054] Figure 6 It is a schematic diagram of visualization results of a generation model and a comparison model provided in an embodiment of the present application;

[0055] Figure 7 is a structural schematic diagram of an image generating device provided by an embodiment of the present application;

[0056] Figure 8 It is a schematic diagram of the structure of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0057] The present application is described more clearly below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the effects of the present application, but are not intended to limit the present application in any form. It should be noted that, for those of ordinary skill in the art, several variations and improvements may be made without departing from the concept of the present application. These all fall within the scope of protection of the present application.

[0058] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0059] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0060] In the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0061] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0062] In addition, the “plurality” mentioned in the embodiments of the present application should be interpreted as two or more.

[0063] In order to solve the problems existing in the related technology, the inventors have found through research that the first generator in the trained generative model can be used to perform shallow feature extraction on the target CT image, as well as multiple serial deep feature extractions, so as to capture features of different levels of the image at multiple stages, better capture the details and contextual information in the image, and improve the overall quality of the generated MRI image.

[0064] Figure 1 FIG. 1 is a flow chart of an image generation method provided by an embodiment of the present application. Figure 1 As shown, the method in the embodiment of the present application may include:

[0065] Step 101: Acquire a target CT image.

[0066] Optionally, the CT images in this embodiment may be CT images of the brain, spine, cardiovascular system, liver and gallbladder, prostate and other parts, as well as pelvic CT images including the bladder and rectum.

[0067] Step 102: Use the first generator in the trained generation model to extract shallow features of the target CT image to obtain a shallow feature map.

[0068] In this embodiment, the generative model may be a generative adversarial network (GAN). The generative model includes a first generator, which inputs a CT image to obtain an MRI image and corresponding MRI image features.

[0069] In a possible implementation, when this embodiment uses the first generator in the trained generation model to perform shallow feature extraction on the target CT image to obtain a shallow feature map, the first generator can be used to perform a preset number of convolutions and cross-processing of feature extraction on the target CT image to obtain a shallow feature map, wherein the feature extraction is performed based on the residual convolution block.

[0070] Reference Figure 2 , the generative model includes a first generator. Figure 3 , the first generator may include multiple convolutional layers and multiple residual convolutional blocks, and the convolutional layers and residual convolutional blocks are cross-set. For example, the first generator may sequentially set a convolutional layer of the same size as the input image (here, the target CT image) and three residual convolutional blocks, and then sequentially set two structures including "a downsampling convolutional layer and three residual convolutional blocks", wherein the convolution kernels of the downsampling convolutional layer in each structure are different, and the convolution kernels of the residual convolutional blocks are different.

[0071] In this way, the resolution of the input image can be maintained by using the same-size convolution layer, while starting to capture the basic features in the input image. The residual convolution block can learn to extract more complex features. After that, the downsampling convolution layer can be used to reduce the spatial dimension of the input image. In a smaller spatial dimension, the next residual convolution block is used to extract deeper features, and finally a shallow feature map is obtained.

[0072] In this embodiment, the convolution layer and the residual convolution block are cross-set, that is, the target CT image is convolved and the feature extraction based on the residual convolution block is cross-processed, which can capture and extract the features of different levels of the target CT image at multiple stages, which helps to improve the overall quality of the generated image.

[0073] In some embodiments, before performing shallow feature extraction on the target CT image using the first generator in the trained generation model, the acquired target CT image may also be denoised, and the processed target CT image may be used as the target CT image.

[0074] Optionally, in this embodiment, a method of manually setting a threshold value may be used for denoising, for example, pixels in the target CT image whose pixel values ​​exceed a preset threshold range are removed as noise.

[0075] Since CT images such as pelvic CT images have complex and diverse anatomical structures, in this embodiment, the obtained shallow feature map can be further subjected to deep feature extraction.

[0076] Step 103: Using the first generator, perform multiple serial deep feature extractions on the shallow feature map to obtain a deep feature map, perform multiple upsampling processes on the deep feature map to obtain target MRI image features, and generate a target MRI image based on the target MRI image features.

[0077] Reference Figure 3 The first generator may also include multiple serial feature processing modules, and use multiple serial feature processing modules to perform multiple serial deep feature extractions on the obtained shallow feature map. Each feature processing module performs a series of feature extraction operations such as feature normalization processing, dynamic unit processing, feature compression processing, and dimension transformation processing. Among them, feature normalization processing normalizes the input features by adjusting the mean and variance of each feature channel of the shallow feature map, which can stabilize the training process in the subsequent training process, promote gradient flow, improve the generalization ability of the model, and accelerate the convergence speed.

[0078] Dynamic unit processing adaptively selects the unit size according to the characteristics of the shallow feature map after normalization, calculates the feature relationship between the elements inside the unit, and then moves the unit to every possible position from the starting point of the shallow feature map after normalization to calculate the features of different units. Each time it moves, the range of the shallow feature map after normalization covered by the unit will change. After that, the feature vectors of all units enter the fully connected layer to capture the complex relationships in the shallow feature map after normalization, integrate local features into global features, and form a complete feature representation, thereby improving the feature processing module's ability to understand complex images. Finally, normalization is performed to maintain the consistency of the feature processing module when processing different layers.

[0079] Feature compression processing reduces the number of channels of the feature map obtained by dynamic unit processing to reduce the parameter parameters and computational complexity of the feature processing module, while extracting important features and removing redundant information. Dimension transformation processing captures more local features of the feature map obtained by feature compression processing through convolution operations, such as edges and textures, and then restores the dimension of the obtained feature map to the dimension of the shallow feature map before feature normalization processing to ensure the integrity of the information.

[0080] Exemplarily, after the first feature processing module is used to perform feature normalization processing, dynamic unit processing, feature compression processing and dimensional transformation processing on the shallow feature map, the next serial feature processing module is used to perform feature normalization processing, dynamic unit processing, feature compression processing and dimensional transformation processing on the first feature map to obtain the second feature map, and so on, until the last serial feature processing module performs feature normalization processing, dynamic unit processing, feature compression processing and dimensional transformation processing on the obtained feature map to obtain a deep feature map.

[0081] Optionally, the first generator may also include multiple upsampling convolution layers, which can be used to gradually restore the spatial dimensions of the deep feature map to the spatial dimensions of the target CT image to obtain the target MRI image features, and then generate the target MRI image based on the target MRI image features.

[0082] In this embodiment, multiple serial deep feature extractions are performed on the obtained shallow feature maps, which can better capture the details and contextual information in the image and generate high-quality images that contain rich details and have semantic consistency.

[0083] The image generation method provided in the embodiment of the present application uses the first generator in the trained generation model to perform shallow feature extraction on the acquired target CT image to capture features of different levels of the image in multiple stages to obtain a shallow feature map. Thereafter, the first generator is used to perform multiple serial deep feature extractions on the shallow feature map to better capture details and contextual information in the image to obtain a deep feature map. The deep feature map is upsampled multiple times to obtain target MRI image features, and a target MRI image is generated based on the target MRI image features. In this way, high-quality conversion from CT images to MRI images can be achieved, and high-quality images that are both rich in details and semantically consistent can be generated, thereby improving the overall quality of the generated MRI images.

[0084] In order to enable the generative model to generate high-quality target MRI images, the generative model can also be trained.

[0085] In a possible implementation, this embodiment may further include steps S1 to S3 before using the first generator in the trained generation model to perform shallow feature extraction on the target CT image.

[0086] S1. Obtain different original CT images and corresponding original MRI images.

[0087] S2. Based on the original CT image, the corresponding original MRI image is registered, and the registered original MRI image is used as the original MRI image. The original CT image and the corresponding original MRI image constitute a training set.

[0088] S3. Train the preset generation model based on the training set to obtain a trained generation model.

[0089] In this embodiment, the MRI image corresponding to the CT image may be the MRI image of the brain, spine, cardiovascular, liver and gallbladder, prostate and other parts, as well as the pelvic MRI image including the bladder and rectum. In this embodiment, the original MRI image is first registered using affine transformation so that the original MRI image is aligned with the corresponding original CT image in spatial position and has the same size. Afterwards, the original CT image and the original MRI image are denoised using a method of manually setting a threshold, for example, the pixel points in the original CT image and the original MRI image whose pixel values ​​exceed the preset threshold range are removed as noise. The denoised original CT image and the denoised original MRI image are used as the original CT image and the original MRI image to form a training set, and the training set is used to train the preset generation model.

[0090] In some embodiments, reference Figure 2 , the generation module also includes a second generator, a first discriminator and a second discriminator. Figure 2In G MRI represents the first generator, G CT represents the second generator, D MRI represents the first discriminator, D CT Denotes a second discriminator. When the preset generation model is trained based on the training set to obtain the trained generation model, steps A1 to A6 may also be included.

[0091] A1. Select a set of original CT images and corresponding original MRI images from the training set.

[0092] A2. Input the original CT image into the first generator to obtain a first pseudo MRI image and a corresponding first pseudo MRI image feature.

[0093] A3. Input the first pseudo MRI image into the second generator to obtain a first pseudo CT image and corresponding first pseudo CT image features; the second generator inputs the MRI image and outputs the CT image and corresponding CT image features.

[0094] A4. Input the first pseudo MRI image and the corresponding original MRI image into the first discriminator, use the first discriminator to perform discriminative processing on the first pseudo MRI image, output a first discriminative probability, use the first discriminator to perform discriminative processing on the original MRI image, and output a second discriminative probability.

[0095] A5. Input the first pseudo CT image and the corresponding original CT image into the second discriminator, use the second discriminator to perform discriminative processing on the first pseudo CT image, output a third discriminative probability, use the second discriminator to perform discriminative processing on the original CT image, and output a fourth discriminative probability.

[0096] A6. Adjust the parameters of the first generator and the second generator according to the features corresponding to the original CT image, the first pseudo CT image features and the cycle consistency loss function, adjust the parameters of the first generator and the first discriminant according to the first discriminant probability, the second discriminant probability and the first generative adversarial loss function, and adjust the parameters of the first generator, the second generator and the second discriminator according to the third discriminant probability, the fourth discriminant probability and the second generative adversarial loss function until the training is completed.

[0097] In this embodiment, the second generator inputs an MRI image and outputs a CT image and corresponding CT image features. The structure of the second generator can refer to the structure of the first generator in the aforementioned embodiment, that is, the second generator also includes multiple convolutional layers and multiple residual convolutional blocks, and the convolutional layers and residual convolutional blocks are cross-set. The cross-set convolutional layers and residual convolutional blocks can be used to capture and extract features of different levels of the MRI image in multiple stages to obtain corresponding shallow feature maps. And the second generator can also include multiple serial feature processing modules, and the multiple serial feature processing modules are used to perform multiple serial deep feature extractions on the obtained shallow feature maps, which can better capture the details and contextual information in the image and obtain the corresponding deep feature maps. And the second generator can also include multiple upsampling convolutional layers, and the upsampling convolutional layers can be used to gradually restore the spatial dimensions of the deep feature maps to the spatial dimensions of the MRI images to obtain CT image features.

[0098] It should be noted that the convolution layers, residual convolution blocks, feature processing modules and upsampling convolution layers mentioned in different steps and modules in this embodiment only represent the convolution layers, residual convolution blocks, feature processing modules and upsampling convolution layers in the current steps and modules, and should be distinguished from the convolution layers, residual convolution blocks, feature processing modules and upsampling convolution layers in other steps and modules.

[0099] Similarly, the shallow feature maps, deep feature maps, etc. obtained in different steps and modules in this embodiment only represent the shallow feature maps, deep feature maps, etc. in the current step and module, and should be distinguished from the shallow feature maps, deep feature maps, etc. in other steps and modules.

[0100] The process of processing the MRI image using the second generator can refer to the process of processing the CT image using the first generator in the aforementioned embodiment, and will not be repeated here.

[0101] In some embodiments, when using the first discriminator to perform discriminant processing on the first pseudo MRI image and outputting a first discriminant probability, and using the first discriminant to perform discriminant processing on the original MRI image and outputting a second discriminant probability, the first discriminator can be used to perform convolution processing on the first pseudo MRI image to obtain a first primary feature map, and based on the channel attention mechanism, the importance of each channel in the first primary feature map relative to each channel except the channel is learned to obtain a first weighted feature map with weights, the first primary feature map and the first weighted feature map are multiplied, and the obtained first secondary feature map is upsampled and feature concatenated to obtain a first output feature map, and based on the first output feature map, a first discriminant probability is obtained.

[0102] And, using the first discriminator, convolution processing is performed on the original MRI image to obtain a second primary feature map, and the importance of each channel in the second primary feature map relative to each channel other than the channel is learned based on the channel attention mechanism to obtain a second weighted feature map with weights, the second primary feature map and the second weighted feature map are multiplied, and the obtained second secondary feature map is upsampled and feature concatenated to obtain a second output feature map, and based on the second output feature map, a second discrimination probability is obtained. Wherein, the discrimination probability indicates the probability value of whether the current image is true or false.

[0103] In this embodiment, refer to Figure 4 , the first discriminator may include a convolution block, an efficient channel attention block and a maximum pooling layer. Among them, the convolution block may include a convolution layer, a batch normalization layer and an activation layer (not shown), and the activation layer may apply a ReLU activation function. After the first pseudo MRI image is subjected to convolution processing, batch normalization processing and activation processing using the convolution block, a first primary feature map is obtained. The efficient channel attention block is used to calculate the one-dimensional convolution kernel size based on the number of channels in the first primary feature map, and the importance of each channel relative to each channel other than the channel is learned based on the one-dimensional convolution to obtain a first weighted feature map with weights, so as to achieve adaptive re-weighting of channel features to enhance feature expression. The first primary feature map and the first weighted feature map are multiplied to obtain a first secondary feature map, and then the maximum pooling layer is used to reduce the size of the first secondary feature map to half of the original size.

[0104] The first discriminator may also include multiple "convolution blocks + efficient channel attention blocks + maximum pooling layers" structures in sequence, for example, a total of three "convolution blocks + efficient channel attention blocks + maximum pooling layers" structures may be included to further enhance feature expression and focus on key features in the image. And after the three "convolution blocks + efficient channel attention blocks + maximum pooling layers" structures, a convolution block is also set to perform convolution processing, batch normalization processing and activation processing on the obtained first-level feature map after pooling, and obtain the first-level feature map after convolution processing.

[0105] Optionally, the first discriminator may also include multiple "transposed convolution layer + convolution block" structures and a convolution layer in sequence, and the number of "transposed convolution layer + convolution block" structures is the same as the number of "convolution block + efficient channel attention block + maximum pooling layer" structures and corresponds one to one. Here, the "transposed convolution layer and convolution block" structure transposes and upsamples the first secondary feature map after convolution processing to obtain the first secondary feature map after upsampling processing. After that, the first secondary feature map after upsampling processing is spliced ​​with the first secondary feature map output by the corresponding "convolution block + efficient channel attention block + maximum pooling layer" structure to finally obtain the first output feature map, and the first output feature map is convolved using a convolution layer to obtain a first discrimination probability pixel by pixel.

[0106] In this embodiment, the process and principle of using the first discriminator to process the original MRI image to obtain the second discriminant probability pixel by pixel can refer to the above-mentioned description of using the first discriminator to process the first pseudo MRI image to obtain the first discriminant probability pixel by pixel, and will not be repeated here.

[0107] In this embodiment, the structure of the second discriminator can refer to the structure of the first discriminator in the aforementioned embodiment. The process and principle of using the second discriminator to discriminate the first pseudo CT image and the corresponding original CT image, and outputting the third discriminant probability and the fourth discriminant probability respectively, can refer to the relevant description of the first discriminator processing the first pseudo MRI image and the corresponding original MRI image in the aforementioned embodiment, which will not be repeated here.

[0108] Thus, after selecting a group of original CT images and corresponding original MRI images from the training set, the original CT images are input into the first generator to obtain the first pseudo MRI image and the corresponding first pseudo MRI image features, and the first pseudo MRI images are input into the second generator to obtain the first pseudo CT image and the corresponding first pseudo CT image features. Also, the first pseudo MRI image and the corresponding original MRI image are input into the first discriminator to obtain the first discriminant probability and the second discriminant probability, respectively, and the first pseudo CT image and the corresponding original CT image are input into the second discriminator to obtain the third discriminant probability and the fourth discriminant probability.

[0109] In a possible implementation, this embodiment may include steps B1 to B4 when adjusting the parameters of the first generator, the second generator, the first discriminator, and the second discriminator.

[0110] B1. Calculate a first loss value according to features corresponding to the original CT image, features of the first pseudo CT image, and a cycle consistency loss function, and determine whether the first loss value is less than a first preset threshold.

[0111] B2. Calculate a second loss value based on the first discrimination probability, the second discrimination probability and the first generative adversarial loss function, and determine whether the second loss value is less than a second preset threshold.

[0112] B3. Calculate a third loss value based on the third discrimination probability, the fourth discrimination probability and the second generative adversarial loss function, and determine whether the third loss value is less than a third preset threshold.

[0113] B4. If the first loss value is less than the first preset threshold, the second loss value is less than the second preset threshold, and the third loss value is less than the third preset threshold, the training is completed and a trained generative model is obtained; otherwise, the first generator and the second generator are adjusted according to the first loss value, the first generator and the first discriminator are adjusted according to the second loss value, and the first generator, the second generator and the second discriminator are adjusted according to the third loss value to obtain a generative model after parameter adjustment; and the next group of original CT images and the corresponding original MRI images are selected from the training set, and the generative model after parameter adjustment is trained until the training is completed to obtain a trained generative model.

[0114] In this embodiment, the SSIM loss function can be added on the basis of the cycle consistency loss function such as the L2 loss function to jointly constrain the cycle consistency loss to enhance the performance of the model. By adding the SSIM loss to the cycle consistency loss, the model can focus on the details in the image while controlling the pixel-level accuracy of the generated image, thereby generating a more realistic image with richer details.

[0115] Cycle consistency loss function L cycle The expression is:

[0116] L cycle =L2+λ·(1-SSIM(x,y))

[0117] in,

[0118] Where x is the feature corresponding to the original CT image, y is the feature of the first pseudo CT image, and μ x is the mean value of x, μ y is the mean value of y, δ x is the variance of x, δ y is the variance of y, δ xy is the covariance of x and y, c1 and c2 are constants used to maintain stability, and λ is a hyperparameter used to balance the weight of SSIM loss.

[0119] Optionally, in step B2, the loss value of the first pseudo MRI image is calculated according to the first pixel-by-pixel discriminant probability, the pixel-by-pixel true label corresponding to the first pseudo MRI image, and the first generative adversarial loss function. The loss value of the original MRI image is calculated according to the second pixel-by-pixel discriminant probability, the pixel-by-pixel true label corresponding to the original MRI image, and the first generative adversarial loss function. And based on the first pseudo MRI image loss value and the original MRI image loss value, the second loss value is obtained. Here, since the first pseudo MRI image is a generated pseudo image, the value of the pixel-by-pixel true label corresponding to the first pseudo MRI image is 0. Since the original MRI image is a real image, the value of the pixel-by-pixel true label corresponding to the original MRI image is 1.

[0120] Similarly, in step B3, the loss value of the first pseudo CT image is calculated according to the third pixel-by-pixel discriminant probability, the pixel-by-pixel true label corresponding to the first pseudo CT image, and the second generative adversarial loss function. The loss value of the original CT image is calculated according to the fourth pixel-by-pixel discriminant probability, the pixel-by-pixel true label corresponding to the original CT image, and the second generative adversarial loss function. And based on the first pseudo CT image loss value and the original CT image loss value, the third loss value is obtained. Here, since the first pseudo CT image is a generated pseudo image, the value of the pixel-by-pixel true label corresponding to the first pseudo CT image is 0. Since the original CT image is a real image, the value of the pixel-by-pixel true label corresponding to the original CT image is 1.

[0121] Exemplarily, in this embodiment, if the first loss value is less than the first preset threshold, the second loss value is less than the second preset threshold, and the third loss value is less than the third preset threshold, it indicates that the performance of the generative model is good at this time, and the training is completed to obtain a trained generative model. Otherwise, it means that the generative model needs to be further trained, and the first generator and the second generator are adjusted according to the first loss value, and the first generator and the first discriminator are adjusted according to the second loss value. The first generator, the second generator, and the second discriminator are adjusted according to the third loss value to obtain a generative model after parameter adjustment. After that, the next group of original CT images and the corresponding original MRI images are selected from the training set, and the above steps A1 to A6 are repeated, that is, the generative model after parameter adjustment is trained until the training is completed to obtain a trained generative model.

[0122] Optionally, the condition for completing the training is that the first loss value is less than a first preset threshold, the second loss value is less than a second preset threshold, and the third loss value is less than a third preset threshold, or the number of training iterations reaches a preset iteration threshold.

[0123] In this embodiment, the generative model includes, in addition to the first generator, a second generator, a first discriminator, and a second discriminator. By training the generative model and adjusting the parameters of the first generator, the second generator, the first discriminator, and the second discriminator, the first generator and the second generator can output a pseudo image closer to a real image, and the first discriminator and the second discriminator can more accurately determine whether the input image is a real image or a pseudo image. This improves the performance of the generative model.

[0124] In the above embodiments, the process of forward training the generative model is described. In practical applications, in order to improve the performance of the generative model, the generative model can be reversely trained while forward training the generative model. The process of reverse training the generative model will be described in the following embodiments.

[0125] In one possible implementation, refer to Figure 5 , the reverse training of the generative model can include:

[0126] Based on a set of original CT images and corresponding original MRI images selected in step A1,

[0127] The original MRI image is input into the second generator to obtain a second pseudo CT image and corresponding second pseudo CT image features.

[0128] The second pseudo CT image is input into the first generator to obtain a second pseudo MRI image and corresponding second pseudo MRI image features.

[0129] The second pseudo CT image and the corresponding original CT image are input into the second discriminator, the second pseudo CT image is discriminated by the second discriminator, and a fifth discriminant probability is output, and the original CT image is discriminated by the second discriminator, and a sixth discriminant probability is output.

[0130] The second MRI image and the corresponding original MRI image are input into the first discriminator, the second MRI image is discriminated by the first discriminator, and the seventh discriminant probability is output; the original MRI image is discriminated by the first discriminator, and the eighth discriminant probability is output.

[0131] Reference Figure 2 and Figure 5 , with G MRI represents the first generator, G CT represents the second generator, D MRI represents the first discriminator, D CT represents the second discriminator.

[0132] The principles and process of reverse training of the generative model in this embodiment can refer to the relevant description of the forward training of the generative model in the aforementioned embodiment, as well as the processing process and principles of the first generator, the second generator, the first discriminator and the second discriminator on the corresponding images can refer to the relevant description in the aforementioned embodiment, which will not be repeated here.

[0133] Exemplarily, a fourth loss value is calculated based on features corresponding to the original MRI image, features of the second pseudo CT image, and a cycle consistency loss function, and it is determined whether the fourth loss value is less than a fourth preset threshold.

[0134] According to the fifth discriminant probability, the sixth discriminant probability and the third generative adversarial loss function, a fifth loss value is calculated, and it is determined whether the fifth loss value is less than a fifth preset threshold.

[0135] According to the seventh discriminant probability, the eighth discriminant probability and the fourth generative adversarial loss function, a sixth loss value is calculated, and it is determined whether the sixth loss value is less than a sixth preset threshold.

[0136] In this way, if the first loss value is less than the first preset threshold, the second loss value is less than the second preset threshold, the third loss value is less than the third preset threshold, the fourth loss value is less than the fourth preset threshold, the fifth loss value is less than the fifth preset threshold, and the sixth loss value is less than the sixth preset threshold, the training is completed and a trained generation model is obtained.

[0137] Otherwise, the first generator and the second generator are adjusted according to the first loss value and the fourth loss value, the first generator and the first discriminator are adjusted according to the second loss value, the first generator, the second generator and the second discriminator are adjusted according to the third loss value and the sixth loss value, and the second generator and the second discriminator are adjusted according to the fifth loss value to obtain the generative model after the parameter adjustment; and the next group of original CT images and the corresponding original MRI images are selected from the training set, and the generative model after the parameter adjustment is trained until the training is completed to obtain the trained generative model.

[0138] Here, the condition for the completion of training is that the loss values ​​are all less than the corresponding preset thresholds, or the number of training iterations reaches the preset iteration threshold.

[0139] In this embodiment, forward training and reverse training are performed on the generation model at the same time, which can improve the performance of the generation model.

[0140] For example, when training the generative model in actual applications, 2000 sets of original pelvic CT images and corresponding original MRI images are used as data sets, of which 1500 sets of images are selected as training sets for training the generative model, and the remaining 500 sets of images are used as test sets for testing the generative model. During the training process, 250 iterations were set. In the first 125 iterations, the model was trained with an initial learning rate of 0.0002, and in the next 125 iterations, the learning rate was set to decay linearly to zero. During the entire training process, the Adam optimizer was used to optimize the parameters of the generator and discriminator networks, and the batchsize was set to 4.

[0141] Afterwards, in order to verify the effectiveness of the trained generative model, Pix2pix, DualGAN and CycleGAN networks were used as comparison models, and 500 sets of images in the test set were used to test the trained generative model and the comparison models. At the same time, in order to ensure the effectiveness and comparability of the comparison results, the comparison model used the same basic hyperparameters as the trained generative model to control the variables in the comparison experiment.

[0142] The visualization results of the generated model and the comparison model in this application are as follows Figure 6 As shown, it can be seen that the MRI images generated by the generation model in this application are superior to the MRI images generated by other comparison models in terms of texture, content, and edges. This proves that the image generation method of this application can clearly map the organ and tissue information in the CT image to the generated MRI image, thereby improving the overall quality of the generated MRI image.

[0143] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0144] Figure 7 is a schematic diagram of the structure of an image generating device provided by an embodiment of the present application. Figure 7 As shown, the image generating device provided in this embodiment may include: an acquisition module 701 , a processing module 702 and a generating module 703 .

[0145] Wherein, the acquisition module 701 is used to acquire a target CT image;

[0146] The processing module 702 is used to use the first generator in the trained generation model to perform shallow feature extraction on the target CT image to obtain a shallow feature map.

[0147] The generation module 703 is used to use the first generator to perform multiple serial deep feature extractions on the shallow feature map to obtain a deep feature map, perform multiple upsampling processes on the deep feature map to obtain target MRI image features, and generate a target MRI image based on the target MRI image features.

[0148] The generation model includes a first generator, which inputs a CT image to obtain an MRI image and corresponding MRI image features.

[0149] Optionally, the processing module 702 is specifically configured to:

[0150] Using the first generator to perform a preset number of convolution and feature extraction cross processing on the target CT image to obtain a shallow feature map;

[0151] Wherein, the feature extraction is performed based on the residual convolution block.

[0152] Optionally, the processing module 702 is further configured to:

[0153] The acquired target CT image is subjected to denoising processing, and the processed target CT image is used as the target CT image.

[0154] Optionally, the image generating device may further include a training module, the training module being used for:

[0155] Obtain different original CT images and corresponding original MRI images;

[0156] Based on the original CT image, the corresponding original MRI image is registered, and the registered original MRI image is used as the original MRI image, and the original CT image and the corresponding original MRI image constitute a training set;

[0157] The preset generation model is trained based on the training set to obtain a trained generation model.

[0158] Optionally, the generation model further includes a second generator, a first discriminator, and a second discriminator; and the training module is further used for:

[0159] Selecting a set of original CT images and corresponding original MRI images from the training set;

[0160] Inputting the original CT image into the first generator to obtain a first pseudo MRI image and a corresponding first pseudo MRI image feature;

[0161] The first pseudo MRI image is input into the second generator to obtain a first pseudo CT image and a corresponding first pseudo CT image feature; the second generator inputs the MRI image and outputs a CT image and a corresponding CT image feature;

[0162] Inputting the first pseudo MRI image and the corresponding original MRI image into the first discriminator, using the first discriminator to perform discriminative processing on the first pseudo MRI image and output a first discriminative probability, and using the first discriminator to perform discriminative processing on the original MRI image and output a second discriminative probability;

[0163] Inputting the first pseudo CT image and the corresponding original CT image into the second discriminator, using the second discriminator to perform discriminative processing on the first pseudo CT image and output a third discriminative probability, and using the second discriminator to perform discriminative processing on the original CT image and output a fourth discriminative probability;

[0164] The first generator and the second generator are adjusted according to the features corresponding to the original CT image, the first pseudo CT image features and the cycle consistency loss function, the first generator and the first discriminant are adjusted according to the first discriminant probability, the second discriminant probability and the first generative adversarial loss function, and the first generator, the second generator and the second discriminator are adjusted according to the third discriminant probability, the fourth discriminant probability and the second generative adversarial loss function until the training is completed.

[0165] Optionally, the training module is also used to:

[0166] Using the first discriminator, convolution processing is performed on the first pseudo MRI image to obtain a first primary feature map, and the importance of each channel in the first primary feature map relative to each channel other than the channel is learned based on the channel attention mechanism to obtain a first weighted feature map with weights, the first primary feature map and the first weighted feature map are multiplied, and upsampling and feature concatenation processing are performed on the obtained first secondary feature map to obtain a first output feature map, and a first discriminant probability is obtained based on the first output feature map;

[0167] Using the first discriminator, convolution processing is performed on the original MRI image to obtain a second primary feature map, and the importance of each channel in the second primary feature map relative to each channel other than the channel is learned based on the channel attention mechanism to obtain a second weighted feature map with weights, the second primary feature map and the second weighted feature map are multiplied, and the obtained second secondary feature map is upsampled and feature concatenated to obtain a second output feature map, and a second discriminant probability is obtained based on the second output feature map;

[0168] Among them, the discrimination probability indicates the probability value of whether the current image is true or false.

[0169] Optionally, the training module is also used to:

[0170] Calculating a first loss value according to the features corresponding to the original CT image, the features of the first pseudo CT image, and a cycle consistency loss function, and determining whether the first loss value is less than a first preset threshold;

[0171] Calculate a second loss value according to the first discrimination probability, the second discrimination probability and the first generative adversarial loss function, and determine whether the second loss value is less than a second preset threshold;

[0172] Calculate a third loss value according to the third discriminant probability, the fourth discriminant probability and the second generative adversarial loss function, and determine whether the third loss value is less than a third preset threshold;

[0173] If the first loss value is less than the first preset threshold, the second loss value is less than the second preset threshold, and the third loss value is less than the third preset threshold, the training is completed and a trained generation model is obtained; otherwise, the first generator and the second generator are adjusted according to the first loss value, the first generator and the first discriminator are adjusted according to the second loss value, and the first generator, the second generator and the second discriminator are adjusted according to the third loss value to obtain a generation model after parameter adjustment; and the next group of original CT images and the corresponding original MRI images are selected from the training set, and the generation model after parameter adjustment is trained until the training is completed to obtain a trained generation model.

[0174] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0175] Figure 8 Schematic diagram of the structure of an electronic device provided by an embodiment of the present application. Figure 8 As shown, the electronic device 800 of this embodiment includes: a processor 810 and a memory 820, wherein the memory 820 stores a computer program 821 that can be run on the processor 810. When the processor 810 executes the computer program 821, the steps in any of the above method embodiments are implemented, for example Figure 1 Alternatively, when the processor 810 executes the computer program 821, the functions of each module / unit in the above-mentioned device embodiments are implemented, for example Figure 7 Functions of modules 701 to 703 are shown.

[0176] Exemplarily, the computer program 821 may be divided into one or more modules / units, one or more modules / units are stored in the memory 820, and are executed by the processor 810 to complete the present application. The one or more modules / units may be a series of computer program instruction segments capable of completing specific functions, and the instruction segments are used to describe the execution process of the computer program 821 in the electronic device 800.

[0177] Those skilled in the art will understand that Figure 8 These are merely examples of electronic devices and do not constitute a limitation of the electronic device. The electronic device may include more or fewer components than those shown in the figure, or a combination of certain components, or different components, such as input and output devices, network access devices, buses, etc.

[0178] The processor 810 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0179] The memory 820 may be an internal storage unit of the electronic device, such as a hard disk or memory of the electronic device, or an external storage device of the electronic device, such as a plug-in hard disk, a smart memory card (SmartMedia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device. The above-mentioned memory 820 may also include both an internal storage unit of the electronic device and an external storage device. The above-mentioned memory 820 is used to store computer programs and other programs and data required by the electronic device. The memory 820 may also be used to temporarily store data that has been output or is to be output.

[0180] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0181] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0182] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0183] In the embodiments provided by the present invention, it should be understood that the disclosed devices / electronic devices and methods can be implemented in other ways. For example, the device / electronic device embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0184] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0185] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0186] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal and software distribution medium, etc.

[0187] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. An image generation method, characterized in that: include: Acquire a target CT image; Using the first generator in the trained generative model to extract shallow features from the target CT image to obtain a shallow feature map; Using the first generator, performing multiple serial deep feature extractions on the shallow feature map to obtain a deep feature map, performing multiple upsampling processes on the deep feature map to obtain target MRI image features, and generating a target MRI image based on the target MRI image features; The generation model includes a first generator, which inputs a CT image to obtain an MRI image and corresponding MRI image features.

2. The image generation method according to claim 1, characterized in that: The method of using the first generator in the trained generative model to extract shallow features from the target CT image to obtain a shallow feature map includes: Using the first generator to perform a preset number of convolution and feature extraction cross processing on the target CT image to obtain a shallow feature map; Wherein, the feature extraction is performed based on the residual convolution block.

3. The image generation method according to claim 1 or 2, characterized in that: Before using the first generator in the trained generation model to extract shallow features from the target CT image, the method further includes: The acquired target CT image is subjected to denoising processing, and the processed target CT image is used as the target CT image.

4. The image generation method according to claim 1, characterized in that: Before using the first generator in the trained generation model to extract shallow features from the target CT image, the method further includes: Obtain different original CT images and corresponding original MRI images; Based on the original CT image, the corresponding original MRI image is registered, and the registered original MRI image is used as the original MRI image, and the original CT image and the corresponding original MRI image constitute a training set; The preset generation model is trained based on the training set to obtain a trained generation model.

5. The image generation method according to claim 4, characterized in that: The generation model also includes a second generator, a first discriminator, and a second discriminator; The step of training a preset generation model based on the training set to obtain a trained generation model includes: Selecting a set of original CT images and corresponding original MRI images from the training set; Inputting the original CT image into the first generator to obtain a first pseudo MRI image and a corresponding first pseudo MRI image feature; The first pseudo MRI image is input into the second generator to obtain a first pseudo CT image and a corresponding first pseudo CT image feature; the second generator inputs the MRI image and outputs a CT image and a corresponding CT image feature; Inputting the first pseudo MRI image and the corresponding original MRI image into the first discriminator, using the first discriminator to perform discriminative processing on the first pseudo MRI image and output a first discriminative probability, and using the first discriminator to perform discriminative processing on the original MRI image and output a second discriminative probability; Inputting the first pseudo CT image and the corresponding original CT image into the second discriminator, using the second discriminator to perform discriminative processing on the first pseudo CT image and output a third discriminative probability, and using the second discriminator to perform discriminative processing on the original CT image and output a fourth discriminative probability; The first generator and the second generator are adjusted according to the features corresponding to the original CT image, the first pseudo CT image features and the cycle consistency loss function, the first generator and the first discriminant are adjusted according to the first discriminant probability, the second discriminant probability and the first generative adversarial loss function, and the first generator, the second generator and the second discriminator are adjusted according to the third discriminant probability, the fourth discriminant probability and the second generative adversarial loss function until the training is completed.

6. The image generation method according to claim 5, characterized in that: The step of inputting the first pseudo MRI image and the corresponding original MRI image into the first discriminator, performing discriminative processing on the first pseudo MRI image using the first discriminator and outputting a first discriminative probability, and performing discriminative processing on the original MRI image using the first discriminator and outputting a second discriminative probability comprises: Using the first discriminator, convolution processing is performed on the first pseudo MRI image to obtain a first primary feature map, and the importance of each channel in the first primary feature map relative to each channel other than the channel is learned based on the channel attention mechanism to obtain a first weighted feature map with weights, the first primary feature map and the first weighted feature map are multiplied, and upsampling and feature concatenation processing are performed on the obtained first secondary feature map to obtain a first output feature map, and a first discriminant probability is obtained based on the first output feature map; Using the first discriminator, convolution processing is performed on the original MRI image to obtain a second primary feature map, and the importance of each channel in the second primary feature map relative to each channel other than the channel is learned based on the channel attention mechanism to obtain a second weighted feature map with weights, the second primary feature map and the second weighted feature map are multiplied, and the obtained second secondary feature map is upsampled and feature concatenated to obtain a second output feature map, and a second discriminant probability is obtained based on the second output feature map; Among them, the discrimination probability indicates the probability value of whether the current image is true or false.

7. The image generation method according to claim 6, characterized in that: The first generator and the second generator are adjusted according to the features corresponding to the original CT image, the first pseudo CT image features and the cycle consistency loss function, the first generator and the first discriminant are adjusted according to the first discriminant probability, the second discriminant probability and the first generation adversarial loss function, and the first generator, the second generator and the second discriminator are adjusted according to the third discriminant probability, the fourth discriminant probability and the second generation adversarial loss function until the training is completed, including: Calculating a first loss value according to the features corresponding to the original CT image, the features of the first pseudo CT image, and a cycle consistency loss function, and determining whether the first loss value is less than a first preset threshold; Calculate a second loss value according to the first discrimination probability, the second discrimination probability and the first generative adversarial loss function, and determine whether the second loss value is less than a second preset threshold; Calculate a third loss value according to the third discriminant probability, the fourth discriminant probability and the second generative adversarial loss function, and determine whether the third loss value is less than a third preset threshold; If the first loss value is less than the first preset threshold, the second loss value is less than the second preset threshold, and the third loss value is less than the third preset threshold, the training is completed and a trained generation model is obtained; otherwise, the first generator and the second generator are adjusted according to the first loss value, the first generator and the first discriminator are adjusted according to the second loss value, and the first generator, the second generator and the second discriminator are adjusted according to the third loss value to obtain a generation model after parameter adjustment; and the next group of original CT images and the corresponding original MRI images are selected from the training set, and the generation model after parameter adjustment is trained until the training is completed to obtain a trained generation model.

8. An image generating device, characterized in that: include: An acquisition module, used for acquiring a target CT image; A processing module, configured to extract shallow features from the target CT image using the first generator in the trained generation model to obtain a shallow feature map; A generating module, configured to use the first generator to perform multiple serial deep feature extractions on the shallow feature map to obtain a deep feature map, perform multiple upsampling processes on the deep feature map to obtain target MRI image features, and generate a target MRI image based on the target MRI image features; The generation model includes a first generator, which inputs a CT image to obtain an MRI image and corresponding MRI image features.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, characterized in that: When the processor executes the computer program, the image generating method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the image generating method according to any one of claims 1 to 7 is implemented.

Images