Medical image generation method and system, electronic equipment and storage medium

Abstract

The invention discloses a medical image generation method and system, electronic equipment and a storage medium. The method comprises the following steps: acquiring a training data set constructed by paired sMRI images and PET images; a medical image generation model comprising a 3D encoder and a 3D decoder is constructed, the 3D encoder comprises a depth separable convolution and a plurality of feature extraction modules comprising MVM blocks and down-sampling, the 3D decoder comprises a plurality of residual blocks, a plurality of up-sampling and a prediction head, and the MVM blocks are used for aggregating input information of the MVM blocks and extracting different scale features; and training the medical image generation model by adopting the training data set to obtain a trained medical image generation model so as to input the sMRI image to be generated into the trained medical image generation model to generate a PET image. According to the method, high-quality conversion from the sMRI image to the PET image can be efficiently and accurately realized, so that the accuracy of the generated image can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of medical image processing, and in particular to a medical image generation method, system, electronic device and storage medium. Background Art

[0002] In clinical practice, positron emission tomography (PET) imaging is important for the early diagnosis of a variety of diseases, such as Alzheimer's disease and cardiovascular disease. However, PET scanning has many limitations, including high cost and risk of radiation exposure, which makes it impossible for many medical centers, especially in underdeveloped areas, to widely provide PET scanning services. In contrast, structural magnetic resonance imaging (sMRI) is less expensive and more popular. Therefore, generating PET images from sMRI data has become a research direction to solve the problem of insufficient PET data.

[0003] Currently, there are many methods trying to achieve the conversion from sMRI to PET images. Although the early traditional 3D convolutional neural networks (3D CNNs) can perform modality conversion, the generated PET images are of low quality. Subsequent networks such as the 3D U-Net network use jump connections to improve the local details and texture information of the synthesized PET images, but there are still deficiencies in the balance between local details and global contours of the image. The 3D-Cycle GAN model retains the basic structure of the image through cycle consistency, but due to the large domain gap between sMRI and PET modalities, the actual conversion effect is still not ideal.

[0004] All of the above existing methods are difficult to achieve high-quality conversion from sMRI to PET images efficiently and accurately, and cannot meet the needs of clinical diagnosis. Summary of the invention

[0005] The present application aims to propose a medical image generation method, system, electronic device and storage medium, which can efficiently and accurately achieve high-quality conversion from sMRI images to PET images, thereby improving the accuracy of generated images.

[0006] In a first aspect, an embodiment of the present application provides a method for generating a medical image, the method comprising: Acquire an sMRI image to be generated, and acquire a training data set constructed by paired sMRI images and PET images; Constructing a medical image generation model including a 3D encoder and a 3D decoder, wherein the 3D encoder includes a depthwise separable convolution and a plurality of feature extraction modules including an MVM block and downsampling, and the 3D decoder includes a plurality of residual blocks, a plurality of upsampling and prediction heads, and the MVM block is used to aggregate input information of the MVM block and extract features of different scales; The medical image generation model is trained using the training data set to obtain a trained medical image generation model, so that the sMRI image to be generated is input into the trained medical image generation model to generate a PET image.

[0007] Compared with the prior art, the first aspect of the present application has the following beneficial effects: The method obtains an sMRI image to be generated, and obtains a training data set constructed by paired sMRI images and PET images; constructs a medical image generation model including a 3D encoder and a 3D decoder, wherein the 3D encoder includes a depth-separable convolution and multiple feature extraction modules including MVM blocks and downsampling, and the 3D decoder includes multiple residual blocks, multiple upsampling and prediction heads, and the MVM block is used to aggregate the input information of the MVM block and extract features of different scales; the training data set is used to train the medical image generation model to obtain a trained medical image generation model, so that the sMRI image to be generated is input into the trained medical image generation model to generate a PET image. In this way, by aggregating and extracting information of different scales, the constructed medical image generation model can better learn image features, and can efficiently and accurately achieve high-quality conversion from sMRI images to PET images, thereby improving the accuracy of generated images.

[0008] In some embodiments, the step of inputting the sMRI image to be generated into the trained medical image generation model to generate a PET image comprises: Processing the sMRI image to be generated by the depthwise separable convolution to obtain a depthwise separable convolution result; Passing the depthwise separable convolution result through the plurality of feature extraction modules including MVM blocks and downsampling to obtain an output result of each feature extraction module; Processing the output results of each feature extraction module by a residual block to obtain a residual block output result corresponding to each feature extraction module; Upsampling the residual block output result corresponding to the last feature extraction module to obtain a first upsampling result; and adding the first upsampling result to the residual block output result corresponding to the previous feature extraction module and taking the average value to obtain a first calculation result; The first calculation result is upsampled to obtain a second upsampled result; and the second upsampled result is added to the residual block output result corresponding to the previous feature extraction module adjacent to the current feature extraction module and the average value is taken to obtain a second calculation result, and so on, until the residual block output result corresponding to the first feature extraction module is added to the calculation result corresponding to the next feature extraction module adjacent to the current feature extraction module and the average value is taken to obtain the calculation result corresponding to the first feature extraction module; Processing the depth-wise separable convolution result through a residual block to obtain a first convolution residual result; Upsampling the calculation result corresponding to the first feature extraction module to obtain a first feature upsampling result; Adding the first feature upsampling result and the first convolution residual result and taking the average value to obtain a first average result; Upsampling the first average result to obtain a second feature upsampling result; Processing the sMRI image to be generated through a plurality of residual blocks to obtain a second convolution residual result; Add the second feature upsampling result and the second convolution residual result and take the average value to obtain a second average result; The second average result is input into the prediction head to generate a PET image.

[0009] In some implementations, the step of passing the depthwise separable convolution result through the plurality of feature extraction modules including MVM blocks and downsampling to obtain an output result of each feature extraction module includes: Inputting the depthwise separable convolution result into the MVM block in the first feature extraction module to obtain a first MVM block output result; Downsampling the output result of the first MVM block to obtain an output result of the first feature extraction module; The output result of the first feature extraction module is used as the input of the MVM block in the next feature extraction module until all feature extraction modules are processed and the output result of each feature extraction module is obtained.

[0010] In some embodiments, the MVM block includes an MVM sub-block and an ASC sub-block, the MVM sub-block is used to extract sequence features of the forward domain, the reverse domain and the mixed domain of the input MVM sub-block feature, the ASC sub-block is used to aggregate the feature information of the input ASC sub-block, and the input of the depth-separable convolution result into the MVM block in the first feature extraction module to obtain the first MVM block output result includes: Processing the depthwise separable convolution result through the ASC sub-block to obtain an ASC sub-block output result; The ASC sub-block output result is subjected to first-layer normalization processing to obtain a first-layer normalized result; Processing the first layer normalization result through the MVM sub-block to obtain an MVM sub-block output result; Adding the ASC sub-block output result and the MVM sub-block output result to obtain a first addition result; Subjecting the first addition result to a second-layer normalization process to obtain a second-layer normalization result; Processing the normalized result of the second layer through a multi-layer perceptron to obtain an output result of the multi-layer perceptron; The multilayer perceptron output result and the first addition result are added to obtain a first MVM block output result.

[0011] In some implementations, the processing the depthwise separable convolution result through the ASC sub-block to obtain an ASC sub-block output result includes: The depth-separable convolution result is processed by a first normalization, a first convolution layer and a first activation function to obtain a first processing result; The depth-separable convolution result is processed by a second normalization, a second convolution layer, and a second activation function to obtain a second processing result; multiplying the first processing result and the second processing result to obtain a multiplication result; The multiplication result is processed by a third normalization, a third convolution layer and a third activation function to obtain a third processing result; The third processing result is added to the depthwise separable convolution result to obtain an ASC sub-block output result.

[0012] In some embodiments, the using the training data set to train the medical image generation model to obtain a trained medical image generation model includes: Constructing perceptual loss functions for medical image generation models; Based on the perceptual loss function, the medical image generation model is trained using the training data set to obtain a trained medical image generation model.

[0013] In some embodiments, the perceptual loss function of constructing a medical image generation model includes: ; in, represents the perceptual loss function, Indicates the number of channels of the feature map, represents the height of the feature map, represents the width of the feature map, Represents the first The feature extraction function of the layer, represents the generated PET image, represents the real PET image, Represents the element-wise squared error.

[0014] In a second aspect, an embodiment of the present application further provides a medical image generation system, the system comprising: A data acquisition unit, used to acquire the sMRI image to be generated, and to acquire a training data set constructed by paired sMRI images and PET images; A model construction unit, configured to construct a medical image generation model comprising a 3D encoder and a 3D decoder, wherein the 3D encoder comprises a depthwise separable convolution and a plurality of feature extraction modules comprising an MVM block and downsampling, the 3D decoder comprises a plurality of residual blocks, a plurality of upsampling and prediction heads, and the MVM block is configured to aggregate input information of the MVM block and extract features of different scales; The image generation unit is used to train the medical image generation model using the training data set to obtain a trained medical image generation model, so as to input the sMRI image to be generated into the trained medical image generation model to generate a PET image.

[0015] In a third aspect, an embodiment of the present application also provides an electronic device comprising at least one control processor and a memory for communicating with the at least one control processor; the memory stores instructions executable by the at least one control processor, and the instructions are executed by the at least one control processor so that the at least one control processor can execute a medical image generation method as described above.

[0016] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute a medical image generation method as described above.

[0017] It can be understood that the beneficial effects of the second to fourth aspects compared with the related art are the same as the beneficial effects of the first aspect compared with the related art. Please refer to the relevant description in the first aspect, and no further details will be given here. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which: Figure 1 is a flowchart of an embodiment of a medical image generation method provided by the present application; Figure 2 It is a schematic diagram of the overall structure of the GenMamba model in the best embodiment of the medical image generation method provided by the present application; Figure 3 is a schematic diagram of the structure of the MVM block in the best embodiment of the medical image generation method provided by the present application; Figure 4It is a schematic diagram of MVM sub-block extraction sequence features in the best embodiment of the medical image generation method provided by the present application; Figure 5 is a schematic diagram of the structure of the ASC sub-block in the best embodiment of the medical image generation method provided by the present application; Figure 6 is a schematic diagram of the structure of an embodiment of a medical image generation system provided by the present application; Figure 7 It is a schematic diagram of the structure of an embodiment of the electronic device provided by the present application. DETAILED DESCRIPTION

[0019] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be understood as limiting the present application.

[0020] In the description of this application, if there is a description of first, second, etc., it is only for the purpose of distinguishing technical features, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.

[0021] It should also be understood that the references to "one embodiment" or "some embodiments" described in the specification of the embodiments of the present application mean that one or more embodiments of the embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Thus, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0022] In the description of the present application, it should be understood that the descriptions involving orientation, such as the orientation or positional relationship indicated as up, down, etc., are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present application.

[0023] In the description of this application, it should be noted that, unless otherwise clearly defined, terms such as setting, installing, connecting, etc. should be understood in a broad sense, and technicians in the relevant technical field can reasonably determine the specific meaning of the above terms in this application based on the specific content of the technical solution.

[0024] In clinical practice, positron emission tomography (PET) imaging is important for the early diagnosis of a variety of diseases, such as Alzheimer's disease and cardiovascular disease. However, PET scanning has many limitations, including high cost and risk of radiation exposure, which makes it impossible for many medical centers, especially in underdeveloped areas, to widely provide PET scanning services. In contrast, structural magnetic resonance imaging (sMRI) is less expensive and more popular. Therefore, generating PET images from sMRI data has become a research direction to solve the problem of insufficient PET data.

[0025] Currently, there are many methods trying to achieve the conversion from sMRI to PET images. Although the early traditional 3D convolutional neural networks (3D CNNs) can perform modality conversion, the generated PET images are of low quality. Subsequent networks such as the 3D U-Net network use jump connections to improve the local details and texture information of the synthesized PET images, but there are still deficiencies in the balance between local details and global contours of the image. The 3D-Cycle GAN model retains the basic structure of the image through cycle consistency, but due to the large domain gap between sMRI and PET modalities, the actual conversion effect is still not ideal.

[0026] All of the above existing methods are difficult to achieve high-quality conversion from sMRI to PET images efficiently and accurately, and cannot meet the needs of clinical diagnosis.

[0027] In order to solve the problem that the above-mentioned existing methods are difficult to efficiently and accurately achieve high-quality conversion from sMRI to PET images, the present application proposes a medical image generation method, system, electronic device and storage medium.

[0028] Reference Figure 1 The present invention provides a method for generating a medical image, which comprises the following steps: Step S100, obtaining an sMRI image to be generated, and obtaining a training data set constructed by paired sMRI images and PET images; Step S200, constructing a medical image generation model including a 3D encoder and a 3D decoder, wherein the 3D encoder includes a depthwise separable convolution and a plurality of feature extraction modules including an MVM block and downsampling, the 3D decoder includes a plurality of residual blocks, a plurality of upsampling and prediction heads, and the MVM block is used to aggregate the input information of the MVM block and extract features of different scales; Step S300: Use the training data set to train the medical image generation model to obtain a trained medical image generation model, so as to input the sMRI image to be generated into the trained medical image generation model to generate a PET image.

[0029] In this embodiment, by acquiring an sMRI image to be generated, and acquiring a training data set constructed by paired sMRI images and PET images; constructing a medical image generation model including a 3D encoder and a 3D decoder, wherein the 3D encoder includes a depthwise separable convolution and multiple feature extraction modules including an MVM block and downsampling, and the 3D decoder includes multiple residual blocks, multiple upsampling and prediction heads, and the MVM block is used to aggregate the input information of the MVM block and extract features of different scales; using the training data set to train the medical image generation model to obtain a trained medical image generation model, so that the sMRI image to be generated is input into the trained medical image generation model to generate a PET image. In this way, by aggregating and extracting information of different scales, the constructed medical image generation model can better learn image features, and can efficiently and accurately achieve high-quality conversion from sMRI images to PET images, thereby improving the accuracy of the generated image.

[0030] The above-mentioned training data set is used to train the medical image generation model to obtain a trained medical image generation model. Appropriate hyperparameters, such as learning rate and number of iterations, can be set during the model training process. The model is trained using the training data set; during the training process, the medical image generation model continuously adjusts the model parameters through back propagation to optimize the model performance.

[0031] The above-mentioned step of inputting the sMRI image to be generated into the trained medical image generation model to generate the PET image may be to input the sMRI image to be generated into the trained medical image generation model, extract feature information of the sMRI image to be generated through a 3D encoder, and then gradually restore the details and structure of the image through a 3D decoder, and convert the features extracted by the 3D encoder into image information of the PET modality.

[0032] In some embodiments, the sMRI image to be generated is input into a trained medical image generation model to generate a PET image, comprising: The sMRI image to be generated is subjected to a depthwise separable convolution process to obtain a depthwise separable convolution result; The depth-wise separable convolution results are passed through multiple feature extraction modules including MVM blocks and downsampling to obtain the output results of each feature extraction module; The output result of each feature extraction module is processed by the residual block to obtain the residual block output result corresponding to each feature extraction module; Upsampling the residual block output result corresponding to the last feature extraction module to obtain a first upsampling result; and adding the first upsampling result to the residual block output result corresponding to the previous feature extraction module and taking the average value to obtain a first calculation result; The first calculation result is upsampled to obtain a second upsampled result; and the second upsampled result is added to the residual block output result corresponding to the previous feature extraction module adjacent to the current feature extraction module and the average value is taken to obtain the second calculation result, and so on, until the residual block output result corresponding to the first feature extraction module is added to the calculation result corresponding to the next feature extraction module adjacent to the current feature extraction module and the average value is taken to obtain the calculation result corresponding to the first feature extraction module; The depth-separable convolution result is processed by the residual block to obtain the first convolution residual result; Upsampling the calculation result corresponding to the first feature extraction module to obtain a first feature upsampling result; The first feature upsampling result and the first convolution residual result are added and averaged to obtain a first average result; Upsampling the first average result to obtain a second feature upsampling result; Processing the sMRI image to be generated through multiple residual blocks to obtain a second convolution residual result; The second feature upsampling result and the second convolution residual result are added and averaged to obtain a second average result; The second average result is input into the prediction head to generate the PET image.

[0033] In this embodiment, the residual block output result corresponding to the last feature extraction module is upsampled to obtain a first upsampled result; the first upsampled result is added to the residual block output result corresponding to the previous feature extraction module and the average value is taken to obtain a first calculation result; the first calculation result is upsampled to obtain a second upsampled result; the second upsampled result is added to the residual block output result corresponding to the previous feature extraction module adjacent to the current feature extraction module and the average value is taken to obtain a second calculation result, and so on, until the residual block output result corresponding to the first feature extraction module is added to the calculation result corresponding to the next feature extraction module adjacent to the current feature extraction module and the average value is taken to obtain a first calculation result. The calculation results corresponding to the feature extraction modules are obtained; the depth-separable convolution results are processed by the residual block to obtain the first convolution residual result; the calculation results corresponding to the first feature extraction module are upsampled to obtain the first feature upsampling result; the first feature upsampling result is added to the first convolution residual result and the average is taken to obtain the first average result; the first average result is upsampled to obtain the second feature upsampling result; the sMRI image to be generated is processed by multiple residual blocks to obtain the second convolution residual result; the second feature upsampling result and the second convolution residual result are added and the average is taken to obtain the second average result; the second average result is input into the prediction head to generate a PET image. In this way, feature information of multiple scales is extracted through multiple feature extraction modules including MVM blocks and downsampling, and then the details and structure of the image are gradually restored by combining the residual block output results with the upsampling results, so that high-quality conversion from sMRI images to PET images can be achieved efficiently and accurately.

[0034] In some implementations, the depthwise separable convolution result is passed through a plurality of feature extraction modules including MVM blocks and downsampling to obtain an output result of each feature extraction module, including: Input the depthwise separable convolution result into the MVM block in the first feature extraction module to obtain the first MVM block output result; Downsampling the output result of the first MVM block to obtain the output result of the first feature extraction module; The output result of the first feature extraction module is used as the input of the MVM block in the next feature extraction module until all feature extraction modules are processed and the output result of each feature extraction module is obtained.

[0035] In this embodiment, the output result of the first MVM block is obtained by inputting the depth-separable convolution result into the MVM block in the first feature extraction module; the output result of the first MVM block is downsampled to obtain the output result of the first feature extraction module; the output result of the first feature extraction module is used as the input of the MVM block in the next feature extraction module, until all feature extraction modules are processed, and the output result of each feature extraction module is obtained. In this way, feature information of different scales is extracted through multiple feature extraction modules, which can lay a good data foundation for the later generation of high-quality images.

[0036] In some embodiments, the MVM block includes an MVM sub-block and an ASC sub-block, the MVM sub-block is used to extract sequence features of the forward domain, the reverse domain, and the mixed domain of the input MVM sub-block feature, the ASC sub-block is used to aggregate the feature information of the input ASC sub-block, and the depth-separable convolution result is input into the MVM block in the first feature extraction module to obtain the first MVM block output result, including: The depth-wise separable convolution result is processed by the ASC sub-block to obtain the ASC sub-block output result; The output result of the ASC sub-block is subjected to the first-layer normalization processing to obtain the first-layer normalized result; The normalized result of the first layer is processed by the MVM sub-block to obtain the MVM sub-block output result; Adding the output result of the ASC sub-block and the output result of the MVM sub-block to obtain a first addition result; The first addition result is subjected to a second-layer normalization process to obtain a second-layer normalized result; The normalized result of the second layer is processed by a multi-layer perceptron to obtain the output result of the multi-layer perceptron; The multilayer perceptron output result and the first addition result are added to obtain the first MVM block output result.

[0037] In this embodiment, the ASC sub-block output result is obtained by processing the depth separable convolution result through the ASC sub-block; the ASC sub-block output result is processed through the first layer normalization to obtain the first layer normalization result; the first layer normalization result is processed through the MVM sub-block to obtain the MVM sub-block output result; the ASC sub-block output result and the MVM sub-block output result are added to obtain the first addition result; the first addition result is processed through the second layer normalization to obtain the second layer normalization result; the second layer normalization result is processed through the multi-layer perceptron to obtain the multi-layer perceptron output result; the multi-layer perceptron output result and the first addition result are added to obtain the first MVM block output result. In this way, the forward domain, reverse domain and mixed domain sequence features of the input MVM sub-block feature are extracted through the MVM sub-block, and the feature information of the input ASC sub-block is aggregated through the ASC sub-block, which can enhance the feature extraction ability and information fusion ability of the medical image generation model. And by extracting the feature information of each domain of different spatial perspectives, the learning ability of the medical image generation model for 3D information can be enhanced.

[0038] In some implementations, the depthwise separable convolution result is processed by the ASC sub-block to obtain an ASC sub-block output result, including: The depth-wise separable convolution result is processed by a first normalization, a first convolution layer, and a first activation function to obtain a first processing result; The depth-separable convolution result is processed by a second normalization, a second convolution layer, and a second activation function to obtain a second processing result; Multiplying the first processing result and the second processing result to obtain a multiplication result; The multiplication result is processed by a third normalization, a third convolution layer, and a third activation function to obtain a third processing result; The third processing result is added to the depthwise separable convolution result to obtain the ASC sub-block output result.

[0039] In this embodiment, by aggregating the feature information of the input ASC sub-blocks, the information fusion capability of the medical image generation model can be enhanced, laying a good data foundation for the subsequent generation of high-quality images.

[0040] In some embodiments, a medical image generation model is trained using a training data set to obtain a trained medical image generation model, including: Constructing perceptual loss functions for medical image generation models; Based on the perceptual loss function, the medical image generation model is trained using the training data set to obtain a trained medical image generation model.

[0041] In this embodiment, a perceptual loss function of a medical image generation model is constructed; based on the perceptual loss function, the medical image generation model is trained using a training data set to obtain a trained medical image generation model. In this way, the PET image generated by the perceptual loss function can focus on details and improve image quality.

[0042] In some embodiments, constructing a perceptual loss function of a medical image generation model includes: ; in, represents the perceptual loss function, Indicates the number of channels of the feature map, represents the height of the feature map, represents the width of the feature map, Represents the first The feature extraction function of the layer, represents the generated PET image, represents the real PET image, Represents the element-wise squared error.

[0043] To facilitate understanding by those skilled in the art, a set of best embodiments is provided below: In clinical practice, positron emission tomography (PET) imaging is important for the early diagnosis of a variety of diseases, such as Alzheimer's disease and cardiovascular disease. However, PET scanning has many limitations, including high cost and risk of radiation exposure, which makes it impossible for many medical centers, especially in underdeveloped areas, to widely provide PET scanning services. In contrast, structural magnetic resonance imaging (sMRI) is less expensive and more popular. Therefore, generating PET images from sMRI data has become a research direction to solve the problem of insufficient PET data.

[0044] Currently, there are many methods that attempt to achieve the conversion from sMRI to PET. Although the early traditional 3D convolutional neural networks (3D CNNs) were able to perform modality conversion, the generated PET images were of low quality. Subsequent networks such as the 3D U-Net network used jump connections to improve the local details and texture information of synthetic PET images, but they still have shortcomings in balancing the local details and global contours of the image. The 3D-Cycle GAN model retains the basic structure of the image through cycle consistency, but due to the large domain gap between sMRI and PET modalities, the actual conversion effect is still not ideal. These methods are difficult to achieve high-quality conversion from sMRI to PET efficiently and accurately, and cannot meet the needs of clinical diagnosis.

[0045] In order to solve the problems existing in the conversion from sMRI to PET in the prior art, this embodiment proposes a deep learning method for generating the corresponding PET modality from the sMRI modality based on the GenMamba model (i.e., the medical image generation model), aiming to improve the quality of generated PET images, enhance the feature extraction capability and information fusion capability of the medical image generation model, and provide more reliable imaging support for clinical diagnosis.

[0046] To achieve the above purpose, the technical solution of this embodiment is as follows: The deep learning method of this embodiment for generating the corresponding PET modality from the sMRI modality based on the GenMamba model includes the following steps: Step S1: Input the subject's sMRI image and pair it with the corresponding PET image. Collect imaging data from the Alzheimer's Disease Neuroimaging Initiative (ADNI) database, where the sMRI image is a T1-weighted MP-RAGE sequence acquired from a 1.5T scanner, usually The voxel size is approximately , the image resolution is ; Process PET images into a unified format.

[0047] Step S2: Construct the GenMamba model (i.e., medical image generation model), which includes data preprocessing (i.e., depthwise separable convolution) of the data input in step S1. Normalize the sMRI image and map the pixel values ​​to interval to eliminate the impact of data scale differences on model training. The overall structure diagram of the GenMamba model is as follows Figure 2 As shown, the GenMamba model includes a 3D encoder and a 3D decoder. The 3D encoder includes a depth-separable convolution, multiple Multi-View Mamba (MVM) blocks and multiple downsampling. The number of MVM blocks is the same as the number of downsampling. An MVM block and a downsampling are combined into a feature extraction module to obtain multiple feature extraction modules (four feature extraction modules are constructed in this embodiment); the 3D decoder includes multiple residual blocks, multiple upsampling (i.e., deconvolution) and a prediction head.

[0048] Step S3: Use the Multi-View Mamba (MVM) sub-block in the 3D encoder to perform multi-scale modeling on the pre-processed data, and aggregate the slice information through the Aggregated Spatial Convolution (ASC) sub-block. Specifically: An sMRI image (i.e., 3D input) is obtained, and the sMRI image is preprocessed by a depthwise separable convolution (Stem) to obtain a depthwise separable convolution result; the depthwise separable convolution result is input into the first feature extraction module to obtain the output result of the first feature extraction module (i.e., the extracted feature), wherein the feature extraction module first processes the obtained input information (which may be the depthwise separable convolution result) through an MVM block to obtain an MVM block output result, and then downsamples the MVM block output result to obtain the feature extraction module output result.

[0049] Specifically, refer to Figure 3 , the MVM block includes an MVM sub-block and an ASC sub-block. The processing method of the MVM block is as follows: the input information (i.e., the extracted image features) obtained by the MVM block is processed by the ASC sub-block to obtain the output result of the ASC sub-block, the output result of the ASC sub-block is processed by the first layer normalization (LayerNorm) to obtain the first layer normalization result, the first layer normalization result is processed by the MVM sub-block to obtain the output result of the MVM sub-block. The output result of the ASC sub-block and the output result of the MVM sub-block are added to obtain the first addition result. The first addition result is processed by the second layer normalization to obtain the second layer normalization result, the second layer normalization result is processed by the multi-layer perceptron (MLP) to obtain the multi-layer perceptron output result, the multi-layer perceptron output result and the first addition result are added to obtain the output result of the MVM block.

[0050] The MVM sub-block models each domain from three different spatial perspectives, as follows: ; in, Represent the sequence features from the forward domain, the reverse domain, and the mixed domain respectively. Because the image features received by the Multi-View Mamba (MVM) sub-block have multiple feature information elements, the multiple feature information elements are arranged in sequence to obtain a column of feature information elements. Figure 4 The forward domain refers to extracting feature information elements in sequence from the front of a column of feature information elements to obtain the sequence features of the forward domain; the reverse domain refers to extracting feature information elements in sequence from the back of a column of feature information elements to obtain the sequence features of the reverse domain; the mixed domain refers to extracting one from the front of a column of feature information elements, and then extracting one from the back of a column of feature information elements to cross-extract feature information elements to obtain the sequence features of the mixed domain, so as to enhance the learning ability of 3D information.

[0051] Specifically, the ASC sub-block processing method is as follows: Figure 5, the input information obtained by the ASC sub-block is subjected to the first normalization (Norm), the first convolutional layer (3×3×3 Conv3D) and the first activation function (ReLU) to obtain the first processing result; the input information obtained by the ASC sub-block is subjected to the second normalization (Norm), the second convolutional layer (1×1×1 Conv3D) and the second activation function (ReLU) to obtain the second processing result; the first processing result and the second processing result are multiplied to obtain the multiplication result, and the multiplication result is subjected to the third normalization (Norm), the third convolutional layer (3×3×3 Conv3D) and the third activation function (ReLU) to obtain the third processing result; finally, the third processing result is added to the input information obtained by the ASC sub-block to obtain the output result of the ASC sub-block.

[0052] The ASC sub-block undergoes specific convolution operations and fusion operations, and the formula is: ; in, Represents the input 3D feature information, Represents the aggregation space module. The ASC sub-block can effectively aggregate the spatiotemporal features between slices.

[0053] The output of the first feature extraction module is used as the input of the second feature extraction module, and the second feature extraction module processes the input information to obtain the output of the second feature extraction module. , and so on, until the output of the last feature extraction module is obtained. In this way, multi-scale features can be extracted through the encoder .

[0054] Step S4: Decoding using a 3D decoder. The 3D decoder includes multiple residual blocks, multiple upsampling (i.e., deconvolution (Transpose Conv)) and a prediction head (Generator Head, the prediction head adopts a network layer structure known to those skilled in the art, which is not described in detail in this embodiment). Through convolution operations (i.e., deconvolution) and combined with jump connections (i.e., residual blocks (Residual Block)), PET modality information is predicted and generated. The decoder gradually restores the details and structure of the image and converts the features extracted by the encoder into image information of the PET modality.

[0055] The specific process of the 3D decoder is as follows: Output the result of the last feature extraction module After the residual block processing, the first residual output result is obtained, and the output result of the previous feature extraction module of the last feature extraction module is obtained. After the residual block is processed, a second residual output result is obtained, the first residual output result is upsampled to obtain a first upsampled result, the first upsampled result and the second residual output result are added and averaged to obtain a first calculation result; The first calculation result is upsampled to obtain the second upsampling result, and the output result of the previous feature extraction module is After the residual block is processed, a third residual output result is obtained, and the third residual output result and the second up-sampling result are added and averaged to obtain a second calculation result; The second calculation result is upsampled to obtain a third upsampling result, and the output result of the previous feature extraction module is After the residual block is processed, a fourth residual output result is obtained, and the fourth residual output result and the third upsampling result are added and averaged to obtain a third calculation result (i.e., the calculation result corresponding to the first feature extraction module); The third calculation result is upsampled to obtain the fourth upsampling result (i.e. the first feature upsampling result), and the depth-separable convolution result is After the residual block is processed, a fifth residual output result (i.e., the first convolution residual result) is obtained, and the fifth residual output result and the fourth upsampling result are added and averaged to obtain a fourth calculation result (i.e., the first average result); The fourth calculation result is upsampled to obtain a fifth upsampling result (i.e., the second feature upsampling result), and the sMRI image is processed by a residual block to obtain a sixth residual output result. , the sixth residual output result is processed by a residual block again to obtain a seventh residual output result (i.e., the second convolution residual result), the seventh residual output result and the fifth upsampling result are added and averaged to obtain a fifth calculation result (i.e., the second average result); The feature information in the fifth calculation result is input into the prediction head for prediction, and the PET image is obtained by prediction, that is, the PET image is generated.

[0056] Step S5: During the generation process, the perceptual loss function is used to guide the generated PET image to focus on details and improve image quality. Using 3D-ResNet10 pre-trained on 23 medical datasets as a feature extractor, the difference between the features of the real PET image and the synthetic image is calculated, and the perceptual loss is obtained by averaging from the spatial dimension to make the synthetic image closer to the features of the real PET image. The calculation formula of the perceptual loss function is: ; in, Represents the first Feature extraction function of the layer; and represent the generated PET image and the real PET image respectively; Respectively represent the number of channels, height and width of the feature map; Represents the element-by-element squared error, and finally averages all positions in the feature map. The pre-trained 3D-ResNet-10 is a model that has been trained on 23 medical datasets; the real PET images and the generated PET images are input into the feature extractor respectively, the features extracted in each layer are normalized on a specific channel, the difference between the input features and the target features is calculated, and the perceptual loss is obtained by averaging from the spatial dimension.

[0057] Step S6: Verify the effectiveness of the MVM sub-block, and compare the SSIM, PSNR, and MAE indicators under different Mamba domain modeling; verify the effectiveness of the ASC sub-block, and compare the changes in the SSIM, PSNR, and MAE indicators before and after using the ASC sub-block; adjust the weight of the perceptual loss function , compared with different The SSIM, PSNR and MAE indicators of the synthetic PET image under the given values ​​are determined to determine the optimal These indicators are used to quantitatively evaluate the similarity and quality difference between the generated PET images and the real PET images.

[0058] This embodiment uses the innovative GenMamba model, combined with MVM, ASC sub-blocks and perceptual loss function, to effectively improve the quality and accuracy of generating PET modality images from sMRI modality, providing more valuable imaging data for clinical diagnosis.

[0059] Reference Figure 6 The present application also provides a medical image generation system, which includes a data acquisition unit 100, a model construction unit 200 and an image generation unit 300, wherein: The data acquisition unit 100 is used to acquire the sMRI image to be generated, and to acquire a training data set constructed by paired sMRI images and PET images; A model construction unit 200 is used to construct a medical image generation model including a 3D encoder and a 3D decoder, wherein the 3D encoder includes a depthwise separable convolution and a plurality of feature extraction modules including an MVM block and downsampling, the 3D decoder includes a plurality of residual blocks, a plurality of upsampling and prediction heads, and the MVM block is used to aggregate input information of the MVM block and extract features of different scales; The image generation unit 300 is used to train the medical image generation model using the training data set to obtain a trained medical image generation model, so as to input the sMRI image to be generated into the trained medical image generation model to generate a PET image.

[0060] In some implementations, the image generation unit 300 may be specifically configured to: The sMRI image to be generated is subjected to a depthwise separable convolution process to obtain a depthwise separable convolution result; The depth-wise separable convolution results are passed through multiple feature extraction modules including MVM blocks and downsampling to obtain the output results of each feature extraction module; The output result of each feature extraction module is processed by the residual block to obtain the residual block output result corresponding to each feature extraction module; Upsampling the residual block output result corresponding to the last feature extraction module to obtain a first upsampling result; and adding the first upsampling result to the residual block output result corresponding to the previous feature extraction module and taking the average value to obtain a first calculation result; The first calculation result is upsampled to obtain a second upsampled result; and the second upsampled result is added to the residual block output result corresponding to the previous feature extraction module adjacent to the current feature extraction module and the average value is taken to obtain the second calculation result, and so on, until the residual block output result corresponding to the first feature extraction module is added to the calculation result corresponding to the next feature extraction module adjacent to the current feature extraction module and the average value is taken to obtain the calculation result corresponding to the first feature extraction module; The depth-separable convolution result is processed by the residual block to obtain the first convolution residual result; Upsampling the calculation result corresponding to the first feature extraction module to obtain a first feature upsampling result; The first feature upsampling result and the first convolution residual result are added and averaged to obtain a first average result; Upsampling the first average result to obtain a second feature upsampling result; Processing the sMRI image to be generated through multiple residual blocks to obtain a second convolution residual result; The second feature upsampling result and the second convolution residual result are added and averaged to obtain a second average result; The second average result is input into the prediction head to generate the PET image.

[0061] In some implementations, the image generation unit 300 may be specifically configured to: Input the depthwise separable convolution result into the MVM block in the first feature extraction module to obtain the first MVM block output result; Downsampling the output result of the first MVM block to obtain the output result of the first feature extraction module; The output result of the first feature extraction module is used as the input of the MVM block in the next feature extraction module until all feature extraction modules are processed and the output result of each feature extraction module is obtained.

[0062] In some implementations, the image generation unit 300 may be specifically configured to: The depth-wise separable convolution result is processed by the ASC sub-block to obtain the ASC sub-block output result; The output result of the ASC sub-block is subjected to the first-layer normalization processing to obtain the first-layer normalized result; The normalized result of the first layer is processed by the MVM sub-block to obtain the MVM sub-block output result; Adding the output result of the ASC sub-block and the output result of the MVM sub-block to obtain a first addition result; The first addition result is subjected to a second-layer normalization process to obtain a second-layer normalized result; The normalized result of the second layer is processed by a multi-layer perceptron to obtain the output result of the multi-layer perceptron; The multilayer perceptron output result and the first addition result are added to obtain the first MVM block output result.

[0063] In some implementations, the image generation unit 300 may be specifically configured to: The depth-wise separable convolution result is processed by a first normalization, a first convolution layer, and a first activation function to obtain a first processing result; The depth-separable convolution result is processed by a second normalization, a second convolution layer, and a second activation function to obtain a second processing result; Multiplying the first processing result and the second processing result to obtain a multiplication result; The multiplication result is processed by a third normalization, a third convolution layer, and a third activation function to obtain a third processing result; The third processing result is added to the depthwise separable convolution result to obtain the ASC sub-block output result.

[0064] In some implementations, the image generation unit 300 may be specifically configured to: Constructing perceptual loss functions for medical image generation models; Based on the perceptual loss function, the medical image generation model is trained using the training data set to obtain a trained medical image generation model.

[0065] In some implementations, the image generation unit 300 may be specifically configured to: ; in, represents the perceptual loss function, Indicates the number of channels of the feature map, represents the height of the feature map, represents the width of the feature map, Represents the first The feature extraction function of the layer, represents the generated PET image, represents the real PET image, Represents the element-wise squared error.

[0066] It should be noted that, since a medical image generation system in this embodiment and a medical image generation method described above are based on the same inventive concept, the corresponding contents in the method embodiment are also applicable to this system embodiment and will not be described in detail here.

[0067] Reference Figure 7 The embodiment of the present application further provides an electronic device, the electronic device comprising: at least one memory; at least one processor; at least one program; The programs are stored in the memory, and the processor executes at least one program to implement the medical image generation method described above in the present disclosure.

[0068] The electronic device may be any intelligent terminal including a mobile phone, a tablet computer, a personal digital assistant (PDA), a vehicle-mounted computer, etc.

[0069] The electronic device according to the embodiment of the present application is described in detail below.

[0070] The processor 1600 may be implemented by a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present disclosure; The memory 1700 may be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1700 may store an operating system and other application programs. When the technical solution provided in the embodiments of this specification is implemented by software or firmware, the relevant program code is stored in the memory 1700, and the processor 1600 calls and executes the medical image generation method of the embodiment of the present disclosure.

[0071] Input / output interface 1800, used to implement information input and output; Communication interface 1900, used to realize communication interaction between the device and other devices, which can be realized through wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WIFI, Bluetooth, etc.); Bus 2000 , which transmits information between various components of the device (e.g., processor 1600 , memory 1700 , input / output interface 1800 , and communication interface 1900 ); The processor 1600 , the memory 1700 , the input / output interface 1800 , and the communication interface 1900 are connected to each other in communication within the device via the bus 2000 .

[0072] The embodiment of the present disclosure further provides a storage medium, which is a computer-readable storage medium. The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the above-mentioned medical image generation method.

[0073] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0074] The embodiments described in the embodiments of the present disclosure are intended to more clearly illustrate the technical solutions of the embodiments of the present disclosure and do not constitute a limitation on the technical solutions provided by the embodiments of the present disclosure. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of the present disclosure are also applicable to similar technical problems.

[0075] Those skilled in the art will appreciate that the technical solutions shown in the figures do not limit the embodiments of the present disclosure and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0076] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0077] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0078] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0079] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0080] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0081] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0082] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0083] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable an electronic device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store programs. The above is a detailed description of the embodiments of the present application in conjunction with the accompanying drawings, but the present application is not limited to the above embodiments. Within the scope of knowledge possessed by ordinary technicians in the relevant technical field, various changes can be made without departing from the purpose of the present application.

[0084] The embodiments of the present application are described in detail above in conjunction with the accompanying drawings, but the present application is not limited to the above embodiments. Various changes can be made within the knowledge scope of ordinary technicians in the relevant technical field without departing from the purpose of the present application.

Claims

1. A medical image generation method, characterized in that: The method comprises: Acquire an sMRI image to be generated, and acquire a training data set constructed by paired sMRI images and PET images; Constructing a medical image generation model including a 3D encoder and a 3D decoder, wherein the 3D encoder includes a depthwise separable convolution and a plurality of feature extraction modules including an MVM block and downsampling, and the 3D decoder includes a plurality of residual blocks, a plurality of upsampling and prediction heads, and the MVM block is used to aggregate input information of the MVM block and extract features of different scales; The medical image generation model is trained using the training data set to obtain a trained medical image generation model, so that the sMRI image to be generated is input into the trained medical image generation model to generate a PET image.

2. The medical image generation method according to claim 1, characterized in that: The step of inputting the sMRI image to be generated into the trained medical image generation model to generate a PET image comprises: Processing the sMRI image to be generated by the depthwise separable convolution to obtain a depthwise separable convolution result; Passing the depthwise separable convolution result through the plurality of feature extraction modules including MVM blocks and downsampling to obtain an output result of each feature extraction module; Processing the output results of each feature extraction module by a residual block to obtain a residual block output result corresponding to each feature extraction module; Upsampling the residual block output result corresponding to the last feature extraction module to obtain a first upsampling result; and adding the first upsampling result to the residual block output result corresponding to the previous feature extraction module and taking the average value to obtain a first calculation result; The first calculation result is upsampled to obtain a second upsampled result; and the second upsampled result is added to the residual block output result corresponding to the previous feature extraction module adjacent to the current feature extraction module and the average value is taken to obtain a second calculation result, and so on, until the residual block output result corresponding to the first feature extraction module is added to the calculation result corresponding to the next feature extraction module adjacent to the current feature extraction module and the average value is taken to obtain the calculation result corresponding to the first feature extraction module; Processing the depth-wise separable convolution result through a residual block to obtain a first convolution residual result; Upsampling the calculation result corresponding to the first feature extraction module to obtain a first feature upsampling result; Adding the first feature upsampling result and the first convolution residual result and taking the average value to obtain a first average result; Upsampling the first average result to obtain a second feature upsampling result; Processing the sMRI image to be generated through a plurality of residual blocks to obtain a second convolution residual result; Add the second feature upsampling result and the second convolution residual result and take the average value to obtain a second average result; The second average result is input into the prediction head to generate a PET image.

3. The medical image generation method according to claim 2, characterized in that: The step of passing the depth-separable convolution result through the plurality of feature extraction modules including MVM blocks and downsampling to obtain an output result of each feature extraction module comprises: Inputting the depthwise separable convolution result into the MVM block in the first feature extraction module to obtain a first MVM block output result; Downsampling the output result of the first MVM block to obtain an output result of the first feature extraction module; The output result of the first feature extraction module is used as the input of the MVM block in the next feature extraction module until all feature extraction modules are processed and the output result of each feature extraction module is obtained.

4. The medical image generation method according to claim 3, characterized in that: The MVM block includes an MVM sub-block and an ASC sub-block, wherein the MVM sub-block is used to extract sequence features of the forward domain, the reverse domain and the mixed domain of the input MVM sub-block feature, and the ASC sub-block is used to aggregate the feature information of the input ASC sub-block, and the input of the depth-separable convolution result into the MVM block in the first feature extraction module to obtain the first MVM block output result includes: Processing the depthwise separable convolution result through the ASC sub-block to obtain an ASC sub-block output result; The ASC sub-block output result is subjected to first-layer normalization processing to obtain a first-layer normalized result; Processing the first layer normalization result through the MVM sub-block to obtain an MVM sub-block output result; Adding the ASC sub-block output result and the MVM sub-block output result to obtain a first addition result; Subjecting the first addition result to a second-layer normalization process to obtain a second-layer normalization result; Processing the normalized result of the second layer through a multi-layer perceptron to obtain an output result of the multi-layer perceptron; The multilayer perceptron output result and the first addition result are added to obtain a first MVM block output result.

5. The medical image generation method according to claim 4, characterized in that: The step of processing the depthwise separable convolution result through the ASC sub-block to obtain an ASC sub-block output result includes: The depth-separable convolution result is processed by a first normalization, a first convolution layer and a first activation function to obtain a first processing result; The depth-separable convolution result is processed by a second normalization, a second convolution layer, and a second activation function to obtain a second processing result; multiplying the first processing result and the second processing result to obtain a multiplication result; The multiplication result is processed by a third normalization, a third convolution layer and a third activation function to obtain a third processing result; The third processing result is added to the depthwise separable convolution result to obtain an ASC sub-block output result.

6. The medical image generation method according to claim 1, characterized in that: The step of training the medical image generation model using the training data set to obtain a trained medical image generation model comprises: Constructing perceptual loss functions for medical image generation models; Based on the perceptual loss function, the medical image generation model is trained using the training data set to obtain a trained medical image generation model.

7. The medical image generation method according to claim 6, characterized in that: The perceptual loss function of constructing the medical image generation model includes: ； in, represents the perceptual loss function, Indicates the number of channels of the feature map, represents the height of the feature map, represents the width of the feature map, Represents the first The feature extraction function of the layer, represents the generated PET image, represents the real PET image, Represents the element-wise squared error.

8. A medical image generation system, characterized in that: The system comprises: A data acquisition unit, used to acquire the sMRI image to be generated, and to acquire a training data set constructed by paired sMRI images and PET images; A model construction unit, configured to construct a medical image generation model comprising a 3D encoder and a 3D decoder, wherein the 3D encoder comprises a depthwise separable convolution and a plurality of feature extraction modules comprising an MVM block and downsampling, the 3D decoder comprises a plurality of residual blocks, a plurality of upsampling and prediction heads, and the MVM block is configured to aggregate input information of the MVM block and extract features of different scales; The image generation unit is used to train the medical image generation model using the training data set to obtain a trained medical image generation model, so as to input the sMRI image to be generated into the trained medical image generation model to generate a PET image.

9. An electronic device, characterized in that: It includes at least one control processor and a memory for communicating with the at least one control processor; the memory stores instructions that can be executed by the at least one control processor, and the instructions are executed by the at least one control processor to enable the at least one control processor to execute the medical image generation method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the medical image generating method according to any one of claims 1 to 7.

Images