A CT-to-PET cross-modality medical image generation method, system and storage medium

Abstract

The application relates to the field of medical images and neural network technologies and discloses a CT-to-PET cross-modality medical image generation method and system and a storage medium. The method comprises the following steps: constructing a prior perception tensor; inputting the prior perception tensor into a prior encoder to obtain a multi-scale feature map; obtaining a corresponding PET image and preprocessing the PET image; adding noise to the preprocessed PET image based on a forward process of a diffusion model; inputting the current time step's noisy PET image and the multi-scale feature map into a denoising U-shaped network based on spatial adaptive modulation to predict a denoised PET image; calculating a total loss based on the PET image and the denoised PET image, and updating network parameters of the prior encoder and the denoising U-shaped network according to the total loss to complete training. The trained prior encoder and the denoising U-shaped network are used to convert a CT image into a PET image. The application effectively improves the structural fidelity and clinical quantitative reliability of cross-modality medical image generation.

Description

Technical Field

[0001] This application relates to the fields of medical imaging and neural network technology, and in particular to a method, system and storage medium for generating cross-modal medical images from CT to PET. Background Technology

[0002] In the field of medical imaging, computed tomography (CT) and positron emission tomography (PET) are two complementary key imaging modalities. The former provides high-resolution anatomical information, while the latter reflects tissue metabolic function information. The fusion of the two has significant clinical value for the diagnosis and treatment of major diseases such as cancer. However, PET imaging is expensive, time-consuming, carries radiation risks, and has low equipment availability, limiting its widespread application. Therefore, using artificial intelligence technology to generate PET images from CT images has become a research hotspot.

[0003] Current mainstream methods are mainly based on Generative Adversarial Networks (GANs), Variational Autoencoders (VAEs), and Diffusion Models. By learning the statistical distribution mapping between the source and target domains, they have made some progress in overall structural similarity and texture realism. However, existing technologies still have the following key problems: First, the models usually perform isotropic uniform optimization on the entire image, ignoring the essential difference between the lesion area and the background area, resulting in blurred lesion edges, smoothing of metabolic hotspots, or the generation of false high metabolic artifacts. Second, when using clinically available prior information (such as lesion masks), simple channel splicing is often used for shallow fusion. As the number of network layers increases, information decays severely, and deep networks lose precise anatomical guidance, failing to guarantee strict alignment between the generated lesions and the real anatomical structures. Third, existing optimization objectives mostly focus on visual perception realism, lacking explicit physical constraints on local metabolic intensity, resulting in large deviations in the quantitative measurement values ​​of the generated PET images, making it difficult to meet the needs of clinical quantitative analysis. In addition, existing diffusion models employ a blind generation paradigm from pure Gaussian noise to images. The random initialization process is prone to losing the subtle anatomical topology of the source CT image, which may cause lesion location drift or geometric deformation.

[0004] In summary, existing technologies cannot achieve differentiated generation guidance between lesion and background regions, struggle to deeply integrate prior lesion information into each level of the generation network, and fail to ensure both strict alignment of anatomical structures and quantitative accuracy of metabolic functions. This results in lesion distortion, anatomical misalignment, and quantitative bias in the generated PET images. Therefore, constructing a cross-modal image generation method that can deeply integrate anatomical and pathological priors and achieve spatial adaptive constraints to improve the structural fidelity and clinical quantitative reliability of the generated images is a pressing technical challenge in this field. Summary of the Invention

[0005] The purpose of this invention is to address at least one deficiency in the existing technology and provide a method, system and storage medium for generating cross-modal medical images from CT to PET. This invention achieves differentiated and accurate generation of lesion areas and anatomical backgrounds by fusing prior perception and spatial adaptive modulation, effectively improving the structural fidelity and clinical quantitative reliability of cross-modal medical image generation.

[0006] To achieve the above objectives, in a first aspect, the present invention provides a method for training a neural network model based on prior perception and spatial adaptive modulation, the method comprising the following steps: Based on the acquired CT images and the corresponding lesion masks, a priori perception tensor is constructed. The prior perception tensor is input into the prior encoder for feature extraction to obtain a multi-scale feature map. The PET image corresponding to the CT image is acquired and preprocessed. Noise is added to the preprocessed PET image based on the forward process of the diffusion model to obtain a noisy PET image. The noisy PET image and multi-scale feature map at the current time step are input into a spatially adaptive modulation-based denoising U-shaped network. The multi-scale feature map is used to modulate the feature maps of the corresponding scales in the encoder and decoder of the denoising U-shaped network to predict the denoised PET image. The total loss is calculated based on the PET image and the denoised PET image, and the network parameters of the prior encoder and the denoised U-shaped network are updated according to the total loss to complete the training.

[0007] Furthermore, the construction of the prior perception tensor based on the CT image sequence and the corresponding lesion mask specifically includes: The CT image is truncated by window width and window level, and the truncated CT values ​​are linearly normalized to obtain a standardized CT image. The standardized CT image is concatenated with the corresponding lesion mask to obtain the prior perception tensor.

[0008] Furthermore, the prior encoder includes multiple cascaded convolutional residual modules, which extract features at different spatial resolutions through step-by-step downsampling to obtain the multi-scale features.

[0009] Furthermore, the modulation of feature maps at corresponding scales in the encoder and decoder of the denoising U-shaped network using the multi-scale features specifically includes: (1). For the feature map to be modulated in any network layer of the denoising U-shaped network, obtain a prior feature map with the same resolution as the feature map to be modulated in the multi-scale prior feature map; (2). Perform group normalization on the feature map to be modulated to obtain a standardized feature map; (3). Based on the prior feature map, spatially adaptive scaling and bias coefficients are generated through convolutional layers; (4) Perform a pixel-by-pixel affine transformation on the normalized feature map using the scaling factor and the bias factor to obtain a modulated feature map, which is then processed by the subsequent layers of the denoising network to finally generate the denoised PET image.

[0010] Furthermore, the group normalization operation specifically includes the following process:

[0011]

[0012]

[0013] in, For the input feature map at the th Each channel, spatial coordinates The original activation value at that location; and Channels Mean and standard deviation of the group; It is a set that covers three spatial dimensions: depth, height, and width, and also includes passageways. All other channels belonging to the same group, i.e. Included in this group All voxel points in each channel; This is a standardized feature map.

[0014] Furthermore, the total loss includes reconstruction loss and local metabolic consistency loss. The reconstruction loss is used to constrain the difference in overall structure between the denoised PET image and the PET image. The local metabolic consistency loss includes mean error and peak error in the region corresponding to the lesion mask, which is used to constrain the difference in metabolic intensity between the denoised PET image and the PET image in the region corresponding to the lesion mask.

[0015] Furthermore, the expression for the total loss is as follows:

[0016] in, The weight hyperparameters are preset to control the proportion of the two types of losses. The reconstruction loss is expressed as follows:

[0017] The expression for the loss of local metabolic consistency is as follows:

[0018] The TopK operation returns the average value of the top K% of pixels with the largest values ​​within the region. For Hadama accumulation, For the initial PET image, The denoised PET image. For pre-set weight hyperparameters, The lesion area.

[0019] Secondly, the present invention also provides a method for generating cross-modal medical images from CT to PET, the method comprising the following steps: Based on the acquired target CT image and the corresponding lesion mask, a target prior perception tensor is constructed. The target prior perception tensor is input into the prior encoder to obtain the first multi-scale feature map, wherein the... The prior encoder is trained according to the method described in the first aspect; Gaussian noise is obtained as the initial PET image to be generated; the PET image to be generated and the first multi-scale feature map are fed into a denoising U-shaped network. The denoising U-shaped network is based on a spatial adaptive modulation module. The first multi-scale feature map is used to modulate the feature maps of the corresponding scale in the encoder and decoder of the denoising U-shaped network to predict the denoising result of the current time step. The denoising result is used to update the PET image to be generated, and the process of iteratively changing the PET image to be generated is performed. The process involves transferring the first multi-scale feature map to a denoising U-shaped network until a preset time step is reached, at which point the final target PET image is output. The denoising U-shaped network is trained according to the method described in the first aspect.

[0020] Thirdly, the present invention also provides a CT-to-PET cross-modal medical image generation system, the system being based on the method described in the second aspect, comprising: Construction module: used to construct the target prior perception tensor based on the acquired target CT image and the corresponding lesion mask; Acquisition module: used to input the target prior perception tensor into the prior encoder to obtain the first multi-scale feature. Figure, wherein the prior encoder is trained according to the method described in the first aspect; Denoising module: used to acquire Gaussian noise as the initial PET image to be generated; feed the PET image to be generated and the first multi-scale feature map into the denoising U-shaped network, the denoising U-shaped network is based on the spatial adaptive modulation module, and uses the first multi-scale feature map to modulate the feature maps of the corresponding scale in the encoder and decoder of the denoising U-shaped network to predict the denoising result of the current time step; Iterative module: Updates the PET image to be generated based on the denoising result, and iteratively executes the process of generating the PET image. The process involves generating a PET image and a first multi-scale feature map, then passing the data to a denoising U-shaped network until a preset time step is reached, at which point the final target PET image is output. The denoising U-shaped network is trained according to the method described in the first aspect.

[0021] Fourthly, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect, or implements the method described in the second aspect.

[0022] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention extracts multi-scale prior features from CT images and lesion masks using a prior encoder, and symmetrically injects these features into the corresponding levels of the encoder and decoder of a U-shaped denoising network through a spatial adaptive modulation module. This achieves deep fusion and guidance of prior information throughout the generation process. This mechanism can adaptively adjust the distribution parameters of the generated features according to spatial location, enhancing the freedom and accuracy of metabolic generation in the lesion region and strengthening the constraint of anatomical structures in the background region. This overcomes the shortcomings of existing technologies, such as indiscriminate processing of lesion and background regions, information attenuation due to only shallow fusion of prior information, and lack of anatomical structure anchors in the generation process. Furthermore, this invention employs an iterative denoising framework, using the previous denoising result as input at each time step. Combined with the continuous constraint of multi-scale prior features, it effectively avoids the problems of loss of subtle anatomical topology, lesion position drift, or geometric deformation under the traditional blind generation paradigm, ultimately outputting high-fidelity anatomical structures and highly accurate PET images with metabolic quantification. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1This is a flowchart of a neural network model training method based on prior perception and spatial adaptive modulation according to Embodiment 1 of the present invention; Figure 2 This is a diffusion process diagram of Embodiment 1 of this aspect; Figure 3 This is a diagram of the denoising network architecture of Embodiment 1 of the present invention; Figure 4 This is a detailed diagram of the feature modulation module in Embodiment 1 of the present invention; Figure 5 This is a block diagram of a CT-to-PET cross-modal medical image generation system according to Embodiment 2 of the present invention. Detailed Implementation

[0025] To enable those skilled in the art to better understand the present invention, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention. It should be understood that the various steps described in the method embodiments of the present invention can be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present invention is not limited in this respect.

[0026] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0027] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0028] It should be noted that the terms "a" and "a plurality of" used in this invention are illustrative rather than restrictive. Those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0029] The names of the messages or information exchanged between the multiple devices in the embodiments of the present invention are for illustrative purposes only and are not intended to limit the scope of these messages or information.

[0030] Example 1 Please see Figure 1 A preferred embodiment of the present invention provides a neural network model training method based on prior perception and spatial adaptive modulation, the method comprising: S1: Construct a priori perceptual tensor based on the acquired CT images and the corresponding lesion masks; S2: Input the prior perception tensor into the prior encoder for feature extraction to obtain a multi-scale feature map; S3: Obtain the PET image corresponding to the CT image and preprocess it. Add noise to the preprocessed PET image based on the forward process of the diffusion model to obtain a noisy PET image. S4: Input the noisy PET image and multi-scale feature map of the current time step into the denoising U-shaped network based on spatial adaptive modulation, and use the multi-scale feature map to modulate the feature maps of the corresponding scale in the encoder and decoder of the denoising U-shaped network to predict the denoised PET image. S5: Calculate the total loss based on the PET image and the denoised PET image, and update the network parameters of the prior encoder and the denoised U-shaped network according to the total loss to complete the training.

[0031] This embodiment extracts multi-scale prior features from CT images and lesion masks using a prior encoder, and symmetrically injects these features into the corresponding levels of the encoder and decoder of the U-shaped denoising network through a spatial adaptive modulation module. This achieves deep fusion and guidance of prior information throughout the generation process. This mechanism can adaptively adjust the distribution parameters of the generated features according to spatial location, enhancing the freedom and accuracy of metabolic generation in the lesion area and strengthening the constraint of anatomical structures in the background area. This overcomes the shortcomings of existing technologies, such as indiscriminate processing of lesion and background areas, information attenuation due to only shallow fusion of prior information, and lack of anatomical structure anchors in the generation process. Furthermore, this invention adopts an iterative denoising framework, using the previous denoising result as input at each time step. Combined with the continuous constraint of multi-scale prior features, it effectively avoids the problems of loss of subtle anatomical topology, lesion position drift, or geometric deformation under the traditional blind generation paradigm, and finally outputs high-fidelity anatomical structures and highly accurate metabolic quantitative PET images.

[0032] In an optional embodiment, before step S1, it is necessary to obtain the raw data required for training, specifically, to obtain CT images from a medical imaging database. and the lesion mask corresponding to the CT image Where H is the image height, W is the image width, and D is the number of image channels, CT images provide anatomical information, and lesion masks indicate the precise spatial location of lesion areas; areas with a value of 1 represent lesions, and areas with a value of 0 represent the background. For ease of subsequent processing, all images are cropped or interpolated to a uniform spatial size. After obtaining the above data, it is necessary to analyze the CT images. and the corresponding lesion mask Constructing the prior-aware tensor includes: (1) The CT image is truncated by window width and window level, and the truncated CT values ​​are linearly normalized to obtain a standardized CT image; specifically, the truncation range [L] is set according to the window width and window level parameters of the CT image. min ,L max ] Pixel values ​​outside this range are truncated to the boundary values, and then a linear normalization formula is applied:

[0033] Map the pixel values ​​of the CT image to [ The standard CT image is obtained by using the interval [1,1] to eliminate numerical differences caused by different equipment and scanning parameters, stabilize the model training process, and obtain standardized CT images. .

[0034] (2). The standardized CT images The corresponding lesion mask Channel splicing is performed to obtain the prior perception tensor, represented as follows:

[0035] Prior perception tensor The number of channels is 2. This tensor integrates anatomical structure information and lesion spatial location information, and serves as the input of the subsequent prior encoder, providing structured prior guidance for the generation process.

[0036] This embodiment constructs a priori perception tensor that integrates anatomical structure and lesion spatial location by stitching CT images and lesion masks in the channel dimension. This provides structured conditional input for the subsequent encoder to extract multi-scale feature maps, enabling the model to clearly distinguish between lesion areas and background areas from an early stage, thereby achieving spatial differentiation guidance during the generation process.

[0037] In an optional embodiment, in step S2, the prior perceptual tensor is input to the prior encoder for feature extraction. The prior encoder consists of multiple cascaded convolutional residual modules. Through progressive downsampling, each stage extracts a feature map corresponding to the spatial resolution, thereby forming a multi-scale feature map (also known as a pyramid) containing details from high-resolution shallow layers to low-resolution deep semantics. In this embodiment, the prior encoder consists of five cascaded convolutional residual modules. It is understood that the number of convolutional residual modules can be selected according to specific circumstances, and this invention does not impose any limitation on this. Regarding the prior perceptual tensor obtained in step S1... After k-level downsampling, the first Level features The calculation process is expressed as follows:

[0038] in, And let . No. Level features The spatial resolution is The feature maps extracted at each level are combined to obtain a multi-scale feature map, represented as follows: .

[0039] This embodiment addresses the challenge of nonlinear mapping between CT and PET modalities by extracting multi-level anatomical prior information, from global organ contours to local tissue textures, through a feature pyramid. These prior features serve as spatial constraints, guiding the subsequent generative network to strictly limit high metabolic signals within the corresponding anatomical structures when synthesizing PET images. This effectively prevents edge overflow or anatomical misalignment of generated PET hotspots, improving the structural fidelity of cross-modal synthesis.

[0040] In an optional embodiment, please refer to Figure 2 Step S3 simulates the degradation process of data distribution based on Markov chains. Specifically, similar to acquiring CT images, PET images corresponding to the CT images in step S1 are obtained from the medical image database. ; then the PET images Perform maximum normalization to Processing is performed to eliminate numerical differences caused by different scanning parameters, thereby stabilizing model training and obtaining preprocessed PET images. Subsequently, noise is added to the preprocessed PET image using a forward process based on the diffusion model. The total diffusion time step is preset to T, and the noise variance sequence is defined. Then calculate the cumulative noise figure. :

[0041] Cumulative noise figure The cumulative multiplication form is: During the training phase, random sampling time steps are used. And sample standard Gaussian noise Noisy PET images can be directly calculated using reparameterization techniques:

[0042] Noisy PET image It serves as the input for the subsequent denoising U-shaped network.

[0043] This embodiment simulates the progressive degradation path from the real data distribution to the noise distribution through the forward noise addition process of the diffusion model, enabling the model to learn the inverse diffusion process at any time step, thereby improving the stability and diversity of the generation process. At the same time, the reparameterization technique is used to directly sample noisy images at any time, avoiding the computational overhead of step-by-step iteration and effectively improving training efficiency.

[0044] In an optional embodiment, for step S4, in a specific implementation, the noisy PET image at the current time step is... With the multi-scale prior feature map extracted in step S2 The inputs are fed into a spatially adaptive modulation-based denoising U-shaped network. This denoising U-shaped network employs an encoder-decoder symmetric architecture and incorporates skip connections to fuse features at different resolution levels. See the structure below. Figure 3 Specifically, in each layer of the encoder and decoder, multi-scale prior feature maps are used to modulate the backbone feature maps of the corresponding scale, so as to achieve depth guidance of the generation process by prior information on anatomical structure and lesions. The modulation process specifically includes the following sub-steps: (1). For the modulated feature map h of any network layer in the denoising U-shaped network, from the multi-scale prior feature map Obtain a prior feature map with the same spatial resolution as the feature map to be modulated. This ensures that the modulation operations are strictly aligned on a spatial scale.

[0045] (2) Perform group normalization on the feature map h to be modulated to obtain the standardized feature map. The normalization operation of the group includes the following steps:

[0046]

[0047]

[0048] in, For the input feature map at the th Each channel, spatial coordinates The original activation value at that location; and Channels Mean and standard deviation of the group; It is a set that covers three spatial dimensions: depth, height, and width, and also includes passageways. All other channels belonging to the same group, i.e. Included in this group All voxel points in each channel; To standardize the feature map, group normalization divides the channels of the feature map into several groups, calculates the mean and standard deviation within each group, and standardizes the data accordingly to eliminate covariate bias and make the feature distribution more stable.

[0049] (3) Based on the acquired prior feature maps Spatially adaptive scaling factors are generated through two independent 1×1 convolutional layers. and bias coefficient This process can be represented as:

[0050]

[0051] SiLU is an activation function used to enhance nonlinear expressive power.

[0052] (4) Utilize the generated scaling factor and bias coefficient Standardized feature maps A pixel-by-pixel affine transformation is performed to obtain the modulated feature map. :

[0053] The modulated feature map will be used as the input to the next network layer, or it will be upsampled and fused with the corresponding layer features of the encoder. Finally, it will be mapped to the denoised PET image through the output layer of the denoising U-shaped network. See details Figure 4 denoised PET image It is expressed as follows: .

[0054] This step employs a spatial adaptive modulation mechanism to inject multi-scale prior features into each layer of the denoising network using pixel-level affine transformations. This allows the model to dynamically adjust its feature responses based on spatial location during generation: enhancing the freedom and accuracy of metabolic generation in lesion regions and strengthening the constraints of anatomical structures in background regions. Compared to traditional channel splicing or simple conditional input methods, this mechanism effectively overcomes the problem of semantic information attenuation in deep networks, ensuring significant improvements in anatomical boundary alignment, lesion edge clarity, and metabolic intensity accuracy in the generated PET images.

[0055] In an optional embodiment, in step S5, the total loss is calculated based on the PET image and the denoised PET image, and the network parameters of the prior encoder and the denoised U-shaped network are updated according to the total loss to complete model training. Specifically, the total loss consists of two parts: reconstruction loss and local metabolic consistency loss, which achieves comprehensive constraints on the model's generation capability through multi-objective joint optimization.

[0056] (1). The reconstruction loss adopts the L1 norm to constrain the denoised PET image. With real PET images The difference in overall structure can be expressed as follows:

[0057] This loss enables the model to maintain the integrity of the anatomical structure and the overall fidelity of the image globally.

[0058] (2) The local metabolic consistency loss is specifically designed to constrain the area covered by the lesion mask M, ensuring that the metabolic intensity of the lesion area remains highly consistent with the real PET image. This loss consists of two parts: first, the overall mean error of the lesion area, constrained by the L1 norm to limit pixel differences within the area; and second, the peak error of the lesion area, which is achieved by extracting the average value of the top K% of pixels with the highest values ​​within the area using a TopK operation, thus constraining the metabolic intensity of the peak area. Its expression is as follows:

[0059] The TopK operation returns the average value of the top K% of pixels with the largest values ​​within the region. For Hadama accumulation, For the initial PET image, The denoised PET image. For pre-set weight hyperparameters, The lesion area.

[0060] (3) The final total loss function is expressed as a weighted sum of the two losses mentioned above:

[0061] in, The weight hyperparameters are pre-set to control the proportion of the two types of losses. During training, the gradient of the total loss with respect to each network parameter is calculated using the backpropagation algorithm, and the weight parameters of the prior encoder and the denoising U-shaped network are iteratively updated using an optimizer (such as Adam) until the model converges.

[0062] This embodiment employs a multi-objective joint optimization approach, simultaneously imposing constraints on both the image space and the lesion region. This enables the model to not only generate PET images with realistic overall structure but also to accurately capture the metabolic intensity distribution within the lesion region. Specifically, the reconstruction loss ensures the consistency of the global anatomical structure, while the local metabolic consistency loss specifically targets and strengthens constraints on lesion regions of clinical interest. In particular, the TopK peak error explicitly guides the model to focus on the regions with the strongest metabolic activity, thereby ensuring that the generated PET images are highly consistent with real images in key clinical indicators such as standard uptake values, and possess reliable quantitative analysis capabilities.

[0063] In a specific implementation, once the model training is complete, the trained prior encoder and denoising U-shaped network can be used to generate cross-modal medical images. Specifically, a method for generating CT-to-PET cross-modal medical images includes the following steps: S100: Construct the target prior perception tensor based on the acquired target CT image and the corresponding lesion mask; S200: Input the target prior perception tensor into the prior encoder to obtain the first multi-scale feature map, wherein... The prior encoder is trained according to the method described above; S300: Obtain Gaussian noise as the initial PET image to be generated; feed the PET image to be generated and the first multi-scale feature map into a denoising U-shaped network. The denoising U-shaped network is based on a spatial adaptive modulation module. It uses the first multi-scale feature map to modulate the feature maps of the corresponding scales in the encoder and decoder of the denoising U-shaped network to predict the denoising result of the current time step. S400: Update the PET image to be generated based on the denoising result, and iteratively execute the step of updating the PET image to be generated. The process of generating the image and the first multi-scale feature map into a denoising U-shaped network continues until a preset time step is reached, at which point the final target PET image is output. The denoising U-shaped network is trained according to the method described above.

[0064] In an optional embodiment, steps S100 and S200 are the same as S1 and S2, and will not be repeated here to avoid repetition. For S300 and S400, in specific implementation, a noise tensor is randomly sampled from a standard Gaussian distribution as the initial PET image to be generated, denoted as... Starting from time step t=T, an iterative denoising process is executed. For the current time step t, the PET image x to be generated is... t The current time step t and the first multi-scale feature map are input into the trained denoising U-shaped network. This denoising U-shaped network adopts an encoder-decoder symmetric architecture, and in each layer, a spatially adaptive modulation module generates spatially adaptive scaling and bias coefficients based on a prior feature map with the same resolution as the current layer. It then performs a pixel-wise affine transformation on the feature map to be modulated after group normalization, thereby achieving depth modulation of the corresponding scale feature maps in the encoder and decoder, and finally predicting the denoising result for the current time step. .

[0065] Next, based on predictions and x at the current moment t Based on the posterior distribution calculation formula of the diffusion model, the state x at the previous time step is derived. t 1. Specifically, calculate the mean of the posterior distribution. and variance And sampled to obtain x t 1:

[0066]

[0067]

[0068]

[0069] x t 1 is taken as the new PET image to be generated, and the above denoising steps are continued, recursively until t=0. The final output x0 is the generated target PET image. Throughout the iteration process, since both the prior encoder and the denoising U-shaped network use the optimal parameters obtained during the training phase, and the spatial adaptive modulation mechanism continuously provides depth guidance for the prior anatomical structure and lesion, the generated PET images maintain high quality in terms of anatomical structure alignment, lesion edge clarity, and metabolic intensity accuracy, and can be directly used for clinical auxiliary diagnosis.

[0070] Example 2 Please see Figure 5An embodiment of the present invention provides a CT-to-PET cross-modal medical image generation system, comprising: Construction module: used to construct the target prior perception tensor based on the acquired target CT image and the corresponding lesion mask; Acquisition module: used to input the target prior perception tensor into the prior encoder to obtain the first multi-scale feature. Figure, wherein the prior encoder is trained according to the method described in the first aspect; Denoising module: used to acquire Gaussian noise as the initial PET image to be generated; feed the PET image to be generated and the first multi-scale feature map into the denoising U-shaped network, the denoising U-shaped network is based on the spatial adaptive modulation module, and uses the first multi-scale feature map to modulate the feature maps of the corresponding scale in the encoder and decoder of the denoising U-shaped network to predict the denoising result of the current time step; Iterative module: Updates the PET image to be generated based on the denoising result, and iteratively executes the process of generating the PET image. The process involves generating a PET image and a first multi-scale feature map, then passing the data to a denoising U-shaped network until a preset time step is reached, at which point the final target PET image is output. The denoising U-shaped network is trained according to the method described in the first aspect.

[0071] The system proposed in this embodiment is based on the method proposed in Embodiment 1. Therefore, the options proposed in Embodiment 1 are also applicable to this embodiment. To avoid repetition, they will not be described again here.

[0072] Furthermore, embodiments of the present invention also provide a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method described in Embodiment 1.

[0073] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0074] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0075] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0076] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0077] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0078] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for training a neural network model based on prior perception and spatial adaptive modulation, characterized in that, The method includes the following steps: Based on the acquired CT images and the corresponding lesion masks, a priori perception tensor is constructed. The prior perception tensor is input into the prior encoder for feature extraction to obtain a multi-scale feature map. The PET image corresponding to the CT image is acquired and preprocessed. Noise is added to the preprocessed PET image based on the forward process of the diffusion model to obtain a noisy PET image. The noisy PET image and multi-scale feature map at the current time step are input into a spatially adaptive modulation-based denoising U-shaped network. The multi-scale feature map is used to modulate the feature maps of the corresponding scales in the encoder and decoder of the denoising U-shaped network to predict the denoised PET image. The total loss is calculated based on the PET image and the denoised PET image, and the network parameters of the prior encoder and the denoised U-shaped network are updated according to the total loss to complete the training.

2. The method according to claim 1, characterized in that, The construction of the prior perception tensor based on CT image sequences and corresponding lesion masks specifically includes: The CT image is truncated by window width and window level, and the truncated CT values ​​are linearly normalized to obtain a standardized CT image. The standardized CT image is concatenated with the corresponding lesion mask to obtain the prior perception tensor.

3. The method according to claim 1, characterized in that, The prior encoder contains multiple cascaded convolutional residual modules, which extract features at different spatial resolutions through step-by-step downsampling to obtain the multi-scale feature map.

4. The method according to claim 1, characterized in that, The modulation of feature maps at corresponding scales in the encoder and decoder of the denoising U-shaped network using the multi-scale feature maps specifically includes: (1). For the feature map to be modulated in any network layer of the denoising U-shaped network, obtain a prior feature map with the same resolution as the feature map to be modulated in the multi-scale feature map; (2). Perform group normalization on the feature map to be modulated to obtain a standardized feature map; (3). Based on the prior feature map, spatially adaptive scaling and bias coefficients are generated through convolutional layers; (4) Perform a pixel-by-pixel affine transformation on the normalized feature map using the scaling factor and the bias factor to obtain a modulated feature map, which is then processed by the subsequent layers of the denoising U-shaped network to finally generate the denoised PET image.

5. The method according to claim 4, characterized in that, The group normalization operation specifically Includes the following processes: in, For the input feature map at the th Each channel, spatial coordinates The original activation value at that location; and Channels Mean and standard deviation of the group; It is a set that covers three spatial dimensions: depth, height, and width, and also includes passageways. All other channels belonging to the same group, i.e. Included in this group All voxel points in each channel; This is a standardized feature map.

6. The method according to claim 4, characterized in that, The total loss includes reconstruction loss and local metabolic consistency loss. The reconstruction loss is used to constrain the difference in overall structure between the denoised PET image and the PET image. The local metabolic consistency loss includes mean error and peak error in the region corresponding to the lesion mask, which is used to constrain the difference in metabolic intensity between the denoised PET image and the PET image in the region corresponding to the lesion mask.

7. The method according to claim 6, characterized in that, The expression for the total loss is as follows: in, The weight hyperparameters are preset to control the proportion of the two types of losses. The reconstruction loss is expressed as follows: The expression for the loss of local metabolic consistency is as follows: The TopK operation returns the average value of the top K% of pixels with the largest values ​​within the region. For Hadama accumulation, For the initial PET image, The denoised PET image. For pre-set weight hyperparameters, The lesion area.

8. A method for generating cross-modal medical images from CT to PET, characterized in that, The method includes the following steps: Based on the acquired target CT image and the corresponding lesion mask, a target prior perception tensor is constructed. The target prior perception tensor is input into the prior encoder to obtain the first multi-scale feature map, wherein the... The prior encoder is trained by the method according to any one of claims 1-7; Gaussian noise is obtained as the initial PET image to be generated; the PET image to be generated and the first multi-scale feature map are fed into a denoising U-shaped network. The denoising U-shaped network is based on a spatial adaptive modulation module. The first multi-scale feature map is used to modulate the feature maps of the corresponding scale in the encoder and decoder of the denoising U-shaped network to predict the denoising result of the current time step. The denoising result is used to update the PET image to be generated, and the process of iteratively generating the PET image to be generated is performed. The process of generating an image and a first multi-scale feature map into a denoising U-shaped network continues until a preset time step is reached, at which point the final target PET image is output, wherein the denoising U-shaped network is trained using the method described in any one of claims 1-7.

9. A CT-to-PET cross-modal medical image generation system, characterized in that, The system is based on the method of claim 8, comprising: Construction module: Used to construct a target prior perception template based on the acquired target CT image and the corresponding lesion mask. quantity; Acquisition module: used to input the target prior perception tensor into the prior encoder to obtain the first multi-scale feature. Figure, wherein the prior encoder is trained by the method according to any one of claims 1-7; Denoising module: used to acquire Gaussian noise as the initial PET image to be generated; feed the PET image to be generated and the first multi-scale feature map into the denoising U-shaped network, the denoising U-shaped network is based on the spatial adaptive modulation module, and uses the first multi-scale feature map to modulate the feature maps of the corresponding scale in the encoder and decoder of the denoising U-shaped network to predict the denoising result of the current time step; Iterative module: Updates the PET image to be generated based on the denoising result, and iteratively executes the process of changing the denoising result... The process of generating a PET image and a first multi-scale feature map to a denoising U-shaped network continues until a preset time step is reached, at which point the final target PET image is output. The denoising U-shaped network is trained according to the method described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-7, or implements the method as described in claim 8.

Images