An organ registration system for an integrated PET / MR based on deep learning

Abstract

The present invention discloses an organ registration system for an integrated PET / MR based on deep learning, which includes an image acquisition module, an image denoising module, an image reconstruction module, and an organ registration module. The present invention belongs to the technical field of image processing, and specifically refers to an organ registration system for an integrated PET / MR based on deep learning. In this solution, low-dose PET / MR images with different noise intensities are defined, a noise adaptability module is introduced to improve the quality of low-dose PET / MR images, and an adaptive texture matching loss is introduced to construct a strongly adaptable denoising module to ensure the accuracy and efficiency of organ registration; combined with adaptive fusion weights, multi-scale feature fusion is performed at different levels to optimize local details; a hierarchical neural structure search is introduced to improve the registration accuracy of the registration system for different types of PET / MR images; the reconstruction structure is optimized based on the proximity contrast loss to improve the subsequent registration efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and specifically refers to an organ registration system for an integrated PET / MR based on deep learning. Background Art

[0002] The organ registration system of PET / MR is a technical system for spatially aligning PET images and MR images. The purpose of this system is to accurately align images of different modalities in space, so as to obtain more accurate anatomical and functional information. However, in general, the PET / MR organ registration system has problems such as improper processing of low-dose PET / MR images with different noise intensities, reducing the registration accuracy, poor adaptability to PET / MR images with complex noise patterns, which in turn leads to loss of image structure and poor organ registration effect; there are problems of registration errors caused by resolution differences and low registration accuracy caused by loss of image details in general PET / MR organ registration systems. Summary of the Invention

[0003] In view of the above situation, in order to overcome the defects of the prior art, the present invention provides an organ registration system for an integrated PET / MR based on deep learning. Aiming at the problems that the general PET / MR organ registration system has improper processing of low-dose PET / MR images with different noise intensities, reducing the registration accuracy, poor adaptability to PET / MR images with complex noise patterns, which in turn leads to loss of image structure and poor organ registration effect, this solution defines low-dose PET / MR images with different noise intensities, introduces a noise adaptability module, stores the high-level hidden features of PET / MR images with different intensities of noise, improves the quality of low-dose PET / MR images, reduces the influence of noise on important structural regions, and introduces an adaptive texture matching loss to construct a denoising module with strong adaptability to ensure the accuracy and efficiency of organ registration; aiming at the problems that the general PET / MR organ registration system has registration errors caused by resolution differences and low registration accuracy caused by loss of image details, this solution processes the global information and local details in PET / MR images based on a global-local design unit, combines adaptive fusion weights, performs multi-scale feature fusion at different levels to optimize local details; introduces hierarchical neural structure search to improve the registration accuracy of the registration system for different types of PET / MR images; optimizes the reconstruction structure based on proximity contrast loss to improve the subsequent registration efficiency.

[0004] The technical solution adopted by the present invention is as follows: An organ registration system for an integrated PET / MR based on deep learning provided by the present invention includes an image acquisition module, an image denoising module, an image reconstruction module, and an organ registration module;

[0005] The image acquisition module acquires historical PET / MR organ registration image data;

[0006] The image denoising module optimizes the denoising effect of low-dose PET / MR images by designing noise modeling, an adaptive memory unit, a generative adversarial network, and multiple loss functions, combining an encoder, a decoder, and a self-attention mechanism, ensuring the convergence of losses on the training set and the test set, and finally achieving high-quality image reconstruction;

[0007] The image reconstruction module performs denoising and reconstruction of low-dose PET / MR images by using a globally-local designed generator network, combining Transformer and CNN modules, and generates high-quality images through multi-scale fusion, hierarchical neural structure search, and perceptual loss, ensuring the convergence of losses in the training and test sets and completing the image reconstruction task;

[0008] The organ registration module learns and predicts the deformation field by combining a variational autoencoder and a U-Net network, realizing the precise spatial alignment of PET and MR images and ensuring the efficient registration of the two-modal images.

[0009] Furthermore, in the image acquisition module, the historical PET / MR organ registration image data includes PET images at different dose levels and MR images of different modalities; the images are structurally annotated; and the collected data is divided into a test set and a training set.

[0010] Furthermore, the image denoising module specifically includes the following:

[0011] A noise modeling unit; defining low-dose PET / MR images with different noise intensities, expressed as: ; the relationship between the conventional-dose PET / MR image and the low-dose PET / MR image is expressed as: ; based on the of the image and the division threshold, the low-dose PET / MR image is divided into high-noise and low-noise ; generating a denoised image of the conventional-dose PET / MR image based on the mapping, expressed as: ; where, P i is the signal intensity at the i-th pixel position, and i is the pixel position index; is the initial value of the signal intensity; T i is the attenuation coefficient; E i is the background noise; N{·} is a normal distribution random process with zero mean and unit variance; I LD is the low-dose PET / MR image; B(·) is the back-projection operation; T ND is the conventional-dose PET / MR projection image; T N is the noise term; G(·) is the mapping function for generating the denoised image; I NDIs a conventional-dose PET / MR image;

[0012] Network architecture design unit; The main network structure of the image denoising module includes an encoder, a noise adaption unit, a decoder, and a discriminator; The encoder and decoder are each composed of multiple BLOGS modules, and skip connections are introduced into the network to prevent gradient vanishing; The GAN discriminator uses a classic Patch-GAN structure and downsamples through convolutional layers to penalize structural mismatches at the patch level and improve the denoising effect; The BLOGS module includes multiple 1x1 convolutional layers, a 3x3 convolutional layer, a GELU activation layer, and a simple self-attention module;

[0013] Noise adaption unit design; To store high-level hidden features of different-intensity noises, a noise adaption unit is proposed, which includes memory terms K h and K l , denoted as: ; Update the memory terms, denoted as: ; Adopt a soft attention reading strategy for noise decoding, denoted as: ; Among them, is the memory term index that best matches the input features; k hi1 and q hj1 are the memory terms and query vectors before update, respectively; is the memory term after update; τ is the update coefficient; is an indicator function indicating whether the current index matches; is feature reconstruction; n1 is the total number of stored memory terms; i1 and j1 are memory term indices; is the similarity between memory terms, obtained based on cosine similarity; T is the transpose operation;

[0014] Loss function design unit; Design a hybrid adversarial loss , denoted as: ; Design a smooth visual loss , denoted as: ; ; Design a multi-scale perception loss , denoted as: ; Design an adaptive texture matching loss , denoted as: ; Obtain the final denoising loss , denoted as: ; Among them, D(·) is the discriminator; G(·) is the generator; and are adversarial weight parameters; E is the expectation operator; is the smooth function, and x is the function variable; is the i2-th low-dose PET / MR image sample; is the weight of the i2-th scale; n2 is the total scale; is the feature extraction function; is the square of the Frobenius norm; is the weight of the i2-th feature layer; is the Gram matrix operation; 、 and are the balanced loss weights;

[0015] The denoising module is determined; when the loss of the denoising module converges for the training set and meets the expected effect for the test set, the denoising module is completed.

[0016] Furthermore, the image reconstruction module specifically includes the following:

[0017] Global-local design unit; The image reconstruction module uses a generator network, which contains multiple reconstruction blocks, and each reconstruction block is composed of a global Transformer module and a local CNN module. The input of the generator is the denoised low-dose PET / MR image, and the output is the reconstructed image; The global Transformer module is expressed as: ; The local CNN module is expressed as: ; ; Among them, is the global Transformer module operation; Q, K, and V are the input query matrix, key matrix, and value matrix respectively, obtained by performing feature mapping on the denoised low-dose PET / MR image; D is the feature dimension; P i3 and P j3 are the computational nodes in the local CNN module and are the feature representations of the PET / MR image; is the pooling operation between nodes; C is the output of the local CNN module; Concat(·) is the concatenation operation; P1, P2, and P3 are the output features of the three computational nodes in the local CNN module; i3 and j3 are the node indices;

[0018] Hierarchical neural structure search unit; In the high-level search stage, determine the distribution of the global Transformer module and the local CNN module, introduce adaptive fusion weights, and obtain the output of the reconstruction block through weighted combination; In the low-level search stage, introduce multi-scale fusion operations to optimize the internal structures of the global Transformer module and the local CNN module respectively; The high-level search is expressed as: ; ; ; ; ; The low-level search is expressed as: ; Among them, and are the outputs of the i4-th reconstructed block and the (i4 - 1)-th reconstructed block respectively; and are the adaptive fusion weights of the global Transformer module and the local CNN module respectively; and are the parameter matrices of the global Transformer module and the local CNN module respectively; exp(·) is the exponential function; and are the feature statistics; represent the mean calculations of the global feature and the local feature respectively; is the feature output of the multi-scale fusion; S1 is the set of features at different scales; is the operation of feature extraction at scale s;

[0019] Reconstruction loss function design unit; introducing the perceptual loss and the proximity contrast loss, the perceptual loss is expressed as: ; the proximity contrast loss is expressed as: ; obtaining the reconstruction loss function is expressed as: ; where, α and β are the weights of the perceptual loss and the proximity contrast loss; is the weight of the feature layer perceptual loss, and l is the feature layer index; is the feature extraction function, which extracts features at the l-th layer based on the pre-trained network; SZF is the undersampled PET / MR image; is the square of the L2 norm; SGT u and SGT r are the pixel values at the u-th position and the r-th position of the target image respectively, and the target image is a high-quality PET / MR image; and are the reconstruction weights; and are the pixel values at the u-th position and the r-th position of the reconstructed image respectively;

[0020] Reconstruction module determination unit; pre-dividing the denoised low-dose PET / MR image into a reconstruction training set and a reconstruction test set; when the reconstruction loss converges to the reconstruction training set loss and achieves the expected effect on the reconstruction test set, the reconstruction module is completed.

[0021] Further, the input of the organ registration module is the PET / MR organ image to be registered that is collected in real time and processed by the image denoising module and the image reconstruction module; the organ registration module combines a variational autoencoder and a U-Net network architecture to achieve spatial alignment between images by predicting the deformation field; the deformation field captures the spatial differences between the PET image and the MR image and is then applied to the registration of the two images. First, the VAE is used to learn the potential deformation features from the input PET and MR images, and then the U-Net network serves as the backbone network of the organ registration module, responsible for accurately predicting the deformation field. The skip connections of the U-Net can effectively prevent the vanishing gradient and retain the detailed information of the image, ensuring that the generated deformation field can accurately represent the spatial mapping between the PET image and the MR image; the generated deformation field represents the spatial transformation of the PET image relative to the MR image, and the pixel points of the PET image are mapped to the space of the MR image through the bilinear interpolation method to achieve the accurate alignment of the two-modal images.

[0022] The beneficial effects achieved by the present invention using the above solutions are as follows:

[0023] (1) Aiming at the problems existing in the general PET / MR organ registration system, such as improper processing of low-dose PET / MR images with different noise intensities, reducing the registration accuracy, poor adaptability to PET / MR images with complex noise patterns, resulting in the loss of image structure and poor organ registration effect. This solution defines low-dose PET / MR images with different noise intensities, introduces a noise adaptation unit to store the high-level hidden features of PET / MR images with different intensities of noise, improves the quality of low-dose PET / MR images, reduces the influence of noise on important structural regions, and introduces an adaptive texture matching loss to construct a highly adaptable denoising module to ensure the accuracy and efficiency of organ registration.

[0024] (2) Aiming at the problems existing in the general PET / MR organ registration system, such as registration errors caused by resolution differences and low registration accuracy caused by the loss of image details. This solution processes the global information and local details in the PET / MR image based on the global-local design unit, combines the adaptive fusion weight, performs multi-scale feature fusion at different levels to optimize the local details; introduces hierarchical neural architecture search to improve the registration accuracy of the registration system for different types of PET / MR images; optimizes the reconstruction structure based on the proximity contrast loss to improve the subsequent registration efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 is a schematic flowchart of an organ registration system for an integrated PET / MR based on deep learning provided by the present invention;

[0026] Figure 2 is a schematic flowchart of the image denoising module;

[0027] Figure 3 It is a schematic diagram of the process of image reconstruction.

[0028] The accompanying drawings are used to provide a further understanding of the present invention, and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation to the present invention. Specific embodiments

[0029] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts fall within the scope of protection of the present invention.

[0030] In the description of the present invention, it should be understood that the terms "upper", "lower", "front", "rear", "left", "right", "top", "bottom", "inner", "outer", etc. indicating the orientation or positional relationship are based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation to the present invention.

[0031] Embodiment 1. Refer to Figure 1 , an organ registration system for an integrated PET / MR based on deep learning provided by the present invention, includes an image acquisition module, an image denoising module, an image reconstruction module, and an organ registration module;

[0032] The image acquisition module acquires historical PET / MR organ registration image data; and sends the data to the image denoising module;

[0033] The image denoising module receives the data sent by the image acquisition module; by designing noise modeling, an adaptive memory unit, a generative adversarial network, and multiple loss functions, combining an encoder, a decoder, and a self-attention mechanism, optimizes the denoising effect of low-dose PET / MR images, ensures the convergence of losses on the training set and the test set, and finally realizes high-quality image reconstruction; and sends the data to the image reconstruction module;

[0034] The image reconstruction module receives the data sent by the image denoising module; by using a globally-local designed generator network, combining Transformer and CNN modules, performs denoising and reconstruction of low-dose PET / MR images, and generates high-quality images through multi-scale fusion, hierarchical neural architecture search, and perceptual loss, and ensures the convergence of losses on the training and test sets to complete the image reconstruction task; and sends the data to the organ registration module;

[0035] The organ registration module receives the data sent by the image reconstruction module; by combining the variational autoencoder and the U-Net network, it learns and predicts the deformation field to achieve the precise spatial alignment of PET and MR images, ensuring the efficient registration of the two-modal images.

[0036] Example 2, refer to Figure 1 , this example is based on the above example. In the image acquisition module, the historical PET / MR organ registration image data includes PET images at different dose levels and MR images of different modalities; and the images are structurally annotated; the collected data is divided into a test set and a training set; the structural annotation specifically refers to contour annotation of each organ in the PET / MR images, indicating the position and shape of each organ.

[0037] Example 3, refer to Figure 1 and Figure 2 , this example is based on the above example. The image denoising module specifically includes the following:

[0038] Noise modeling unit; define low-dose PET / MR images with different noise intensities, expressed as: ; the relationship between the conventional-dose PET / MR image and the low-dose PET / MR image is expressed as: ; based on the of the image and the division threshold, divide the low-dose PET / MR image into high-noise and low-noise ; when the of the pixels in the PET / MR image is higher than the division threshold, divide the PET / MR image into high-noise; generate a denoised image of the conventional-dose PET / MR image based on the mapping, expressed as: ; where, P i is the signal intensity at the i-th pixel position, and i is the pixel position index; is the initial value of the signal intensity; T i is the attenuation coefficient; E i is the background noise; N{·} is a normal distribution random process with zero mean and unit variance; I LD is the low-dose PET / MR image; B(·) is the back-projection operation; T ND is the conventional-dose PET / MR projection image; T N is the noise term; G(·) is the mapping function for generating the denoised image; I ND is the conventional-dose PET / MR image;

[0039] Network architecture design unit; The main network structure of the image denoising module includes an encoder, a noise adaptability unit, a decoder, and a discriminator; The encoder and decoder are each composed of multiple BLOGS modules, and skip connections are introduced into the network to prevent gradient vanishing; The GAN discriminator uses a classical Patch-GAN structure and downsamples through convolutional layers to penalize structural mismatches at the patch level and improve the denoising effect; The BLOGS module includes multiple 1x1 convolutional layers, a 3x3 convolutional layer, a GELU activation layer, and a simple self-attention module;

[0040] Noise adaptability unit design; To store high-level hidden features of different intensities of noise, a noise adaptability unit is proposed, which includes memory terms K h and K l , expressed as: ; Update the memory term, expressed as: ; Adopt a soft attention reading strategy for noise decoding, expressed as: ; Among them, is the memory term index that best matches the input feature; k hi1 and q hj1 are the memory term and query vector before update, respectively; is the memory term after update; τ is the update coefficient; is an indicator function indicating whether the current index matches; is the feature reconstruction; n1 is the total number of stored memory terms; i1 and j1 are memory term indices; is the similarity between memory terms, obtained based on cosine similarity; T is the transpose operation;

[0041] Loss function design unit; Design a hybrid adversarial loss , expressed as: ; Design a smooth visual loss , expressed as: ; ; Design a multi-scale perception loss , expressed as: ; Design an adaptive texture matching loss , expressed as: ; Obtain the final denoising loss , expressed as: ; Among them, D(·) is the discriminator; G(·) is the generator; and are adversarial weight parameters; E is the expectation operator; is the smoothing function, and x is the function variable; is the i2-th low-dose PET / MR image sample; is the weight of the i2-th scale; n2 is the total number of scales; is the feature extraction function; is the square of the Frobenius norm; is the weight of the i2-th feature layer; is the Gram matrix operation; 、 and are the balanced loss weights;

[0042] The denoising module is determined; when the loss of the denoising module converges for the training set and meets the expected effect for the test set, the denoising module is completed.

[0043] By performing the above operations, for the general PET / MR organ registration system, there are problems such as improper processing of low-dose PET / MR images with different noise intensities, reducing the registration accuracy, poor adaptability to PET / MR images with complex noise patterns, and thus resulting in loss of image structure and poor organ registration effect. In this solution, by defining low-dose PET / MR images with different noise intensities, introducing a noise adaptation unit, storing the high-level hidden features of PET / MR images with different intensities of noise, improving the quality of low-dose PET / MR images, reducing the influence of noise on important structural regions, introducing an adaptive texture matching loss to construct a highly adaptable denoising module, ensuring the accuracy and efficiency of organ registration.

[0044] Example 4, refer to Figure 1 and Figure 3 , based on the above example, the image reconstruction module specifically includes the following:

[0045] Global-local design unit; the image reconstruction module uses a generator network, which contains multiple reconstruction blocks, and each reconstruction block is composed of a global Transformer module and a local CNN module. The input of the generator is the denoised low-dose PET / MR image, and the output is the reconstructed image; the global Transformer module is expressed as: ; the local CNN module is expressed as: ; ; where, is the global Transformer module operation; Q, K, and V are the input query matrix, key matrix, and value matrix respectively, obtained by performing feature mapping on the denoised low-dose PET / MR image; D is the feature dimension; P i3 and P j3 are the calculation nodes in the local CNN module and are the feature representations of the PET / MR image; is the pooling operation between nodes; C is the output of the local CNN module; Concat(·) is the connection operation; P1, P2, and P3 are the output features of the three calculation nodes in the local CNN module; i3 and j3 are the node indices;

[0046] Hierarchical neural structure search unit; in the high-level search stage, determine the distribution of the global Transformer module and the local CNN module, introduce adaptive fusion weights, and obtain the output of the reconstruction block through weighted combination; in the low-level search stage, introduce multi-scale fusion operations to optimize the internal structures of the global Transformer module and the local CNN module respectively; the high-level search is expressed as: ; ; ; ; ; the low-level search is expressed as: ; where, and are the outputs of the i4-th reconstruction block and the (i4 - 1)-th reconstruction block respectively; and are the adaptive fusion weights of the global Transformer module and the local CNN module respectively; and are the parameter matrices of the global Transformer module and the local CNN module respectively; exp(·) is the exponential function; and are feature statistics; represent the mean calculations of the global feature and the local feature respectively; is the feature output of multi-scale fusion; S1 is the set of features at different scales; is the operation of feature extraction at scale s;

[0047] Reconstruction loss function design unit; introduce perceptual loss and proximity contrast loss, and the perceptual loss is expressed as: ; the proximity contrast loss is expressed as: ; obtain the reconstruction loss function which is expressed as: ; where, α and β are the weights of the perceptual loss and the proximity contrast loss; is the weight of the feature layer perceptual loss, and l is the feature layer index; is the feature extraction function, which extracts features at the l-th layer based on the pre-trained network; SZF is the undersampled PET / MR image; is the square of the L2 norm; SGT u and SGT r are the pixel values at the u-th position and the r-th position of the target image respectively, and the target image is a high-quality PET / MR image; and are the reconstruction weights; and They are the pixel values of the reconstructed image at the u-th position and the r-th position, respectively.

[0048] The reconstruction module determination unit; the denoised low-dose PET / MR image is pre-divided into a reconstruction training set and a reconstruction test set; when the reconstruction loss converges to the reconstruction training set loss and reaches the expected effect for the reconstruction test set, the reconstruction module is completed.

[0049] By performing the above operations, aiming at the problems of registration error caused by resolution differences and low registration accuracy caused by loss of image details in the general PET / MR organ registration system, this solution processes the global information and local details in the PET / MR image based on the global-local design unit, combines the adaptive fusion weight, performs multi-scale feature fusion at different levels to optimize the local details; introduces hierarchical neural architecture search to improve the registration accuracy of the registration system for different types of PET / MR images; optimizes the reconstruction structure based on the proximity contrast loss to improve the subsequent registration efficiency.

[0050] Example Five, refer to Figure 1 , this example is based on the above example. The input of the organ registration module is the PET / MR organ image to be registered that is collected in real time and processed by the image denoising module and the image reconstruction module; the organ registration module adopts the combination of the variational autoencoder and the U-Net network architecture to achieve spatial alignment between images by predicting the deformation field; the deformation field captures the spatial differences between the PET image and the MR image and is then applied to the registration of the two images. First, the VAE is used to learn the potential deformation features from the input PET and MR images, and then the U-Net network serves as the backbone network of the organ registration module, responsible for accurately predicting the deformation field. The skip connections of the U-Net can effectively prevent the vanishing gradient and retain the detailed information of the image, ensuring that the generated deformation field can accurately represent the spatial mapping between the PET image and the MR image; the generated deformation field represents the spatial transformation of the PET image relative to the MR image, and the pixel points of the PET image are mapped to the space of the MR image through the bilinear interpolation method to achieve the accurate alignment of the two modal images.

[0051] It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or device.

[0052] Although embodiments of the present invention have been shown and described, those of ordinary skill in the art will appreciate that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention.

[0053] The present invention and its embodiments have been described above. Such description is not restrictive. What is shown in the drawings is only one of the embodiments of the present invention, and the actual structure is not limited thereto. In summary, if those of ordinary skill in the art are inspired by it and, without departing from the purpose of the present invention, design similar structural modes and embodiments to this technical solution without creative efforts, they shall fall within the protection scope of the present invention.

Claims

1. An integrated PET / MR organ registration system based on deep learning, characterized by: The system includes an image acquisition module, an image denoising module, an image reconstruction module and an organ registration module; The image acquisition module acquires historical PET / MR organ registration image data; The image denoising module optimizes the denoising effect of low-dose PET / MR images by designing noise modeling, adaptive memory units, generative adversarial networks and multiple loss functions, combining encoders, decoders and self-attention mechanisms, ensuring loss convergence on training sets and test sets, and ultimately achieving high-quality image reconstruction; The image reconstruction module uses a globally-locally designed generator network combined with Transformer and CNN modules to denoise and reconstruct low-dose PET / MR images, and generates high-quality images through multi-scale fusion, hierarchical neural structure search and perceptual loss, and ensures the loss convergence of training and test sets to complete the image reconstruction task; The organ registration module combines the variational autoencoder and the U-Net network to learn and predict the deformation field, achieve accurate spatial alignment of PET and MR images, and ensure efficient registration of the two modality images; The image denoising module includes a noise modeling unit, which defines low-dose PET / MR images with different noise intensities, expressed as: ; The relationship between conventional-dose PET / MR images and low-dose PET / MR images is expressed as: ; Image-based and the segmentation threshold to classify low-dose PET / MR images into high-noise and low noise ; Based on the mapping, the denoised image of the conventional dose PET / MR image is generated, which is expressed as: ; Among them, P i is the signal strength at the i-th pixel position, i is the pixel position index; is the initial value of signal strength; T i is the attenuation coefficient; E i is the background noise; N{·} is a normally distributed random process with zero mean and unit variance; I LD is a low-dose PET / MR image; B(·) is a back-projection operation; T ND is a conventional dose PET / MR projection image; T N is the noise term; G(·) is the mapping function for generating the denoised image; I ND It is a conventional dose PET / MR image; The image reconstruction module specifically includes the following contents: Global-local design unit; the image reconstruction module uses a generator network, which contains multiple reconstruction blocks. Each reconstruction block consists of a global Transformer module and a local CNN module. The input of the generator is the denoised low-dose PET / MR image, and the output is the reconstructed image; the global Transformer module is expressed as: ; The local CNN module is expressed as: ; ;in, is the global Transformer module operation; Q, K, and V are the input query matrix, key matrix, and value matrix, respectively, obtained by feature mapping the denoised low-dose PET / MR images; D is the feature dimension; P i3 and P j3 It is a computational node in the local CNN module and is the feature representation of the PET / MR image; is the pooling operation between nodes; C is the output of the local CNN module; Concat(·) is the concatenation operation; P1, P2, and P3 are the output features of the three computing nodes in the local CNN module; i3 and j3 are node indices; Hierarchical neural structure search unit; in the high-level search stage, the distribution of the global Transformer module and the local CNN module is determined, and the adaptive fusion weight is introduced to obtain the output of the reconstructed block through weighted combination; in the low-level search stage, the multi-scale fusion operation is introduced to optimize the internal structure of the global Transformer module and the local CNN module respectively; the high-level search is expressed as: ; ; ; ; ; Low-level search is expressed as: ;in, and are the outputs of the i4th reconstruction block and the i4-1th reconstruction block respectively; and They are the adaptive fusion weights of the global Transformer module and the local CNN module; and are the parameter matrices of the global Transformer module and the local CNN module respectively; exp(·) is the exponential function; and is the characteristic statistic; Respectively represent the mean calculation of global features and local features; is the feature output of multi-scale fusion; S1 is the feature set of different scales; It is an operation to extract features at scale s; Reconstruction loss function design unit; introduce perceptual loss and adjacent contrast loss, perceptual loss It is expressed as: ; Proximity contrast loss It is expressed as: ; Get the reconstruction loss function , expressed as: ; Among them, α and β are the weights of perceptual loss and adjacent contrast loss; is the weight of the feature layer perceptual loss, l is the feature layer index; is a feature extraction function that extracts features at layer l based on the pre-trained network; SZF is an undersampled PET / MR image; is the square of the L2 norm; SGT u and SGT r are the pixel values ​​at the u-th position and the r-th position of the target image, respectively. The target image is a high-quality PET / MR image; and is the reconstruction weight; and are the pixel values ​​of the reconstructed image at the u-th position and the r-th position respectively; Reconstruction module judgment unit; pre-dividing the denoised low-dose PET / MR images into a reconstruction training set and a reconstruction test set; when the reconstruction loss converges to the reconstruction training set loss and achieves the expected effect on the reconstruction test set, the reconstruction module is established.

2. The deep learning-based integrated PET / MR organ registration system according to claim 1, characterized in that: The image denoising module specifically includes the following contents: Noise modeling unit; Network architecture design unit; the main network structure of the image denoising module includes an encoder, a noise adaptive unit, a decoder and a discriminator; the encoder and decoder are composed of multiple BLOGS modules respectively, and the network introduces jump connections to prevent gradient disappearance; the GAN discriminator uses the classic Patch-GAN structure, and downsamples the convolution layer to penalize structural mismatches at the patch level to improve the denoising effect; the BLOGS module includes multiple 1x1 convolutional layers, a 3x3 convolutional layer, a GELU activation layer and a simple self-attention module; Noise adaptive unit design: In order to store high-level hidden features of noises of different intensities, a noise adaptive unit is proposed, which contains the memory items K of strong noise and weak noise. h and K l , expressed as: ; Update the memory item, expressed as: ; Use soft attention reading strategy for noise decoding, expressed as: ;in, is the memory item index that best matches the input feature; k hi1 and q hj1 are the memory item and query vector before updating respectively; is the updated memory term; τ is the update coefficient; It is an indicator function, indicating whether the current index matches; is feature reconstruction; n1 is the total number of stored memory items; i1 and j1 are memory item indices; is the similarity between memory items, obtained based on cosine similarity; T is the transposition operation; Loss function design unit; design hybrid adversarial loss , expressed as: ; Design to smooth visual loss , expressed as: ; ; Design multi-scale perceptual loss , expressed as: ; Design adaptive texture matching loss , expressed as: ; Get the final denoising loss , expressed as: ; Where D(·) is the discriminator; G(·) is the generator; and is the adversarial weight parameter; E is the expectation operator; is a smooth function, x is the function variable; is the i2th low-dose PET / MR image sample; is the weight of the i2th scale; n2 is the total scale; is the feature extraction function; is the square of the Frobenius norm; is the weight of the i2th feature layer; It is a Gram matrix operation; , and is the balance loss weight; Denoising module determination: When the loss of the denoising module on the training set converges and the expected effect on the test set meets the standard, the denoising module is established.

3. The deep learning-based integrated PET / MR organ registration system according to claim 1, characterized in that: In the image acquisition module, the historical PET / MR organ registration image data includes PET images of different dose levels and MR images of different modalities; the images are structurally annotated; and the acquired data are divided into a test set and a training set.

4. The deep learning-based integrated PET / MR organ registration system according to claim 1, characterized in that: The input of the organ registration module is the PET / MR organ image to be registered, which is collected in real time and processed by the image denoising module and the image reconstruction module; the organ registration module adopts a combination of a variational autoencoder and a U-Net network architecture to achieve spatial alignment between images by predicting a deformation field; the deformation field captures the spatial difference between the PET image and the MR image, and is then applied to the registration of the two images. First, VAE is used to learn potential deformation features from the input PET and MR images, and then the U-Net network, as the backbone network of the organ registration module, is responsible for accurately predicting the deformation field. The jump connection of U-Net can effectively prevent the gradient from disappearing and retain the detailed information of the image, ensuring that the generated deformation field can accurately represent the spatial mapping between the PET image and the MR image; the generated deformation field represents the spatial transformation of the PET image relative to the MR image, and the pixels of the PET image are mapped to the space of the MR image through a bilinear interpolation method to achieve accurate alignment of the two modality images.

Images