Integrated PET / MR organ registration system based on deep learning

Abstract

The invention discloses an integrated PET / MR organ registration system based on deep learning. The system comprises an image acquisition module, an image denoising module, an image reconstruction module and an organ registration module. The invention belongs to the technical field of image processing, and particularly relates to an integrated PET / MR (positron emission tomography / magnetic resonance) organ registration system based on deep learning, which defines low-dose PET / MR images with different noise intensities, introduces a noise adaptability module, improves the quality of the low-dose PET / MR images, and introduces adaptive texture matching loss to construct a denoising module with high adaptability. The precision and high efficiency of organ registration are ensured; performing multi-scale feature fusion on different levels in combination with an adaptive fusion weight, and optimizing local details; hierarchical neural structure search is introduced to improve the registration precision of the registration system in different types of PET / MR images; the reconstruction structure is optimized based on the adjacent contrast loss, and the subsequent registration efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an integrated PET / MR organ registration system based on deep learning. Background Art

[0002] The PET / MR organ registration system is a technical system for spatially aligning PET images and MR images. The purpose of this system is to accurately align images of different modalities in space to obtain more accurate anatomical and functional information. However, the general PET / MR organ registration system has problems such as improper processing of low-dose PET / MR images with different noise intensities, which reduces the registration accuracy, and poor adaptability to PET / MR images with complex noise patterns, which leads to loss of image structure and poor organ registration effect; the general PET / MR organ registration system has problems such as resolution differences leading to registration errors and loss of image details leading to low registration accuracy. Summary of the invention

[0003] In view of the above situation, in order to overcome the defects of the prior art, the present invention provides an integrated PET / MR organ registration system based on deep learning. The general PET / MR organ registration system has the problems of improper processing of low-dose PET / MR images with different noise intensities, reduced registration accuracy, poor adaptability to PET / MR images with complex noise patterns, and thus loss of image structure and poor organ registration effect. This scheme defines low-dose PET / MR images with different noise intensities, introduces a noise adaptability module, stores high-level hidden features of PET / MR images with different noise intensities, improves the quality of low-dose PET / MR images, and reduces noise. In order to solve the problem of low registration accuracy caused by resolution difference and loss of image details in general PET / MR organ registration system, this scheme processes global information and local details in PET / MR images based on global-local design units, combines adaptive fusion weights, performs multi-scale feature fusion at different levels, and optimizes local details. Hierarchical neural structure search is introduced to improve the registration accuracy of the registration system in different types of PET / MR images. The reconstruction structure is optimized based on adjacent contrast loss to improve the efficiency of subsequent registration.

[0004] The technical solution adopted by the present invention is as follows: the present invention provides an integrated PET / MR organ registration system based on deep learning, including an image acquisition module, an image denoising module, an image reconstruction module and an organ registration module;

[0005] The image acquisition module acquires historical PET / MR organ registration image data;

[0006] The image denoising module optimizes the denoising effect of low-dose PET / MR images by designing noise modeling, adaptive memory units, generative adversarial networks and multiple loss functions, combining encoders, decoders and self-attention mechanisms, ensuring loss convergence on training sets and test sets, and ultimately achieving high-quality image reconstruction;

[0007] The image reconstruction module uses a globally-locally designed generator network combined with Transformer and CNN modules to denoise and reconstruct low-dose PET / MR images, and generates high-quality images through multi-scale fusion, hierarchical neural structure search and perceptual loss, and ensures the loss convergence of training and test sets to complete the image reconstruction task;

[0008] The organ registration module combines the variational autoencoder and the U-Net network to learn and predict the deformation field, achieve accurate spatial alignment of PET and MR images, and ensure efficient registration of the two modal images.

[0009] Furthermore, in the image acquisition module, the historical PET / MR organ registration image data includes PET images of different dose levels and MR images of different modalities; and the images are structurally annotated; and the acquired data are divided into a test set and a training set.

[0010] Furthermore, the image denoising module specifically includes the following contents:

[0011] Noise modeling unit: defines low-dose PET / MR images with different noise intensities, expressed as: ; The relationship between conventional-dose PET / MR images and low-dose PET / MR images is expressed as: ; Image-based and the segmentation threshold to classify low-dose PET / MR images into high-noise and low noise ; Based on the mapping, the denoised image of the conventional dose PET / MR image is generated, which is expressed as: ; Among them, P i is the signal strength at the i-th pixel position, i is the pixel position index; is the initial value of signal strength; T i is the attenuation coefficient; E i is the background noise; N{·} is a normally distributed random process with zero mean and unit variance; I LD is a low-dose PET / MR image; B(·) is a back-projection operation; T ND is a conventional dose PET / MR projection image; T N is the noise term; G(·) is the mapping function for generating the denoised image; I NDIt is a conventional dose PET / MR image;

[0012] Network architecture design unit; the main network structure of the image denoising module includes an encoder, a noise adaptive unit, a decoder and a discriminator; the encoder and decoder are composed of multiple BLOGS modules respectively, and the network introduces jump connections to prevent gradient disappearance; the GAN discriminator uses the classic Patch-GAN structure, and downsamples the convolution layer to penalize structural mismatches at the patch level to improve the denoising effect; the BLOGS module includes multiple 1x1 convolutional layers, a 3x3 convolutional layer, a GELU activation layer and a simple self-attention module;

[0013] Noise adaptive unit design: In order to store high-level hidden features of noises of different intensities, a noise adaptive unit is proposed, which contains the memory items K of strong noise and weak noise. h and K l , expressed as: ; Update the memory item, expressed as: ; Use soft attention reading strategy for noise decoding, expressed as: ;in, is the memory item index that best matches the input feature; k hi1 and q hj1 are the memory item and query vector before updating respectively; is the updated memory term; τ is the update coefficient; It is an indicator function, indicating whether the current index matches; is feature reconstruction; n1 is the total number of stored memory items; i1 and j1 are memory item indices; is the similarity between memory items, obtained based on cosine similarity; T is the transposition operation;

[0014] Loss function design unit; design hybrid adversarial loss , expressed as: ; Design to smooth visual loss , expressed as: ; ; Design multi-scale perceptual loss , expressed as: ; Design adaptive texture matching loss , expressed as: ; Get the final denoising loss , expressed as: ; Where D(·) is the discriminator; G(·) is the generator; and is the adversarial weight parameter; E is the expectation operator; is a smooth function, x is the function variable; is the i2th low-dose PET / MR image sample; is the weight of the i2th scale; n2 is the total scale; is the feature extraction function; is the square of the Frobenius norm; is the weight of the i2th feature layer; is a Gram matrix operation; , and is the balance loss weight;

[0015] Denoising module determination: When the loss of the denoising module on the training set converges and the expected effect on the test set meets the standard, the denoising module is established.

[0016] Furthermore, the image reconstruction module specifically includes the following contents:

[0017] Global-local design unit; the image reconstruction module uses a generator network, which contains multiple reconstruction blocks. Each reconstruction block consists of a global Transformer module and a local CNN module. The input of the generator is the denoised low-dose PET / MR image, and the output is the reconstructed image; the global Transformer module is expressed as: ; The local CNN module is expressed as: ; ;in, is the global Transformer module operation; Q, K, and V are the input query matrix, key matrix, and value matrix, respectively, obtained by feature mapping the denoised low-dose PET / MR images; D is the feature dimension; P i3 and P j3 It is a computational node in the local CNN module and is the feature representation of the PET / MR image; is the pooling operation between nodes; C is the output of the local CNN module; Concat(·) is the concatenation operation; P1, P2, and P3 are the output features of the three computing nodes in the local CNN module; i3 and j3 are node indices;

[0018] Hierarchical neural structure search unit; in the high-level search stage, the distribution of the global Transformer module and the local CNN module is determined, and the adaptive fusion weight is introduced to obtain the output of the reconstructed block through weighted combination; in the low-level search stage, the multi-scale fusion operation is introduced to optimize the internal structure of the global Transformer module and the local CNN module respectively; the high-level search is expressed as: ; ; ; ; ; Low-level search is expressed as: ;in, and are the outputs of the i4th reconstruction block and the i4-1th reconstruction block respectively; and They are the adaptive fusion weights of the global Transformer module and the local CNN module; and are the parameter matrices of the global Transformer module and the local CNN module respectively; exp(·) is the exponential function; and is the characteristic statistic; Respectively represent the mean calculation of global features and local features; is the feature output of multi-scale fusion; S1 is the feature set of different scales; It is an operation to extract features at scale s;

[0019] Reconstruction loss function design unit; introduce perceptual loss and adjacent contrast loss, perceptual loss It is expressed as: ; Proximity contrast loss It is expressed as: ; Get the reconstruction loss function , expressed as: ; Among them, α and β are the weights of perceptual loss and adjacent contrast loss; is the weight of the feature layer perceptual loss, l is the feature layer index; is a feature extraction function that extracts features at layer l based on the pre-trained network; SZF is an undersampled PET / MR image; is the square of the L2 norm; SGT u and SGT r are the pixel values ​​at the u-th position and the r-th position of the target image, respectively. The target image is a high-quality PET / MR image; and is the reconstruction weight; and are the pixel values ​​of the reconstructed image at the u-th position and the r-th position respectively;

[0020] Reconstruction module judgment unit; pre-dividing the denoised low-dose PET / MR images into a reconstruction training set and a reconstruction test set; when the reconstruction loss converges to the reconstruction training set loss and achieves the expected effect on the reconstruction test set, the reconstruction module is established.

[0021] Furthermore, the input of the organ registration module is the PET / MR organ image to be registered, which is collected in real time and processed by the image denoising module and the image reconstruction module; the organ registration module adopts a combination of a variational autoencoder and a U-Net network architecture to achieve spatial alignment between images by predicting a deformation field; the deformation field captures the spatial difference between the PET image and the MR image, and is then applied to the registration of the two images. First, VAE is used to learn potential deformation features from the input PET and MR images, and then the U-Net network, as the backbone network of the organ registration module, is responsible for accurately predicting the deformation field. The jump connection of U-Net can effectively prevent the gradient from disappearing and retain the detailed information of the image, ensuring that the generated deformation field can accurately represent the spatial mapping between the PET image and the MR image; the generated deformation field represents the spatial transformation of the PET image relative to the MR image, and the pixels of the PET image are mapped to the space of the MR image through a bilinear interpolation method to achieve accurate alignment of the two modality images.

[0022] The beneficial effects achieved by the present invention using the above scheme are as follows:

[0023] (1) In view of the problems that the general PET / MR organ registration system has problems such as improper processing of low-dose PET / MR images with different noise intensities, reduced registration accuracy, and poor adaptability to PET / MR images with complex noise patterns, which leads to loss of image structure and poor organ registration effect, this scheme defines low-dose PET / MR images with different noise intensities, introduces noise adaptive units, stores high-level hidden features of PET / MR images with different noise intensities, improves the quality of low-dose PET / MR images, reduces the impact of noise on important structural areas, and introduces adaptive texture matching loss to construct a highly adaptable denoising module to ensure the accuracy and efficiency of organ registration.

[0024] (2) In order to address the problems of resolution differences leading to registration errors and image detail loss leading to low registration accuracy in general PET / MR organ registration systems, this scheme processes the global information and local details in PET / MR images based on a global-local design unit, combines adaptive fusion weights, performs multi-scale feature fusion at different levels, and optimizes local details. It introduces a hierarchical neural structure search to improve the registration accuracy of the registration system for different types of PET / MR images. It optimizes the reconstructed structure based on the adjacent contrast loss to improve the efficiency of subsequent registration. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 A schematic diagram of a process flow of an integrated PET / MR organ registration system based on deep learning provided by the present invention;

[0026] Figure 2 This is a flow chart of the image denoising module;

[0027] Figure 3 Schematic diagram of the image reconstruction process.

[0028] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention. DETAILED DESCRIPTION

[0029] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0030] In the description of the present invention, it should be understood that terms such as “upper”, “lower”, “front”, “back”, “left”, “right”, “top”, “bottom”, “inside” and “outside” indicating directions or positional relationships are based on the directions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore should not be understood as limiting the present invention.

[0031] Example 1, see Figure 1 The present invention provides an integrated PET / MR organ registration system based on deep learning, comprising an image acquisition module, an image denoising module, an image reconstruction module and an organ registration module;

[0032] The image acquisition module acquires historical PET / MR organ registration image data; and sends the data to the image denoising module;

[0033] The image denoising module receives data sent by the image acquisition module; optimizes the denoising effect of low-dose PET / MR images by designing noise modeling, adaptive memory units, generative adversarial networks and multiple loss functions, combining encoders, decoders and self-attention mechanisms, ensuring loss convergence on training sets and test sets, and ultimately achieving high-quality image reconstruction; and sends the data to the image reconstruction module;

[0034] The image reconstruction module receives data sent by the image denoising module; performs denoising and reconstruction of low-dose PET / MR images by using a globally-locally designed generator network in combination with Transformer and CNN modules, generates high-quality images through multi-scale fusion, hierarchical neural structure search and perceptual loss, ensures loss convergence of training and test sets, and completes the image reconstruction task; and sends the data to the organ registration module;

[0035] The organ registration module receives data sent by the image reconstruction module; by combining the variational autoencoder and the U-Net network, it learns and predicts the deformation field, realizes the precise spatial alignment of the PET and MR images, and ensures the efficient registration of the two modal images.

[0036] Example 2, see Figure 1 This embodiment is based on the above embodiment. In the image acquisition module, the historical PET / MR organ registration image data includes PET images of different dose levels and MR images of different modalities; and the images are structurally annotated; the acquired data are divided into a test set and a training set; the structural annotation specifically includes contour annotation of each organ in the PET / MR image to indicate the position and shape of each organ.

[0037] Example 3, see Figure 1 and Figure 2 This embodiment is based on the above embodiment, and the image denoising module specifically includes the following contents:

[0038] Noise modeling unit: defines low-dose PET / MR images with different noise intensities, expressed as: ; The relationship between conventional-dose PET / MR images and low-dose PET / MR images is expressed as: ; Image-based and the segmentation threshold to classify low-dose PET / MR images into high-noise and low noise ; When there are pixels in the PET / MR image When it is higher than the classification threshold, the PET / MR image is classified as high noise; based on the mapping, the denoised image of the conventional dose PET / MR image is generated, which is expressed as: ; Among them, P i is the signal strength at the i-th pixel position, i is the pixel position index; is the initial value of signal strength; T i is the attenuation coefficient; E i is the background noise; N{·} is a normally distributed random process with zero mean and unit variance; I LD is a low-dose PET / MR image; B(·) is a back-projection operation; T ND is a conventional dose PET / MR projection image; T N is the noise term; G(·) is the mapping function for generating the denoised image; I ND It is a conventional dose PET / MR image;

[0039] Network architecture design unit; the main network structure of the image denoising module includes an encoder, a noise adaptive unit, a decoder and a discriminator; the encoder and decoder are composed of multiple BLOGS modules respectively, and the network introduces jump connections to prevent gradient disappearance; the GAN discriminator uses the classic Patch-GAN structure, and downsamples the convolution layer to penalize structural mismatches at the patch level to improve the denoising effect; the BLOGS module includes multiple 1x1 convolutional layers, a 3x3 convolutional layer, a GELU activation layer and a simple self-attention module;

[0040] Noise adaptive unit design: In order to store high-level hidden features of noises of different intensities, a noise adaptive unit is proposed, which contains the memory items K of strong noise and weak noise. h and K l , expressed as: ; Update the memory item, expressed as: ; Use soft attention reading strategy for noise decoding, expressed as: ;in, is the memory item index that best matches the input feature; k hi1 and q hj1 are the memory item and query vector before updating respectively; is the updated memory term; τ is the update coefficient; It is an indicator function, indicating whether the current index matches; is feature reconstruction; n1 is the total number of stored memory items; i1 and j1 are memory item indices; is the similarity between memory items, obtained based on cosine similarity; T is the transposition operation;

[0041] Loss function design unit; design hybrid adversarial loss , expressed as: ; Design to smooth visual loss , expressed as: ; ; Design multi-scale perceptual loss , expressed as: ; Design adaptive texture matching loss , expressed as: ; Get the final denoising loss , expressed as: ; Where D(·) is the discriminator; G(·) is the generator; and is the adversarial weight parameter; E is the expectation operator; is a smooth function, x is the function variable; is the i2th low-dose PET / MR image sample; is the weight of the i2th scale; n2 is the total scale; is the feature extraction function; is the square of the Frobenius norm; is the weight of the i2th feature layer; is a Gram matrix operation; , and is the balance loss weight;

[0042] Denoising module determination: When the loss of the denoising module on the training set converges and the expected effect on the test set meets the standard, the denoising module is established.

[0043] By performing the above operations, the general PET / MR organ registration system has the problems of improper processing of low-dose PET / MR images with different noise intensities, reduced registration accuracy, poor adaptability to PET / MR images with complex noise patterns, and further resulting in loss of image structure and poor organ registration effect. This scheme defines low-dose PET / MR images with different noise intensities, introduces noise adaptive units, stores high-level hidden features of PET / MR images with different noise intensities, improves the quality of low-dose PET / MR images, reduces the impact of noise on important structural areas, introduces adaptive texture matching loss to construct a highly adaptable denoising module, and ensures the accuracy and efficiency of organ registration.

[0044] Example 4, see Figure 1 and Figure 3 This embodiment is based on the above embodiment, and the image reconstruction module specifically includes the following contents:

[0045] Global-local design unit; the image reconstruction module uses a generator network, which contains multiple reconstruction blocks. Each reconstruction block consists of a global Transformer module and a local CNN module. The input of the generator is the denoised low-dose PET / MR image, and the output is the reconstructed image; the global Transformer module is expressed as: ; The local CNN module is expressed as: ; ;in, is the global Transformer module operation; Q, K, and V are the input query matrix, key matrix, and value matrix, respectively, obtained by feature mapping the denoised low-dose PET / MR images; D is the feature dimension; P i3 and P j3 It is a computational node in the local CNN module and is the feature representation of the PET / MR image; is the pooling operation between nodes; C is the output of the local CNN module; Concat(·) is the concatenation operation; P1, P2, and P3 are the output features of the three computing nodes in the local CNN module; i3 and j3 are node indices;

[0046] Hierarchical neural structure search unit; in the high-level search stage, the distribution of the global Transformer module and the local CNN module is determined, and the adaptive fusion weight is introduced to obtain the output of the reconstructed block through weighted combination; in the low-level search stage, the multi-scale fusion operation is introduced to optimize the internal structure of the global Transformer module and the local CNN module respectively; the high-level search is expressed as: ; ; ; ; ; Low-level search is expressed as: ;in, and are the outputs of the i4th reconstruction block and the i4-1th reconstruction block respectively; and They are the adaptive fusion weights of the global Transformer module and the local CNN module; and are the parameter matrices of the global Transformer module and the local CNN module respectively; exp(·) is the exponential function; and is the characteristic statistic; Respectively represent the mean calculation of global features and local features; is the feature output of multi-scale fusion; S1 is the feature set of different scales; It is an operation to extract features at scale s;

[0047] Reconstruction loss function design unit; introduce perceptual loss and adjacent contrast loss, perceptual loss It is expressed as: ; Proximity contrast loss It is expressed as: ; Get the reconstruction loss function , expressed as: ; Among them, α and β are the weights of perceptual loss and adjacent contrast loss; is the weight of the feature layer perceptual loss, l is the feature layer index; is a feature extraction function that extracts features at layer l based on the pre-trained network; SZF is an undersampled PET / MR image; is the square of the L2 norm; SGT u and SGT r are the pixel values ​​at the u-th position and the r-th position of the target image, respectively. The target image is a high-quality PET / MR image; and is the reconstruction weight; and are the pixel values ​​of the reconstructed image at the u-th position and the r-th position respectively;

[0048] Reconstruction module judgment unit; pre-dividing the denoised low-dose PET / MR images into a reconstruction training set and a reconstruction test set; when the reconstruction loss converges to the reconstruction training set loss and achieves the expected effect on the reconstruction test set, the reconstruction module is established.

[0049] By performing the above operations, in order to solve the problems of resolution difference leading to registration error and image detail loss leading to low registration accuracy in general PET / MR organ registration systems, this scheme processes global information and local details in PET / MR images based on a global-local design unit, combines adaptive fusion weights, performs multi-scale feature fusion at different levels, and optimizes local details; introduces hierarchical neural structure search to improve the registration accuracy of the registration system for different types of PET / MR images; optimizes the reconstruction structure based on adjacent contrast loss to improve subsequent registration efficiency.

[0050] Example 5, see Figure 1 This embodiment is based on the above embodiment. The input of the organ registration module is the PET / MR organ image to be registered, which is collected in real time and processed by the image denoising module and the image reconstruction module. The organ registration module adopts a combination of a variational autoencoder and a U-Net network architecture to achieve spatial alignment between images by predicting a deformation field. The deformation field captures the spatial difference between the PET image and the MR image, and is then applied to the registration of the two images. First, VAE is used to learn potential deformation features from the input PET and MR images. Then, the U-Net network is used as the backbone network of the organ registration module to accurately predict the deformation field. The jump connection of U-Net can effectively prevent the gradient from disappearing and retain the detailed information of the image, ensuring that the generated deformation field can accurately represent the spatial mapping between the PET image and the MR image. The generated deformation field represents the spatial transformation of the PET image relative to the MR image. The pixel points of the PET image are mapped to the space of the MR image through a bilinear interpolation method to achieve accurate alignment of the two modality images.

[0051] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0052] While the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that many changes, modifications, substitutions and variations can be made to the embodiments without departing from the principles and spirit of the invention.

[0053] The present invention and its embodiments are described above, and such description is not restrictive. The drawings show only one embodiment of the present invention, and the actual structure is not limited thereto. In short, if ordinary technicians in the field are inspired by it, without departing from the purpose of the invention, they can design a structure and embodiment similar to the technical solution without creativity, which should belong to the protection scope of the present invention.

Claims

1. An integrated PET / MR organ registration system based on deep learning, characterized by: The system includes an image acquisition module, an image denoising module, an image reconstruction module and an organ registration module; The image acquisition module acquires historical PET / MR organ registration image data; The image denoising module optimizes the denoising effect of low-dose PET / MR images by designing noise modeling, adaptive memory units, generative adversarial networks and multiple loss functions, combining encoders, decoders and self-attention mechanisms, ensuring loss convergence on training sets and test sets, and ultimately achieving high-quality image reconstruction; The image reconstruction module uses a globally-locally designed generator network combined with Transformer and CNN modules to denoise and reconstruct low-dose PET / MR images, and generates high-quality images through multi-scale fusion, hierarchical neural structure search and perceptual loss, and ensures the loss convergence of training and test sets to complete the image reconstruction task; The organ registration module combines the variational autoencoder and the U-Net network to learn and predict the deformation field, achieve accurate spatial alignment of PET and MR images, and ensure efficient registration of the two modality images; The image denoising module includes a noise modeling unit, which defines low-dose PET / MR images with different noise intensities, expressed as: ; The relationship between conventional-dose PET / MR images and low-dose PET / MR images is expressed as: ; Image-based and the segmentation threshold to classify low-dose PET / MR images into high-noise and low noise ; Based on the mapping, the denoised image of the conventional dose PET / MR image is generated, which is expressed as: ; Among them, P i is the signal strength at the i-th pixel position, i is the pixel position index; is the initial value of signal strength; T i is the attenuation coefficient; E i is the background noise; N{·} is a normally distributed random process with zero mean and unit variance; I LD is a low-dose PET / MR image; B(·) is a back-projection operation; T ND is a conventional dose PET / MR projection image; T N is the noise term; G(·) is the mapping function for generating the denoised image; I ND This is a conventional dose PET / MR image.

2. The deep learning-based integrated PET / MR organ registration system according to claim 1, characterized in that: The image denoising module specifically includes the following contents: Noise modeling unit; Network architecture design unit; the main network structure of the image denoising module includes an encoder, a noise adaptive unit, a decoder and a discriminator; the encoder and decoder are composed of multiple BLOGS modules respectively, and the network introduces jump connections to prevent gradient disappearance; the GAN discriminator uses the classic Patch-GAN structure, and downsamples the convolution layer to penalize structural mismatches at the patch level to improve the denoising effect; the BLOGS module includes multiple 1x1 convolutional layers, a 3x3 convolutional layer, a GELU activation layer and a simple self-attention module; Noise adaptive unit design: In order to store high-level hidden features of noises of different intensities, a noise adaptive unit is proposed, which contains the memory items K of strong noise and weak noise. h and K l , expressed as: ; Update the memory item, expressed as: ; Use soft attention reading strategy for noise decoding, expressed as: ;in, is the memory item index that best matches the input feature; k hi1 and q hj1 are the memory item and query vector before updating respectively; is the updated memory term; τ is the update coefficient; It is an indicator function, indicating whether the current index matches; is feature reconstruction; n1 is the total number of stored memory items; i1 and j1 are memory item indices; is the similarity between memory items, obtained based on cosine similarity; T is the transposition operation; Loss function design unit; design hybrid adversarial loss , expressed as: ; Design to smooth visual loss , expressed as: ; ; Design multi-scale perceptual loss , expressed as: ; Design adaptive texture matching loss , expressed as: ; Get the final denoising loss , expressed as: ; Where D(·) is the discriminator; G(·) is the generator; and is the adversarial weight parameter; E is the expectation operator; is a smooth function, x is the function variable; is the i2th low-dose PET / MR image sample; is the weight of the i2th scale; n2 is the total scale; is the feature extraction function; is the square of the Frobenius norm; is the weight of the i2th feature layer; It is a Gram matrix operation; , and is the balance loss weight; Denoising module determination: When the loss of the denoising module on the training set converges and the expected effect on the test set meets the standard, the denoising module is established.

3. The deep learning-based integrated PET / MR organ registration system according to claim 2, characterized in that: The image reconstruction module specifically includes the following contents: Global-local design unit; the image reconstruction module uses a generator network, which contains multiple reconstruction blocks. Each reconstruction block consists of a global Transformer module and a local CNN module. The input of the generator is the denoised low-dose PET / MR image, and the output is the reconstructed image; the global Transformer module is expressed as: ; The local CNN module is expressed as: ; ;in, is the global Transformer module operation; Q, K, and V are the input query matrix, key matrix, and value matrix, respectively, obtained by feature mapping the denoised low-dose PET / MR images; D is the feature dimension; P i3 and P j3 It is a computational node in the local CNN module and is the feature representation of the PET / MR image; is the pooling operation between nodes; C is the output of the local CNN module; Concat(·) is the concatenation operation; P1, P2, and P3 are the output features of the three computing nodes in the local CNN module; i3 and j3 are node indices; Hierarchical neural structure search unit; in the high-level search stage, the distribution of the global Transformer module and the local CNN module is determined, and the adaptive fusion weight is introduced to obtain the output of the reconstructed block through weighted combination; in the low-level search stage, the multi-scale fusion operation is introduced to optimize the internal structure of the global Transformer module and the local CNN module respectively; the high-level search is expressed as: ; ; ; ; ; Low-level search is expressed as: ;in, and are the outputs of the i4th reconstruction block and the i4-1th reconstruction block respectively; and They are the adaptive fusion weights of the global Transformer module and the local CNN module; and are the parameter matrices of the global Transformer module and the local CNN module respectively; exp(·) is the exponential function; and is the characteristic statistic; Respectively represent the mean calculation of global features and local features; is the feature output of multi-scale fusion; S1 is the feature set of different scales; It is an operation to extract features at scale s; Reconstruction loss function design unit; introduce perceptual loss and adjacent contrast loss, perceptual loss It is expressed as: ; Proximity contrast loss It is expressed as: ; Get the reconstruction loss function , expressed as: ; Among them, α and β are the weights of perceptual loss and adjacent contrast loss; is the weight of the feature layer perceptual loss, l is the feature layer index; is a feature extraction function that extracts features at layer l based on the pre-trained network; SZF is an undersampled PET / MR image; is the square of the L2 norm; SGT u and SGT r are the pixel values ​​at the u-th position and the r-th position of the target image, respectively. The target image is a high-quality PET / MR image; and is the reconstruction weight; and are the pixel values ​​of the reconstructed image at the u-th position and the r-th position respectively; Reconstruction module judgment unit; pre-dividing the denoised low-dose PET / MR images into a reconstruction training set and a reconstruction test set; when the reconstruction loss converges to the reconstruction training set loss and achieves the expected effect on the reconstruction test set, the reconstruction module is established.

4. The deep learning-based integrated PET / MR organ registration system according to claim 1, characterized in that: In the image acquisition module, the historical PET / MR organ registration image data includes PET images of different dose levels and MR images of different modalities; the images are structurally annotated; and the acquired data are divided into a test set and a training set.

5. The deep learning-based integrated PET / MR organ registration system according to claim 1, characterized in that: The input of the organ registration module is the PET / MR organ image to be registered, which is collected in real time and processed by the image denoising module and the image reconstruction module; the organ registration module adopts a combination of a variational autoencoder and a U-Net network architecture to achieve spatial alignment between images by predicting a deformation field; the deformation field captures the spatial difference between the PET image and the MR image, and is then applied to the registration of the two images. First, VAE is used to learn potential deformation features from the input PET and MR images, and then the U-Net network, as the backbone network of the organ registration module, is responsible for accurately predicting the deformation field. The jump connection of U-Net can effectively prevent the gradient from disappearing and retain the detailed information of the image, ensuring that the generated deformation field can accurately represent the spatial mapping between the PET image and the MR image; the generated deformation field represents the spatial transformation of the PET image relative to the MR image, and the pixels of the PET image are mapped to the space of the MR image through a bilinear interpolation method to achieve accurate alignment of the two modality images.

Images