Method, system and device for magnetic particle imaging image denoising based on feature fusion

Abstract

The present application belongs to the field of magnetic particle imaging, and particularly relates to a magnetic particle imaging image denoising method, system and device based on feature fusion, aiming to solve the problem that the existing MPI denoising method is difficult to balance noise removal and image detail preservation. The method comprises: collecting an MPI image to be denoised as an input image; performing denoising processing on the input image based on a pre-trained feature fusion denoising network model to obtain a denoised MPI image; the feature fusion denoising network model comprises a feature extraction module, a feature fusion module and a feature regression module; the feature extraction module comprises a noise feature extractor and a content feature extractor; the feature fusion module comprises two channel attention sub-modules, two convolution layers and a spatial attention sub-module; and the feature regression module is constructed based on the convolution layers. The present application can achieve good denoising effect while preserving the structural details of the MPI image.

Description

Technical Field

[0001] This invention belongs to the field of magnetic particle imaging, and specifically relates to a method, system, and device for denoising magnetic particle imaging images based on feature fusion. Background Technology

[0002] Magnetic Particle Imaging (MPI) is a tomographic imaging method that uses the nonlinear response of magnetic nanoparticles to create three-dimensional, dynamic, real-time images of lesions or other biomarkers. Compared to existing medical imaging techniques such as Magnetic Resonance Imaging (MRI) and Computed Tomography (CT), MPI offers advantages such as high sensitivity, no depth limitations, and high safety, making it a promising new medical imaging method with broad application prospects.

[0003] In current practical applications, the signal-to-noise ratio (SNR) of MPI images is low due to the influence of the equipment itself and external environmental factors. In multicolor MPI, the low SNR is insufficient for clinical needs when imaging multiple concentration samples. Therefore, improving the SNR of MPI images is a challenge and difficulty for current MPI equipment. Existing methods mainly improve MPI imaging quality by processing noise in the signal domain. Due to the high sensitivity and real-time nature of MPI imaging, the noise in the imaging is complex and variable, and it is difficult to construct statistical models in the signal domain for effective quantitative analysis of information and noise. Furthermore, existing methods struggle to simultaneously remove image noise and preserve image structural details. Based on this, this invention proposes a feature fusion learning-based magnetic particle imaging image denoising method, aiming to effectively remove interference noise from images while preserving image structural details, thereby improving the imaging quality of MPI. Summary of the Invention

[0004] To address the aforementioned problems in existing technologies, specifically the difficulty of simultaneously achieving noise removal and image detail preservation in current MPI denoising methods, this invention proposes a feature fusion-based magnetic particle imaging image denoising method, which includes:

[0005] S100: Acquire the MPI image to be denoised as the input image;

[0006] S200, the input image is denoised based on a pre-trained feature fusion denoising network model to obtain a denoised MPI image;

[0007] The feature fusion denoising network model includes a feature extraction module, a feature fusion module, and a feature regression module.

[0008] The feature extraction module includes a noise feature extractor and a content feature extractor; the inputs to both the noise feature extractor and the content feature extractor are noisy MPI images.

[0009] Both the noise feature extractor and the content feature extractor are constructed based on three downsampling layers and three upsampling layers; the three downsampling layers are respectively the first downsampling layer, the second downsampling layer, and the third downsampling layer; the three upsampling layers are respectively the first upsampling layer, the second upsampling layer, and the third upsampling layer.

[0010] The first downsampling layer, the second downsampling layer, the third downsampling layer, the first upsampling layer, the second upsampling layer, and the third upsampling layer are connected sequentially; the first downsampling layer and the third downsampling layer, and the second downsampling layer and the second upsampling layer are connected across layers through a convolutional attention module;

[0011] The convolutional attention module is constructed based on a sequentially connected channel attention submodule and spatial attention submodule;

[0012] The channel attention submodule is used to perform average pooling on the input features, multiply the features before average pooling with the features after average pooling, and use the result of the multiplication as the output of the channel attention submodule.

[0013] The spatial attention submodule is used to convolve the input features, perform max pooling on the convolved features, multiply the features before max pooling with the features after max pooling, and use the result of the multiplication as the output of the spatial attention submodule.

[0014] The feature fusion module includes two channel attention submodules, two convolutional layers, and one spatial attention submodule. The inputs of the two channel attention submodules are the outputs of the noise feature extractor and the content feature extractor, respectively. The outputs of the two channel attention submodules are concatenated and combined, and then the fused features are extracted through the two convolutional layers. The fused features are input into the spatial attention submodule for processing, and the processed fused features are used as the output of the feature fusion module.

[0015] The feature regression module is built based on a convolutional layer and is used to perform convolution operations on the output of the feature fusion module to output a denoised MPI image.

[0016] In some preferred embodiments, each of the three downsampling layers and the three upsampling layers is based on 3×3 convolution operations, batch normalization, and Dropout operations.

[0017] In some preferred embodiments, the feature fusion denoising network model is trained as follows:

[0018] A100 acquires a noise-free MPI simulation image and a real noise image of the MPI device. The real noise image is randomly cropped and processed to obtain an amplified noise image. The image processing includes rotation, white balance transformation, and gamma transformation.

[0019] A200 multiplies the amplified noisy image by a set intensity coefficient and superimposes it with the noise-free simulated MPI image to obtain an MPI image with real noise; a training dataset is constructed based on the MPI image with real noise and its corresponding ground truth image.

[0020] A300: Input the MPI images with real noise in the training dataset into the feature fusion denoising network model to obtain the denoised MPI images, which are used as the feature fusion learning output images. Also, obtain the features extracted by the noise feature extractor and the content feature extractor in the feature fusion denoising network model as noise features and content features, respectively.

[0021] A400 processes the noise features using a pre-built noise feature regressor, and then subtracts the processed noise features from the MPI image with real noise input to the feature fusion denoising network model to obtain a noise learning output image; the content features are processed using a pre-built content feature regressor to obtain a content learning output image; both the noise feature regressor and the content feature regressor are built based on convolutional layers;

[0022] A500 updates the model parameters of the feature fusion denoising network model by calculating the loss value through a pre-constructed loss function based on the noise learning output image, the content learning output image, and the feature fusion learning output image, combined with the ground truth image.

[0023] A600, repeat steps A300-A500 until a trained feature fusion and denoising network model is obtained.

[0024] In some preferred embodiments, the noise-free MPI simulation image is obtained by:

[0025] A particle distribution image is randomly generated; the grayscale value in the particle distribution image represents the concentration of magnetic particles.

[0026] The voltage signal corresponding to the particle distribution image is generated according to the MPI principle, and the image is reconstructed using the x-space reconstruction algorithm to obtain a noise-free MPI simulation image.

[0027] In some preferred embodiments, the real noise image of the MPI device is obtained by placing a phantom without injected magnetic particles into the MPI device for air sampling to obtain the real noise image of the MPI device.

[0028] In some preferred embodiments, the pre-constructed loss function is:

[0029]

[0030] Among them, I content I1 represents the ground truth image, and I2, I3 represent the noise learning output image, the feature fusion learning output image, and the content learning output image, respectively.

[0031] In a second aspect, the present invention proposes a magnetic particle imaging image denoising system based on feature fusion, the system comprising: an image acquisition module and an image denoising module;

[0032] The image acquisition module is configured to acquire the MPI image to be denoised as the input image;

[0033] The image denoising module is configured to perform denoising processing on the input image based on a pre-trained feature fusion denoising network model to obtain a denoised MPI image.

[0034] The feature fusion denoising network model includes a feature extraction module, a feature fusion module, and a feature regression module.

[0035] The feature extraction module includes a noise feature extractor and a content feature extractor; the inputs to both the noise feature extractor and the content feature extractor are noisy MPI images.

[0036] Both the noise feature extractor and the content feature extractor are constructed based on three downsampling layers and three upsampling layers; the three downsampling layers are respectively the first downsampling layer, the second downsampling layer, and the third downsampling layer; the three upsampling layers are respectively the first upsampling layer, the second upsampling layer, and the third upsampling layer.

[0037] The first downsampling layer, the second downsampling layer, the third downsampling layer, the first upsampling layer, the second upsampling layer, and the third upsampling layer are connected sequentially; the first downsampling layer and the third downsampling layer, and the second downsampling layer and the second upsampling layer are connected across layers through a convolutional attention module;

[0038] The convolutional attention module is constructed based on a sequentially connected channel attention submodule and spatial attention submodule;

[0039] The channel attention submodule is used to perform average pooling on the input features, multiply the features before average pooling with the features after average pooling, and use the result of the multiplication as the output of the channel attention submodule.

[0040] The spatial attention submodule is used to convolve the input features, perform max pooling on the convolved features, multiply the features before max pooling with the features after max pooling, and use the result of the multiplication as the output of the spatial attention submodule.

[0041] The feature fusion module includes two channel attention submodules, two convolutional layers, and one spatial attention submodule. The inputs of the two channel attention submodules are the outputs of the noise feature extractor and the content feature extractor, respectively. The outputs of the two channel attention submodules are concatenated and combined, and then the fused features are extracted through the two convolutional layers. The fused features are input into the spatial attention submodule for processing, and the processed fused features are used as the output of the feature fusion module.

[0042] The feature regression module is built based on a convolutional layer and is used to perform convolution operations on the output of the feature fusion module to output a denoised MPI image.

[0043] A third aspect of the present invention provides an electronic device comprising: at least one processor; and a memory communicatively connected to at least one of the processors; wherein the memory stores instructions executable by the processor for implementing the above-described feature fusion-based magnetic particle imaging image denoising method.

[0044] In a fourth aspect, the present invention provides a computer-readable storage medium, characterized in that the computer-readable storage medium stores computer instructions for execution by a computer to implement the above-described feature fusion-based magnetic particle imaging image denoising method.

[0045] The beneficial effects of this invention are:

[0046] This invention can maintain the structural details of MPI images while achieving good noise reduction.

[0047] This invention processes noise generated during magnetic particle imaging in the image domain. It extracts noise and content features of MPI images by designing two feature extractors and corresponding loss functions, and introduces an attention mechanism to achieve efficient fusion of the two features. Image prediction is performed using the fused features, which can remove noise from MPI images while preserving image details. The denoising effect is particularly significant when the magnetic particle concentration is low. Attached Figure Description

[0048] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings.

[0049] Figure 1 This is a flowchart illustrating a feature fusion-based magnetic particle imaging image denoising method according to an embodiment of the present invention.

[0050] Figure 2 This is a schematic diagram of the framework of a feature fusion-based magnetic particle imaging image denoising system according to an embodiment of the present invention.

[0051] Figure 3 This is a simplified structural diagram of a feature fusion denoising network model according to an embodiment of the present invention;

[0052] Figure 4 This is a schematic diagram of the feature extraction module according to an embodiment of the present invention;

[0053] Figure 5 This is a schematic diagram of the structure of a convolutional attention module according to an embodiment of the present invention;

[0054] Figure 6 This is a schematic diagram of the feature fusion module according to an embodiment of the present invention;

[0055] Figure 7 This is a simplified flowchart illustrating the training process of a feature fusion denoising network model according to an embodiment of the present invention.

[0056] Figure 8 This is a schematic diagram of the structure of a computer system suitable for implementing an electronic device according to an embodiment of the present invention. Detailed Implementation

[0057] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0058] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0059] A method for denoising magnetic particle imaging images based on feature fusion according to the first embodiment of the present invention, such as... Figure 1 As shown, the method includes:

[0060] S100: Acquire the MPI image to be denoised as the input image;

[0061] S200, the input image is denoised based on a pre-trained feature fusion denoising network model to obtain a denoised MPI image;

[0062] The feature fusion denoising network model includes a feature extraction module, a feature fusion module, and a feature regression module.

[0063] The feature extraction module includes a noise feature extractor and a content feature extractor; the inputs to both the noise feature extractor and the content feature extractor are noisy MPI images.

[0064] Both the noise feature extractor and the content feature extractor are constructed based on three downsampling layers and three upsampling layers; the three downsampling layers are respectively the first downsampling layer, the second downsampling layer, and the third downsampling layer; the three upsampling layers are respectively the first upsampling layer, the second upsampling layer, and the third upsampling layer.

[0065] The first downsampling layer, the second downsampling layer, the third downsampling layer, the first upsampling layer, the second upsampling layer, and the third upsampling layer are connected sequentially; the first downsampling layer and the third downsampling layer, and the second downsampling layer and the second upsampling layer are connected across layers through a convolutional attention module;

[0066] The convolutional attention module is constructed based on a sequentially connected channel attention submodule and spatial attention submodule;

[0067] The channel attention submodule is used to perform average pooling on the input features, multiply the features before average pooling with the features after average pooling, and use the result of the multiplication as the output of the channel attention submodule.

[0068] The spatial attention submodule is used to convolve the input features, perform max pooling on the convolved features, multiply the features before max pooling with the features after max pooling, and use the result of the multiplication as the output of the spatial attention submodule.

[0069] The feature fusion module includes two channel attention submodules, two convolutional layers, and one spatial attention submodule. The inputs of the two channel attention submodules are the outputs of the noise feature extractor and the content feature extractor, respectively. The outputs of the two channel attention submodules are concatenated and combined, and then the fused features are extracted through the two convolutional layers. The fused features are input into the spatial attention submodule for processing, and the processed fused features are used as the output of the feature fusion module.

[0070] The feature regression module is built based on a convolutional layer and is used to perform convolution operations on the output of the feature fusion module to output a denoised MPI image.

[0071] To more clearly illustrate the feature fusion-based magnetic particle imaging image denoising method of the present invention, the steps of one embodiment of the method of the present invention will be described in detail below with reference to the accompanying drawings.

[0072] In the following embodiments, the structure and training process of the feature fusion denoising network model are first described in detail, and then the process of denoising the MPI image to be denoised by the feature fusion-based magnetic particle imaging image denoising method is described in detail.

[0073] 1. The structure and training process of the feature fusion denoising network model, such as... Figure 7 As shown

[0074] A100 acquires a noise-free MPI simulation image and a real noise image of the MPI device. The real noise image is randomly cropped and processed to obtain an amplified noise image. The image processing includes rotation, white balance transformation, and gamma transformation.

[0075] In this embodiment, an MPI simulation program is used to obtain a noise-free MPI simulation image. Real noise is then added to the MPI simulation image to obtain a noisy image. Based on this, a corresponding training dataset is constructed to train the model. Specifically:

[0076] A110, Simulation of Magnetic Particle Imaging Process: The MPI simulation program used in this invention mainly consists of four parts: data reading, image reconstruction, image post-processing, and image export. The original input data is a randomly generated particle distribution image, where the grayscale values ​​represent the magnetic particle concentration. After data reading, the corresponding voltage signal is generated according to the MPI principle, and the x-space reconstruction algorithm is used to reconstruct the image of the magnetic particle voltage signal.

[0077] A120, Construction of Real Noise Image Dataset: To obtain real noise images from MPI devices, a commercial MPI device was used for empty sampling. A phantom without injected magnetic particles was placed in the device, the default scanning mode was set, the average number of scans was 1, and the empty sampling of noise images began. The acquired noise images were randomly cropped to 128×128 pixels, and then subjected to image processing operations such as rotation, white balance transformation, and gamma transformation to amplify the samples and obtain amplified noise images.

[0078] A200 multiplies the amplified noisy image by a set intensity coefficient and superimposes it with the noise-free simulated MPI image to obtain an MPI image with real noise; a training dataset is constructed based on the MPI image with real noise and its corresponding ground truth image.

[0079] In this embodiment, the amplified noise sample (i.e., the noise image) is multiplied by a set intensity coefficient σ and then superimposed onto a noise-free simulated MPI image to obtain an MPI image with real noise. A training dataset is constructed based on the MPI image with real noise and its corresponding ground truth image. That is, the simulated image with superimposed device noise (i.e., the MPI image with real noise) is used as the network input, and the noise-free simulated image is used as the training ground truth image (i.e., the ground truth image). During training, this invention preferably divides the training dataset into a training set, a validation set, and a test set in a 7:1:2 ratio and inputs them into the model for training.

[0080] A300: Input the MPI images with real noise in the training dataset into the feature fusion denoising network model to obtain the denoised MPI images, which are used as the feature fusion learning output images. Also, obtain the features extracted by the noise feature extractor and the content feature extractor in the feature fusion denoising network model as noise features and content features, respectively.

[0081] In this embodiment, the feature fusion denoising network model includes a feature extraction module, a feature fusion module, and a feature regression module;

[0082] The feature extraction module consists of two feature extractors based on an autoencoder network architecture. These two feature extractors are constrained by a loss function, thus extracting noise features and content features from the MPI image, respectively. The two feature extractors are a noise feature extractor and a content feature extractor, as shown below. Figure 3 As shown; the inputs to both the noise feature extractor and the content feature extractor are noisy MPI images;

[0083] like Figure 4As shown, both the noise feature extractor and the content feature extractor are constructed based on three downsampling layers and three upsampling layers. After each downsampling, the size of the output feature map is half the size of the input feature map. After each upsampling, the size of the output feature map is twice the size of the input feature map, to achieve multi-scale feature extraction (e.g., Figure 4 (As shown in 1, 1 / 2, 1 / 4); the three downsampling layers are respectively the first downsampling layer, the second downsampling layer, and the third downsampling layer; the three upsampling layers are respectively the first upsampling layer, the second upsampling layer, and the third upsampling layer; each of the three downsampling layers and the three upsampling layers is composed of 3×3 convolution operation, batch normalization and Dropout operation (i.e., connected in sequence).

[0084] The first downsampling layer, the second downsampling layer, the third downsampling layer, the first upsampling layer, the second upsampling layer, and the third upsampling layer are connected sequentially; the first downsampling layer and the third downsampling layer, and the second downsampling layer and the second upsampling layer are connected across layers through a convolutional attention module;

[0085] like Figure 5 As shown, the convolutional attention module is constructed based on a channel attention submodule and a spatial attention submodule connected in sequence;

[0086] The channel attention submodule is used to perform average pooling on the input features, multiply the features before average pooling with the features after average pooling, and use the result of the multiplication as the output of the channel attention submodule.

[0087] The spatial attention submodule is used to convolve the input features, perform max pooling on the convolved features, multiply the features before max pooling with the features after max pooling, and use the result of the multiplication as the output of the spatial attention submodule.

[0088] In this way, downsampling is performed through convolution operations to retain more features. Finally, a convolutional attention mechanism is used to combine low-order detail features and high-order semantic features to improve the representational ability of the extractor.

[0089] like Figure 6 As shown, the feature fusion module includes two channel attention submodules, two convolutional layers, and one spatial attention submodule. The inputs to the two channel attention submodules are the outputs of the noise feature extractor and the content feature extractor, respectively, used to filter out important feature channels. Then, the outputs of the two channel attention submodules are concatenated and combined (i.e.,...). Figure 6(Connected by channel), extracting fused features through two convolutional layers, and finally inputting the fused features into the spatial attention submodule for processing to determine the most important region in the feature map, and using the processed fused features as the output of the feature fusion module;

[0090] The feature regression module is built on a convolutional layer and is used to perform convolution operations on the output of the feature fusion module to output a denoised MPI image, thus realizing the conversion from fused features to a predicted image.

[0091] The MPI images with real noise in the training dataset are input into the feature fusion denoising network model to obtain the denoised MPI images, which are used as the feature fusion learning output images. The features extracted by the noise feature extractor and the content feature extractor in the feature fusion denoising network model are obtained as noise features and content features, respectively.

[0092] A400 processes the noise features using a pre-built noise feature regressor, and then subtracts the processed noise features from the MPI image with real noise input to the feature fusion denoising network model to obtain a noise learning output image; the content features are processed using a pre-built content feature regressor to obtain a content learning output image; both the noise feature regressor and the content feature regressor are built based on convolutional layers;

[0093] In this embodiment, the noise features are processed by a pre-built noise feature regressor, and the difference between the processed noise features and the MPI image with real noise input to the feature fusion denoising network model is calculated to obtain a noise learning output image; the content features are processed by a pre-built content feature regressor to obtain a content learning output image, such as... Figure 3 As shown.

[0094] A500 updates the model parameters of the feature fusion denoising network model by calculating the loss value through a pre-constructed loss function based on the noise learning output image, the content learning output image, and the feature fusion learning output image, combined with the ground truth image.

[0095] In this embodiment, the pre-constructed loss function is:

[0096]

[0097] Among them, I content I1 represents the ground truth image, and I2, I3 represent the noise learning output image, the feature fusion learning output image, and the content learning output image, respectively.

[0098] A600, repeat steps A300-A500 until a trained feature fusion and denoising network model is obtained.

[0099] In this embodiment, the feature fusion denoising network model is trained cyclically until a well-trained feature fusion denoising network model is obtained.

[0100] 2. A method for denoising magnetic particle imaging images based on feature fusion

[0101] S100: Acquire the MPI image to be denoised as the input image.

[0102] In this embodiment, the MPI image to be denoised is acquired.

[0103] S200, the input image is denoised based on a pre-trained feature fusion denoising network model to obtain a denoised MPI image.

[0104] In this embodiment, the MPI image to be denoised is input into a trained feature fusion denoising network model for denoising, resulting in a denoised MPI image.

[0105] A second embodiment of the present invention provides a magnetic particle imaging image denoising system based on feature fusion, such as... Figure 2 As shown, the system includes: an image acquisition module 100 and an image denoising module 200;

[0106] The image acquisition module 100 is configured to acquire the MPI image to be denoised as the input image;

[0107] The image denoising module 200 is configured to perform denoising processing on the input image based on a pre-trained feature fusion denoising network model to obtain a denoised MPI image.

[0108] The feature fusion denoising network model includes a feature extraction module, a feature fusion module, and a feature regression module.

[0109] The feature extraction module includes a noise feature extractor and a content feature extractor; the inputs to both the noise feature extractor and the content feature extractor are noisy MPI images.

[0110] Both the noise feature extractor and the content feature extractor are constructed based on three downsampling layers and three upsampling layers; the three downsampling layers are respectively the first downsampling layer, the second downsampling layer, and the third downsampling layer; the three upsampling layers are respectively the first upsampling layer, the second upsampling layer, and the third upsampling layer.

[0111] The first downsampling layer, the second downsampling layer, the third downsampling layer, the first upsampling layer, the second upsampling layer, and the third upsampling layer are connected sequentially; the first downsampling layer and the third downsampling layer, and the second downsampling layer and the second upsampling layer are connected across layers through a convolutional attention module;

[0112] The convolutional attention module is constructed based on a sequentially connected channel attention submodule and spatial attention submodule;

[0113] The channel attention submodule is used to perform average pooling on the input features, multiply the features before average pooling with the features after average pooling, and use the result of the multiplication as the output of the channel attention submodule.

[0114] The spatial attention submodule is used to convolve the input features, perform max pooling on the convolved features, multiply the features before max pooling with the features after max pooling, and use the result of the multiplication as the output of the spatial attention submodule.

[0115] The feature fusion module includes two channel attention submodules, two convolutional layers, and one spatial attention submodule. The inputs of the two channel attention submodules are the outputs of the noise feature extractor and the content feature extractor, respectively. The outputs of the two channel attention submodules are concatenated and combined, and then the fused features are extracted through the two convolutional layers. The fused features are input into the spatial attention submodule for processing, and the processed fused features are used as the output of the feature fusion module.

[0116] The feature regression module is built based on a convolutional layer and is used to perform convolution operations on the output of the feature fusion module to output a denoised MPI image.

[0117] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process and related descriptions of the system described above can be found in the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0118] It should be noted that the feature fusion-based magnetic particle imaging image denoising system provided in the above embodiments is only an example of the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the modules or steps in the embodiments of the present invention can be further decomposed or combined. For example, the modules in the above embodiments can be merged into one module, or further divided into multiple sub-modules to complete all or part of the functions described above. The names of the modules and steps involved in the embodiments of the present invention are only for distinguishing the various modules or steps and are not considered as an improper limitation of the present invention.

[0119] A third embodiment of the present invention provides an electronic device comprising at least one processor and a memory communicatively connected to at least one of the processors; wherein the memory stores instructions executable by the processor to implement the feature fusion-based magnetic particle imaging image denoising method of claim 1.

[0120] A computer-readable storage medium according to a fourth embodiment of the present invention stores computer instructions for execution by a computer to implement the feature fusion-based magnetic particle imaging image denoising method of the claims above.

[0121] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process and related descriptions of the electronic devices and computer-readable storage media described above can be referred to the corresponding processes in the foregoing method examples, and will not be repeated here.

[0122] The following is for reference. Figure 8 It shows a schematic diagram of the structure of a computer system suitable for implementing the system, method, and device embodiments of this application. Figure 8 The server shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0123] like Figure 8 As shown, the computer system includes a Central Processing Unit (CPU) 801, which can perform various appropriate actions and processes based on programs stored in Read Only Memory (ROM) 802 or programs loaded from storage section 808 into Random Access Memory (RAM) 803. RAM 803 also stores various programs and data required for system operation. The CPU 801, ROM 802, and RAM 803 are interconnected via bus 804. An Input / Output (I / O) interface 805 is also connected to bus 804.

[0124] The following components are connected to I / O interface 805: an input section 806 including a keyboard, mouse, etc.; an output section 807 including a cathode ray tube, liquid crystal display, etc., and speakers, etc.; a storage section 808 including a hard disk, etc.; and a communication section 809 including a network interface card, such as a local area network card, modem, etc. The communication section 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to I / O interface 805 as needed. A removable medium 811, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 810 as needed so that computer programs read from it can be installed into storage section 808 as needed.

[0125] Specifically, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 809, and / or installed from removable medium 811. When the computer program is executed by CPU 801, it performs the functions defined in the methods of this application. It should be noted that the computer-readable medium described above in this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, RAM, ROM, erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on a computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, etc., or any suitable combination thereof.

[0126] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0127] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0128] The terms “first”, “second”, etc., are used to distinguish similar objects, not to describe or indicate a specific order or sequence.

[0129] The term "comprising" or any other similar term is intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus / device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent in such process, method, article, or apparatus / device.

[0130] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after such changes or substitutions will all fall within the scope of protection of the present invention.

Claims

1. A method for denoising magnetic particle imaging images based on feature fusion, characterized in that, The method includes: S100: Acquire the MPI image to be denoised as the input image; S200, the input image is denoised based on a pre-trained feature fusion denoising network model to obtain a denoised MPI image; The feature fusion denoising network model includes a feature extraction module, a feature fusion module, and a feature regression module. The feature extraction module includes a noise feature extractor and a content feature extractor; the inputs to both the noise feature extractor and the content feature extractor are noisy MPI images. Both the noise feature extractor and the content feature extractor are constructed based on three downsampling layers and three upsampling layers; the three downsampling layers are respectively the first downsampling layer, the second downsampling layer, and the third downsampling layer; the three upsampling layers are respectively the first upsampling layer, the second upsampling layer, and the third upsampling layer. The first downsampling layer, the second downsampling layer, the third downsampling layer, the first upsampling layer, the second upsampling layer, and the third upsampling layer are connected sequentially; the first downsampling layer and the third upsampling layer, and the second downsampling layer and the second upsampling layer are connected across layers through a convolutional attention module; The convolutional attention module is constructed based on a sequentially connected channel attention submodule and spatial attention submodule; The channel attention submodule is used to perform average pooling on the input features, multiply the features before average pooling with the features after average pooling, and use the result of the multiplication as the output of the channel attention submodule. The spatial attention submodule is used to convolve the input features, perform max pooling on the convolved features, multiply the features before max pooling with the features after max pooling, and use the result of the multiplication as the output of the spatial attention submodule. The feature fusion module includes two channel attention submodules, two convolutional layers, and one spatial attention submodule. The inputs of the two channel attention submodules are the outputs of the noise feature extractor and the content feature extractor, respectively. The outputs of the two channel attention submodules are concatenated and combined, and then the fused features are extracted through the two convolutional layers. The fused features are input into the spatial attention submodule for processing, and the processed fused features are used as the output of the feature fusion module. The feature regression module is built based on a convolutional layer and is used to perform convolution operations on the output of the feature fusion module to output a denoised MPI image.

2. The method for denoising magnetic particle imaging images based on feature fusion according to claim 1, characterized in that, Each of the three downsampling layers and three upsampling layers is based on 3×3 convolution operations, batch normalization, and Dropout operations.

3. The method for denoising magnetic particle imaging images based on feature fusion according to claim 2, characterized in that, The training method for the feature fusion denoising network model is as follows: A100 acquires a noise-free MPI simulation image and a real noise image of the MPI device. The real noise image is randomly cropped and processed to obtain an amplified noise image. The image processing includes rotation, white balance transformation, and gamma transformation. A200 multiplies the amplified noisy image by a set intensity coefficient and superimposes it with the noise-free simulated MPI image to obtain an MPI image with real noise. A training dataset is constructed based on MPI images with real noise and their corresponding ground truth images; A300: Input the MPI images with real noise in the training dataset into the feature fusion denoising network model to obtain the denoised MPI images, which are used as the feature fusion learning output images. Also, obtain the features extracted by the noise feature extractor and the content feature extractor in the feature fusion denoising network model as noise features and content features, respectively. A400 processes the noise features using a pre-built noise feature regressor, and then subtracts the processed noise features from the MPI image with real noise input to the feature fusion denoising network model to obtain a noise learning output image; the content features are processed using a pre-built content feature regressor to obtain a content learning output image; both the noise feature regressor and the content feature regressor are built based on convolutional layers; A500 updates the model parameters of the feature fusion denoising network model by calculating the loss value through a pre-constructed loss function based on the noise learning output image, the content learning output image, and the feature fusion learning output image, combined with the ground truth image. A600, repeat steps A300-A500 until a trained feature fusion and denoising network model is obtained.

4. The method for denoising magnetic particle imaging images based on feature fusion according to claim 3, characterized in that, The noise-free MPI simulation image is obtained by the following method: A particle distribution image is randomly generated; the grayscale value in the particle distribution image represents the concentration of magnetic particles. The voltage signal corresponding to the particle distribution image is generated according to the MPI principle, and the image is reconstructed using the x-space reconstruction algorithm to obtain a noise-free MPI simulation image.

5. The method for denoising magnetic particle imaging images based on feature fusion according to claim 3, characterized in that, The method for obtaining the true noise image of the MPI device is as follows: a phantom without injected magnetic particles is placed into the MPI device for air sampling to obtain the true noise image of the MPI device.

6. The method for denoising magnetic particle imaging images based on feature fusion according to claim 3, characterized in that, The pre-constructed loss function is: ； in, For the true value image, These are the noise learning output image, the feature fusion learning output image, and the content learning output image, respectively.

7. A magnetic particle imaging image denoising system based on feature fusion, characterized in that, The system includes: an image acquisition module and an image denoising module; The image acquisition module is configured to acquire the MPI image to be denoised as the input image; The image denoising module is configured to perform denoising processing on the input image based on a pre-trained feature fusion denoising network model to obtain a denoised MPI image. The feature fusion denoising network model includes a feature extraction module, a feature fusion module, and a feature regression module. The feature extraction module includes a noise feature extractor and a content feature extractor; the inputs to both the noise feature extractor and the content feature extractor are noisy MPI images. Both the noise feature extractor and the content feature extractor are constructed based on three downsampling layers and three upsampling layers; the three downsampling layers are respectively the first downsampling layer, the second downsampling layer, and the third downsampling layer; the three upsampling layers are respectively the first upsampling layer, the second upsampling layer, and the third upsampling layer. The first downsampling layer, the second downsampling layer, the third downsampling layer, the first upsampling layer, the second upsampling layer, and the third upsampling layer are connected sequentially; the first downsampling layer and the third upsampling layer, and the second downsampling layer and the second upsampling layer are connected across layers through a convolutional attention module; The convolutional attention module is constructed based on a sequentially connected channel attention submodule and spatial attention submodule; The channel attention submodule is used to perform average pooling on the input features, multiply the features before average pooling with the features after average pooling, and use the result of the multiplication as the output of the channel attention submodule. The spatial attention submodule is used to convolve the input features, perform max pooling on the convolved features, multiply the features before max pooling with the features after max pooling, and use the result of the multiplication as the output of the spatial attention submodule. The feature fusion module includes two channel attention submodules, two convolutional layers, and one spatial attention submodule. The inputs of the two channel attention submodules are the outputs of the noise feature extractor and the content feature extractor, respectively. The outputs of the two channel attention submodules are concatenated and combined, and then the fused features are extracted through the two convolutional layers. The fused features are input into the spatial attention submodule for processing, and the processed fused features are used as the output of the feature fusion module. The feature regression module is built based on a convolutional layer and is used to perform convolution operations on the output of the feature fusion module to output a denoised MPI image.

8. An electronic device, characterized in that, include: At least one processor; and a memory communicatively connected to at least one of the processors; The memory stores instructions that can be executed by the processor to implement the feature fusion-based magnetic particle imaging image denoising method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that are executed by a computer to implement the feature fusion-based magnetic particle imaging image denoising method according to any one of claims 1-6.

Images