Magnetic particle image reconstruction method and system based on multi-mode diffusion model

Abstract

The invention belongs to the field of image reconstruction of magnetic particle imaging and multi-modal generative artificial intelligence models, particularly relates to a magnetic particle image reconstruction method and system based on a multi-modal diffusion model, and aims to solve the problem that there is no method capable of combining a generative model and a multi-modal imaging technology to realize accurate quantitative reconstruction of MPI. The method comprises the following steps: carrying out integral modeling and discretization operation on a first excitation signal to obtain discrete signals, fitting and superposing the discrete signals to obtain a first response signal, calculating a loss function, and training a model based on the loss function; processing the second time sequence signal by using the trained particle response model to obtain a second response signal; calculating based on the second response signal and the system matrix, gradually adding noise, and constraining to obtain a noise constrained image; and gradually denoising and recovering the noise constraint image in the last step to obtain a final reconstructed image. According to the invention, rapid and high-quality MPI image reconstruction becomes possible.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of image reconstruction and multi-modal generative artificial intelligence model of magnetic particle imaging, and particularly relates to a magnetic particle image reconstruction method and system based on a multi-modal diffusion model. BACKGROUND

[0002] Magnetic particle imaging (MPI) is a new medical imaging technology that has been widely used in various biomedical fields. MPI has two main reconstruction algorithms: system matrix inverse problem-based reconstruction and X-space mapping-based reconstruction. The X-space-based algorithm is faster in reconstruction, but the image quality and quantification are poor; the system matrix-based reconstruction algorithm is better in image resolution and quality, and thus has become the mainstream imaging method. The difference between different system matrix reconstruction algorithms mainly lies in the design of the regularization term, which aims to alleviate the ill-conditioned nature of inverse problem solving. However, the system matrix-based reconstruction method relies on a large amount of prior information and needs to obtain the reconstruction result through a complex solving process. This kind of method often involves many parameters, resulting in poor robustness of reconstruction and being easily affected by parameter setting and noise, thereby affecting the image quality; in recent years, deep learning technology has also gradually explored in the field of MPI image reconstruction. Some studies have used fully connected layers and UNet-based models for MPI image reconstruction, and verified the feasibility of deep learning technology in MPI image reconstruction on simulated data. At the same time, another study has used a deep learning model to fit the regularization term, which has achieved higher quality image reconstruction in real data compared to the traditional fixed form regularization term. In addition, some image post-processing algorithms have been proposed to improve the quality of system matrix reconstruction images. Although existing deep learning algorithms improve the visual quality of images, they are usually end-to-end techniques, lacking the combination with the actual physical process. This leads to the reconstructed image being too sharp, which has a large error with the real magnetic particle concentration distribution, affecting the quantitative accuracy of MPI image reconstruction. The image reconstruction of MPI is essentially solving a linear inverse problem. In recent years, generative artificial intelligence models (such as diffusion models) have shown significant potential in solving inverse problems in medical imaging. At the same time, multi-modal fusion technology has played an important role in various medical imaging problems. However, there is currently no accurate and quantitative reconstruction algorithm that can combine generative models and multi-modal imaging technology to achieve accurate and quantitative reconstruction of MPI. SUMMARY

[0003] To address the aforementioned problems in existing technologies—namely, the lack of an accurate and quantitative reconstruction algorithm that combines generative models with multimodal imaging techniques to achieve precise quantitative reconstruction of magnetic particle images (MPI)—this invention provides a magnetic particle image reconstruction method based on a multimodal diffusion model. The method includes: A first excitation signal is acquired, and an integral model is performed on the first excitation signal to obtain a first time-series signal. The first time-series signal is discretized to obtain a discrete signal. Each discrete signal is fitted and then superimposed to obtain a first response signal. A loss function is calculated based on the first response signal and the first time-series signal. A particle response model is trained based on the loss function to obtain a trained particle response model. The second response signal is obtained by processing the second time-series signal corresponding to the second excitation signal obtained by integral modeling using the trained particle response model. The reconstructed image is obtained by calculating the second response signal and the system matrix. The reconstructed image is then gradually noise-added and constrained to obtain a noise-constrained image. The final reconstructed image is obtained by progressively denoising and restoring the noise-constrained image based on Bayes' theorem and Gaussian distribution.

[0004] In a preferred embodiment, obtaining a first time-series signal by integral modeling of the first excitation signal includes: ; in, It's temperature. It's viscosity. It is Boltzmann's constant. The magnetic moment per unit particle. As the first excitation signal, It is the first timing signal. It is the spatial distribution of particles. It is a particle response model. This indicates the particle response to the first excitation signal. t It is time. r It represents a spatial location.

[0005] In a preferred embodiment, the discrete signal is: ; in, For position i discrete signals, For position i The spatial distribution of particles, where i is the position. It is a discrete signal.

[0006] In a preferred embodiment, the first response signal is obtained by fitting and superimposing the individual discrete signals separately, including: Utilizing a first neural operator Fitting the discrete signals for each position yields a response signal for each position; ; wherein, y is the response signal for position ; is a first neural operator; is the temperature, is the viscosity, is the Boltzmann constant, is the unit particle magnetic moment, is the discrete signal for position i ; Superimposing the response signals for each position and processing by a second neural operator yields a first response signal: ; wherein, is the first response signal, is a second neural operator.

[0007] In a preferred embodiment, calculating a loss function based on the first response signal and a first temporal signal comprises: calculating a loss function based on the first response signal and a first temporal signal comprises: ; wherein, is the loss function, is a frequency domain representation of the first response signal after Fourier transformation, is a representation of the first temporal signal after Fourier transformation.

[0008] In a preferred embodiment, calculating a loss function based on the first response signal and a first temporal signal comprises: ; wherein, u is a Fourier transform of the second response signal, S is a system matrix obtained by measurement, is the reconstructed image.

[0009] Stepwise adding noise to the reconstructed image and constraining yields a noise constrained image for each step comprises: for the th step, adding noise to the reconstructed image yields a noise image; ; ; wherein, is the tNoisy image in step 1; t 1 represents the number of time steps. Standard Gaussian noise from random sampling; It is a noise estimation model; Then, the constraint parameters are used to constrain the noise image at step t1 to obtain the constrained noise image, specifically: ; in, These are pre-defined hyperparameters. u Let S be the Fourier transform of the second response signal, and S be the system matrix obtained through measurement. yes The pseudo-inverse matrix, I It is the identity matrix. For the first t The noise-constrained image after step 1, where F is the intermediate feature. The first constraint parameter, This is the second constraint parameter.

[0010] In a preferred embodiment, the constraint parameters include a first constraint parameter and a second constraint parameter; The first constraint parameter is: calculate the average and maximum values ​​of each pixel of the intermediate feature F in the channel dimension to obtain two new feature maps with 1 channel. Then, concatenate the two new feature maps with 1 channel to obtain the first feature map. Input the first feature map into a convolutional layer with a kernel size of 7 to obtain the convolutional feature. Finally, activate the convolutional feature using the sigmoid function. Finally, the first constraint parameter is calculated. ; The second constraint parameter is obtained by inputting the intermediate features into a convolutional layer with a kernel size of 3. ; in, A convolutional layer with a kernel size of 3. The maximum value of each pixel in the intermediate feature F. The average value of each pixel in the intermediate feature F is... A convolutional layer with a kernel size of 7.

[0011] In a preferred embodiment, performing stepwise denoising and restoration to obtain the final reconstructed image includes: For the t Step 1 to the t Step 1-1: ; ; in, It is the image distribution during the noise-addition process in forward diffusion. It is a forward diffusion process that adds noise. Standard Gaussian noise is randomly sampled. for t 1 Distribution of step images, I It is the identity matrix. The distribution of the image at step t1-1, This is multimodal structural imaging data; p (.) represents the distribution. For the first Noisy images of the steps, For the constrained noise image at step t-1, For the constrained noise image at step t1, These are pre-defined hyperparameters. u Let S be the Fourier transform of the second response signal, and S be the system matrix obtained through measurement. The first constraint parameter, Here, F is the second constraint parameter, and F is the intermediate feature. yes The pseudo-inverse matrix, It is the first t Noisy image in step 1; It is the first diffusion parameter of the model. It is the second diffusion parameter of the model.

[0012] A second aspect of the present invention proposes a magnetic particle image reconstruction system based on a multimodal diffusion model, the system comprising: The data acquisition module is used to acquire a first excitation signal, perform integral modeling on the first excitation signal to obtain a first time-series signal, discretize the first time-series signal to obtain a discrete signal, fit and superimpose each discrete signal to obtain a first response signal, calculate a loss function based on the first response signal and the first time-series signal, and train a particle response model based on the loss function to obtain a trained particle response model. The signal processing module is used to process the second time-series signal corresponding to the second excitation signal obtained by integral modeling using the trained particle response model to obtain the second response signal. The image noise constraint module is used to calculate the reconstructed image based on the second response signal and the system matrix, and to gradually add noise and constrain the reconstructed image to obtain a noise-constrained image. The image denoising module is used to progressively denoise and restore the noise-constrained image in the final step based on Bayes' theorem and Gaussian distribution to obtain the final reconstructed image.

[0013] The beneficial effects of this invention are: (1) The application generates more accurate first response signals and second response signals by processing the excitation signal using the trained particle response model. This helps to overcome the image quality problems caused by parameter setting and noise in traditional system matrix-based reconstruction algorithms, thereby improving the quality and resolution of the final reconstructed image; (2) The application combines generative artificial intelligence models (such as diffusion models) and multi-modal imaging technology. This method can more accurately reflect the real magnetic particle concentration distribution and provide more realistic and accurate quantitative reconstruction results. Through steps such as integral modeling, discretization and fitting of the first time signal, and particle response model training based on loss function, this method provides a new approach to solving linear inverse problems. In particular, by gradually adding noise, constraint to obtain a noise-constrained image, and using the Bayes theorem and Gaussian distribution iterative denoising to recover the image, the stability and robustness of the inverse problem solving are enhanced; (3) The reconstruction algorithm based on the system matrix in the application has advantages in image quality. The deep learning model is used to fit the regularization term and the generative model is used to process the inverse problem, which not only improves the quality of image reconstruction, but also improves the calculation efficiency, making it possible to quickly and high-quality MPI image reconstruction. By integrating the latest generative artificial intelligence model with traditional MPI reconstruction methods, the application aims to achieve more accurate and quantitative MPI image reconstruction, while solving the problems of poor image quality, insufficient quantitative accuracy and low calculation efficiency in existing methods. BRIEF DESCRIPTION OF DRAWINGS

[0014] Other features, objects and advantages of the application will become more apparent from the following detailed description of non-limiting embodiments, made with reference to the accompanying drawings: Figure 1 is a schematic diagram of a magnetic particle image reconstruction method based on a multi-modal diffusion model according to an embodiment of the application; Figure 2 is an image reconstruction model diagram based on multi-modal model diffusion according to an embodiment of the application; Figure 3 is an image reconstruction model diagram based on multi-modal diffusion of zero value range decomposition according to an embodiment of the application; Figure 4 is a structural schematic diagram of a computer system of a server for implementing the method, system and device embodiments of the application. DETAILED DESCRIPTION

[0015] The application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only intended to explain the related application, and not to limit the application. In addition, it should be noted that only the parts related to the application are shown in the drawings for ease of description.

[0016] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict. The present application will be described in detail below with reference to the drawings and in combination with the embodiments.

[0017] The present application provides a magnetic particle image reconstruction method based on a multi-modal diffusion model, the method comprising: obtaining a first excitation signal and a second excitation signal, respectively integrating and modeling the first excitation signal and the second excitation signal to obtain a first time sequence signal and a second time sequence signal, discretizing the first time sequence signal to obtain a discrete signal, respectively fitting and superimposing each discrete signal to obtain a first response signal, calculating a loss function based on the first response signal and the first time sequence signal, training a particle response model based on the loss function to obtain a trained particle response model; using the trained particle response model to process the second time sequence signal to obtain a second response signal; based on the second response signal and a system matrix, a reconstructed image is calculated, and the reconstructed image is gradually denoised and constrained to obtain a noise-constrained image; based on the Bayesian theorem and Gaussian distribution, the noise-constrained image of the last step is gradually denoised and recovered to obtain a final reconstructed image.

[0018] In order to more clearly illustrate the magnetic particle image reconstruction method based on the multi-modal diffusion model of the present application, the following will be combined with Figure 1 The steps in the embodiments of the present application will be described in detail.

[0019] The magnetic particle image reconstruction method based on the multi-modal diffusion model of the first embodiment of the present application is described in detail as follows: obtaining a first excitation signal and a second excitation signal, respectively integrating and modeling the first excitation signal and the second excitation signal to obtain a first time sequence signal and a second time sequence signal, discretizing the first time sequence signal to obtain a discrete signal, respectively fitting and superimposing each discrete signal to obtain a first response signal, calculating a loss function based on the first response signal and the first time sequence signal, training a particle response model based on the loss function to obtain a trained particle response model; In the present embodiment, the first excitation signal is integrated and modeled to obtain the first time sequence signal, which comprises: ; wherein, is the temperature, is the viscosity, is the Boltzmann constant, is the unit particle magnetic moment, and the device parameters include the magnetic field strength and the frequency, which are embodied in the excitation signal . The particle parameters include a unit particle magnetic moment, which is related to a diameter and a saturation magnetization of the particle. The device parameters include a magnetic field strength and a frequency, which are embodied in the excitation signal.

[0020] The particle response model is a core factor affecting the accuracy of the generated data of MPI. Although many different response models have been proposed to describe this relationship, such as the Langevin model and the Debye model, the underlying motion and response of the magnetic particle are very complex, and the existing fixed mathematical form model is difficult to accurately describe, resulting in a large difference between the simulated signal and the actual measured signal. This mismatch will have a negative impact on the quality of the generated data, and then affect the training accuracy and generalization ability of the model. In order to solve this problem, the present application adopts a data-driven idea, and fits the response model of the particle through FNO, instead of setting a specific mathematical model, and the specific idea is as follows.

[0021] In the present embodiment, the discrete signals are: ; wherein, is a discrete signal of a position i , is a spatial distribution of the particle at a position , i is a position, i is a discrete signal.

[0022] The first response signal is obtained by fitting each discrete signal and then superimposing them, and includes: The first neural operator is used to fit the discrete signal of each position to obtain the response signal of each position; ; wherein, is a response signal of a position ; is the first neural operator; is a temperature, is a viscosity, is a Boltzmann constant, is a unit particle magnetic moment, is a discrete signal of a position i ; The first response signal is obtained by superimposing the response signal of each position and then processing it by the second neural operator: ; wherein, is the first response signal, is the second neural operator.

[0023] ​In the embodiment, the loss function is calculated based on the first response signal and the first time sequence signal, and the loss function comprises: ; wherein, is the loss function, is a frequency domain representation of the first response signal after Fourier transform, is a representation of the first time sequence signal after Fourier transform.

[0024] After the training is completed, a corresponding phantom is constructed based on a magnetic resonance imaging (MRI) dataset IXI, digital phantoms of different magnetic particle concentration distributions and corresponding structural imaging data are obtained, and then a response signal corresponding to the digital phantoms is generated by using the trained system, so that a large number of multi-modal data pairs close to the real data distribution are obtained for subsequent training and optimization of a multi-modal diffusion reconstruction model of an image reconstruction model, as shown in Figure 2 .

[0025] The second response signal is obtained by processing the second time sequence signal by using the trained particle response model; The reconstructed image is obtained based on the second response signal and the system matrix, and the noise constraint image is obtained by gradually adding noise and constraint to the reconstructed image; In the embodiment, the noise constraint image of each step is obtained by gradually adding noise and constraint to the reconstructed image based on the second response signal and the system matrix, and the noise constraint image of each step comprises: The reconstruction algorithm based on the system matrix is essentially to solve an inverse problem wherein, is a frequency domain representation of the received signal after Fourier transform, is a measured system matrix, c 0 is a to-be-solved reconstructed image. The purpose of the forward diffusion is to change the reconstructed image c into Gaussian noise , and the process can be represented as . The core of the forward diffusion is to train a noise estimation model to predict the noise added to the original image in each step, and the reconstructed image is calculated based on the second response signal and the system matrix, and the noise constraint image of each step comprises: ; wherein, u is the Fourier transform of the second response signal, S is a measured system matrix, is the reconstructed image.

[0026] The noise constraint image of each step is obtained by gradually adding noise and constraint to the reconstructed image, and the noise constraint image of each step comprises: For the first step, the noise image is obtained by adding noise to the reconstructed image; ; ; in, For the first t Noisy image in step 1; t 1 represents the number of time steps. Standard Gaussian noise from random sampling; It is a noise estimation model; in, The hyperparameters are fixed in advance to control the noise intensity at each step, and is randomly sampled standard Gaussian noise. However, this diffusion process is inherently unconditional, meaning it lacks constraints and correspondence with actual physical information. To integrate measured physical information with the diffusion process, this invention redesigns the diffusion generation process based on the idea of ​​zero-domain decomposition, enabling the introduction of measured physical information during the diffusion process. This improved method not only maintains the generative capability of the diffusion model but also enhances the physical consistency and accuracy of the reconstructed image. u The Fourier transform of the second response signal; Will replace This invention, used in the forward diffusion process, adjusts the original distribution to a conditional distribution based on the system matrix and the received signal, achieving a correspondence between image diffusion and measured physical information. This invention further constrains the diffusion process using multimodal structural imaging data. Specifically, this invention utilizes a pre-trained fixed-parameter model, VGG, to encode intermediate features, and finally implements constraints through a conditional modulation network. The intermediate image of the above iterative process can be further represented as: The constrained noise image is obtained by constraining the noise image at step t1 using constraint parameters, specifically as follows: The constrained noise image is obtained by constraining the noise image at step t1 using constraint parameters, specifically as follows: ; in, These are pre-defined hyperparameters. u Let S be the Fourier transform of the second response signal, and S be the system matrix obtained through measurement. yes The pseudo-inverse matrix, I It is the identity matrix. For the first t The noise-constrained image after step 1, where F is the intermediate feature. The first constraint parameter, This is the second constraint parameter.

[0027] In this embodiment, the constraint parameters include a first constraint parameter and a second constraint parameter; The first constraint parameter is: calculate the average and maximum values ​​of each pixel of the intermediate feature F in the channel dimension to obtain two new feature maps with 1 channel. Then, concatenate the two new feature maps with 1 channel to obtain the first feature map. Input the first feature map into a convolutional layer with a kernel size of 7 to obtain the convolutional feature. Finally, activate the convolutional feature using the sigmoid function. Finally, the first constraint parameter is calculated. ; The second constraint parameter is obtained by inputting the intermediate features into a convolutional layer with a kernel size of 3. ; in, A convolutional layer with a kernel size of 3. The maximum value of each pixel in the intermediate feature F. The average value of each pixel in the intermediate feature F is... A convolutional layer with a kernel size of 7.

[0028] The final reconstructed image is obtained by progressively denoising and restoring the noise-constrained image based on Bayes' theorem and Gaussian distribution.

[0029] For the t Step 1 to the t Step 1-1: In this embodiment, the reverse reconstruction process involves iteratively denoising the noisy image to gradually recover the final reconstructed image. Its core lies in constructing... arrive The recursive formula, i.e., the distribution Using Bayes' theorem, this distribution can be decomposed into: ; Where p is the probability; During the forward diffusion process, this invention uses an image decomposed from the zero-domain. Simultaneously, constraints from measured physical information and structural imaging data need to be added during the reverse reconstruction process. This formula can be written as: ; All three distributions can be modeled by Gaussian distributions. Through parameter renormalization, the recursive formula for inverse reconstruction can be obtained as follows: ; in, It is the image distribution during the noise-addition process in forward diffusion. It is a forward diffusion process that adds noise. Standard Gaussian noise is randomly sampled. fort 1 a distribution of the step image, I is a unit matrix, is a distribution of the t1-1 step image, is a multi-modal structural imaging data; p is a distribution representation, is a tth step noise image, is a t-1th step noise image, is a t1th step noise image, is a pre-fixed hyper parameter, is a Fourier transform of the second response signal, S is a measured system matrix, u is a first constraint parameter, is a second constraint parameter, F is an intermediate feature, is a pseudo-inverse matrix of is a tth step noise image; is a model first diffusion parameter, is a model second diffusion parameter. t Thus, the inverse process can generate a corresponding MPI high-quality image by randomly generating a noise According to the above iterative formula, under the condition constraints of the measured physical information and the structural image, the corresponding MPI high-quality image is gradually generated. The diffusion and reconstruction model image can be seen as follows .

[0030] The above embodiment describes each step in the above order, but those skilled in the art can understand that, in order to achieve the effect of the embodiment, the different steps do not have to be executed in such an order, and they can be executed simultaneously (in parallel) or in a reversed order, and these simple changes are within the protection scope of the present application. Figure 3 The second embodiment of the present application is a magnetic particle image reconstruction system based on a multi-modal diffusion model, which comprises: A data acquisition module is used to acquire a first excitation signal and a second excitation signal, to obtain a first time sequence signal and a second time sequence signal by integrating modeling the first excitation signal and the second excitation signal respectively, to obtain a discrete signal by discretizing the first time sequence signal, and to obtain a first response signal by fitting and superimposing each discrete signal respectively, to calculate a loss function based on the first response signal and the first time sequence signal, and to obtain a trained particle response model by training the particle response model based on the loss function;

[0031] A signal processing module is used to process the second time sequence signal to obtain a second response signal by using the trained particle response model.

[0032] The second embodiment of the present application is a magnetic particle image reconstruction system based on a multi-modal diffusion model, which comprises: A data acquisition module is used to acquire a first excitation signal and a second excitation signal, to obtain a first time sequence signal and a second time sequence signal by integrating modeling the first excitation signal and the second excitation signal respectively, to obtain a discrete signal by discretizing the first time sequence signal, and to obtain a first response signal by fitting and superimposing each discrete signal respectively, to calculate a loss function based on the first response signal and the first time sequence signal, and to obtain a trained particle response model by training the particle response model based on the loss function; A signal processing module is used to process the second time sequence signal to obtain a second response signal by using the trained particle response model. The image noise adding constraint module is configured to calculate a reconstructed image based on the second response signal and the system matrix, and to add noise to the reconstructed image step by step to obtain a noise constraint image. The image denoising module is configured to denoise the noise constraint image of the last step based on the Bayes theorem and the Gaussian distribution to obtain a final reconstructed image.

[0033] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system and the related description described above can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

[0034] It should be noted that the magnetic particle image reconstruction system based on the multi-modal diffusion model provided in the above embodiments is only exemplified by the division of the above functional modules, and in actual application, the above functions can be completed by different functional modules according to needs, that is, the modules or steps in the embodiments of the present application can be further decomposed or combined, for example, the modules of the above embodiments can be combined into one module, or can be further split into multiple sub-modules to complete all or part of the functions described above. The names of the modules and steps involved in the embodiments of the present application are only for distinguishing the modules and steps, and should not be considered as an improper limitation of the present application.

[0035] The third embodiment of the present application is an electronic device, which comprises at least one processor and a memory connected with the at least one processor in communication; wherein the memory stores instructions executable by the processor, and the instructions are used to be executed by the processor to implement the magnetic particle image reconstruction method based on the multi-modal diffusion model.

[0036] The fourth embodiment of the present application is a computer readable storage medium, which stores computer instructions, and the computer instructions are used to be executed by the computer to implement the magnetic particle image reconstruction method based on the multi-modal diffusion model.

[0037] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the storage device and the processing device described above and the related description can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

[0038] Those skilled in the art should clearly understand that the modules and method steps of each example described in combination with the embodiments disclosed herein can be realized by electronic hardware, computer software or a combination of both. The programs corresponding to the software modules and method steps can be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. In order to clearly illustrate the interchangeability of electronic hardware and software, the components and steps of each example have been generally described in the above description. Whether the functions are performed by electronic hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0039] Reference is made below to Figure 4 which shows a structural schematic diagram of a computer system of a server for implementing the embodiments of the method, system and device of the present application. Figure 4 The server shown is merely an example and should not impose any limitation on the functions and use range of the embodiments of the present application.

[0040] As Figure 4 shown, the computer system includes a central processing unit (CPU) 601 which can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 602 or loaded from a storage portion 608 into a random access memory (RAM) 603. Various programs and data required for system operation are also stored in the RAM 603. The CPU 601, the ROM 602 and the RAM 603 are connected to each other through a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0041] The following components are connected to the I / O interface 605: an input section 606 including input devices such as a keyboard and mouse; an output section 607 including output devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), and a speaker; a storage section 608 including a hard disk; and a communication section 609 including a network interface card such as a LAN (Local Area Network) card, a modem, and the like. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the I / O interface 605 as necessary. A removable medium 611 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, and the like is attached to the drive 610 as necessary, so that a computer program read out therefrom is installed in the storage section 608 as necessary.

[0042] In particular, the processes described above with reference to the flow charts can be implemented as a computer software program according to embodiments of the present disclosure. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer readable medium, the computer program comprising program code for performing the methods illustrated by the flow charts. In such embodiments, the computer program can be downloaded and installed from a network via the communication section 609, and / or installed from the removable medium 611. When the computer program is executed by the central processing unit (CPU) 601, the above-described functions defined in the methods of the present application are performed. It should be noted that the computer readable medium of the present application can be a computer readable signal medium or a computer readable storage medium or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any suitable combination of the above. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, the computer readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus or device. In the present application, the computer readable signal medium can include a data signal carried in a baseband or as part of a carrier wave, in which the computer readable program code is carried. Such a propagated data signal can take any of a variety of forms, including but not limited to electro-magnetic, optical, or any suitable combination thereof. The computer readable signal medium can also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate or transport a program for use by or in connection with an instruction execution system, apparatus or device. The program code contained on the computer readable medium can be transmitted by any suitable medium, including but not limited to wireless, wire line, optical fiber cable, RF, etc., or any suitable combination of the above.

[0043] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0044] The computer program instructions can also be loaded onto a computer or other programmable information processing apparatus to cause a series of operations to be performed on the computer or other programmable information processing apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable information processing apparatus implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0045] The terms "first", "second", etc. are used to distinguish between similar objects, and are not used to describe or indicate a particular order or sequence.

[0046] The terms "include", "comprise" and the like are used synonymously to denote a non-exclusive inclusion, such that processes, methods, articles, or apparatuses / apparatuses that are described as "including", "comprising" or the like have the possibilities that, in addition to the described elements, further elements are also present.

[0047] The technical scheme of the present application has been described in combination with the preferred embodiments shown in the drawings, but it is easy for those skilled in the art to understand that the protection scope of the present application is obviously not limited to these specific embodiments. Those skilled in the art can make equivalent changes or replacements to the related technical features without departing from the principles of the present application, and the technical schemes after the changes or replacements will all fall within the protection scope of the present application.

Claims

1. A magnetic particle image reconstruction method based on a multimodal diffusion model, characterized in that, The method includes: A first excitation signal is acquired, and an integral model is performed on the first excitation signal to obtain a first time-series signal. The first time-series signal is discretized to obtain a discrete signal. Each discrete signal is fitted and then superimposed to obtain a first response signal. A loss function is calculated based on the first response signal and the first time-series signal. A particle response model is trained based on the loss function to obtain a trained particle response model. The second response signal is obtained by processing the second time-series signal corresponding to the second excitation signal obtained by integral modeling using the trained particle response model. The reconstructed image is obtained by calculating the second response signal and the system matrix. The reconstructed image is then gradually noise-added and constrained to obtain a noise-constrained image. The final reconstructed image is obtained by progressively denoising and restoring the noise-constrained image based on Bayes' theorem and Gaussian distribution.

2. The magnetic particle image reconstruction method based on a multimodal diffusion model according to claim 1, characterized in that, The first time-series signal obtained by integral modeling the first excitation signal includes: ； in, It's temperature. It's viscosity. It is Boltzmann's constant. The magnetic moment per unit particle. As the first excitation signal, It is the first timing signal. It is the spatial distribution of particles. It is a particle response model. This indicates the particle response to the first excitation signal. t It is time. r It represents a spatial location.

3. The magnetic particle image reconstruction method based on a multimodal diffusion model according to claim 2, characterized in that, The discrete signal is: ； in, For position i discrete signals, For position i The spatial distribution of particles, i For location, It is a discrete signal.

4. The magnetic particle image reconstruction method based on a multimodal diffusion model according to claim 3, characterized in that, The first response signal is obtained by fitting and superimposing each discrete signal separately, including: Using the first neural operator For each position The response signal at each position is obtained by fitting the discrete signal; ； in, For position The response signal; It is the first neural operator; It's temperature. It's viscosity. It is Boltzmann's constant. The magnetic moment per unit particle. For position i Discrete signals; The response signals at each location are superimposed, and then processed by the second neural operator to obtain the first response signal: ； in, As the first response signal, This is the second neural operator.

5. The magnetic particle image reconstruction method based on a multimodal diffusion model according to claim 4, characterized in that, The loss function is calculated based on the first response signal and the first time series signal, including: ； in, For loss function, The frequency domain representation of the first response signal after Fourier transform. This is the representation of the first time-series signal after performing a Fourier transform.

6. The magnetic particle image reconstruction method based on a multimodal diffusion model according to claim 5, characterized in that, The reconstructed image calculated based on the second response signal and the system matrix includes: ； in, u Let S be the Fourier transform of the second response signal, and S be the system matrix obtained through measurement. To reconstruct the image.

7. The magnetic particle image reconstruction method based on a multimodal diffusion model according to claim 6, characterized in that, The reconstructed image is progressively noise-added and constrained to obtain a noise-constrained image at each step, including: For the The first step is to add noise to the reconstructed image to obtain a noisy image; ； ； in, For the first t Noisy image in step 1; t 1 represents the number of time steps. Standard Gaussian noise from random sampling; It is a noise estimation model; Then, the constraint parameters are used to constrain the noise image at step t1 to obtain the constrained noise image, specifically: ； in, These are pre-defined hyperparameters. u Let S be the Fourier transform of the second response signal, and S be the system matrix obtained through measurement. yes The pseudo-inverse matrix, I It is the identity matrix. For the first t The noise-constrained image after step 1, where F is the intermediate feature. The first constraint parameter, This is the second constraint parameter.

8. The magnetic particle image reconstruction method based on a multimodal diffusion model according to claim 7, characterized in that, The constraint parameters include a first constraint parameter and a second constraint parameter; The first constraint parameter is: calculate the average and maximum values ​​of each pixel of the intermediate feature F in the channel dimension to obtain two new feature maps with 1 channel. Then, concatenate the two new feature maps with 1 channel to obtain the first feature map. Input the first feature map into a convolutional layer with a kernel size of 7 to obtain the convolutional feature. Finally, activate the convolutional feature using the sigmoid function to obtain the first constraint parameter. ； The second constraint parameter is obtained by inputting the intermediate features into a convolutional layer with a kernel size of 3. ； in, A convolutional layer with a kernel size of 3. The maximum value of each pixel in the intermediate feature F. The average value of each pixel in the intermediate feature F. A convolutional layer with a kernel size of 7.

9. The magnetic particle image reconstruction method based on a multimodal diffusion model according to claim 8, characterized in that, The final reconstructed image obtained by progressive denoising and restoration includes: For the t Step 1 to the t Step 1-1: ； ； in, It is the image distribution during the noise-addition process in forward diffusion. It is a forward diffusion process that adds noise. Standard Gaussian noise is randomly sampled. for t 1 Distribution of step images, I It is the identity matrix. The distribution of the image at step t1-1, This is multimodal structural imaging data; p (.) represents the distribution. For the first Noisy images of the steps, For the constrained noise image at step t-1, For the constrained noise image at step t1, These are pre-defined hyperparameters. u Let S be the Fourier transform of the second response signal, and S be the system matrix obtained through measurement. The first constraint parameter, Here, F is the second constraint parameter, and F is the intermediate feature. yes The pseudo-inverse matrix, It is the first t Noisy image in step 1; It is the first diffusion parameter of the model. It is the second diffusion parameter of the model.

10. A magnetic particle image reconstruction system based on a multimodal diffusion model, characterized in that, The system includes: The data acquisition module is used to acquire a first excitation signal, perform integral modeling on the first excitation signal to obtain a first time-series signal, discretize the first time-series signal to obtain a discrete signal, fit and superimpose each discrete signal to obtain a first response signal, calculate a loss function based on the first response signal and the first time-series signal, and train a particle response model based on the loss function to obtain a trained particle response model. The signal processing module is used to process the second time-series signal corresponding to the second excitation signal obtained by integral modeling using the trained particle response model to obtain the second response signal. The image noise constraint module is used to calculate the reconstructed image based on the second response signal and the system matrix, and to gradually add noise and constrain the reconstructed image to obtain a noise-constrained image. The image denoising module is used to progressively denoise and restore the noise-constrained image in the final step based on Bayes' theorem and Gaussian distribution to obtain the final reconstructed image.

Images