Magnetic particle image reconstruction method and system based on multi-modal diffusion model

Abstract

The application belongs to the field of image reconstruction and multi-modal generative artificial intelligence model of magnetic particle imaging, and particularly relates to a magnetic particle image reconstruction method and system based on a multi-modal diffusion model, aiming to solve the problem that no method can combine a generative model with multi-modal imaging technology to realize accurate quantitative reconstruction of MPI. The application comprises: modeling and discretizing a first excitation signal to obtain a discrete signal, fitting each discrete signal respectively and then superimposing to obtain a first response signal, calculating a loss function, and training a model based on the loss function; using the trained particle response model to process a second time sequence signal to obtain a second response signal; calculating based on the second response signal and a system matrix, gradually adding noise, and constraining to obtain a noise-constrained image; and gradually denoising and restoring the noise-constrained image of the last step to obtain a final reconstructed image. The application makes it possible to realize fast and high-quality MPI image reconstruction.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of image reconstruction and multi-modal generative artificial intelligence model of magnetic particle imaging, and particularly relates to a magnetic particle image reconstruction method and system based on a multi-modal diffusion model. BACKGROUND

[0002] Magnetic particle imaging (MPI) is a new medical imaging technology that has been widely used in various biomedical fields. MPI has two main reconstruction algorithms: system matrix inverse problem-based reconstruction and X-space mapping-based reconstruction. The X-space-based algorithm is faster in reconstruction, but the image quality and quantification are poor; the system matrix-based reconstruction algorithm is better in image resolution and quality, and thus has become the mainstream imaging method. The difference between different system matrix reconstruction algorithms mainly lies in the design of the regularization term, which aims to alleviate the ill-conditioned nature of inverse problem solving. However, the system matrix-based reconstruction method relies on a large amount of prior information and needs to obtain the reconstruction result through a complex solving process. This kind of method often involves many parameters, resulting in poor robustness of reconstruction and being easily affected by parameter setting and noise, thereby affecting the image quality; in recent years, deep learning technology has also gradually explored in the field of MPI image reconstruction. Some researches have used fully connected layers and UNet-based models for MPI image reconstruction, and verified the feasibility of deep learning technology in MPI image reconstruction on simulation data. At the same time, another research has used a deep learning model to fit the regularization term, which has achieved higher quality image reconstruction in real data compared with the traditional fixed form regularization term. In addition, some image post-processing algorithms have been proposed to improve the quality of system matrix reconstruction images. Although the existing deep learning algorithms improve the visual quality of the image, they are usually end-to-end techniques, lacking the combination with the actual physical process. This leads to the reconstructed image being too sharp, which has a large error with the real magnetic particle concentration distribution, affecting the quantitative accuracy of MPI image reconstruction.

[0003] The image reconstruction of MPI is essentially solving a linear inverse problem. In recent years, generative artificial intelligence models (such as diffusion models) have shown significant potential in dealing with inverse problems in medical imaging. At the same time, multi-modal fusion technology has played an important role in various medical imaging problems. However, there is currently no accurate and quantitative reconstruction algorithm that can combine generative models and multi-modal imaging technology to achieve accurate and quantitative reconstruction of MPI. SUMMARY

[0004] In order to solve the above problems in the prior art, that is, there is no accurate and quantitative reconstruction algorithm at present, which can combine the generative model with the multi-modal imaging technology to realize the accurate quantitative reconstruction of MPI, the application provides a magnetic particle image reconstruction method based on a multi-modal diffusion model, the method comprises:

[0005] The first excitation signal is obtained, the first time sequence signal is obtained by integrating modeling the first excitation signal, the discrete signal is obtained by discretizing the first time sequence signal, the first response signal is obtained by fitting and superimposing each discrete signal, the loss function is calculated based on the first response signal and the first time sequence signal, and the particle response model is trained based on the loss function to obtain the trained particle response model;

[0006] The second response signal is obtained by processing the second time sequence signal corresponding to the second excitation signal obtained by integrating modeling, using the trained particle response model;

[0007] The reconstruction image is calculated based on the second response signal and the system matrix, the reconstruction image is gradually denoised and constrained to obtain the noise constraint image;

[0008] The final reconstruction image is obtained by gradually denoising and recovering the noise constraint image of the last step based on the Bayes theorem and the Gaussian distribution.

[0009] In a preferred embodiment, the first time sequence signal is obtained by integrating modeling the first excitation signal, comprising:

[0010] ;

[0011] Wherein, is the temperature, is the viscosity, is the Boltzmann constant, is the unit particle magnetic moment, is the first excitation signal, is the first time sequence signal, is the spatial distribution of particles, is the particle response model, indicates the particle response to the first excitation signal, t is the time, r represents the spatial position.

[0012] In a preferred embodiment, the discrete signal is:

[0013] ;

[0014] Wherein, is the discrete signal of the position i is the discrete signal of the position is the discrete signal of the position iThe spatial distribution of particles, where i is the position. It is a discrete signal.

[0015] In a preferred embodiment, the first response signal is obtained by fitting and superimposing the individual discrete signals separately, including:

[0016] Using the first neural operator For each position The response signal at each position is obtained by fitting the discrete signal;

[0017] ;

[0018] in, For position The response signal; This is the first neural operator; It's temperature. It's viscosity. It is Boltzmann's constant. The magnetic moment per unit particle. For position i Discrete signals;

[0019] The response signals at each location are superimposed, and then processed by the second neural operator to obtain the first response signal:

[0020] ;

[0021] in, As the first response signal, This is the second neural operator.

[0022] In a preferred embodiment, calculating the loss function based on the first response signal and the first time-series signal includes:

[0023] The loss function is calculated based on the first response signal and the first time series signal, including:

[0024] ;

[0025] in, For loss function, The frequency domain representation of the first response signal after Fourier transform. This is the representation of the first time-series signal after performing a Fourier transform.

[0026] In a preferred embodiment, calculating the reconstructed image based on the second response signal and the system matrix includes:

[0027] ;

[0028] in, uLet S be the Fourier transform of the second response signal, and S be the system matrix obtained through measurement. To reconstruct the image.

[0029] The reconstructed image is progressively noise-added and constrained to obtain a noise-constrained image at each step, including:

[0030] For the The first step is to add noise to the reconstructed image to obtain a noisy image;

[0031] ;

[0032] ;

[0033] in, For the first t Noisy image in step 1; t 1 represents the number of time steps. Standard Gaussian noise from random sampling; It is a noise estimation model;

[0034] Then, the constraint parameters are used to constrain the noise image at step t1 to obtain the constrained noise image, specifically:

[0035] ;

[0036] in, These are pre-defined hyperparameters. u Let S be the Fourier transform of the second response signal, and S be the system matrix obtained through measurement. yes The pseudo-inverse matrix, I It is the identity matrix. For the first t The noise-constrained image after step 1, where F is the intermediate feature. The first constraint parameter, This is the second constraint parameter.

[0037] In a preferred embodiment, the constraint parameters include a first constraint parameter and a second constraint parameter;

[0038] The first constraint parameter is: calculate the average and maximum values ​​of each pixel of the intermediate feature F in the channel dimension to obtain two new feature maps with 1 channel. Then, concatenate the two new feature maps with 1 channel to obtain the first feature map. Input the first feature map into a convolutional layer with a kernel size of 7 to obtain the convolutional feature. Finally, activate the convolutional feature using the sigmoid function. Finally, the first constraint parameter is calculated.

[0039] ;

[0040] The second constraint parameter is obtained by inputting the intermediate features into a convolutional layer with a kernel size of 3.

[0041] ;

[0042] in, A convolutional layer with a kernel size of 3. The maximum value of each pixel in the intermediate feature F. The average value of each pixel in the intermediate feature F is... A convolutional layer with a kernel size of 7.

[0043] In a preferred embodiment, performing stepwise denoising and restoration to obtain the final reconstructed image includes:

[0044] For the t Step 1 to the t Step 1-1:

[0045] ; ;

[0046] in, It is the image distribution during the noise-addition process in forward diffusion. It is a forward diffusion process that adds noise. The noise is standard Gaussian noise sampled randomly. for t 1 Distribution of step images I It is the identity matrix. The distribution of the image at step t1-1, This is multimodal structural imaging data; p (.) represents the distribution. For the first Noisy images of the steps, For the constrained noise image at step t-1, For the constrained noise image at step t1, These are pre-defined hyperparameters. u Let S be the Fourier transform of the second response signal, and S be the system matrix obtained through measurement. The first constraint parameter, Here, F is the second constraint parameter, and F is the intermediate feature. yes The pseudo-inverse matrix, It is the first t Noisy image in step 1; It is the first diffusion parameter of the model. It is the second diffusion parameter of the model.

[0047] In a second aspect, the present application provides a magnetic particle image reconstruction system based on a multi-modal diffusion model, the system comprising:

[0048] The data acquisition module is configured to acquire a first excitation signal, perform integral modeling on the first excitation signal to obtain a first time sequence signal, discretize the first time sequence signal to obtain discrete signals, superimpose the discrete signals after fitting each discrete signal to obtain a first response signal, calculate a loss function based on the first response signal and the first time sequence signal, and train a particle response model based on the loss function to obtain a trained particle response model.

[0049] The signal processing module is configured to use the trained particle response model to process a second time sequence signal corresponding to a second excitation signal obtained by integral modeling to obtain a second response signal.

[0050] The image noise constraint module is configured to calculate a reconstructed image based on the second response signal and a system matrix, and to obtain a noise constraint image by gradually adding noise to the reconstructed image.

[0051] The image denoising module is configured to gradually denoise and recover the noise constraint image obtained in the last step based on Bayes' theorem and Gaussian distribution to obtain a final reconstructed image.

[0052] The present application has the following advantages:

[0053] (1) The present application uses a trained particle response model to process an excitation signal, which can generate more accurate first and second response signals. This helps to overcome the image quality problems caused by parameter setting and noise in traditional system matrix-based reconstruction algorithms, thereby improving the quality and resolution of the final reconstructed image.

[0054] (2) The present application combines generative artificial intelligence models (such as diffusion models) and multi-modal imaging technology, which can more accurately reflect the true magnetic particle concentration distribution and provide more realistic and accurate quantitative reconstruction results. Through integral modeling, discretization, and fitting of the first time sequence signal, and training of the particle response model based on the loss function, this method provides a new approach to solving linear inverse problems. In particular, the process of gradually adding noise, constraining to obtain a noise constraint image, and iteratively denoising and recovering the image using Bayes' theorem and Gaussian distribution enhances the stability and robustness of inverse problem solving.

[0055] (3) The reconstruction algorithm based on the system matrix in the application has advantages in image quality, adopts a deep learning model to fit a regular term and a generative model to process an inverse problem, improves the quality of image reconstruction, and improves the calculation efficiency, so that fast and high-quality MPI image reconstruction is possible. By integrating the latest generative artificial intelligence model and the traditional MPI reconstruction method, more accurate and quantitative MPI image reconstruction is achieved, and the problems of poor image quality, insufficient quantitative accuracy and low calculation efficiency in the existing method are solved. BRIEF DESCRIPTION OF DRAWINGS

[0056] Other features, objects and advantages of the application will become more apparent from the following detailed description of non-limiting embodiments, made with reference to the attached drawings:

[0057] Figure 1 is a schematic diagram of a magnetic particle image reconstruction method based on a multi-modal diffusion model according to an embodiment of the application;

[0058] Figure 2 is an image reconstruction model diagram based on multi-modal model diffusion according to an embodiment of the application;

[0059] Figure 3 is an image reconstruction model diagram based on multi-modal diffusion according to an embodiment of the application;

[0060] Figure 4 is a structural schematic diagram of a computer system of a server for implementing the method, system and device embodiments of the application. DETAILED DESCRIPTION

[0061] The application will be further described in detail below with reference to the drawings and embodiments. It can be understood that the specific embodiments described herein are only used to explain the related application, and not to limit the application. In addition, it should be noted that, for the sake of description, only the parts related to the application are shown in the drawings.

[0062] It should be noted that the embodiments and features in the embodiments in the application can be combined with each other without conflict. The application will be described in detail below with reference to the drawings and embodiments.

[0063] The application provides a magnetic particle image reconstruction method based on a multi-modal diffusion model, the method comprising:

[0064] Acquire a first excitation signal and a second excitation signal, perform integral modeling on the first excitation signal and the second excitation signal respectively to obtain a first time series signal and a second time series signal, discretize the first time series signal to obtain a discrete signal, fit each discrete signal and superimpose them to obtain a first response signal, calculate a loss function based on the first response signal and the first time series signal, and train a particle response model based on the loss function to obtain a trained particle response model.

[0065] The second time-series signal is obtained by processing the trained particle response model.

[0066] The reconstructed image is obtained by calculating the second response signal and the system matrix. The reconstructed image is then gradually noise-added and constrained to obtain a noise-constrained image.

[0067] The final reconstructed image is obtained by progressively denoising and restoring the noise-constrained image based on Bayes' theorem and Gaussian distribution.

[0068] To more clearly explain the magnetic particle image reconstruction method based on a multimodal diffusion model of this invention, the following will be combined with... Figure 1 The steps in the embodiments of the present invention will be described in detail below.

[0069] The magnetic particle image reconstruction method based on a multimodal diffusion model according to the first embodiment of the present invention is described in detail below:

[0070] Acquire a first excitation signal and a second excitation signal, perform integral modeling on the first excitation signal and the second excitation signal respectively to obtain a first time series signal and a second time series signal, discretize the first time series signal to obtain a discrete signal, fit each discrete signal and superimpose them to obtain a first response signal, calculate a loss function based on the first response signal and the first time series signal, and train a particle response model based on the loss function to obtain a trained particle response model.

[0071] In this embodiment, the process of integral modeling the first excitation signal to obtain the first time-series signal includes:

[0072] ;

[0073] in, It's temperature. It's viscosity. It is Boltzmann's constant. The magnetic moment per unit particle is given. Device parameters include magnetic field strength and frequency, and the excitation signal is used for this purpose. This is reflected in Particle parameters include the magnetic moment per unit particle, which is related to the particle's diameter and saturation magnetization. Device parameters include magnetic field strength and frequency, which are reflected in the excitation signal.

[0074] The particle response model is the core factor affecting the accuracy of the generated data of MPI. Although many different response models have been proposed to describe this relationship, such as the Langevin model and the Debye model, the underlying motion and response of magnetic particles are very complex, and existing fixed mathematical form models are difficult to accurately describe, resulting in a large difference between the simulated signal and the actual measured signal. This mismatch will have a negative impact on the quality of the generated data, and then affect the training accuracy and generalization ability of the model. In order to solve this problem, the present application adopts a data-driven approach to fit the response model of the particles through FNO, rather than setting a specific mathematical model, and the specific idea is as follows.

[0075] In this embodiment, the discrete signal is:

[0076] ;

[0077] wherein, is the discrete signal of the position i , is the spatial distribution of the particles at the position , i is the position, i and is the discrete signal.

[0078] The first response signal is obtained by fitting and superimposing each discrete signal, including:

[0079] The first neural operator is used to fit the discrete signal of each position to obtain the response signal of each position;

[0080] ;

[0081] wherein, is the response signal of the position ; is the first neural operator; is the temperature, is the viscosity, is the Boltzmann constant, is the unit particle magnetic moment, is the discrete signal of the position i ;

[0082] The response signals of each position are superimposed, and then processed by the second neural operator to obtain the first response signal:

[0083] ;

[0084] wherein, is the first response signal, is the second neural operator.

[0085] ​In the embodiment, the loss function is calculated based on the first response signal and the first time sequence signal, and the loss function comprises:

[0086]

[0087] wherein, is the loss function, is a frequency domain representation of the first response signal after Fourier transform, is a representation of the first time sequence signal after Fourier transform.

[0088] After the training is completed, a corresponding phantom is constructed based on a magnetic resonance imaging (MRI) dataset IXI, digital phantoms of different magnetic particle concentration distributions and corresponding structural imaging data are obtained, and then a response signal corresponding to the digital phantoms is generated by using the trained system, so that a large number of multi-modal data pairs close to the real data distribution are obtained, which are used for subsequent training and optimization of a multi-modal diffusion reconstruction model, such as an image reconstruction model shown in Figure 2 .

[0089] The second response signal is obtained by processing the second time sequence signal by using the trained particle response model.

[0090] The reconstructed image is obtained based on the second response signal and the system matrix, and the reconstructed image is gradually added with noise and constrained to obtain a noise-constrained image.

[0091] In the embodiment, the noise-constrained image at each step is obtained by gradually adding noise to the reconstructed image based on the second response signal and the system matrix, and the reconstructed image is gradually added with noise and constrained to obtain a noise-constrained image.

[0092] The reconstruction algorithm based on the system matrix is essentially to solve an inverse problem wherein, is a frequency domain representation of the received signal after Fourier transform, is a measured system matrix, c 0 is a to-be-solved reconstructed image. The purpose of the forward diffusion is to change the reconstructed image c into Gaussian noise , and the process can be represented as . The core of the forward diffusion is to train a noise estimation model to predict the noise added to the original image at each step, and the reconstructed image is obtained based on the second response signal and the system matrix.

[0093]

[0094] wherein, u is a Fourier transform of the second response signal, S is a measured system matrix, is a reconstructed image.

[0095] ​​The reconstructed image is progressively noise-added and constrained to obtain a noise-constrained image at each step, including:

[0096] For the The first step is to add noise to the reconstructed image to obtain a noisy image;

[0097] ;

[0098] ;

[0099] in, For the first t Noisy image in step 1; t 1 represents the number of time steps. Standard Gaussian noise from random sampling; It is a noise estimation model;

[0100] in, The hyperparameters are fixed in advance to control the noise intensity at each step, and is randomly sampled standard Gaussian noise. However, this diffusion process is inherently unconditional, meaning it lacks constraints and correspondence with actual physical information. To integrate measured physical information with the diffusion process, this invention redesigns the diffusion generation process based on the idea of ​​zero-domain decomposition, enabling the introduction of measured physical information during the diffusion process. This improved method not only maintains the generative capability of the diffusion model but also enhances the physical consistency and accuracy of the reconstructed image. u The Fourier transform of the second response signal;

[0101] Will replace This invention, used in the forward diffusion process, adjusts the original distribution to a conditional distribution based on the system matrix and the received signal, achieving a correspondence between image diffusion and measured physical information. This invention further constrains the diffusion process using multimodal structural imaging data. Specifically, this invention utilizes a pre-trained fixed-parameter model, VGG, to encode intermediate features, and finally implements constraints through a conditional modulation network. The intermediate image of the above iterative process can be further represented as:

[0102] The constrained noise image is obtained by constraining the noise image at step t1 using constraint parameters, specifically as follows:

[0103] The constrained noise image is obtained by constraining the noise image at step t1 using constraint parameters, specifically as follows:

[0104] ;

[0105] in, These are pre-defined hyperparameters. uS is a system matrix measured, is a pseudo-inverse matrix of I is a unit matrix, is a noise constraint image after the first step of noise image constraint, t 1step, F is an intermediate feature, is a first constraint parameter, is a second constraint parameter.

[0106] In the embodiment, the constraint parameters include a first constraint parameter and a second constraint parameter;

[0107] The first constraint parameter is: calculating the average value and the maximum value of each pixel point of the intermediate feature F in the channel dimension to obtain two new feature maps with a channel number of 1, then splicing the two new feature maps with a channel number of 1 to obtain a first feature map, inputting the first feature map into a convolution layer with a convolution kernel size of 7 to obtain a convolution feature, and finally activating the convolution feature by using a sigmoid function, and finally calculating to obtain the first constraint parameter;

[0108] ;

[0109] The second constraint parameter is: inputting the intermediate feature into a convolution layer with a convolution kernel size of 3 to obtain the second constraint parameter:

[0110] ;

[0111] wherein, the convolution layer with a convolution kernel size of 3, is the maximum value of each pixel point of the intermediate feature F, is the average value of each pixel point of the intermediate feature F, the convolution layer with a convolution kernel size of 7.

[0112] The noise constraint image of the last step is gradually denoised and recovered to obtain a final reconstruction image based on the Bayes theorem and Gaussian distribution.

[0113] For the first t 1step to the first t 1-1step:

[0114] In the embodiment, the process of inverse reconstruction is to gradually recover the final reconstruction image by iteratively denoising the noise image multiple times, and the core is to construct a recursive formula of to , that is, a distribution . According to the Bayes theorem, the distribution can be decomposed as:

[0115] ;

[0116] Where p is the probability;

[0117] During the forward diffusion process, this invention uses an image decomposed from the zero-domain. Simultaneously, constraints from measured physical information and structural imaging data need to be added during the reverse reconstruction process. This formula can be written as:

[0118] ;

[0119] All three distributions can be modeled by Gaussian distributions. Through parameter renormalization, the recursive formula for inverse reconstruction can be obtained as follows:

[0120] ;

[0121] in, It is the image distribution during the noise-addition process in forward diffusion. It is a forward diffusion process that adds noise. The noise is standard Gaussian noise sampled randomly. for t 1 Distribution of step images I It is the identity matrix. The distribution of the image at step t1-1, This is multimodal structural imaging data; p (.) represents the distribution. For the first Noisy images of the steps, For the constrained noise image at step t-1, For the constrained noise image at step t1, These are pre-defined hyperparameters. u Let S be the Fourier transform of the second response signal, and S be the system matrix obtained through measurement. The first constraint parameter, Here, F is the second constraint parameter, and F is the intermediate feature. yes The pseudo-inverse matrix, It is the first t Noisy image in step 1; It is the first diffusion parameter of the model. It is the second diffusion parameter of the model.

[0122] Therefore, the reverse process can be achieved by randomly generating noise. Based on the above iterative formula, and under the constraints of measured physical information and structural images, corresponding high-quality MPI images are generated step by step. The model diagrams for diffusion and reconstruction are shown below. Figure 3 .

[0123] Although the above embodiments are described in the above-mentioned order, it is understood by those skilled in the art that, in order to achieve the effects of the embodiments, the different steps do not have to be executed in such an order, and they can be executed simultaneously (in parallel) or in a reversed order, and these simple changes are within the protection scope of the present application.

[0124] The magnetic particle image reconstruction system based on the multi-modal diffusion model according to the second embodiment of the present application comprises:

[0125] The data acquisition module is configured to acquire the first excitation signal and the second excitation signal, perform integral modeling on the first excitation signal and the second excitation signal to obtain a first time sequence signal and a second time sequence signal, discretize the first time sequence signal to obtain a discrete signal, fit each discrete signal and then superimpose them to obtain a first response signal, calculate a loss function based on the first response signal and the first time sequence signal, and train a particle response model based on the loss function to obtain a trained particle response model.

[0126] The signal processing module is configured to process the second time sequence signal using the trained particle response model to obtain a second response signal.

[0127] The image noise constraint module is configured to calculate a reconstructed image based on the second response signal and a system matrix, and perform step-by-step noise addition and constraint on the reconstructed image to obtain a noise constraint image.

[0128] The image denoising module is configured to perform step-by-step denoising and recovery on the noise constraint image of the last step based on the Bayes theorem and Gaussian distribution to obtain a final reconstructed image.

[0129] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process and related description of the system described above can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0130] It should be noted that the magnetic particle image reconstruction system based on the multi-modal diffusion model provided in the above embodiments is only exemplified by the division of the above functional modules, and in actual applications, the above functions can be completed by different functional modules according to needs, that is, the modules or steps in the embodiments of the present application can be further decomposed or combined, for example, the modules of the above embodiments can be combined into one module, or can be further split into multiple sub-modules to complete all or part of the functions described above. The names of the modules and steps involved in the embodiments of the present application are only for distinguishing the modules and steps, and should not be considered as an improper limitation of the present application.

[0131] The electronic device of the third embodiment of the application comprises at least one processor and a memory in communication connection with the at least one processor, wherein the memory stores instructions executable by the processor, and the instructions are used to be executed by the processor to implement the magnetic particle image reconstruction method based on the multi-modal diffusion model.

[0132] The computer readable storage medium of the fourth embodiment of the application stores computer instructions, and the computer instructions are used to be executed by the computer to implement the magnetic particle image reconstruction method based on the multi-modal diffusion model.

[0133] Those skilled in the art can clearly understand the specific working processes of the storage device and the processing device and the related descriptions described above for the convenience and brevity of description, which can refer to the corresponding processes in the foregoing method embodiments, and will not be described here.

[0134] Those skilled in the art should realize that the modules and method steps of each example described in combination with the embodiments disclosed herein can be realized by electronic hardware, computer software or a combination of both. The programs corresponding to the software modules and method steps can be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM or any other form of storage medium known in the art. In order to clearly illustrate the interchangeability of electronic hardware and software, the components and steps of each example have been generally described in the foregoing description. Whether the functions are performed by electronic hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the application.

[0135] Reference will now be made to the following description Figure 4 which shows the structural schematic diagram of a computer system of a server for implementing the method, system and device embodiments of the application. Figure 4 The server shown is only an example and should not impose any limitation on the functions and use range of the embodiments of the application.

[0136] As Figure 4As shown, the computer system includes a central processing unit (CPU) 601 which can perform various appropriate actions and processes in accordance with a program stored in a read only memory (ROM) 602 or a program loaded from a storage section 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the system are also stored. The CPU 601, the ROM 602, and the RAM 603 are connected to each other through a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0137] Connected to the I / O interface 605 are an input section 606 including a keyboard, a mouse, etc.; an output section 607 including a display such as a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the I / O interface 605 as required. A removable recording medium 611 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is attached to the drive 610 as required, so that a computer program read out therefrom is installed in the storage section 608 as required.

[0138] In particular, the processes described above with reference to the flow charts can be implemented as a computer software program according to embodiments of the present disclosure. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer readable medium, the computer program comprising program code for performing the methods illustrated by the flow charts. In such embodiments, the computer program can be downloaded and installed from a network via the communication section 609, and / or installed from the removable medium 611. When the computer program is executed by the central processing unit (CPU) 601, the above-described functions defined in the methods of the present application are performed. It should be noted that the computer readable medium of the present application can be a computer readable signal medium or a computer readable storage medium or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any suitable combination of the above. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, the computer readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus or device. In the present application, the computer readable signal medium can include a data signal carried in a baseband or as part of a carrier wave, in which the computer readable program code is carried. Such a propagated data signal can take any of a variety of forms, including but not limited to electro-magnetic, optical, or any suitable combination thereof. The computer readable signal medium can also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate or transport a program for use by or in connection with an instruction execution system, apparatus or device. The program code contained on the computer readable medium can be transmitted by any suitable medium, including but not limited to wireless, wire line, optical fiber cable, RF, etc., or any suitable combination of the above.

[0139] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0140] The computer program instructions can also be loaded onto a computer or other programmable information processing apparatus to cause a series of operations to be performed on the computer or other programmable information processing apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable information processing apparatus implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0141] The terms "first", "second", etc. are used to distinguish between similar objects, and are not used to describe or indicate a particular order or sequence.

[0142] The terms "comprises", "comprising", or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus.

[0143] The technical scheme of the present application has been described in combination with the preferred embodiments shown in the drawings, but it is easy for those skilled in the art to understand that the protection scope of the present application is obviously not limited to these specific embodiments. Those skilled in the art can make equivalent changes or replacements to the related technical features without departing from the principles of the present application, and the technical schemes after the changes or replacements will all fall within the protection scope of the present application.

Claims

1. A magnetic particle image reconstruction method based on a multi-modal diffusion model, characterized in that, The method comprises: obtaining a first excitation signal, performing integral modeling on the first excitation signal to obtain a first time sequence signal, discretizing the first time sequence signal to obtain discrete signals, fitting each discrete signal respectively and then superimposing to obtain a first response signal, calculating a loss function based on the first response signal and the first time sequence signal, and training a particle response model based on the loss function to obtain a trained particle response model; the integral modeling on the first excitation signal to obtain the first time sequence signal comprises: ； wherein is the temperature, is the viscosity, is the Boltzmann constant, is the unit particle magnetic moment, is a first excitation signal, is a first timing signal, is a spatial distribution of particles, is a particle response model, t is time, and r represents a spatial position; the fitting of each discrete signal respectively and then the superimposing to obtain the first response signal comprises: Utilizing a first neural operator Fitting a discrete signal for each location to obtain a response signal for each location; ； wherein is a position of a response signal; is a first neural operator; is a temperature, is a viscosity, is a Boltzmann constant, is a spatial distribution of particles at position i, is a unit particle magnetic moment, is a discrete signal at position i; the superimposing of the response signals of each position, and then processing by a second neural operator to obtain the first response signal: ； wherein is a first response signal, is a second neural operator; using the trained particle response model to process a second time sequence signal corresponding to a second excitation signal obtained by integral modeling to obtain a second response signal; based on the second response signal and a system matrix, a reconstructed image is calculated, and the reconstructed image is gradually added with noise and constrained to obtain a noise-constrained image; based on the Bayes theorem and the Gaussian distribution, the noise-constrained image of the last step is gradually denoised and recovered to obtain a final reconstructed image.

2. The multi-modal diffusion model based magnetic particle image reconstruction method according to claim 1, characterized in that, The discrete signals are: ； wherein is the discrete signal at position i, i being the position, is the discrete signal.

3. The multi-modal diffusion model based magnetic particle image reconstruction method according to claim 2, characterized in that, the calculation of the loss function based on the first response signal and the first time sequence signal comprises: ； wherein is a loss function, is a frequency domain representation of the first response signal after Fourier transformation, is a representation of the first time series after Fourier transformation.

4. The multi-modal diffusion model based magnetic particle image reconstruction method according to claim 3, characterized in that, the calculation of the reconstructed image based on the second response signal and the system matrix comprises: ； where u is the Fourier transform of the second response signal, S is the system matrix obtained by measurement, to reconstruct the image.

5. The multi-modal diffusion model based magnetic particle image reconstruction method according to claim 4, characterized in that, the gradual addition of noise and constraint of the reconstructed image to obtain a noise-constrained image at each step comprises: For the first Step, noise is added to the reconstructed image to obtain a noisy image; ； ； wherein, is the noise image of the t1 step; t1 is the time step number, is a standard Gaussian noise sampled randomly; is a noise estimation model; then, a constraint parameter is used to constrain the noise image at the t1th step to obtain a constraint noise image, specifically: ； wherein, is a pre-fixed hyper parameter, u is the Fourier transform of the second response signal, S is the measured system matrix, is the pseudo inverse matrix of , I is the identity matrix, is the noise constrained image after the constraint of the noise image of the t1th step, F is the intermediate feature, is the first constraint parameter, is the second constraint parameter.

6. The multi-modal diffusion model based magnetic particle image reconstruction method according to claim 5, characterized in that, the constraint parameter comprises a first constraint parameter and a second constraint parameter; the first constraint parameter is: calculating the average value and the maximum value of each pixel point of the intermediate feature F in the channel dimension to obtain two new feature maps with a channel number of 1, then splicing the two new feature maps with a channel number of 1 to obtain a first feature map, inputting the first feature map into a convolution layer with a convolution kernel size of 7 to obtain a convolution feature, and finally activating the convolution feature by using a sigmoid function to obtain the first constraint parameter; ； the second constraint parameter is: inputting the intermediate feature into a convolution layer with a convolution kernel size of 3 to obtain the second constraint parameter: ； wherein, a convolution layer with a kernel size of 3, a maximum value for each pixel point of the intermediate feature F, an average value for each pixel point of the intermediate feature F, a convolution layer with a kernel size of 7.

7. The multi-modal diffusion model based magnetic particle image reconstruction method according to claim 6, characterized in that, the gradual denoising and recovery to obtain the final reconstructed image comprises: for the t1th to the t1-1th steps: ； ； in, It is the image distribution during the noise-addition process in forward diffusion. It is a forward diffusion process that adds noise. The noise is standard Gaussian noise sampled randomly. Let I be the distribution of the image at step t1, and let I be the identity matrix. The distribution of the image at step t1-1, This represents multimodal structural imaging data; p(.) is the distribution representation. For the first Noisy images of the steps, For the constrained noise image at step t-1, For the constrained noise image at step t1, These are pre-fixed hyperparameters, where u is the Fourier transform of the second response signal, and S is the system matrix obtained through measurement. The first constraint parameter, Here, F is the second constraint parameter, and F is the intermediate feature. yes The pseudo-inverse matrix, It is the noisy image at step t1; It is the first diffusion parameter of the model. It is the second diffusion parameter of the model.

8. A multi-modal diffusion model based magnetic particle image reconstruction system, characterized in that, the system comprises: a data acquisition module, configured to obtain a first excitation signal, perform integral modeling on the first excitation signal to obtain a first time sequence signal, discretize the first time sequence signal to obtain discrete signals, fit each discrete signal respectively and then superimpose to obtain a first response signal, calculate a loss function based on the first response signal and the first time sequence signal, and train a particle response model based on the loss function to obtain a trained particle response model; a signal processing module, configured to use the trained particle response model to process a second time sequence signal corresponding to a second excitation signal obtained by integral modeling to obtain a second response signal; an image noise adding and constraining module, configured to calculate a reconstructed image based on the second response signal and a system matrix, and gradually add noise and constrain the reconstructed image to obtain a noise-constrained image; an image denoising module, configured to gradually denoise and recover the noise-constrained image of the last step based on the Bayes theorem and the Gaussian distribution to obtain a final reconstructed image. The first time sequence signal is obtained by integrating modeling the first excitation signal, and the first response signal is obtained by fitting each discrete signal respectively and then superimposing the fitted signals: ； wherein is the temperature, is the viscosity, is the Boltzmann constant, is the unit particle magnetic moment, is a first excitation signal, is a first timing signal, is a spatial distribution of particles, is a particle response model, t is time, and r represents a spatial location; The first response signal is obtained by fitting each discrete signal respectively and then superimposing the fitted signals: Utilizing a first neural operator Fitting a discrete signal for each location to obtain a response signal for each location; ； wherein is a position of a response signal; is a first neural operator; is a temperature, is a viscosity, is the Boltzmann constant, is a spatial distribution of particles at position i, is a unit particle magnetic moment, is a discrete signal at position i; The first response signal is obtained by superimposing the response signals of each position and then processing the superimposed signals by the second neural operator: ； wherein is a first response signal, is a second neural operator.

Images