Mathematical information driven magnetic particle imaging gradient reconstruction method, model training method and system

Abstract

The present invention provides a magnetic particle imaging gradient reconstruction method driven by mathematical information, a model training method and a system, and relates to the field of artificial intelligence technology, and in particular to the field of biomedical molecular imaging and magnetic particle imaging reconstruction technology. The method comprises: inputting a system matrix and a response signal for an object to be imaged into a conditional generation model, and outputting conditional reference information; looping through the following operations to obtain a target reconstructed image for the object to be imaged; using an initial inference image as a current inference image, inputting the current inference image and conditional reference information into a conditional gradient estimation model, and outputting the current estimated image; inputting a system matrix and a response signal into a mathematical information iterative extraction model, and outputting the current mathematical information; inputting the current estimated image and the current mathematical information into a mathematical information transformation driving model, and outputting an optimized estimated image, and looping through the above operations with the optimized estimated image as the current inference image.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, in particular to the field of biomedical molecular imaging and magnetic particle imaging reconstruction technology, and more specifically to a mathematical information-driven magnetic particle imaging gradient reconstruction method, model training method and system. Background Art

[0002] Magnetic Particle Imaging (MPI) is a new biomedical imaging technology that can use the nonlinear response of superparamagnetic iron oxide particles in a composite magnetic field to directly detect their distribution in the body. It has the advantages of high sensitivity, high temporal and spatial resolution, zero background tissue depth signal, and no ionizing radiation. It has good application prospects in the biomedical field, such as vascular imaging, cancer detection, cell tracing, and perfusion imaging.

[0003] MPI reconstruction is to reconstruct the voltage signal obtained from the MPI device into a distribution map of magnetic particle concentration. The reconstruction method based on and the reconstruction method based on the system matrix. The quality and clarity of the magnetic particle distribution images obtained by the reconstruction method are poor, which affects the resolution and contrast of the reconstructed image, and further affects the interpretation of the MPI results. The reconstruction method based on the system matrix usually uses a calibration scan to measure the system matrix composed of multiple harmonic components before reconstruction, models the relationship between the spatial concentration distribution of magnetic particles and their frequency response, and then uses the system matrix and the magnetic particle response signal to solve the linear inverse problem, decode the spatial concentration distribution of magnetic particles, and reconstruct the MPI image. However, the complex background noise leads to the pathological nature of the linear inverse problem, making the solution of the inverse problem unstable and limiting the solution accuracy. Summary of the invention

[0004] In view of this, the present invention provides a mathematical information driven magnetic particle imaging gradient reconstruction method, model training method and system.

[0005] One aspect of the present invention provides a mathematical information driven magnetic particle imaging gradient reconstruction method, which is applied to a trained mathematical information driven magnetic particle imaging gradient reconstruction model system, wherein the trained mathematical information driven magnetic particle imaging gradient reconstruction model system includes a conditional generation model, a conditional gradient estimation model, a mathematical information iterative extraction model, and a mathematical information transformation driving model; the above method includes: inputting a system matrix and a response signal for an object to be imaged into the conditional generation model, outputting conditional reference information, wherein the conditional reference information includes an initial reconstructed image for the object to be imaged or a feature map of the initial reconstructed image; and cyclically performing the following operations until the number of cycles reaches a preset cycle threshold value When , a target reconstructed image for the object to be imaged is obtained; the initial reasoning image is used as the current reasoning image, the current reasoning image and the conditional reference information are input into the conditional gradient estimation model, and the current estimated image is output, wherein the initial reasoning image includes a Gaussian noise image; the system matrix and the response signal are input into the mathematical information iterative extraction model, and the current mathematical information is output, wherein the current mathematical information represents the mathematical information for the object to be imaged at the current number of cycles; the current estimated image and the current mathematical information are input into the mathematical information transformation driving model to optimize the current estimated image, and the optimized estimated image is output, and the above operation is cyclically performed with the optimized estimated image as the current reasoning image.

[0006] Another aspect of the present invention provides a training method for a mathematical information driven magnetic particle imaging gradient reconstruction model system, wherein the mathematical information driven magnetic particle imaging gradient reconstruction model system includes a conditional generation model, a conditional gradient estimation model, a mathematical information iterative extraction model and a mathematical information transformation driving model; the method includes: obtaining training sample data, wherein the training sample data includes a sample system matrix, a simulated MPI image and a simulated response signal; inputting the sample system matrix and the simulated response signal into the conditional generation model, and outputting sample condition reference information, wherein the sample condition reference information includes an initial reconstructed image corresponding to the simulated MPI image or a feature map of the initial reconstructed image; using the initial inference image as the current sample inference image, and inputting the current sample inference image and the sample condition reference information into the conditional gradient estimation model. The invention discloses a method for extracting magnetic particle imaging gradient reconstruction model, wherein the mathematical information of the magnetic particle imaging gradient is extracted from the sample system matrix and the simulated response signal, and outputs the mathematical information of the current sample, wherein the mathematical information of the current sample represents the mathematical information of the object to be imaged at the current number of cycles; the mathematical information transformation driving model is used to optimize the current sample estimation image and the mathematical information of the current sample, and outputs the optimized sample estimation image; the network parameters of the magnetic particle imaging gradient reconstruction model system driven by the mathematical information are iteratively adjusted based on the optimized sample estimation image and the simulated MPI image until the first iteration stop condition is reached, and the model system obtained when the first iteration stop condition is reached is used as the trained magnetic particle imaging gradient reconstruction model system driven by the mathematical information.

[0007] Another aspect of the present invention provides a mathematical information driven magnetic particle imaging gradient reconstruction system, which is applied to a trained mathematical information driven magnetic particle imaging gradient reconstruction model system, wherein the trained mathematical information driven magnetic particle imaging gradient reconstruction model system includes a conditional generation model, a conditional gradient estimation model, a mathematical information iterative extraction model and a mathematical information transformation driving model; the mathematical information driven magnetic particle imaging gradient reconstruction system includes: a conditional reference information generation module, which is used to input a system matrix and a response signal for an object to be imaged into the conditional generation model, and output conditional reference information, wherein the conditional reference information includes an initial reconstructed image for the object to be imaged or a feature map of the initial reconstructed image; a loop execution module, which is used to loop execute The following operations are performed until the number of cycles reaches a preset cycle threshold, and a target reconstructed image for the object to be imaged is obtained; the initial reasoning image is used as the current reasoning image, the current reasoning image and conditional reference information are input into the conditional gradient estimation model, and the current estimated image is output, wherein the initial reasoning image includes a Gaussian noise image; the system matrix and the response signal are input into the mathematical information iterative extraction model, and the current mathematical information is output, wherein the current mathematical information represents the mathematical information of the object to be imaged at the current number of cycles; the current estimated image and the current mathematical information are input into the mathematical information transformation driving model to optimize the current estimated image, and the optimized estimated image is output, and the above operations are performed cyclically with the optimized estimated image as the current reasoning image.

[0008] Another aspect of the present invention provides an electronic device, comprising: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method as described above.

[0009] Another aspect of the present invention provides a computer-readable storage medium storing computer-executable instructions, which are used to implement the above method when executed.

[0010] Another aspect of the present invention provides a computer program product, the computer program product comprising computer executable instructions, the instructions being used to implement the method as described above when being executed.

[0011] According to an embodiment of the present invention, the system matrix and the response signal for the object to be imaged are input into a conditional generation model to output conditional reference information; then the initial reasoning image, i.e., the Gaussian noise image, is used as the current reasoning image, and the current reasoning image and the conditional reference information are input into a conditional gradient estimation model to output the current estimated image; then the system matrix and the response signal are input into a mathematical information iterative extraction model to output the current mathematical information, wherein the current mathematical information represents the mathematical information for the object to be imaged at the current number of cycles; thereafter, the current estimated image and the current mathematical information are input into a mathematical information transformation driving model to optimize the current estimated image, and the optimized estimated image is output, and the above operation is cyclically performed with the optimized estimated image as the current reasoning image until the number of cycles reaches a preset cycle threshold, thereby obtaining a target reconstructed image for the object to be imaged. Since the technical solution of the present invention adopts mathematical information drive, the conditional gradient estimation model and the mathematical information iterative extraction model are combined to provide a set of good inputs for the mathematical information transformation driving model, and filtering correction is cyclically performed to implement data consistency constraints, which helps to improve the fidelity and accuracy of MPI reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The above and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0013] Figure 1 A flow chart of a magnetic particle imaging gradient reconstruction method driven by mathematical information according to an embodiment of the present invention is shown.

[0014] Figure 2 A schematic diagram of a magnetic particle imaging gradient reconstruction method driven by mathematical information according to an embodiment of the present invention is shown.

[0015] Figure 3 A structural schematic diagram of a conditional gradient estimation model according to an embodiment of the present invention is shown.

[0016] Figure 4 A schematic structural diagram of a direct feedthrough convolution block according to an embodiment of the present invention is shown.

[0017] Figure 5 A schematic structural diagram of a time feature embedding layer according to an embodiment of the present invention is shown.

[0018] Figure 6 A schematic diagram of the structure of a mathematical information iterative extraction model according to an embodiment of the present invention is shown.

[0019] Figure 7 A schematic diagram of the structure of a mathematical information transformation module according to an embodiment of the present invention is shown.

[0020] Figure 8A structural schematic diagram of a neural filter correction module according to an embodiment of the present invention is shown.

[0021] Fig. 9 A flow chart of a training method for a magnetic particle imaging gradient reconstruction model system driven by mathematical information according to an embodiment of the present invention is shown.

[0022] Fig.10 A schematic diagram of a training method for a magnetic particle imaging gradient reconstruction model system driven by mathematical information according to an embodiment of the present invention is shown.

[0023] Fig.11 A block diagram of a magnetic particle imaging gradient reconstruction system driven by mathematical information according to an embodiment of the present invention is shown.

[0024] Fig.12 A block diagram of an electronic device suitable for implementing the method described above according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0025] Below, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present invention. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of embodiments of the present invention. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of concepts of the present invention.

[0026] The terms used herein are only for describing specific embodiments and are not intended to limit the present invention. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0027] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0028] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0029] In the embodiments of the present invention, the collection, updating, analysis, processing, use, transmission, provision, disclosure, storage and other aspects of the data involved (for example, including but not limited to user personal information) are in compliance with the provisions of relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. In particular, necessary measures are taken for user personal information to prevent illegal access to user personal information data and maintain the security of user personal information, network security and national security.

[0030] In the embodiment of the present invention, before obtaining or collecting the user's personal information, the user's authorization or consent is obtained.

[0031] Figure 1 A flow chart of a magnetic particle imaging gradient reconstruction method driven by mathematical information according to an embodiment of the present invention is shown.

[0032] The mathematical information driven magnetic particle imaging gradient reconstruction method according to the embodiment of the present invention can be applied to a trained mathematical information driven magnetic particle imaging gradient reconstruction model system, and the trained mathematical information driven magnetic particle imaging gradient reconstruction model system can be applied to a magnetic particle imaging (MPI) device. The trained mathematical information driven magnetic particle imaging gradient reconstruction model system includes a conditional generation model, a conditional gradient estimation model, a mathematical information iterative extraction model, and a mathematical information transformation driving model. The mathematical information driven magnetic particle imaging gradient reconstruction method may include the following: Figure 1 Operations S110~S140 are shown.

[0033] In operation S110, the system matrix and the response signal for the object to be imaged are input into the condition generation model, and condition reference information is output, wherein the condition reference information includes an initial reconstructed image for the object to be imaged or a feature map of the initial reconstructed image.

[0034] According to an embodiment of the present invention, the system matrix is ​​a system matrix of an MPI device, which can be obtained by measurement. The response signal of the object to be imaged is generated by the MPI device.

[0035] According to an embodiment of the present invention, the initial reconstructed image may be a least squares reconstructed image or an initial iterative reconstruction result of a reconstructed MPI image of the object to be imaged. The initial reconstructed image may be calculated by iterating a first preset number of times using a conventional iterative reconstruction algorithm. For example, the initial reconstructed image may be calculated using an algebraic reconstruction technique (ART) or an alternating direction method of multipliers (ADMM). It should be noted that the initial reconstructed image is calculated by iterating a small number of steps and is an image that has not yet been reconstructed. It can be understood that the initial reconstructed image is an intermediate result of iteratively calculating the reconstructed image.

[0036] According to an embodiment of the present invention, the feature map of the initial reconstructed image may be obtained by performing feature extraction on the initial reconstructed image.

[0037] In operation S120, the initial reasoning image is used as the current reasoning image, the current reasoning image and the conditional reference information are input into the conditional gradient estimation model, and the current estimation image is output. The initial reasoning image includes a Gaussian noise image.

[0038] According to an embodiment of the present invention, the current estimation image represents an intermediate estimation result at the current cycle number.

[0039] In operation S130, the system matrix and the response signal are input into a mathematical information iterative extraction model, and current mathematical information is output, wherein the current mathematical information represents mathematical information of the object to be imaged at the current cycle number.

[0040] In operation S140, the current estimated image and the current mathematical information are input into the mathematical information transformation driving model to optimize the current estimated image, and the optimized estimated image is output. The optimized estimated image is used as the current inference image, and operations S120 to S140 are repeatedly executed until the number of loops reaches a preset loop threshold to obtain a target reconstructed image for the object to be imaged.

[0041] According to an embodiment of the present invention, the system matrix and the response signal for the object to be imaged are input into a conditional generation model to output conditional reference information; then the initial reasoning image, i.e., the Gaussian noise image, is used as the current reasoning image, and the current reasoning image and the conditional reference information are input into a conditional gradient estimation model to output the current estimated image; then the system matrix and the response signal are input into a mathematical information iterative extraction model to output the current mathematical information, wherein the current mathematical information represents the mathematical information for the object to be imaged at the current number of cycles; thereafter, the current estimated image and the current mathematical information are input into a mathematical information transformation driving model to optimize the current estimated image, and the optimized estimated image is output, and the above operation is cyclically performed with the optimized estimated image as the current reasoning image until the number of cycles reaches a preset cycle threshold, thereby obtaining a target reconstructed image for the object to be imaged. Since the technical solution of the present invention adopts mathematical information drive, the conditional gradient estimation model and the mathematical information iterative extraction model are combined to provide a set of good inputs for the mathematical information transformation driving model, and filtering correction is cyclically performed to implement data consistency constraints, which helps to improve the fidelity and accuracy of MPI reconstruction.

[0042] According to an embodiment of the present invention, the conditional gradient estimation model, the mathematical information iterative extraction model and the mathematical information transformation driving model in the mathematical information driven magnetic particle imaging gradient reconstruction model system can be formalized as a reverse stochastic differential equation that is time-conditioned, mathematical information-driven and used to propagate the probability density of the spatial concentration distribution of magnetic particles. That is, the MPI reconstruction process based on the system matrix is ​​reformulated as a reverse stochastic differential equation, as shown in the following formula (1):

[0043] (1);

[0044] in, is the drift function, is the diffusion function, is the gradient operator with respect to the spatial concentration distribution c of magnetic particles, is the probability density of the magnetic particle spatial concentration distribution c at time point t and conditional on the response signal u, It is the reverse standard Brownian motion. Specifically, in computer implementation, continuous time is discretized into several time points, with T as the initial time point and 0 as the final time point. The discretized reverse stochastic differential equation uses the reasoning result of the previous time point as the reasoning input of the next time point, and performs cyclic reasoning from time point T through a shared conditional gradient estimation model, a mathematical information iterative extraction model, and a mathematical information transformation driving model, and makes the mean of the reasoning results converge to the real magnetic particle spatial concentration distribution at time point 0, that is, the MPI reconstructed image. Usually, the reasoning input is initialized as a Gaussian noise image.

[0045] Reference below Figure 2~Figure 8 , combined with specific embodiments Figure 1 The method shown is further explained.

[0046] Figure 2 A schematic diagram of a magnetic particle imaging gradient reconstruction method driven by mathematical information according to an embodiment of the present invention is shown.

[0047] The mathematical information driven magnetic particle imaging gradient reconstruction method in this embodiment discretizes the continuous time into several time points, with T as the initial time point and 0 as the final time point. Figure 2 In the example, the reasoning process at time point t can include the following steps:

[0048] Taking T as the initial time point, at the current time point t=T, that is, when the current round is the first round, the system matrix 210 and the response signal 220 are input into the conditional generation model 230, and the conditional reference information is output; then, the conditional reference information is input into the conditional gradient estimation model 250, and the initial reasoning image 240 is input into the conditional gradient estimation model 250 as the current reasoning image; the conditional gradient estimation model 250 processes and outputs the current estimated image according to the conditional reference information and the initial reasoning image 240. On this basis, the system matrix 210 and the response signal 220 are input into the mathematical information iteration module of the mathematical information iteration extraction model 260, and at the same time, the driving image output by the mathematical information iteration module at the previous time point t+1 is input into the mathematical information iteration module to calculate the driving image at the current time point t, and the driving image at the current time point t is input into the mathematical information extraction module to extract mathematical information, and the current mathematical information is output. Then, the current estimated image output by the conditional gradient estimation model 250 and the current mathematical information output by the mathematical information iterative extraction model 260 are spliced ​​and input into the mathematical information transformation driving model 270, and the current estimated image is optimized using the current mathematical information to output the optimized estimated image; and the optimized estimated image is input into the conditional gradient estimation model 250 as the current reasoning image for the next round of reasoning, that is, when 0<t<T, the optimized estimated image is input into the conditional gradient estimation model 250 as the current reasoning image for reasoning, and the reasoning is repeated until t=0, and the optimized estimated image output by the mathematical information transformation driving model 270 is used as the target reconstructed image 280.

[0049] According to an embodiment of the present invention, the conditional gradient estimation model includes a feature extraction module and a gradient estimation module; the current reasoning image and conditional reference information are input into the conditional gradient estimation model, and outputting the current estimated image includes: inputting the current reasoning image and conditional reference information into the feature extraction module for feature extraction, and outputting the current image features, wherein the current image features represent the features of the current reasoning image extracted under the condition of the conditional reference information; inputting the current image features into the gradient estimation module, estimating the conditional logarithmic probability density gradient of the current reasoning image, and calculating the current estimated image based on the conditional logarithmic probability density gradient.

[0050] It should be noted that, for the convenience of description, the embodiment of the present invention records the conditional logarithmic probability density gradient as the score, and marks the expected logarithmic probability density gradient as the expected score. For example, the current expected score is the current expected logarithmic probability density gradient, and the current estimated score is the conditional logarithmic probability density gradient of the current estimate.

[0051] According to an embodiment of the present invention, in the reasoning at any time point t, the feature extraction module connects and extracts the conditional reference information and the current reasoning image, i.e., the optimized estimated image (0<t<T) or the initial reasoning image (t=T) at the previous time point t+1, by channel to obtain the current image features as the input of the gradient estimation module; the gradient estimation module estimates the score of the current reasoning image and calculates the current estimated image based on the score.

[0052] Figure 3 A structural schematic diagram of a conditional gradient estimation model according to an embodiment of the present invention is shown.

[0053] like Figure 3 As shown, the conditional gradient estimation model includes a feature extraction module 310 and a gradient estimation module 320. The feature extraction module 310 includes three feature extractors connected in sequence, which serve as a first feature extractor 311, a second feature extractor 312, and a third feature extractor 313 respectively. The first feature extractor 311, the second feature extractor 312, and the third feature extractor 313 each extract a feature, and the scale of the extracted features is halved to obtain three groups of features in total, and the three groups of features are input into the gradient estimation module.

[0054] The gradient estimation module 320 includes a time encoder 321, a feature-score transformer 322, three score estimators connected in sequence as a first score estimator 323, a second score estimator 324, a third score estimator 325 and an image calculator 326. Among them, the time encoder 321 is used to encode the time point t into a time feature; the feature-score converter 322 is used to convert the output of the third feature extractor 313 into a score domain; three score estimators connected in sequence are used to estimate the score scale by scale; the first score estimator 323 takes the time feature output by the time encoder 321, the output of the third feature extractor 313 connected by channel and the output of the feature-score converter 322 as input, the second score estimator 324 takes the time feature output by the time encoder 321, the output of the second feature extractor 312 connected by channel and the output of the first score estimator 323 as input, and the third score estimator 325 takes the time feature output by the time encoder 321, the output of the first feature extractor 311 connected by channel and the output of the second score estimator 324 as input; the output of the third score estimator 325 is used as the final score, and then the score is input into the image calculator 326, and the estimated image at the current time point t is calculated in combination with the score to obtain the current estimated image. The image calculator may include the formula (1) shown above.

[0055] It should be noted that in this embodiment, is the score at the current time point, corresponding to the score output by the third score estimator.

[0056] According to an embodiment of the present invention, each feature extractor includes three direct feedthrough convolution self-attention blocks and a feature downsampler, and the input is continuously passed through the three direct feedthrough convolution self-attention blocks to extract features, and the feature downsampler is used to halve the scale of the extracted features as the output of the feature extractor. The direct feedthrough convolution self-attention block may include a direct feedthrough convolution block and a global self-attention layer connected in sequence; the feature downsampler may include a 3×3 convolution layer and a maximum pooling layer connected in sequence.

[0057] According to an embodiment of the present invention, the global self-attention layer normalizes the input feature groups and calculates the learnable vectors Q, K, and V in combination with the multi-channel convolution layer; then Q and K are combined to calculate the weight matrix A through dot product and soft maximization; then, based on A and V, the attention vector is calculated as the output through weighted summation.

[0058] According to an embodiment of the present invention, the temporal encoder includes a sine-cosine coding layer and a fully connected coding unit connected in sequence; the feature-score transformer includes a direct feedthrough convolution self-attention block and a direct feedthrough convolution block connected in sequence, and each score estimator includes three direct feedthrough convolution blocks and a score upsampler; the first score estimator, the second score estimator, and the third score estimator continuously pass the input through three direct feedthrough convolution blocks to estimate the score at the current scale, and use the score upsampler to multiply the estimated score scale.

[0059] According to an embodiment of the present invention, the sine-cosine coding layer first calculates the half dimension half=N / 2 by the initially given time feature dimension N; combines half to calculate the coding base base=ln(10000) / half by logarithm and division; combines base to calculate the precoding value of each feature dimension by exponentiation and multiplication ; Multiply the time point t with the scaling factor scale, and calculate the outer product outer of the multiplication result and pre; take the sine function and cosine function of outer respectively, and connect them by channel as the output.

[0060] According to an embodiment of the present invention, the fully connected encoding unit includes a linear layer, a SiLU activation layer, and a linear layer connected in sequence.

[0061] According to an embodiment of the present invention, the scoring upsampler includes a nearest neighbor upsampling layer and a 3×3 convolutional layer connected in sequence.

[0062] Figure 4 A schematic structural diagram of a direct feedthrough convolution block according to an embodiment of the present invention is shown.

[0063] like Figure 4 As shown, the direct feedthrough convolution block 410 may include a SiLU activation layer 411, a 3×3 convolution layer 412, a temporal feature embedding layer 413, a SiLU activation layer 414, a 3×3 convolution layer 415 and a jumper connection from input to output, which are connected in sequence.

[0064] Figure 5 A schematic structural diagram of a time feature embedding layer according to an embodiment of the present invention is shown.

[0065] like Figure 5 As shown, the temporal feature embedding layer 413 includes a SiLU activation layer 4131, a linear layer 4132, and a jumper connection from input to output, wherein the temporal feature is input into the SiLU activation layer.

[0066] According to an embodiment of the present invention, a mathematical information iterative extraction model includes a mathematical information iterative module and a mathematical information extraction module; inputting a system matrix and a response signal into the mathematical information iterative extraction model, and outputting the current mathematical information includes: inputting the system matrix and the response signal into the mathematical information iterative module, calculating the current iteration result according to the system matrix, the response signal and the iteration result output by the mathematical information iterative module in the previous round, and obtaining the current driving image, wherein the current driving image includes mathematical information; inputting the current driving image into the mathematical information extraction module, extracting the mathematical information contained in the current driving image, and outputting the current mathematical information.

[0067] According to an embodiment of the present invention, the mathematical information iteration module can be formalized as an iterative linear inverse problem solver, and the number of iteration steps can be formalized as discrete time points, which is used to iteratively calculate the MPI image as the current driving image through the system matrix and the response signal, combined with the iterative result of the previous time point; wherein the solver can be an iterative reconstruction algorithm solver, such as an ART solver or an ADMM solver.

[0068] According to an embodiment of the present invention, calculating the current iteration result according to the system matrix, the response signal and the iteration result output by the mathematical information iteration module in the previous round includes: calculating the intermediate result of the reconstructed image by iterating the second preset number of times using the iterative reconstruction algorithm according to the system matrix, the response signal and the iteration result output by the mathematical information iteration module in the previous round, and obtaining the current iteration result. The second preset number of times can be a small number of iterations.

[0069] According to an embodiment of the present invention, the mathematical information extraction module includes three direct feedthrough convolutional self-attention blocks connected in sequence and a time encoder that inputs time features to the three direct feedthrough convolutional self-attention blocks.

[0070] Figure 6 A schematic diagram of the structure of a mathematical information iterative extraction model according to an embodiment of the present invention is shown.

[0071] like Figure 6As shown, the mathematical information iterative extraction model includes a mathematical information iterative module 610 and a mathematical information extraction module 620 . The mathematical information extraction module 620 includes a direct feedthrough convolution self-attention block 621, a direct feedthrough convolution self-attention block 622, a direct feedthrough convolution self-attention block 623 and a time encoder 624; the system matrix, the response signal and the iteration result of the mathematical information iteration module at the previous time point t+1 are input into the mathematical information iteration module 610, and the iteration result at the current time point t is output; the iteration result at the current time point t is input into the direct feedthrough convolution self-attention block 621, the direct feedthrough convolution self-attention block 622, and the direct feedthrough convolution self-attention block 623 in sequence, and at the same time, the time feature of the time point t is input into the direct feedthrough convolution self-attention block 621, the direct feedthrough convolution self-attention block 622, and the direct feedthrough convolution self-attention block 623 through the time encoder 624, and according to the time feature and the iteration result at the current time point t, the mathematical information contained in the iteration result at the current time point t is extracted to obtain the current mathematical information.

[0072] According to an embodiment of the present invention, a mathematical information transformation driving model includes a mathematical information transformation module and a neural filtering correction module; a current estimated image and current mathematical information are input into the mathematical information transformation driving model to optimize the current estimated image, and outputting the optimized estimated image includes: inputting the current mathematical information into the mathematical information transformation module, encoding the current mathematical information and transforming it to the domain of the current estimated image to obtain intermediate mathematical information; inputting the current estimated image and the intermediate mathematical information into the neural filtering correction module, using the intermediate mathematical information to drive the filtering correction optimization of the current estimated image to obtain the optimized estimated image.

[0073] According to an embodiment of the present invention, a mathematical information transformation driven model includes a mathematical information transformation module and a neural filter correction module; in the reasoning at any time point t, the mathematical information transformation module encodes the current mathematical information at time point t and transforms it to the domain of the current estimated image as intermediate mathematical information; the neural filter correction module combines the current estimated image and the intermediate mathematical information, drives the filter correction optimization for the current estimated image through the intermediate mathematical information, performs data consistency constraints, and the neural filter correction module outputs the optimized estimated image, that is, the reasoning result of the mathematical information driven magnetic particle imaging gradient reconstruction model system at the current time point t.

[0074] Figure 7 A schematic diagram of the structure of a mathematical information transformation module according to an embodiment of the present invention is shown.

[0075] like Figure 7As shown, the mathematical information conversion module 700 includes a direct feedthrough convolution self-attention block 710, a direct feedthrough convolution self-attention block 720, and a direct feedthrough convolution block 730 connected in sequence, and a time encoder 740 that inputs time features to the direct feedthrough convolution self-attention block 710, the direct feedthrough convolution self-attention block 720, and the direct feedthrough convolution block 730. Figure 6 The current mathematical information output by the mathematical information extraction module 620 shown in the figure is input into the mathematical information transformation module 700, which encodes the current mathematical information and transforms it into the domain of the current estimated image to obtain and output intermediate mathematical information.

[0076] Figure 8 A structural schematic diagram of a neural filter correction module according to an embodiment of the present invention is shown.

[0077] like Figure 8 As shown, the neural filter correction module 800 includes a prediction variance estimator 810, an observation variance estimator 820, and a gain controller 830. In this embodiment, the inverse stochastic differential equation can be remodeled as a set of state space equations, namely

[0078] Equation of state: (2);

[0079] Observation equation: (3);

[0080] in, Corresponding to the current estimated image at time point t output by the conditional gradient estimation model, Corresponding to the optimized estimated image at time point t+1 output by the neural filter correction module, that is, the output of the mathematical information transformation driving model, The inference noise corresponding to time point t, The observed value corresponding to time point t, The observation noise corresponding to time point t.

[0081] In one example, the neural filter correction module 800 can be formalized as a Kalman filter for the above-mentioned state-space equation, and the current estimated image is regarded as the predicted value of the Kalman filter, and the intermediate mathematical information is regarded as the observed value of the Kalman filter.

[0082] In the reasoning at any time point t, the prediction variance estimator 810 connects the current estimated image at the current time point t, the current estimated image at the previous time point t+1, the optimized estimated image at the previous time point t+1, the variance of the optimized estimated image at the previous time point t+1, and the diffusion function g(t) of the inverse stochastic differential equation (Formula (1)) at the current time point t by channel to estimate the prediction variance at the current time point t; the observation variance estimator 820 connects the prediction variance at the current time point t, the intermediate mathematical information at the current time point t, the intermediate mathematical information at the previous time point t+1, and the observation variance at the previous time point t+1 by channel to estimate the observation variance at the current time point t; the gainer 830 connects the current estimated image at the current time point t, the prediction variance at the current time point t, the intermediate mathematical information at the current time point t, and the observation variance at the current time point t by channel to calculate the optimized estimated image at the current time point t and the variance of the optimized estimated image.

[0083] According to an embodiment of the present invention, the prediction variance estimator and the observation variance estimator both include a linear layer and a variance estimation network, encode the input through the linear layer and then output the prediction / observation variance through the variance estimation network; the gainer includes a linear layer, a mean estimation network, and a variance estimation network, encode the input through the linear layer and then output the optimized estimated image and the optimized estimated image variance through the mean estimation network and the variance estimation network respectively.

[0084] According to an embodiment of the present invention, the mean estimation network includes a SiLU activation layer, a 1×1 convolutional layer, three 3×3 convolutional layers, and a jump line connection from input to output, which are connected in sequence.

[0085] According to an embodiment of the present invention, the variance estimation network includes a SiLU activation layer, a 1×1 convolution layer, a global self-attention layer, and a jump line connection from input to output, which are connected in sequence.

[0086] According to an embodiment of the present invention, by establishing a magnetic particle imaging gradient reconstruction method driven by mathematical information, the problem of instability and limited accuracy of the MPI reconstruction method based on the system matrix in solving the ill-conditioned linear inverse problem is solved, the reconstruction quality is improved, and stable, high-fidelity, and high-precision MPI reconstruction is achieved. In addition, by formalizing the MPI reconstruction process based on the system matrix into an inverse stochastic differential equation that is time-conditioned, driven by mathematical information, and used to propagate the probability density of the spatial concentration distribution of magnetic particles, there is no need to solve the linear inverse problem, and the MPI image stable reasoning is achieved; in addition, by using mathematical information drive, combining the conditional gradient estimation model and the mathematical information iterative extraction model to provide a set of good inputs for the mathematical information transformation drive model, and cyclically performing filtering correction to achieve data consistency constraints, it is helpful to improve the fidelity and accuracy of MPI reconstruction.

[0087] Fig. 9 A flow chart of a training method for a magnetic particle imaging gradient reconstruction model system driven by mathematical information according to an embodiment of the present invention is shown.

[0088] The mathematical information driven magnetic particle imaging gradient reconstruction model system includes a conditional generation model, a conditional gradient estimation model, a mathematical information iterative extraction model and a mathematical information transformation driving model. The training method of this embodiment includes the following steps: Fig. 9 Operation S910 to operation S960 are shown.

[0089] In operation S910, training sample data is acquired, where the training sample data includes a sample system matrix, a simulated MPI image, and a simulated response signal.

[0090] According to an embodiment of the present invention, the training sample data can be obtained in the following manner: measuring the sample background noise S of the MPI device b And the sample system matrix S, and combine the sample background noise and the sample system matrix to calculate the noise-free system matrix ; Combined with the noise-free system matrix and the MPI forward model , simulate the MPI image c, and then calculate the response signal under different noise levels according to the simulated MPI image to construct the simulated response signal ,Right now:

[0091] (4);

[0092] in, To simulate background noise; simulate response signal Contains the response signals under different simulated background noise conditions.

[0093] In operation S920, the sample system matrix and the simulated response signal are input into a condition generation model, and sample condition reference information is output, wherein the sample condition reference information includes an initial reconstructed image corresponding to the simulated MPI image or a feature map of the initial reconstructed image.

[0094] In operation S930, an initial reasoning image is used as a current sample reasoning image, the current sample reasoning image and sample condition reference information are input into a conditional gradient estimation model, and a current sample estimation image is output, wherein the initial reasoning image includes a Gaussian noise image.

[0095] In operation S940, the sample system matrix and the simulation response signal are input into a mathematical information iterative extraction model, and current sample mathematical information is output, wherein the current sample mathematical information represents mathematical information of the object to be imaged at the current cycle number.

[0096] In operation S950, the current sample estimation image and the current sample mathematical information are input into a mathematical information transformation driving model to optimize the current sample estimation image, and the optimized sample estimation image is output.

[0097] In operation S960, the network parameters of the mathematical information driven magnetic particle imaging gradient reconstruction model system are iteratively adjusted based on the optimized sample estimation image and the simulated MPI image until the first iteration stop condition is reached, and the model system obtained when the first iteration stop condition is reached is used as the trained mathematical information driven magnetic particle imaging gradient reconstruction model system.

[0098] According to an embodiment of the present invention, the first iteration stop condition may include minimizing the mean square error between the optimized sample estimation image and the simulated MPI image. Iteratively adjusting the network parameters of the magnetic particle imaging gradient reconstruction model system driven by mathematical information based on the optimized sample estimation image and the simulated MPI image may include: determining the mean square error between the optimized sample estimation image and the simulated MPI image as a loss value, and adjusting the network parameters of the magnetic particle imaging gradient reconstruction model system driven by mathematical information when the loss value does not meet a preset threshold.

[0099] According to an embodiment of the present invention, the processing procedures of the conditional generation model, the conditional gradient estimation model, the mathematical information iterative extraction model and the mathematical information transformation driving model are similar to the processing procedures of the conditional generation model, the conditional gradient estimation model, the mathematical information iterative extraction model and the mathematical information transformation driving model of the trained mathematical information driven magnetic particle imaging gradient reconstruction model system in the above-mentioned mathematical information driven magnetic particle imaging gradient reconstruction method, and are not repeated here.

[0100] According to an embodiment of the present invention, the training method further includes training the conditional gradient estimation model separately. It should be noted that, in the process of training the conditional gradient estimation model separately, the conditional gradient estimation model can output the score estimated by the gradient estimation module, and update the parameters of the conditional gradient estimation model by minimizing the mean square error between the expected score and the score estimated by the gradient estimation module. The specific method is as follows: input the current forward reasoning image and the sample condition reference information into the conditional gradient estimation model, estimate the current conditional logarithmic probability density gradient, that is, the current estimated score, wherein the current forward reasoning image is the forward reasoning image of the simulated MPI image under the current expected logarithmic probability density gradient, that is, the current expected score; determine the first loss value according to the current expected score and the current estimated score; iteratively adjust the network parameters of the conditional gradient estimation model based on the first loss value until the first loss value meets the second iteration stop condition, and use the model obtained when the second iteration stop condition is met as the trained conditional gradient estimation model. Among them, the second iteration stop condition can be to minimize the mean square error between the current expected score and the current estimated score.

[0101] Operation S930 may also include inputting the current sample reasoning image and the sample condition reference information into the trained conditional gradient estimation model, and outputting the current sample estimation image. Operation S960 may also include iteratively adjusting the network parameters of the mathematical information iterative extraction model and the mathematical information transformation driving model based on the optimized sample estimation image and the simulated MPI image.

[0102] According to an embodiment of the present invention, the current forward reasoning image and the current expected score can be determined by the following method: the simulated MPI image is propagated through the forward stochastic differential equation (Formula (5)) at any discrete time point t, thereby obtaining the reasoning input of the reverse stochastic differential equation at the corresponding time point t, i.e., the current forward reasoning image, and then the expected unconditional score at the corresponding time point t is calculated based on the current forward reasoning image. ; Calculate the expected score at the corresponding time point t by combining the expected unconditional score with the Bayesian formula .

[0103] (5)

[0104] Where w is the standard Brownian motion.

[0105] In one example, the total number of training epochs when training the conditional gradient estimation model alone can be 1000.

[0106] According to an embodiment of the present invention, the mathematical information iterative extraction model includes a mathematical information iterative module and a mathematical information extraction module; after the conditional gradient estimation model is trained, before the sample system matrix and the simulation response signal are input into the mathematical information iterative extraction model and the current sample mathematical information is output, it can also include: pre-training the mathematical information iterative extraction model and the mathematical information transformation driving model by using the following operations:

[0107] Input the sample system matrix and the simulation response signal into the mathematical information iteration module, calculate the current iteration result according to the sample system matrix, the simulation response signal and the iteration result output by the mathematical information iteration module in the previous round, and obtain the current sample driving image;

[0108] Input the current sample estimation image and the current sample driving image into the Kalman filter for image optimization to obtain the current sample optimized image, wherein the current sample optimized image can be used as the current pre-training label;

[0109] Input the current sample driving image into the mathematical information extraction module, extract the mathematical information contained in the current sample driving image, and output the current sample mathematical information;

[0110] Inputting the current sample estimation image and the current sample mathematical information into the mathematical information transformation driving model, optimizing the current sample estimation image, and outputting the optimized sample estimation image;

[0111] Determine a second loss value based on the current pre-training label and the optimized sample estimation image;

[0112] Iteratively adjust the network parameters of the mathematical information extraction module and the mathematical information transformation driving model based on the second loss value until the second loss value satisfies the third iteration stop condition, and use the mathematical information iterative extraction model and the mathematical information transformation driving model obtained when the third iteration stop condition is satisfied as the pre-trained mathematical information iterative extraction model and the pre-trained mathematical information transformation driving model. The third iteration stop condition may be to minimize the mean square error between the current pre-trained label and the optimized sample estimated image.

[0113] In one example, the total number of training epochs in the pre-training phase can be 500, where, in machine learning, an epoch refers to the process of inputting a dataset into the model once in its entirety.

[0114] According to an embodiment of the present invention, after pre-training of the mathematical information iterative extraction model and the mathematical information transformation driving model is completed, the pre-training parameters are pre-loaded, and the parameters of the pre-trained mathematical information extraction module and the pre-trained mathematical information transformation driving model are fine-tuned; the fine-tuning stage can specifically include: inputting the current sample driving image output by the mathematical information iterative module into the pre-trained mathematical information extraction module, extracting the mathematical information contained in the current sample driving image, and outputting the current sample mathematical information; inputting the current sample estimation image and the current sample mathematical information output by the conditional gradient estimation model into the pre-trained mathematical information transformation driving model, optimizing the current sample estimation image, and outputting the optimized sample estimation image; determining the third loss value according to the optimized sample estimation image and the simulated MPI image; fine-tuning the network parameters of the pre-trained mathematical information extraction module and the pre-trained mathematical information transformation driving model based on the third loss value until the third loss value meets the fourth iteration stop condition, and using the model system obtained when the fourth iteration stop condition is met as the trained mathematical information driven magnetic particle imaging gradient reconstruction model system. The fourth iteration stopping condition may be to minimize the mean square error between the simulated MPI image and the output of the magnetic particle imaging gradient reconstruction model system driven by mathematical information at time point 0.

[0115] In one example, the total number of training epochs during the fine-tuning phase can be 100.

[0116] Fig.10 A schematic diagram of a training method for a magnetic particle imaging gradient reconstruction model system driven by mathematical information according to an embodiment of the present invention is shown.

[0117] like Fig.10 As shown, the training method of this embodiment includes a first-stage training process for a conditional gradient estimation model and a second-stage training process for a mathematical information extraction module and a mathematical information transformation driving model.

[0118] The training method of the first stage includes: inputting the sample system matrix 1011 and the simulation response signal 1012 into the conditional generation model 1013 to obtain sample condition reference information; inputting the sample condition reference information and the current forward reasoning image 1014 into the conditional gradient estimation model 1015, and outputting the current estimated score 1017, wherein the current forward reasoning image 1014 is the forward reasoning image of the simulated MPI image under the current expected score; then determining the first loss value 1018 according to the current expected score 1016 and the current estimated score 1017; iteratively adjusting the network parameters of the conditional gradient estimation model 1015 based on the first loss value 1018 until the first loss value 1018 meets the second iteration stop condition, and using the model obtained when the second iteration stop condition is met as the trained conditional gradient estimation model.

[0119] The training method of the second stage includes a pre-training part and a fine-tuning part; the training method of the pre-training part includes:

[0120] The sample system matrix 1021 and the simulation response signal 1022 are input into the conditional generation model 1023 to obtain the conditional reference information, and the conditional reference information and the initial reasoning image (t=T) 1024 or the output of the Kalman filter in the previous round (0<t<T) are input into the conditional gradient estimation model 1025 trained in the first stage, and the sample estimation image at time point t is output; at the same time, the sample system matrix 1021, the simulation response signal 1022 and the sample driving image at time point t+1 are input into the mathematical information iteration module 1026, and the sample driving image at time point t is output; then, the sample estimation image at time point t and the sample driving image at time point t are spliced ​​and input into the Kalman filter 1027, and the sample optimization image 1028 at time point t is output, and the sample optimization image at time point t is used as the pre-trained label at time point t;

[0121] On this basis, the sample driving image at time point t is input into the mathematical information extraction module 1029, and the sample mathematical information at time point t is output; the sample estimation image at time point t and the sample mathematical information at time point t are input into the mathematical information transformation driving model 1030 for optimization, and the optimized sample estimation image at time point t is output; thereafter, the second loss value 1032 is determined according to the pre-trained label at time point t and the optimized sample estimation image at time point t, and the network parameters of the mathematical information extraction module 1029 and the mathematical information transformation driving model 1030 are iteratively adjusted based on the second loss value 1032 until the second loss value 1032 meets the third iteration stop condition, thereby obtaining the pre-trained mathematical information extraction module and the pre-trained mathematical information transformation driving model.

[0122] The training methods for the fine-tuning part include:

[0123] The sample system matrix 1041 and the simulation response signal 1042 are input into the conditional generation model 1043 to obtain the conditional reference information. The conditional reference information and the initial reasoning image (t=T) 1044 or the output (0<t<T) of the pre-trained mathematical information transformation driving model 1048 in the previous round are input into the conditional gradient estimation model 1045 trained in the first stage, and the sample estimation image at time point t is output; at the same time, the sample system matrix 1041, the simulation response signal 1042 and the sample driving image at time point t+1 are input into the mathematical information iteration module 1046, and the sample driving image at time point t is output. The sample driving image at time point t is input into the pre-trained mathematical information extraction module 1046. 47, output the mathematical information at time point t; then input the estimated image at time point t and the mathematical information at time point t into the pre-trained mathematical information transformation driving model 1048, output the optimized estimated image at time point t, and repeat this cycle until t=0, output the target reconstructed image 1049; then determine the third loss value 1051 according to the target reconstructed image 1049 and the simulated MPI image, iteratively adjust the network parameters of the pre-trained mathematical information extraction module 1047 and the pre-trained mathematical information transformation driving model 1048 based on the third loss value 1051, until the third loss value 1051 meets the fourth iteration stop condition, and obtain the trained mathematical information driven magnetic particle imaging gradient reconstruction model system.

[0124] Fig.11 A block diagram of a magnetic particle imaging gradient reconstruction system 1100 driven by mathematical information according to an embodiment of the present invention is shown.

[0125] The mathematical information driven magnetic particle imaging gradient reconstruction system 1100 can be applied to a trained mathematical information driven magnetic particle imaging gradient reconstruction model system. The trained mathematical information driven magnetic particle imaging gradient reconstruction model system includes a conditional generation model, a conditional gradient estimation model, a mathematical information iterative extraction model and a mathematical information transformation driving model. The mathematical information driven magnetic particle imaging gradient reconstruction system 1100 can include: Fig.11 The conditional reference information generating module 1110 and the loop executing module 1120 are shown.

[0126] The conditional reference information generation module 1110 is used to input the system matrix and the response signal for the object to be imaged into the conditional generation model, and output conditional reference information, wherein the conditional reference information includes an initial reconstructed image for the object to be imaged or a feature map of the initial reconstructed image.

[0127] The loop execution module 1120 is used to loop the following operations until the number of loops reaches a preset loop threshold, thereby obtaining a target reconstructed image for the object to be imaged; using the initial reasoning image as the current reasoning image, inputting the current reasoning image and conditional reference information into the conditional gradient estimation model, and outputting the current estimated image, wherein the initial reasoning image includes a Gaussian noise image; inputting the system matrix and the response signal into the mathematical information iterative extraction model, and outputting the current mathematical information, wherein the current mathematical information represents the mathematical information for the object to be imaged at the current number of loops; inputting the current estimated image and the current mathematical information into the mathematical information transformation driving model to optimize the current estimated image, outputting the optimized estimated image, and looping the above operations with the optimized estimated image as the current reasoning image.

[0128] According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits, or at least part of the functions of any one of them can be implemented in one module. According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits can be split into multiple modules for implementation. According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits can be at least partially implemented as hardware circuits, such as field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), systems on chips, systems on substrates, systems on packages, application specific integrated circuits (ASICs), or can be implemented by hardware or firmware in any other reasonable way of integrating or packaging the circuit, or by any one of the three implementation methods of software, hardware, and firmware, or by a proper combination of any of them. Alternatively, according to the embodiments of the present invention, one or more of the modules, submodules, units, and subunits can be at least partially implemented as computer program modules, and when the computer program modules are run, the corresponding functions can be executed.

[0129] For example, any multiple of the conditional reference information generation module 1110 and the loop execution module 1120 can be combined in one module / unit / sub-unit for implementation, or any one of the modules / units / sub-units can be split into multiple modules / units / sub-units. Alternatively, at least part of the functions of one or more of these modules / units / sub-units can be combined with at least part of the functions of other modules / units / sub-units and implemented in one module / unit / sub-unit. According to an embodiment of the present invention, at least one of the conditional reference information generation module 1110 and the loop execution module 1120 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware and firmware or in a suitable combination of any of them. Alternatively, at least one of the conditional reference information generating module 1110 and the loop executing module 1120 may be at least partially implemented as a computer program module, and when the computer program module is executed, the corresponding function may be executed.

[0130] It should be noted that the mathematical information driven magnetic particle imaging gradient reconstruction system part in the embodiment of the present invention corresponds to the mathematical information driven magnetic particle imaging gradient reconstruction method part in the embodiment of the present invention. The description of the mathematical information driven magnetic particle imaging gradient reconstruction system part specifically refers to the mathematical information driven magnetic particle imaging gradient reconstruction method part, which will not be repeated here.

[0131] Fig.12 A block diagram of an electronic device suitable for implementing the method described above according to an embodiment of the present invention is shown. Fig.12 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0132] like Fig.12As shown, the electronic device 1200 according to an embodiment of the present invention includes a processor 1201, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1202 or a program loaded from a storage part 1208 to a random access memory (RAM) 1203. The processor 1201 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 1201 may also include an onboard memory for caching purposes. The processor 1201 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0133] In RAM 1203, various programs and data required for the operation of electronic device 1200 are stored. Processor 1201, ROM 1202 and RAM 1203 are connected to each other via bus 1204. Processor 1201 performs various operations of the method flow according to the embodiment of the present invention by executing the program in ROM 1202 and / or RAM 1203. It should be noted that the program can also be stored in one or more memories other than ROM 1202 and RAM 1203. Processor 1201 can also perform various operations of the method flow according to the embodiment of the present invention by executing the program stored in the one or more memories.

[0134] According to an embodiment of the present invention, the electronic device 1200 may further include an input / output (I / O) interface 1205, which is also connected to the bus 1204. The electronic device 1200 may further include one or more of the following components connected to the input / output (I / O) interface 1205: an input portion 1206 including a keyboard, a mouse, etc.; an output portion 1207 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 1208 including a hard disk, etc.; and a communication portion 1209 including a network interface card such as a LAN card, a modem, etc. The communication portion 1209 performs communication processing via a network such as the Internet. A drive 1210 is also connected to the input / output (I / O) interface 1205 as needed. A removable medium 1211, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1210 as needed, so that a computer program read therefrom is installed into the storage portion 1208 as needed.

[0135] According to an embodiment of the present invention, the method flow according to an embodiment of the present invention can be implemented as a computer software program. For example, an embodiment of the present invention includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 1209, and / or installed from the removable medium 1211. When the computer program is executed by the processor 1201, the above-mentioned functions defined in the system of the embodiment of the present invention are executed. According to an embodiment of the present invention, the system, equipment, device, module, unit, etc. described above can be implemented by a computer program module.

[0136] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiment; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present invention is implemented.

[0137] According to an embodiment of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium. For example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device.

[0138] For example, according to an embodiment of the present invention, the computer-readable storage medium may include the ROM 1202 and / or the RAM 1203 described above and / or one or more memories other than the ROM 1202 and the RAM 1203 .

[0139] An embodiment of the present invention also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method provided by the embodiment of the present invention. When the computer program product runs on an electronic device, the program code is used to enable the electronic device to implement the above method provided by the embodiment of the present invention.

[0140] When the computer program is executed by the processor 1201, the above functions defined in the system / device of the embodiment of the present invention are executed. According to the embodiment of the present invention, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0141] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 1209, and / or installed from the removable medium 1211. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0142] According to an embodiment of the present invention, the program code for executing the computer program provided by the embodiment of the present invention can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0143] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram may represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box may also occur in an order different from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions. It can be understood by those skilled in the art that the features recorded in the various embodiments of the present invention can be combined and / or combined in various ways, even if such a combination or combination is not explicitly recorded in the present invention. In particular, without departing from the spirit and teachings of the present invention, the features recorded in the various embodiments of the present invention can be combined and / or combined in various ways. All these combinations and / or combinations fall within the scope of the present invention.

[0144] The embodiments of the present invention are described above. However, these embodiments are only for the purpose of illustration, and are not intended to limit the scope of the present invention. Although each embodiment is described above, it does not mean that the measures in each embodiment cannot be used in combination advantageously. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.

Claims

1. A method for gradient reconstruction of magnetic particle imaging driven by mathematical information, applied to a trained model system for gradient reconstruction of magnetic particle imaging driven by mathematical information, characterized in that: The trained mathematical information driven magnetic particle imaging gradient reconstruction model system includes a conditional generation model, a conditional gradient estimation model, a mathematical information iterative extraction model and a mathematical information transformation driving model; The method comprises: Inputting a system matrix and a response signal for the object to be imaged into the condition generation model, and outputting condition reference information, wherein the condition reference information includes an initial reconstructed image for the object to be imaged or a feature map of the initial reconstructed image; The following operations are executed cyclically until the number of cycles reaches a preset cycle threshold, thereby obtaining a target reconstructed image for the object to be imaged; Taking an initial reasoning image as a current reasoning image, inputting the current reasoning image and the conditional reference information into the conditional gradient estimation model, and outputting a current estimation image, wherein the initial reasoning image includes a Gaussian noise image; Inputting the system matrix and the response signal into the mathematical information iterative extraction model, and outputting current mathematical information, wherein the current mathematical information represents the mathematical information of the object to be imaged at the current cycle number; The current estimated image and the current mathematical information are input into the mathematical information transformation driving model to optimize the current estimated image, output the optimized estimated image, and use the optimized estimated image as the current reasoning image to loop through the above operations.

2. The method according to claim 1, characterized in that The conditional reference information includes an initial reconstructed image; The step of inputting the system matrix and the response signal to the object to be imaged into the condition generation model and outputting the condition reference information comprises: The system matrix and the response signal for the object to be imaged are input into the conditional generation model, and the intermediate result of the reconstructed image is calculated by iterating the first preset number of times using an iterative reconstruction algorithm according to the system matrix and the response signal to obtain the initial reconstructed image.

3. The method according to claim 1, characterized in that The conditional gradient estimation model includes a feature extraction module and a gradient estimation module; The step of inputting the current reasoning image and the conditional reference information into the conditional gradient estimation model and outputting the current estimated image comprises: Inputting the current reasoning image and the conditional reference information into the feature extraction module for feature extraction, and outputting current image features, wherein the current image features represent features of the current reasoning image extracted under the conditional reference information; The current image features are input into the gradient estimation module, the conditional logarithmic probability density gradient of the current inference image is estimated, and the current estimated image is calculated based on the conditional logarithmic probability density gradient.

4. The method according to claim 1, characterized in that: The mathematical information iterative extraction model includes a mathematical information iterative module and a mathematical information extraction module; The step of inputting the system matrix and the response signal into the mathematical information iterative extraction model and outputting the current mathematical information comprises: Inputting the system matrix and the response signal into the mathematical information iteration module, calculating the current iteration result according to the system matrix, the response signal and the iteration result output by the mathematical information iteration module in the previous round, and obtaining the current driving image, wherein the current driving image includes mathematical information; The current driving image is input into the mathematical information extraction module, the mathematical information contained in the current driving image is extracted, and the current mathematical information is output.

5. The method according to claim 4, characterized in that The calculating of the current iteration result according to the system matrix, the response signal and the iteration result output by the mathematical information iteration module in the previous round comprises: According to the system matrix, the response signal and the iteration result output by the mathematical information iteration module in the previous round, an intermediate result of the reconstructed image is calculated by iterating the iterative reconstruction algorithm for a second preset number of times to obtain the current iteration result.

6. The method according to claim 1, characterized in that The mathematical information transformation driving model includes a mathematical information transformation module and a neural filter correction module; The step of inputting the current estimated image and the current mathematical information into the mathematical information transformation driving model to optimize the current estimated image, and outputting the optimized estimated image comprises: Inputting the current mathematical information into the mathematical information transformation module, encoding the current mathematical information and transforming it into the domain of the current estimated image to obtain intermediate mathematical information; The current estimated image and the intermediate mathematical information are input into the neural filter correction module, and the intermediate mathematical information is used to drive the filter correction optimization of the current estimated image to obtain the optimized estimated image.

7. A training method for a magnetic particle imaging gradient reconstruction model system driven by mathematical information, characterized in that: The mathematical information driven magnetic particle imaging gradient reconstruction model system comprises a conditional generation model, a conditional gradient estimation model, a mathematical information iterative extraction model and a mathematical information transformation driving model; the method comprises: Acquire training sample data, wherein the training sample data includes a sample system matrix, a simulated MPI image, and a simulated response signal; Inputting the sample system matrix and the simulated response signal into the condition generation model, and outputting sample condition reference information, wherein the sample condition reference information includes an initial reconstructed image corresponding to the simulated MPI image or a feature map of the initial reconstructed image; Taking an initial reasoning image as a current sample reasoning image, inputting the current sample reasoning image and the sample condition reference information into the conditional gradient estimation model, and outputting a current sample estimation image, wherein the initial reasoning image includes a Gaussian noise image; Inputting the sample system matrix and the simulation response signal into the mathematical information iterative extraction model, and outputting current sample mathematical information, wherein the current sample mathematical information represents mathematical information for the object to be imaged at the current cycle number; Inputting the current sample estimation image and the current sample mathematical information into the mathematical information transformation driving model to optimize the current sample estimation image, and outputting the optimized sample estimation image; Based on the optimized sample estimation image and the simulated MPI image, the network parameters of the mathematical information driven magnetic particle imaging gradient reconstruction model system are iteratively adjusted until the first iteration stopping condition is reached, and the model system obtained when the first iteration stopping condition is reached is used as the trained mathematical information driven magnetic particle imaging gradient reconstruction model system.

8. The method according to claim 7, characterized in that Also includes: Inputting the current forward reasoning image and the sample condition reference information into the conditional gradient estimation model to estimate the current conditional logarithmic probability density gradient, wherein the current forward reasoning image is a forward reasoning image of the simulated MPI image under the current expected logarithmic probability density gradient; Determine a first loss value according to the current expected logarithmic probability density gradient and the current conditional logarithmic probability density gradient; Iteratively adjusting the network parameters of the conditional gradient estimation model based on the first loss value until the first loss value satisfies a second iteration stop condition, and using the model obtained when the second iteration stop condition is satisfied as the trained conditional gradient estimation model; The step of inputting the current sample reasoning image and the sample condition reference information into the conditional gradient estimation model and outputting the current sample estimation image comprises: Inputting the current sample reasoning image and the sample condition reference information into the trained conditional gradient estimation model, and outputting the current sample estimation image; The network parameters of the magnetic particle imaging gradient reconstruction model system that iteratively adjusts mathematical information based on the optimized sample estimation image and the simulated MPI image include: The network parameters of the mathematical information iterative extraction model and the mathematical information transformation driving model are iteratively adjusted based on the optimized sample estimation image and the simulated MPI image.

9. The method according to claim 8, characterized in that The mathematical information iterative extraction model includes a mathematical information iterative module and a mathematical information extraction module; Before inputting the sample system matrix and the simulation response signal into the mathematical information iterative extraction model and outputting the current sample mathematical information, the method further includes: pre-training the mathematical information iterative extraction model and the mathematical information transformation driving model by performing the following operations: Inputting the sample system matrix and the simulation response signal into the mathematical information iteration module, calculating the current iteration result according to the sample system matrix, the simulation response signal and the iteration result output by the mathematical information iteration module in the previous round, and obtaining the current sample driving image; Input the current sample estimation image and the current sample driving image into a Kalman filter for image optimization, and output a current sample optimized image, wherein the current sample optimized image is a current pre-trained label; Input the current sample driving image into the mathematical information extraction module, extract the mathematical information contained in the current sample driving image, and output the current sample mathematical information; Inputting the current sample estimation image and the current sample mathematical information into the mathematical information transformation driving model, optimizing the current sample estimation image, and outputting the optimized sample estimation image; Determine a second loss value according to the current pre-training label and the optimized sample estimation image; Iteratively adjust the network parameters of the mathematical information extraction module and the mathematical information transformation driving model based on the second loss value until the second loss value satisfies a third iterative stopping condition, and use the mathematical information iterative extraction model and the mathematical information transformation driving model obtained when the third iterative stopping condition is satisfied as the pre-trained mathematical information iterative extraction model and the pre-trained mathematical information transformation driving model.

10. A mathematical information driven magnetic particle imaging gradient reconstruction model system, characterized in that: The mathematical information driven magnetic particle imaging gradient reconstruction model system includes a conditional generation model, a conditional gradient estimation model, a mathematical information iterative extraction model and a mathematical information transformation driving model; The conditional generation model is used to generate conditional reference information according to the system matrix and the response signal for the object to be imaged, wherein the conditional reference information includes an initial reconstructed image for the object to be imaged or a feature map of the initial reconstructed image; The conditional gradient estimation model is used to use the initial reasoning image or the optimized estimated image output by the mathematical information transformation driving model as the current reasoning image, and generate the current estimated image according to the current reasoning image and the conditional reference information, wherein the initial reasoning image includes a Gaussian noise image; The mathematical information iterative extraction model is used to generate current mathematical information according to the system matrix and the response signal, wherein the current mathematical information represents the mathematical information of the object to be imaged at the current cycle number; The mathematical information transformation driving model is used to optimize the current estimated image according to the current estimated image and the current mathematical information to generate an optimized estimated image.

Images