Magnetic particle high-resolution reconstruction method based on fast feature fusion

Abstract

The present application belongs to the field of magnetic particle imaging, and particularly relates to a magnetic particle high-resolution reconstruction method based on fast feature fusion, aiming at solving the problems of slow system matrix calibration, low resolution of reconstructed magnetic particle image, limited signal-to-noise ratio and the like of the existing magnetic particle reconstruction method. The method comprises: collecting a system matrix generated using a standard particle sample as a low-resolution system matrix; collecting a voltage signal of a phantom to be imaged; inputting the low-resolution system matrix into a trained fast feature fusion Transformer model to obtain a high-resolution system matrix; and reconstructing the high-resolution system matrix based on the voltage signal to obtain a high-resolution MPI image. The present application improves the resolution of the MPI image by fast super-resolution of the system matrix, avoids the time-consuming calibration in the system matrix reconstruction, and can improve the resolution and signal-to-noise ratio of any system matrix without regard to the scanning mode of the system matrix.

Description

Technical Field

[0001] The present invention belongs to the field of magnetic particle imaging, and in particular relates to a magnetic particle high-resolution reconstruction method based on rapid feature fusion. Background Art

[0002] In clinical testing and diagnosis, how to accurately achieve highly sensitive, high-resolution in vivo imaging and localization of target lesions has always been a hot topic and a challenging issue in international research. Magnetic particle imaging (MPI) is a new tracer-based imaging method with advantages such as being radiation-free and highly sensitive. Currently, magnetic particle imaging devices have shown broad application prospects in pre-clinical research, including early lesion detection and tumor detection. However, the current resolution of MPI cannot meet application requirements. In the system matrix method, a high-resolution system matrix needs to be acquired by collecting signals at each voxel position one by one. This process is complex and time-consuming, and the acquired system matrix is ​​extremely sensitive to system parameters. This limitation limits the application of MPI in rapid imaging and dynamic monitoring.

[0003] Thanks to the development of big data and computing power, deep learning-based methods for medical image resolution enhancement have broad prospects. By introducing super-resolution algorithms, a high-resolution system matrix can be recovered from a low-resolution system matrix, eliminating the need for tedious system matrix calibration and significantly reducing calibration time.

[0004] Based on this, the present invention proposes a magnetic particle high-resolution reconstruction method based on rapid feature fusion. Summary of the Invention

[0005] In order to solve the above-mentioned problems in the prior art, namely, to solve the problems of slow system matrix calibration, low resolution of reconstructed magnetic particle images, and limited signal-to-noise ratio in the existing magnetic particle reconstruction methods, the first aspect of the present invention proposes a magnetic particle high-resolution reconstruction method based on fast feature fusion, which comprises:

[0006] S100, acquiring a system matrix generated using a standard particle sample as a low-resolution system matrix; acquiring a voltage signal of the phantom to be imaged within the imaging field of view of the magnetic particle imaging device;

[0007] S200, inputting the low-resolution system matrix into a trained fast feature fusion Transformer model to obtain a high-resolution system matrix;

[0008] S300, reconstructing the high-resolution system matrix based on the voltage signal to obtain a high-resolution MPI image;

[0009] The fast feature fusion Transformer model includes a shallow feature extractor, a local feature fusion convolutional neural network, a global feature fusion Transformer module, and a high-resolution system matrix recovery module.

[0010] The shallow feature extractor is used to extract shallow features of the low-resolution system matrix;

[0011] The local feature fusion convolutional neural network is used to perform local feature aggregation on the shallow features to obtain local features;

[0012] The global feature fusion Transformer module is used to perform global feature extraction on the shallow features through continuous global Transformer layers to obtain global features;

[0013] The high-resolution system matrix recovery module is used to fuse the shallow features, the local features, and the global features, and obtain a high-resolution system matrix through processing through a convolution layer and a pixel shuffle layer.

[0014] In some preferred embodiments, the fast feature fusion Transformer model is trained by:

[0015] A100 uses a simulation platform to generate system matrices with different gradients and particle sizes by adjusting the gradient value and the size of the magnetic particles, which serve as high-resolution system matrices.

[0016] A200, downsampling the high-resolution system matrix to obtain a low-resolution system matrix;

[0017] A300, inputs the low-resolution system matrix into a pre-built fast feature fusion Transformer model to obtain a high-resolution system matrix after super-resolution;

[0018] A400, combining the super-resolved high-resolution system matrix and its corresponding labels, calculates a loss value using a pre-built loss function, and then updates the parameters of a pre-built fast feature fusion Transformer model;

[0019] Step A500: loop A300-A400 until a trained fast feature fusion Transformer model is obtained.

[0020] In some preferred embodiments, the shallow feature extractor is composed of convolutional layers of a set size.

[0021] In some preferred embodiments, the local feature fusion convolutional neural network includes three local feature aggregation units and a 1×1 convolution;

[0022] Each local feature aggregation unit includes a first local feature aggregation module, a channel aggregation module, and a second local feature aggregation module connected in sequence;

[0023] The channel aggregation module is constructed based on a global pooling layer, a 1×1 convolution, a RELU activation layer, a 1×1 convolution, a sigmoid layer, a cross product layer, and a fusion layer connected in sequence; the cross product layer is used to multiply the output of the first local feature aggregation module with the output of the sigmoid layer; the fusion layer is used to fuse the output of the cross product layer with the first feature;

[0024] Except for the first local feature aggregation unit, the first feature input to the fusion layer in other local feature aggregation units is the output of the second local feature aggregation module in the previous local feature aggregation unit; the first feature input to the fusion layer in the first local feature aggregation unit is the output of the first local feature aggregation module in the last local feature aggregation unit;

[0025] The input of the first local feature aggregation unit is the shallow feature; the input of other local feature aggregation units is the output of the first local feature aggregation unit in the previous local feature aggregation unit;

[0026] The outputs of the local feature aggregation modules in the first local feature aggregation unit, the second local feature aggregation unit, and the third local feature aggregation unit are spliced ​​along the channel dimension, and after splicing, they are processed by 1×1 convolution as the output of the local feature fusion convolutional neural network;

[0027] The local feature aggregation module includes multiple polarization fusion modules connected in sequence, each polarization fusion module is connected to a 1×1 convolution, and the input of each 1×1 convolution is the output of all previous polarization fusion modules and the input of the first polarization fusion module. The output of the last 1×1 convolution is fused with the input of the first polarization fusion module to serve as the output of the local feature aggregation module;

[0028] The polarization fusion module includes six 1×1 convolutions and a polarization attention module; the six 1×1 convolutions serve as the first convolution layer, the second convolution layer, the third convolution layer, the fourth convolution layer, the fifth convolution layer, and the sixth convolution layer respectively;

[0029] The output of the first convolutional layer serves as the input of the second convolutional layer and the third convolutional layer respectively; the output of the second convolutional layer, the output of the third convolutional layer and the output of the first convolutional layer are spliced ​​along the channel dimension as the input of the fifth convolutional layer; the output of the second convolutional layer serves as the input of the fourth convolutional layer; the output of the third convolutional layer, the output of the fourth convolutional layer and the output of the fifth convolutional layer are spliced ​​along the channel dimension as the input of the polarized attention module; the output of the polarized attention module serves as the output of the sixth convolutional layer; the output of the sixth convolutional layer is fused with the output of the first convolutional layer as the output of the polarized fusion module.

[0030] In some preferred embodiments, the polarized attention module includes a channel attention module and a spatial attention module; the output of the polarized attention module is the concatenation of the output of the channel attention module and the output of the spatial attention module;

[0031] The output of the channel attention module is:

[0032]

[0033]

[0034] in, represents the output of the channel attention module, represents the input of the polarized attention module, represents the channel characteristics, 、 and Represent the sigmoid function, reshaping operation, and SoftMax function respectively. Represents convolution.

[0035] In some preferred embodiments, the output of the spatial attention module is:

[0036]

[0037]

[0038] in, represents the output of the spatial attention module, Represents spatial features, Represents global pooling.

[0039] In some preferred embodiments, the global feature fusion Transformer module includes multiple global Transformer layers; the global Transformer layer is constructed based on a strip window attention layer, a first fusion layer, a feedforward network, and a second fusion layer connected in sequence;

[0040] The input of the strip window attention layer in the first global Transformer layer is the shallow features; except for the first one, the input of the strip window attention layer in other global Transformer layers is the output of the previous global Transformer layer;

[0041] The first fusion layer is used to fuse the input and output of the strip window attention layer; the second fusion layer is used to fuse the input and output of the feedforward network.

[0042] In some preferred embodiments, the stripe window attention layer evenly divides the input shallow features along the channel dimension, and applies vertical stripe windows and horizontal stripe windows to capture features respectively;

[0043] Self-attention calculation is performed on the features captured by the vertical stripe window and the features captured by the horizontal stripe window respectively, and then they are spliced ​​along the channel dimension as the output of the strip window attention layer.

[0044] In some preferred embodiments, the self-attention calculation is performed on the features captured by the vertical stripe window, and the method is as follows:

[0045]

[0046] in, Represents the result of self-attention calculation, 、 They represent the query features and value features obtained by mapping the captured features through different weight matrices. represents the scale factor, T Indicates transpose.

[0047] In some preferred embodiments, the loss function of the fast feature fusion Transformer model during training is:

[0048]

[0049] in, represents the total loss, Keep the loss for details, is the multi-scale gradient consistency loss, and are all weight parameters;

[0050]

[0051]

[0052] in, Indicates the current index position in the system matrix i Neighborhood, represents other locations in the neighborhood of this location, is the total number of pixels in the system matrix, and Represent the reconstructed high-resolution system moment and the reference high-resolution system matrix respectively. i The pixel value of the position, and Represent the reconstructed high-resolution system moment and the reference high-resolution system matrix respectively. j The pixel value of the position, and Respectively expressed in scale s The reconstructed high-resolution system matrix and the reference high-resolution system matrix under is the gradient operator.

[0053] Beneficial effects of the present invention:

[0054] 1) The present invention can use super-resolution technology to innovatively combine CNN and Transformer to learn and extract local and global features of the MPI system matrix, and directly super-resolve the low-resolution system matrix in an end-to-end manner;

[0055] 2) The present invention can super-resolve a low-resolution system matrix of any grid size, proactively predict a high-resolution system matrix, and thus obtain a high-resolution MPI image;

[0056] 3) The present invention can avoid complex and time-consuming system matrix calibration. Based on the existing low-resolution system matrix, the calibration time is greatly reduced, and a high signal-to-noise ratio and high-resolution system matrix is ​​quickly obtained;

[0057] 4) The present invention is applicable to all types of system matrix reconstruction methods, including magnetic field free point and magnetic field free line scanning methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Other features, objects and advantages of the present application will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings.

[0059] Figure 1 1 is a flow chart of a method for high-resolution reconstruction of magnetic particles based on rapid feature fusion according to an embodiment of the present invention;

[0060] Figure 2 Schematic diagram of the training and application process of a fast feature fusion Transformer model according to an embodiment of the present invention;

[0061] Figure 3 2. It is a schematic diagram of the structure of a fast feature fusion Transformer model according to an embodiment of the present invention;

[0062] Figure 4 1 is a schematic diagram of the structure of a local feature fusion convolutional neural network of a fast feature fusion Transformer model according to an embodiment of the present invention;

[0063] Figure 5 1 is a structural diagram of a global feature fusion Transformer of a fast feature fusion Transformer model according to an embodiment of the present invention. DETAILED DESCRIPTION

[0064] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0065] The present application will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are intended only to illustrate the relevant invention and are not intended to limit the invention. It should also be noted that, for ease of description, only portions relevant to the relevant invention are shown in the accompanying drawings.

[0066] It should be noted that, unless there is any conflict, the embodiments and features in the embodiments of this application can be combined with each other.

[0067] The magnetic particle high-resolution reconstruction method based on rapid feature fusion of the present invention is as follows: Figure 1 As shown, the following steps are included:

[0068] S100, acquiring a system matrix generated using a standard particle sample as a low-resolution system matrix; acquiring a voltage signal of the phantom to be imaged within the imaging field of view of the magnetic particle imaging device;

[0069] S200, inputting the low-resolution system matrix into a trained fast feature fusion Transformer model to obtain a high-resolution system matrix;

[0070] S300, reconstructing the high-resolution system matrix based on the voltage signal to obtain a high-resolution MPI image;

[0071] The fast feature fusion Transformer model includes a shallow feature extractor, a local feature fusion convolutional neural network, a global feature fusion Transformer module, and a high-resolution system matrix recovery module.

[0072] The shallow feature extractor is used to extract shallow features of the low-resolution system matrix;

[0073] The local feature fusion convolutional neural network is used to perform local feature aggregation on the shallow features to obtain local features;

[0074] The global feature fusion Transformer module is used to evenly split the shallow features along the channel dimension, perform self-attention calculation after even splitting, and splice along the channel dimension to obtain global features;

[0075] The high-resolution system matrix recovery module is used to fuse the shallow features, the local features, and the global features, and obtain a high-resolution system matrix through processing through a convolution layer and a pixel shuffle layer.

[0076] In order to more clearly illustrate the method for high-resolution reconstruction of magnetic particles based on rapid feature fusion of the present invention, each step in an embodiment of the method of the present invention is described in detail below with reference to the accompanying drawings.

[0077] In the following embodiments, the training method of the fast feature fusion Transformer model is first described, and then the process of obtaining high-resolution MPI images by the magnetic particle high-resolution reconstruction method based on fast feature fusion is explained.

[0078] 1. Fast feature fusion Transformer model training method, such as Figure 2 shown

[0079] A100 uses a simulation platform to generate system matrices with different gradients and particle sizes by adjusting the gradient value and the size of the magnetic particles, which serve as high-resolution system matrices.

[0080] In this embodiment, the data set is generated by simulating the magnetic particle signal generation process in real situations using a system matrix reconstruction simulation platform. Different system matrices are generated by adjusting the gradient value and particle size, i.e., a high-resolution system matrix.

[0081] In addition, before the simulation begins, various parameters of the magnetic particles must be input, including vacuum magnetic permeability, particle diameter, saturation magnetization, magnetic torque, temperature, Boltzmann constant, and magnetic field gradient.

[0082] A200, downsampling the high-resolution system matrix to obtain a low-resolution system matrix;

[0083] A300, inputs the low-resolution system matrix into a pre-built fast feature fusion Transformer model to obtain a high-resolution system matrix after super-resolution;

[0084] In this embodiment, the fast feature fusion Transformer model includes a shallow feature extractor, a local feature fusion convolutional neural network, a global feature fusion Transformer module and a high-resolution system matrix recovery module. Figure 3 As shown;

[0085] Shallow feature extractor is used to extract shallow features of low-resolution system matrix (in actual application, it is the collected low-resolution system matrix). The extracted shallow features are F S ; The shallow feature extractor is composed of a convolutional layer of a set size, and the present invention preferably uses a 3×3 convolutional layer.

[0086] Local feature fusion convolutional neural network, such as Figure 4 As shown in (a), for extracting local features from the system matrix, the local feature fusion convolutional neural network includes a symmetrical convolutional neural network architecture, including a three-layer network (i.e., three local feature aggregation units), each layer of the network includes two local feature aggregation modules and a channel aggregation module; the two local feature aggregation modules are located at the top and bottom (respectively as the first local feature aggregation module and the second local feature aggregation module), and the channel aggregation module is located between the two local feature aggregation modules; the three-layer network is placed in parallel;

[0087] The local feature aggregation module consists of three layers of polarization fusion modules and three layers of 1×1 convolution layers. Except for the first local feature aggregation unit, the first feature of the fusion layer input in other local feature aggregation units is the output of the second local feature aggregation module in the previous local feature aggregation unit; the first feature of the fusion layer input in the first local feature aggregation unit is the output of the first local feature aggregation module in the last local feature aggregation unit; the input of the first local feature aggregation unit is the shallow feature; the input of other local feature aggregation units is the output of the first local feature aggregation unit in the previous local feature aggregation unit; the outputs of each local feature aggregation module in the first local feature aggregation unit, the second local feature aggregation unit and the third local feature aggregation unit are spliced ​​along the channel dimension, and after splicing, they are processed by 1×1 convolution as the output of the local feature fusion convolutional neural network; that is, the local feature aggregation module fuses features from different levels through dense connections, as shown in the following formula:

[0088] (1)

[0089] in, and Represents the local feature aggregation module of the i-th layer in the top and bottom branches respectively, and the two modules share parameters. represents the i-th layer channel aggregation module; the channel aggregation module is constructed based on the global pooling layer, 1×1 convolution, RELU activation layer, 1×1 convolution, sigmoid layer, cross product layer, and fusion layer connected in sequence; the cross product layer is used to multiply the output of the first local feature aggregation module with the output of the sigmoid layer; the fusion layer is used to fuse the output of the cross product layer with the first feature;

[0090] The outputs of all local feature aggregation modules are concatenated and passed through a 1×1 convolutional layer to obtain the final local features. F LS .

[0091] Local feature aggregation module, such as Figure 4 As shown in (b), it consists of three layers of polarization fusion modules and three layers of 1×1 convolution layers (specifically, the local feature aggregation module includes multiple polarization fusion modules connected in sequence, each polarization fusion module is connected to a 1×1 convolution, and the input of each 1×1 convolution is the output of all previous polarization fusion modules and the input of the first polarization fusion module. The output of the last 1×1 convolution is fused with the input of the first polarization fusion module to serve as the output of the local feature aggregation module). The local feature aggregation module fuses features from different levels through dense connections.

[0092] (2)

[0093] and represents the input and output of the local feature aggregation module, represents the polarization fusion module at layer i;

[0094] The polarization fusion module uses a multi-branch architecture, and each branch adjusts the receptive field size by reducing the number of channels in the convolution layer, thereby extracting features of different scales in parallel; the polarization fusion module uses a polarization attention module to obtain local features, such as Figure 4As shown in (c) in the figure; the polarization fusion module specifically includes six 1×1 convolutions and a polarization attention module; the six 1×1 convolutions are respectively used as the first convolution layer, the second convolution layer, the third convolution layer, the fourth convolution layer, the fifth convolution layer, and the sixth convolution layer; the output of the first convolution layer is respectively used as the input of the second convolution layer and the third convolution layer; the output of the second convolution layer, the output of the third convolution layer and the output of the first convolution layer are spliced ​​along the channel dimension as the input of the fifth convolution layer; the output of the second convolution layer is used as the input of the fourth convolution layer; the output of the third convolution layer, the output of the fourth convolution layer, and the output of the fifth convolution layer are spliced ​​along the channel dimension as the input of the polarization attention module; the output of the polarization attention module is used as the output of the sixth convolution layer; the output of the sixth convolution layer is fused with the output of the first convolution layer as the output of the polarization fusion module.

[0095] The polarization attention module contains channel attention and spatial attention, which is used to extract channel statistics and spatial context features, such as Figure 4 As shown in (d);

[0096] Channel attention extracts the channel features of the system matrix by obtaining the channel polarization factor:

[0097] (3)

[0098] in, 、 and Represent the sigmoid function, reshaping operation, and SoftMax function respectively. represents convolution;

[0099] The output of channel attention is:

[0100] (4)

[0101] Spatial attention extracts the spatial context features of the system matrix by obtaining the spatial polarization factor:

[0102] (5)

[0103] The output of spatial attention is:

[0104] (6)

[0105] The output of the channel attention is added to the output of the spatial attention to obtain the output of the polarization attention module.

[0106] The global feature fusion Transformer module includes multiple global Transformer layers (the global Transformer layer is constructed based on the sequentially connected strip window attention layer, the first fusion layer, the feedforward network, and the second fusion layer; the input of the strip window attention layer in the first global Transformer layer is the shallow feature; except for the first one, the input of the strip window attention layer in other global Transformer layers is the output of the previous global Transformer layer; the first fusion layer is used to fuse the input and output of the strip window attention layer; the second fusion layer is used to fuse the input and output of the feedforward network), such as Figure 5 As shown in (a) in the figure, the process of connecting multiple global Transformer layers in series to efficiently learn the global dependency features of the system matrix is ​​as follows:

[0107] (7)

[0108] in, Indicates the i Global Transformer layers The extracted features, the features output by the last global Transformer layer are the global features extracted by the global feature fusion Transformer module .

[0109] The strip window attention layer divides the shallow features of the input into two independent blocks along the channel dimension, and uses the vertical stripe window and the horizontal stripe window to capture the features respectively; the self-attention calculation is performed on the features captured by the vertical stripe window and the features captured by the horizontal stripe window respectively, such as Figure 5 As shown in (b) in .

[0110] The vertical stripe window is first calculated using the weight matrix and The linear layers map the input features to query features Sum value characteristics :

[0111] (8)

[0112] in, Represents the features captured by the vertical stripe window.

[0113] The vertical stripe window performs self-attention calculation on the captured features (preferably performing proportional dot product operations on each head in multi-head attention) as follows:

[0114] (9)

[0115] in, Represents the result of self-attention calculation, 、 They represent the query features and value features obtained by mapping the captured features through different weight matrices. represents the scale factor, represents transpose;

[0116] Specifically, the present invention employs a stripe attention mechanism, which, through anisotropic window design, adapts to the anisotropic characteristics of the system matrix and improves the relevance and efficiency of attention calculations. The global Transformer layer applies multi-head computation within each stripe window, further enhancing the model's ability to capture features of different scales and directions.

[0117] The orthogonal features (i.e., the features obtained by the vertical stripe window and the features obtained by the horizontal stripe window) are spliced ​​together along the channel dimension to obtain the output features of the stripe window attention layer (i.e., the stripe window attention mechanism).

[0118] (10)

[0119] in, represents the features obtained by the horizontal stripe window, Represents the output of the strip window attention layer.

[0120] The high-resolution system matrix recovery module is used to fuse shallow features, local features, and global features, and process them through a convolution layer (preferably a 3×3 convolution layer) and a pixel shuffle layer to obtain a high-resolution system matrix, as shown in the following formula:

[0121] (11)

[0122] in, represents the high-resolution system matrix recovery module, represents the high-resolution system matrix, 、 They represent local features and global features respectively.

[0123] A400, combining the super-resolved high-resolution system matrix and its corresponding labels, calculates a loss value using a pre-built loss function, and then updates the parameters of a pre-built fast feature fusion Transformer model;

[0124] In this embodiment, the objective function (i.e., loss function) of the fast feature fusion Transformer model is optimized on a training set containing N pairs of samples:

[0125] (12)

[0126] (13)

[0127] (14)

[0128] in, represents the total loss, Keep the loss for details, is the multi-scale gradient consistency loss, and are weight parameters, Indicates the current index position in the system matrix i Neighborhood, represents other locations in the neighborhood of this location, is the total number of pixels in the system matrix, and Represent the reconstructed high-resolution system moment and the reference high-resolution system matrix respectively. i The pixel value of the position, and Represent the reconstructed high-resolution system moment and the reference high-resolution system matrix respectively. j The pixel value of the position, and Respectively expressed in scale s The reconstructed high-resolution system matrix and the reference high-resolution system matrix under is the gradient operator.

[0129] Step A500: loop A300-A400 until a trained fast feature fusion Transformer model is obtained.

[0130] In this embodiment, the model is trained using the training set to obtain a trained super-resolution model (i.e., the optimal fast feature fusion Transformer model). During the network training process, the Adam back-propagation algorithm is used to perform model optimization training. β1,β2 They are set to 0.9 and 0.999 respectively. The convolution parameters are initialized by the Xavier initializer and the initial learning rate is set to 1e-4.

[0131] In summary, the present invention uses the low-gradient system matrix as input and the high-resolution system matrix as a label to train the neural network, optimize the loss function, calculate the loss function based on the output results and the label system matrix, update the neural network model parameters (including the weight coefficient in the local feature fusion convolutional neural network, the number of attention heads in the strip attention, the proportional coefficient, etc.), calculate the loss function on the validation set, update and save the network model parameters that minimize the loss function, and after the iterative training is completed, save the optimal neural network model.

[0132] 2. High-resolution magnetic particle reconstruction method based on fast feature fusion

[0133] S100, acquiring a system matrix generated using a standard particle sample as a low-resolution system matrix; acquiring a voltage signal of the phantom to be imaged within the imaging field of view of the magnetic particle imaging device;

[0134] S200, inputting the low-resolution system matrix into a trained fast feature fusion Transformer model to obtain a high-resolution system matrix;

[0135] S300 , reconstructing the high-resolution system matrix based on the voltage signal to obtain a high-resolution MPI image.

[0136] A magnetic particle high-resolution reconstruction system based on rapid feature fusion according to a second embodiment of the present invention includes:

[0137] A voltage signal acquisition module is configured to acquire a system matrix generated using a standard particle sample as a low-resolution system matrix; and to acquire a voltage signal of the phantom to be imaged within the imaging field of view of the magnetic particle imaging device;

[0138] a super-resolution module configured to input the low-resolution system matrix into a trained fast feature fusion Transformer model to obtain a high-resolution system matrix;

[0139] In this embodiment, the low-resolution system matrix may be pre-processed by transformation first, and the pre-processed low-resolution system matrix is ​​input into a trained fast feature fusion Transformer model to obtain a high-resolution system matrix.

[0140] The image output module is configured to reconstruct the high-resolution system matrix based on the voltage signal to obtain a high-resolution MPI image.

[0141] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working process and related instructions of the system described above can refer to the corresponding process in the aforementioned method embodiment and will not be repeated here.

[0142] It should be noted that the magnetic particle high-resolution reconstruction system based on rapid feature fusion provided in the above embodiment is merely illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the modules or steps in the embodiments of the present invention can be further decomposed or combined. For example, the modules in the above embodiment can be combined into one module or further divided into multiple sub-modules to complete all or part of the functions described above. The names of the modules and steps involved in the embodiments of the present invention are merely for the purpose of distinguishing the modules or steps and are not to be considered as improper limitations of the present invention.

[0143] An electronic device according to a third embodiment of the present invention comprises: at least one processor; and a memory communicatively connected to at least one of the processors; wherein the memory stores instructions executable by the processor, the instructions being executed by the processor to implement the above-mentioned method for high-resolution magnetic particle reconstruction based on rapid feature fusion.

[0144] A fourth embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to be executed by the computer to implement the above-mentioned method for high-resolution magnetic particle reconstruction based on rapid feature fusion.

[0145] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes and related instructions of the electronic device and computer-readable storage medium described above can refer to the corresponding processes in the aforementioned method examples and will not be repeated here.

[0146] Those skilled in the art should be able to appreciate that the modules and method steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two, and the programs corresponding to the software modules and method steps can be placed in random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs, or any other form of storage medium known in the art. In order to clearly illustrate the interchangeability of electronic hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in electronic hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0147] The terms "first", "second", "third", etc. are used to distinguish similar objects rather than to describe or indicate a particular order or sequence.

[0148] Thus far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art may make equivalent changes or substitutions to the relevant technical features, and the technical solutions after such changes or substitutions will fall within the scope of protection of the present invention.

Claims

1. A method for high-resolution reconstruction of magnetic particles based on rapid feature fusion, characterized in that: The method includes: S100, acquiring a system matrix generated using a standard particle sample as a low-resolution system matrix; acquiring a voltage signal of the phantom to be imaged within the imaging field of view of the magnetic particle imaging device; S200, inputting the low-resolution system matrix into a trained fast feature fusion Transformer model to obtain a high-resolution system matrix; S300, reconstructing the high-resolution system matrix based on the voltage signal to obtain a high-resolution MPI image; The fast feature fusion Transformer model includes a shallow feature extractor, a local feature fusion convolutional neural network, a global feature fusion Transformer module, and a high-resolution system matrix recovery module. The shallow feature extractor is used to extract shallow features of the low-resolution system matrix; The local feature fusion convolutional neural network is used to perform local feature aggregation on the shallow features to obtain local features; The global feature fusion Transformer module is used to perform global feature extraction on the shallow features through continuous global Transformer layers to obtain global features; The high-resolution system matrix recovery module is used to fuse the shallow features, the local features, and the global features, and obtain a high-resolution system matrix through processing through a convolution layer and a pixel shuffle layer.

2. The method for high-resolution magnetic particle reconstruction based on rapid feature fusion according to claim 1, characterized in that: The training method of the fast feature fusion Transformer model is as follows: A100 uses a simulation platform to generate system matrices with different gradients and particle sizes by adjusting the gradient value and the size of the magnetic particles, which serve as high-resolution system matrices. A200, downsampling the high-resolution system matrix to obtain a low-resolution system matrix; A300, inputs the low-resolution system matrix into a pre-built fast feature fusion Transformer model to obtain a high-resolution system matrix after super-resolution; A400, combining the super-resolved high-resolution system matrix and its corresponding labels, calculates a loss value using a pre-built loss function, and then updates the parameters of a pre-built fast feature fusion Transformer model; Step A500: loop A300-A400 until a trained fast feature fusion Transformer model is obtained.

3. The method for high-resolution magnetic particle reconstruction based on rapid feature fusion according to claim 1, characterized in that: The shallow feature extractor consists of convolutional layers of set size.

4. The method for high-resolution magnetic particle reconstruction based on rapid feature fusion according to claim 1, characterized in that: The local feature fusion convolutional neural network includes three local feature aggregation units and 1×1 convolution; Each local feature aggregation unit includes a first local feature aggregation module, a channel aggregation module, and a second local feature aggregation module connected in sequence; The channel aggregation module is constructed based on a global pooling layer, a 1×1 convolution, a RELU activation layer, a 1×1 convolution, a sigmoid layer, a cross product layer, and a fusion layer connected in sequence; the cross product layer is used to multiply the output of the first local feature aggregation module with the output of the sigmoid layer; the fusion layer is used to fuse the output of the cross product layer with the first feature; Except for the first local feature aggregation unit, the first feature input to the fusion layer in other local feature aggregation units is the output of the second local feature aggregation module in the previous local feature aggregation unit; the first feature input to the fusion layer in the first local feature aggregation unit is the output of the first local feature aggregation module in the last local feature aggregation unit; The input of the first local feature aggregation unit is the shallow feature; the input of other local feature aggregation units is the output of the first local feature aggregation unit in the previous local feature aggregation unit; The outputs of the local feature aggregation modules in the first local feature aggregation unit, the second local feature aggregation unit, and the third local feature aggregation unit are spliced ​​along the channel dimension, and after splicing, they are processed by 1×1 convolution as the output of the local feature fusion convolutional neural network; The local feature aggregation module includes multiple polarization fusion modules connected in sequence, each polarization fusion module is connected to a 1×1 convolution, and the input of each 1×1 convolution is the output of all previous polarization fusion modules and the input of the first polarization fusion module. The output of the last 1×1 convolution is fused with the input of the first polarization fusion module to serve as the output of the local feature aggregation module; The polarization fusion module includes six 1×1 convolutions and a polarization attention module; the six 1×1 convolutions serve as the first convolution layer, the second convolution layer, the third convolution layer, the fourth convolution layer, the fifth convolution layer, and the sixth convolution layer respectively; The output of the first convolutional layer serves as the input of the second convolutional layer and the third convolutional layer respectively; the output of the second convolutional layer, the output of the third convolutional layer and the output of the first convolutional layer are spliced ​​along the channel dimension as the input of the fifth convolutional layer; the output of the second convolutional layer serves as the input of the fourth convolutional layer; the output of the third convolutional layer, the output of the fourth convolutional layer and the output of the fifth convolutional layer are spliced ​​along the channel dimension as the input of the polarized attention module; the output of the polarized attention module serves as the output of the sixth convolutional layer; the output of the sixth convolutional layer is fused with the output of the first convolutional layer as the output of the polarized fusion module.

5. The method for high-resolution magnetic particle reconstruction based on rapid feature fusion according to claim 4, characterized in that: The polarized attention module includes a channel attention module and a spatial attention module; the output of the polarized attention module is the concatenation of the output of the channel attention module and the output of the spatial attention module; The output of the channel attention module is: ； ； in, represents the output of the channel attention module, represents the input of the polarized attention module, represents the channel characteristics, 、 and Represent the sigmoid function, reshaping operation, and SoftMax function respectively. Represents convolution.

6. The method for high-resolution magnetic particle reconstruction based on rapid feature fusion according to claim 5, characterized in that: The output of the spatial attention module is: ； ； in, represents the output of the spatial attention module, Represents spatial features, Represents global pooling.

7. The method for high-resolution magnetic particle reconstruction based on rapid feature fusion according to claim 1, characterized in that: The global feature fusion Transformer module includes multiple global Transformer layers; the global Transformer layer is constructed based on the sequentially connected strip window attention layer, the first fusion layer, the feedforward network, and the second fusion layer; The input of the strip window attention layer in the first global Transformer layer is the shallow features; except for the first one, the input of the strip window attention layer in other global Transformer layers is the output of the previous global Transformer layer; The first fusion layer is used to fuse the input and output of the strip window attention layer; the second fusion layer is used to fuse the input and output of the feedforward network.

8. The method for high-resolution magnetic particle reconstruction based on rapid feature fusion according to claim 7, characterized in that: The strip window attention layer divides the input shallow features evenly along the channel dimension and uses vertical stripe windows and horizontal stripe windows to capture features respectively; Self-attention calculation is performed on the features captured by the vertical stripe window and the features captured by the horizontal stripe window respectively, and then they are spliced ​​along the channel dimension as the output of the strip window attention layer.

9. The method for high-resolution magnetic particle reconstruction based on rapid feature fusion according to claim 8, characterized in that: The self-attention calculation is performed on the features captured by the vertical stripe window, and the method is as follows: ； in, Represents the result of self-attention calculation, 、 They represent the query features and value features obtained by mapping the captured features through different weight matrices. represents the scale factor, T Indicates transpose.

10. The method for high-resolution magnetic particle reconstruction based on rapid feature fusion according to claim 2, characterized in that: The loss function of the fast feature fusion Transformer model during training is: ； in, represents the total loss, Keep the loss for details, is the multi-scale gradient consistency loss, and are all weight parameters; ； ； in, Indicates the current index position in the system matrix i Neighborhood, represents other locations in the neighborhood of this location, is the total number of pixels in the system matrix, and Represent the reconstructed high-resolution system moment and the reference high-resolution system matrix respectively. i The pixel value of the position, and Represent the reconstructed high-resolution system moment and the reference high-resolution system matrix respectively. j The pixel value of the position, and Respectively expressed in scale s The reconstructed high-resolution system matrix and the reference high-resolution system matrix under is the gradient operator.

Images