Magnetic particle high-resolution reconstruction method based on rapid feature fusion

Abstract

The invention belongs to the field of magnetic particle imaging, particularly relates to a magnetic particle high-resolution reconstruction method based on fast feature fusion, and aims to solve the problems that an existing magnetic particle reconstruction method is slow in system matrix calibration, low in reconstructed magnetic particle image resolution, limited in signal-to-noise ratio and the like. The method comprises the following steps: collecting a system matrix generated by using a standard particle sample as a low-resolution system matrix; acquiring a voltage signal of the to-be-imaged phantom; inputting the low-resolution system matrix into the trained fast feature fusion Transform model to obtain a high-resolution system matrix; and based on the voltage signal, reconstructing the high-resolution system matrix to obtain a high-resolution MPI image. According to the method, the resolution of the MPI image is improved by carrying out rapid super-division on the system matrix, tedious and time-consuming calibration in system matrix reconstruction is avoided, in addition, resolution and signal-to-noise ratio improvement can be carried out on any system matrix, and a scanning mode related to the system matrix is not needed.

Description

Technical Field

[0001] The present invention belongs to the field of magnetic particle imaging, and particularly relates to a magnetic particle high-resolution reconstruction method based on fast feature fusion. Background Art

[0002] In clinical detection and diagnosis, how to accurately achieve in-vivo high-sensitivity, high-resolution imaging and localization of target lesions has always been a research hotspot and challenging problem internationally. Magnetic particle imaging technology (MPI) is a brand-new imaging method based on tracers, which has advantages such as non-radiation and high sensitivity. At present, magnetic particle imaging equipment shows broad application prospects in pre-clinical research, including directions such as early lesion examination and tumor detection. However, the current resolution of MPI cannot meet the application requirements. In the system matrix method, a high-resolution system matrix needs to collect signals at each voxel position one by one to obtain. This process is complex and time-consuming, and the collected system matrix is extremely sensitive to system parameters. This defect limits the application of MPI in fast imaging and dynamic monitoring.

[0003] Benefiting from the development of big data and computing power, the medical image resolution enhancement method based on deep learning has broad development prospects. By introducing a super-resolution algorithm, a high-resolution system matrix can be restored from a low-resolution system matrix without going through a cumbersome system matrix calibration process, greatly reducing the calibration time.

[0004] Based on this, the present invention proposes a magnetic particle high-resolution reconstruction method based on fast feature fusion. Summary of the Invention

[0005] In order to solve the above problems in the prior art, that is, to solve the problems of slow system matrix calibration, low resolution of the reconstructed magnetic particle image, and limited signal-to-noise ratio in the existing magnetic particle reconstruction method, in the first aspect of the present invention, a magnetic particle high-resolution reconstruction method based on fast feature fusion is proposed. The method includes: S100, collecting the system matrix generated using standard particle samples as the low-resolution system matrix; collecting the voltage signal of the phantom to be imaged within the imaging field of view of the magnetic particle imaging device; S200, inputting the low-resolution system matrix into a trained fast feature fusion Transformer model to obtain a high-resolution system matrix; S300, reconstructing the high-resolution system matrix based on the voltage signal to obtain a high-resolution MPI image; wherein, the fast feature fusion Transformer model includes a shallow feature extractor, a local feature fusion convolutional neural network, a global feature fusion Transformer module, and a high-resolution system matrix recovery module; The shallow feature extractor is used to extract the shallow features of the low-resolution system matrix; The local feature fusion convolutional neural network is used to perform local feature aggregation on the shallow features to obtain local features; The global feature fusion Transformer module is used to perform global feature extraction on the shallow features through consecutive global Transformer layers to obtain global features; The high-resolution system matrix restoration module is used to fuse the shallow features, the local features, and the global features, and through processing by a convolutional layer and a pixel shuffle layer, a high-resolution system matrix is obtained.

[0006] In some preferred embodiments, the training method of the fast feature fusion Transformer model is as follows: A100, using a simulation platform, different gradients and different particle size system matrices are generated by adjusting the gradient value size and the magnetic particle size, and used as the high-resolution system matrix; A200, downsampling the high-resolution system matrix to obtain a low-resolution system matrix; A300, inputting the low-resolution system matrix into a pre-constructed fast feature fusion Transformer model to obtain a super-resolved high-resolution system matrix; A400, combining the super-resolved high-resolution system matrix and its corresponding label, calculating a loss value through a pre-constructed loss function, and further updating the parameters of the pre-constructed fast feature fusion Transformer model; Step A500, loop A300 - A400 until a trained fast feature fusion Transformer model is obtained.

[0007] In some preferred embodiments, the shallow feature extractor is composed of convolutional layers of a set size.

[0008] In some preferred embodiments, the local feature fusion convolutional neural network includes three local feature aggregation units and 1×1 convolution; Each local feature aggregation unit includes a first local feature aggregation module, a channel aggregation module, and a second local feature aggregation module connected in sequence; The channel aggregation module is constructed based on a global pooling layer, a 1×1 convolution, a RELU activation layer, a 1×1 convolution, a sigmoid layer, a multiplication layer, and a fusion layer connected in sequence; the multiplication layer is used to multiply the output of the first local feature aggregation module by the output of the sigmoid layer; the fusion layer is used to fuse the output of the multiplication layer with the first feature; Except for the first local feature aggregation unit, the first feature input to the fusion layer in other local feature aggregation units is the output of the second local feature aggregation module in the previous local feature aggregation unit; the first feature input to the fusion layer in the first local feature aggregation unit is the output of the first local feature aggregation module in the last local feature aggregation unit; The input of the first local feature aggregation unit is the shallow feature; the input of other local feature aggregation units is the output of the first local feature aggregation unit in the previous local feature aggregation unit; The outputs of the local feature aggregation modules in the first local feature aggregation unit, the second local feature aggregation unit, and the third local feature aggregation unit are concatenated along the channel dimension, and after being processed by a 1×1 convolution, they are used as the output of the local feature fusion convolutional neural network; The local feature aggregation module includes a plurality of sequentially connected polarization fusion modules, and a 1×1 convolution is connected after each polarization fusion module. The input of each 1×1 convolution is the output of all the previous polarization fusion modules and the input of the first polarization fusion module. After the output of the last 1×1 convolution is fused with the input of the first polarization fusion module, it is used as the output of the local feature aggregation module; The polarization fusion module includes six 1×1 convolutions and one polarization attention module; the six 1×1 convolutions are respectively used as the first convolutional layer, the second convolutional layer, the third convolutional layer, the fourth convolutional layer, the fifth convolutional layer, and the sixth convolutional layer; The output of the first convolutional layer is respectively used as the input of the second convolutional layer and the third convolutional layer; the output of the second convolutional layer, the output of the third convolutional layer, and the output of the first convolutional layer are concatenated along the channel dimension and used as the input of the fifth convolutional layer; the output of the second convolutional layer is used as the input of the fourth convolutional layer; the output of the third convolutional layer, the output of the fourth convolutional layer, and the output of the fifth convolutional layer are concatenated along the channel dimension and used as the input of the polarization attention module; the output of the polarization attention module is used as the output of the sixth convolutional layer; the output of the sixth convolutional layer is fused with the output of the first convolutional layer and used as the output of the polarization fusion module.

[0009] In some preferred embodiments, the polarization attention module includes a channel attention module and a spatial attention module; the output of the polarization attention module is the concatenation of the output of the channel attention module and the output of the spatial attention module; The output of the channel attention module is: Wherein, represents the output of the channel attention module, represents the input of the polarization attention module, represents the channel features, , and represent the sigmoid function, reshaping operation, and SoftMax function respectively, represents convolution.

[0010] In some preferred embodiments, the output of the spatial attention module is: where, represents the output of the spatial attention module, represents the spatial features, represents global pooling.

[0011] In some preferred embodiments, the global feature fusion Transformer module includes a plurality of global Transformer layers; the global Transformer layer is constructed based on a bar window attention layer, a first fusion layer, a feed-forward network, and a second fusion layer connected in sequence; The input of the bar window attention layer in the first global Transformer layer is the shallow features; except for the first one, the input of the bar window attention layer in other global Transformer layers is the output of the previous global Transformer layer; The first fusion layer is used to fuse the input and output of the bar window attention layer; the second fusion layer is used to fuse the input and output of the feed-forward network.

[0012] In some preferred embodiments, the bar window attention layer evenly divides the input shallow features along the channel dimension, and respectively applies a vertical stripe window and a horizontal stripe window to capture features; Self-attention calculations are respectively performed on the features captured by the vertical stripe window and the features captured by the horizontal stripe window, and then they are concatenated along the channel dimension as the output of the bar window attention layer.

[0013] In some preferred embodiments, the method for performing self-attention calculation on the features captured by the vertical stripe window is: where, represents the result of self-attention calculation, , respectively represent the query feature and value feature obtained by feature mapping of the captured features through different weight matrices, represents the scaling factor, T represents the transpose.

[0014] In some preferred embodiments, for the Fast Feature Fusion Transformer model, its loss function during training is: where, represents the total loss, is the detail preservation loss, is the multi-scale gradient consistency loss, and are both weight parameters; where, represents the neighborhood of the current index position in the system matrix i of, represents other positions in the neighborhood of this position, is the total number of pixels in the system matrix, and respectively represent the pixel values at the position in the reconstructed high-resolution system matrix and the reference high-resolution system matrix i of, and respectively represent the pixel values at the position in the reconstructed high-resolution system matrix and the reference high-resolution system matrix j of, and respectively represent the reconstructed high-resolution system matrix and the reference high-resolution system matrix at scale s of, is the gradient operator.

[0015] Advantages of the present invention: 1) The present invention can, by means of super-resolution technology, innovatively combine CNN and Transformer to learn and extract local and global features of the MPI system matrix, and directly perform super-resolution on the low-resolution system matrix in an end-to-end manner; 2) The present invention can perform super-resolution on low-resolution system matrices of any grid size, prospectively predict high-resolution system matrices, and thus obtain high-resolution MPI images; 3) The present invention can avoid complex and time-consuming system matrix calibration. Based on the existing low-resolution system matrix, it greatly reduces the calibration time and quickly obtains a high-signal-to-noise ratio and high-resolution system matrix; 4) The present invention is applicable to all system matrix reconstruction methods, including magnetic field free point and magnetic field free line scan methods. Description of the Drawings

[0016] Other features, objects, and advantages of the present application will become more apparent from the following detailed description of non-limiting embodiments read in conjunction with the accompanying drawings.

[0017] Figure 1 is a schematic flowchart of a magnetic particle high-resolution reconstruction method based on fast feature fusion according to an embodiment of the present invention; Figure 2 is a schematic diagram of the training process and application process of a fast feature fusion Transformer model according to an embodiment of the present invention; Figure 3 is a schematic structural diagram of a fast feature fusion Transformer model according to an embodiment of the present invention; Figure 4 is a schematic structural diagram of a local feature fusion convolutional neural network of a fast feature fusion Transformer model according to an embodiment of the present invention; Figure 5 is a schematic structural diagram of a global feature fusion Transformer of a fast feature fusion Transformer model according to an embodiment of the present invention. Detailed Embodiments

[0018] To make the objectives, technical solutions, and advantages of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Apparently, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0019] The following further details the present application in conjunction with the drawings and embodiments. It can be understood that the specific embodiments described herein are only for explaining the relevant invention and not for limiting the invention. Additionally, it should be noted that only parts related to the relevant invention are shown in the drawings for ease of description.

[0020] It should be noted that, without conflict, the embodiments in the present application and the features in the embodiments may be combined with each other.

[0021] The magnetic particle high-resolution reconstruction method based on fast feature fusion of the present invention, as Figure 1 shown, includes the following steps: S100, collect the system matrix generated using a standard particle sample as a low-resolution system matrix; collect the voltage signal of the phantom to be imaged within the imaging field of view of the magnetic particle imaging device; S200. Input the low-resolution system matrix into the trained fast feature fusion Transformer model to obtain a high-resolution system matrix; S300. Based on the voltage signal, reconstruct the high-resolution system matrix to obtain a high-resolution MPI image; Among them, the fast feature fusion Transformer model includes a shallow feature extractor, a local feature fusion convolutional neural network, a global feature fusion Transformer module, and a high-resolution system matrix recovery module; The shallow feature extractor is used to extract the shallow features of the low-resolution system matrix; The local feature fusion convolutional neural network is used to perform local feature aggregation on the shallow features to obtain local features; The global feature fusion Transformer module is used to evenly split the shallow features along the channel dimension. After average splitting, perform self-attention calculation and splicing along the channel dimension to obtain global features; The high-resolution system matrix recovery module is used to fuse the shallow features, the local features, and the global features, and after processing through a convolutional layer and a pixel shuffle layer, obtain a high-resolution system matrix.

[0022] To more clearly illustrate the magnetic particle high-resolution reconstruction method based on fast feature fusion of the present invention, the following will, in conjunction with the accompanying drawings, elaborate on each step in an embodiment of the method of the present invention.

[0023] In the following embodiments, first, the training method of the fast feature fusion Transformer model will be described, and then the process of obtaining a high-resolution MPI image through the magnetic particle high-resolution reconstruction method based on fast feature fusion will be elaborated.

[0024] 1. The training method of the fast feature fusion Transformer model is as Figure 2 shown A100. Use a simulation platform to generate system matrices with different gradients and different particle sizes by adjusting the gradient value and the magnetic particle diameter, as the high-resolution system matrix; In this embodiment, by simulating the magnetic particle signal generation process in the real situation, a system matrix reconstruction simulation platform is used to generate a dataset. By adjusting the gradient value and the particle size to generate different system matrices, which are the high-resolution system matrices.

[0025] In addition, before the simulation starts, various parameters of the magnetic particles need to be input, and the parameters include the vacuum permeability, particle diameter, saturation magnetization, magnetic moment, temperature, Boltzmann constant, and magnetic field gradient.

[0026] A200 downsamples the high-resolution system matrix to obtain a low-resolution system matrix; A300 inputs the low-resolution system matrix into a pre-constructed fast feature fusion Transformer model to obtain a super-resolved high-resolution system matrix; In this embodiment, the fast feature fusion Transformer model includes a shallow feature extractor, a local feature fusion convolutional neural network, a global feature fusion Transformer module, and a high-resolution system matrix restoration module, as Figure 3 shown; The shallow feature extractor is used to extract the shallow features of the low-resolution system matrix (the low-resolution system matrix collected during actual application), and the extracted shallow features are F S ; The shallow feature extractor consists of a convolutional layer with a set size, and in the present invention, it is preferably a 3×3 convolutional layer.

[0027] The local feature fusion convolutional neural network, as Figure 4 shown in (a) of, is used to extract the local features in the system matrix. The local feature fusion convolutional neural network adopts a symmetric convolutional neural network architecture, including three layers of networks (i.e., three local feature aggregation units). Each layer of the network includes two local feature aggregation modules and one channel aggregation module; the two local feature aggregation modules are respectively located at the top and bottom (as the first local feature aggregation module and the second local feature aggregation module respectively), and the channel aggregation module is located between the two local feature aggregation modules; the three layers of networks are placed in parallel; The local feature aggregation module is composed of three layers of polarization fusion modules and three layers of 1×1 convolutional layers. Except for the first local feature aggregation unit, the first feature input to the fusion layer in other local feature aggregation units is the output of the second local feature aggregation module in the previous local feature aggregation unit; the first feature input to the fusion layer in the first local feature aggregation unit is the output of the first local feature aggregation module in the last local feature aggregation unit; the input of the first local feature aggregation unit is the shallow feature; the input of other local feature aggregation units is the output of the first local feature aggregation module in the previous local feature aggregation unit; the outputs of the local feature aggregation modules in the first local feature aggregation unit, the second local feature aggregation unit, and the third local feature aggregation unit are concatenated along the channel dimension, and after concatenation, they are processed by 1×1 convolution to be used as the output of the local feature fusion convolutional neural network; that is, the local feature aggregation module fuses the features from different levels in a densely connected manner, as shown in the following formula: (1) where, and Represents the i-th layer local feature aggregation module in the top and bottom branches respectively, and the two modules share parameters. represents the i-th channel aggregation module; the channel aggregation module is constructed based on the global pooling layer, 1×1 convolution, RELU activation layer, 1×1 convolution, sigmoid layer, cross-product layer, and fusion layer connected in sequence; the cross-product layer is used to multiply the output of the first local feature aggregation module with the output of the sigmoid layer; the fusion layer is used to fuse the output of the cross-product layer with the first feature; The outputs of all local feature aggregation modules are concatenated and passed through a 1×1 convolutional layer to obtain the final local features. F LS .

[0028] Local feature aggregation module, such as Figure 4 As shown in (b), it is composed of three layers of polarization fusion modules and three layers of 1×1 convolution layers (specifically: the local feature aggregation module includes multiple polarization fusion modules connected in sequence, each polarization fusion module is connected to a 1×1 convolution, and the input of each 1×1 convolution is the output of all previous polarization fusion modules and the input of the first polarization fusion module, and the output of the last 1×1 convolution is fused with the input of the first polarization fusion module to serve as the output of the local feature aggregation module); the local feature aggregation module fuses features from different levels by dense connection; (2) and represents the input and output of the local feature aggregation module, represents the i-th layer polarization fusion module; The polarization fusion module uses a multi-branch architecture, and each branch adjusts the receptive field size by reducing the convolution layer with the number of channels, thereby extracting features of different scales in parallel; the polarization fusion module uses a polarization attention module to obtain local features, such as Figure 4as shown in (c) of [reference]; the polarization fusion module specifically includes six 1×1 convolutions and one polarization attention module; the six 1×1 convolutions are respectively used as the first convolution layer, the second convolution layer, the third convolution layer, the fourth convolution layer, the fifth convolution layer, and the sixth convolution layer; the output of the first convolution layer is respectively used as the input of the second convolution layer and the third convolution layer; the outputs of the second convolution layer, the third convolution layer, and the output of the first convolution layer are concatenated along the channel dimension and used as the input of the fifth convolution layer; the output of the second convolution layer is used as the input of the fourth convolution layer; the outputs of the third convolution layer, the fourth convolution layer, and the fifth convolution layer are concatenated along the channel dimension and used as the input of the polarization attention module; the output of the polarization attention module is used as the output of the sixth convolution layer; the output of the sixth convolution layer is fused with the output of the first convolution layer and used as the output of the polarization fusion module.

[0029] The polarization attention module includes channel attention and spatial attention, and is used to extract channel statistics and spatial context features, such as Figure 4 shown in (d) of [reference]; The channel attention extracts the channel features of the system matrix by obtaining the channel polarization factor: (3) where , and respectively represent the sigmoid function, the reshaping operation, and the SoftMax function, represents convolution; The output of the channel attention is: (4) The spatial attention extracts the spatial context features of the system matrix by obtaining the spatial polarization factor: (5) The output of the spatial attention is: (6) The output of the channel attention is added to the output of the spatial attention to obtain the output of the polarization attention module.

[0030] The global feature fusion Transformer module includes multiple global Transformer layers (the global Transformer layer is constructed based on a bar window attention layer, a first fusion layer, a feed-forward network, and a second fusion layer connected in sequence; the input of the bar window attention layer in the first global Transformer layer is the shallow feature; except for the first one, the input of the bar window attention layer in other global Transformer layers is the output of the previous global Transformer layer; the first fusion layer is used to fuse the input and output of the bar window attention layer; the second fusion layer is used to fuse the input and output of the feed-forward network), as shown in Figure 5 (a) in (7) Among them, represents the feature extracted by the i th global Transformer layer. The feature output by the last global Transformer layer is the global feature extracted by the global feature fusion Transformer module. .

[0031] The bar window attention layer evenly divides the input shallow feature along the channel dimension into two independent pieces, and respectively applies a vertical stripe window and a horizontal stripe window to capture features; self-attention calculations are respectively performed on the features captured by the vertical stripe window and the features captured by the horizontal stripe window, as shown in Figure 5 (b) in

[0032] The vertical stripe window first uses linear layers with weight matrices of and to map the input features into query features and value features respectively: (8) Among them, represents the feature captured by the vertical stripe window.

[0033] The vertical stripe window performs self-attention calculation on the captured features (preferably, each head in the multi-head attention performs a scaled dot product operation), and its method is: (9) Among them, represents the result of the self-attention calculation, , respectively represent the query feature and the value feature obtained by feature mapping of the captured features through different weight matrices, represents the scale factor, represents the transpose; That is, the present invention adopts a strip attention mechanism. Through an anisotropic window design, it conforms to the anisotropic characteristics of the system matrix, improving the relevance and efficiency of attention calculation. The global Transformer layer applies multi-head calculation within each stripe window, further enhancing the model's ability to capture features of different scales and directions.

[0034] The orthogonal features (i.e., the features obtained from the vertical stripe window and the features obtained from the horizontal stripe window) are concatenated along the channel dimension to obtain the output features of the stripe window attention layer (i.e., the stripe window attention mechanism).

[0035] (10) wherein, represents the features obtained from the horizontal stripe window, represents the output of the bar window attention layer.

[0036] A high-resolution system matrix restoration module is used to fuse shallow features, local features, and global features, and after being processed by a convolutional layer (preferably a 3×3 convolutional layer) and a pixel shuffle layer, a high-resolution system matrix is obtained, as shown in the following formula: (11) wherein, represents the high-resolution system matrix restoration module, represents the high-resolution system matrix, , respectively represent local features and global features.

[0037] A400, combined with the super-resolved high-resolution system matrix and its corresponding label, calculates the loss value through a pre-constructed loss function, and then updates the parameters of the pre-constructed fast feature fusion Transformer model; In this embodiment, the objective function (i.e., the loss function) of the fast feature fusion Transformer model is optimized in the model on a training set containing N pairs of samples: (12) (13) (14) wherein, represents the total loss, is the detail preservation loss, is the multi-scale gradient consistency loss, and are both weight parameters, represents the neighborhood of the current index position in the system matrix i ; represents other positions in the neighborhood of this position, is the total number of pixels in the system matrix, and respectively represent the pixel values at the positions in the reconstructed high-resolution system matrix and the reference high-resolution system matrix i ; and respectively represent the pixel values at the positions in the reconstructed high-resolution system matrix and the reference high-resolution system matrix j ; and respectively represent the reconstructed high-resolution system matrix and the reference high-resolution system matrix at scale s ; is the gradient operator.

[0038] Step A500, loop A300 - A400 until a trained fast feature fusion Transformer model is obtained.

[0039] In this embodiment, the model is trained using the training set to obtain a trained super-resolution model (i.e., the optimal fast feature fusion Transformer model). During the process of training the network, the Adam backpropagation algorithm is used for model optimization training. β1,β2 are respectively set to 0.9 and 0.999. The convolutional parameters are initialized by the Xavier initializer, and the initial learning rate is set to 1e - 4.

[0040] In summary, the present invention uses the low-gradient system matrix as the input, the high-resolution system matrix as the label, trains the neural network, optimizes the loss function, calculates the loss function according to the output result and the label system matrix, updates the neural network model parameters (including the weight coefficients in the local feature fusion convolutional neural network, the number of attention heads in the strip attention, the scale factor, etc.), calculates the loss function on the validation set, updates and saves the network model parameters that minimize the loss function. After the iterative training is completed, the optimal neural network model is saved.

[0041] 2. Magnetic particle high-resolution reconstruction method based on fast feature fusion S100, collect the system matrix generated using the standard particle sample as the low-resolution system matrix; collect the voltage signal of the phantom to be imaged within the imaging field of view of the magnetic particle imaging device; S200. Input the low-resolution system matrix into the trained fast feature fusion Transformer model to obtain a high-resolution system matrix; S300. Based on the voltage signal, reconstruct the high-resolution system matrix to obtain a high-resolution MPI image.

[0042] A magnetic particle high-resolution reconstruction system based on fast feature fusion according to the second embodiment of the present invention includes: A voltage signal acquisition module configured to collect a system matrix generated using a standard particle sample as a low-resolution system matrix; and collect the voltage signal of the phantom to be imaged within the imaging field of view of the magnetic particle imaging device; A super-resolution module configured to input the low-resolution system matrix into the trained fast feature fusion Transformer model to obtain a high-resolution system matrix; In this embodiment, the low-resolution system matrix can also be preprocessed by transformation, and the preprocessed low-resolution system matrix is input into the trained fast feature fusion Transformer model to obtain a high-resolution system matrix.

[0043] An image output module configured to reconstruct the high-resolution system matrix based on the voltage signal to obtain a high-resolution MPI image.

[0044] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working process and related descriptions of the above-described system can refer to the corresponding process in the foregoing method embodiment, and will not be repeated here.

[0045] It should be noted that the magnetic particle high-resolution reconstruction system based on fast feature fusion provided in the above embodiment is only illustrated by the division of the above functional modules. In actual applications, the above functions can be allocated to different functional modules according to needs, that is, the modules or steps in the embodiments of the present invention can be further decomposed or combined. For example, the modules in the above embodiment can be combined into one module, or further split into multiple sub-modules to complete all or part of the functions described above. The names of the modules and steps involved in the embodiments of the present invention are only for distinguishing each module or step, and are not regarded as an improper limitation of the present invention.

[0046] An electronic device according to the third embodiment of the present invention includes: at least one processor; and a memory communicatively connected to at least one of the processors; wherein, the memory stores instructions executable by the processor, and the instructions are used to be executed by the processor to implement the above-mentioned magnetic particle high-resolution reconstruction method based on fast feature fusion.

[0047] A computer-readable storage medium according to a fourth embodiment of the present invention, the computer-readable storage medium stores computer instructions, and the computer instructions are used to be executed by the computer to implement the above-mentioned magnetic particle high-resolution reconstruction method based on fast feature fusion.

[0048] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes and related descriptions of the above-described electronic devices and computer-readable storage media can refer to the corresponding processes in the foregoing method examples, and will not be repeated here.

[0049] Those skilled in the art should be able to realize that the modules and method steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. The programs corresponding to the software modules and method steps can be placed in random access memory (RAM), memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs, or any other form of storage medium known in the art. For the sake of clearly illustrating the interchangeability of electronic hardware and software, the composition and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in the form of electronic hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0050] The terms "first", "second", "third", etc. are used to distinguish similar objects, rather than to describe or represent a specific order or sequence.

[0051] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the protection scope of the present invention.

Claims

1. A high-resolution reconstruction method for magnetic particles based on rapid feature fusion, characterized in that, The method includes: S100, acquiring a system matrix generated using a standard particle sample as a low-resolution system matrix; acquiring voltage signals of a phantom to be imaged within the imaging field of view of a magnetic particle imaging device; S200, inputting the low-resolution system matrix into a trained fast feature fusion Transformer model to obtain a high-resolution system matrix; S300, reconstructing the high-resolution system matrix based on the voltage signals to obtain a high-resolution MPI image; wherein, the fast feature fusion Transformer model includes a shallow feature extractor, a local feature fusion convolutional neural network, a global feature fusion Transformer module, and a high-resolution system matrix restoration module; The shallow feature extractor is used to extract shallow features of the low-resolution system matrix; The local feature fusion convolutional neural network is used to perform local feature aggregation on the shallow features to obtain local features; The global feature fusion Transformer module is used to perform global feature extraction on the shallow features through consecutive global Transformer layers to obtain global features; The high-resolution system matrix restoration module is used to fuse the shallow features, the local features, and the global features, and through processing by a convolutional layer and a pixel shuffle layer, to obtain a high-resolution system matrix.

2. The high-resolution reconstruction method of magnetic particles based on fast feature fusion according to claim 1, wherein The training method of the fast feature fusion Transformer model is as follows: A100, using a simulation platform to generate system matrices with different gradients and different particle sizes by adjusting the gradient value and the magnetic particle size as high-resolution system matrices; A200, downsampling the high-resolution system matrix to obtain a low-resolution system matrix; A300, inputting the low-resolution system matrix into a pre-constructed fast feature fusion Transformer model to obtain a super-resolved high-resolution system matrix; A400, combining the super-resolved high-resolution system matrix and its corresponding label, calculating a loss value through a pre-constructed loss function, and further updating the parameters of the pre-constructed fast feature fusion Transformer model; Step A500, repeating A300 - A400 until a trained fast feature fusion Transformer model is obtained.

3. The magnetic particle high-resolution reconstruction method based on fast feature fusion according to claim 1, characterized in that, The shallow feature extractor is composed of convolutional layers of a set size.

4. The magnetic particle high-resolution reconstruction method based on fast feature fusion according to claim 1, wherein The local feature fusion convolutional neural network includes three local feature aggregation units and 1×1 convolution; Each local feature aggregation unit includes a first local feature aggregation module, a channel aggregation module, and a second local feature aggregation module connected in sequence; The channel aggregation module is constructed based on a global pooling layer, a 1×1 convolution, a RELU activation layer, a 1×1 convolution, a sigmoid layer, a multiplication layer, and a fusion layer connected in sequence; the multiplication layer is used to multiply the output of the first local feature aggregation module by the output of the sigmoid layer; the fusion layer is used to fuse the output of the multiplication layer with the first feature; Except for the first local feature aggregation unit, the first feature input to the fusion layer in other local feature aggregation units is the output of the second local feature aggregation module in the previous local feature aggregation unit; the first feature input to the fusion layer in the first local feature aggregation unit is the output of the first local feature aggregation module in the last local feature aggregation unit; The input of the first local feature aggregation unit is the shallow feature; the input of other local feature aggregation units is the output of the first local feature aggregation unit in the previous local feature aggregation unit; The outputs of the local feature aggregation modules in the first local feature aggregation unit, the second local feature aggregation unit, and the third local feature aggregation unit are concatenated along the channel dimension, and after being processed by a 1×1 convolution, they serve as the output of the local feature fusion convolutional neural network; The local feature aggregation module includes a plurality of sequentially connected polarization fusion modules, and each polarization fusion module is connected to a 1×1 convolution. The input of each 1×1 convolution is the output of all the previous polarization fusion modules and the input of the first polarization fusion module. After the output of the last 1×1 convolution is fused with the input of the first polarization fusion module, it serves as the output of the local feature aggregation module; The polarization fusion module includes six 1×1 convolutions and one polarization attention module; the six 1×1 convolutions are respectively used as the first convolutional layer, the second convolutional layer, the third convolutional layer, the fourth convolutional layer, the fifth convolutional layer, and the sixth convolutional layer; The output of the first convolutional layer is respectively used as the input of the second convolutional layer and the third convolutional layer; the output of the second convolutional layer, the output of the third convolutional layer, and the output of the first convolutional layer are concatenated along the channel dimension and used as the input of the fifth convolutional layer; the output of the second convolutional layer is used as the input of the fourth convolutional layer; the output of the third convolutional layer, the output of the fourth convolutional layer, and the output of the fifth convolutional layer are concatenated along the channel dimension and used as the input of the polarization attention module; the output of the polarization attention module is used as the output of the sixth convolutional layer; the output of the sixth convolutional layer is fused with the output of the first convolutional layer and used as the output of the polarization fusion module.

5. The method for high-resolution reconstruction of magnetic particles based on fast feature fusion according to claim 4, wherein The polarization attention module includes a channel attention module and a spatial attention module; the output of the polarization attention module is the concatenation of the output of the channel attention module and the output of the spatial attention module; The output of the channel attention module is: ； ； Among them, represents the output of the channel attention module, represents the input of the polarization attention module, represents the channel feature, 、 and respectively represent the sigmoid function, reshaping operation, and SoftMax function, represents convolution.

6. The high-resolution reconstruction method of magnetic particles based on fast feature fusion according to claim 5, wherein The output of the spatial attention module is: ； ； Among them, represents the output of the spatial attention module, represents the spatial feature, represents global pooling.

7. The method for high-resolution reconstruction of magnetic particles based on fast feature fusion according to claim 1, characterized in that, The global feature fusion Transformer module includes a plurality of global Transformer layers; the global Transformer layer is constructed based on a sequentially connected strip window attention layer, a first fusion layer, a feed-forward network, and a second fusion layer; The input of the strip window attention layer in the first global Transformer layer is the shallow feature; except for the first one, the input of the strip window attention layer in other global Transformer layers is the output of the previous global Transformer layer; The first fusion layer is used to fuse the input and output of the strip window attention layer; the second fusion layer is used to fuse the input and output of the feed-forward network.

8. The high-resolution reconstruction method of magnetic particles based on fast feature fusion according to claim 7, wherein, The strip window attention layer evenly slices the input shallow features along the channel dimension, and respectively applies vertical stripe windows and horizontal stripe windows to capture features; Self-attention calculations are respectively performed on the features captured by the vertical stripe window and the features captured by the horizontal stripe window, and then they are concatenated along the channel dimension as the output of the strip window attention layer.

9. The magnetic particle high-resolution reconstruction method based on fast feature fusion according to claim 8, characterized in that The method for performing self-attention calculation on the features captured by the vertical stripe window is as follows: ； Among them, represents the self-attention calculation result, , respectively represent the query feature and value feature obtained by feature mapping of the captured features through different weight matrices, represents the scaling factor, T represents the transpose.

10. The method for high-resolution reconstruction of magnetic particles based on fast feature fusion according to claim 2, wherein For the fast feature fusion Transformer model, its loss function during training is: ； Among them, represents the total loss, is the detail preservation loss, is the multi-scale gradient consistency loss, and are both weight parameters; ； ； Among them, represents the neighborhood of the current index position in the system matrix i , represents other positions in the neighborhood of this position is the total number of pixels in the system matrix and represent the pixel values at the positions in the reconstructed high-resolution system matrix and the reference high-resolution system matrix respectively i , and represent the pixel values at the positions in the reconstructed high-resolution system matrix and the reference high-resolution system matrix respectively j , and represent the reconstructed high-resolution system matrix and the reference high-resolution system matrix at scale s respectively is the gradient operator

Images