A method for time-series image reconstruction of magnetic nanoparticles based on dual-domain collaborative processing

Abstract

The present invention belongs to the field of magnetic nanoparticle time-series image reconstruction, and specifically relates to a magnetic nanoparticle time-series image reconstruction method and system based on dual-domain collaborative processing. The method aims to solve the problems of excessive short-term signal noise in existing MPI time-series image reconstruction and the need to balance the global receptive field of time series and computational cost in multi-frame reconstruction. The present invention comprises: adaptive noise suppression in the signal domain by combining multi-order graph convolution with adjacent frames, processing time-series images in the image domain by a U-type Mamba model, calculating a loss function, and iteratively updating the multi-order graph convolution and U-type Mamba parameter values according to the gradient of the loss function. The present invention performs collaborative processing in both the signal and image domains, can specifically denoise short-term signal noise, and can well balance computational cost and global receptive field, thereby achieving more efficient output of dynamic videos that focus on image temporal correlation and present image temporal information.

Description

Technical Field

[0001] The present invention belongs to the technical field of magnetic nanoparticle time-series image reconstruction, and in particular relates to a magnetic nanoparticle time-series image reconstruction method and system based on dual-domain collaborative processing. Background Art

[0002] Magnetic nanoparticle imaging (MPI), as an emerging imaging technology, offers the advantages of high sensitivity and specificity. It plays an important role in vascular and perfusion imaging, neuroimaging, tumor imaging, and cell tracking. The system matrix method is commonly used to reconstruct images from signals received by magnetic nanoparticle imaging devices. This method uses the system matrix as a mapping between the MPI image and the MPI signal, and calculates the MPI image from the received frequency-domain voltage signal. This method can obtain accurate measurement results, but measuring the system matrix requires calibration scanning, which is very time-consuming. Therefore, developing algorithms to achieve fast MPI reconstruction is crucial.

[0003] In recent years, deep learning models have been introduced into the field of MPI. Many studies have applied deep learning techniques to system matrix calibration, MPI signal denoising, image reconstruction, or image post-processing. Representative examples include novel Transformer-based deep learning methods for accelerating MPI calibration, which enable rapid system matrix calibration. Most of these methods focus on optimizing the intermediate steps of traditional MPI reconstruction methods, with few studies exploring the feasibility of implementing end-to-end MPI reconstruction. Furthermore, there is currently no research on MPI time-series image reconstruction methods, which are crucial for analyzing brain function and vascular perfusion.

[0004] Traditional time-series image reconstruction methods only generate images at the corresponding moment from the signal at a certain moment, ignoring the continuity of the time flow in the time series and the connection between the previous and subsequent images. The generated time-series images do not pay attention to the correlation between image time and do not achieve better spatiotemporal consistency. Therefore, for MPI brain functional imaging or vascular imaging problems, it is necessary to construct an algorithm that directly converts the received one-dimensional voltage signal into a time-series imaging of magnetic nanoparticles while maintaining the spatiotemporal consistency of the image, thereby achieving fast and accurate end-to-end MPI time-series image reconstruction.

[0005] Currently, MPI time-series image reconstruction faces two main challenges: excessive noise in short-time signals and the need to balance the global receptive field of the time series and the computational cost when reconstructing multiple frames.

[0006] Graph convolutional networks (GCNs) can capture complex relationships between nodes and the structural features of graphs by performing convolution operations on them. Since MPI signals rarely change between adjacent frames, we construct a graph structure for joint analysis, preserving the original signal while removing noise and achieving better reconstruction. This involves processing the input from the signal domain. U-net, a U-shaped convolutional neural network consisting of an encoder (downsampling path) and a decoder (upsampling path), has been widely demonstrated to play an important role in medical image analysis. However, convolution focuses on local information and cannot capture long-range contextual features. The Transformer architecture can capture consistent information about time series images based on the global receptive field of self-attention. However, the attention mechanism has high computational overhead and grows quadratically with the input size, resulting in a high computational cost for long-sequence data. Mamba, a new selective structural state-space model, uses a simple and effective selection mechanism and hardware-aware algorithms to make model training and inference more efficient. When applied to time series image processing in the image domain, it avoids the locality of convolution and the computational complexity of the Transformer, achieving a good balance between computational cost and global receptive field.

[0007] In summary, it is necessary to propose an MPI signal processing algorithm of multi-order graph convolution and an MPI video processing framework of U-type Mamba, which can be processed collaboratively in both signal and image domains to achieve more efficient output of dynamic videos that focus on image temporal correlation and present image timing information. Summary of the Invention

[0008] In order to solve the above-mentioned problems in the prior art, namely, the problem that the short-term signal noise is too large during the existing MPI time-series image reconstruction and the balance between the time-series global receptive field and the computational cost is required during multi-frame reconstruction, the first aspect of the present invention proposes a magnetic nanoparticle time-series image reconstruction method based on dual-domain collaborative processing, which includes the following steps:

[0009] Obtain a time-domain MPI signal, perform framing and Fourier transform on it to generate F frames of frequency-domain data, select d frequency points, integrate the frequency-domain features of c channels, and construct an input sequence of dimension (b, F, c, d) according to the batch size b;

[0010] Each frame signal of the input sequence is used as a node and the frequency signal as the node feature to construct a graph structure;

[0011] Based on the graph structure, a multi-order graph convolutional network is used to combine adjacent frames to perform signal denoising and output a time series MPI signal. ;

[0012] The timing of the MPI signal After the linear projection layer transformation, the generated dimension becomes The time series image data is fed into the U-type Mamba model to obtain the reconstructed MPI time series image, where N represents the image size:

[0013] The U-type Mamba model divides the time series image data into blocks according to the number of frames: , and linearly expand the channel to c' through the linear projection layer to obtain a transformation feature of dimension (b, B×F, c'), where B refers to the image size and c' represents the number of channels; the transformation feature is compressed to (b, F, 8c', N / 8, N / 8) encoding features through a multi-level bidirectional Mamba spatiotemporal processing block and a downsampling block; the encoding feature generates a decoding feature of dimension (b, F, c', N, N) through a multi-level bidirectional Mamba spatiotemporal processing block and an upsampling block. After each level of downsampling, the current level encoding feature is concatenated and fused with the corresponding level decoding feature. The feature information of the last level of concatenation is reduced in dimension by two-dimensional convolution to generate a dimension of MPI timing image.

[0014] In some preferred embodiments, the magnetic nanoparticle time-series image reconstruction method based on dual-domain collaborative processing further includes optimizing the MPI time-series image:

[0015] Calculate the mean square error (MSE) based on the MPI time series image and the actual image as a first loss value;

[0016] According to the time series MPI signal output by multi-order graph convolution And the L1 norm of the clean signal as the second loss value;

[0017] The first loss value and the second loss value are weightedly fused to obtain a comprehensive loss value; a peak signal-to-noise ratio pSNR is calculated based on the first loss value to evaluate the quality of the MPI time series image reconstruction and obtain a quality evaluation result;

[0018] Evaluate the performance of the multi-order graph convolutional network and the U-shaped Mamba model according to the comprehensive loss value, and obtain an evaluation model performance result;

[0019] Based on the quality assessment results, each parameter is updated by the gradient descent algorithm in each iteration; based on the performance results of the evaluation model, each parameter is updated by minimizing the loss value in each iteration; the parameters include parameters of the multi-order graph convolutional network and parameters of the U-type Mamba model.

[0020] In some preferred embodiments, the c channels are: a real channel and an imaginary channel for separating frequency domain data.

[0021] In some preferred embodiments, the timing MPI signal , and the method to obtain it is:

[0022]

[0023]

[0024] ;

[0025] ;

[0026] Where, Representation node Features, Representation node of The characteristics of the order neighbors, represents the similarity operation, Representation node surrounding Step node, Representation node The degree, Representation node Features, is the activation function, Indicates that the features are spliced. represents the similarity coefficient, For Matrices of the same dimensions, Represents the transpose of a matrix.

[0027] In some preferred embodiments, the timing MPI signal After the linear projection layer transformation, the generated dimension becomes The time series image data is:

[0028] Through the known dimension of M×d system matrix and the time series MPI signal of dimension (b, F, c, d) Multiply to get a matrix of dimension (b, F, c, M), and then arrange the M-dimensional features into an N×N matrix to generate a matrix of dimension Time series image data; where M=N 2 .

[0029] In some preferred embodiments, N=64.

[0030] In some preferred embodiments, the method for iteratively compressing to a coding feature of dimension (b, F, 8c', N / 8, N / 8) is:

[0031] The spatiotemporal features with input dimensions of (b, F, c', N, N) are processed sequentially through a first-level bidirectional Mamba spatiotemporal processing block and a first-level downsampling block, and the output is a coded feature with dimensions of (b, F, 2c', N / 2, N / 2);

[0032] The (b, F, 2c', N / 2, N / 2) coded feature is processed sequentially through a two-level bidirectional Mamba spatiotemporal processing block and a two-level downsampling block, outputting a coded feature of dimension (b, F, 4c', N / 4, N / 4);

[0033] The (b, F, 4c', N / 4, N / 4) coding feature is processed in sequence by a three-level bidirectional Mamba spatiotemporal processing block and a three-level downsampling block, and the output dimension is a coding feature of (b, F, 8c', N / 8, N / 8).

[0034] In some preferred embodiments, the method for iteratively generating decoding features of dimension (b, F, c', N, N) is:

[0035] The encoding feature of (b, F, 8c', N / 8, N / 8) is processed by two bidirectional Mamba spatiotemporal processing blocks, and the output feature is spliced with the encoding feature of (b, F, 8c', N / 8, N / 8). The spliced feature is processed by a first-level upsampling block and a fourth-level bidirectional Mamba spatiotemporal processing block, and the output dimension is a decoding feature of (b, F, 4c', N / 4, N / 4);

[0036] The encoding features of the dimension (b, F, 4c', N / 4, N / 4) and the decoding features of the dimension (b, F, 4c', N / 4, N / 4) are concatenated, and after concatenation, they are processed in sequence by a two-level upsampling block and a five-level bidirectional Mamba spatiotemporal processing block to output decoding features of the dimension (b, F, 2c', N / 2, N / 2);

[0037] The encoding features with the dimension of (b, F, 2c', N / 2, N / 2) and the decoding features with the dimension of (b, F, 2c', N / 2, N / 2) are spliced, and after splicing, they are processed by a three-level upsampling block and a six-level bidirectional Mamba spatiotemporal processing block, and the decoding features with the dimension of (b, F, c', N, N) are output.

[0038] In some preferred embodiments, the feature information of the last level of splicing fusion is reduced to a dimension of The MPI timing image is:

[0039] Reduce c' to 1 and obtain the dimension MPI timing image.

[0040] The second invention of the present invention proposes a magnetic nanoparticle time-series image reconstruction system based on dual-domain collaborative processing. Based on the magnetic nanoparticle time-series image reconstruction method based on dual-domain collaborative processing, the system includes a signal domain signal processing module and an image domain time-series image processing module. The signal domain signal processing module is configured to: obtain a time domain MPI signal, frame the time domain MPI signal, perform Fourier transform on the time domain MPI signal to generate F frames of frequency domain data, screen d frequency points, integrate the frequency domain features of c channels, and construct an input sequence with a dimension of (b, F, c, d) according to a batch size b;

[0041] Each frame signal of the input sequence is used as a node and the frequency signal as the node feature to construct a graph structure;

[0042] Based on the graph structure, a multi-order graph convolutional network is used to combine adjacent frames to perform signal denoising and output a time series MPI signal. ;

[0043] The timing of the MPI signal After the linear projection layer transformation, the generated dimension becomes The time series image data is fed into the U-type Mamba model to obtain the reconstructed MPI time series image, where N represents the image size:

[0044] The image domain time series image processing module is configured as follows: the U-shaped Mamba model divides the time series image data into blocks according to the number of frames. , and linearly expand the channel to c' through the linear projection layer to obtain a transformation feature of dimension (b, B×F,c'), where B refers to the image size and c' represents the number of channels; the transformation feature is compressed to (b,F,8c',N / 8,N / 8) encoding features through a multi-level bidirectional Mamba spatiotemporal processing block and a downsampling block; the encoding feature generates a decoding feature of dimension (b,F,c',N,N) through a multi-level bidirectional Mamba spatiotemporal processing block and an upsampling block. After each level of downsampling, the current level encoding feature is concatenated and fused with the corresponding level decoding feature. The feature information of the last level of concatenation is reduced in dimension by two-dimensional convolution to generate a dimension of MPI timing image.

[0045] Beneficial effects of the present invention:

[0046] 1) Receive time-domain MPI signals and adaptively suppress them in the signal domain using multi-order graph convolution combined with adjacent frames. This allows for targeted denoising of short-duration signal noise. In the image domain, the U-shaped Mamba model is used to process time-series images, achieving a good balance between computational cost and global receptive field. This collaborative processing in both the signal and image domains enables more efficient output of dynamic videos that prioritize temporal correlations and present temporal information.

[0047] 2) In the training of multi-order graph convolution for MPI signal denoising and in the training of U-shaped Mamba for MPI video processing, the loss functions are calculated respectively. According to the gradient of the loss function, the parameter values are iteratively updated in the direction that makes the function value decrease the fastest, thereby improving the efficiency and quality of generating MPI time series images. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Other features, objects and advantages of the present application will become more apparent upon reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0049] Figure 1 It is a data flow diagram of the present invention;

[0050] Figure 2 It is a multi-order graph convolutional network graph of the present invention;

[0051] Figure 3 This is a structural diagram of the U-shaped Mamba of the present invention;

[0052] Figure 4 It is a bidirectional Mamba block structure diagram of the present invention;

[0053] Figure 5 This is a structural diagram of the bidirectional Mamba spatiotemporal processing block of the present invention. DETAILED DESCRIPTION

[0054] The present application will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are intended only to illustrate the relevant invention and are not intended to limit the invention. It should also be noted that, for ease of description, only portions relevant to the relevant invention are shown in the accompanying drawings.

[0055] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0056] In order to more clearly illustrate the magnetic nanoparticle time-series image reconstruction method based on dual-domain collaborative processing of the present invention, the following is combined with Figures 1 to 5 Each step in the embodiment of the present invention is described in detail.

[0057] The first embodiment of the present invention proposes a magnetic nanoparticle time-series image reconstruction method based on dual-domain collaborative processing, see Figure 1 , the method comprises the following steps:

[0058] Step 1: Obtain a time-domain MPI signal, frame it, and perform Fourier transform on it to generate F frames of frequency-domain data, also known as the frequency-domain MPI signal. D frequency points are selected, and the frequency-domain features of c channels are integrated. An input sequence ‌ of dimension (b, F, c, d) is constructed according to a batch size b, denoted as ‌x.

[0059] In this embodiment, the specific process of acquiring the time-domain MPI signal is as follows: in the MPI system, the selected field is a uniform gradient field that can construct a region with a magnetic field intensity close to zero (Field Free Region, FFR); the induced voltage signal of the receiving coil is referred to as the time-domain MPI signal, which is the time-domain signal generated by the change in magnetic susceptibility of the object to be measured (the object to be measured containing magnetic particles, such as blood vessels or brain function injected with magnetic particles) in the FFR and its vicinity under the time-varying oscillating driving field. The time-domain MPI voltage signal is framed (sliding window) to generate F frames of data, and Fourier transform is performed to generate F frames of frequency domain data, i.e., frequency domain MPI signals. d frequency points are selected, the frequency domain features of c channels are integrated, and an input sequence x of dimension (b, F, c, d) is constructed according to a batch size b. The c channels include virtual multidimensional channels generated by time-frequency decomposition of a single receiving coil. Here, the number of virtual multidimensional channels is 2, which are the real and imaginary parts of the complex number (frequency domain MPI signal).

[0060] Step 2, see Figure 2 , using each frame signal of the input sequence as a node and the frequency signal as the node feature, a graph structure is constructed based on the K-nearest principle;

[0061] Based on the graph structure, a multi-order graph convolutional network is used to combine adjacent frames to perform signal denoising and output a time series MPI signal. ;

[0062] In this embodiment, the connection range between nodes in the K-neighbor principle is 5. For two frames of signal, if the adjacent range is within 5 frames, it is considered that there is an edge;

[0063] The general graph convolutional network only considers the relationship between adjacent nodes. In this paper, a multi-order graph convolution structure is constructed with an order of 3, that is, the connection between nodes with an order of K=3 and within 3 is considered;

[0064] The timing of the MPI signal , and the method to obtain it is:

[0065] ;

[0066] ;

[0067] ;

[0068] ;

[0069] Where, Representation node Features, Representation node of The characteristics of the order neighbors, represents the similarity operation, Representation node surrounding Step node, Representation node The degree, Representation node Features, is the activation function, Indicates that the features are spliced. represents the similarity coefficient, For Matrices of the same dimensions, Represents the transpose of a matrix;

[0070] Step 3, the timing MPI signal After the linear projection layer, the generated dimension becomes The time series image data enters the U-shaped Mamba model to obtain the reconstructed MPI time series image, where N represents the image size;

[0071] In this embodiment, the linear projection layer is converted into: through the known dimension of M×d system matrix and the dimension of (b, F, c, d) video frame sequence Multiply to get a matrix of dimension (b, F, c, M), and then arrange the M-dimensional features into an N×N matrix to generate a matrix of dimension Time series image data, where M=N 2 ;N=64;

[0072] Step 4: The state space model (SSM) is a model used to describe the state representation of the sequence at each time step and predict its next state based on the input. The core equations of the SSM include the state equation and the output equation:

[0073] ;

[0074] in, Represents any given time potential state, Indicates the updated status. Indicates input, Indicates output, These three matrices are parameters, Matrix affects the previous state , Matrix influence input , Matrices transform states into outputs, which can be learned;

[0075] However, in SSM, after learning, entering the inference stage, the matrix If the model is fixed, it will not be possible to perform targeted reasoning on different inputs.

[0076] Mamba applies a selection mechanism that allows the model to selectively process information, filtering out information irrelevant to the problem while retaining relevant information for a long time. The Mamba architecture combines the SSM and MLP blocks of the Transformer into a single block. The parameters of the SSM are parameterized as a function of the input, enabling the model to adaptively adjust its behavior based on the input (Selective SSM).

[0077] Based on the above technical status, a U-shaped Mamba model was constructed, see Figure 3 The U-type Mamba model divides the temporal image data into blocks according to the number of frames (specifically through block embedding: Patch Embedding is , b refers to the batch size, B refers to the image size, the blocks are divided by the number of frames, the block size is the image size, and the number of blocks is the number of frames F, which is expressed here as , in order to make it an input form that Mamba can receive, and linearly expand the channel to c' (c' represents the number of channels, specifically through linear projection Linear projection linear expansion), and obtain the transformation feature of dimension; the transformation feature is compressed to (b, F, 8c', N / 8, N / 8) encoding feature through multi-level bidirectional Mamba spatiotemporal processing block and downsampling block; the encoding feature is generated by multi-level bidirectional Mamba spatiotemporal processing block and upsampling block to generate decoding feature of dimension (b, F, c', N, N). After each level of downsampling, the current level encoding feature is spliced and fused with the corresponding level decoding feature. The feature information of the last level of splicing and fusion is reduced in dimension by two-dimensional convolution (Conv2d) to generate a dimension of MPI timing images;

[0078] In this embodiment, the bidirectional Mamba spatiotemporal processing block models the time series and generates dimensions of Spatiotemporal features; the spatiotemporal features are compressed to a dimension of The encoding features of the image are used to complete one iteration; the first input is the spatiotemporal features, and the subsequent input is the output of the previous iteration. This is repeated three times in sequence, and the iterative compression is performed to a dimension of (b, F, 8c', N / 8, N / 8) encoding features; the encoding features of the dimension (b, F, 8c', N / 8, N / 8) are processed by the bidirectional Mamba spatiotemporal processing block and the upsampling block (Patch Expanding), N is doubled, and c' is halved to generate a decoding feature of the dimension (b, F, 4c', N / 4, N / 4), and one iteration is completed; the first input is the encoding feature of the dimension (b, F, 8c', N / 8, N / 8), and the subsequent input is the output of the previous iteration. This is repeated three times in sequence, and the decoding feature of the dimension (b, F, c', N, N) is generated. After each level of downsampling, the encoding features of the current level are concatenated and fused with the decoding features of the corresponding level. The feature information of the last level of concatenation is reduced in dimension by two-dimensional convolution (Conv2d) to generate a dimension of The MPI time series image, the two-dimensional convolution dimension reduction is to reduce c' to 1, and obtain the dimension MPI timing images;

[0079] The method for iterative compression to the encoding feature of dimension (b,F,8c',N / 8,N / 8) is:

[0080] The spatiotemporal features with input dimensions of (b, F, c', N, N) are processed sequentially through a first-level bidirectional Mamba spatiotemporal processing block and a first-level downsampling block, and the output dimension is (b, F, 2c', N / 2, N / 2) encoded features.

[0081] The (b, F, 2c', N / 2, N / 2) coded feature is processed in sequence by a second-level bidirectional Mamba spatiotemporal processing block and a second-level downsampling block, and the output dimension is a coded feature of (b, F, 4c', N / 4, N / 4);

[0082] The (b, F, 4c', N / 4, N / 4) coded feature is processed sequentially through a three-level bidirectional Mamba spatiotemporal processing block and a three-level downsampling block, and the output dimension is a coded feature of (b, F, 8c', N / 8, N / 8);

[0083] The method for iteratively generating decoding features of dimension (b, F, c', N, N) is:

[0084] The encoding feature of (b, F, 8c', N / 8, N / 8) is processed by two bidirectional Mamba spatiotemporal processing blocks, and the output feature is spliced with the encoding feature of (b, F, 8c', N / 8, N / 8). The spliced feature is processed by a first-level upsampling block and a fourth-level bidirectional Mamba spatiotemporal processing block, and the output dimension is a decoding feature of (b, F, 4c', N / 4, N / 4);

[0085] The encoding features of the dimension (b, F, 4c', N / 4, N / 4) and the decoding features of the dimension (b, F, 4c', N / 4, N / 4) are concatenated, and after concatenation, they are processed in sequence by a two-level upsampling block and a five-level bidirectional Mamba spatiotemporal processing block to output decoding features of the dimension (b, F, 2c', N / 2, N / 2);

[0086] The encoding features of the dimension (b, F, 2c', N / 2, N / 2) and the decoding features of the dimension (b, F, 2c', N / 2, N / 2) are concatenated, and after concatenation, a three-level upsampling block and a six-level bidirectional Mamba spatiotemporal processing block are processed to output decoding features of the dimension (b, F, c', N, N);

[0087] Among them, see Figure 4 、 5 The specific working process of the bidirectional Mamba spatiotemporal processing block is as follows:

[0088] First, the input block sequence is rearranged into a dimensional shape that conforms to the Mamba module (type) can accept. , normalized through a normalization layer;

[0089] The normalized sequence is then linearly projected into two vectors x and q, where x represents the input feature for state-space model (SSM) processing. It is then processed h forward and h backward to extract features from two directions, where the processing is convolution and SSM in sequence.

[0090] q is used to control the weight of the forward and backward outputs. The results of the forward and backward processing are weighted and summed by q to obtain the bidirectional output y;

[0091] y is mapped to the original feature dimension through a linear projection layer, and is residually connected with the input sequence and added to obtain the output of the current bidirectional Mamba block;

[0092] Before the bidirectional Mamba block outputs, it undergoes “dimensional rearrangement” to convert the output sequence into a shape that can be accepted by downsampling / upsampling, for example ;

[0093] The above is the process of bidirectional Mamba block processing. The bidirectional processing design enables the model to extract features from both forward and backward directions simultaneously, thereby more comprehensively capturing the global context information in the image;

[0094] The output sequence processed by the bidirectional Mamba block is temporally encoded, that is, the temporal frame dimension is highlighted. It is then mapped to a specific dimension through a linear projection layer and input into the self-attention block for processing. The self-attention block is based on the self-attention mechanism, which is an attention mechanism that associates different positions of a single sequence to calculate the representation of the same sequence. When processing time-series MPI data, the self-attention block dynamically adjusts the degree of attention paid to each video frame to capture the complex dependencies within the frame. The working process of the self-attention block is as follows:

[0095] First, a query vector (Q), a key vector (K), and a value vector (V) are calculated for each element in the input sequence;

[0096] For each pair of elements, calculate the attention score , and then through Activation function derives attention weight , the specific calculation formula is:

[0097] Indicates the elements of the query vector, Indicates the a key vector of elements, is the dimension of the key vector;

[0098]

[0099] in, The standard expression of the activation function is, , calculate each pair of attention scores Value Regards Substitute the standard expression of the softmax activation function and we get The query vector of elements, The attention weights of the key vector of elements , that is, the vector is converted into a probability distribution through the softmax function, the weight value of each pair of elements is between 0 and 1 and the sum of the weight values of all elements is 1;

[0100] Finally, the value vector of each element The corresponding attention weight After multiplication and summing, we get The output of the elements after the self-attention mechanism :

[0101]

[0102] In the formula, n represents the total number of elements;

[0103] After processing the sequence self-attention block, the extraction of temporal features is enhanced, and then the linear layer is used to restore the dimension of the data before inputting the self-attention block;

[0104] The above is the complete process of the bidirectional Mamba spatiotemporal processing block. The bidirectional Mamba block enhances features spatially, and the self-attention block enhances features temporally. After one bidirectional Mamba spatiotemporal processing block, the time series image is enhanced.

[0105] In this embodiment, the magnetic nanoparticle time-series image reconstruction method based on dual-domain collaborative processing further includes optimizing the MPI time-series image:

[0106] Calculate the mean square error MSE based on the MPI time series image and the actual image as the first loss value; and calculate the time series MPI signal output by the multi-order graph convolution. And the L1 norm of the clean signal as the second loss value;

[0107] The first loss value and the second loss value are weightedly fused to obtain a comprehensive loss value; the peak signal-to-noise ratio pSNR is calculated based on the first loss, the quality of the MPI time series image reconstruction is evaluated, and a quality assessment result is obtained; the performance of the multi-order graph convolutional network and the U-type Mamba model is evaluated based on the comprehensive loss value to obtain an evaluation model performance result;

[0108] Based on the quality assessment results, each parameter is updated by the gradient descent algorithm in each iteration; based on the performance results of the evaluation model, each parameter is updated by minimizing the loss value in each iteration; the parameters include parameters of the multi-order graph convolutional network and the parameters of the U-shaped Mamba model, reducing the prediction error.

[0109] Although the various steps in the above embodiment are described in the above-mentioned order, those skilled in the art will understand that in order to achieve the effect of this embodiment, different steps do not have to be executed in such an order. They can be executed simultaneously (in parallel) or in a reverse order. These simple changes are within the scope of protection of the present invention.

[0110] A second embodiment of the present invention is a magnetic nanoparticle time-series image reconstruction system based on dual-domain collaborative processing. Based on the magnetic nanoparticle time-series image reconstruction method based on dual-domain collaborative processing, the system includes a signal domain signal processing module and an image domain time-series image processing module. The signal domain signal processing module is configured to: obtain a time domain MPI signal, frame the time domain MPI signal, perform Fourier transform on the time domain MPI signal to generate F frames of frequency domain data, screen d frequency points, integrate the frequency domain features of c channels, and construct an input sequence of dimension (b, F, c, d) according to a batch size b;

[0111] Each frame signal of the input sequence is used as a node and the frequency signal as the node feature to construct a graph structure;

[0112] Based on the graph structure, a multi-order graph convolutional network is used to combine adjacent frames to perform signal denoising and output a time series MPI signal. ;

[0113] The timing of the MPI signal After the linear projection layer transformation, the generated dimension becomes The time series image data is fed into the U-type Mamba model to obtain the reconstructed MPI time series image, where N represents the image size:

[0114] The image domain time series image processing module is configured as follows: the U-shaped Mamba model divides the time series image data into blocks according to the number of frames. , and linearly expand the channel to c' through the linear projection layer to obtain a transformation feature of dimension (b, B×F,c'), where B refers to the image size and c' represents the number of channels; the transformation feature is compressed to (b,F,8c',N / 8,N / 8) encoding features through a multi-level bidirectional Mamba spatiotemporal processing block and a downsampling block; the encoding feature generates a decoding feature of dimension (b,F,c',N,N) through a multi-level bidirectional Mamba spatiotemporal processing block and an upsampling block. After each level of downsampling, the current level encoding feature is concatenated and fused with the corresponding level decoding feature. The feature information of the last level of concatenation is reduced in dimension by two-dimensional convolution to generate a dimension of MPI timing image.

[0115] It should be noted that the above embodiment provides a magnetic nanoparticle time-series image reconstruction system based on dual-domain collaborative processing, which is illustrated only by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the modules or steps in the embodiments of the present invention can be further decomposed or combined. For example, the modules in the above embodiments can be combined into one module or further divided into multiple sub-modules to complete all or part of the functions described above. The names of the modules and steps involved in the embodiments of the present invention are merely for distinguishing the modules or steps and are not to be regarded as improper limitations of the present invention.

[0116] Thus far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art may make equivalent changes or substitutions to the relevant technical features, and the technical solutions after such changes or substitutions will fall within the scope of protection of the present invention.

[0117] Those skilled in the art should be able to appreciate that the modules and method steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two, and the programs corresponding to the software modules and method steps can be placed in random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs, or any other form of storage medium known in the art. In order to clearly illustrate the interchangeability of electronic hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in electronic hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0118] The terms "first", "second", etc. are used to distinguish similar objects, rather than to describe or indicate a particular order or sequence.

[0119] The term "comprise" or any other similar term is intended to cover non-exclusive inclusion such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed or inherent to such process, method, article, or apparatus.

[0120] Thus far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art may make equivalent changes or substitutions to the relevant technical features, and the technical solutions after such changes or substitutions will fall within the scope of protection of the present invention.

Claims

1. A magnetic nanoparticle time-series image reconstruction method based on dual-domain collaborative processing, characterized in that: The method comprises the following steps: Acquire a time-domain MPI signal, perform framing and Fourier transform on it to generate F frames of frequency-domain data, select d frequency points, integrate the frequency-domain features of c channels, and construct an input sequence of dimensions (b, F, c, d) according to a batch size b; the c channels include virtual multidimensional channels generated by time-frequency decomposition of a single receiving coil; Each frame signal of the input sequence is used as a node and the frequency signal as the node feature to construct a graph structure; Based on the graph structure, a multi-order graph convolutional network is used to combine adjacent frames to perform signal denoising and output a time series MPI signal. ; The timing of the MPI signal After the linear projection layer transformation, the generated dimension becomes The time series image data is fed into the U-type Mamba model to obtain the reconstructed MPI time series image, where N represents the image size: The U-type Mamba model divides the time series image data into blocks according to the number of frames: , and linearly expand the channel to c' through the linear projection layer to obtain a transformation feature of dimension (b, B×F, c'), where B refers to the image size and c' represents the number of channels; the transformation feature is compressed to (b, F, 8c', N / 8, N / 8) encoding features through a multi-level bidirectional Mamba spatiotemporal processing block and a downsampling block; the encoding feature generates a decoding feature of dimension (b, F, c', N, N) through a multi-level bidirectional Mamba spatiotemporal processing block and an upsampling block. After each level of downsampling, the current level encoding feature is concatenated and fused with the corresponding level decoding feature. The feature information of the last level of concatenation is reduced in dimension by two-dimensional convolution to generate a dimension of MPI timing image.

2. The method for reconstructing magnetic nanoparticle time-series images based on dual-domain collaborative processing according to claim 1, characterized in that: The method for reconstructing magnetic nanoparticle time-series images based on dual-domain collaborative processing further includes optimizing the MPI time-series images: Calculate the mean square error (MSE) based on the MPI time series image and the actual image as a first loss value; According to the time series MPI signal output by multi-order graph convolution And the L1 norm of the clean signal as the second loss value; The first loss value and the second loss value are weightedly fused to obtain a comprehensive loss value; a peak signal-to-noise ratio pSNR is calculated based on the first loss value to evaluate the quality of the MPI time series image reconstruction and obtain a quality evaluation result; Evaluate the performance of the multi-order graph convolutional network and the U-shaped Mamba model according to the comprehensive loss value, and obtain an evaluation model performance result; Based on the quality assessment results, each parameter is updated by a gradient descent algorithm in each iteration; Based on the performance results of the evaluation model, each parameter is updated by minimizing the loss value in each iteration; the parameters include parameters of the multi-order graph convolutional network and parameters of the U-shaped Mamba model.

3. The method for reconstructing magnetic nanoparticle time-series images based on dual-domain collaborative processing according to claim 1, characterized in that: The c channels are: a real channel and an imaginary channel for separating frequency domain data.

4. The method for reconstructing magnetic nanoparticle time-series images based on dual-domain collaborative processing according to claim 1, characterized in that: The timing of the MPI signal , and the method to obtain it is: ； ； ； ； Where, Representation node Features, Representation node of The characteristics of the order neighbors, represents the similarity operation, Representation node surrounding Step node, Representation node The degree, Representation node Features, is the activation function, Indicates that the features are spliced. represents the similarity coefficient, For Matrices of the same dimensions, Represents the transpose of a matrix.

5. The method for reconstructing magnetic nanoparticle time-series images based on dual-domain collaborative processing according to claim 1, characterized in that: The timing of the MPI signal After the linear projection layer transformation, the generated dimension becomes The time series image data is: Through the known dimension of M×d system matrix and the time series MPI signal of dimension (b, F, c, d) Multiply to get a matrix of dimension (b, F, c, M), and then arrange the M-dimensional features into an N×N matrix to generate a matrix of dimension Time series image data; where M=N 2 .

6. The method for reconstructing magnetic nanoparticle time-series images based on dual-domain collaborative processing according to claim 5, characterized in that: Said N=64.

7. The method for reconstructing magnetic nanoparticle time-series images based on dual-domain collaborative processing according to claim 1, characterized in that: The method for iterative compression to the encoding feature of dimension (b,F,8c',N / 8,N / 8) is: The spatiotemporal features with input dimensions of (b, F, c', N, N) are processed sequentially through a first-level bidirectional Mamba spatiotemporal processing block and a first-level downsampling block, and the output is a coded feature with dimensions of (b, F, 2c', N / 2, N / 2); The (b, F, 2c', N / 2, N / 2) coded feature is processed sequentially through a two-level bidirectional Mamba spatiotemporal processing block and a two-level downsampling block, outputting a coded feature of dimension (b, F, 4c', N / 4, N / 4); The (b, F, 4c', N / 4, N / 4) coding feature is processed in sequence by a three-level bidirectional Mamba spatiotemporal processing block and a three-level downsampling block, and the output dimension is a coding feature of (b, F, 8c', N / 8, N / 8).

8. The method for reconstructing magnetic nanoparticle time-series images based on dual-domain collaborative processing according to claim 7, characterized in that: The method for iteratively generating decoding features of dimension (b, F, c', N, N) is: The encoding feature of (b, F, 8c', N / 8, N / 8) is processed by two bidirectional Mamba spatiotemporal processing blocks, and the output feature is spliced with the encoding feature of (b, F, 8c', N / 8, N / 8). The spliced feature is processed by a first-level upsampling block and a fourth-level bidirectional Mamba spatiotemporal processing block, and the output dimension is a decoding feature of (b, F, 4c', N / 4, N / 4); The encoding features of the dimension (b, F, 4c', N / 4, N / 4) and the decoding features of the dimension (b, F, 4c', N / 4, N / 4) are concatenated, and after concatenation, they are processed in sequence by a two-level upsampling block and a five-level bidirectional Mamba spatiotemporal processing block to output decoding features of the dimension (b, F, 2c', N / 2, N / 2); The encoding features with the dimension of (b, F, 2c', N / 2, N / 2) and the decoding features with the dimension of (b, F, 2c', N / 2, N / 2) are spliced, and after splicing, they are processed by a three-level upsampling block and a six-level bidirectional Mamba spatiotemporal processing block, and the decoding features with the dimension of (b, F, c', N, N) are output.

9. The method for reconstructing magnetic nanoparticle time-series images based on dual-domain collaborative processing according to claim 1, characterized in that: The feature information of the last level of splicing fusion is reduced to a dimension by two-dimensional convolution. The MPI timing image is: Reduce c' to 1 and obtain the dimension MPI timing image.

10. A magnetic nanoparticle time-series image reconstruction system based on dual-domain collaborative processing, based on the magnetic nanoparticle time-series image reconstruction method based on dual-domain collaborative processing according to any one of claims 1 to 9, the system comprising a signal domain signal processing module and an image domain time-series image processing module, the signal domain signal processing module being configured to: acquire a time-domain MPI signal, frame the time-domain MPI signal, perform Fourier transform on the signal, generate F frames of frequency-domain data, select d frequency points, integrate frequency-domain features of c channels, and construct an input sequence of dimensions (b, F, c, d) according to a batch size b; Each frame signal of the input sequence is used as a node and the frequency signal as the node feature to construct a graph structure; Based on the graph structure, a multi-order graph convolutional network is used to combine adjacent frames to perform signal denoising and output a time series MPI signal. ; The timing of the MPI signal After the linear projection layer transformation, the generated dimension becomes The time series image data enters the U-type Mamba model to obtain the reconstructed MPI time series image, where N represents the image size: The image domain time series image processing module is configured as follows: the U-shaped Mamba model divides the time series image data into blocks according to the number of frames. , and linearly expand the channel to c' through the linear projection layer to obtain a transformation feature of dimension (b, B×F,c'), where B refers to the image size and c' represents the number of channels; the transformation feature is compressed to (b,F,8c',N / 8,N / 8) encoding features through a multi-level bidirectional Mamba spatiotemporal processing block and a downsampling block; the encoding feature generates a decoding feature of dimension (b,F,c',N,N) through a multi-level bidirectional Mamba spatiotemporal processing block and an upsampling block. After each level of downsampling, the current level encoding feature is concatenated and fused with the corresponding level decoding feature. The feature information of the last level of concatenation is reduced in dimension by two-dimensional convolution to generate a dimension of MPI timing image.

Images