Magnetic nanoparticle time sequence image reconstruction method based on double-domain cooperative processing

Abstract

The invention belongs to the field of magnetic nanoparticle time sequence image reconstruction, particularly relates to a magnetic nanoparticle time sequence image reconstruction method and system based on double-domain cooperative processing, and aims to solve the problems that signal noise is too large in short time during existing MPI time sequence image reconstruction, and a time sequence global receptive field and calculation cost balance are needed during multi-frame reconstruction. The method comprises the steps of carrying out adaptive noise suppression in a signal domain through multi-order graph convolution in combination with an adjacent frame, processing a time sequence image in an image domain through a U-shaped Mama model, calculating a loss function, and iteratively updating multi-order graph convolution and U-shaped Mama parameter values according to the gradient of the loss function. According to the method, the signal domain and the image domain are cooperatively processed, short-time signal noise can be specifically denoised, the calculation cost and the global receptive field can be well balanced, and a dynamic video which pays attention to image time correlation and presents image time sequence information can be more efficiently output.

Description

Technical Field

[0001] The present invention belongs to the technical field of magnetic nanoparticle time-series image reconstruction, and in particular relates to a magnetic nanoparticle time-series image reconstruction method and system based on dual-domain collaborative processing. Background Art

[0002] Magnetic Particle Imaging (MPI) is an emerging imaging technology with the advantages of high sensitivity and high specificity. It plays an important role in vascular and perfusion imaging, neural imaging, tumor imaging, cell tracking, etc. The system matrix method is often used to reconstruct images from signals received from magnetic nanoparticle imaging devices. The system matrix reconstruction method uses the system matrix as a mapping between MPI images and MPI signals, and generates MPI images from the received frequency domain voltage signals. This method can obtain accurate measurement results, but the measurement of the system matrix requires calibration scanning, which is very time-consuming. Therefore, it is crucial to develop an algorithm to achieve fast MPI reconstruction.

[0003] In recent years, deep learning models have been introduced into the field of MPI. Many studies have applied deep learning technology to system matrix calibration, MPI signal denoising, image reconstruction or image post-processing, among which a representative one is a new deep learning method based on Transformer to accelerate MPI calibration, which can perform fast system matrix calibration. Most of them are aimed at optimizing the intermediate process of traditional MPI reconstruction methods. Few studies have explored the possibility of achieving end-to-end MPI reconstruction. Moreover, there is no research on MPI time-series image reconstruction methods, which is very important for the analysis of brain function and vascular perfusion.

[0004] Traditional time-series image reconstruction methods only generate images at the corresponding moments from the signals at a certain moment, ignoring the continuity of the time flow in the time series and the connection between the previous and subsequent images. The generated time-series images do not pay attention to the correlation between image time and do not achieve better spatiotemporal consistency. Therefore, for MPI brain function imaging or vascular imaging problems, it is necessary to construct an algorithm that directly converts the received one-dimensional voltage signal into time-series imaging of magnetic nanoparticles while maintaining the spatiotemporal consistency of the image, thereby achieving fast and accurate end-to-end MPI time-series image reconstruction.

[0005] There are two main challenges in MPI time series image reconstruction: too much noise in short-time signals and a balance between the global receptive field of the time series and the computational cost when reconstructing multiple frames.

[0006] Graph convolutional network (GCN) can capture the complex relationship between nodes and the structural characteristics of the graph by performing convolution operations on the graph. Since the MPI signal does not change much between adjacent frames, we construct a graph structure to perform joint analysis, retain the original signal, remove noise, and achieve better reconstruction. This is to process the input from the signal domain; U-net is composed of two parts: encoder (downsampling path) and decoder (upsampling path). The U-shaped convolutional neural network has been widely proven to play an important role in medical image analysis, but convolution focuses more on local information and cannot capture long-distance context features; Transformer architecture can obtain the consistency information of time series images based on the global receptive field of self-attention, but the attention mechanism has a large computational overhead and will grow quadratically with the input scale, so the computational cost for long sequence data is high; Mamba, as a new selective structural state space model, makes the model training and reasoning more efficient through a simple and effective selection mechanism and hardware-aware algorithm. When applied to the image domain for time series image processing, it can avoid the locality of convolution and the computational complexity of Transformer, and balance the computational cost and global receptive field well.

[0007] In summary, it is necessary to propose an MPI signal processing algorithm of multi-order graph convolution and an MPI video processing framework of U-type Mamba, which can be processed collaboratively from both the signal and image domains to achieve more efficient output of dynamic videos that focus on image temporal correlation and present image timing information. Summary of the invention

[0008] In order to solve the above problems in the prior art, namely, the existing MPI time-series image reconstruction has too much short-time signal noise, and the time-series global receptive field and computational cost balance are required for multi-frame reconstruction, the first aspect of the present invention proposes a magnetic nanoparticle time-series image reconstruction method based on dual-domain collaborative processing, the method comprising the following steps: Obtain a time-domain MPI signal, perform framing and Fourier transformation on the time-domain MPI signal to generate F frames of frequency-domain data, select d frequency points, integrate the frequency-domain features of c channels, and construct an input sequence of dimension (b, F, c, d) according to a batch size b; Each frame signal of the input sequence is used as a node and the frequency point signal is used as the node feature to construct a graph structure; Based on the graph structure, a multi-order graph convolutional network is used to combine adjacent frames to perform signal denoising and output a time-series MPI signal. ; The timing of the MPI signal After the linear projection layer transformation, the generated dimension becomes The time series image data enters the U-type Mamba model to obtain the reconstructed MPI time series image, where N represents the image size: The U-type Mamba model divides the time series image data into blocks according to the number of frames: , and linearly expand the channel to c' through the linear projection layer to obtain a transformation feature with a dimension of (b, B×F, c'), where B represents the image size and c' represents the number of channels; the transformation feature is compressed to a coding feature of (b, F, 8c', N / 8, N / 8) through a multi-level bidirectional Mamba spatiotemporal processing block and a downsampling block; the coding feature is generated through a multi-level bidirectional Mamba spatiotemporal processing block and an upsampling block to generate a decoding feature with a dimension of (b, F, c', N, N). After each level of downsampling, the current level coding feature is concatenated and fused with the corresponding level decoding feature. The feature information of the last level of concatenation is reduced in dimension by two-dimensional convolution to generate a dimension of MPI timing image.

[0009] In some preferred embodiments, the magnetic nanoparticle time-series image reconstruction method based on dual-domain collaborative processing further includes optimizing the MPI time-series image: Calculate a mean square error (MSE) based on the MPI time series image and the actual image as a first loss value; According to the time series MPI signal output by multi-order graph convolution And the L1 norm of the clean signal as the second loss value; The first loss value and the second loss value are weighted and fused to obtain a comprehensive loss value; a peak signal-to-noise ratio pSNR is calculated according to the first loss value to evaluate the quality of MPI time series image reconstruction and obtain a quality evaluation result; Evaluate the performance of the multi-order graph convolutional network and the U-shaped Mamba model according to the comprehensive loss value, and obtain the evaluation model performance result; Based on the quality assessment results, each parameter is updated by a gradient descent algorithm in each iteration; based on the evaluation model performance results, each parameter is updated by minimizing the loss value in each iteration; the parameters include parameters of a multi-order graph convolutional network and parameters of a U-type Mamba model.

[0010] In some preferred embodiments, the c channels are: a real channel and an imaginary channel for separating frequency domain data.

[0011] In some preferred embodiments, the timing MPI signal , and the method to obtain it is:

[0012]

[0013] ; ; In the formula, Representation Node Features, Representation Node of The characteristics of the order neighbors, represents the similarity operation, Representation Node Around Step node, Representation Node The degree, Representation Node Features, is the activation function, Indicates that the features are spliced. represents the similarity coefficient, For Matrices of the same dimension, Represents the transpose of a matrix.

[0014] In some preferred embodiments, the timing MPI signal After the linear projection layer transformation, the generated dimension becomes The time series image data is: Through the known dimension of M×d system matrix and the time-series MPI signal of dimension (b, F, c, d) Multiply them to get a matrix of dimension (b, F, c, M), and then arrange the M-dimensional features into an N×N matrix to generate a matrix of dimension Time series image data; where M=N 2 .

[0015] In some preferred embodiments, N=64.

[0016] In some preferred embodiments, the method for iteratively compressing to a coding feature of dimension (b, F, 8c', N / 8, N / 8) is: The spatiotemporal features with input dimension (b, F, c', N, N) are processed by the first-level bidirectional Mamba spatiotemporal processing block and the first-level downsampling block in turn, and the output dimension is (b, F, 2c', N / 2, N / 2) encoding features; The (b, F, 2c', N / 2, N / 2) coding feature is processed by a secondary bidirectional Mamba spatiotemporal processing block and a secondary downsampling block in sequence, and an output coding feature with a dimension of (b, F, 4c', N / 4, N / 4) is obtained; The (b, F, 4c', N / 4, N / 4) coding feature is processed in sequence by a three-level bidirectional Mamba spatiotemporal processing block and a three-level downsampling block, and an output coding feature with a dimension of (b, F, 8c', N / 8, N / 8) is obtained.

[0017] In some preferred embodiments, the method for iteratively generating a decoding feature with a dimension of (b, F, c', N, N) is: The features output after the coding features of (b, F, 8c', N / 8, N / 8) are processed by two bidirectional Mamba spatiotemporal processing blocks are concatenated with the coding features of (b, F, 8c', N / 8, N / 8), and the concatenated features are processed by a first-level upsampling block and a fourth-level bidirectional Mamba spatiotemporal processing block, and the output dimension is a decoding feature of (b, F, 4c', N / 4, N / 4); The coding features of the dimension (b, F, 4c', N / 4, N / 4) and the decoding features of the dimension (b, F, 4c', N / 4, N / 4) are concatenated, and after concatenation, they are processed in turn by a two-level upsampling block and a five-level bidirectional Mamba spatiotemporal processing block to output a decoding feature of the dimension (b, F, 2c', N / 2, N / 2); The encoding features of the dimension (b, F, 2c', N / 2, N / 2) and the decoding features of the dimension (b, F, 2c', N / 2, N / 2) are spliced, and after splicing, they are processed by a three-level upsampling block and a six-level bidirectional Mamba spatiotemporal processing block to output decoding features of the dimension (b, F, c', N, N).

[0018] In some preferred embodiments, the feature information of the last level of splicing and fusion is reduced in dimension by two-dimensional convolution to generate a dimension of The MPI timing image is: Reduce c' to 1, and obtain the dimension MPI timing image.

[0019] The second invention of the present invention proposes a magnetic nanoparticle time-series image reconstruction system based on dual-domain collaborative processing. Based on the magnetic nanoparticle time-series image reconstruction method based on dual-domain collaborative processing, the system includes a signal domain signal processing module and an image domain time-series image processing module. The signal domain signal processing module is configured to: obtain a time domain MPI signal, divide the time domain MPI signal into frames, perform Fourier transformation to generate F frames of frequency domain data, select d frequency points, integrate the frequency domain features of c channels, and construct an input sequence with a dimension of (b, F, c, d) according to a batch size b; Each frame signal of the input sequence is used as a node and the frequency point signal is used as the node feature to construct a graph structure; Based on the graph structure, a multi-order graph convolutional network is used to combine adjacent frames to perform signal denoising and output a time-series MPI signal. ; The timing of the MPI signal After the linear projection layer transformation, the generated dimension becomes The time series image data enters the U-type Mamba model to obtain the reconstructed MPI time series image, where N represents the image size: The image domain time series image processing module is configured as follows: the U-type Mamba model divides the time series image data into blocks according to the number of frames: , and linearly expand the channel to c' through the linear projection layer to obtain a transformation feature with a dimension of (b, B×F,c'), where B represents the image size and c' represents the number of channels; the transformation feature is compressed to a coding feature of (b,F,8c',N / 8,N / 8) through a multi-level bidirectional Mamba spatiotemporal processing block and a downsampling block; the coding feature is generated through a multi-level bidirectional Mamba spatiotemporal processing block and an upsampling block to generate a decoding feature with a dimension of (b,F,c',N,N). After each level of downsampling, the current level coding feature is concatenated and fused with the corresponding level decoding feature. The feature information of the last level of concatenation is reduced in dimension by two-dimensional convolution to generate a dimension of MPI timing image.

[0020] Beneficial effects of the present invention: 1) Receive the time-domain MPI signal, and perform adaptive noise suppression in the signal domain by combining multi-order graph convolution with adjacent frames. This can denoise the short-time signal noise in a targeted manner. In the image domain, the U-shaped Mamba model is used to process the time series image, which can well balance the computational cost and the global receptive field. In other words, the signal and image domains are processed in a coordinated manner to achieve more efficient output of dynamic videos that focus on image temporal association and present image temporal information. 2) In the training of multi-order graph convolution for MPI signal denoising and in the training of U-type Mamba for MPI video processing, the loss functions are calculated respectively, and according to the gradient of the loss function, the parameter values ​​are iteratively updated along the direction that makes the function value decrease the fastest, so as to improve the efficiency and quality of generating MPI time series images. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings: Figure 1 It is a data flow chart of the present invention; Figure 2 It is a multi-order graph convolutional network graph of the present invention; Figure 3 It is a structural diagram of the U-shaped Mamba of the present invention; Figure 4 It is a bidirectional Mamba block structure diagram of the present invention; Figure 5 It is a structural diagram of the bidirectional Mamba space-time processing block of the present invention. DETAILED DESCRIPTION

[0022] The present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the relevant invention, rather than to limit the invention. It is also necessary to explain that, for ease of description, only the parts related to the relevant invention are shown in the accompanying drawings.

[0023] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0024] In order to more clearly illustrate the magnetic nanoparticle time-series image reconstruction method based on dual-domain collaborative processing of the present invention, the following is combined with Figures 1 to 5 Each step in the embodiment of the present invention is described in detail.

[0025] The first embodiment of the present invention proposes a magnetic nanoparticle time-series image reconstruction method based on dual-domain collaborative processing, see Figure 1 , the method comprises the following steps: Step 1: obtain the time domain MPI signal, divide the time domain MPI signal into frames and perform Fourier transform to generate F frames of frequency domain data, i.e., frequency domain MPI signal, select d frequency points, integrate the frequency domain features of c channels, and construct an input sequence ‌ of dimension (b, F, c, d) according to the batch size b, denoted as ‌x; In this embodiment, the specific process of acquiring the time-domain MPI signal is as follows: in the MPI system, the selection field is a uniform gradient field, which can construct a region with a magnetic field intensity close to zero (Field Free Region, FFR); the induced voltage signal of the receiving coil is referred to as the time-domain MPI signal, which is a time-domain signal generated by the change in magnetic susceptibility of the object to be tested (the object to be tested containing magnetic particles, such as blood vessels and brain function injected with magnetic particles) in the FFR and its vicinity under the time-varying oscillation driving field. The time-domain MPI voltage signal is framed (sliding window) to generate F frame data, and Fourier transform is performed to generate F frame frequency domain data, that is, the frequency domain MPI signal, d frequency points are selected, the frequency domain features of c channels are integrated, and an input sequence x of dimension (b, F, c, d) is constructed according to the batch size b. The c channels include virtual multidimensional channels generated by time-frequency decomposition of a single receiving coil. Here, the virtual multidimensional channel is 2, which are the real part and imaginary part of the complex number (frequency domain MPI signal). Step 2, see Figure 2 , taking each frame signal of the input sequence as a node and the frequency point signal as the node feature, a graph structure is constructed, which is specifically constructed according to the K-neighbor principle; Based on the graph structure, a multi-order graph convolutional network is used to combine adjacent frames to perform signal denoising and output a time-series MPI signal. ; In this embodiment, the connection range between nodes in the K-neighbor principle is 5. For two frame signals, if the adjacent range is within 5 frames, it is considered that there is an edge; The general graph convolution network only considers the relationship between adjacent nodes. In the present invention, a multi-order graph convolution structure is constructed with an order of 3, that is, the connection between nodes with an order of K=3 and within 3 is considered; The timing of the MPI signal , and the method to obtain it is: ; ; ; ; In the formula, Representation Node Features, Representation Node of The characteristics of the order neighbors, represents the similarity operation, Representation Node Around Step node, Representation Node The degree, Representation Node Features, is the activation function, Indicates that the features are spliced. represents the similarity coefficient, For Matrices of the same dimension, Represents the transpose of a matrix; Step 3, the timing MPI signal After the linear projection layer, the generated dimension becomes The time series image data enters the U-type Mamba model to obtain the reconstructed MPI time series image, where N represents the image size; In this embodiment, the linear projection layer is converted into: through the known dimension of M×d system matrix and the dimension of (b, F, c, d) video frame sequence Multiply them to get a matrix of dimension (b, F, c, M), and then arrange the M-dimensional features into an N×N matrix to generate a matrix of dimension Time series image data, where M=N 2 ; Said N=64; Step 4: The state space model (SSM) is a model used to describe the state representation of the sequence at each time step and predict its next state based on the input. The core equations of the SSM include the state equation and the output equation: ; in, At any given time potential state, Indicates the updated status. Indicates input, Indicates the output, These three matrices are parameters, Matrix affects the previous state , Matrix influence input , Matrices transform states into outputs, which can be learned; However, in SSM, after learning, entering the inference phase, the matrix If the data is fixed, it will not be possible to perform targeted reasoning on different inputs. Mamba applies a selection mechanism that allows the model to selectively process information, filter out information that is not relevant to the problem, and remember information that is relevant to the problem for a long time; the Mamba architecture combines the SSM and the MLP blocks of the Transformer into a single block, and the parameters of the SSM are parameterized as a function of the input, allowing the model to adaptively adjust its behavior based on the input (Selective SSM); Based on the above technical status, a U-shaped Mamba model was constructed, see Figure 3 The U-type Mamba model divides the temporal image data into blocks according to the number of frames (specifically through block embedding: Patch Embedding is , b refers to batch size, B refers to image size, blocks are divided by the number of frames, the block size is the image size, and the number of blocks is the number of frames F, which is expressed here as , in order to make it an input form that Mamba can receive, and linearly expand the channel to c' (c' represents the number of channels, specifically through linear projection Linear projection linear expansion), and obtain the transformation feature of dimension; the transformation feature is compressed to (b, F, 8c', N / 8, N / 8) encoding feature through multi-level bidirectional Mamba spatiotemporal processing blocks and downsampling blocks; the encoding feature is generated by multi-level bidirectional Mamba spatiotemporal processing blocks and upsampling blocks. The decoding feature of the dimension is (b, F, c', N, N). After each level of downsampling, the current level encoding feature is spliced ​​and fused with the corresponding level decoding feature. The feature information of the last level of splicing and fusion is reduced in dimension by two-dimensional convolution (Conv2d) to generate a dimension of MPI time series images; In this embodiment, the bidirectional Mamba spatiotemporal processing block models the time series and generates dimensions of Spatiotemporal features; the spatiotemporal features are compressed to a dimension of The encoding features of the above are used to complete one iteration; the first input is the spatiotemporal features, and the subsequent input is the output of the previous iteration. This is repeated three times in sequence, and the iterative compression is performed to a dimension of (b, F, 8c', N / 8, N / 8) encoding features; the encoding features of the dimension (b, F, 8c', N / 8, N / 8) are processed by the bidirectional Mamba spatiotemporal processing block and the upsampling block (Patch Expanding), N is doubled, and c' is halved to generate a decoding feature of the dimension (b, F, 4c', N / 4, N / 4), and one iteration is completed; the first input is the encoding features of the dimension (b, F, 8c', N / 8, N / 8), and the subsequent input is the output of the previous iteration. This is repeated three times in sequence, and the decoding features of the dimension (b, F, c', N, N) are generated iteratively. After each level of downsampling, the encoding features of the current level are concatenated and fused with the decoding features of the corresponding level. The feature information of the last level of concatenation and fusion is reduced in dimension by two-dimensional convolution (Conv2d) to generate a decoding feature of the dimension The MPI time series image, the two-dimensional convolution dimension reduction is to reduce c' to 1, and the dimension is MPI time series images; The method of iterative compression to the dimension (b, F, 8c', N / 8, N / 8) encoding features is: The spatiotemporal features with input dimension (b, F, c', N, N) are processed by the first-level bidirectional Mamba spatiotemporal processing block and the first-level downsampling block in turn, and the output dimension is (b, F, 2c', N / 2, N / 2) encoding features; The (b, F, 2c', N / 2, N / 2) coding feature is processed by the secondary bidirectional Mamba spatiotemporal processing block and the secondary downsampling block in sequence, and the output dimension is a coding feature of (b, F, 4c', N / 4, N / 4); The (b, F, 4c', N / 4, N / 4) coding feature is processed by a three-level bidirectional Mamba spatiotemporal processing block and a three-level downsampling block in sequence, and the output dimension is a coding feature of (b, F, 8c', N / 8, N / 8); The method for iteratively generating decoding features of dimension (b, F, c', N, N) is: The features output after the coding features of (b, F, 8c', N / 8, N / 8) are processed by two bidirectional Mamba spatiotemporal processing blocks are concatenated with the coding features of (b, F, 8c', N / 8, N / 8), and the concatenated features are processed by a first-level upsampling block and a fourth-level bidirectional Mamba spatiotemporal processing block, and the output dimension is a decoding feature of (b, F, 4c', N / 4, N / 4); The coding features of the dimension (b, F, 4c', N / 4, N / 4) and the decoding features of the dimension (b, F, 4c', N / 4, N / 4) are concatenated, and after concatenation, they are processed in turn by a two-level upsampling block and a five-level bidirectional Mamba spatiotemporal processing block to output a decoding feature of the dimension (b, F, 2c', N / 2, N / 2); The coding features of the dimension (b, F, 2c', N / 2, N / 2) and the decoding features of the dimension (b, F, 2c', N / 2, N / 2) are concatenated, and after concatenation, a three-level upsampling block and a six-level bidirectional Mamba spatiotemporal processing block are processed to output a decoding feature of the dimension (b, F, c', N, N); Among them, see Figure 4 , 5 The specific working process of the bidirectional Mamba space-time processing block is as follows: First, the input block sequence is rearranged into a dimensional shape that conforms to the Mamba module (type) can accept. , normalized through a normalization layer; Then the normalized sequence is linearly projected into two vectors x and q, where x represents the input feature for state space model (SSM) processing, and h forward and h backward processing are performed on it to extract features from two directions, where the processing is convolution and SSM performed in sequence; q is used to control the weight of the forward and backward outputs. The results of the forward and backward processing are weighted and summed by q to obtain the bidirectional output y. y is mapped to the original feature dimension through a linear projection layer, and is residually connected with the input sequence and added to obtain the output of the current bidirectional Mamba block; Before the bidirectional Mamba block outputs, it undergoes “dimensional rearrangement” to convert the output sequence into a shape that can be accepted by downsampling / upsampling, for example ; The above is the process of bidirectional Mamba block processing. The design of bidirectional processing enables the model to extract features from both the forward and backward directions at the same time, thereby capturing the global context information in the image more comprehensively. The output sequence processed by the bidirectional Mamba block is encoded in time frames, that is, the time frame dimension is highlighted, and then mapped to a specific dimension through a linear projection layer, and input into the self-attention block for processing. The self-attention block is based on the self-attention mechanism, which is an attention mechanism that associates different positions of a single sequence to calculate the representation of the same sequence. When the self-attention block processes the time series MPI data, it dynamically adjusts the degree of attention to each video frame, thereby capturing the complex dependencies within the frame. The working process of the self-attention block is as follows: First, a query vector (Q), a key vector (K), and a value vector (V) are calculated for each element in the input sequence; For each pair of elements, calculate the attention score , and then through Activation function derives attention weight , the specific calculation formula is: Indicates elements of the query vector, Indicates elements of the key vector, is the dimension of the key vector;

[0026] in, The standard expression of the activation function is, , calculate each pair of attention scores Value Regards Substituting the standard expression of the softmax activation function, we get The query vector of elements, The attention weights of the key vector of elements , that is, the vector is converted into a probability distribution through the softmax function, the weight value of each pair of elements is between 0 and 1 and the sum of the weight values ​​of all elements is 1; Finally, the value vector of each element The corresponding attention weight After multiplication and summing, we get The output of the elements after the self-attention mechanism :

[0027] In the formula, n represents the total number of elements; After processing the sequence self-attention block, the extraction of temporal features is enhanced, and then the dimension of the data before inputting the self-attention block is restored through the linear layer; The above is the complete process of the bidirectional Mamba spatiotemporal processing block. The bidirectional Mamba block enhances the features spatially, and the self-attention block enhances the features temporally. After one bidirectional Mamba spatiotemporal processing block, the time series image is enhanced. In this embodiment, the magnetic nanoparticle time-series image reconstruction method based on dual-domain collaborative processing further includes optimizing the MPI time-series image: The mean square error MSE is calculated based on the MPI time series image and the actual image as the first loss value; the time series MPI signal output by the multi-order graph convolution is And the L1 norm of the clean signal as the second loss value; The first loss value and the second loss value are weighted and fused to obtain a comprehensive loss value; the peak signal-to-noise ratio pSNR is calculated according to the first loss, the quality of MPI time series image reconstruction is evaluated, and a quality evaluation result is obtained; the performance of the multi-order graph convolutional network and the U-type Mamba model is evaluated according to the comprehensive loss value, and a model performance evaluation result is obtained; Based on the quality assessment results, each parameter is updated by a gradient descent algorithm in each iteration; based on the evaluation model performance results, each parameter is updated by minimizing the loss value in each iteration; the parameters include parameters of a multi-order graph convolutional network and parameters of a U-type Mamba model to reduce prediction errors.

[0028] Although the various steps in the above embodiment are described in the above-mentioned order, those skilled in the art can understand that in order to achieve the effect of this embodiment, different steps do not have to be executed in such an order. They can be executed simultaneously (in parallel) or in a reverse order. These simple changes are within the scope of protection of the present invention.

[0029] The second embodiment of the present invention is a magnetic nanoparticle time-series image reconstruction system based on dual-domain collaborative processing. Based on the magnetic nanoparticle time-series image reconstruction method based on dual-domain collaborative processing, the system includes a signal domain signal processing module and an image domain time-series image processing module. The signal domain signal processing module is configured to: obtain a time domain MPI signal, divide the time domain MPI signal into frames, perform Fourier transformation to generate F frames of frequency domain data, select d frequency points, integrate the frequency domain features of c channels, and construct an input sequence with a dimension of (b, F, c, d) according to a batch size b; Each frame signal of the input sequence is used as a node and the frequency point signal is used as the node feature to construct a graph structure; Based on the graph structure, a multi-order graph convolutional network is used to combine adjacent frames to perform signal denoising and output a time-series MPI signal. ; The timing of the MPI signal After the linear projection layer transformation, the generated dimension becomes The time series image data enters the U-type Mamba model to obtain the reconstructed MPI time series image, where N represents the image size: The image domain time series image processing module is configured as follows: the U-type Mamba model divides the time series image data into blocks according to the number of frames: , and linearly expand the channel to c' through the linear projection layer to obtain a transformation feature with a dimension of (b, B×F,c'), where B represents the image size and c' represents the number of channels; the transformation feature is compressed to a coding feature of (b,F,8c',N / 8,N / 8) through a multi-level bidirectional Mamba spatiotemporal processing block and a downsampling block; the coding feature is generated through a multi-level bidirectional Mamba spatiotemporal processing block and an upsampling block to generate a decoding feature with a dimension of (b,F,c',N,N). After each level of downsampling, the current level coding feature is concatenated and fused with the corresponding level decoding feature. The feature information of the last level of concatenation is reduced in dimension by two-dimensional convolution to generate a dimension of MPI timing image.

[0030] It should be noted that the magnetic nanoparticle time-series image reconstruction system based on dual-domain collaborative processing provided in the above embodiment is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the modules or steps in the embodiments of the present invention can be decomposed or combined. For example, the modules in the above embodiments can be combined into one module, or further divided into multiple sub-modules to complete all or part of the functions described above. The names of the modules and steps involved in the embodiments of the present invention are only for distinguishing the modules or steps, and are not regarded as improper limitations of the present invention.

[0031] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

[0032] Those skilled in the art should be able to appreciate that the modules and method steps of each example described in conjunction with the embodiments disclosed herein can be implemented with electronic hardware, computer software, or a combination of the two, and the programs corresponding to the software modules and method steps can be placed in random access memory (RAM), memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs, or any other form of storage medium known in the technical field. In order to clearly illustrate the interchangeability of electronic hardware and software, the composition and steps of each example have been generally described in the above description according to the function. Whether these functions are performed in electronic hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0033] The terms "first", "second", etc. are used to distinguish similar objects rather than to describe or indicate a particular order or sequence.

[0034] The term "comprise" or any other similar term is intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus / device that includes a list of elements includes not only those elements but also other elements not expressly listed, or also includes elements inherent to such process, method, article, or apparatus / device.

[0035] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

Claims

1. A magnetic nanoparticle time-series image reconstruction method based on dual-domain collaborative processing, characterized in that: The method comprises the following steps: Obtain a time-domain MPI signal, perform frame division and Fourier transformation on the time-domain MPI signal to generate F frames of frequency-domain data, select d frequency points, integrate the frequency-domain features of c channels, and construct an input sequence of dimensions (b, F, c, d) according to a batch size b; construct a graph structure with each frame signal of the input sequence as a node and a frequency point signal as a node feature; Based on the graph structure, a multi-order graph convolutional network is used to combine adjacent frames to perform signal denoising and output a time-series MPI signal. ; The timing of the MPI signal After the linear projection layer transformation, the generated dimension becomes The time series image data enters the U-type Mamba model to obtain the reconstructed MPI time series image, where N represents the image size: The U-type Mamba model divides the time series image data into blocks according to the number of frames: , and linearly expand the channel to c' through the linear projection layer to obtain a transformation feature with a dimension of (b, B×F, c'), where B represents the image size and c' represents the number of channels; the transformation feature is compressed to a coding feature of (b, F, 8c', N / 8, N / 8) through a multi-level bidirectional Mamba spatiotemporal processing block and a downsampling block; the coding feature is generated through a multi-level bidirectional Mamba spatiotemporal processing block and an upsampling block to generate a decoding feature with a dimension of (b, F, c', N, N). After each level of downsampling, the current level coding feature is concatenated and fused with the corresponding level decoding feature. The feature information of the last level of concatenation is reduced in dimension by two-dimensional convolution to generate a dimension of MPI timing image.

2. The method for reconstructing magnetic nanoparticle time-series images based on dual-domain collaborative processing according to claim 1, characterized in that: The magnetic nanoparticle time-series image reconstruction method based on dual-domain collaborative processing also includes optimizing the MPI time-series image: Calculate the mean square error MSE according to the MPI time series image and the actual image as the first loss value; According to the time series MPI signal output by multi-order graph convolution And the L1 norm of the clean signal as the second loss value; The first loss value and the second loss value are weighted and fused to obtain a comprehensive loss value; the peak signal-to-noise ratio pSNR is calculated according to the first loss value, the quality of MPI time series image reconstruction is evaluated, and a quality evaluation result is obtained; the performance of the multi-order graph convolutional network and the U-type Mamba model is evaluated according to the comprehensive loss value, and a performance evaluation model result is obtained; Based on the quality assessment results, updating each parameter by a gradient descent algorithm in each iteration; Based on the evaluation model performance results, each parameter is updated by minimizing the loss value in each iteration; the parameters include parameters of a multi-order graph convolutional network and parameters of a U-type Mamba model.

3. The method for reconstructing magnetic nanoparticle time-series images based on dual-domain collaborative processing according to claim 1, characterized in that: The c channels are: a real channel and an imaginary channel for separating frequency domain data.

4. The method for reconstructing magnetic nanoparticle time-series images based on dual-domain collaborative processing according to claim 1, characterized in that: The timing of the MPI signal , and the method to obtain it is: ； ； ； ； In the formula, Representation Node Features, Representation Node of The characteristics of the order neighbors, represents the similarity operation, Representation Node Around Step node, Representation Node The degree, Representation Node Features, is the activation function, Indicates that the features are spliced. represents the similarity coefficient, For Matrices of the same dimension, Represents the transpose of a matrix.

5. The method for reconstructing magnetic nanoparticle time-series images based on dual-domain collaborative processing according to claim 1, characterized in that: The timing of the MPI signal After the linear projection layer transformation, the generated dimension becomes The time series image data is: Through the known dimension of M×d system matrix and the dimension of (b, F, c, d) time series MPI signal Multiply them to get a matrix of dimension (b, F, c, M), and then arrange the M-dimensional features into an N×N matrix to generate a matrix of dimension Time series image data; where M=N 2 .

6. The method for reconstructing magnetic nanoparticle time-series images based on dual-domain collaborative processing according to claim 5, characterized in that: Said N=64.

7. The method for reconstructing magnetic nanoparticle time-series images based on dual-domain collaborative processing according to claim 1, characterized in that: The method of iterative compression to the dimension (b, F, 8c', N / 8, N / 8) encoding features is: The spatiotemporal features with input dimension (b, F, c', N, N) are processed by the first-level bidirectional Mamba spatiotemporal processing block and the first-level downsampling block in turn, and the output dimension is (b, F, 2c', N / 2, N / 2) encoding features; The (b, F, 2c', N / 2, N / 2) coding feature is processed by a secondary bidirectional Mamba spatiotemporal processing block and a secondary downsampling block in sequence, and an output coding feature with a dimension of (b, F, 4c', N / 4, N / 4) is obtained; The (b, F, 4c', N / 4, N / 4) coding feature is processed in sequence by a three-level bidirectional Mamba spatiotemporal processing block and a three-level downsampling block, and an output coding feature with a dimension of (b, F, 8c', N / 8, N / 8) is obtained.

8. The method for reconstructing magnetic nanoparticle time-series images based on dual-domain collaborative processing according to claim 7, characterized in that: The method for iteratively generating decoding features of dimension (b, F, c', N, N) is: The features output after the coding features of (b, F, 8c', N / 8, N / 8) are processed by two bidirectional Mamba spatiotemporal processing blocks are concatenated with the coding features of (b, F, 8c', N / 8, N / 8), and the concatenated features are processed by a first-level upsampling block and a fourth-level bidirectional Mamba spatiotemporal processing block, and the output dimension is a decoding feature of (b, F, 4c', N / 4, N / 4); The coding features of the dimension (b, F, 4c', N / 4, N / 4) and the decoding features of the dimension (b, F, 4c', N / 4, N / 4) are concatenated, and after concatenation, they are processed in turn by a two-level upsampling block and a five-level bidirectional Mamba spatiotemporal processing block to output a decoding feature of the dimension (b, F, 2c', N / 2, N / 2); The encoding features of the dimension (b, F, 2c', N / 2, N / 2) and the decoding features of the dimension (b, F, 2c', N / 2, N / 2) are spliced, and after splicing, they are processed by a three-level upsampling block and a six-level bidirectional Mamba spatiotemporal processing block to output decoding features of the dimension (b, F, c', N, N).

9. The method for reconstructing magnetic nanoparticle time-series images based on dual-domain collaborative processing according to claim 1, characterized in that: The feature information of the last level of splicing and fusion is reduced by two-dimensional convolution to generate a dimension of The MPI timing image is: Reduce c' to 1, and obtain the dimension MPI timing image.

10. A magnetic nanoparticle time-series image reconstruction system based on dual-domain collaborative processing, based on a magnetic nanoparticle time-series image reconstruction method based on dual-domain collaborative processing according to any one of claims 1 to 9, the system comprising a signal domain signal processing module and an image domain time-series image processing module, wherein the signal domain signal processing module is configured to: obtain a time domain MPI signal, perform frame division and Fourier transformation on the time domain MPI signal to generate F frames of frequency domain data, select d frequency points, integrate the frequency domain features of c channels, and construct an input sequence of dimension (b, F, c, d) according to a batch size b; construct a graph structure using each frame signal of the input sequence as a node and the frequency point signal as a node feature; Based on the graph structure, a multi-order graph convolutional network is used to combine adjacent frames to perform signal denoising and output a time-series MPI signal. ; The timing of the MPI signal After the linear projection layer transformation, the generated dimension becomes The time series image data enters the U-type Mamba model to obtain the reconstructed MPI time series image, where N represents the image size: The image domain time series image processing module is configured as follows: the U-type Mamba model divides the time series image data into blocks according to the number of frames: , and linearly expand the channel to c' through the linear projection layer to obtain a transformation feature with a dimension of (b, B×F, c'), where B represents the image size and c' represents the number of channels; the transformation feature is compressed to a coding feature of (b, F, 8c', N / 8, N / 8) through a multi-level bidirectional Mamba spatiotemporal processing block and a downsampling block; the coding feature is generated through a multi-level bidirectional Mamba spatiotemporal processing block and an upsampling block to generate a decoding feature with a dimension of (b, F, c', N, N). After each level of downsampling, the current level coding feature is concatenated and fused with the corresponding level decoding feature. The feature information of the last level of concatenation is reduced in dimension by two-dimensional convolution to generate a dimension of MPI timing image.

Images