Magnetic nanoparticle device and method for three-dimensional rapid and direct imaging of nervous system

Abstract

The present invention belongs to the field of magnetic particle imaging technology, and specifically relates to a magnetic nanoparticle device and method for three-dimensional rapid and direct imaging of the nervous system, aiming to solve the problems of low temporal resolution of magnetic particle signal detection in existing magnetic particle imaging technology, low imaging quality, slow speed, large amount of calculation, and complex calculation method. The device of the present invention: signal generator 1 outputs a voltage signal and amplifies it and then connects it to a selection coil and a drive coil; signal generator 2 outputs a voltage signal and then connects it to an excitation coil; there are two selection coils, two as a pair, arranged in the Y direction; there are six drive coils, excitation coils, and receiving coils, two as a pair, respectively arranged in the X direction, the Y direction, and the Z direction; a high-pass filter filters the signal received by the receiving coil and imports it into a computer through a signal acquisition card to reconstruct the magnetic particle image. The present invention improves the temporal resolution of magnetic particle signal detection and proposes a method for effectively reconstructing MPI time-series video.

Description

Technical Field

[0001] The invention belongs to the technical field of magnetic particle imaging, and in particular relates to a magnetic nanoparticle device and method for three-dimensional rapid and direct imaging of a nervous system. Background Art

[0002] Since its invention in 1991, BOLD-fMRI imaging technology for human has been widely used in cognitive tasks such as vision and hearing. In the United States, more than 90% of psychology research topics are subjected to bold-fMRI scans. In addition to humans, fMRI technology has also been used to study various cognitive activities of awake monkeys. With the development of technology, researchers have found that when superparamagnetic iron oxide nanoparticles (SPIONs) are injected into monkeys before scanning, the functional signal sensitivity measured is 5 times that of the Bold signal, and the detected brain function activation and brain connection strength are greatly increased (that is, injection of SPIONs can significantly improve the sensitivity of primate fMRI imaging). Therefore, injection of SPIONs is currently a routine step in macaque fMRI imaging.

[0003] However, the Bold-fMRI method uses deoxyhemoglobin as a natural contrast agent, and the observed hemodynamic changes caused by stimulation are relatively weak. Studies usually require the average of a large number of subjects to achieve statistical significance. In addition, its sensitivity for detecting brain neural activity is very low. Studies have shown that BOLD-fMRI can only capture less than 2% of changes in brain neural activity. Although the SPION-fMRI method has improved sensitivity compared to the Bold-fMRI method, in essence, both SPION-fMRI and BOLD-fMRI use MRI to indirectly image changes in intravascular contrast agent content. Therefore, the improvement in the sensitivity of SPION-fMRI for brain function detection is very limited.

[0004] Magnetic nanoparticle imaging (MPI) is a new imaging method used to reconstruct the concentration distribution of superparamagnetic nanoparticles in the object to be tested. MPI imaging requires the use of tracers, and signals can only be generated when the tracer exists in the imaging area. Superparamagnetic iron oxide nanoparticles (SPION) are generally used as tracers. The MPI device has no background signal interference from the human body itself, which makes the MPI image have excellent contrast and high sensitivity, and the signal intensity is proportional to the tracer concentration. It is an inspection method that can obtain quantitative data. These characteristics make it an ideal paradigm for applications such as stem cell tracking, angiography, and targeted drug delivery.

[0005] fMPI is a natural way to image cerebral blood volume (CBV), which detects changes in the CBV of the test subject through fast MPI time-series imaging. Because changes in CBV caused by brain activity cause changes in the distribution of SPIONs, when the external alternating excitation magnetic field in the MPI device acts on the SPIONs, the SPIONs will generate continuous magnetic response signals of a specific frequency, which can be directly detected by the MPI device. And fMPI has a natural sensitivity to hemodynamics and blood volume changes related to brain activation, and can produce highly sensitive continuous magnetic response signals.

[0006] However, whether it is fMRI or fMPI, the current imaging method is essentially to realize the conversion of one-dimensional signal sequence into two-dimensional image sequence. In the field of fMRI, the sequence using T2×WI contrast can best reflect the change of local T2 value, and compared with SE sequence and gradient echo sequence, gradient echo sequence is more sensitive to the change of magnetic sensitivity. Therefore, the BOLD sequence is generally performed using a gradient echo sequence with T2×WI contrast. The reason why three-dimensional time-series fMRI video is not generated directly from the BOLD sequence is, on the one hand, because the imaging time of a single element in the sequence is long, and on the other hand, because there is a lack of end-to-end processing signal sequence to video method.

[0007] In summary, it is necessary to study the feasibility of quickly and directly generating MPI time-series functional videos from one-dimensional signal sequences based on MPI. Neural response activities have received particular attention in recent years. Reconstructing MPI time-series functional videos by detecting neural response activities is an important prerequisite for evaluating whether fMPI can be used as a potential clinical diagnostic technology and tool. It is also an important prerequisite for evaluating whether it can provide ideas for directly and quickly generating time-series fMRI videos from BOLD sequences for the reconstruction of traditional fMRI.

[0008] There are two types of traditional MPI reconstruction techniques: one relies on the system matrix to pre-characterize the signal response of SPION, and the other is the X-space algorithm. Both traditional methods have their shortcomings and are not suitable for the reconstruction of MPI time-series functional videos.

[0009] With the development of neural networks in the field of computer vision, the methods suitable for fMPI reconstruction are gradually increasing. In addition to recognizing and learning traditional two-dimensional semantic information, the three-dimensional network also recognizes and learns semantic information and dependencies in the time dimension. For videos, the visual signals can be divided into two parts: content and motion. Content specifies the objects in the video, while motion describes their dynamics. Therefore, in addition to learning the content model of the object, the generative model also needs to learn a reasonable motion model of the object. Therefore, many networks in the field of video generation are designed with the idea of ​​"separation", or separating the foreground and background information in the video, or separating the content and motion information in the video. Based on this, the present invention combines the GAN network and the multi-task idea to propose a magnetic nanoparticle device and method for three-dimensional fast and direct imaging of the nervous system. Summary of the invention

[0010] In order to solve the above problems in the prior art, that is, to solve the problems of low time resolution of magnetic particle signal detection in the existing magnetic particle imaging technology, low imaging quality, slow speed, large amount of calculation, and complex calculation method, the present invention proposes a magnetic nanoparticle device for three-dimensional fast and direct imaging of the nervous system, the device includes: a signal generator, a power amplifier, a selection coil, a driving coil, an excitation coil, a receiving coil, a high-pass filter, and a signal acquisition card;

[0011] The signal generator comprises a signal generator 1 and a signal generator 2; the signal generator 1 outputs a voltage signal and is connected to the selection coil and the driving coil after being amplified by the power amplifier; the signal generator 2 outputs a voltage signal and is connected to the excitation coil;

[0012] There are two selection coils, which are arranged as a pair in the Y direction; the two selection coils are arranged in a Maxwell configuration and have the same axis;

[0013] There are six drive coils, which are arranged in pairs in the X direction, the Y direction, and the Z direction respectively; the two drive coils in the same direction are arranged in a Helmholtz configuration and have the same axis; the drive coils in the same direction are located inside the selection coils in the same direction;

[0014] There are six excitation coils, which are arranged in pairs in the X direction, the Y direction, and the Z direction respectively; the two excitation coils in the same direction are arranged in a Helmholtz configuration and have the same axis; the excitation coils in the same direction are located inside the driving coils in the same direction;

[0015] There are six receiving coils, which are arranged in pairs in the X direction, the Y direction and the Z direction respectively; the two receiving coils in the same direction are arranged symmetrically and in parallel with the same axis; the receiving coils in the same direction are located inside the exciting coils in the same direction; the magnetic field free zone is in the enclosed space formed by the receiving coils;

[0016] The high-pass filter filters out interference signals of the magnetic particle signals generated at the FFP positions at three latitudes received by the receiving coil; the signal acquisition card is used to import the signals filtered by the high-pass filter into a computer for magnetic particle image reconstruction.

[0017] In some preferred embodiments, the selection coil, the drive coil, the excitation coil, and the receiving coil are all annular coils, and the selection coil, the drive coil, the excitation coil, and the receiving coil in the same direction have the same axis.

[0018] In a second aspect of the present invention, a magnetic nanoparticle method for three-dimensional rapid and direct imaging of the nervous system is proposed. Based on the magnetic nanoparticle device for three-dimensional rapid and direct imaging of the nervous system, the method comprises:

[0019] S100, collecting one-dimensional MPI signals by using the constructed magnetic nanoparticle device for three-dimensional rapid and direct imaging of the nervous system to construct a one-dimensional signal sequence;

[0020] S200, inputting the one-dimensional signals in the one-dimensional signal sequence into the generator in the pre-trained 3D-Multitask-fMPI-GAN generative adversarial network in sequence, and reconstructing through the reconstruction task branch of the generator to obtain a three-dimensional time-series grayscale motion video;

[0021] The 3D-Multitask-fMPI-GAN generative adversarial network includes a multi-task generator, a 3D discriminator, and a 2D discriminator; the multi-task generator includes a reconstruction task branch and a segmentation task branch; the multi-task generator includes a reconstruction task branch and a segmentation task branch; the reconstruction task branch is used to generate a one-dimensional signal sequence input into the multi-task generator into a three-dimensional temporal grayscale motion video; the segmentation task branch is used to generate a one-dimensional signal sequence input into the multi-task generator into a three-dimensional temporal approximate binary motion video; the 3D discriminator is used to combine the generated three-dimensional temporal grayscale motion video with the real three-dimensional temporal grayscale motion video in the corresponding data set to determine the authenticity of the three-dimensional temporal grayscale motion video; the 2D discriminator is used to determine the authenticity of the two-dimensional image based on the two-dimensional image after frame sampling of the generated three-dimensional temporal grayscale motion video and the real three-dimensional temporal grayscale motion video in the corresponding data set.

[0022] In some preferred embodiments, the multi-task generator includes a front linear layer, a bidirectional two-layer GRU network, M sequentially connected upsampling blocks, a reconstruction task branch, and a segmentation task branch;

[0023] The linear layer is used to perform linear transformation processing on each one-dimensional signal in the input one-dimensional signal sequence;

[0024] The input of the bidirectional double-layer GRU network: one is the signal sequence of all time steps after the linear layer processing, and the other is the initialization state; the initialization state is a standard normal distribution; the output of the bidirectional double-layer GRU network is the output sequence processed by the bidirectional double-layer GRU network, excluding the final hidden state, and the output sequence is output to the first upsampling block;

[0025] The upsampling block includes 3D Position Embedding, Video Swin transformer Block, and 3D Time-patch expanding;

[0026] The 3D Position Embedding is used to initialize a parameter matrix matching the dimension of the input tensor of the upsampling block as position encoding information embedding, and directly add it to the input tensor, and use the added tensor as the first tensor;

[0027] The Video Swin transformer Block is used to calculate the importance weight of the first tensor through a self-attention mechanism, and weight the first tensor based on the importance weight to obtain a second tensor;

[0028] The 3D Time-patch expanding is used to double the number of channels of the second tensor through an internal linear layer without changing the number of video frames, and after doubling, rearrange the tensors so that the number of channels of the tensor becomes half of the number of channels of the output tensor of the Video Swin transformer Block, and at the same time double the width and height of each frame of the image;

[0029] There are a total of M upsampling blocks in the network, so the above upsampling block processing process is cycled M times in total until the output of the last upsampling block meets the image height, width, and channel number requirements to be received by the subsequent multi-task branch; wherein, starting from the second upsampling block, the input of each sampling block is the output of the previous sampling block;

[0030] The reconstruction task branch is used to process the output of the last upsampling block through N Video Swintransformer Blocks and 3D convolution layers in sequence to obtain a three-dimensional time-series grayscale motion video; N is an even number;

[0031] The segmentation task branch is used to process the output of the last upsampling block through N VideoSwin transformer Blocks, 3D convolution layers, and activation layers in sequence to obtain a three-dimensional temporal approximate binary motion video.

[0032] In some preferred embodiments, the self-attention mechanism includes 3D window self-attention and 3D sliding window self-attention.

[0033] In some preferred embodiments, the 3D discriminator adopts a video swin transformer as a basic framework, and a time-image fusion cross-attention module is added after the basic calculation modules of the video swin transformer including 3D W-MSA and 3D SW-MSA; the time-image fusion cross-attention module is used to calculate the dependency between different time dimensions and the spatial semantic information within the same frame, and weightedly combine the time information and spatial information to update the model parameters; the dependency between different time dimensions is the dependency between different frames of the three-dimensional temporal grayscale motion video.

[0034] In some preferred embodiments, the 2D discriminator is constructed based on a swin transformer.

[0035] In some preferred embodiments, the loss function of the multi-task generator of the 3D-Multitask-fMPI-GAN generative adversarial network during training is:

[0036]

[0037] in, represents the loss of the multi-task generator, represents the real image, Indicates signal, represents the one-dimensional signal sequence of the input, It represents the 2D image of the i-th frame of the i-th signal in the input one-dimensional MPI signal sequence, obtained by sampling all frames of the three-dimensional time-series grayscale motion video generated by the multi-task generator. It represents the T-frame two-dimensional image obtained after all frames of the three-dimensional temporal approximate binary motion video generated by the multi-task generator are sampled, corresponding to the i-th frame two-dimensional image of the i-th signal in the input one-dimensional MPI signal sequence, represents the three-dimensional temporal grayscale motion video generated by the multi-task generator, It represents the 2D image of the i-th frame of the i-th signal in the input one-dimensional MPI signal sequence, obtained by sampling all frames of the three-dimensional time-series grayscale motion video in the training set. It represents the 2D image of the i-th frame of the i-th signal in the input one-dimensional MPI signal sequence, obtained by sampling all frames of the three-dimensional time-series binary motion video in the training set. represents the mean absolute error function, represents the BCE function, represents the Dice function, , , They represent the penalty items corresponding to the mean absolute error function, BCE function, and Dice function, respectively. , Respectively represent 2D discriminator and 3D discriminator, represents mathematical expectation.

[0038] In some preferred embodiments, the loss functions of the 3D discriminator and the 2D discriminator of the 3D-Multitask-fMPI-GAN generative adversarial network during training are:

[0039]

[0040]

[0041] in, , Respectively represent the loss of 2D discriminator and the loss of 3D discriminator, Represents the 3D temporal grayscale motion video in the training set.

[0042] Beneficial effects of the present invention:

[0043] The present invention improves the time resolution of magnetic particle signal detection, simplifies the complexity and amount of calculation, improves the quality of magnetic particle imaging, and proposes a method for effectively reconstructing MPI time-series video.

[0044] 1) In practice, there is another scheme in which the excitation coil and the drive coil in the magnetic particle imaging device share the same coil. This scheme distinguishes the functions of the excitation coil and the drive coil by passing currents of different parameters through the same coil. The present invention distinguishes the excitation coil and the drive coil from the physical structure, which can achieve more precise control and adjustment of the driving working magnetic field and the high-frequency time-varying excitation magnetic field. The excitation coil can additionally provide alternating magnetic fields of different frequencies or different waveforms to more effectively stimulate the nonlinear response of magnetic nanoparticles. In addition, the drive coil, the excitation coil, and the receiving coil are all placed in the three dimensions of XYZ, which can realize the three-dimensional spatial movement of the FFP area, and the scanning of the object to be measured is more comprehensive and accurate.

[0045] 2) The present invention can use simulation software to generate training data in advance. After the training is completed and the fixed model parameters are obtained, the corresponding reconstructed MPI time series function video can be directly obtained by inputting a one-dimensional signal sequence. An end-to-end processing method is adopted, and there is no need to generate a two-dimensional image transition. The spatiotemporal feature information is integrated in the generation process, which can reduce the amount of calculation and improve the generation quality of the MPI time series function video. Compared with two-dimensional images without time series information, MPI time series function videos can display dynamically changing MPI images and provide more useful information required clinically. In addition, compared with the solution of first generating a time series image sequence and then splicing it into a video, this solution is faster and less time-consuming, and can also provide new ideas for the reconstruction of traditional fMRI.

[0046] 3) The present invention adopts a multi-task design concept. The reconstruction task is responsible for generating a three-dimensional time-series grayscale motion video as the main task, while the segmentation task is responsible for generating a three-dimensional time-series approximate binary motion video as an auxiliary task. When the quality of the auxiliary task is improved (the image foreground and background segmentation is more accurate), the boundaries of the single-frame images in the three-dimensional time-series grayscale motion video generated by the reconstruction task can be guided to be clearer and sharper, while reducing the burden of the reconstruction task, making it easier to focus on the high-dimensional features of the reconstructed image, and leaving the details to the segmentation task (because the segmentation task and the reconstruction task have a common network part, they can share weights during training, which can serve as an auxiliary to better guide the completion of the reconstruction task). BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings.

[0048] Figure 1 It is a schematic structural diagram of a magnetic nanoparticle device for three-dimensional rapid and direct imaging of the nervous system according to an embodiment of the present invention;

[0049] Figure 2 It is a detailed structural schematic diagram of each coil in a magnetic nanoparticle device for three-dimensional rapid and direct imaging of a nervous system according to an embodiment of the present invention;

[0050] Figure 3 is a structural diagram of a task generator according to an embodiment of the present invention;

[0051] Figure 4 It is a schematic diagram of the processing process of a 2D discriminator and a 3D discriminator in one embodiment of the present invention;

[0052] Figure 5 is a schematic diagram of a phantom model made according to an embodiment of the present invention;

[0053] Figure 6It is a schematic diagram of injecting a magnetic nanoparticle reagent into a phantom model in one embodiment of the present invention. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0055] The present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the relevant invention, rather than to limit the invention. It should be noted that, in the absence of conflict, the embodiments in the present application and the features in the embodiments can be combined with each other.

[0056] The present invention provides a magnetic nanoparticle device and method for three-dimensional rapid and direct imaging of the nervous system. It can realize the generation and collection of one-dimensional magnetic particle signal sequences by a three-dimensional MPI device, and the input of the collected signal sequence into a multi-task generative adversarial network, so as to directly and quickly reconstruct the MPI time-series functional video to visualize the neural response activity. The details are as follows:

[0057] The first embodiment of the present invention provides a magnetic nanoparticle device for three-dimensional rapid and direct imaging of the nervous system, such as Figure 1 , 2 (i.e., the main three-dimensional MPI device), including: a signal generator, a power amplifier, a selection coil, a driving coil, an excitation coil, a receiving coil, a high-pass filter, and a signal acquisition card; all coils are circular coils;

[0058] The signal generator includes a signal generator 1 and a signal generator 2; the signal generator 1 outputs a voltage signal. In order to improve the zero field line gradient formed subsequently and thus improve the resolution of the MPI system, a large current is usually required to achieve a more refined zero field line. Therefore, the output voltage needs to be amplified by a power amplifier and connected to the selection coil and the drive coil after amplification; the signal generator 2 outputs a voltage signal and then connects to the excitation coil;

[0059] There are two selection coils, which are arranged as a pair in the Y direction. The two selection coils are arranged in a Maxwell configuration (i.e., symmetrically placed and with current in opposite directions) and have the same axis. This will generate a magnetic field whose magnetic field strength decreases as it moves away from the selection coil (static gradient magnetic field). The SPION particles in the device are in a magnetic saturation state under the strong magnetic field part of the static gradient magnetic field, and there is an area with zero magnetic flux density near the center, which is called the magnetic field free region (FFR). The magnetic field free region can generally be divided into two types according to its shape: magnetic field free point (FFP) and magnetic field free line (FFL). In this device, FFP is formed. Compared with FFL, FFP has a faster imaging speed and is more suitable for three-dimensional imaging in the scheme. The SPION particles in the FFP area are magnetically unsaturated and can also be affected by subsequent magnetic fields.

[0060] There are six driving coils, which are arranged in pairs in the X, Y and Z directions respectively; the two driving coils in the same direction are arranged in a Helmholtz configuration (i.e., symmetrically placed and with current flowing in the same direction) and have the same axis; the driving coils in the same direction are located on the inner side of the selection coils in the same direction; after the driving coils are connected to the current, a working magnetic field is generated that can be superimposed on the magnetic field of the selection coil. The intensity and direction of the working magnetic field can be changed by adjusting the current size and the direction of power flow, thereby realizing the movement of the FFP position in three-dimensional space, thereby realizing the scanning of the sample to be tested.

[0061] There are six excitation coils, two of which are arranged as a pair in the X, Y, and Z directions respectively; the two excitation coils in the same direction are arranged in a Helmholtz configuration with the same axis; the excitation coils in the same direction are located on the inner side of the driving coils in the same direction; after a high-frequency current is passed through the excitation coils near the FFP, a high-frequency time-varying excitation magnetic field can be generated, which acts on the magnetically unsaturated SPION particles in the FFP, pushing the magnetic particles to repeatedly magnetize into a saturated state, and generating a nonlinear response. This nonlinear response generates a signal component at the harmonic, and the magnetic particle signal to be detected comes from the harmonic component. Because the driving coil can realize the movement of the FFP position in three-dimensional space, the excitation coils must also be placed in three dimensions, so that a high-frequency time-varying excitation magnetic field can be generated in all three latitudes to act on the FFP.

[0062] There are six receiving coils, which are arranged in pairs in the X direction, Y direction and Z direction respectively; the two receiving coils in the same direction are symmetrically arranged in parallel and have the same axis; the receiving coils in the same direction are located on the inner side of the excitation coils in the same direction; the magnetic field free zone is in the enclosed space formed by the receiving coils; wherein the selection coils, driving coils, excitation coils and receiving coils in the same direction all have the same axis.

[0063] The function of the receiving coil is to receive the magnetic particle signals generated at the FFP position in three latitudes. It is closest to the central area and belongs to the innermost coil, which is convenient for detecting changes in magnetic flux density and realizing high-sensitivity detection of nonlinear response signals of magnetic nanoparticles. Since the received magnetic particle signals come from high-order harmonics, which are mixed with background interference noise in the detection process and fundamental interference signals of the excitation field, a high-pass filter must be connected after the receiving coil converts the magnetic particle signals into voltage signals.

[0064] The high-pass filter filters out interference signals of the magnetic particle signals generated at the FFP positions at three latitudes received by the receiving coil; the signal acquisition card is used to import the signals filtered by the high-pass filter into the computer for magnetic particle image reconstruction.

[0065] In summary, the device has a total of 4 layers of structure from the inside to the outside, namely: 6 receiving coils placed symmetrically in the XYZ direction, 6 excitation coils arranged symmetrically in the Helmholtz configuration in the XYZ direction, 6 driving coils arranged symmetrically in the Helmholtz configuration in the XYZ direction, and 2 selection coils arranged symmetrically in the Maxwell configuration in the Y direction. The selection coil is responsible for forming the FFP, the driving coil is responsible for controlling its position movement in three latitudes, and the excitation coil forms a high-frequency time-varying excitation magnetic field after being energized, which acts on the magnetically unsaturated SPION particles in the FFP. The receiving coil receives the magnetic particle signal and transmits it to the subsequent high-pass filter and other components, thereby realizing the three-dimensional spatial movement of the FFP area, and scanning the object to be tested more comprehensively and accurately. Another reason for adopting this solution is that the subsequent software method needs to realize the corresponding reconstructed MPI timing function video directly from the input one-dimensional signal sequence, which requires the accuracy and information amount of the collected signal. The realization of three-dimensional spatial movement, excitation and reception of the FFP area can provide more accurate information on the position, contour or time change of the object to be tested, and provides hardware support for the subsequent software solution.

[0066] It should be noted that the magnetic nanoparticle device for three-dimensional rapid and direct imaging of the nervous system provided in the above embodiment is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the modules or steps in the embodiments of the present invention can be decomposed or combined. For example, the modules in the above embodiment can be combined into one module, or further divided into multiple sub-modules to complete all or part of the functions described above. The names of the modules and steps involved in the embodiments of the present invention are only for distinguishing the modules or steps, and are not regarded as improper limitations of the present invention.

[0067] In order to solve the problem of being unable to effectively reconstruct MPI time-series video, a second embodiment of the present invention provides a magnetic nanoparticle method for three-dimensional rapid and direct imaging of the nervous system, based on the magnetic nanoparticle device for three-dimensional rapid and direct imaging of the nervous system; the method comprises:

[0068] S100, collecting one-dimensional MPI signals by using the constructed magnetic nanoparticle device for three-dimensional rapid and direct imaging of the nervous system to construct a one-dimensional signal sequence;

[0069] S200, the one-dimensional signals in the one-dimensional signal sequence are sequentially input into the generator in the pre-trained 3D-Multitask-fMPI-GAN generative adversarial network, and reconstructed through the reconstruction task branch of the generator to obtain a three-dimensional time-series grayscale motion video.

[0070] In the following, the structure and training process of the 3D-Multitask-fMPI-GAN generative adversarial network are first described, and then the process of obtaining three-dimensional temporal grayscale motion video through the magnetic nanoparticle method of three-dimensional fast and direct imaging of the nervous system is described.

[0071] The structure and training process of the 3D-Multitask-fMPI-GAN generative adversarial network are as follows:

[0072] Step 1: Prepare training set and test set data. First, use MATLAB to generate a time-series grayscale image sequence. For each time-series grayscale image sequence, create an empty time-series image sequence with the same dimension, fill the empty sequence with 0, and then traverse the original time-series grayscale image sequence frame by frame and pixel by pixel. If the pixel of the original time-series grayscale image sequence at the traversed position is not 0, then set the pixel at the corresponding position of the empty time-series image sequence to 1, so that the time-series binary image sequence corresponding to the time-series grayscale image sequence can be obtained. Next, confirm the device parameters and use the MPIRF simulation software to generate the corresponding one-dimensional signal sequence for all the original time-series grayscale image sequences. Add a time dimension to each frame in the generated time-series grayscale image sequence and time-series binary image sequence, and superimpose the two-dimensional images in the time dimension in their corresponding sequence order to generate a three-dimensional time-series grayscale motion video and a three-dimensional time-series binary motion video. Most of the obtained paired data is used as a training set to train the network, and a small part is used as a test set. Under the same parameters, the three-dimensional MPI device of the above hardware device is used to collect the signal sequence data of the pre-made phantom as another part of the test set.

[0073] Step 2: Build the network. The 3D-Multitask-fMPI-GAN generative adversarial network includes a multi-task generator, a 3D (temporal) discriminator, a 2D (spatial) discriminator, and a loss function. Figure 3As shown, the 3D discriminator and the 2D discriminator are Figure 4 As shown, fMPI is functional magnetic particle imaging.

[0074] The multi-task generator has two tasks: reconstruction and segmentation. The reconstruction task is the main task, which is responsible for generating three-dimensional time-series grayscale motion video (i.e. Figure 3 The segmentation task is an auxiliary task responsible for generating a three-dimensional temporal approximate binary motion video (i.e. Figure 3 temporal approximation of binary video in ).

[0075] The multi-task generator can be divided into two parts: the task common part and the task branch part.

[0076] The task-sharing part mainly includes linear layers, bidirectional two-layer GRU networks (gated recurrent networks), M upsampling blocks formed by cross-stacking of VideoSwin transformer Block (video shift window transformation block) and 3D Time-patch expanding layers (three-dimensional time-patch expansion blocks or three-dimensional time-patch expansion layers), Swintransformer, that is, shift window transformation.

[0077] There are two branches in the task branch part: reconstruction task branch and segmentation task branch. The internal structures of the two branches are basically the same, both mainly including N Video Swin transformer blocks (N is an even number) and 3D convolution layers, but there is an activation layer at the end of the segmentation task branch. The details are as follows:

[0078] The linear layer is used to perform linear transformation on each one-dimensional signal in the input one-dimensional signal sequence to meet the input feature dimension required by the subsequent network.

[0079] GRU network is a special RNN network. More specifically, compared with another special RNN network, LSTM network, GRU network merges the input gate and forget gate of LSTM into an update gate (determines the extent to which the state information of the previous time step is brought into the state information of the current time step), removes the output gate, adds a reset gate (determines the extent to which the state information of the previous time step is ignored), and merges the units and hidden states of the intermediate steps, so that the final GRU model is simpler (containing only two gates) than the typical LSTM model, learns faster, and requires less training data. In the present invention, because the reconstruction task is to directly reconstruct a three-dimensional time-series motion video through a one-dimensional signal (without generating a two-dimensional image transition in the middle), higher requirements are placed on the learning of time-dependent information between signals, so a two-layer network is used to deepen the hidden layer and strengthen the model's learning and memory ability for time-dependent information. Compared with the ordinary unidirectional GRU network, the bidirectional GRU learns the forward and backward information of the sequence at the same time, which can capture more dependencies between sequences and improve the performance of the model. The length of the overall MPI signal sequence to be processed is dozens of frames, which is not a long sequence. If too many layers of recurrent networks are used, overfitting may occur. In summary, a bidirectional two-layer GRU network is selected and initialized with a normal distribution.

[0080] The function of the bidirectional double-layer GRU network is to internally cycle the length of the sequence to be processed (T) times in both directions. The forward network polls the input one-dimensional signal sequence in forward order, and the reverse network polls in reverse order to learn the temporal information relationship therein. That is, the input of the bidirectional double-layer GRU network: one is the signal sequence of all time steps after the linear layer processing, and the other is the initialization state; the initialization state is a standard normal distribution; the output of the bidirectional double-layer GRU network: only the output sequence processed by the bidirectional double-layer GRU network is used (because the calculation of the bidirectional double-layer GRU network at each time step is no longer performed explicitly, but implicitly inside the function, so its output is directly the processed sequence, but in fact it has another output called "hidden state" in the code, but it is not used in the present invention, so it is not used, and only the processed output sequence is used), and output to the first upsampling block, and the inputs of the remaining upsampling blocks are all the outputs of the previous sampling block;

[0081] The upsampling block includes 3D Position Embedding, Video Swintransformer Block, and 3D Time-patch expanding; the function of the upsampling block is to convert a one-dimensional signal sequence into a corresponding three-dimensional time-series motion video.

[0082] 3D Position Embedding: For the tensor with the dimension of Batch_size×Channel×Time×Height×Width input to the upsampling block, use the code to initialize a parameter matrix that matches the dimension of the input tensor according to the input tensor of the upsampling block, and directly add it to the input tensor, and use the added tensor as the first tensor; this is equivalent to assigning an independent absolute position code to each spatiotemporal position, and these codes will be updated during the training process to learn the spatiotemporal related information of the input data. 3DPosition Embedding is a learnable parameterized spatiotemporal absolute position code, which, in conjunction with the relative position code in the subsequent Video Swin transformer Block, can show good spatiotemporal flexibility when processing videos.

[0083] Video Swin transformer Block: Calculate the importance weight of the first tensor through the self-attention mechanism, weight the first tensor based on the importance weight to obtain the second tensor; the self-attention mechanism includes 3D window self-attention and 3D sliding window self-attention. 3D Time-patchexpanding: Double the number of channels of the second tensor through a linear layer. After doubling, rearrange the tensors so that the number of channels of the tensor becomes half of the number of channels of the output tensor of the Video Swin transformer Block, and double the width and height of each frame of the image;

[0084] That is, the tensor processed by 3D Position Embedding continues to enter the Video Swin transformer Block, and learns the texture and distribution features of each frame in the video and the time dependency between different frames in the video through mechanisms such as 3D window self-attention (3D W-MSA) and 3D sliding window self-attention (3D SW-MSA) in the Video Swin transformer Block, and gradually increases the resolution of each frame image in the video and reduces the number of channels of each frame image through the 3D Time-patch expanding operation (for example, the tensor dimension before input is Batch_size×Channel×Time×8×8, and after upsampling it becomes Batch_size×Channel / 2×Time×16×16, where Batch_size represents the number of samples during training, Channel represents the number of channels, and Time represents the number of video frames), and then inputs it to the next level upsampling block (a total of M upsampling blocks, M cycles). Specifically, 3D Time-patch expanding doubles the number of channels of the video tensor through a linear layer without changing the number of video frames (Time), and then rearranges the tensor to make the number of channels half of the number of channels of the tensor output by the Video Swintransformer Block of the same level, and at the same time doubles the width and height of each frame in the video, so that upsampling can be achieved. The reason for using 3D Time-patchexpanding is that traditional upsampling methods such as transposed convolution are prone to uneven overlap during the upsampling process, resulting in a "chessboard effect" in the image after upsampling, reducing the upsampling quality. However, 3D Time-patch expanding only has linear transformation and pixel rearrangement, which includes all the required information in the channel dimension, does not contain transposed convolution operations, and can effectively avoid the "chessboard effect" and improve the upsampling quality.

[0085] The reconstruction task branch is used to process the output of the last upsampling block through N (N is an even number) VideoSwin transformer Blocks and 3D convolution layers in sequence to obtain a three-dimensional time-series grayscale motion video; the role of the Video Swin transformer Block is the same as that in the above upsampling block and will not be repeated here. The role of the 3D convolution layer is to adjust the number of channels of each frame image in the video without changing the number of frames and the resolution of each frame by reasonably setting the convolution kernel parameters, and finally obtain a reconstructed three-dimensional time-series motion video, and the video meets the required number of frames, resolution of each frame and number of channels.

[0086] The segmentation task branch is used to process the output of the last upsampling block through N (N is an even number) VideoSwin transformer blocks, 3D convolution layers, and activation layers in sequence to obtain a three-dimensional temporal approximate binary motion video. Compared with the reconstruction task branch, the segmentation task branch has an activation layer. Since the function of the segmentation task branch is to segment the foreground and background of each frame of the reconstructed three-dimensional temporal motion video to generate a three-dimensional temporal approximate binary motion video. Therefore, the Sigmoid function should be set in the activation layer to achieve that the pixels in each frame of the reconstructed three-dimensional temporal approximate binary motion video are mapped to the 0-1 interval and as close to 0 or 1 as possible for subsequent calculations.

[0087] The 3D discriminator uses the video swin transformer as the basic framework. After the basic calculation modules of 3D W-MSA and 3D SW-MSA are included in the framework, the time-image fusion cross-attention (Time-patch-Cross-attention) module is added. The Time-patch-Cross-attention module is used to calculate the dependencies between different time dimensions (i.e., different frames of 3D temporal motion video) and the spatial semantic information within the same frame, and weightedly combine the time information and spatial information to help update the model parameters. The Time-patch-Cross-attention module is mainly composed of the following subcomponents: multi-head self-attention, time cross-attention branch, spatial patch attention branch, and weighted fusion and output mapping. Multi-head self-attention implements the standard multi-head self-attention mechanism to complete the interaction and fusion of information. The time cross-attention branch is for the input video tensor V (Batch_size×Channel×Time×Height×Width), and the dimension transformation is completed through 3D convolution and tensor reordering operations internally, and then V is sent to the multi-head self-attention to capture the global dependency between different frames. After completing the self-attention calculation, it is restored back to a tensor of (Batch_size×Hidden_dim×Time×Height×Width) to align with the output of the subsequent spatial patch attention branch. The spatial patch attention branch first performs frame-by-frame patch division and embedding on the video tensor V(Batch_size×Channel×Time×Height×Width), and then completes the dimension transformation through serialization processing (the tensor dimension arrangement is now (Batch_size×Time,N_patches,Embed_dim)). Then it is sent to the multi-head self-attention to capture the spatial information of the same frame or across frames. After completing the self-attention calculation, the output needs to be split back to the original spatial layout (by indexing the patch back to the corresponding area of ​​each frame), so as to reconstruct the feature tensor of (Batch_size×Hidden_dim×Time×Height×Width) shape. Weighted fusion and output mapping For the feature representations obtained from the temporal span attention branch and the spatial patch attention branch, two sets of learnable parameters α and β are used for weighted summation to achieve fusion. Finally, a 3D convolution is used to map the fused features from hidden_dim back to the original number of channels Channel, and the output shape is the same as the input, i.e. (Batch_size×Channel×Time×Height×Width). The function of the 3D discriminator is to judge the authenticity of the 3D time-series grayscale motion video generated by the multi-task generator and the 3D time-series grayscale motion video in the training set.

[0088] As mentioned above, the reconstruction task places high demands on the learning of time-dependent information. Therefore, before using the 2D discriminator, firstly, in units of frames, all frames of the three-dimensional time-series grayscale motion video generated by the multi-task generator and the three-dimensional time-series grayscale motion video in the training set are sampled, and all frames are extracted to obtain T-frame two-dimensional images. The T-frame two-dimensional images are input into the 2D discriminator, and the 2D discriminator is applied to each frame independently to obtain T recognition results and calculate the average. If some frames in the three-dimensional time-series grayscale motion video generated by the multi-task generator are of low quality or differ too much from the time motion information of the previous and next frames, the discriminant operation on each frame can identify the indexes of these frames and impose penalties and constraints, so as to promote the generator to learn better. As mentioned above, the reconstruction task places high demands on the learning of time-dependent information. Therefore, before using the 2D discriminator, firstly, in units of frames, all frames of the three-dimensional time-series grayscale motion video generated by the multi-task generator and the three-dimensional time-series grayscale motion video in the training set are sampled, and all frames are extracted to obtain T-frame two-dimensional images. Input T frames of 2D images into the 2D discriminator, apply the 2D discriminator to each frame independently, obtain T recognition results and calculate the average. If some frames in the 3D time-series grayscale motion video generated by the multi-task generator are of low quality or have a large difference in temporal motion information from the previous and next frames, performing a discriminative operation on each frame can identify the indexes of these frames and impose penalties and constraints, which can help the generator learn better.

[0089] To stabilize the training process, the loss function is adjusted based on Hinge GAN Loss. To conform to the form of Hinge GANLoss, the two discriminators are finally set to the Tanh activation function so that the output judgment probability range is (-1,1). The average judgment results of all two discriminators for the generated 3D temporal grayscale motion video (Part A), the mean absolute error (MAE) in the frame dimension between the 3D temporal grayscale motion video generated by the multi-task generator and the real 3D temporal grayscale motion video in the training set (Part B), the average BCE error in the frame dimension between the 3D temporal approximate binary motion video generated by the multi-task generator and the real 3D temporal binary motion video in the training set (Part C), and the average Dice error in the frame dimension between the 3D temporal approximate binary motion video generated by the multi-task generator and the real 3D temporal binary motion video in the training set (Part D) are used as the loss function of the multi-task generator. Among them, the average frame errors of parts B, C, and D are also multiplied by a pre-set hyperparameter , , As a penalty term. Because the average frame error of parts B, C, and D is smaller than the value of the discriminator's judgment result, only by multiplying the penalty term can the learning process and generation results of the multi-task generator be effectively constrained.

[0090] The loss function of the multi-task generator of the 3D-Multitask-fMPI-GAN generative adversarial network during training is:

[0091]

[0092] in, represents the loss of the multi-task generator, represents the real image, Indicates signal, represents the one-dimensional signal sequence of the input, It represents the 2D image of the i-th frame of the i-th signal in the input one-dimensional MPI signal sequence, obtained by sampling all frames of the three-dimensional time-series grayscale motion video generated by the multi-task generator. It represents the T-frame two-dimensional image obtained after all frames of the three-dimensional temporal approximate binary motion video generated by the multi-task generator are sampled, corresponding to the i-th frame two-dimensional image of the i-th signal in the input one-dimensional MPI signal sequence, represents the three-dimensional temporal grayscale motion video generated by the multi-task generator, It represents the 2D image of the i-th frame of the i-th signal in the input one-dimensional MPI signal sequence, obtained by sampling all frames of the three-dimensional time-series grayscale motion video in the training set. It represents the 2D image of the i-th frame of the i-th signal in the input one-dimensional MPI signal sequence, obtained by sampling all frames of the three-dimensional time-series binary motion video in the training set. represents the mean absolute error function, represents the BCE function, represents the Dice function, , , They represent the penalty items corresponding to the mean absolute error function, BCE function, and Dice function, respectively. , Respectively represent 2D discriminator and 3D discriminator, represents mathematical expectation.

[0093] The loss functions of the 3D discriminator and 2D discriminator are:

[0094]

[0095]

[0096] in, , Respectively represent the loss of 2D discriminator and the loss of 3D discriminator, Represents the 3D temporal grayscale motion video in the training set.

[0097] Step 3: Train the network: Use the one-dimensional signal sequence generated by the simulation software as the input of the multi-task generator to obtain the three-dimensional time-series grayscale motion video output by the network. Use it and the three-dimensional time-series grayscale motion video in the training set as the input of the 3D discriminator. Sampling all frames of the three-dimensional time-series grayscale motion videos from two different sources to obtain T-frame two-dimensional images, and polling the single-frame image obtained by the T-frame two-dimensional image as the input of the 2D discriminator. Combine the discriminator recognition results (Part A) and the remaining three parts (Parts B, C, and D) to construct a loss function for iterative training. When the iteration requirements are met, the network training ends. Among them, the 3D-Multitask-fMPI-GAN network is constructed using the Pytorch2.5.1, cuda 12.1 framework, and trained using 4 NVIDIA GeForce RTX 4090s. The training process is optimized using the Adam algorithm. The learning rate of the 3D-Multitask-fMPI-GAN network generator is set to 0.00001, and the learning rates of the 3D discriminator and 2D discriminator are both set to 0.000001. It is planned to iterate for 100 epochs.

[0098] Step 4: Test the network: During the test phase, only the reconstruction task branch of the multi-task generator part of the network is used. First, the one-dimensional signal sequence in the paired data generated by the simulation software in the test set is input into the generator of the network, and the obtained three-dimensional time-series grayscale motion video is compared with the corresponding three-dimensional time-series grayscale motion video in the test set. The performance of the model on the test set is measured by three indicators: frame average root mean square error (RMSE), frame average peak signal-to-noise ratio (PSNR), and frame average structural similarity index (SSIM). After the test passes, the signal sequence data of the pre-made phantom of the three-dimensional MPI device in another part of the test set is input into the generator of the network, and the reconstructed MPI time-series function video is output to observe the generation effect.

[0099] That is, in actual use, the process of reconstruction of the three-dimensional rapid and direct imaging of the nervous system by magnetic nanoparticles is as follows:

[0100] S100, collecting one-dimensional MPI signals by using the constructed magnetic nanoparticle device for three-dimensional rapid and direct imaging of the nervous system to construct a one-dimensional signal sequence;

[0101] S200, the one-dimensional signals in the one-dimensional signal sequence are sequentially input into the generator in the pre-trained 3D-Multitask-fMPI-GAN generative adversarial network, and reconstructed through the reconstruction task branch of the generator to obtain a three-dimensional time-series grayscale motion video.

[0102] In summary, traditional system matrix-based image reconstruction is usually achieved by solving a linear equation with a system function, which maps the measured signal spectrum to the spatial concentration. This function includes the nonlinear magnetic response of the particle and the measurement conditions. Due to the ill-conditioned nature of the inverse problem, the calculation is large, the process is complex and time-consuming when solving it. In practice, the X-space method cannot operate in an ideal adiabatic environment. Using the Langevin model to describe the magnetization process of magnetic particles under the adiabatic assumption will ignore the relaxation effect of the particles, resulting in blurry and inaccurate x-space reconstructed images. And if time series imaging is performed, as the experimental time increases, the relaxation effect of the particles will become more obvious, and the reconstructed image will become more blurred. Therefore, the above two methods are limited by the amount of calculation and the calculation method, and are not suitable for reconstructing MPI time series functional videos. In traditional fMRI methods, the reason why three-dimensional time series fMRI videos are not generated directly from BOLD sequences is, on the one hand, because the imaging time of a single element in the sequence is long, and on the other hand, because there is a lack of end-to-end processing signal sequence to video method. The present invention adopts an end-to-end processing method, does not need to generate a two-dimensional image transition, integrates spatiotemporal feature information in the generation process, reduces the amount of calculation and can improve the generation quality of the MPI timing function video.

[0103] In order to further understand the magnetic nanoparticle method for three-dimensional rapid and direct imaging of the nervous system of the present invention, a specific example is given below:

[0104] 1. Build MPI equipment

[0105] Purchase and manufacture parts and connect them according to the above MPI 3D hardware device diagram to build a 3D MPI device. Connect the built hardware device part to the computer host;

[0106] 2. Prepare training set and test set data

[0107] First, MATLAB is used to generate 10,000 time-series grayscale image sequences of three different models, namely, geometric model, letter model and resolution model. After obtaining the time-series grayscale image sequences, for each time-series grayscale image sequence, an empty time-series image sequence with the same dimension is created, and the empty sequence is filled with 0. Then, the original time-series grayscale image sequence is traversed frame by frame and pixel by pixel. If the pixel of the original time-series grayscale image sequence at the traversed position is not 0, the pixel at the corresponding position of the empty time-series image sequence is set to 1, so that the time-series binary image sequence corresponding to the time-series grayscale image sequence can be obtained. The MPIRF simulation software is used to generate the corresponding one-dimensional signal sequence for all the original time-series grayscale image sequences. A time dimension is added to each frame in the generated time-series grayscale image sequence and time-series binary image sequence, and the two-dimensional images are superimposed in the time dimension in their corresponding sequence order to generate 10,000 three-dimensional time-series grayscale motion videos and three-dimensional time-series binary motion videos. 8000 pairs of the 10000 paired data (including one-dimensional signal sequence, three-dimensional time-series grayscale motion video and three-dimensional time-series binary motion video) were used as training data, and the remaining 2000 pairs were used as test data sets. At the same time, the magnetic nanoparticle reagent was injected into different phantom models that were prepared in advance, such as Figure 5 , 6 As shown, 100 sets of signal sequence data of phantom models are actually collected using a three-dimensional MPI device as another part of the test set;

[0108] 3. Training 3D-Multitask-fMPI-GAN to generate adversarial models

[0109] The network architecture in the above software method is used to build a 3D-Multitask-fMPI-GAN generative adversarial network. In the multi-task generator, the one-dimensional signal sequence in the training set is input as a condition to obtain the three-dimensional temporal grayscale motion video and the three-dimensional temporal approximate binary motion video output by the multi-task generator. The input of the 3D discriminator is the three-dimensional temporal grayscale motion video output by the generator and the real three-dimensional temporal grayscale motion video paired with the one-dimensional signal sequence in the training set, and the output is the judgment of whether the input three-dimensional temporal grayscale motion video is a real video. In units of frames, the two-dimensional image sequences from two different sources are obtained by sampling all frames of the three-dimensional temporal grayscale motion video generated by the multi-task generator and the three-dimensional temporal grayscale motion video in the training set. The single frame image in the two-dimensional image sequence is polled as the input of the 2D discriminator, and the output is the judgment of whether the input image is a real image. The loss function is constructed by combining the discriminator recognition result (part A) and the other three parts (parts B, C, and D) for iterative training. The generator and discriminator improve the quality of generated videos and single-frame images during training. Use PSNR, SSIM, and RMSE values ​​as evaluation indicators to determine the average difference between the corresponding frames of the 3D time-series grayscale motion video in the training set and the generated 3D time-series grayscale motion video, and select the model with the smallest RMSE value and the highest SSIM value on the training set as the best model, which is used as the test model to accept the test set data.

[0110] 4. Testing and deploying the 3D-Multitask-fMPI-GAN generation model

[0111] In the test phase, only the reconstruction task branch of the multi-task generator part of the network is used. The simulated one-dimensional signal sequence in the test set is used as the input of the generator, and the reconstructed three-dimensional time-series motion video is output. In the paired data obtained by the simulation software in the test set, evaluation indicators such as PSNR, SSIM, and RMSE are used to evaluate the quality of the generated three-dimensional time-series grayscale motion video and the gap between the real three-dimensional time-series grayscale motion video. The specific calculation method is to traverse and calculate the PSNR, SSIM, and RMSE values ​​of each frame of the generated three-dimensional time-series grayscale motion video and the corresponding three-dimensional time-series grayscale motion video in the test set and calculate the average value. After the completion of this step of the test, the signal sequence data of the pre-made phantom of another part of the three-dimensional MPI device in the test set is input into the generation model to observe the quality of the obtained MPI time-series functional video. After the quality is accepted, the trained generation model is deployed in the three-dimensional MPI device built in the first step for subsequent use.

[0112] 5. Animal experiments based on macaques to perform time-series video imaging of their nervous system activities

[0113] About ten macaques with basically the same growth, health and other characteristics were selected as experimental subjects. All of them were injected with ketamine for anesthesia to facilitate subsequent experiments. After the experimental monkeys were stable, the heads of the experimental monkeys were fixed with a stereotaxic bracket of moderate hardness and softness to ensure that their heads did not shift during the experiment. All experimental monkeys were intramuscularly injected with the same volume and concentration of magnetic particle solution. After the injection, a small amount of physiological saline was injected to flush the tube to ensure that all magnetic nanoparticles entered the body and participated in the blood circulation. The anesthetized experimental monkeys were placed in the MPI scanning space for time-series MPI signal acquisition. No tasks were performed during the entire acquisition process and no external stimulation was applied to the experimental monkeys. Five rounds of resting time series signals were collected for each experimental monkey, and the sequence signals of different rounds of each experimental monkey were saved separately. The sequence signals collected in the same round of each experimental monkey were taken out respectively and input into the multi-task generator (only using the reconstruction task branch) of the 3D-Multitask-fMPI-GAN generative adversarial network deployed in the fourth step to obtain the corresponding MPI three-dimensional time-series brain function video of each experimental monkey. The above process was repeated for 5 rounds to obtain the brain function video results of ten experimental monkeys in 5 rounds. The obtained brain function video was preprocessed, including spatial normalization, smoothing, filtering, etc.

[0114] Since MPI only images magnetic nanoparticles SPIONs and not brain tissue structures, the same experimental monkey will also be imaged using fMRI equipment to collect high-definition T1 structural images of the monkey. Then, each frame of the MPI image in the MPI time-series functional video will be registered to the corresponding frame of the T1 structural image through the registration method, and the pre-processed brain function video will be registered frame by frame.

[0115] At the same round of latitude, the brain function video results of different experimental monkeys after registration were compared to observe the changes of brain function videos with individual changes. At the same experimental monkey individual latitude, the changes of brain function videos of the same experimental monkey individual after registration over time were compared with the changes of rounds.

[0116] This can verify: 1) The ability of the three-dimensional MPI device to collect brain function signals and provide hardware support for subsequent software solutions; 2) The ability of the 3D-Multitask-fMPI-GAN generative adversarial network to directly and quickly generate MPI time-series functional videos from one-dimensional signal sequences without the need to generate two-dimensional image transitions.

[0117] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process and related instructions of the method described above can refer to the corresponding process in the aforementioned device example, and will not be repeated here.

[0118] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the device, method and computer program product according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based device that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0119] The terms "first", "second", etc. are used to distinguish similar objects rather than to describe or indicate a particular order or sequence.

[0120] The term "comprise" or any other similar term is intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus / device that includes a list of elements includes not only those elements but also other elements not expressly listed, or also includes elements inherent to such process, method, article, or apparatus / device.

[0121] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

Claims

1. A magnetic nanoparticle method for three-dimensional rapid and direct imaging of the nervous system, based on a magnetic nanoparticle device for three-dimensional rapid and direct imaging of the nervous system, characterized in that: The device has a total of 4 layers of structure from the inside to the outside, namely: 6 receiving coils placed symmetrically in the XYZ direction, 6 excitation coils arranged symmetrically in the Helmholtz configuration in the XYZ direction, 6 driving coils arranged symmetrically in the Helmholtz configuration in the XYZ direction, and 2 selection coils arranged symmetrically in the Maxwell configuration in the Y direction. The selection coil is responsible for forming the FFP, and the driving coil is responsible for controlling its position movement in three latitudes. When the excitation coil is energized, a high-frequency time-varying excitation magnetic field is formed to act on the magnetically unsaturated SPION particles in the FFP. The receiving coil receives the magnetic particle signal and transmits it to the subsequent high-pass filter and other components. The method includes: S100, collecting one-dimensional MPI signals by using the constructed magnetic nanoparticle device for three-dimensional rapid and direct imaging of the nervous system to construct a one-dimensional signal sequence; S200, inputting the one-dimensional signals in the one-dimensional signal sequence into the generator in the pre-trained 3D-Multitask-fMPI-GAN generative adversarial network in sequence, and reconstructing through the reconstruction task branch of the generator to obtain a three-dimensional time-series grayscale motion video; The 3D-Multitask-fMPI-GAN generative adversarial network includes a multi-task generator, a 3D discriminator, and a 2D discriminator; the multi-task generator includes a reconstruction task branch and a segmentation task branch; the reconstruction task branch is used to generate a one-dimensional signal sequence input into the multi-task generator into a three-dimensional temporal grayscale motion video; the segmentation task branch is used to generate a one-dimensional signal sequence input into the multi-task generator into a three-dimensional temporal approximate binary motion video; the 3D discriminator is used to combine the generated three-dimensional temporal grayscale motion video with the real three-dimensional temporal grayscale motion video in the corresponding data set to determine the authenticity of the three-dimensional temporal grayscale motion video; the 2D discriminator is used to determine the authenticity of the two-dimensional image based on the two-dimensional image obtained after frame sampling of the generated three-dimensional temporal grayscale motion video and the real three-dimensional temporal grayscale motion video in the corresponding data set.

2. The magnetic nanoparticle method for three-dimensional rapid and direct imaging of the nervous system according to claim 1, characterized in that: The multi-task generator includes a front linear layer, a bidirectional two-layer GRU network, M sequentially connected upsampling blocks, a reconstruction task branch, and a segmentation task branch; The linear layer is used to perform linear transformation processing on each one-dimensional signal in the input one-dimensional signal sequence; The input of the bidirectional double-layer GRU network: one is the signal sequence of all time steps after the linear layer processing, and the other is the initialization state; the initialization state is a standard normal distribution; the output of the bidirectional double-layer GRU network is the output sequence processed by the bidirectional double-layer GRU network, excluding the final hidden state, and the output sequence is output to the first upsampling block; The upsampling block includes 3D Position Embedding, Video Swin transformer Block, and 3D Time-patch expanding; The 3D Position Embedding is used to initialize a parameter matrix matching the dimension of the input tensor of the upsampling block as position encoding information embedding, and directly add it to the input tensor, and use the added tensor as the first tensor; The Video Swin transformer Block is used to calculate the importance weight of the first tensor through a self-attention mechanism, and weight the first tensor based on the importance weight to obtain a second tensor; The 3D Time-patch expanding is used to double the number of channels of the second tensor through an internal linear layer without changing the number of video frames, and after doubling, rearrange the tensors so that the number of channels of the tensor becomes half of the number of channels of the output tensor of the Video Swin transformer Block, and at the same time double the width and height of each frame of the image; There are a total of M upsampling blocks in the network, so the above upsampling block processing process is cycled M times in total until the output of the last upsampling block meets the image height, width, and channel number requirements to be received by the subsequent multi-task branch; wherein, starting from the second upsampling block, the input of each sampling block is the output of the previous sampling block; The reconstruction task branch is used to process the output of the last upsampling block through N Video Swintransformer Blocks and 3D convolution layers in sequence to obtain a three-dimensional time-series grayscale motion video; N is an even number; The segmentation task branch is used to process the output of the last upsampling block through N Video Swintransformer Blocks, 3D convolution layers, and activation layers in sequence to obtain a three-dimensional temporal approximate binary motion video.

3. The magnetic nanoparticle method for three-dimensional rapid and direct imaging of the nervous system according to claim 2, characterized in that: The self-attention mechanism includes 3D window self-attention and 3D sliding window self-attention.

4. The magnetic nanoparticle method for three-dimensional rapid and direct imaging of the nervous system according to claim 1, characterized in that: The 3D discriminator adopts the video swin transformer as the basic framework, and adds a time-image fusion cross attention module after the basic calculation modules of the video swin transformer including 3D W-MSA and 3D SW-MSA; The time-image fusion cross-attention module is used to calculate the dependency between different time dimensions and the spatial semantic information within the same frame, and weightedly combine the time information and spatial information to update the model parameters; the dependency between the different time dimensions is the dependency between different frames of the three-dimensional time-series grayscale motion video.

5. The magnetic nanoparticle method for three-dimensional rapid and direct imaging of the nervous system according to claim 1, characterized in that: The 2D discriminator is built based on the swin transformer.

6. The magnetic nanoparticle method for three-dimensional rapid and direct imaging of the nervous system according to claim 1, characterized in that: The loss function of the multi-task generator of the 3D-Multitask-fMPI-GAN generative adversarial network during training is: ； in, represents the loss of the multi-task generator, represents the real image, Indicates signal, represents the one-dimensional signal sequence of the input, It represents the 2D image of the i-th frame of the i-th signal in the input one-dimensional MPI signal sequence, obtained by sampling all frames of the three-dimensional time-series grayscale motion video generated by the multi-task generator. It represents the T-frame two-dimensional image obtained after all frames of the three-dimensional temporal approximate binary motion video generated by the multi-task generator are sampled, corresponding to the i-th frame two-dimensional image of the i-th signal in the input one-dimensional MPI signal sequence, represents the three-dimensional temporal grayscale motion video generated by the multi-task generator, It represents the 2D image of the i-th frame of the i-th signal in the input one-dimensional MPI signal sequence, obtained by sampling all frames of the three-dimensional time-series grayscale motion video in the training set. It represents the 2D image of the i-th frame of the i-th signal in the input one-dimensional MPI signal sequence, obtained by sampling all frames of the three-dimensional time-series binary motion video in the training set. represents the mean absolute error function, represents the BCE function, represents the Dice function, , , They represent the penalty items corresponding to the mean absolute error function, BCE function, and Dice function, respectively. , Respectively represent 2D discriminator and 3D discriminator, represents mathematical expectation.

7. The magnetic nanoparticle method for three-dimensional rapid and direct imaging of the nervous system according to claim 6, characterized in that: The loss functions of the 3D discriminator and 2D discriminator of the 3D-Multitask-fMPI-GAN generative adversarial network during training are: ； ； in, , Respectively represent the loss of 2D discriminator and the loss of 3D discriminator, Represents the 3D temporal grayscale motion video in the training set.

8. A magnetic nanoparticle device for three-dimensional rapid and direct imaging of the nervous system, based on a magnetic nanoparticle method for three-dimensional rapid and direct imaging of the nervous system according to any one of claims 1 to 7, characterized in that: The device includes: a signal generator, a power amplifier, a selection coil, a driving coil, an excitation coil, a receiving coil, a high-pass filter, and a signal acquisition card; The signal generator comprises a signal generator 1 and a signal generator 2; the signal generator 1 outputs a voltage signal and is connected to the selection coil and the driving coil after being amplified by the power amplifier; the signal generator 2 outputs a voltage signal and is connected to the excitation coil; There are two selection coils, which are arranged as a pair in the Y direction; the two selection coils are arranged in a Maxwell configuration and have the same axis; There are six drive coils, which are arranged in pairs in the X direction, the Y direction, and the Z direction respectively; the two drive coils in the same direction are arranged in a Helmholtz configuration and have the same axis; the drive coils in the same direction are located inside the selection coils in the same direction; There are six excitation coils, which are arranged in pairs in the X direction, the Y direction, and the Z direction respectively; the two excitation coils in the same direction are arranged in a Helmholtz configuration and have the same axis; the excitation coils in the same direction are located inside the driving coils in the same direction; There are six receiving coils, which are arranged in pairs in the X direction, the Y direction and the Z direction respectively; the two receiving coils in the same direction are arranged symmetrically and in parallel with the same axis; the receiving coils in the same direction are located inside the exciting coils in the same direction; the magnetic field free zone is in the enclosed space formed by the receiving coils; The high-pass filter filters out interference signals of the magnetic particle signals generated at the FFP positions at three latitudes received by the receiving coil; the signal acquisition card is used to import the signals filtered by the high-pass filter into a computer for magnetic particle image reconstruction.

9. The magnetic nanoparticle device for three-dimensional rapid and direct imaging of the nervous system according to claim 8, characterized in that: The selection coil, the drive coil, the excitation coil, and the receiving coil are all annular coils, and the selection coil, the drive coil, the excitation coil, and the receiving coil in the same direction have the same axis.

Images