Time-sequential magnetic nanoparticle imaging device and method for detecting neural response activity

Abstract

The present invention belongs to the field of magnetic particle imaging technology, and specifically relates to a time-series magnetic nanoparticle imaging device and method for detecting neural response activities, aiming to solve the problems of low sensitivity of magnetic particle signal detection, large coil group, large amount of calculation and complex calculation method required to achieve high magnetic field strength. The device of the present invention: the signal generator outputs a voltage signal and amplifies it to connect the selection coil and the driving coil; there are two selection coils, which are arranged in Maxwell configuration and have the same axis; the driving coil is centered on FFR, and a driving coil is placed every 90 degrees up, down, left, right, front and back; the excitation coil and the receiving coil are coaxial with the selection coil; the high-pass filter filters the signal received by the receiving coil and imports it into the computer through the signal acquisition card to reconstruct the time-series fMPI image sequence. The present invention improves the sensitivity of magnetic particle signal detection, reduces the volume of the coil group, simplifies the calculation complexity and amount, and improves the robustness of magnetic particle imaging.

Description

Technical Field

[0001] The present invention belongs to the technical field of magnetic particle imaging, and in particular relates to a time-series magnetic nanoparticle imaging device and method for detecting neural response activities. Background Art

[0002] Since its invention in 1991, BOLD-fMRI imaging technology applied to humans has been widely used in cognitive tasks such as vision and hearing. In the United States, more than 90% of psychology research topics are scanned with bold-fMRI. In addition to humans, fMRI technology has also been used to study various cognitive activities of awake monkeys. With the development of technology, researchers have found that when superparamagnetic iron oxide nanoparticles (SPION) are injected into monkeys before scanning, the functional signal sensitivity measured is 5 times that of the Bold signal, and the detected brain function activation and brain connection strength are greatly increased (that is, injection of SPION can significantly improve the sensitivity of primate fMRI imaging). Therefore, injection of SPION is currently a routine step in macaque fMRI imaging. However, the bold-fMRI method uses deoxyhemoglobin as a natural contrast agent, and the observed hemodynamic changes caused by stimulation are weak. Studies usually require the average of a large number of subjects to achieve statistical significance. And its sensitivity to brain neural activity detection is very low. Studies have shown that BOLD-fMRI can only capture less than 2% of changes in brain neural activity. Although the SPION-fMRI method has improved sensitivity compared to the BOLD-fMRI method, in essence, both SPION-fMRI and BOLD-fMRI use MRI to indirectly image the changes in the content of contrast agents in blood vessels. Therefore, the improvement of the sensitivity of SPION-fMRI in brain function detection is very limited.

[0003] Magnetic nanoparticle imaging (MPI) is a new imaging method used to reconstruct the concentration distribution of superparamagnetic nanoparticles in the object to be tested. MPI imaging requires the use of tracers, and signals can only be generated when the tracer exists in the imaging area. Superparamagnetic iron oxide nanoparticles (SPION) are generally used as tracers. The MPI device has no background signal interference from the human body itself, which makes the MPI image have excellent contrast and high sensitivity, and the signal intensity is proportional to the tracer concentration. It is an inspection method that can obtain quantitative data. These characteristics make it an ideal paradigm for applications such as stem cell tracking, angiography, and targeted drug delivery.

[0004] fMPI is a natural way to image cerebral blood volume (CBV), which detects changes in the CBV of the test subject through fast MPI time-series imaging. Because changes in CBV caused by brain activity cause changes in the distribution of SPIONs, when the external alternating excitation magnetic field in the MPI device acts on the SPIONs, the SPIONs will generate continuous magnetic response signals of a specific frequency, which can be directly detected by the MPI device. And fMPI has a natural sensitivity to hemodynamics and blood volume changes related to brain activation, and can produce highly sensitive continuous magnetic response signals.

[0005] Therefore, it is necessary to study the feasibility of fMPI as a substitute for fMRI to detect neural response activity. In recent years, neural response activity has received particular attention. By detecting neural response activity, completing fMPI time series imaging is an important prerequisite for evaluating whether fMPI can be used as a potential clinical diagnostic technology and tool.

[0006] However, the existing magnetic particle imaging equipment coils are made of copper, which has a general conductive effect, unclear FFR edges, and low sensitivity in detecting magnetic particle signals. To achieve a higher magnetic field strength, the coil group needs to be made larger, which increases the size and cost of the magnetic particle imaging system.

[0007] In addition, there are two types of traditional MPI reconstruction techniques. One relies on the system matrix to pre-characterize the signal response of SPION, and the other reconstruction technique is the X-space algorithm. Image reconstruction based on the system matrix is ​​usually achieved by solving a linear equation with a system function, which maps the measured signal spectrum to the spatial concentration. This function includes the nonlinear magnetic response of the particle and the measurement conditions. Due to the pathological nature of the inverse problem, the amount of calculation is large and the process is complicated when solving it. The X-space method ignores the relaxation of the particles. In addition, the above two methods are limited by the amount of calculation and the calculation method, and are not suitable for reconstructing three-dimensional time-series image sequences.

[0008] With the development of neural networks, more and more methods are suitable for MPI and fMPI reconstruction. Among them, MoCo-GAN is particularly suitable for time-series reconstruction. The continuity of 3D time-series image sequences in the time dimension allows them to be regarded as videos, and the visual signals in videos can be divided into two parts: content and motion. Content specifies the objects in the video, while motion describes their dynamics. Therefore, in addition to learning the content model of the object, the generative model also needs to learn a reasonable motion model of the object. For this reason, the motion and content decomposition generative adversarial network (MoCo-GAN) framework is suitable for video / 3D time-series image sequence generation.

[0009] Based on this, the present invention combines the GAN network to propose a time-series magnetic nanoparticle imaging device and method for detecting neural response activities. Summary of the invention

[0010] In order to solve the above problems in the prior art, that is, to solve the problems that the existing magnetic particle imaging technology has low sensitivity in detecting magnetic particle signals, requires a larger coil group to achieve a higher magnetic field strength, and has a large amount of calculation and a complex calculation method, the present invention proposes a time-series magnetic nanoparticle imaging device for detecting neural response activities, the device comprising: a signal generator, a power amplifier, a selection coil, a driving coil, an excitation coil, a receiving coil, a high-pass filter, and a signal acquisition card;

[0011] The signal generator outputs a voltage signal which is amplified by the power amplifier and then connected to the selection coil and the drive coil;

[0012] There are two selection coils, one as a first selection coil and the other as a second selection coil; the first selection coil and the second selection coil are arranged in a Maxwell configuration and have the same axis;

[0013] There are six drive coils, which are respectively used as a first drive coil, a second drive coil, a third drive coil, a fourth drive coil, a fifth drive coil, and a sixth drive coil;

[0014] The first drive coil and the second drive coil are symmetrically arranged in parallel and have the same axis; the first drive coil is arranged on the inner side of the first selection coil, and the axis of the first drive coil is the same as the axis of the first selection coil; the third drive coil, the fourth drive coil, the fifth drive coil, and the sixth drive coil are arranged in a plane perpendicular to the axis of the first drive coil, and the axes of the four drive coils are parallel to the axis of the first drive coil; the third drive coil and the fourth drive coil are symmetrically arranged, and the fifth drive coil and the sixth drive coil are symmetrically arranged, and the two symmetry lines are orthogonal;

[0015] The excitation coil and the receiving coil are both coaxial with the first selection coil; the receiving coil is arranged outside the first selection coil; the excitation coil is arranged outside the receiving coil;

[0016] The high-pass filter filters out interference signals from the signal received by the receiving coil; the signal acquisition card is used to import the signal filtered by the high-pass filter into a computer for magnetic particle image reconstruction.

[0017] In some preferred embodiments, the selection coil, the drive coil, the excitation coil, and the receiving coil are all circular coils.

[0018] In some preferred embodiments, the selection coil, the drive coil, the excitation coil, and the receiving coil are all made of NbTi superconducting wire.

[0019] In a second aspect of the present invention, a time-sequential magnetic nanoparticle imaging method for detecting neural response activity is proposed. Based on the above-mentioned time-sequential magnetic nanoparticle imaging device for detecting neural response activity, the method comprises:

[0020] S100, collecting one-dimensional MPI signals by means of the constructed time-series magnetic nanoparticle imaging device for detecting neural response activities, and constructing a one-dimensional signal sequence;

[0021] S200, inputting the one-dimensional signals in the one-dimensional signal sequence into a generator in a pre-trained generative adversarial network in sequence, thereby obtaining a reconstructed MPI three-dimensional time-series image sequence, that is, a time-series motion video;

[0022] The generative adversarial network includes a generator, a 3D discriminator, and a 2D discriminator; the generator is used to generate an input one-dimensional signal sequence into an MPI three-dimensional time-series image sequence; the 3D discriminator is used to combine the generated MPI three-dimensional time-series image sequence with the real MPI three-dimensional time-series image sequence label to determine whether the generated MPI three-dimensional time-series image sequence is a real image sequence; the 2D discriminator is used to determine whether the two-dimensional image of the generated MPI three-dimensional time-series image sequence after random frame downsampling is a real image based on the two-dimensional image after random frame downsampling of the generated MPI three-dimensional time-series image sequence and the real MPI three-dimensional time-series image sequence label.

[0023] In some preferred embodiments, the generator includes a front linear layer, a GRU cell network, an upsampling block, a rear linear layer, and a splicing layer;

[0024] The front linear layer is used to perform linear transformation processing on each one-dimensional signal in the input one-dimensional signal sequence;

[0025] The input of the GRUcell network in the first time step is: one is the signal of the current time step after the processing of the front linear layer, and the other is the initialization state; the initialization state is a standard normal distribution; except for the first time step, the input of the GRUcell network in the remaining time steps is: one is the signal of the current time step after the processing of the front linear layer, and the other is the state of the GRUcell network output in the previous time step;

[0026] The upsampling block includes Swin Block and Patch expanding; the Swin Block includes window self-attention and sliding window self-attention; the upsampling block is used to perform window self-attention and sliding window self-attention processing on the signal output by the GRU cell network, and perform Patch expanding operation after processing;

[0027] The post-linear layer is used to perform linear transformation processing on the output of the upsampling block to obtain a reconstructed image;

[0028] The splicing layer is used to splice the reconstructed images to obtain an MPI three-dimensional time-series image sequence.

[0029] In some preferred embodiments, the 3D discriminator adopts the video swin transformer as a basic framework, and a cross attention module is added after the 3D W-MSA and 3D SW-MSA modules of the video swin transformer.

[0030] In some preferred embodiments, the 2D discriminator is constructed based on a swin transformer.

[0031] Beneficial effects of the present invention:

[0032] The present invention improves the sensitivity of magnetic particle signal detection, reduces the volume of the coil group, simplifies the complexity and amount of calculation, and improves the robustness of magnetic particle imaging.

[0033] 1) The present invention replaces the coil group with NbTi superconducting wire, and then uses a high-pass filter to make the FFR edge clear, control the FFR shift more convenient and quick, and the magnetic particle signal detection sensitivity is high, and the degree of interference from external signals is not large. And because the coil material is NbTi, a large coil group is not required to achieve a higher magnetic field strength. The volume of the coil group can be slightly increased while the voltage is increased, which is more friendly to the volume and cost control of the magnetic particle imaging device;

[0034] 2) This application combines a generative adversarial network with an end-to-end processing approach to simplify the computational complexity and amount of magnetic particle imaging and improve the robustness of magnetic particle imaging. Compared with single-frame MPI images that do not contain time series information, three-dimensional time-series image sequences can display dynamically changing MPI images and provide more useful information required clinically. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings.

[0036] Figure 1 is a schematic structural diagram of a time-series magnetic nanoparticle imaging device for detecting neural response activities according to an embodiment of the present invention;

[0037] Figure 2 It is a schematic diagram of the structure of a generator of a generative adversarial network according to an embodiment of the present invention;

[0038] Figure 3 It is a schematic diagram of the processing process of a 2D discriminator and a 3D discriminator in a generative adversarial network according to an embodiment of the present invention. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0040] The present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the relevant invention, rather than to limit the invention. It should be noted that, in the absence of conflict, the embodiments in the present application and the features in the embodiments can be combined with each other.

[0041] A time-series magnetic nanoparticle imaging device for detecting neural response activity according to a first embodiment of the present invention is as follows: Figure 1 As shown, the device includes: a signal generator, a power amplifier, a selection coil, a driving coil, an excitation coil, a receiving coil, a high-pass filter, and a signal acquisition card;

[0042] The signal generator outputs a voltage signal which is amplified by the power amplifier and then connected to the selection coil and the driving coil;

[0043] There are two selection coils, one as a first selection coil and the other as a second selection coil; the first selection coil and the second selection coil are arranged in a Maxwell configuration and have the same axis;

[0044] There are six drive coils, which are respectively used as a first drive coil, a second drive coil, a third drive coil, a fourth drive coil, a fifth drive coil, and a sixth drive coil;

[0045] The first drive coil and the second drive coil are symmetrically arranged in parallel and have the same axis; the first drive coil is arranged on the inner side of the first selection coil, and the axis of the first drive coil is the same as the axis of the first selection coil; the third drive coil, the fourth drive coil, the fifth drive coil, and the sixth drive coil are arranged in a plane perpendicular to the axis of the first drive coil, and the axes of the four drive coils are parallel to the axis of the first drive coil; the third drive coil and the fourth drive coil are symmetrically arranged, and the fifth drive coil and the sixth drive coil are symmetrically arranged, and the two symmetry lines are orthogonal;

[0046] The excitation coil and the receiving coil are both coaxial with the first selection coil; the receiving coil is arranged outside the first selection coil; the excitation coil is arranged outside the receiving coil;

[0047] The high-pass filter filters out interference signals from the signal received by the receiving coil; the signal acquisition card is used to import the signal filtered by the high-pass filter into a computer for magnetic particle image reconstruction.

[0048] In order to more clearly illustrate a time-series magnetic nanoparticle imaging device for detecting neural response activities of the present invention, each module in an embodiment of the device of the present invention is described in detail below in conjunction with the accompanying drawings.

[0049] The present invention provides a time-series magnetic nanoparticle imaging device and method for detecting neural response activities. It can realize the generation and collection of one-dimensional magnetic particle signals by MPI equipment, and the reconstruction of MPI three-dimensional time-series image sequence from the collected signals to visualize neural response activities. The details are as follows:

[0050] A sequential magnetic nanoparticle imaging device for detecting neural response activities of the present invention comprises: a signal generator, a power amplifier, a selection coil, a driving coil, an excitation coil, a receiving coil, a high-pass filter, and a signal acquisition card;

[0051] The signal generator outputs a voltage signal which is amplified by the power amplifier and then connected to the selection coil and the driving coil;

[0052] There are two selection coils, one as a first selection coil and the other as a second selection coil; the first selection coil and the second selection coil are arranged in a Maxwell configuration (i.e., symmetrically arranged in parallel) and have the same axis, forming a Maxwell coil pair;

[0053] There are six drive coils, which are respectively used as a first drive coil, a second drive coil, a third drive coil, a fourth drive coil, a fifth drive coil, and a sixth drive coil;

[0054] The first drive coil and the second drive coil are symmetrically arranged in parallel and have the same axis; the first drive coil is arranged on the inner side of the first selection coil, and the axis of the first drive coil is the same as the axis of the first selection coil; the third drive coil, the fourth drive coil, the fifth drive coil, and the sixth drive coil are arranged in a plane perpendicular to the axis of the first drive coil, and the axes of the four drive coils are parallel to the axis of the first drive coil; the third drive coil and the fourth drive coil are symmetrically arranged, and the fifth drive coil and the sixth drive coil are symmetrically arranged, and the two symmetry lines are orthogonal;

[0055] The excitation coil and the receiving coil are both coaxial with the first selection coil; the receiving coil is arranged outside the first selection coil; the excitation coil is arranged outside the receiving coil;

[0056] The selection coil, driving coil, exciting coil and receiving coil are all circular coils, and are made of NbTi superconducting wire.

[0057] The high-pass filter filters out interference signals from the signal received by the receiving coil; the signal acquisition card is used to import the signal filtered by the high-pass filter into the computer for magnetic particle image reconstruction. That is, the computer is used to collect one-dimensional MPI signals through the constructed time-series magnetic nanoparticle imaging device for detecting neural response activities to construct a one-dimensional signal sequence; the one-dimensional signals in the one-dimensional signal sequence are sequentially input into the generator in the pre-trained generative adversarial network, thereby obtaining a reconstructed MPI three-dimensional time-series image sequence. For the specific process, see the second embodiment for details.

[0058] The specific working process of the time-series magnetic nanoparticle imaging device for detecting neural response activity is as follows:

[0059] First, the signal generator outputs a voltage signal. If the output voltage value is too small, the magnetic field strength generated by the coil group will be too small, which is not enough to stimulate the SPION particles to produce nonlinear magnetization response or the magnetic particle signal is not clear. Therefore, the output voltage needs to be amplified by a power amplifier.

[0060] The power amplifier is connected to the selection coil and the driving coil. The selection coil arranged in the Maxwell configuration will generate a magnetic field whose magnetic field strength decreases as it moves away from the selection coil (static gradient magnetic field). The SPION particles in the device are in a magnetic saturation state under a strong magnetic field, and there is a region with zero magnetic field near the center of the symmetrical selection coil connection line, which is called the magnetic field free region (FFR). The SPION particles in the region are magnetically unsaturated and can also be magnetized by the subsequent magnetic field (because SPION particles have the property of nonlinear magnetization). There is a transition region between the FFR and other non-zero magnetic regions of the gradient magnetic field. Most of the errors and noise in the general MPI imaging process come from this. If the rate of change of the magnetic field gradient in the gradient magnetic field is small, the edge of the zero magnetic region will be unclear, affecting the detection accuracy. Therefore, NbTi superconducting wire is used as the selection coil material, so that the magnetic field gradient generated by the symmetrically placed superconducting selection coil has a large rate of change around the FFR, making the boundary between the FFR edge and the non-FFR region clear.

[0061] After the drive coil is connected to the current, it generates a working magnetic field that can be superimposed with the magnetic field of the selection coil, which is used to drive the FFR to move in space and scan the sample to be tested. If the intensity of the driving magnetic field is too small, it is difficult to affect the magnetic field of the selection coil, that is, it is difficult to control the position change of the FFR. Therefore, NbTi superconducting wire is used as the driving coil material, so that the driving magnetic field intensity generated under the same current and voltage conditions is greater, which is convenient for controlling the displacement of the FFR.

[0062] When the excitation coil is powered on near the FFR, it can generate a high-frequency time-varying excitation magnetic field, which acts on the magnetically unsaturated SPION particles in the FFR, pushing the magnetic particles to repeatedly magnetize into a saturated state, and generating a nonlinear response. This nonlinear response generates a signal component at the harmonic, and the magnetic particle signal comes from the harmonic component. The signal generated at this time is generally weak and mixed with external interference noise. Therefore, using NbTi superconducting wire as the excitation coil material can increase the intensity of the excitation magnetic field and effectively improve the problem of weak magnetic particle signals.

[0063] The receiving coil can receive the magnetic particle signal. Using NbTi superconducting wire as the receiving coil material can, on the one hand, make the receiving coil more sensitive to the signal detection, and on the other hand, can also improve the problem of weak received magnetic particle signals. Since the received magnetic particle signal comes from high-order harmonics, which are mixed with background interference noise in the detection process and fundamental interference signals of the excitation field, after the receiving coil converts the magnetic particle signal into a voltage signal, it must be connected to a high-pass filter to filter out the fundamental interference and background interference, and then use the signal acquisition card to import the collected signal from the MPI device into the computer for subsequent software processing.

[0064] In summary, with the FFR as the center, a driving coil is placed every 90 degrees up, down, left, right, front and back, for a total of 6. Two selection coils are arranged in Maxwell configuration outside the upper and lower driving coils. The selection coil and the driving coil together form the FFR and control its position. One receiving coil and one excitation coil are placed on the top of the selection coil, with the receiving coil closer to the driving coil. After the excitation coil is powered on, a signal is applied to the FFR, and the receiving coil receives the magnetic particle signal and transmits it to the subsequent high-pass filter and other components.

[0065] It should be noted that the time-series magnetic nanoparticle imaging device for detecting neural response activities provided in the above embodiment is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the modules or steps in the embodiments of the present invention can be decomposed or combined. For example, the modules in the above embodiment can be combined into one module, or further divided into multiple sub-modules to complete all or part of the functions described above. The names of the modules and steps involved in the embodiments of the present invention are only for distinguishing the modules or steps, and are not regarded as improper limitations of the present invention.

[0066] A time-sequential magnetic nanoparticle imaging method for detecting neural response activity according to a second embodiment of the present invention is based on the time-sequential magnetic nanoparticle imaging device for detecting neural response activity described above; the method comprises:

[0067] S100, collecting one-dimensional MPI signals by means of the constructed time-series magnetic nanoparticle imaging device for detecting neural response activities, and constructing a one-dimensional signal sequence;

[0068] S200, inputting the one-dimensional signals in the one-dimensional signal sequence into the generator in the pre-trained generative adversarial network in sequence, thereby obtaining a reconstructed MPI three-dimensional time-series image sequence, that is, a time-series motion video.

[0069] In the following, the structure and training process of the generative adversarial network are first described, and then the process of obtaining the MPI three-dimensional time-series image sequence by the time-series magnetic nanoparticle imaging method for detecting neural response activities is described.

[0070] The structure and training process of the generative adversarial network are as follows:

[0071] Step 1: Prepare training and test data. First, use MATLAB to generate a time series binary image sequence. Then, confirm the equipment parameters and use the MPIRF simulation software to generate corresponding one-dimensional signal sequences for all time series binary image sequences. Add a time dimension to all generated two-dimensional images, and superimpose the two-dimensional images in the time dimension according to their corresponding sequence order to generate an MPI three-dimensional time series image sequence. Most of the obtained paired sequences are used as training sets to train the network, and a small part is used as a test set. Under the same parameters, the measured signal sequence of the MPI device of the above-mentioned hardware device is processed in the same way as the training set, and the obtained paired sequences are used as part of the test set.

[0072] Step 2: Build the network: The network consists of a generator, a 3D (temporal) discriminator, a 2D (spatial) discriminator, and a loss function.

[0073] The generator includes a front linear layer, a GRU cell network, an upsampling block, a rear linear layer, and a concatenation layer, such as Figure 2 As shown;

[0074] The front linear layer (multiple) is used to perform linear transformation processing on each one-dimensional signal in the input one-dimensional signal sequence (i.e., to expand the information capacity and nonlinearize the input one-dimensional signal sequence to prevent problems such as network overfitting due to uneven distribution of input information), and then input them into the GRU cell network respectively after processing;

[0075] The function of the GRUcell network is to receive the one-dimensional signal sequence after MLP, learn the temporal information relationship, embed the signal sequence into the temporal relationship, and then output it to the upsampling block; the GRUcell network is a special GRU network. The GRUcell network processes the input within a single time step and returns the output within a single time step. Its advantages are that it is more flexible and has better model granularity. In this reconstruction algorithm, it is necessary to dynamically adjust the sequence length and record the output of all time steps, while the ordinary GRU network can only get the output of the last time step, so the GRUcell network is used in this reconstruction algorithm.

[0076] The function of the GRU cell network is to loop the length of the sequence to be processed (T) times, poll the one-dimensional signal sequence, receive the output of the previous time step as the input state of this time, and use the signal polled this time as input to learn the time sequence information relationship. Finally, save the output of this time step and submit it to the next time step.

[0077] The upsampling block includes Swin Block and Patch expanding; the Swin Block includes window self-attention and sliding window self-attention;

[0078] The function of the upsampling block is to convert a one-dimensional signal into a corresponding two-dimensional image. During the conversion process, the texture and distribution characteristics of the image are learned through mechanisms such as window self-attention (W-MSA) and sliding window self-attention (SW-MSA), and the image resolution is gradually increased and the number of image channels is reduced through the Patch expanding operation, which is then input to the next upsampling layer.

[0079] A post-linear layer, used for performing a linear transformation process on the output of the upsampling block to obtain a reconstructed image;

[0080] The stitching layer is used to stitch the reconstructed images to obtain the MPI three-dimensional time-series image sequence. That is, the function of the stitching layer is to add a time dimension to the T-frame two-dimensional image output by the upsampling block, and then superimpose them in the time dimension according to the order in the one-dimensional signal sequence (that is, the two-dimensional image adds a time dimension, and then superimposes them in the time dimension according to the order in the one-dimensional signal sequence) to obtain the generated MPI three-dimensional time-series image sequence (that is, Figure 2 3D time-series image sequences in .

[0081] The 3D discriminator uses the video swin transformer as the basic framework, and adds a cross-attention module after the 3D W-MSA and 3DSW-MSA calculation modules in the framework. Cross-attention is used to calculate the relevant information between different time dimensions (i.e., different frames of the 3D time-series image sequence), and use this information to help update the model parameters. The function of the 3D discriminator is to determine the authenticity of the MPI 3D time-series image sequence generated by the generator and the MPI 3D time-series image sequence in the training set.

[0082] The 2D discriminator uses the swin transformer as the basic framework, and adds a classification head after the framework (that is, a linear layer is connected at the end of the network to output the probability that the 2D discriminator considers the input to be true). Before using the 2D discriminator, the MPI 3D time-series image sequence generated by the generator and the MPI 3D time-series image sequence in the training set are randomly downsampled in frames according to a fixed downsampling ratio to obtain k frames of 2D images. The k frames of 2D images are input into the 2D discriminator, and the 2D discriminator is applied independently to each frame to obtain k recognition results and calculate the average. The role of the 2D discriminator is to judge the authenticity of 2D images from different sources, such as Figure 3 shown.

[0083] Step 3: Train the network: Use the one-dimensional signal sequence generated by the simulation software as the input of the generator to obtain the MPI three-dimensional time-series image sequence output by the GAN network. Use it and the MPI three-dimensional time-series image sequence in the training set as the input of the 3D discriminator. Use the two-dimensional image obtained by random frame downsampling of the MPI three-dimensional time-series image sequence from two different sources as the input of the 2D discriminator for iterative training. When the iteration requirements are met, the network training ends.

[0084] Time-sequential magnetic nanoparticle imaging method for detecting neural response activity:

[0085] S100, collecting one-dimensional MPI signals by means of the constructed time-series magnetic nanoparticle imaging device for detecting neural response activities, and constructing a one-dimensional signal sequence;

[0086] S200, inputting the one-dimensional signals in the one-dimensional signal sequence into the generator in the pre-trained generative adversarial network in sequence, thereby obtaining a reconstructed MPI three-dimensional time-series image sequence, that is, a time-series motion video.

[0087] That is, the training set data of the present invention is first generated by MATLAB as a time-series binary image sequence. Then, the equipment parameters are confirmed, and the MPIRF simulation software is used to generate corresponding one-dimensional signal sequences from all time-series binary image sequences. A time dimension is added to all the generated two-dimensional images, and the two-dimensional images are superimposed in the time dimension according to their corresponding sequence order to generate an MPI three-dimensional time-series image sequence. The one-dimensional signal sequence and the MPI three-dimensional time-series image sequence obtained above are used together as training set data. The test set data is composed of part of the data generated by the simulation software and part of the actual data collected by the MPI device of the above-mentioned hardware device. The training set data is first used to train the conditional GAN ​​network, and after the training is completed, the test set data is used to verify whether the network performance meets the requirements, so as to realize the GAN network to quickly and high-resolution reconstruct the time-series function MPI data to visualize the neural response activity.

[0088] In order to further understand the time-series magnetic nanoparticle imaging method for detecting neural response activities of the present invention, a specific example is given below:

[0089] 1. Build MPI equipment

[0090] According to the above fMPI hardware device diagram, purchase parts and NbTi superconducting wires and connect them to build the MPI device. Connect the built hardware device to the computer host;

[0091] 2. Prepare training set and test set data

[0092] First, MATLAB is used to generate 10,000 time-series binary image sequences of three different models, namely, geometric model, letter model and resolution model. After obtaining the time-series binary image sequences, the MPIRF simulation software is used to generate the corresponding 10,000 one-dimensional signal sequences. A time dimension is added to all the generated two-dimensional images, and the two-dimensional images are superimposed in the time dimension according to their corresponding sequence order to generate the corresponding 10,000 MPI three-dimensional time-series image sequences. 8,000 pairs of the 10,000 paired data are used as training data, and the remaining 2,000 pairs are used as test data sets. At the same time, the MPI device is used to actually collect 100 sets of paired data of different models as test sets;

[0093] 3. Training fMPI-GAN to generate adversarial models

[0094] The generative model is built using the network architecture of conditional GAN. In the generator, a one-dimensional signal sequence is input as a condition. PSNR and SSIM values ​​are used as evaluation indicators to judge the difference between the training image sequence and the generated image sequence. The model with the highest PSNR value on the training set is selected as the best model as the test model for the test set data. The input of the 3D discriminator is the image sequence output by the generator and the real MPI three-dimensional time-series image sequence paired with the one-dimensional signal sequence in the training set. The output is the judgment of whether the input image sequence is a real image sequence. The input of the 2D discriminator is two two-dimensional images from two different sources obtained by downsampling the MPI three-dimensional time-series image sequence generated by the generator and the MPI three-dimensional time-series image sequence in the training set in the same downsampling ratio in frames. The output is the judgment of whether the input image is a real image. The generator and the discriminator improve the quality of the generated images during training.

[0095] 4. Testing and deploying the fMPI-GAN generation model

[0096] During the test, only the network's generative model (i.e., generator) is used. The one-dimensional signal sequence in the test set is used as the input of the generator, and the output is a reconstructed MPI three-dimensional time-series image sequence. Evaluation indicators such as PSNR and SSIM are used to evaluate the quality of the generated image sequence. The specific calculation method is to traverse and calculate the PSNR and SSIM values ​​of each corresponding image between the generated image sequence and the corresponding real image sequence in the test set and calculate the average value. After the test is completed, the trained generative model is deployed in the MPI device built in the first step for subsequent use.

[0097] 5. Study the relationship between the signal sequence generated by neural response activity and the generated MPI 3D time-series image sequence

[0098] Difference analysis: Analyze the neural response signal sequences that generate different MPI three-dimensional time-series image sequences, and calculate the time-domain feature differences between the sequences: such as mean, variance, etc. In addition, perform short-time Fourier transform (STFT) on these signal sequences to obtain time-frequency graph features, and compare the time-domain feature differences with the differences between MPI three-dimensional time-series image sequences (SSIM) to obtain the relationship between signal sequence differences and MPI three-dimensional time-series image sequence differences (SSIM).

[0099] Similarity analysis: The similarity between the neural response signal sequences that generate similar MPI three-dimensional time-series image sequences is measured, and the Euclidean distance, cosine similarity and correlation coefficient between the signal sequences are calculated to obtain the relationship between the signal sequence similarity index score and the MPI three-dimensional time-series image sequence similarity score (SSIM).

[0100] An electronic device according to the third embodiment of the present invention comprises at least one processor; and a memory communicatively connected to at least one of the processors; wherein the memory stores instructions executable by the processor, and the instructions are used to be executed by the processor to implement the time-series magnetic nanoparticle imaging method for detecting neural response activities as claimed in the claim above.

[0101] A computer-readable storage medium according to a fourth embodiment of the present invention stores computer instructions, and the computer instructions are used to be executed by the computer to implement the above-mentioned time-series magnetic nanoparticle imaging method for detecting neural response activities.

[0102] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process and related instructions of the electronic device and computer-readable storage medium described above can refer to the corresponding process in the aforementioned method example and will not be repeated here.

[0103] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the device, method and computer program product according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based device that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0104] The terms "first", "second", etc. are used to distinguish similar objects rather than to describe or indicate a particular order or sequence.

[0105] The term "comprise" or any other similar term is intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus / device that includes a list of elements includes not only those elements but also other elements not expressly listed, or also includes elements inherent to such process, method, article, or apparatus / device.

[0106] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

Claims

1. A time-series magnetic nanoparticle imaging device for detecting neural response activity, characterized in that: The device includes: a signal generator, a power amplifier, a selection coil, a driving coil, an excitation coil, a receiving coil, a high-pass filter, and a signal acquisition card; The signal generator outputs a voltage signal which is amplified by the power amplifier and then connected to the selection coil and the driving coil; There are two selection coils, one as a first selection coil and the other as a second selection coil; the first selection coil and the second selection coil are arranged in a Maxwell configuration and have the same axis; There are six drive coils, which are respectively used as a first drive coil, a second drive coil, a third drive coil, a fourth drive coil, a fifth drive coil, and a sixth drive coil; The first drive coil and the second drive coil are symmetrically arranged in parallel and have the same axis; the first drive coil is arranged on the inner side of the first selection coil, and the axis of the first drive coil is the same as the axis of the first selection coil; the third drive coil, the fourth drive coil, the fifth drive coil, and the sixth drive coil are arranged in a plane perpendicular to the axis of the first drive coil, and the axes of the four drive coils are parallel to the axis of the first drive coil; the third drive coil and the fourth drive coil are symmetrically arranged, and the fifth drive coil and the sixth drive coil are symmetrically arranged, and the two symmetry lines are orthogonal; The excitation coil and the receiving coil are both coaxial with the first selection coil; the receiving coil is arranged outside the first selection coil; the excitation coil is arranged outside the receiving coil; The high-pass filter filters out interference signals from the signal received by the receiving coil; the signal acquisition card is used to import the signal filtered by the high-pass filter into a computer for magnetic particle image reconstruction.

2. The time-series magnetic nanoparticle imaging device for detecting neural response activity according to claim 1, characterized in that: The selection coil, the driving coil, the exciting coil, and the receiving coil are all circular coils.

3. The time-series magnetic nanoparticle imaging device for detecting neural response activity according to claim 2, characterized in that: The selection coil, the drive coil, the excitation coil, and the receiving coil are all made of NbTi superconducting wire.

4. A method for detecting neural response activity by sequential magnetic nanoparticle imaging, based on the sequential magnetic nanoparticle imaging device for detecting neural response activity according to any one of claims 1 to 3, characterized in that: The method includes: S100, collecting one-dimensional MPI signals by means of the constructed time-series magnetic nanoparticle imaging device for detecting neural response activities, and constructing a one-dimensional signal sequence; S200, inputting the one-dimensional signals in the one-dimensional signal sequence into a generator in a pre-trained generative adversarial network in sequence, thereby obtaining a reconstructed MPI three-dimensional time-series image sequence, that is, a time-series motion video; The generative adversarial network includes a generator, a 3D discriminator, and a 2D discriminator; the generator is used to generate an input one-dimensional signal sequence into an MPI three-dimensional time-series image sequence; the 3D discriminator is used to combine the generated MPI three-dimensional time-series image sequence with the real MPI three-dimensional time-series image sequence label to determine whether the generated MPI three-dimensional time-series image sequence is a real image sequence; the 2D discriminator is used to determine whether the two-dimensional image of the generated MPI three-dimensional time-series image sequence after random frame downsampling is a real image based on the two-dimensional image after random frame downsampling of the generated MPI three-dimensional time-series image sequence and the real MPI three-dimensional time-series image sequence label.

5. The method for time-series magnetic nanoparticle imaging for detecting neural response activity according to claim 4, characterized in that: The generator includes a front linear layer, a GRU cell network, an upsampling block, a rear linear layer, and a splicing layer; The front linear layer is used to perform linear transformation processing on each one-dimensional signal in the input one-dimensional signal sequence; The input of the GRUcell network in the first time step is: one is the signal of the current time step after the processing of the front linear layer, and the other is the initialization state; the initialization state is a standard normal distribution; except for the first time step, the input of the GRUcell network in the remaining time steps is: one is the signal of the current time step after the processing of the front linear layer, and the other is the state of the GRUcell network output in the previous time step; The upsampling block includes Swin Block and Patch expanding; the Swin Block includes window self-attention and sliding window self-attention; the upsampling block is used to perform window self-attention and sliding window self-attention processing on the signal output by the GRU cell network, and perform Patch expanding operation after processing; The post-linear layer is used to perform linear transformation processing on the output of the upsampling block to obtain a reconstructed image; The splicing layer is used to splice the reconstructed images to obtain an MPI three-dimensional time-series image sequence.

6. The method for time-series magnetic nanoparticle imaging for detecting neural response activity according to claim 4, characterized in that: The 3D discriminator adopts the video swin transformer as the basic framework, and a cross attention module is added after the 3D W-MSA and 3D SW-MSA modules of the video swin transformer.

7. The method for time-series magnetic nanoparticle imaging for detecting neural response activity according to claim 4, characterized in that: The 2D discriminator is built based on the swin transformer.

Images