Magnetic particle image sparse scanning reconstruction method, system and device and storage medium

Abstract

The invention belongs to the technical field of magnetic particle imaging, particularly relates to a magnetic particle image sparse scanning reconstruction method, system and device and a storage medium, and aims to solve the problems that the MPI time resolution of magnetic particle imaging is low, and the image reconstruction performance and the reasoning speed are unbalanced. The method comprises the following steps: determining a magnetic particle image to be sparsely scanned and reconstructed; inputting the magnetic particle image to be subjected to sparse scanning reconstruction into a magnetic particle image sparse scanning reconstruction model to obtain a sparse scanning reconstructed magnetic particle image output by the magnetic particle image sparse scanning reconstruction model; wherein the magnetic particle image sparse scanning reconstruction model is obtained by sequentially training a teacher model comprising a multi-scale implicit content driving module and a student model comprising a knowledge distillation loss function based on sample data of a selected training set. According to the method, the imaging time resolution can be improved, and the image reconstruction performance and the reasoning speed are improved.

Description

Background Art

[0002] Magnetic particle imaging (MPI) is an emerging tomographic imaging technology with high sensitivity and potential for real-time imaging. It has played an important role in the fields of vascular imaging and cell tracing, and is expected to bring new possibilities for real-time imaging of three-dimensional living organisms. However, the current MPI requires multiple measurement averaging to perform signal denoising in order to obtain high-quality signals. Multiple measurement averaging will greatly reduce the temporal resolution of MPI. Taking the public dataset Open MPI as an example, the single 3D MPI measurement time is 0.02154s, and the peak signal to noise ratio (PNSR) of the reconstructed image is about 25dB; the measurement time for 100 times is 2.154s, and the PNSR of the reconstructed image is about 41dB. It can be seen that although multiple measurement averaging greatly improves the image quality (PSNR increases from 25dB to 41dB), it also greatly reduces the imaging temporal resolution (the number of reconstructed frames per second (FPS) decreases from 46.42 to 0.46). In magnetic particle imaging, sparse scanning can significantly shorten the scanning time by reducing the amount of sampled data, which is very beneficial for application scenarios that require fast imaging. Therefore, how to reduce or replace the multiple measurement averaging process through sparse scanning reconstruction technology is crucial for real-time imaging of MPI.

[0003] In recent years, deep learning has been widely used to enhance the performance of medical imaging. There have been many studies on algorithms for medical images based on deep learning. For example, in the fields of computed tomography (CT) and positron emission tomography (PET), neural networks are used to denoise low-dose images to obtain high-dose images, thereby reducing radiation hazards to the human body. In the field of magnetic resonance imaging (MRI), neural networks are used to reduce the number of K-space sampling times and improve image quality. In the field of MPI, although there have been some studies on signal and image denoising, there is still a lack of denoising research on reducing measurement time and the average number of measurements. Summary of the invention

[0004] In order to solve the above-mentioned problems in the prior art, namely, the low temporal resolution of magnetic particle imaging (MPI) and the imbalance between image reconstruction performance and inference speed, the present invention provides a sparse scanning and reconstruction method, system, device and storage medium for magnetic particle images, which improves the imaging temporal resolution by driving image modeling with latent content, and compresses the model by utilizing knowledge distillation technology based on attention mask, so as to achieve a good balance between image reconstruction performance and inference speed.

[0005] One aspect of the present invention provides a method for sparse scanning and reconstruction of magnetic particle images, comprising:

[0006] determining a magnetic particle image to be reconstructed by sparse scanning;

[0007] Inputting the magnetic particle image to be sparsely scanned and reconstructed into a sparse scanning and reconstruction model of a magnetic particle image to obtain a sparsely scanned and reconstructed magnetic particle image output by the sparse scanning and reconstruction model of the magnetic particle image;

[0008] The magnetic particle image sparse scanning reconstruction model is obtained by training a teacher model including a multi-scale latent content driving module and a student model including a knowledge distillation loss function in sequence based on sample data of a selected training set.

[0009] Preferably, the magnetic particle image sparse scanning reconstruction model is obtained by training a teacher model including a multi-scale latent content driving module and a student model including a knowledge distillation loss function in sequence based on selected training sample data, and includes:

[0010] Randomly selecting sample data of a training set, and inputting the sample data of the training set into the teacher model including the multi-scale latent content driving module for training, so as to update the parameters of the teacher model;

[0011] The sample data of the randomly selected training set is input into the student model including the knowledge distillation loss function, and the parameters of the student model are updated based on the total loss function of the student model to obtain the sparse scanning and reconstruction model of the magnetic particle image.

[0012] Preferably, the step of inputting the sample data of the training set into the teacher model including the multi-scale latent content driving module for training to update the parameters of the teacher model comprises:

[0013] Inputting the sample data of the training set into parallel multi-scale convolution kernels, outputting intermediate feature signals of convolution kernels of different scales respectively, and splicing the intermediate feature signals of convolution kernels of different scales into a composite feature signal;

[0014] Inputting the intermediate feature signals of the convolution kernels of different scales into the latent content activation module to calculate the attention mask signals of different scales, and obtaining the activation map of the multi-scale composite signal based on the average value of the attention mask signals of different scales;

[0015] Performing point-to-point multiplication on the composite feature signal and the activation map of the multi-scale composite signal, and taking the corresponding multi-scale features of each layer as residuals after a convolution operation to obtain a final output feature signal;

[0016] The final output feature signal is multiplied by the pseudo-inverse of the system matrix, and a predicted image is obtained through two layers of convolution. The loss function of the teacher model is obtained based on the predicted image, and the parameters of the teacher model are updated based on the loss function of the teacher model.

[0017] Preferably, the loss function based on the teacher model updates the parameters of the teacher model, and the formula is as follows:

[0018]

[0019]

[0020] Where η is the learning rate, is the loss function of the teacher model, θ T are the parameters of the teacher model, is the predicted image, x c is the image corresponding to the sample data of the training set, || 1 is the absolute value operator.

[0021] Preferably, the total loss function of the student model includes a direct loss function and a knowledge distillation loss function; wherein the knowledge distillation loss function includes a feature similarity loss function and a spatial similarity graph loss function, and the direct loss function is the loss function of the teacher model.

[0022] Preferably, the formula of the feature similarity loss function is as follows:

[0023]

[0024] in, is the feature similarity loss function, F′ T are the multi-scale convolution output and the final output of the preset layer of the teacher model, respectively. is an activation map of the multi-scale composite signal corresponding to the preset layer of the teacher model, The preset layer module of the student model only contains two consecutive simple convolutions, respectively, corresponding to the output, || 1 is the absolute value operator.

[0025] Preferably, the formula of the spatial similarity graph loss function is as follows:

[0026]

[0027] in, is the spatial similarity graph loss function, G′ T is the signal similarity graph of the hidden layer feature graph of the teacher model, G′ T =f′ T ×Transpose(f′ T ), is the activation map of the multi-scale composite signal corresponding to the preset layer of the teacher model, F′ T is the final output of the multi-scale convolution of the preset layer of the teacher model, G′ S is the signal similarity graph of the hidden layer feature graph of the student model, G′ S =f′ S ×Transpose(f′ S ), is the activation map of the multi-scale composite signal corresponding to the preset layer of the student model, F′ S The final output of the multi-scale convolution of the preset layer of the student model, is a similarity graph of the multi-scale composite signal of the teacher model, is the similarity graph of the multi-scale composite signal of the student model, is the mean square error calculator.

[0028] Another aspect of the present invention provides a magnetic particle image sparse scanning and reconstruction system, comprising:

[0029] An image determination unit, used for determining a magnetic particle image to be reconstructed by sparse scanning;

[0030] An image reconstruction unit, used for inputting the magnetic particle image to be sparsely scanned and reconstructed into a sparse scanning and reconstruction model of a magnetic particle image, and obtaining the sparsely scanned and reconstructed magnetic particle image output by the sparse scanning and reconstruction model of the magnetic particle image;

[0031] The magnetic particle image sparse scanning reconstruction model is obtained by training a teacher model including a multi-scale latent content driving module and a student model including a knowledge distillation loss function in sequence based on sample data of a selected training set.

[0032] According to a third aspect of the present invention, an electronic device is provided, comprising:

[0033] at least one processor;

[0034] and a memory communicatively coupled to at least one of the processors;

[0035] The memory stores instructions executable by the processor, and the instructions are used to be executed by the processor to implement the above-mentioned magnetic particle image sparse scanning and reconstruction method.

[0036] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to be executed by the computer to implement the above-mentioned magnetic particle image sparse scanning and reconstruction method.

[0037] Beneficial effects of the present invention:

[0038] (1) Improving the imaging time resolution by reducing the artificial intelligence algorithm of multiple MPI measurement averaging;

[0039] (2) The multi-scale latent content driven module is used to reconstruct the MPI signal, which improves the image reconstruction performance;

[0040] (3) The knowledge distillation technology based on attention mask compresses the model and improves the reasoning speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0042] Figure 1 It is a flow chart of a method for sparse scanning and reconstruction of magnetic particle images;

[0043] Figure 2 This is a flow chart of teacher model training based on the multi-scale latent content driven module;

[0044] Figure 3 It is a structural schematic diagram of a magnetic particle image sparse scanning and reconstruction system;

[0045] Figure 4 It is a structural diagram of a computer system of a server for implementing the method, system, and device embodiments of the present application. DETAILED DESCRIPTION

[0046] The present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the relevant invention, rather than to limit the invention. It is also necessary to explain that, for ease of description, only the parts related to the relevant invention are shown in the accompanying drawings.

[0047] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0048] The present invention provides a sparse scanning and reconstruction method for magnetic particle images. The method improves the imaging temporal resolution by driving image modeling with latent content, and compresses the model by using knowledge distillation technology based on attention mask, thereby achieving a good balance between image reconstruction performance and reasoning speed, thereby solving the problems of low temporal resolution of magnetic particle imaging MPI and imbalance between image reconstruction performance and reasoning speed.

[0049] A magnetic particle image sparse scanning and reconstruction method of the present invention comprises:

[0050] determining a magnetic particle image to be reconstructed by sparse scanning;

[0051] Inputting the magnetic particle image to be sparsely scanned and reconstructed into a sparse scanning and reconstruction model of a magnetic particle image to obtain a sparsely scanned and reconstructed magnetic particle image output by the sparse scanning and reconstruction model of the magnetic particle image;

[0052] The magnetic particle image sparse scanning reconstruction model is obtained by training a teacher model including a multi-scale latent content driving module and a student model including a knowledge distillation loss function in sequence based on sample data of a selected training set.

[0053] In order to more clearly explain the magnetic particle image sparse scanning reconstruction method of the present invention, the following is combined with Figure 1 Each step in the embodiment of the present invention is described in detail.

[0054] The magnetic particle image sparse scanning and reconstruction method of the first embodiment of the present invention includes steps S101 to S102, each of which is described in detail as follows:

[0055] Step S101, determining a magnetic particle image to be sparsely scanned and reconstructed;

[0056] Step S102, inputting the magnetic particle image to be sparsely scanned and reconstructed into a sparse scanning and reconstruction model of a magnetic particle image, and obtaining a sparsely scanned and reconstructed magnetic particle image output by the sparse scanning and reconstruction model of the magnetic particle image;

[0057] The magnetic particle image sparse scanning reconstruction model is obtained by training a teacher model including a multi-scale latent content driving module and a student model including a knowledge distillation loss function in sequence based on sample data of a selected training set.

[0058] Specifically, sample data of the training set is selected: a set of corresponding images corresponding to the noisy MPI signal and the clean MPI signal is used as the training set, and batch data is randomly selected for training.

[0059] The teacher model consists of stacked multi-scale latent content driven (MLCD) modules. The input features of each module are the output features of the previous layer. Except for the first and last layer prediction output modules, the feature size and number of channels will not be changed. The training of the student model requires a trained teacher model. The depth of the student model is only half of that of the teacher model. At the same time, the convolution module of the student model only contains two ordinary convolution operations, and does not contain multi-scale convolution modules and latent content activation modules. The loss of the student model consists of two parts, direct loss and knowledge distillation loss.

[0060] It should be noted that the construction of the training set comes from the simulation data of the experimental data set. XCAD is an X-ray-based measured blood vessel data set. Using the segmentation mask of the blood vessel and the forward simulation method of MPI, the MPI signal and reconstructed image of the corresponding blood vessel phantom were simulated. White noise (20dB) was added to the MPI signal and the noise image was reconstructed as the input of the model, and the image reconstructed by the signal without noise was the clean image. The simulation scanning time for each image was 0.01s. XCAD contains a total of 126 images, each with a size of (512, 512). When generating data, a part of each image was randomly cropped (160, 160) and then scaled to (40, 40). This process was repeated 100 times for each image, and the signal corresponding to the image was obtained after processing. For the original 126 samples, 120 samples were randomly selected as the training set, 3 samples were selected as the validation set, and 3 samples were selected as the test set. Since 100 data were randomly simulated for each sample, the final training set contains 12,000 sample data, and the validation set and test set contain 300 sample data.

[0061] The present invention improves the temporal resolution of imaging by driving image modeling with latent content, and compresses the model by using knowledge distillation technology based on attention mask, thus achieving a good balance between image reconstruction performance and inference speed.

[0062] Although the various steps in the above embodiment are described in the above-mentioned order, those skilled in the art can understand that in order to achieve the effect of this embodiment, different steps do not have to be executed in such an order. They can be executed simultaneously (in parallel) or in a reverse order. These simple changes are within the scope of protection of the present invention.

[0063] Based on the above embodiment, the magnetic particle image sparse scanning reconstruction model is obtained by training a teacher model including a multi-scale latent content driving module and a student model including a knowledge distillation loss function in sequence based on the selected training sample data, and includes:

[0064] Randomly selecting sample data of a training set, and inputting the sample data of the training set into the teacher model including the multi-scale latent content driving module for training, so as to update the parameters of the teacher model;

[0065] Specifically, the steps for training the teacher model are as follows:

[0066] Step 1: Input a set of images corresponding to noisy MPI signals and clean MPI signals as the training set, and randomly select batch data for training.

[0067] Step 2: Calculate the multi-scale spatial attention mask based on the noise input. For each sample, first convolve the signal with convolution kernels of different scales. Then, on each scale output, calculate the average and maximum value of a single signal point on the channel. Finally, use the activation function to determine the saliency of each signal point, and add and average the multi-scale saliency masks to get the final attention mask.

[0068] Step 3: Calculate the loss. The multi-layer convolutional model finally outputs a reconstructed high-quality MPI image, which is compared with the actual MPI image averaged over multiple measurements to calculate the loss and update the model parameters.

[0069] The sample data of the randomly selected training set is input into the student model including the knowledge distillation loss function, and the parameters of the student model are updated based on the total loss function of the student model to obtain the sparse scanning and reconstruction model of the magnetic particle image.

[0070] Specifically, the steps for training the student model are as follows:

[0071] Step 1: Input the training set and randomly select batch data for training.

[0072] Step 2: Based on the trained teacher model, the attention mask of each sample is obtained for the final loss function calculation.

[0073] Step 3: For each sample, calculate the intermediate hidden vector expression of the sample in the teacher model and the student model. The hidden vector contains two parts, namely the noise estimation and the hidden layer output, which are used for the next step of loss function calculation.

[0074] Step 4: The loss function is divided into three parts. The first part is the direct loss, that is, the loss calculated by comparing the output of the student model with the actual MPI image averaged by multiple measurements. This part is consistent with the teacher model. The second part is the noise similarity loss. For the noise estimate obtained in the third step, the attention mask obtained in the second step is used to calculate the spatial similarity map of the noise estimate expression, and the similarity map of the student model is made close to the similarity map of the teacher model, thereby obtaining the noise similarity loss. The third part is the hidden layer similarity loss, which is similar to the noise similarity loss. The spatial similarity map of the hidden layer output is calculated, and the hidden layer similarity loss is obtained. Finally, all loss functions are used to update the student model parameters, and the teacher model parameters are always fixed.

[0075] Based on the above embodiment, the step of inputting the sample data of the training set into the teacher model including the multi-scale latent content driving module for training to update the parameters of the teacher model includes the following steps S201-S204:

[0076] Step S201, inputting the sample data of the training set into parallel multi-scale convolution kernels, outputting intermediate feature signals of convolution kernels of different scales respectively, and splicing the intermediate feature signals of convolution kernels of different scales into a composite feature signal;

[0077] For example, the input signal will first go through a parallel multi-scale convolution process, with convolution kernel sizes of 1, 3, 5, and 7, respectively, outputting intermediate features under different scale convolutions, which will then be concatenated into a composite feature signal. l Represents the input feature signal of the lth layer, then the process can be expressed as:

[0078] F l (k) =Conv k (F l ); (1)

[0079]

[0080] in, represents the composite feature body signal of the lth layer, || is the concatenation operator, Conv k It is a convolution operator with a kernel size of k.

[0081] Step S202, inputting the intermediate feature signals of the convolution kernels of different scales into the latent content activation module to calculate the attention mask signals of different scales, and obtaining the activation map of the multi-scale composite signal based on the average values ​​of the attention mask signals of different scales;

[0082] For example, for the feature signals at different convolution scales mentioned above The attention mask signals of different scales will be calculated through a latent content activation (LCA) module. For the feature signal input to LCA, the average and maximum values ​​of each point will be calculated in the channel dimension to obtain two new feature maps with a channel number of 1. Then these two feature signals are concatenated and input into a convolution layer with a convolution kernel size of 7. Finally, the sigmoid function is used for activation to calculate the attention mask signal. The calculation process can be expressed as:

[0083]

[0084] in, represents the attention mask signal of the lth layer at the k-th convolution scale.

[0085] Each scale of the intermediate feature signal is input into a different LCA module to calculate the attention signal under the scale feature, and the four attention signals are averaged to obtain the final activation map of the multi-scale composite signal, that is:

[0086]

[0087] Step S203, performing point-to-point multiplication on the composite feature signal and the activation map of the multi-scale composite signal, and taking the corresponding multi-scale features of each layer as residuals after a convolution operation to obtain a final output feature signal;

[0088] Specifically, after obtaining the composite feature signal and the composite signal activation map, we perform point-to-point multiplication on the two, enhance the representation in the feature signal according to the important areas in the activation map, and suppress the expression in the unimportant areas. Then, we perform another convolution operation on the output result, and add the multi-scale features of this layer as the residual connection as the final output. The calculation process can be expressed as:

[0089]

[0090] Among them, F′ l This is the final output feature signal of this layer.

[0091] Step S204, multiplying the final output feature signal by the pseudo-inverse of the system matrix, and obtaining a predicted image through two layers of convolution, obtaining the loss function of the teacher model based on the predicted image, and updating the parameters of the teacher model based on the loss function of the teacher model.

[0092] Specifically, after multiple stacking of feature signals, the prediction output of the last layer is multiplied by the pseudo-inverse of the system matrix, and the final prediction result is obtained through two layers of convolution modules, namely:

[0093]

[0094]

[0095] Among them, F′ L is the output feature signal of the last layer of MLCD, and the prediction output module is a convolution layer with a convolution kernel size of 1. is the predicted image.

[0096] Based on the above embodiment, the loss function based on the teacher model updates the parameters of the teacher model, and the formula is as follows:

[0097]

[0098]

[0099] Where η is the learning rate, is the loss function of the teacher model, θ T are the parameters of the teacher model, is the predicted image, x c is the image corresponding to the sample data of the training set, || 1 It is an absolute value operator, indicating L1 loss.

[0100] In other words, the final loss function of the teacher model is called L1 loss, and it is used to update the teacher model parameters, namely:

[0101]

[0102]

[0103] Based on the above embodiment, the total loss function of the student model includes a direct loss function and a knowledge distillation loss function; wherein the knowledge distillation loss function includes a feature similarity loss function and a spatial similarity graph loss function, and the direct loss function is the loss function of the teacher model.

[0104] Specifically, the total loss function of the student model is expressed as:

[0105]

[0106] The total loss function is used to update the student model parameters, while the parameters of the teacher model remain fixed.

[0107] Among them, the direct loss is the difference between the signal predicted by the student model and the clean signal || 1 The L1 loss obtained by calculation is consistent with the teacher model. The focus is on the knowledge distillation loss It consists of two parts: feature similarity loss and spatial similarity loss This loss comes from the output of the middle hidden layer of the two models. Since the number of layers of the student model is only half of that of the teacher model, the output of each layer of the student model is aligned with the output of the next two layers of the teacher model. That is, if is the feature of the first layer of the student model, then the feature of the 21th layer of the teacher model needs to be used to calculate the loss.

[0108] Based on the above embodiment, the formula of the feature similarity loss function is as follows:

[0109]

[0110] in, is the feature similarity loss function, F′ T are the multi-scale convolution output and the final output of the preset layer of the teacher model, respectively. is an activation map of the multi-scale composite signal corresponding to the preset layer of the teacher model, The preset layer module of the student model only contains two consecutive simple convolutions, respectively, corresponding to the output, || 1 is the absolute value operator.

[0111] For example, now without considering the number of layers, remember F′ T are the multi-scale convolution output and final output of a layer of the teacher model, respectively. is the activation map of the multi-scale composite signal corresponding to this layer. Similarly, the student model layer module only contains two consecutive simple convolutions, so are the corresponding outputs respectively.

[0112] The feature similarity loss The calculation formula is:

[0113]

[0114] Based on the above embodiment, the formula of the spatial similarity graph loss function is as follows:

[0115]

[0116] in, is the spatial similarity graph loss function, G′ T is the signal similarity graph of the hidden layer feature graph of the teacher model, G′ T =f′ T ×Transpose(f′ T ), is the activation map of the multi-scale composite signal corresponding to the preset layer of the teacher model, F′ T is the final output of the multi-scale convolution of the preset layer of the teacher model, G′ S is the signal similarity graph of the hidden layer feature graph of the student model, G′ S =f′ S ×Transpose(f′ S ), is the activation map of the multi-scale composite signal corresponding to the preset layer of the student model, F′ S The final output of the multi-scale convolution of the preset layer of the student model, is a similarity graph of the multi-scale composite signal of the teacher model, is the similarity graph of the multi-scale composite signal of the student model, is the mean square error calculator.

[0117] For example, for the spatial similarity map loss, the spatial similarity map of each feature is calculated. T As an example, its dimension is (c, h, 1). As with feature similarity loss, we first use Multiply with the feature points, that is:

[0118]

[0119] Transpose and multiply the one-dimensional matrix to obtain the similarity graph G′ T :

[0120] G′ T =f′ T ×Transpose(f′ T ); (17)

[0121] Then the similarity graph loss between the teacher model and the student model can be expressed as:

[0122]

[0123] The magnetic particle image sparse scanning and reconstruction system of the second embodiment of the present invention is as follows: Figure 3 As shown, including:

[0124] An image determination unit 301 is used to determine a magnetic particle image to be reconstructed by sparse scanning;

[0125] An image reconstruction unit 302 is used to input the magnetic particle image to be sparsely scanned and reconstructed into a sparse scanning and reconstruction model of a magnetic particle image, and obtain the sparsely scanned and reconstructed magnetic particle image output by the sparse scanning and reconstruction model of the magnetic particle image;

[0126] The magnetic particle image sparse scanning reconstruction model is obtained by training a teacher model including a multi-scale latent content driving module and a student model including a knowledge distillation loss function in sequence based on sample data of a selected training set.

[0127] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process and related instructions of the system described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0128] It should be noted that the magnetic particle image sparse scanning and reconstruction system provided in the above embodiment is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the modules or steps in the embodiments of the present invention can be decomposed or combined. For example, the modules in the above embodiment can be combined into one module, or further divided into multiple sub-modules to complete all or part of the functions described above. The names of the modules and steps involved in the embodiments of the present invention are only for distinguishing the modules or steps, and are not regarded as improper limitations of the present invention.

[0129] An electronic device according to a third embodiment of the present invention includes:

[0130] at least one processor;

[0131] and a memory communicatively coupled to at least one of the processors;

[0132] The memory stores instructions executable by the processor, and the instructions are used to be executed by the processor to implement the above-mentioned magnetic particle image sparse scanning and reconstruction method.

[0133] A fourth embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to be executed by the computer to implement the above-mentioned magnetic particle image sparse scanning and reconstruction method.

[0134] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process and related instructions of the storage device and processing device described above can refer to the corresponding process in the aforementioned method embodiment and will not be repeated here.

[0135] Those skilled in the art should be able to appreciate that the modules and method steps of each example described in conjunction with the embodiments disclosed herein can be implemented with electronic hardware, computer software or a combination of the two, and the programs corresponding to the software modules and method steps can be placed in random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs, or any other form of storage medium known in the technical field. In order to clearly illustrate the interchangeability of electronic hardware and software, the composition and steps of each example have been generally described in the above description according to the function. Whether these functions are performed in electronic hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0136] Reference below Figure 4 , which shows a schematic diagram of the structure of a computer system of a server for implementing the method, system, and device embodiments of the present application. Figure 4 The server shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0137] like Figure 4 As shown, the computer system includes a central processing unit (CPU) 401, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 402 or the program loaded from the storage part 408 to the random access memory (RAM) 403. Various programs and data required for system operation are also stored in the RAM 403. The CPU 401, ROM 402 and RAM 403 are connected to each other through a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0138] The following components are connected to the I / O interface 405: an input section 406 including a keyboard, a mouse, etc.; an output section 407 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker; a storage section 408 including a hard disk, etc.; and a communication section 409 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 409 performs communication processing via a network such as the Internet. A drive 410 is also connected to the I / O interface 405 as needed. A removable medium 411, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 410 as needed so that a computer program read therefrom is installed into the storage section 408 as needed.

[0139] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 409, and / or installed from the removable medium 411. When the computer program is executed by the central processing unit (CPU) 401, the above-mentioned functions defined in the method of the present application are executed. It should be noted that the above-mentioned computer-readable medium of the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, - but not limited to - a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection with one or more conductors, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, an apparatus or a device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in combination with an instruction execution system, an apparatus or a device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0140] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0141] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0142] The terms "first", "second", etc. are used to distinguish similar objects rather than to describe or indicate a particular order or sequence.

[0143] The term "comprise" or any other similar term is intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus / device that includes a list of elements includes not only those elements but also other elements not expressly listed, or also includes elements inherent to such process, method, article, or apparatus / device.

[0144] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

Claims

1. A method for sparse scanning and reconstruction of magnetic particle images, characterized in that: include: determining a magnetic particle image to be reconstructed by sparse scanning; Inputting the magnetic particle image to be sparsely scanned and reconstructed into a sparse scanning and reconstruction model of a magnetic particle image to obtain a sparsely scanned and reconstructed magnetic particle image output by the sparse scanning and reconstruction model of the magnetic particle image; The magnetic particle image sparse scanning reconstruction model is obtained by training a teacher model including a multi-scale latent content driving module and a student model including a knowledge distillation loss function in sequence based on sample data of a selected training set.

2. The magnetic particle image sparse scanning and reconstruction method according to claim 1, characterized in that: The magnetic particle image sparse scanning reconstruction model is obtained by training a teacher model including a multi-scale latent content driving module and a student model including a knowledge distillation loss function in sequence based on selected training sample data, and includes: Randomly selecting sample data of a training set, and inputting the sample data of the training set into the teacher model including the multi-scale latent content driving module for training, so as to update the parameters of the teacher model; The sample data of the randomly selected training set is input into the student model including the knowledge distillation loss function, and the parameters of the student model are updated based on the total loss function of the student model to obtain the sparse scanning and reconstruction model of the magnetic particle image.

3. The magnetic particle image sparse scanning and reconstruction method according to claim 2, characterized in that: The step of inputting the sample data of the training set into the teacher model including the multi-scale latent content driving module for training to update the parameters of the teacher model comprises: Inputting the sample data of the training set into parallel multi-scale convolution kernels, outputting intermediate feature signals of convolution kernels of different scales respectively, and splicing the intermediate feature signals of convolution kernels of different scales into a composite feature signal; Inputting the intermediate feature signals of the convolution kernels of different scales into the latent content activation module to calculate the attention mask signals of different scales, and obtaining the activation map of the multi-scale composite signal based on the average value of the attention mask signals of different scales; Performing point-to-point multiplication of the composite feature signal and the activation map of the multi-scale composite signal, and taking the corresponding multi-scale features of each layer as residuals after a convolution operation to obtain a final output feature signal; The final output feature signal is multiplied by the pseudo-inverse of the system matrix, and a predicted image is obtained through two layers of convolution. The loss function of the teacher model is obtained based on the predicted image, and the parameters of the teacher model are updated based on the loss function of the teacher model.

4. The magnetic particle image sparse scanning and reconstruction method according to claim 3, characterized in that: The loss function based on the teacher model updates the parameters of the teacher model, and the formula is as follows: Where η is the learning rate, is the loss function of the teacher model, θ T are the parameters of the teacher model, is the predicted image, x c is the image corresponding to the sample data of the training set, and ||1 is the absolute value operator.

5. The magnetic particle image sparse scanning and reconstruction method according to claim 2, 3 or 4, characterized in that: The total loss function of the student model includes a direct loss function and a knowledge distillation loss function; wherein the knowledge distillation loss function includes a feature similarity loss function and a spatial similarity graph loss function, and the direct loss function is the loss function of the teacher model.

6. The magnetic particle image sparse scanning and reconstruction method according to claim 2, characterized in that: The formula of the feature similarity loss function is as follows: in, is the feature similarity loss function, F′ T are the multi-scale convolution output and the final output of the preset layer of the teacher model, respectively. is an activation map of the multi-scale composite signal corresponding to the preset layer of the teacher model, The preset layer module of the student model only contains the corresponding outputs of two consecutive simple convolutions, and ||1 is an absolute value operator.

7. The magnetic particle image sparse scanning and reconstruction method according to claim 2, characterized in that: The formula of the spatial similarity graph loss function is as follows: in, is the spatial similarity graph loss function, G′ T is the signal similarity graph of the hidden layer feature graph of the teacher model, G′ T =f′ T ×Transpose(f′ T ), is the activation map of the multi-scale composite signal corresponding to the preset layer of the teacher model, F′ T is the final output of the multi-scale convolution of the preset layer of the teacher model, G′ S is the signal similarity graph of the hidden layer feature graph of the student model, G′ S =f′ S ×Transpose(f′ S ), is the activation map of the multi-scale composite signal corresponding to the preset layer of the student model, F′ S The final output of the multi-scale convolution of the preset layer of the student model, is a similarity graph of the multi-scale composite signal of the teacher model, is the similarity graph of the multi-scale composite signal of the student model, is the mean square error calculator.

8. A magnetic particle image sparse scanning and reconstruction system, characterized in that: include: An image determination unit, used for determining a magnetic particle image to be reconstructed by sparse scanning; An image reconstruction unit, used for inputting the magnetic particle image to be sparsely scanned and reconstructed into a sparse scanning and reconstruction model of a magnetic particle image, and obtaining the sparsely scanned and reconstructed magnetic particle image output by the sparse scanning and reconstruction model of the magnetic particle image; The magnetic particle image sparse scanning reconstruction model is obtained by training a teacher model including a multi-scale latent content driving module and a student model including a knowledge distillation loss function in sequence based on sample data of a selected training set.

9. An electronic device, characterized in that: include: at least one processor; and a memory communicatively coupled to at least one of the processors; The memory stores instructions executable by the processor, and the instructions are used to be executed by the processor to implement the magnetic particle image sparse scanning and reconstruction method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to be executed by the computer to implement the magnetic particle image sparse scanning and reconstruction method according to any one of claims 1 to 7.

Images