Nuclear magnetic resonance pure chemical shift data optimization method and device based on deep learning

Abstract

The invention discloses a nuclear magnetic resonance pure chemical shift data optimization method and device based on deep learning, and the method comprises the steps: constructing a nuclear magnetic resonance pure chemical shift data optimization model, first training data and second training data, carrying out the pre-training of the nuclear magnetic resonance pure chemical shift data optimization model through employing the first training data, and carrying out the pre-training of the second training data; a pre-trained nuclear magnetic resonance pure chemical shift data optimization model is obtained, the pre-trained nuclear magnetic resonance pure chemical shift data optimization model is finely adjusted through second training data, and a finely-adjusted nuclear magnetic resonance pure chemical shift data optimization model is obtained; and extracting an FID signal in the nuclear magnetic resonance pure chemical shift data before optimization and performing Fourier transform to obtain a corresponding Fourier transform result, and inputting the fine-tuned nuclear magnetic resonance pure chemical shift data optimization model to obtain a frequency domain signal of the optimized nuclear magnetic resonance pure chemical shift data. The problems that an existing deep learning model is insufficient in generalization and low in precision are solved.

Description

Technical Field

[0001] The present invention relates to the field of nuclear magnetic resonance spectroscopy detection, and in particular to a method and device for optimizing nuclear magnetic resonance pure chemical shift data based on deep learning. Background Art

[0002] Nuclear magnetic resonance (NMR) is a method for analyzing molecular structure and dynamics. Due to its non-invasive characteristics, it is also widely used in the fields of biology, medicine and chemical analysis. For conventional NMR spectroscopy, researchers only need to perform a simple Fourier transform operation on the FID signal collected from the magnetic resonance spectrometer to obtain the required spectrum. However, due to the limitations of conventional magnetic resonance spectroscopy methods such as high peak overlap of one-dimensional spectra and low sampling efficiency of multidimensional spectra, pure chemical shift technology has been proposed to improve the resolution of spectra. Pure chemical shift magnetic resonance spectroscopy selectively flips spin nuclei of specific frequencies to achieve the reunion of scalar coupling evolution, so that the multiple peaks formed by scalar coupling splitting in conventional spectra are simplified to single peaks, thereby improving the resolution of the spectrum. However, after being processed by pure chemical shift magnetic resonance spectroscopy, the modulation of the signal caused by the residual scalar coupling will cause some periodic pseudo peaks to appear in the spectrum, which are the so-called "block sideband pseudo peaks". In addition, in view of the long experimental time of high-dimensional magnetic resonance spectroscopy, researchers usually undersample the signal to accelerate the experiment, and then use the prior properties of the magnetic resonance spectroscopy signal such as sparsity and low rank to restore high-quality magnetic resonance spectra through corresponding signal reconstruction methods. Since undersampling will cause undersampling artifacts and reduce the signal-to-noise ratio of the magnetic resonance spectrum, removing the block sideband pseudo-peaks in the process of reconstructing the spectrum becomes a more complicated task.

[0003] At present, the mainstream methods for removing such pseudo-peaks are traditional algorithms and deep learning-based methods. Traditional algorithms rely on complex mathematical models and optimization algorithms, and are generally sensitive to noise, have large computational complexity, are time-consuming, and are overly dependent on manually designed mathematical models. The deep learning-based method uses a large amount of simulated pure chemical shift spectrum data to train the neural network, but the actual experimental conditions are complex and changeable, and the simulated data set can only represent the distribution of a small class of experimental data. The trained network often has insufficient generalization ability when dealing with different types of spectra under different experimental conditions. For example, when the experimental signal length or undersampling template is different from the training data set, there will be problems such as incomplete pseudo-peak removal or small peaks being missed. Summary of the invention

[0004] The purpose of this application is to propose a method and device for optimizing pure chemical shift data of nuclear magnetic resonance based on deep learning, in order to address the problems of insufficient generalization and low precision of existing deep learning algorithms mentioned above.

[0005] In a first aspect, the present invention provides a method for optimizing pure chemical shift data of nuclear magnetic resonance based on deep learning, comprising the following steps:

[0006] Obtain pure chemical shift data of NMR before optimization;

[0007] Constructing a nuclear magnetic resonance pure chemical shift data optimization model, first training data and second training data, wherein the nuclear magnetic resonance pure chemical shift data optimization model includes a first convolution module, a plurality of residual modules, a first convolution layer and a J-coupling modulation simulation module connected in sequence, wherein the residual module includes two second convolution modules connected in sequence, and the output feature of the first second convolution module is added to the output feature of the second second convolution module as the output feature of the residual module;

[0008] Using the first training data to pre-train the nuclear magnetic resonance pure chemical shift data optimization model to obtain a pre-trained nuclear magnetic resonance pure chemical shift data optimization model, and using the second training data to fine-tune the pre-trained nuclear magnetic resonance pure chemical shift data optimization model to obtain a fine-tuned nuclear magnetic resonance pure chemical shift data optimization model;

[0009] The FID signal in the pure chemical shift data of nuclear magnetic resonance before optimization is extracted and Fourier transformed to obtain the corresponding Fourier transform result, the Fourier transform result of the FID signal in the pure chemical shift data of nuclear magnetic resonance before optimization is input into the fine-tuned optimization model of pure chemical shift data of nuclear magnetic resonance, the Fourier transform result is input into the first convolution module to obtain the first feature, the input feature passes through several residual modules in sequence to obtain the second feature, the second feature is input into the first convolution layer to obtain the frequency domain feature, the frequency domain feature is input into the J-coupled modulation simulation module to obtain the frequency domain signal of the optimized pure chemical shift data of nuclear magnetic resonance.

[0010] Preferably, the frequency domain features are input into the J-coupling modulation simulation module, first converted to the time domain by inverse Fourier transform, and then point-by-point multiplication is performed with the equivalent envelope signal f(t) of the J-coupling modulation to generate a time domain feature that includes the simulated J-coupling modulation effect. The time domain features are then converted to the frequency domain by Fourier transform and output as the frequency domain signal of the optimized nuclear magnetic resonance pure chemical shift data.

[0011] Preferably, in the pre-training process of the NMR pure chemical shift data optimization model, the equivalent envelope signal f(t) in the J-coupling modulation simulation module is fixed, and the parameters of the first convolution module, the plurality of residual modules and the first convolution layer are adjusted; at this time, the equivalent envelope signal f(t) is set to the initial value f input (t), the expression is as follows:

[0012]

[0013] Among them, J input represents the equivalent factor of J-coupled modulation, and τ represents the duration of each data block in the pseudo-two-dimensional sampling mode, which is described by the following formula:

[0014]

[0015] Among them, SW1 represents the indirect dimensional spectrum width of pseudo two-dimensional sampling, and t2 represents the evolution time in each data block, which is calculated from the coordinate time t of the FID signal of the magnetic resonance pure chemical shift spectrum data before optimization, as shown in the following formula:

[0016] t2=(t mod τ);

[0017] Wherein, "mod" represents the modulo operation, the sampling interval of the time coordinate t is 1 / SW, and SW represents the direct dimensional spectrum width of the pseudo two-dimensional sampling;

[0018] In the fine-tuning process of the pre-trained NMR pure chemical shift data optimization model, the parameters of the equivalent envelope signal f(t) in the J-coupled modulation simulation module, the first convolution module, several residual modules and the first convolution layer are adjusted, and the equivalent envelope signal f(t) in the J-coupled modulation simulation module is adjusted from the initial value f input (t) Start adjustment.

[0019] Preferably, the construction process of the first training data is as follows:

[0020] Generate Fourier transform results and labels of FID signals of pure chemical shift data of nuclear magnetic resonance before optimization in the first training data using a mathematical model based on Fourier transform and a specific attenuation function;

[0021] The mathematical model is expressed as:

[0022]

[0023] Among them, s ideal (t) is the FID signal of the ideal pure chemical shift data of NMR, s origin (t) is the FID signal of the pure chemical shift data of NMR before optimization, N is the total number of peak components in a single sample, and for each sample, N is set to a random integer between 7 and 10; A i 、F i 、T 2i , and J i denote the amplitude, frequency, transverse relaxation time and scalar coupling constant of the ith spectral peak. For each spectral peak component, A i Set to a random number between 1 and 30, F i Set to a random number between 80 and 2048, T2i Set to a random number between 0.08 and 0.6, J i Set to a random number between 2 and 12; t1 represents the indirect dimension evolution time, j represents the imaginary unit, n(t) represents Gaussian white noise, NUS(t) represents the sampling template of the Poisson exponential distribution segment NUS, and the sampling template is shown in the following formula:

[0024]

[0025] Among them, t′ p Indicates the coordinate time corresponding to the pth sampling point, where p is the sampling point subscript, generated according to Poisson distribution, and the total number of sampling points in each sampling segment is set to 128;

[0026] The FID signal s of the ideal pure chemical shift data of NMR will be constructed origin The Fourier transform result of (t) is used as the label;

[0027] The construction process of the second training data is as follows:

[0028] The pure chemical shift data of nuclear magnetic resonance before optimization collected in the experiment are obtained, the FID signal in the pure chemical shift data of nuclear magnetic resonance before optimization is extracted and Fourier transformed to obtain the corresponding Fourier transform result, which is the Fourier transform result of the FID signal of the pure chemical shift data of nuclear magnetic resonance before optimization in the second training data.

[0029] Preferably, the loss function used in the pre-training process of the NMR pure chemical shift data optimization model is a mean square error loss function between the frequency domain signal of the optimized NMR pure chemical shift data and the label obtained by inputting the Fourier transform result of the FID signal of the NMR pure chemical shift data before optimization in the second training data into the NMR pure chemical shift data optimization model;

[0030] The loss function used in the pre-training process of the NMR pure chemical shift data optimization model is:

[0031]

[0032] in,‖·‖ F represents the F norm, ‖·‖1 represents the L1 norm, λ represents the regularization parameter, and s′ origin (t) represents the FID signal of the pure chemical shift data of NMR before optimization in the second training data, s′ predict(t) represents the frequency domain signal of the optimized pure chemical shift data of nuclear magnetic resonance obtained by inputting the Fourier transform result of the FID signal of the pure chemical shift data of nuclear magnetic resonance before optimization in the second training data into the pre-trained pure chemical shift data optimization model of nuclear magnetic resonance, and FT(·) represents the Fourier transform function.

[0033] Preferably, the convolution kernel size of the first convolution layer is 1×1; the first convolution module includes a second convolution layer and a first LeakyReLU activation function layer connected in sequence, and the convolution kernel size of the second convolution layer is 15×1;

[0034] The number of residual modules is 7, and the calculation process of the residual module is as follows:

[0035] Y k =CONV k1 (X k )+CONV k2 (CONV k1 (X k ));

[0036] Among them, X k represents the input features of the kth residual module, CONV k1 (·) represents the output features of the first second convolution module in the kth residual module, CONV k2 (·) represents the output feature of the second convolution module in the kth residual module, Y k Represents the output features of the kth residual module;

[0037] Both second convolution modules include a third convolution layer and a second LeakyReLU activation function layer connected in sequence, and the convolution kernel size of the third convolution layer is 29×1.

[0038] In a second aspect, the present invention provides a device for optimizing pure chemical shift data of nuclear magnetic resonance based on deep learning, comprising:

[0039] A data acquisition module, configured to acquire pure chemical shift data of nuclear magnetic resonance before optimization;

[0040] A model building module is configured to build a nuclear magnetic resonance pure chemical shift data optimization model, first training data and second training data, wherein the nuclear magnetic resonance pure chemical shift data optimization model includes a first convolution module, a plurality of residual modules, a first convolution layer and a J-coupling modulation simulation module connected in sequence, the residual module includes two second convolution modules connected in sequence, and the output feature of the first second convolution module is added to the output feature of the second second convolution module as the output feature of the residual module;

[0041] a training and fine-tuning module, configured to use the first training data to pre-train the nuclear magnetic resonance pure chemical shift data optimization model to obtain a pre-trained nuclear magnetic resonance pure chemical shift data optimization model, and use the second training data to fine-tune the pre-trained nuclear magnetic resonance pure chemical shift data optimization model to obtain a fine-tuned nuclear magnetic resonance pure chemical shift data optimization model;

[0042] The optimization module is configured to extract the FID signal in the pure chemical shift data of nuclear magnetic resonance before optimization and perform Fourier transform to obtain the corresponding Fourier transform result, input the Fourier transform result of the FID signal in the pure chemical shift data of nuclear magnetic resonance before optimization into the optimization model of pure chemical shift data of nuclear magnetic resonance after fine-tuning, input the Fourier transform result into the first convolution module to obtain the first feature, the input feature passes through several residual modules in sequence to obtain the second feature, the second feature is input into the first convolution layer to obtain the frequency domain feature, the frequency domain feature is input into the J-coupled modulation simulation module to obtain the frequency domain signal of the optimized pure chemical shift data of nuclear magnetic resonance.

[0043] In a third aspect, the present invention provides an electronic device comprising one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner in the first aspect.

[0044] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any implementation manner in the first aspect.

[0045] In a fifth aspect, the present invention provides a computer program product, comprising a computer program, which, when executed by a processor, implements the method described in any implementation manner in the first aspect.

[0046] Compared with the prior art, the present invention has the following beneficial effects:

[0047] (1) After the first stage of pre-training, the parameters of the NMR pure chemical shift data optimization model based on deep learning proposed in the present invention have reached good initial values. The NMR pure chemical shift data optimization model has the ability to perform preliminary optimization on pure chemical shift experimental data, including removing undersampling artifacts and pseudo peaks generated by residual J coupling modulation to a certain extent. This makes it possible to complete the second stage of single-sample customized optimization without too many iterations, thereby improving the efficiency of the algorithm and taking into account both the algorithm efficiency and the accuracy of a single sample. It is particularly suitable for efficient processing and optimization of data under complex experimental conditions.

[0048] (2) The NMR pure chemical shift data optimization method based on deep learning proposed in the present invention performs unsupervised training on the experimental data to be optimized during the fine-tuning process of the second stage, that is, customized training is further performed on individual data based on the results of the first stage network. The training process is guided by the physical model and does not require an ideal spectrum as a reference. This scheme has stronger generalization and can obtain accurate reconstruction results when used for experimental data collected under different experimental conditions.

[0049] (3) The deep learning-based NMR pure chemical shift data optimization method proposed in the present invention can complement the under-sampled experimental signal points for fast-sampled data, and can accurately remove the periodic pseudo-peaks caused by residual J-coupling modulation. It has fast processing speed, high accuracy in reconstructing spectral peaks, and strong robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0051] Figure 1 A schematic diagram of a process of optimizing a pure chemical shift data of nuclear magnetic resonance based on deep learning according to an embodiment of the present application;

[0052] Figure 2 A schematic diagram of the structure of a nuclear magnetic resonance pure chemical shift data optimization model of a nuclear magnetic resonance pure chemical shift data optimization method based on deep learning according to an embodiment of the present application;

[0053] Figure 3 A schematic diagram of a training process of a nuclear magnetic resonance pure chemical shift data optimization model of a nuclear magnetic resonance pure chemical shift data optimization method based on deep learning according to an embodiment of the present application;

[0054] Figure 4 An optimized nuclear magnetic resonance pure chemical shift spectrum obtained by processing the fully sampled nuclear magnetic resonance pure chemical shift data using the pre-trained nuclear magnetic resonance pure chemical shift data optimization model obtained through the first stage pre-training process in the nuclear magnetic resonance pure chemical shift data optimization method based on deep learning in an embodiment of the present application;

[0055] Figure 5 An optimized NMR pure chemical shift spectrum obtained by processing the fully sampled NMR pure chemical shift data using the fine-tuned NMR pure chemical shift data optimization model obtained through the second-stage fine-tuning process in the NMR pure chemical shift data optimization method based on deep learning in an embodiment of the present application;

[0056] Figure 6 An optimized nuclear magnetic resonance pure chemical shift spectrum obtained by processing under-sampled nuclear magnetic resonance pure chemical shift data using a pre-trained nuclear magnetic resonance pure chemical shift data optimization model obtained through a first-stage pre-training process in a nuclear magnetic resonance pure chemical shift data optimization method based on deep learning in an embodiment of the present application;

[0057] Figure 7 An optimized NMR pure chemical shift spectrum obtained by processing under-sampled NMR pure chemical shift data using the fine-tuned NMR pure chemical shift data optimization model obtained through the second-stage fine-tuning process in the NMR pure chemical shift data optimization method based on deep learning in an embodiment of the present application;

[0058] Figure 8 A schematic diagram of a deep learning-based NMR pure chemical shift data optimization device according to an embodiment of the present application;

[0059] Fig. 9 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0061] Figure 1 A method for optimizing pure chemical shift data of nuclear magnetic resonance based on deep learning is shown in an embodiment of the present application, comprising the following steps:

[0062] S1, obtain the pure chemical shift data of NMR before optimization.

[0063] For the convenience of description, the relevant professional terms appearing in the specific implementation methods are first explained:

[0064] FID (Free Induction Decay): Free Induction Decay;

[0065] FT (Fourier Transform): Fourier transform;

[0066] IFT (Inverse Fourier Transform): Inverse Fourier transform;

[0067] ReLU (Rectified Linear Unit): non-linear activation function;

[0068] NUS (Non-Uniform Sampling): undersampling of Poisson exponential distribution segment;

[0069] Adam parameter optimization algorithm: Adaptive Moment Estimation.

[0070] Specifically, the pure chemical shift data of nuclear magnetic resonance to be optimized is obtained and used as the pure chemical shift data of nuclear magnetic resonance before optimization. The pure chemical shift data of nuclear magnetic resonance can be obtained by experimental collection.

[0071] S2, constructing a nuclear magnetic resonance pure chemical shift data optimization model, first training data and second training data, the nuclear magnetic resonance pure chemical shift data optimization model includes a first convolution module, a plurality of residual modules, a first convolution layer and a J-coupling modulation simulation module connected in sequence, the residual module includes two second convolution modules connected in sequence, and the output feature of the first second convolution module is added to the output feature of the second second convolution module as the output feature of the residual module.

[0072] In a specific embodiment, the convolution kernel size of the first convolution layer is 1×1; the first convolution module includes a second convolution layer and a first LeakyReLU activation function layer connected in sequence, and the convolution kernel size of the second convolution layer is 15×1;

[0073] The number of residual modules is 7, and the calculation process of the residual module is as follows:

[0074] Y k =CONV k1 (X k )+CONV k2 (CONV k1 (X k ));

[0075] Among them, X k represents the input features of the kth residual module, CONV k1 (·) represents the output features of the first second convolution module in the kth residual module, CONV k2 (·) represents the output feature of the second convolution module in the kth residual module, Y k Represents the output features of the kth residual module;

[0076] Both second convolution modules include a third convolution layer and a second LeakyReLU activation function layer connected in sequence, and the convolution kernel size of the third convolution layer is 29×1.

[0077] Specifically, Figure 2 As shown, first, a NMR pure chemical shift data optimization model based on deep learning is constructed. The model includes a first convolutional module (Convolutional Block1) consisting of a second convolutional layer and a LeakyReLU activation function (ActivationFunction) layer, followed by N residual modules, each of which contains two second convolutional modules (Convolutional Block2), wherein each second convolutional module contains a third convolutional layer and a LeakyReLU activation function layer, and a first convolutional layer (Convolutional Layer) is connected after these N residual modules. The output features of the first convolutional module are input into the first residual module, the output features of the previous residual module are input into the next residual module, and the output features of the last residual module are input into the first convolutional layer. After the first convolutional layer, there is a J-coupling modulation simulation module for simulating the effect of J-coupling modulation. Specifically, the J-coupled modulation simulation module first converts the frequency domain features input to the J-coupled modulation simulation module into the time domain through inverse Fourier transform, and then performs point-by-point multiplication with the equivalent envelope signal f(t) of the J-coupled modulation to generate time domain features containing the simulated J-coupled modulation effect, and finally converts the time domain features into the frequency domain through Fourier transform and outputs the frequency domain signal, which corresponds to the Fourier transform result of the FID signal of the optimized nuclear magnetic resonance pure chemical shift data. The convolution layer in the nuclear magnetic resonance pure chemical shift data optimization model is used to extract the features of the input data, the LeakyReLU activation function layer is used to introduce nonlinear characteristics, and the residual module is used to form an effective feature fusion to solve the gradient disappearance problem in the deep network, thereby improving the training effect and performance of the network.

[0078] S3, using the first training data to pre-train the nuclear magnetic resonance pure chemical shift data optimization model to obtain a pre-trained nuclear magnetic resonance pure chemical shift data optimization model, and using the second training data to fine-tune the pre-trained nuclear magnetic resonance pure chemical shift data optimization model to obtain a fine-tuned nuclear magnetic resonance pure chemical shift data optimization model.

[0079] In a specific embodiment, during the pre-training process of the NMR pure chemical shift data optimization model, the equivalent envelope signal f(t) in the J-coupling modulation simulation module is fixed, and the parameters of the first convolution module, the plurality of residual modules and the first convolution layer are adjusted; at this time, the equivalent envelope signal f(t) is set to the initial value f input (t), the expression is as follows:

[0080]

[0081] Among them, Jinput represents the equivalent factor of J-coupled modulation, and τ represents the duration of each data block in the pseudo-two-dimensional sampling mode, which is described by the following formula:

[0082]

[0083] Among them, SW1 represents the indirect dimensional spectrum width of pseudo two-dimensional sampling, and t2 represents the evolution time in each data block, which is calculated from the coordinate time t of the FID signal of the magnetic resonance pure chemical shift spectrum data before optimization, as shown in the following formula:

[0084] t2=(t mod τ);

[0085] Wherein, "mod" represents the modulo operation, the sampling interval of the time coordinate t is 1 / SW, and SW represents the direct dimensional spectrum width of the pseudo two-dimensional sampling;

[0086] In the fine-tuning process of the pre-trained NMR pure chemical shift data optimization model, the parameters of the equivalent envelope signal f(t) in the J-coupled modulation simulation module, the first convolution module, several residual modules and the first convolution layer are adjusted, and the equivalent envelope signal f(t) in the J-coupled modulation simulation module is adjusted from the initial value f input (t) Start adjustment.

[0087] In a specific embodiment, the construction process of the first training data is as follows:

[0088] Generate Fourier transform results and labels of FID signals of pure chemical shift data of nuclear magnetic resonance before optimization in the first training data using a mathematical model based on Fourier transform and a specific attenuation function;

[0089] The mathematical model is expressed as:

[0090]

[0091] Among them, s ideal (t) is the FID signal of the ideal pure chemical shift data of NMR, s origin (t) is the FID signal of the pure chemical shift data of NMR before optimization, N is the total number of peak components in a single sample, and for each sample, N is set to a random integer between 7 and 10; A i 、F i 、T 2i , and J i denote the amplitude, frequency, transverse relaxation time and scalar coupling constant of the ith spectral peak. For each spectral peak component, A i Set to a random number between 1 and 30, F i Set to a random number between 80 and 2048, T 2iSet to a random number between 0.08 and 0.6, J i Set to a random number between 2 and 12; t1 represents the indirect dimension evolution time, j represents the imaginary unit, n(t) represents Gaussian white noise, NUS(t) represents the sampling template of the Poisson exponential distribution segment NUS, and the sampling template is shown in the following formula:

[0092]

[0093] Among them, t′ p Indicates the coordinate time corresponding to the pth sampling point, where p is the sampling point subscript, generated according to Poisson distribution, and the total number of sampling points in each sampling segment is set to 128;

[0094] The FID signal s of the ideal pure chemical shift data of NMR will be constructed origin The Fourier transform result of (t) is used as the label;

[0095] The construction process of the second training data is as follows:

[0096] The pure chemical shift data of nuclear magnetic resonance before optimization collected in the experiment are obtained, the FID signal in the pure chemical shift data of nuclear magnetic resonance before optimization is extracted and Fourier transformed to obtain the corresponding Fourier transform result, which is the Fourier transform result of the FID signal of the pure chemical shift data of nuclear magnetic resonance before optimization in the second training data.

[0097] In a specific embodiment, the loss function used in the pre-training process of the NMR pure chemical shift data optimization model is a mean square error loss function between the frequency domain signal of the optimized NMR pure chemical shift data and the label obtained by inputting the Fourier transform result of the FID signal of the NMR pure chemical shift data before optimization in the second training data into the NMR pure chemical shift data optimization model;

[0098] The loss function used in the pre-training process of the NMR pure chemical shift data optimization model is:

[0099]

[0100] in,‖·‖ F represents the F norm, ‖·‖1 represents the L1 norm, λ represents the regularization parameter, and s′ origin (t) represents the FID signal of the pure chemical shift data of NMR before optimization in the second training data, s′ predict (t) represents the frequency domain signal of the optimized pure chemical shift data of nuclear magnetic resonance obtained by inputting the Fourier transform result of the FID signal of the pure chemical shift data of nuclear magnetic resonance before optimization in the second training data into the pre-trained pure chemical shift data optimization model of nuclear magnetic resonance, and FT(·) represents the Fourier transform function.

[0101] Specifically, refer to Figure 3 , constructing the first training data for simulation according to the mathematical model of the nuclear magnetic resonance free decay (FID) signal, performing the first stage of data-driven pre-training on the nuclear magnetic resonance pure chemical shift data optimization model, and obtaining the pre-trained nuclear magnetic resonance pure chemical shift data optimization model, when the experimental signal to be optimized (the nuclear magnetic resonance pure chemical shift data before optimization collected in the experiment) is input into the pre-trained nuclear magnetic resonance pure chemical shift data optimization model, the output obtained is the frequency domain signal of the optimized nuclear magnetic resonance pure chemical shift data, and the frequency domain signal of the optimized nuclear magnetic resonance pure chemical shift data is a pseudo-peak signal that has completed partial optimization.

[0102] In terms of the construction of the first training data for simulation, in order to simulate the dynamic evolution of the FID signal, the embodiments of the present application follow the physical characteristics of the nuclear magnetic resonance signal and use a mathematical model based on Fourier transform and a specific attenuation function to generate the data. The mathematical model contains multiple frequency components and different attenuation characteristics, ensuring the consistency of the characteristics of the simulation data and the experimental data and its reliability as a training set. In order to be closer to the complexity of real data, the embodiments of the present application introduce random noise into the model and adjust the signal-to-noise ratio according to the actual acquisition conditions. In addition, considering the non-uniform undersampling technology commonly used in actual experiments, the embodiments of the present application also introduce a corresponding simulation process into the above-mentioned mathematical model. origin The Fourier transform result of (t) is used as the input signal of the model, s ideal The Fourier transform result of (t) is used as the label. The training parameters include learning rate, batch size, number of training rounds, optimization algorithm, etc.

[0103] In one embodiment, during the pre-training process of the first stage, the dimension of the first training data is S1: 5000×4096×2, where 5000 is the number of samples of the training data, 4096 is the input signal length, and 2 is the channel for distinguishing the real part and the imaginary part. The dimension of the output data is S2: 1×4096×2, where 4096 is the input signal length, and 2 is the channel for distinguishing the real part and the imaginary part. In other embodiments, the parameters can be selected according to actual needs.

[0104] In the embodiment of the present application, the kernel size of the second convolutional layer is set to 15, the number of residual modules N is set to 7, and the kernel size of the third convolutional layer is set to 29. In the embodiment of the present application, the Adam optimizer is selected, the learning rate of the Adam optimizer is set to 0.0001, and the number of training rounds is set to 100. In the pre-training process of the first stage, the number of training data samples as labels is set to 5000, the output of the model and the label are calculated by the mean square loss function (MSE) to obtain the loss value, the parameter gradient is obtained by the back propagation algorithm, and the network parameters are updated by the Adam optimization algorithm according to the calculated parameter gradient, until the loss value converges to end the pre-training.

[0105] For the obtained pre-trained NMR pure chemical shift data optimization model, single-sample customized training is performed for the NMR pure chemical shift data collected from the experiment to be optimized, and the network model parameters are fine-tuned. The input is the FID signal of the NMR pure chemical shift data to be optimized collected from a single experiment. The output of the pre-trained NMR pure chemical shift data optimization model after each iteration is substituted into the loss function for calculation, and the neural network parameters are updated through the back-propagation algorithm to minimize the loss function. The optimization process is based on a known physical model, guiding the pre-trained NMR pure chemical shift data optimization model to further learn the characteristics of the experimental data, thereby improving the adaptability and accuracy of the model under specific experimental data.

[0106] For the pure chemical shift data of nuclear magnetic resonance obtained under different experimental conditions, there is no need to repeat the pre-training process. Only a small amount of iterative optimization is performed on the specific pure chemical shift data of nuclear magnetic resonance during the fine-tuning process to achieve pure pure chemical shift spectrum output. In other words, after completing the pre-training process and obtaining the pre-trained pure chemical shift data optimization model of nuclear magnetic resonance, for various experimental signals to be optimized, only the fine-tuning process iteration with a time scale of seconds is required to obtain the frequency domain signal of the optimized pure chemical shift data of nuclear magnetic resonance. In an embodiment of the present application, the network parameters are optimized by a few-step iterative training in the fine-tuning process, and the learning rate of the Adam optimizer is set to 0.001, and the number of training rounds is set to 50.

[0107] Furthermore, the equivalent envelope signal f(t) of the J-coupled modulation in the J-coupled modulation simulation module is set as a non-trainable parameter of the model during the first stage of pre-training, and its value is always the same as the initial value f input (t) are equal, and are set as trainable parameters of the model during the second stage of fine-tuning, with slight optimization changes in the values. The parameters of the remaining structures during the pre-training process and fine-tuning process must be trainable.

[0108] S4, extract the FID signal in the pure chemical shift data of nuclear magnetic resonance before optimization and perform Fourier transform to obtain the corresponding Fourier transform result, input the Fourier transform result of the FID signal in the pure chemical shift data of nuclear magnetic resonance before optimization into the fine-tuned pure chemical shift data optimization model of nuclear magnetic resonance, input the Fourier transform result into the first convolution module to obtain the first feature, the input feature passes through several residual modules in sequence to obtain the second feature, the second feature is input into the first convolution layer to obtain the frequency domain feature, the frequency domain feature is input into the J-coupled modulation simulation module to obtain the frequency domain signal of the optimized pure chemical shift data of nuclear magnetic resonance.

[0109] In a specific embodiment, the frequency domain features are input into the J-coupling modulation simulation module, first converted to the time domain through inverse Fourier transform, and then point-by-point multiplication is performed with the equivalent envelope signal f(t) of the J-coupling modulation to generate a time domain feature that includes the simulated J-coupling modulation effect. The time domain features are then converted to the frequency domain through Fourier transform and output as the frequency domain signal of the optimized nuclear magnetic resonance pure chemical shift data.

[0110] Specifically, after the fine-tuned NMR pure chemical shift data optimization model is deployed, the Fourier transform result of the FID signal in the NMR pure chemical shift data before optimization obtained in the experiment can be input into the fine-tuned NMR pure chemical shift data optimization model to predict the frequency domain signal of the optimized NMR pure chemical shift data.

[0111] The above steps S1-S4 do not necessarily represent the order between the steps, but are represented by step symbols, and the order between the steps can be adjusted.

[0112] The present invention is described below using specific examples.

[0113] The pure chemical shift data of NMR before optimization were selected from the pure chemical shift spectrum of fully sampled NMR and the pure chemical shift spectrum of 30% NUS, respectively, to demonstrate that the present invention has the functions of removing periodic pseudo peaks caused by J coupling modulation and filling undersampled signals. In this example, the pure chemical shift spectrum of NMR of propylene glycol sample was used for optimization, and the results are shown in FIG. Figures 4 to 7 As shown, Figure 4 and Figure 5 is the result corresponding to the input of the fully sampled pure chemical shift spectrum of NMR, Figure 4 (a) and Figure 5 (a) is the pure chemical shift spectrum of NMR corresponding to the FID signal of the fully sampled propylene glycol sample collected in the experiment. Figure 4 (b) and Figure 5 (b) are the output results of the pre-trained NMR pure chemical shift data optimization model obtained after the first stage pre-training of the embodiments of the present application. Figure 6 and Figure 7 The result corresponding to the pure chemical shift spectrum of NMR with 30% NUS input is: Figure 6 (a) and Figure 7 (a) is the pure chemical shift spectrum of NMR corresponding to the FID signal of the propylene glycol sample after 30% NUS. Figure 6 (b) and Figure 7 (b) are the output results of the fine-tuned NMR pure chemical shift data optimization model obtained in the second stage of the embodiment of the present application. As can be seen from the figure, Figure 4 (b) and Figure 6 The reconstructed signals of (b) all contain obvious pseudo peaks, and the spectra are not clean and smooth enough. Figure 5 (b) and Figure 7 The reconstructed signal of (b) is a good fit to the ideal resonance pure chemical shift spectrum, with clear and complete component peaks and no spurious peaks.

[0114] Further references Figure 8 As an implementation of the methods shown in the above figures, the present application provides an embodiment of a nuclear magnetic resonance pure chemical shift data optimization device based on deep learning, and the device embodiment is similar to Figure 1 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0115] The embodiment of the present application provides a nuclear magnetic resonance pure chemical shift data optimization device based on deep learning, comprising:

[0116] A data acquisition module 1 is configured to acquire pure chemical shift data of nuclear magnetic resonance before optimization;

[0117] The model building module 2 is configured to build a nuclear magnetic resonance pure chemical shift data optimization model, first training data and second training data, wherein the nuclear magnetic resonance pure chemical shift data optimization model includes a first convolution module, a plurality of residual modules, a first convolution layer and a J-coupling modulation simulation module connected in sequence, the residual module includes two second convolution modules connected in sequence, and the output feature of the first second convolution module is added to the output feature of the second second convolution module as the output feature of the residual module;

[0118] The training and fine-tuning module 3 is configured to use the first training data to pre-train the nuclear magnetic resonance pure chemical shift data optimization model to obtain the pre-trained nuclear magnetic resonance pure chemical shift data optimization model, and use the second training data to fine-tune the pre-trained nuclear magnetic resonance pure chemical shift data optimization model to obtain the fine-tuned nuclear magnetic resonance pure chemical shift data optimization model;

[0119] The optimization module 4 is configured to extract the FID signal in the pure chemical shift data of the nuclear magnetic resonance before optimization and perform Fourier transform to obtain the corresponding Fourier transform result, input the Fourier transform result of the FID signal in the pure chemical shift data of the nuclear magnetic resonance before optimization into the optimization model of the pure chemical shift data of the nuclear magnetic resonance after fine-tuning, input the Fourier transform result into the first convolution module to obtain the first feature, the input feature passes through several residual modules in sequence to obtain the second feature, the second feature is input into the first convolution layer to obtain the frequency domain feature, the frequency domain feature is input into the J-coupled modulation simulation module to obtain the frequency domain signal of the optimized pure chemical shift data of the nuclear magnetic resonance.

[0120] Fig. 9 Schematic diagram of the hardware structure of the electronic device provided by the embodiment of the present invention. Fig. 9 As shown, the electronic device of this embodiment includes: a processor 901 and a memory 902; wherein the memory 902 is used to store computer-executable instructions; the processor 901 is used to execute the computer-executable instructions stored in the memory to implement the various steps performed by the electronic device in the above embodiment. For details, please refer to the relevant description in the above method embodiment.

[0121] Optionally, the memory 902 may be independent or integrated with the processor 901 .

[0122] When the memory 902 is independently provided, the electronic device further includes a bus 903 for connecting the memory 902 and the processor 901 .

[0123] The embodiment of the present invention further provides a computer storage medium, in which computer execution instructions are stored. When the processor 901 executes the computer execution instructions, the above method is implemented.

[0124] The embodiment of the present invention further provides a computer program product, including a computer program. When the computer program is executed by the processor 901, the above method is implemented.

[0125] In the embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules is only a logical function division, and there may be other division methods in actual implementation, such as multiple modules can be combined or integrated into another system, or some features can be ignored or not executed. Another point, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.

[0126] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to implement the solution of this embodiment.

[0127] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, each module may exist physically separately, or two or more modules may be integrated into one unit. The unit formed by the above modules may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0128] The above-mentioned integrated module implemented in the form of a software function module can be stored in a computer-readable storage medium. The above-mentioned software function module is stored in a storage medium, including a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor 901 to perform some steps of the methods of various embodiments of the present application.

[0129] It should be understood that the processor 901 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or the processor 901 may be any conventional processor 901, etc. The steps of the method disclosed in the invention may be directly embodied in the hardware processor 901 for execution, or may be executed by a combination of hardware and software modules in the processor 901.

[0130] The memory 902 may include a high-speed RAM memory, and may also include a non-volatile storage NVM, such as at least one disk memory, and may also be a USB flash drive, a mobile hard disk, a read-only memory, a magnetic disk, or an optical disk.

[0131] The bus 903 may be an Industry Standard Architecture (ISA), a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus 903 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus 903 in the drawings of the present application is not limited to only one bus 903 or one type of bus 903.

[0132] The above storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The storage medium can be any available medium that can be accessed by a general or special purpose computer.

[0133] An exemplary storage medium is coupled to the processor 901, so that the processor 901 can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor 901. The processor 901 and the storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor 901 and the storage medium can also exist as discrete components in an electronic device or a main control device.

[0134] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0135] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for optimizing pure chemical shift data of nuclear magnetic resonance based on deep learning, characterized in that: The following steps are involved: Obtain pure chemical shift data of NMR before optimization; Constructing a nuclear magnetic resonance pure chemical shift data optimization model, first training data and second training data, wherein the nuclear magnetic resonance pure chemical shift data optimization model includes a first convolution module, a plurality of residual modules, a first convolution layer and a J-coupling modulation simulation module connected in sequence, wherein the residual module includes two second convolution modules connected in sequence, and the output feature of the first second convolution module is added to the output feature of the second second convolution module as the output feature of the residual module; Using the first training data to pre-train the nuclear magnetic resonance pure chemical shift data optimization model to obtain a pre-trained nuclear magnetic resonance pure chemical shift data optimization model, and using the second training data to fine-tune the pre-trained nuclear magnetic resonance pure chemical shift data optimization model to obtain a fine-tuned nuclear magnetic resonance pure chemical shift data optimization model; Extract the FID signal in the nuclear magnetic resonance pure chemical shift data before the optimization and perform Fourier transform to obtain the corresponding Fourier transform result, input the Fourier transform result of the FID signal in the nuclear magnetic resonance pure chemical shift data before the optimization into the nuclear magnetic resonance pure chemical shift data optimization model after fine-tuning, input the Fourier transform result into a first convolution module to obtain a first feature, the input feature passes through a plurality of residual modules in sequence to obtain a second feature, the second feature is input into a first convolution layer to obtain a frequency domain feature, the frequency domain feature is input into a J-coupled modulation simulation module to obtain a frequency domain signal of the optimized nuclear magnetic resonance pure chemical shift data.

2. The method for optimizing pure chemical shift data of nuclear magnetic resonance based on deep learning according to claim 1, characterized in that: The frequency domain features are input into the J-coupling modulation simulation module, first converted to the time domain by inverse Fourier transform, and then point-by-point multiplication is performed with the equivalent envelope signal f(t) of the J-coupling modulation to generate a time domain feature containing the simulated J-coupling modulation effect. The time domain features are then converted to the frequency domain by Fourier transform and then output to obtain a frequency domain signal of the optimized nuclear magnetic resonance pure chemical shift data.

3. The method for optimizing pure chemical shift data of nuclear magnetic resonance based on deep learning according to claim 2, characterized in that: In the pre-training process of the NMR pure chemical shift data optimization model, the equivalent envelope signal f(t) in the J-coupling modulation simulation module is fixed, and the parameters of the first convolution module, the plurality of residual modules and the first convolution layer are adjusted; at this time, the equivalent envelope signal f(t) is set to the initial value f input (t), the expression is as follows: Among them, J input represents the equivalent factor of J-coupled modulation, and τ represents the duration of each data block in the pseudo-two-dimensional sampling mode, which is described by the following formula: Among them, SW1 represents the indirect dimensional spectrum width of pseudo two-dimensional sampling, and t2 represents the evolution time in each data block, which is calculated from the coordinate time t of the FID signal of the magnetic resonance pure chemical shift spectrum data before optimization, as shown in the following formula: t2=(t mod τ); Where "mod" means modulo operation, the sampling interval of time coordinate t is 1 / SW, and SW represents the direct dimensional spectrum width of pseudo two-dimensional sampling; In the fine-tuning process of the pre-trained NMR pure chemical shift data optimization model, the parameters of the equivalent envelope signal f(t), the first convolution module, the plurality of residual modules and the first convolution layer in the J-coupled modulation simulation module are adjusted, and the equivalent envelope signal f(t) in the J-coupled modulation simulation module is adjusted from the initial value f input (t) Start adjustment.

4. The method for optimizing pure chemical shift data of nuclear magnetic resonance based on deep learning according to claim 3, characterized in that: The construction process of the first training data is as follows: Generate Fourier transform results and labels of FID signals of pure chemical shift data of nuclear magnetic resonance before optimization in the first training data using a mathematical model based on Fourier transform and a specific attenuation function; The mathematical model is expressed as: Among them, s ideal (t) is the FID signal of the ideal pure chemical shift data of NMR, s origin (t) is the FID signal of the pure chemical shift data of NMR before optimization, N is the total number of peak components in a single sample, and for each sample, N is set to a random integer between 7 and 10; A i 、F i , T 2i , and J i denote the amplitude, frequency, transverse relaxation time and scalar coupling constant of the ith spectral peak. For each spectral peak component, A i Set to a random number between 1 and 30, F i Set to a random number between 80 and 2048, T 2i Set to a random number between 0.08 and 0.6, J i Set to a random number between 2 and 12; t1 represents the indirect dimension evolution time, j represents the imaginary unit, n(t) represents Gaussian white noise, NUS(t) represents the sampling template of the Poisson exponential distribution segment NUS, and the sampling template is shown in the following formula: Among them, t′ p Indicates the coordinate time corresponding to the pth sampling point, where p is the sampling point subscript, generated according to Poisson distribution, and the total number of sampling points in each sampling segment is set to 128; The FID signal s of the ideal pure chemical shift data of NMR will be constructed origin The Fourier transform result of (t) is used as the label; The construction process of the second training data is as follows: The pure chemical shift data of nuclear magnetic resonance before optimization collected in the experiment are obtained, and the FID signal in the pure chemical shift data of nuclear magnetic resonance before optimization is extracted and Fourier transformed to obtain the corresponding Fourier transform result, which is the Fourier transform result of the FID signal of the pure chemical shift data of nuclear magnetic resonance before optimization in the second training data.

5. The method for optimizing pure chemical shift data of nuclear magnetic resonance based on deep learning according to claim 4, characterized in that: The loss function used in the pre-training process of the NMR pure chemical shift data optimization model is a mean square error loss function between the frequency domain signal of the optimized NMR pure chemical shift data and the label obtained by inputting the Fourier transform result of the FID signal of the NMR pure chemical shift data before optimization in the second training data into the NMR pure chemical shift data optimization model; The loss function used in the pre-training process of the NMR pure chemical shift data optimization model is: in,‖·‖ F represents the F norm, ‖·‖1 represents the L1 norm, λ represents the regularization parameter, and s′ origin (t) represents the FID signal of the pure chemical shift data of NMR before optimization in the second training data, s′ predict (t) represents the frequency domain signal of the optimized pure chemical shift data of nuclear magnetic resonance obtained by inputting the Fourier transform result of the FID signal of the pure chemical shift data of nuclear magnetic resonance before optimization in the second training data into the pre-trained pure chemical shift data optimization model of nuclear magnetic resonance, and FT(·) represents the Fourier transform function.

6. The method for optimizing pure chemical shift data of nuclear magnetic resonance based on deep learning according to claim 1, characterized in that: The convolution kernel size of the first convolution layer is 1×1; the first convolution module includes a second convolution layer and a first LeakyReLU activation function layer connected in sequence, and the convolution kernel size of the second convolution layer is 15×1; The number of the residual modules is 7, and the calculation process of the residual module is as follows: Y k =CONV k1 (X k )+CONV k2 (CONV k1 (X k )); Among them, X k represents the input features of the kth residual module, CONV k1 (·) represents the output features of the first second convolution module in the kth residual module, CONV k2 (·) represents the output feature of the second convolution module in the kth residual module, Y k Represents the output features of the kth residual module; The two second convolution modules each include a third convolution layer and a second LeakyReLU activation function layer connected in sequence, and the convolution kernel size of the third convolution layer is 29×1.

7. A deep learning-based nuclear magnetic resonance pure chemical shift data optimization device, characterized in that: include: A data acquisition module, configured to acquire pure chemical shift data of nuclear magnetic resonance before optimization; A model building module is configured to build a nuclear magnetic resonance pure chemical shift data optimization model, first training data and second training data, wherein the nuclear magnetic resonance pure chemical shift data optimization model includes a first convolution module, a plurality of residual modules, a first convolution layer and a J-coupling modulation simulation module connected in sequence, wherein the residual module includes two second convolution modules connected in sequence, and the output feature of the first second convolution module is added to the output feature of the second second convolution module as the output feature of the residual module; a training and fine-tuning module, configured to use the first training data to pre-train the nuclear magnetic resonance pure chemical shift data optimization model to obtain a pre-trained nuclear magnetic resonance pure chemical shift data optimization model, and use the second training data to fine-tune the pre-trained nuclear magnetic resonance pure chemical shift data optimization model to obtain a fine-tuned nuclear magnetic resonance pure chemical shift data optimization model; The optimization module is configured to extract the FID signal in the pure chemical shift data of nuclear magnetic resonance before the optimization and perform Fourier transform to obtain the corresponding Fourier transform result, input the Fourier transform result of the FID signal in the pure chemical shift data of nuclear magnetic resonance before the optimization into the optimization model of the pure chemical shift data of nuclear magnetic resonance after fine-tuning, input the Fourier transform result into the first convolution module to obtain the first feature, the input feature passes through several residual modules in sequence to obtain the second feature, the second feature is input into the first convolution layer to obtain the frequency domain feature, the frequency domain feature is input into the J-coupled modulation simulation module to obtain the frequency domain signal of the optimized pure chemical shift data of nuclear magnetic resonance.

8. An electronic device, comprising: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images