A knowledge representation learning method for nanopore single-molecule sensing signals

Abstract

The present invention discloses a knowledge representation learning method for nanopore single-molecule sensing signals, which is a learning method based on instance discrimination transformer, and is applicable to machine learning and nanopore single-molecule sensing technology; the method of the present invention includes: constructing a transformer-based nanopore single-molecule sensing signal knowledge representation learning neural network model; creating a nanopore single-molecule sensing signal data set and performing appropriate preprocessing on the signal data; setting the initial values ​​of each parameter in the neural network model, training the model with the goal of minimizing the loss function to achieve instance-level discrimination for nanopore single-molecule sensing signals; inputting the sensing signal to be processed into the knowledge representation corresponding to the trained model output signal. The present invention adopts a transformer structure in the encoder part, realizes the sparse distribution of the signal in the knowledge representation space, and directly mines pattern information from the signal data, which can be applied to different types of downstream signal analysis tasks, and has strong versatility and effectiveness.

Description

Technical Field

[0001] The present invention belongs to the technical field of data analysis and mining, and specifically relates to a knowledge representation learning method for nanopore single-molecule perception signals, and in particular to a knowledge representation learning method for nanopore single-molecule perception signals based on an instance discriminant transformer. Background Art

[0002] In recent years, nanopore single-molecule sensing technology has received widespread attention due to its ultra-high spatial resolution and label-free detection advantages, and has broad prospects in the fields of medical diagnosis and biochemical testing. The principle of nanopore single-molecule sensing technology is based on the changes in the inhibitory current generated when molecules pass through nanopores under voltage drive. These inhibitory current changes are related to the structural characteristics and physicochemical characteristics of the molecules themselves. By analyzing the inhibitory current signal, that is, the information on the molecular pore-passing behavior, the molecular structural characteristic information can be indirectly understood. At present, the mainstream nanopore single-molecule sensing signal analysis is to extract the signal from the current recording data and then perform statistical analysis on the amplitude, duration or other statistical characteristics of the signal. The statistical characteristics of the inhibitory current signal limit the in-depth and detailed understanding of the single-molecule pore-passing behavior.

[0003] In recent years, researchers have begun to realize the limitations of statistical analysis and have improved the methodology for the analysis of single-molecule sensing signals in nanopores. Some studies have combined machine learning algorithms to classify blocking signals, but the feature information of the training model is still statistical features. For example, Wang et al. used statistical indicators such as the mean and variance of the signal to construct feature vectors and combined them with various machine learning classification algorithms such as Gradient Boost, CART, XGBoost, Random Forest, and KNN to distinguish microRNA, siRNA, tRNA, and 5S rRNA. Liu et al. combined similar features with machine learning classification algorithms such as KNN, SVM, and Decision Trees to classify and distinguish multiple proteins such as Lysozyme, Apo / Holo-myoglobin, and ACTR / NCBD. Diaz et al. used four statistical features, mean, height, levels, and dwelltime, combined with DNN, CNN, LSTM, and XGBoost algorithm models to distinguish and identify the signals of four bases A, T, C, and G passing through molybdenum disulfide solid-state nanopores. Another part of the study was artificially designed and extracted information from the signal data as feature training classification models. For example, Wei et al. extracted shapelets from the perception signal and used shapelets as features combined with cross entropy loss to build a classification model to distinguish two base sequence via signals with only a single base difference. Fu et al. used EEMD, VMD, ITD combined with the time-frequency features extracted by Hilbert Transform to train the ResNet classification model to distinguish two similar base sequence signals.

[0004] Although the existing technology has achieved certain technical results in the analysis of perceptual signals, the fundamental problem of the existing perceptual signal analysis is that it is difficult to use statistical features or artificial features to effectively describe the molecular via behavior mechanism without directly learning and mining the molecular via behavior pattern information implicit in the signal. Summary of the invention

[0005] In view of the problems existing in the prior art, the present invention discloses a knowledge representation learning method for nanopore single-molecule perception signals, and in particular, relates to a knowledge representation learning method for nanopore single-molecule perception signals based on instance discriminant transformer, which is suitable for machine learning and nanopore single-molecule perception technology.

[0006] According to the technical solution of the patent of the present invention, a method for learning knowledge representation of nanopore single-molecule perception signals is provided, which is a method for learning knowledge representation of nanopore single-molecule perception signals based on instance discriminant transformer, and is suitable for machine learning and nanopore single-molecule perception technology.

[0007] Furthermore, the nanopore single molecule sensing signal knowledge representation learning method comprises the following steps:

[0008] Step 1: Construct a transformer-based neural network model for learning knowledge representation of nanopore single-molecule sensing signals;

[0009] Step 2: Create a nanopore single-molecule sensing signal dataset and perform appropriate preprocessing on the signal data;

[0010] Step 3: Set the initial values ​​of the parameters in the neural network model, and train the model to achieve instance-level discrimination of nanopore single-molecule sensing signals with the goal of minimizing the loss function;

[0011] Step 4: Input the perception signal to be processed into the knowledge representation corresponding to the output signal of the trained model.

[0012] Among them, the neural network model in step one includes a signal embedding processing part, a transformer encoder part and a signal instance discrimination part. The signal embedding processing part is responsible for preprocessing and signal embedding conversion of the signal, and converting the signal embedding into a signal embedding form suitable for transformer processing.

[0013] Furthermore, the transformer encoder part converts the signal into corresponding knowledge representation through multi-layer neural network calculation; the signal instance discrimination part uses the knowledge representation as the signal feature to determine the instance category to which the signal belongs.

[0014] Preferably, the nanopore single molecule sensing signal data set in step 2 includes nanopore single molecule sensing signal data extracted by experimental collection or obtained by theoretical simulation. The nanopore single molecule is a protein, polypeptide, polysaccharide, DNA, RNA or other marker molecules.

[0015] Furthermore, the nanopore single-molecule sensing signal dataset in step 2 needs to be appropriately preprocessed to meet the input data form requirements for model training or knowledge representation extraction.

[0016] Preferably, in step three, the model is trained with the goal of minimizing the loss function, and the loss function is used to guide the model to mine the molecular via behavior pattern information in the signal data set to achieve instance-level discrimination of the signal.

[0017] The loss function in step 3 is defined as the following negative log-likelihood function J(θ):

[0018]

[0019] v=f θ (xi )

[0020] Among them, x i is the input signal of instance category i, f θ (·) is a multi-layer neural network model for learning signal knowledge representation, θ represents all parameters in the neural network model that need to be optimized and learned, v is the learned knowledge representation, and P(i|v) is the conditional probability that the input signal is judged to be instance category i.

[0021] Furthermore, the conditional probability P(i|v) in the loss function in step 3 uses the following non-parametric variant:

[0022]

[0023] in It measures the possibility that the learned knowledge representation v is judged as instance category i, and τ is the temperature parameter that modulates the distribution of v in the knowledge representation space.

[0024] Furthermore, the calculation of the conditional probability P(i|v) in step 3 can use additional storage to record all learned knowledge representations v, so as to reduce the computing power required for calculating the conditional probability in each training.

[0025] Compared with the prior art, the method of the present invention has the following beneficial effects:

[0026] 1. The nanopore single-molecule sensing signal knowledge representation learning method provided by the present invention converts the nanopore single-molecule sensing signal from one-dimensional sequence data to two-dimensional data form, and adopts a transformer structure in the encoder part.

[0027] 2. The method of the present invention utilizes the nanopore single-molecule perception signal dataset to train a multi-layer neural network model based on transformer design to achieve the instance discrimination task of nanopore single-molecule perception signals.

[0028] 2. The method of the present invention can obtain the knowledge representation of the corresponding signal by extracting the output of the transformer encoder part; using the multi-head self-attention mechanism of the transformer, the molecular permeation behavior pattern information in the perception signal is comprehensively and meticulously mined and learned to form a general signal knowledge representation.

[0029] 4. The knowledge representation obtained by the method of the present invention realizes the sparse distribution of signals in the knowledge representation space, and it directly mines pattern information from signal data, which can be applied to different types of downstream signal analysis tasks, and has strong versatility and effectiveness.

[0030] 5. The non-parametric softmax variant loss function constructed for the purpose of signal instance discrimination in the method of the present invention can control the sparse distribution of the perception signal in the knowledge representation space after the signal data is mapped, thereby ensuring the effectiveness of the knowledge representation finally extracted. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 is a flow chart of the nanopore single-molecule sensing signal knowledge representation learning method according to the present invention;

[0032] Figure 2 is a schematic diagram of the internal structure of the nanopore single-molecule sensing signal knowledge representation learning model according to the present invention;

[0033] Figure 3 is a schematic diagram of the structure of a transformer encoder applicable to the present invention;

[0034] Figure 4 It is a simplified diagram of data mapping conversion for implementing instance discrimination auxiliary tasks based on the nanopore single-molecule perception signal knowledge representation learning model of the present invention. DETAILED DESCRIPTION

[0035] The following will be combined with the drawings in the embodiments of the patent of the present invention to clearly and completely describe the technical solutions in the embodiments of the patent of the present invention. Obviously, the described embodiments are only part of the embodiments of the patent of the present invention, not all of them. Based on the embodiments in the patent of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the patent of the present invention.

[0036] The present invention proposes a nanopore single-molecule perception signal knowledge representation learning method, which is a nanopore single-molecule perception signal knowledge representation learning method based on an instance-discriminative transformer. The nanopore single-molecule perception signal knowledge representation learning model based on an instance-discriminative transformer introduces transformer, multi-layer perceptron, multi-head attention mechanism, layer normalization and other structures to create a multi-layer neural network model, and combines the created nanopore single-molecule perception signal data set training model to directly mine the via behavior pattern information in the nanopore single-molecule perception signal, and establish a mapping conversion model from signal to signal knowledge representation, so that the learned model can be used to extract the knowledge representation corresponding to the nanopore single-molecule perception signal, thereby realizing the sparse distribution of the single-molecule perception signal in the knowledge representation space for subsequent data analysis tasks.

[0037] A knowledge representation learning method for nanopore single-molecule perception signals, which constructs a transformer-based neural network model to achieve instance-level discrimination of perception signals as the learning goal training model, learns to mine the molecular perforation behavior pattern information in the signal data, establishes a mapping from perception signals to corresponding knowledge representation, and realizes the sparse distribution of perception signals in the knowledge representation space.

[0038] The following further describes the nanopore single molecule sensing signal knowledge representation learning method of the present invention with reference to the accompanying drawings and in combination with preferred embodiments. Figure 1 The nanopore single molecule sensing signal knowledge representation learning method according to the present invention is a nanopore single molecule sensing signal knowledge representation learning method based on an instance-based transformer. The nanopore single molecule sensing signal knowledge representation learning method comprises the following steps:

[0039] Step 1: Construct a transformer-based neural network model for learning knowledge representation of nanopore single-molecule sensing signals;

[0040] Step 2: Create a nanopore single-molecule sensing signal dataset and perform appropriate preprocessing on the signal dataset;

[0041] Step 3: Set the initial values ​​of the parameters in the neural network model, train the model with the goal of minimizing the loss function, and achieve instance-level discrimination of the nanopore single-molecule sensing signal;

[0042] Step 4: Input the perception signal to be processed into the trained model and output the knowledge representation corresponding to the signal.

[0043] The neural network model in step one includes a signal embedding processing part, a transformer encoder part and a signal instance discrimination part. The signal embedding processing part is responsible for preprocessing and signal embedding conversion of the signal, converting the signal embedding into a signal embedding form suitable for transformer processing; the transformer encoder part converts the signal into the corresponding knowledge representation through multi-layer neural network calculation; the signal instance discrimination part uses the knowledge representation as the signal feature to determine the instance category to which the signal belongs.

[0044] The nanopore single molecule sensing signal dataset in step 2 includes nanopore single molecule sensing signal data obtained by experimental collection extraction or theoretical simulation; the nanopore single molecule is a protein, polypeptide, polysaccharide, DNA, RNA or other marker molecules. Preferably, the nanopore single molecule sensing signal dataset in step 2 needs to be appropriately preprocessed for the nanopore single molecule sensing signal dataset to meet the input data form requirements for model training or knowledge representation extraction.

[0045] The nanopore single-molecule sensing signal knowledge in step three indicates that the initial values ​​of the parameters in the learning model are generated by a Gaussian random function, and the parameters of the last layer of the signal instance discrimination part are initially set to zero.

[0046] Furthermore, in step three, the model is trained with the goal of minimizing the loss function, and the loss function is used to guide the model to mine the molecular via behavior pattern information in the signal data set to achieve instance-level discrimination of the signal, that is, the training model regards N signal instances as N independent instance classes.

[0047] The loss function in step 3 is defined as the following negative log-likelihood function J(θ):

[0048]

[0049] v=f θ (x i )

[0050] Among them, x i is the input signal of instance category i, f θ (·) is a multi-layer neural network model for learning signal knowledge representation, θ represents all parameters in the neural network model that need to be optimized and learned, v is the learned knowledge representation, and P(i|v) is the conditional probability that the input signal is judged to be instance category i.

[0051] Preferably, the conditional probability P(i|v) in the loss function in step 3 uses the following non-parametric variant:

[0052]

[0053] in It measures the possibility that the learned knowledge representation v is judged as instance category i, and τ is the temperature parameter that modulates the distribution of v in the knowledge representation space.

[0054] Preferably, in step 3, the calculation of the conditional probability P(i|v) can use additional storage to record all learned knowledge representations v, so as to reduce the computing power required when calculating the conditional probability in each training.

[0055] Figure 2The figure shows the internal structure of the knowledge representation learning model of the transformer nanopore single-molecule perception signal according to the example discrimination of the present invention, which includes three parts: signal embedding processing, transformer encoder, and signal instance discrimination head. Among them, the signal embedding processing performs dimension conversion, data splitting, linear mapping and other operations on the nanopore single-molecule perception signal, and finally converts the nanopore single-molecule perception signal into a suitable embedding form as the input signal of the transformer encoder. In a preferred embodiment of the present invention, the dimension conversion is: a one-dimensional (1D) sequence nanopore single-molecule perception signal with a data length of 256 is processed and converted into a signal data in the form of two-dimensional (2D) data of 16×16 ( Figure 2 The data is split into: split the converted two-dimensional signal data into 4×4 signal image blocks of size 4×4 ( Figure 2 The linear mapping is: convert the signal image blocks obtained by data splitting into multi-dimensional data tensors ( Figure 2 The multidimensional data tensor after linear mapping conversion will add an additional category embedding as the input of the transformer encoder. The corresponding category embedding output calculated by the transformer encoder can be used as the general knowledge representation of the signal.

[0056] Figure 3 The transformer encoder structure suitable for the present invention is shown, which includes eight interconnected transformer encoding modules. Each transformer encoding module includes a multi-head self-attention calculation module, a multi-layer perceptron, two layer normalization modules and two shortcut connections. The input of each transformer encoding module is calculated and processed by the layer normalization module, the multi-head self-attention calculation module, the layer normalization module, and the multi-layer perceptron in turn. One of the shortcut connection encoding modules is input to the multi-head self-attention module calculation result and added to it to form the first residual neural network structure, and the other shortcut connection adds the addition result of the previous shortcut connection to the multi-layer perceptron calculation result to form a second residual neural network structure.

[0057] The signal instance discrimination head performs calculations based on the signal knowledge representation output by the transformer encoder to determine the instance category corresponding to the signal. In the preferred embodiment of the present invention, a multilayer perceptron is used.

[0058] In the nanopore single molecule perception signal knowledge representation learning method of the present invention, the nanopore single molecule perception signal data used in model training is obtained by collecting and extracting the nanopore single molecule perception experiment or by simulating and calculating the corresponding perception experiment. The types of the nanopore single molecule include proteins, polypeptides, polysaccharides, DNA, RNA and other marker molecules.

[0059] Furthermore, the nanopore single-molecule sensing signal data needs to be preprocessed accordingly to meet the basic requirements of the model for input data. Data preprocessing mainly includes four main processes: data cleaning, data labeling, data normalization, and data filling. Data cleaning, exploratory analysis of experimental data and data cleaning based on the analysis results to eliminate invalid signals. Invalid signals include short signals that cannot clearly reflect the characteristic information of molecular perforation behavior and ultra-long signals generated by non-perforation behavior of molecules stagnating in the nanopore for too long or finally withdrawing from the nanopore. Data labeling, sorting the extracted signal data and labeling data according to the corresponding experiments and molecular types. Data normalization is to retain the original amplitude distribution information of the signal data and reduce the computing power requirements for model training. In a preferred embodiment, the signal data is normalized using the nanopore opening current as a reference, so that the data value range is standardized to [0,1]. Data filling, the length of the input signal data is unified. In another embodiment, the data is resampled to ensure uniform length of signal data, so that the knowledge representation learning model pays more attention to the morphological features of the signal in terms of amplitude dimension.

[0060] In step three, according to the transformer nanopore single molecule perception signal knowledge representation learning model and its training parameters, the transformer neural network model is trained with the goal of achieving signal instance discrimination, and finally the construction of the nanopore single molecule perception signal knowledge representation learning model is completed.

[0061] In the method of the present invention, the auxiliary task of signal instance discrimination is to regard each perceptual signal instance in the data set as an independent class, and set the model training goal to identify and distinguish each signal instance class, that is, the model needs to achieve instance-level discrimination of signal data. Figure 4 As shown in FIG. 1 , after being processed by the signal knowledge representation learning model, the distance between different signal data instances will be magnified as much as possible in the knowledge representation space, thereby achieving sparse distribution of signal data in the knowledge representation space.

[0062] According to the method of the present invention, it is assumed that the signal data set for model training has a total of n signals x1, x2, ..., x n and n instance categories, and the knowledge representations corresponding to the signals are v1,v2,…,v n And v i =f θ (x i ). If the knowledge of the signal is expressed as v = f θ(x), the conditional probability that the signal is identified as the i-th type of signal can be expressed as P(i|v). The model learning goal can be determined as maximizing the joint probability The equivalent loss function can be defined as the following negative log-likelihood function J(θ):

[0063]

[0064] Among them, x i is the input signal of instance category i, f θ (·) is a multi-layer neural network model for learning signal knowledge representation, θ represents all parameters in the neural network model that need to be optimized, v is the learned knowledge representation, P(i|v)=P(i|f θ (x i )) is the input signal x i The conditional probability of being judged as instance category i. The softmax function can be initially selected to represent the conditional probability P(i|v):

[0065]

[0066] where w i is the weight vector parameter of instance class i, which needs to be obtained by training the model through the data set, and It measures the likelihood that v is classified as instance class i.

[0067] Therefore, we can consider w i Replace with v i And by limiting ||v||=1 through L2 normalization, the knowledge representation space learned by the model can be limited to the multi-dimensional unit sphere space, and the following non-parametric variant of softmax is obtained:

[0068]

[0069] Where τ is the temperature parameter of the knowledge distillation theory, which is used to modulate the distribution of v in the knowledge representation space. This variant does not require training weight parameters, so that the model training goal focuses on mining the characteristic information of signal data and its related knowledge representation, further ensuring the universality of knowledge representation.

[0070] The design of this loss function requires a lot of computing power when the amount of training data is large. In the preferred embodiment of the present invention, the noise contrast estimation technology is used to optimize the computational complexity of the loss function to reduce the computing power required for model training; the optimizer for neural network model training selects the Adam optimizer. When optimizing network parameters, the Adam optimizer can adaptively adjust the first-order momentum and second-order momentum of the gradient, effectively avoiding the neural network model from converging to the local optimal solution during optimization and accelerating the overall efficiency of the optimization.

[0071] In step 4, the nanopore single-molecule sensing signal to be processed is preprocessed and then input into the signal knowledge representation learning neural network model, and the knowledge representation corresponding to the signal is output.

[0072] According to the nanopore single-molecule sensing signal knowledge representation learning method of the present invention, the trained transformer neural network model processes the signal extremely quickly and can complete the knowledge representation extraction of the signal in less than 0.1 seconds under most hardware conditions, which has great advantages in practical applications.

[0073] Compared with the prior art, the nanopore single-molecule perception signal knowledge representation learning method provided by the present invention converts the nanopore single-molecule perception signal from one-dimensional sequence data to two-dimensional data form, and adopts a transformer structure in the encoder part, and uses the transformer's multi-head self-attention mechanism to comprehensively and meticulously mine and learn the molecular perforation behavior pattern information in the perception signal. The non-parametric softmax variant loss function constructed for the purpose of signal instance discrimination can control the sparse distribution of the perception signal in the knowledge representation space after the signal data is mapped, and ensure the effectiveness of the knowledge representation finally extracted.

[0074] The above is only a preferred specific implementation of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the embodiments of the present invention should be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A knowledge representation learning method for nanopore single-molecule sensing signals, characterized in that: It is a knowledge representation learning method for nanopore single-molecule sensing signals based on instance-discriminative transformer, which is applicable to machine learning and nanopore single-molecule sensing technology; The nanopore single molecule sensing signal knowledge representation learning method comprises the following steps: Step 1: Construct a transformer-based neural network model for learning knowledge representation of nanopore single-molecule sensing signals; Step 2: Create a nanopore single-molecule sensing signal dataset and perform appropriate preprocessing on the signal data; Step 3: Set the initial values ​​of the parameters in the neural network model, train the model with the goal of minimizing the loss function, use the loss function to guide the model, mine the molecular perforation behavior pattern information in the signal data set, and realize instance-level discrimination of nanopore single-molecule sensing signals; the loss function is defined as the following negative log-likelihood function J(θ): Among them, x i is the input signal of instance category i, f θ (x i ) is a multi-layer neural network model for learning signal knowledge representation, θ represents all parameters in the neural network model that need to be optimized and learned, v is the learned knowledge representation, and P(i|v) is the conditional probability that the input signal is judged to be instance category i; The conditional probability P(i|v) in the loss function uses the following non-parametric variant: in It measures the possibility that the learned knowledge representation v is judged as instance category i, and τ is the temperature parameter that modulates the distribution of v in the knowledge representation space; Step 4: Input the perception signal to be processed into the knowledge representation corresponding to the output signal of the trained model.

2. The nanopore single molecule sensing signal knowledge representation learning method according to claim 1, characterized in that: The neural network model in step one includes a signal embedding processing part, a transformer encoder part and a signal instance discrimination part. The signal embedding processing part is responsible for preprocessing the signal and converting the signal embedding into a signal embedding form suitable for processing by the transformer encoder.

3. The nanopore single molecule sensing signal knowledge representation learning method according to claim 2, characterized in that: The transformer encoder part converts the signal into the corresponding knowledge representation through multi-layer neural network calculation; the signal instance discrimination part uses the knowledge representation as the signal feature to determine the instance category to which the signal belongs.

4. The nanopore single molecule sensing signal knowledge representation learning method according to claim 1, characterized in that: The nanopore single-molecule sensing signal data set in step 2 is composed of nanopore single-molecule sensing signal data extracted by experimental collection or obtained by theoretical simulation.

5. The nanopore single molecule sensing signal knowledge representation learning method according to claim 1, characterized in that: The nanopore single molecule is a protein, polypeptide, polysaccharide, DNA, RNA or other marker molecules.

6. The nanopore single molecule sensing signal knowledge representation learning method according to claim 5, characterized in that: In step 2, appropriate preprocessing is performed on the nanopore single-molecule sensing signal dataset to meet the input data form requirements for model training or knowledge representation extraction.

Images