Pathological DNA identification method, device, electronic device and storage medium

Abstract

The present invention provides a method, device, electronic device and storage medium for identifying diseased DNA, the method comprising: obtaining a DNA sequence to be tested and preprocessing the DNA sequence to be tested to obtain multiple DNA sequence fragments; for each DNA sequence fragment, inputting it into a trained first neural network model to output a first feature vector corresponding to the DNA sequence fragment; for each first feature vector, inputting it into a trained second neural network model to output a second feature vector corresponding to the first feature vector, and adding the first feature vector and the corresponding second feature vector to obtain a combined feature vector; inputting each combined feature vector into a trained classifier to obtain a classification result, the classification result indicating whether the input DNA sequence to be tested is a diseased DNA. The present invention can improve the accuracy of diseased DNA identification.

Description

Technical Field

[0001] The present invention relates to the field of bioinformatics and neural network application technology, and in particular to a diseased DNA recognition method, device, electronic equipment and storage medium. Background Art

[0002] Cell-free DNA (cfDNA) in body fluids such as plasma, urine, and cerebrospinal fluid is mainly degraded DNA fragments released into body fluids by apoptotic or necrotic cells. In the early stages of tissue lesions and cancer, when patients have no obvious clinical symptoms, the state of DNA in diseased cells or cancer cells has changed and will be released into body fluids. Therefore, trace amounts of circulating diseased DNA (ctDNA) can be detected in the body fluids of patients with cancer or various diseases. With the development of high-throughput sequencing technology, ctDNA is becoming a new liquid biopsy method for disease diagnosis by identifying trace amounts of disease signals in large amounts of cfDNA sequencing data.

[0003] However, the existing technical means have poor recognition ability for diseased DNA sequences. Therefore, it is necessary for the present invention to provide a diseased DNA recognition method based on a neural network. Summary of the invention

[0004] The present invention provides a diseased DNA recognition method, device, electronic device and storage medium, which are used to solve the problem that the recognition ability of diseased DNA sequences is poor and a large amount of labeled data is required in the prior art.

[0005] In a first aspect, an embodiment of the present invention provides a method for identifying diseased DNA, the method comprising:

[0006] Acquiring a DNA sequence to be tested and preprocessing the DNA sequence to be tested to obtain a plurality of DNA sequence fragments;

[0007] For each DNA sequence fragment, input it into a trained first neural network model to output a first feature vector corresponding to the DNA sequence fragment, wherein the first feature vector contains autocorrelation information between different parts in the DNA sequence fragment;

[0008] For each first feature vector, input it into the trained second neural network model to output a second feature vector corresponding to the first feature vector, and add the first feature vector and the second feature vector corresponding thereto to obtain a merged feature vector, wherein the second feature vector contains long-term and short-term dependency relationship information between different parts in the first feature vector;

[0009] Each of the combined feature vectors is input into a trained classifier to obtain a classification result, wherein the classification result indicates whether the input DNA sequence to be tested is a diseased DNA.

[0010] In one embodiment of the present invention, for each DNA sequence fragment, inputting it into a trained first neural network model to output a first feature vector corresponding to the DNA sequence fragment comprises:

[0011] Performing embedded coding on the DNA sequence fragment to obtain a first coding vector, a second coding vector, and a third coding vector having a preset fixed dimension, wherein the first coding vector contains content coding information of the DNA sequence fragment, the second coding vector contains position coding information of the DNA sequence fragment, and the third coding vector contains segment coding information;

[0012] Adding the first coding vector, the second coding vector, and the third coding vector to obtain a combined coding vector;

[0013] Performing self-attention processing on the merged encoding vector to obtain a corresponding self-attention vector;

[0014] The self-attention vector is normalized and residual processed to obtain the first feature vector corresponding to the DNA sequence fragment.

[0015] In one embodiment of the present invention, for each first feature vector, inputting it into a trained second neural network model to output a second feature vector corresponding to the first feature vector comprises:

[0016] Inputting the first feature vector as an initial vector into the trained second neural network model;

[0017] Each neural unit in the trained second neural network model performs the following operations:

[0018] Filtering the received initial vector or the vector output by the previous neural unit according to a preset rule to discard some elements;

[0019] Performing supplementation processing on the filtered vector to select information stored in the unit according to a preset rule to replace the discarded elements;

[0020] determining an output vector of the neural unit based on the supplemented vector;

[0021] Wherein, the last level of neural unit in the trained second neural network model outputs the second feature vector.

[0022] In one embodiment of the present invention, the method further comprises:

[0023] Before obtaining the DNA sequence to be tested, the first neural network model, the second neural network model and the classifier are trained in the following manner:

[0024] Collecting a data set for training, wherein the data set includes annotated diseased DNA sequences and non-lesioned DNA sequences;

[0025] Dividing the data set into a training set and a test set according to a preset ratio, and using the training set and the test set to train and test the first neural network model, the second neural network model and the classifier, respectively;

[0026] Based on the test results, the parameters of the first neural network model, the second neural network model and the classifier are adjusted until the preset accuracy requirements are reached to obtain the trained first neural network model, the second neural network model and the classifier.

[0027] In one embodiment of the present invention, before collecting the data set for training, the method further includes:

[0028] Obtaining a preset DNA sequence, wherein the preset DNA sequence is a published whole genome DNA sequence;

[0029] Preprocessing the preset DNA sequence to obtain multiple DNA sequence fragments;

[0030] The first neural network model is pre-trained using the DNA sequence fragment.

[0031] In one embodiment of the present invention, the step of preprocessing the preset DNA sequence to obtain a plurality of DNA sequence fragments comprises:

[0032] Deleting the fragment containing unknown bases in the preset DNA sequence; and

[0033] The DNA sequence obtained after deleting the unknown base is divided into the plurality of DNA sequence fragments according to a preset length.

[0034] In one embodiment of the present invention, the pre-training of the first neural network model using the DNA sequence fragments includes:

[0035] Before the DNA sequence fragment is embedded in the coding process, a portion of the bases in the DNA sequence fragment are masked, and for each masked base, the base at the masked base position is predicted based on the upstream and downstream sequences of the base, and the masked base is replaced by the predicted base.

[0036] In one embodiment of the present invention, the training of the first neural network model, the second neural network model and the classifier further includes:

[0037] In the process of training the first neural network model, the second neural network model and the classifier, the first neural network model, the second neural network model and the classifier are trained by gradient descent method and optimizing cross entropy loss function until the preset accuracy requirement is reached.

[0038] In one embodiment of the present invention, after obtaining the trained first neural network model, the second neural network model and the classifier, the method further includes:

[0039] Constructing an objective function according to a preset evaluation index to evaluate the training results of the first neural network model, the second neural network model and the classifier;

[0040] The preset evaluation index includes any combination of the following: precision, recall and accuracy.

[0041] In one embodiment of the present invention, the precision, recall and accuracy are calculated according to the following formulas:

[0042] Precision: TP / (TP+FP);

[0043] Recall rate: TP / (TP+FN);

[0044] Accuracy: (TP+TN) / (TP+FP+FN+TN);

[0045] Among them, FN means that it is identified as non-pathological DNA, but it is actually pathological DNA; FP means that it is identified as containing pathological DNA, but it is actually non-pathological DNA; TN means that it is identified as a non-pathological DNA fragment, but it is actually non-pathological DNA; TP means that it is identified as pathological DNA, but it is actually pathological DNA.

[0046] In one embodiment of the present invention, the first neural network model is a converter-based bidirectional encoding representation model, and the second network model is a long short-term memory network model.

[0047] In one embodiment of the present invention, the converter-based bidirectional encoding representation model is a neural network model based on a multi-head self-attention mechanism, and it has a 12-layer network structure, and its hidden layer has 768 dimensions.

[0048] The embodiment of the present invention further provides a diseased DNA identification device, the device comprising:

[0049] A preprocessing module, used for obtaining a DNA sequence to be tested and preprocessing the DNA sequence to be tested to obtain a plurality of DNA sequence fragments;

[0050] A first feature vector generating module is used for inputting each DNA sequence fragment into a trained first neural network model to output a first feature vector corresponding to the DNA sequence fragment, wherein the first feature vector contains autocorrelation information between different parts of the DNA sequence fragment;

[0051] A second feature vector generating module is used for inputting each first feature vector into a trained second neural network model to output a second feature vector corresponding to the first feature vector, and adding the first feature vector and the second feature vector corresponding thereto to obtain a merged feature vector, wherein the second feature vector contains information on long-term and short-term dependencies between different parts in the first feature vector;

[0052] The classification module is used to input each of the combined feature vectors into a trained classifier to obtain a classification result, wherein the classification result indicates whether the input DNA sequence to be tested is a diseased DNA.

[0053] In one embodiment of the present invention, the device further comprises:

[0054] A training module, wherein the training module is used to:

[0055] Collecting a data set for training, wherein the data set includes annotated diseased DNA sequences and non-lesioned DNA sequences;

[0056] Dividing the data set into a training set and a test set according to a preset ratio, and using the training set and the test set to train and test the first neural network model, the second neural network model and the classifier, respectively;

[0057] Based on the test results, the parameters of the first neural network model, the second neural network model and the classifier are adjusted until the preset accuracy requirements are reached to obtain the trained first neural network model, the second neural network model and the classifier.

[0058] In a third aspect, the present invention further provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of any one of the diseased DNA identification methods described in the first aspect are implemented.

[0059] In a fourth aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any one of the diseased DNA identification methods described in the first aspect.

[0060] The present invention provides a diseased DNA identification method, device, electronic device and storage medium, which adopts a pre-trained first neural network model to avoid building a model from scratch and save training time and computing resources. The diseased DNA identification method that combines the first neural model and the second neural network model can improve the accuracy of diseased DNA identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other embodiments can be obtained based on these drawings without paying creative work.

[0062] Figure 1 It is a schematic diagram of the process of the diseased DNA identification method provided by an embodiment of the present invention;

[0063] Figure 2 is a schematic diagram of pathological DNA identification provided by an embodiment of the present invention;

[0064] Figure 3 is a flowchart of a neural network model training provided by an embodiment of the present invention;

[0065] Figure 4 is a schematic diagram of attention processing provided by an embodiment of the present invention;

[0066] Figure 5 is a schematic diagram of a long short-term memory network provided by an embodiment of the present invention;

[0067] Figure 6 It is a schematic diagram of the structure of the diseased DNA identification device provided by the present invention;

[0068] Figure 7 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0069] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0070] The terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein.

[0071] The technical background of the present invention is described below:

[0072] The proportion of lesion DNA (ctDNA) in body fluid cell-free DNA (cfDNA) has been shown to be consistent with the lesion burden, so identifying ctDNA in cfDNA can be an effective method to estimate the presence and severity of lesions.

[0073] Although many studies in recent years have applied deep learning to DNA sequences, including the architecture of convolutional neural networks (CNNs), models of recurrent neural networks (RNNs), such as long short-term memory networks (LSTMs) and gated recurrent unit (GRU) networks. However, CNNs are generally unable to capture semantic dependencies in remote contexts because they have the limitation of extracting local features by the size of filters. RNN models (LSTM, GRU), although able to learn long-term dependencies to process all past states and compress contextual information into bottleneck sequences with long inputs, suffer from the problem of gradient vanishing and low efficiency when executed sequentially. In addition, most existing models require a large amount of labeled data, resulting in limited performance and applicability in data-scarce scenarios, where high-quality data with labels is expensive and time-consuming to obtain.

[0074] Based on this, the present invention provides a diseased DNA identification method, device, electronic device and storage medium to solve the problem that the prior art has poor recognition ability for diseased DNA sequences and requires a large amount of labeled data.

[0075] The present invention adopts a pre-trained first neural network model, which can avoid building a model from scratch and save training time and computing resources. By combining the diseased DNA recognition method of the first neural model and the second neural network model, the accuracy of diseased DNA recognition can be improved. At the same time, the present invention can still have a high accuracy in diseased DNA recognition when there is only a small amount of labeled data.

[0076] Combine the following Figure 1-Figure 7 The present invention describes a diseased DNA identification method, device, electronic device and storage medium.

[0077] Figure 1 is a schematic diagram of a process for identifying pathological DNA provided by an embodiment of the present invention, Figure 2 Schematic diagram of pathological DNA identification provided by an embodiment of the present invention, such as Figure 1 , Figure 2 The embodiment of the present invention provides a method for identifying diseased DNA, the method comprising:

[0078] Step 101: obtain a DNA sequence to be tested and pre-process the DNA sequence to be tested to obtain a plurality of DNA sequence fragments.

[0079] It should be noted that if the DNA sequence to be tested is relatively short (less than 512 bases in length), then the DNA sequence does not need to be preprocessed, that is, it does not need to be cut into multiple DNA sequence fragments. If the DNA sequence to be tested is relatively long (for example, greater than 512 bases), then the DNA sequence needs to be preprocessed, that is, it needs to be cut into multiple DNA sequence fragments in units of 512 bases.

[0080] Step 102: for each DNA sequence fragment, input it into a trained first neural network model to output a first feature vector corresponding to the DNA sequence fragment, wherein the first feature vector contains autocorrelation information between different parts in the DNA sequence fragment.

[0081] Exemplarily, step 102 includes:

[0082] Step 1021, embedding coding is performed on the DNA sequence fragment to obtain a first coding vector, a second coding vector, and a third coding vector having a preset fixed dimension, wherein the first coding vector contains content coding information of the DNA sequence fragment, the second coding vector contains position coding information of the DNA sequence fragment, and the third coding vector contains segment coding information.

[0083] Step 1022: Add the first coding vector, the second coding vector, and the third coding vector to obtain a merged coding vector.

[0084] Specifically, the input vector generates a word embedding vector and a fragment embedding vector for each base in each DNA sequence fragment in the DNA sequence, and the word embedding vector and the fragment embedding vector are summed to obtain the input vector, and the position embedding vector of each base in the DNA sequence is obtained through position encoding. The position embedding vector and the input vector are summed to obtain the base vector (i.e., the encoding vector) corresponding to each base in the entire DNA sequence fragment.

[0085] Step 1023, perform self-attention processing on the merged coding vector to obtain a corresponding self-attention vector.

[0086] Step 1024, normalize and perform residual processing on the self-attention vector to obtain the first feature vector corresponding to the DNA sequence fragment.

[0087] Specifically, the base vector is subjected to self-attention processing and then addition and normalization processing. The addition here refers to adding the base vector before inputting attention and the vector obtained after attention processing. The vector obtained after the addition is then normalized. The normalization here refers to layer normalization. Finally, it is transmitted through a feedforward network, and then the front and back vectors transmitted by the feedforward network are added and normalized to obtain the first feature vector.

[0088] Step 103, for each first feature vector, input it into the trained second neural network model to output a second feature vector corresponding to the first feature vector, and add the first feature vector and the corresponding second feature vector to obtain a merged feature vector, wherein the second feature vector contains long-term and short-term dependency information between different parts in the first feature vector.

[0089] Exemplarily, step 103 includes:

[0090] Step 1031: input the first feature vector as an initial vector into the trained second neural network model.

[0091] Step 1032: Each neural unit in the trained second neural network model performs the following operations:

[0092] The received initial vector or the vector output by the previous neural unit is filtered according to a preset rule to discard some elements.

[0093] The filtered vector is supplemented to select the information stored in the unit according to the preset rules to replace the discarded elements.

[0094] An output vector of the neural unit is determined based on the supplemented vector.

[0095] Wherein, the last level of neural unit in the trained second neural network model outputs the second feature vector.

[0096] Step 104: input each of the combined feature vectors into a trained classifier to obtain a classification result, wherein the classification result indicates whether the input DNA sequence to be tested is a diseased DNA.

[0097] Exemplarily, the first neural network model is a bidirectional encoder representation model from transformers (BERT), and the second network model is a long short-term memory network model (LSTM).

[0098] Before performing the above steps to predict the DNA sequence to be tested, the first neural network model, the second neural network model and the classifier need to be trained. The following specifically describes the training and testing of the first neural network model, the second neural network model and the classifier.

[0099] Figure 3 is a flowchart of a neural network model training provided by an embodiment of the present invention, such as Figure 3 Before obtaining the DNA sequence to be tested, the first neural network model, the second neural network model and the classifier are trained in the following manner:

[0100] Step 301: Collect a data set for training, wherein the data set includes labeled diseased DNA sequences and non-pathological DNA sequences.

[0101] Step 302: divide the data set into a training set and a test set according to a preset ratio, and use the training set and the test set to train and test the first neural network model, the second neural network model and the classifier, respectively.

[0102] Step 303: Adjust the parameters of the first neural network model, the second neural network model and the classifier based on the test results until the preset accuracy requirements are met to obtain the trained first neural network model, the second neural network model and the classifier.

[0103] The above steps 301 to 303 are described in detail below.

[0104] In the above step 301, the data set for training is collected, and the data set includes annotated diseased DNA sequences and non-pathological DNA sequences, including:

[0105] Before step 301, the first neural network model is pre-trained by obtaining a preset DNA sequence to obtain a preliminarily trained first neural network model, that is, the first neural network model is pre-trained. Specifically, the pre-training includes:

[0106] Step 3011, obtaining a preset DNA sequence, wherein the preset DNA sequence is a publicly available whole genome DNA sequence, and preprocessing the preset DNA sequence to obtain a plurality of DNA sequence fragments.

[0107] Exemplarily, the preset DNA sequence is the whole genome data of humans, a DNA sequence consisting of approximately 3.2 billion bases, and the DNA sequence may include adenine (A), thymine (T), cytosine (C), guanine (G), modifications on different bases (such as methylation modification of cytosine, which can be artificially represented by other unused letters) and unknown bases (N).

[0108] For example, a DNA sequence fragment is:

[0109] ATCGGCTTCATTGGCG.

[0110] The step of preprocessing the preset DNA sequence includes: deleting the fragment containing the unknown base N in the sequence, and randomly dividing the DNA sequence obtained after deleting the unknown base N into multiple DNA sequence fragments of different lengths according to preset lengths.

[0111] Therefore, the goal of preprocessing the genome data is to delete the unknown base N containing large fragments in the DNA sequence.

[0112] Exemplarily, for example, the DNA sequence is divided into multiple DNA sequence segments according to a preset length of 512 (representing 512 bases), and the preset length may also be any other length.

[0113] Step 3012: pre-train the first neural network model using the preset DNA sequence to obtain a preliminarily trained first neural network model.

[0114] Exemplarily, before embedding a DNA sequence fragment, a portion of the bases in the DNA sequence fragment are masked, and for each masked base, the base at the masked base position is predicted based on the upstream and downstream sequences of the base, and the masked base is replaced by the predicted base.

[0115] Exemplarily, the input vector is generated for each base in each DNA sequence fragment in the DNA sequence. A word embedding vector and a fragment embedding vector are summed to obtain the input vector. The position embedding vector of each base in the DNA sequence is obtained by position encoding. The position embedding vector and the input vector are summed to obtain a base vector corresponding to each base in the entire DNA sequence fragment (such as Figure 2 and Figure 4 shown).

[0116] For example, in the DNA sequence fragment "CLS AT CGGCTTCATTGGCG SEP", CLS indicates the start symbol of a DNA sequence fragment, SEP indicates the end symbol of a DNA sequence fragment, one base is one word, and 16 words are obtained. Each word is represented by a 768-dimensional vector. For example, word A is represented by a [0.001, 0.123, 0.234, ...] vector, which contains 768 floating-point numbers. It should be noted that the number of bases in a word can be set arbitrarily, such as setting two bases per word, three bases per word, and five bases per word.

[0117] Among them, the DNA sequence is processed by embedding (embedding, which can be understood as "vector"), and three embeddings are added, including Token Embeddings word embedding, Segment Embeddings segment embedding and Position Embeddings position embedding three vectors. Token Embeddings encodes each base, and Position Embeddings is a position encoding, that is, the position information of the base is added to the base vector to facilitate the distinction of bases at different positions. Segment Embeddings are all uniformly set to 0, that is, there is only one DNA sequence fragment, which means that only one DNA sequence fragment is input each time, for example, only one DNA sequence fragment with a length of 512 is input each time.

[0118] Exemplarily, the base vector is subjected to self-attention processing and then addition and normalization processing, where the addition refers to adding the base vector before inputting the attention and the vector obtained after the attention processing, and then the vector obtained after the addition is normalized, where the normalization refers to layer normalization, and finally transmitted through a feedforward network, and then the front and back vectors transmitted by the feedforward network are added and normalized to obtain a pre-training result.

[0119] Among them, the pre-training results include word vectors (i.e., the first feature vector), the output of each layer of the neural network, and the attention score. The word vectors represent the relationship between words and the relationship between each word and the DNA sequence. Each word is composed of a preset number of bases.

[0120] For example, the self-attention process is as follows Figure 4 The scaled dot product attention mechanism includes dot product, scaling, masking (optional operation), and normalization of the input query vector and key vector, and dot product of the result of the normalization operation with the value vector. The attention processing function is expressed as follows:

[0121]

[0122] Among them, the input values ​​of Q, K, and V are all base vectors, d k represents the adjustment factor, and softmax represents the normalization function.

[0123] Based on the above attention processing function, the Embedding vector of the same DNA sequence fragment is linearly transformed to obtain the Query vector (i.e. the first encoding vector), the Key vector (i.e. the second encoding vector), and the Value vector (i.e. the third encoding vector), and then the attention score is calculated (the attention score is the Attention(Q,K,V) of the above function). The specific operation is that each base vector must be linearly multiplied with the three weight matrices Wq, Wk, and Wv to generate the three vectors of Query(Q), Key(K), and Value(V) corresponding to each base.

[0124] For example: Query(Q)=Embedding*Wq; Key(K)=Embedding*Wq; Value(V)=Embedding*Wq.

[0125] The self-attention mechanism of the first neural network model pays attention to all bases in a certain base pair sequence in the DNA sequence, that is, it performs a dot product between the query vector Q of this base and the key vector K of each base. Then, the weight is normalized by the softmax function, and the dot product is performed on the value vector V to obtain the weighted sum of all bases, and the relative weight of each base to this base is calculated, which is finally used as the output vector of this base.

[0126] The feedforward network of the first neural network model inputs the vectors of all different base words output after Attention into a fully connected layer respectively to complete the feedforward network propagation.

[0127] It should be noted that the addition and standardization processing of the first neural network model is a residual network, which adds the input of the layer to its standardized output. In addition, the first neural network model also introduces a multi-head attention mechanism (Multi-Head Attention), which is equivalent to multiple Attention integration (for example, using n Attentions), and then concatenating the n outputs to obtain the final output, thereby improving the effect and stability of the algorithm. The self-attention mechanism of the first neural network model is used to obtain the corresponding relationship between each base in the DNA sequence and other bases, that is, the vector representation of the corresponding global DNA sequence semantic information. The first neural network model adopts the self-attention mechanism to overcome the shortcomings of RNN that it cannot be parallel and runs slowly. At the same time, MaskLM (Mask Language Model is referred to as Mask LM, that is, a pre-trained language model based on the Mask mechanism) is used to achieve the training of a deep bidirectional pre-trained model, so that the DNA sequence information is larger and the DNA semantic understanding is more accurate. After training the first neural network model, the pre-training result is obtained, and the model parameters corresponding to the pre-training result are saved.

[0128] Exemplarily, the model parameters are mainly some weights w, biases b and hyperparameters (the number of layers and neurons in the neural network) of each layer in the neural network.

[0129] For example, if you need to perform a linear transformation on the input vector (Embedding, also known as "embedding"), you will use the linear formula y=wx+b, where x is the input Embedding vector, and w and b are parameters.

[0130] Exemplarily, the first neural network model is a BERT model, and the second neural network model is a Long Short-Term Memory (LSTM) network model. LSTM is a special RNN that is mainly used to solve the gradient vanishing and gradient exploding problems in the long sequence training process. In short, compared with ordinary RNN, LSTM can perform better in longer sequences.

[0131] The full name of the BERT model is Bidirectional Encoder Representations from Transformers. The BERT model has a 12-layer network structure, with a hidden layer of 768 dimensions and a 12-head mode. The BERT model has the following advantages:

[0132] First, MLM (Masked Language Model) can be used to pre-train bidirectional Transformers (which is a Seq2Seq model consisting of an encoder and a decoder) to generate deep bidirectional language representations.

[0133] Second, after pre-training, the BERT model only needs to add an additional output layer for fine-tuning to achieve better performance in a variety of downstream tasks. In this process, there is no need to make task-specific structural modifications to the BERT model.

[0134] It can be seen that the first neural network model of the present invention adopts the Transformer feature extractor of the BERT model, replacing the inherent mode of the traditional commonly used CNN or RNN, and utilizing its ability to extract the dynamic features of the DNA sequence, the sequence structure features and its own advantages of parallel computing. In addition, due to its multi-head self-attention mechanism, it can obtain DNA sequence features and DNA context information with stronger correlation. Then the word vector obtained by training and the feature vector output by LSTM are summed to form a residual structure, and then the result vector obtained by the summation is combined with a softmax classifier through a fully connected layer to obtain different types of probabilities to achieve the classification of DNA sequences. The present invention can improve the accuracy and stability of DNA classification through the combined application of the first neural network model and the second neural network model, and has good feasibility and effectiveness.

[0135] Exemplarily, the DNA sequence is obtained by extracting DNA from tissue or body fluids and using a first-generation or second-generation sequencing method. For example, for liver cancer, the DNA sequence label derived from liver cancer tissue is 1, and the DNA sequence label derived from non-liver cancer tissue is 0. The length of tumor and non-lesion DNA sequences can be set to a range of, for example, 100-512, representing 100 to 512 bases. It should be noted that liver cancer tissue is not pure, and non-liver cancer cells also exist, so the DNA sequence of liver cancer tissue needs to be filtered to remove the same sequence as the DNA of non-liver cancer tissue.

[0136] In addition, the pathological DNA detection method described in the embodiment of the present invention can be used to identify various tumor diseases, such as liver cancer, lung cancer, etc. The detection method can also be used to identify non-tumor lesions, such as hepatitis, cirrhosis, etc. If a certain disease needs to be detected, the corresponding input DNA sequence of the lesion and the corresponding non-lesion DNA sequence can be used. In addition, the recognition result of the embodiment of the present invention is a binary classification result, that is, pathological DNA or non-lesion DNA.

[0137] In the above step 302, the data set is divided into a training set and a test set according to a preset ratio, and the first neural network model, the second neural network model and the classifier are trained and tested using the training set and the test set respectively.

[0138] For example, the training set and the test set may be divided into a ratio of 8:2. It is understandable that the division may also be performed in other ratios, such as 6:4. The present invention does not limit the preset ratio, which may be set according to actual conditions.

[0139] Exemplarily, the steps of respectively training and testing the first neural network model, the second neural network model, and the classifier using the training set and the test set include:

[0140] Step 3021, input the training set and the test set into the first neural network model respectively, and output the word vector (ie, the first feature vector).

[0141] Exemplarily, the word vector is a vector learned by the first neural network model, which represents the relationship between words and the relationship between each word and the DNA sequence, and each word is composed of a preset number of bases. For example, one word represents a vector, and each word is composed of one or more bases.

[0142] Step 3022, input the word vector into the second neural network model, and output a feature vector (i.e., a second feature vector), wherein the feature vector represents the learned sequence features of the diseased DNA and the non-pathological DNA.

[0143] Exemplarily, the first neural network model (e.g., BERT model) is connected to the second neural network model (e.g., LSTM model) in the following manner:

[0144] The word vector of the sequence is obtained by using the Get_Sequence_Output() method of the BERT model, that is, the word vector output by the BERT model is used as the input of the LSTM model.

[0145] Figure 5 is a schematic diagram of a long short-term memory network provided by an embodiment of the present invention, such as Figure 5As shown. After being processed by the forget gate, input gate, and output gate of the LSTM model. Among them, the forget gate is used to decide whether to retain or discard the information output by the previous unit; the memory gate is used to determine what kind of new information is stored in the unit module (i.e., neural unit). The memory gate is mainly for the attribute information discarded in the forget gate. The corresponding new attribute information is found in the current unit module and added to supplement the discarded attribute information. The output value is determined based on the information of this unit module. The data is filtered, screened, and updated through the forget gate, input gate, and output gate to extract higher-dimensional and effective DNA sequence features.

[0146] For example, Figure 5 Middle C t-1 Indicates the unit state at the previous moment (the previous moment can be understood as the previous word). t-1 Represents the output value of LSTM at the previous moment. t Represents the input value of the network at the current moment (the current moment can be understood as the current word). t represents the cell state at the current moment. ht represents the output value of LSTM at the current moment. σ represents the sigmoid function, and the formula is Tanh is the tanh activation function, the formula is f t represents the forget gate, i t represents the memory gate, O t represents the output gate, Represents the value after transformation by the tanh activation function.

[0147] Exemplarily, for example, the second neural network model outputs a 16*768-dimensional feature vector (ie, a second feature vector), and each word is represented by a 768-dimensional vector.

[0148] Step 3203, adding the word vector (i.e., the first feature vector) and the feature vector (i.e., the second feature vector) to obtain a combined feature vector, inputting the combined feature vector into the classifier, and finally outputting a test result of pathological DNA or non-pathological DNA.

[0149] For example, the word vector output by the BERT model and the feature vector output by the LSTM model are added together to form a residual structure, which can prevent gradient explosion and disappearance. Finally, the result vector obtained by the addition is input into the classifier for softmax classification, and the final output layer can obtain a binary classification result of whether it is diseased DNA.

[0150] Exemplarily, the output binary classification structure is a 2-dimensional vector, the value range of each number is [0,1], and the sum of the two numbers is 1. For example, the output binary classification results are 0.23 (indicating the probability of non-lesional DNA) and 0.77 (indicating the probability of diseased DNA). This result shows that the DNA sequence is more likely to be diseased DNA.

[0151] In the above step 303, the parameters of the first neural network model, the second neural network model and the classifier are adjusted based on the test results until the preset accuracy requirements are met, so as to obtain the trained first neural network model, the second neural network model and the classifier, including:

[0152] Step 3031, judging the test result.

[0153] Exemplarily, in the process of training the first neural network model, the second neural network model and the classifier, the first neural network model, the second neural network model and the classifier are trained by gradient descent method and optimizing cross entropy loss function until the preset accuracy requirement is reached.

[0154] Exemplarily, the expression of the cross entropy loss function is:

[0155]

[0156] Where N is the total number of training samples, that is, the sum of diseased DNA and non-lesioned DNA, y (i) represents the true label (0 or 1) of the i-th sample, Represents the predicted probability of the i-th sample predicted by the neural network. The predicted probability is in the range of [0,1]. It means summing from the 1st sample to the Nth sample.

[0157] Exemplarily, the parameters of the neural network are optimized by continuously reducing the loss function. For example, the conditions for stopping training the first neural network model, the second neural network model, and the classifier include: first, by setting the total number of training rounds, such as 500 rounds, 1000 rounds, or 10000 rounds. Second, observe the loss function. For example, when the difference between the average loss function of the first 5 rounds and the last 5 rounds is less than 0.01, the training can be stopped; or if the loss function does not decrease after 5 consecutive rounds, the training can also be stopped. The conditions for stopping training in the present invention are not limited to the above two methods. After the training is completed, the relevant parameters of the first neural network model, the second neural network model, and the classifier are saved.

[0158] Step 3032: If the test result does not reach the preset accuracy, adjust the model parameters of the first neural network model, the second neural network model and the classifier, and continue to train and test the first neural network model, the second neural network model and the classifier.

[0159] Step 3033: If the test result reaches the preset accuracy, the training is stopped to obtain the trained first neural network model, the second neural network model and the classifier.

[0160] Exemplarily, after obtaining the trained first neural network model, the second neural network model and the classifier, constructing an objective function according to a preset evaluation index to evaluate the training results of the first neural network model, the second neural network model and the classifier;

[0161] The preset evaluation index includes any combination of the following: precision, recall and accuracy.

[0162] Since the detection result of the embodiment of the present invention is a classification problem, the classification problem can be evaluated by evaluation indicators (including precision, recall and accuracy). Precision P refers to the proportion of samples predicted as positive and correctly by the classifier to all samples predicted as positive, and recall R refers to the proportion of samples predicted as positive and correctly by the classifier to all samples that are actually positive. Accuracy is the proportion of accurate predictions in all samples (including positive and negative samples). For example:

[0163] Negative samples: non-lesional DNA

[0164] Positive sample: diseased DNA

[0165] FN: False Negative, identified as a negative sample, but in fact it is a positive sample.

[0166] FP: False Positive, identified as a positive sample, but in fact it is a negative sample.

[0167] TN: True Negative, identified as a negative sample, is actually a negative sample.

[0168] TP: True Positive, identified as a positive sample, in fact, is also a proof sample.

[0169] The formula for accuracy is Accuracy = (TP + TN) / (TP + FP + FN + TN);

[0170] Definition: For a given test data set, the ratio of the number of samples correctly classified by the classifier to the total number of samples. The higher the accuracy, the better.

[0171] The formula for precision is Precision = TP / (TP+FP);

[0172] Definition: How many of the predicted positive samples are actually positive samples. The higher the accuracy, the better.

[0173] The formula for recall is Recall = TP / (TP+FN);

[0174] Definition: How many positive samples in the sample are predicted correctly. The higher the recall, the better.

[0175] Exemplarily, the trained first neural network model, the second neural network model and the classifier are used to predict the DNA sequence to be tested. The model parameters in the prediction stage are the same as the optimal model parameters in the training stage and remain unchanged. Then, the unlabeled DNA sequence to be tested is obtained, and the DNA sequence to be tested is converted into the form of a first feature vector through the trained first neural network model, and sent to the trained second neural network model to output a second feature vector, and then the first feature vector and the second feature vector are added and input into the trained classifier to obtain a prediction result, which is the probability of diseased DNA and the probability of non-pathological DNA.

[0176] It should be noted that the pathological DNA identification method provided in the embodiment of the present invention can be applied to various cancer early screening fields based on DNA sequence identification. For example, early detection of various solid tumors and blood cancers by identifying pathological DNA in plasma, early screening of brain cancer by identifying pathological DNA in cerebrospinal fluid in the brain, and early cancer screening of the urinary system by identifying pathological DNA in urine. This method can also be applied to non-cancer fields based on DNA sequence identification, such as hepatitis, cirrhosis, etc. Early detection of cirrhosis of the liver by identifying pathological DNA of cirrhosis in plasma.

[0177] In summary, the embodiment of the present invention provides a method for identifying diseased DNA by combining a first neural network model and a second neural network model, using a first neural network model (such as a BERT pre-trained language model) to represent a first feature vector at the sequence level for a DNA sequence, and inputting the obtained first feature vector into a second neural network model (such as an LSTM model) to further obtain high-level semantic features, and outputting a second feature vector. At the same time, the first feature vector output by the first neural network model and the second feature vector output by the second neural network model are jump-connected to form a residual structure to prevent gradient disappearance or gradient explosion, so that the second neural network model can more accurately understand the information of the DNA sequence and improve the accuracy of diseased DNA identification.

[0178] In addition, the embodiment of the present invention has been experimented on a public data set. Through the method of multiple training and multiple testing, not only the robustness of the model is improved, but also the recognition ability of the diseased DNA sequence is enhanced. The results show that the accuracy and recall rate of the diseased DNA recognition model proposed in the embodiment of the present invention are greatly improved. The embodiment of the present invention also uses the idea of ​​a pre-trained model (i.e., the first neural network model) to avoid building a model from scratch. By fine-tuning the first neural network model, a large amount of training time and computing resources can be saved, further improving the detection efficiency.

[0179] Therefore, the diseased DNA recognition model proposed in the embodiment of the present invention is compared with the traditional CNN and RNN models through experiments. The experimental results on public data sets show that the diseased DNA recognition method proposed in the embodiment of the present invention has higher accuracy, precision and recall rate.

[0180] The diseased DNA identification device provided by the present invention is described below. The diseased DNA identification device described below and the diseased DNA identification method described above can be referenced to each other.

[0181] Figure 6 FIG. 1 is a schematic diagram of the structure of the diseased DNA identification device provided by the present invention. Figure 6 The embodiment of the present invention provides a diseased DNA identification device 500, which includes a pre-processing module 501, a first feature vector generation module 502, a second feature vector generation module 503 and a classification module 504.

[0182] The preprocessing module 501 is used to obtain a DNA sequence to be tested and preprocess the DNA sequence to be tested to obtain a plurality of DNA sequence fragments.

[0183] The first feature vector generating module 502 is used to input each DNA sequence fragment into a trained first neural network model to output a first feature vector corresponding to the DNA sequence fragment, wherein the first feature vector contains autocorrelation information between different parts in the DNA sequence fragment.

[0184] The second feature vector generating module 503 is used to input each first feature vector into the trained second neural network model to output a second feature vector corresponding to the first feature vector, and add the first feature vector and the corresponding second feature vector to obtain a merged feature vector, wherein the second feature vector contains the long-term and short-term dependency information between different parts in the first feature vector.

[0185] The classification module 504 is used to input each of the combined feature vectors into a trained classifier to obtain a classification result, wherein the classification result indicates whether the input DNA sequence to be tested is a diseased DNA.

[0186] Exemplarily, the diseased DNA identification device further includes:

[0187] Training module 505 ( Figure 6 (not shown), the training module 505 is used for:

[0188] Collecting a data set for training, wherein the data set includes annotated diseased DNA sequences and non-lesioned DNA sequences;

[0189] Dividing the data set into a training set and a test set according to a preset ratio, and using the training set and the test set to train and test the first neural network model, the second neural network model and the classifier, respectively;

[0190] Based on the test results, the parameters of the first neural network model, the second neural network model and the classifier are adjusted until the preset accuracy requirements are reached to obtain the trained first neural network model, the second neural network model and the classifier.

[0191] Exemplarily, the first feature vector generating module 502 is further used for:

[0192] Performing embedded coding on the DNA sequence fragment to obtain a first coding vector, a second coding vector, and a third coding vector having a preset fixed dimension, wherein the first coding vector contains content coding information of the DNA sequence fragment, the second coding vector contains position coding information of the DNA sequence fragment, and the third coding vector contains segment coding information;

[0193] Adding the first coding vector, the second coding vector, and the third coding vector to obtain a combined coding vector;

[0194] Performing self-attention processing on the merged encoding vector to obtain a corresponding self-attention vector;

[0195] The self-attention vector is normalized and residual processed to obtain the first feature vector corresponding to the DNA sequence fragment.

[0196] Exemplarily, the second feature vector generating module 503 is further used for:

[0197] Inputting the first feature vector as an initial vector into the trained second neural network model;

[0198] Each neural unit in the trained second neural network model performs the following operations:

[0199] Filtering the received initial vector or the vector output by the previous neural unit according to a preset rule to discard some elements;

[0200] Performing supplementation processing on the filtered vector to select information stored in the unit according to a preset rule to replace the discarded elements;

[0201] determining an output vector of the neural unit based on the supplemented vector;

[0202] Wherein, the last level of neural unit in the trained second neural network model outputs the second feature vector.

[0203] Exemplarily, the training module 505 is further used for:

[0204] Obtaining a preset DNA sequence, wherein the preset DNA sequence is a published whole genome DNA sequence;

[0205] Preprocessing the preset DNA sequence to obtain multiple DNA sequence fragments;

[0206] The first neural network model is pre-trained using the DNA sequence fragment.

[0207] Exemplarily, the training module 505 is further used for:

[0208] Deleting the fragment containing unknown bases in the DNA sequence; and

[0209] The DNA sequence obtained after deleting the unknown base is divided into the multiple DNA sequence fragments of different lengths according to a preset length.

[0210] Exemplarily, the training module 505 is further used for:

[0211] Before the DNA sequence fragment is embedded in the coding process, a portion of the bases in the DNA sequence fragment are masked, and for each masked base, the base at the masked base position is predicted based on the upstream and downstream sequences of the base, and the masked base is replaced by the predicted base.

[0212] Exemplarily, the training module 505 is further used for:

[0213] In the process of training the first neural network model, the second neural network model and the classifier, the first neural network model, the second neural network model and the classifier are trained by gradient descent method and optimizing cross entropy loss function until the preset accuracy requirement is reached.

[0214] Exemplarily, the training module 505 is further used for:

[0215] Constructing an objective function according to a preset evaluation index to evaluate the training results of the first neural network model, the second neural network model and the classifier;

[0216] The preset evaluation index includes any combination of the following: precision, recall and accuracy.

[0217] Exemplarily, the training module 505 is further used for:

[0218] The precision, recall and accuracy are calculated according to the following formulas:

[0219] Precision: TP / (TP+FP);

[0220] Recall rate: TP / (TP+FN);

[0221] Accuracy: (TP+TN) / (TP+FP+FN+TN);

[0222] Among them, FN means that it is identified as non-pathological DNA, but it is actually pathological DNA; FP means that it is identified as containing pathological DNA, but it is actually non-pathological DNA; TN means that it is identified as a non-pathological DNA fragment, but it is actually non-pathological DNA; TP means that it is identified as pathological DNA, but it is actually pathological DNA.

[0223] Exemplarily, the first neural network model is a converter-based bidirectional encoding representation model, and the second network model is a long short-term memory network model.

[0224] Exemplarily, the converter-based bidirectional encoding representation model is a neural network model based on a multi-head self-attention mechanism, and has a 12-layer network structure, and its hidden layer has 768 dimensions.

[0225] Figure 7 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 7 As shown, the electronic device may include: a processor (Processor) 610, a communication interface (Communications Interface) 620, a memory (Memory) 630 and a communication bus 640, wherein the processor 610, the communication interface 620, and the memory 630 communicate with each other through the communication bus 640. The processor 610 may call the logic instructions in the memory 630 to execute the diseased DNA identification method, which includes:

[0226] Acquiring a DNA sequence to be tested and preprocessing the DNA sequence to be tested to obtain a plurality of DNA sequence fragments;

[0227] For each DNA sequence fragment, input it into a trained first neural network model to output a first feature vector corresponding to the DNA sequence fragment, wherein the first feature vector contains autocorrelation information between different parts in the DNA sequence fragment;

[0228] For each first feature vector, input it into the trained second neural network model to output a second feature vector corresponding to the first feature vector, and add the first feature vector and the second feature vector corresponding thereto to obtain a merged feature vector, wherein the second feature vector contains long-term and short-term dependency relationship information between different parts in the first feature vector;

[0229] Each of the combined feature vectors is input into a trained classifier to obtain a classification result, wherein the classification result indicates whether the input DNA sequence to be tested is a diseased DNA.

[0230] In addition, the logic instructions in the above-mentioned memory 630 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0231] On the other hand, the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the diseased DNA identification method provided by the above-mentioned methods.

[0232] In yet another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to perform the above-mentioned diseased DNA identification methods.

[0233] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0234] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0235] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for identifying diseased DNA, characterized in that: The method comprises: Acquiring a DNA sequence to be tested and preprocessing the DNA sequence to be tested to obtain a plurality of DNA sequence fragments; For each DNA sequence fragment, input it into a trained first neural network model to output a first feature vector corresponding to the DNA sequence fragment, wherein the first feature vector contains autocorrelation information between different parts of the DNA sequence fragment, including: performing embedded coding processing on the DNA sequence fragment to obtain a first coding vector, a second coding vector, and a third coding vector with a preset fixed dimension, wherein the first coding vector contains content coding information of the DNA sequence fragment, the second coding vector contains position coding information of the DNA sequence fragment, and the third coding vector contains segment coding information; adding the first coding vector, the second coding vector, and the third coding vector to obtain a merged coding vector; performing self-attention processing on the merged coding vector to obtain a corresponding self-attention vector; performing normalization and residual processing on the self-attention vector to obtain the first feature vector corresponding to the DNA sequence fragment; the first neural network model is a pre-trained BERT model; For each first feature vector, input it into the trained second neural network model to output a second feature vector corresponding to the first feature vector, and add the first feature vector and the second feature vector corresponding thereto to obtain a merged feature vector, wherein the second feature vector contains long-term and short-term dependency relationship information between different parts in the first feature vector; the second neural network model is an LSTM model; Each of the combined feature vectors is input into a trained classifier to obtain a classification result, wherein the classification result indicates whether the input DNA sequence to be tested is a diseased DNA; the classifier is a Softmax classifier.

2. The method for identifying diseased DNA according to claim 1, characterized in that: The step of inputting each first feature vector into a trained second neural network model to output a second feature vector corresponding to the first feature vector comprises: Inputting the first feature vector as an initial vector into the trained second neural network model; Each neural unit in the trained second neural network model performs the following operations: Filtering the received initial vector or the vector output by the previous neural unit according to a preset rule to discard some elements; Performing supplementation processing on the filtered vector to select information stored in the unit according to a preset rule to replace the discarded elements; determining an output vector of the neural unit based on the supplemented vector; Wherein, the last level of neural unit in the trained second neural network model outputs the second feature vector.

3. The method for identifying diseased DNA according to claim 1, characterized in that: The method further comprises: Before obtaining the DNA sequence to be tested, the first neural network model, the second neural network model and the classifier are trained in the following manner: Collecting a data set for training, wherein the data set includes annotated diseased DNA sequences and non-lesioned DNA sequences; Dividing the data set into a training set and a test set according to a preset ratio, and using the training set and the test set to train and test the first neural network model, the second neural network model and the classifier, respectively; Based on the test results, the parameters of the first neural network model, the second neural network model and the classifier are adjusted until the preset accuracy requirements are reached to obtain the trained first neural network model, the second neural network model and the classifier.

4. The method for identifying diseased DNA according to claim 3, characterized in that: Before collecting the data set for training, the method further includes: Obtaining a preset DNA sequence, wherein the preset DNA sequence is a published whole genome DNA sequence; Preprocessing the preset DNA sequence to obtain multiple DNA sequence fragments; The first neural network model is pre-trained using the DNA sequence fragment.

5. The method for identifying diseased DNA according to claim 4, characterized in that: The step of obtaining a plurality of DNA sequence fragments after preprocessing the preset DNA sequence comprises: Deleting the fragment containing unknown bases in the preset DNA sequence; and The DNA sequence obtained after deleting the unknown base is divided into the plurality of DNA sequence fragments according to a preset length.

6. The method for identifying diseased DNA according to claim 4, characterized in that: The pre-training of the first neural network model using the DNA sequence fragments comprises: Before the DNA sequence fragment is embedded in the coding process, a portion of the bases in the DNA sequence fragment are masked, and for each masked base, the base at the masked base position is predicted based on the upstream and downstream sequences of the base, and the masked base is replaced by the predicted base.

7. The method for identifying diseased DNA according to claim 3, characterized in that: The training of the first neural network model, the second neural network model and the classifier further includes: In the process of training the first neural network model, the second neural network model and the classifier, the first neural network model, the second neural network model and the classifier are trained by gradient descent method and optimizing cross entropy loss function until the preset accuracy requirement is reached.

8. The method for identifying diseased DNA according to claim 3, characterized in that: After obtaining the trained first neural network model, second neural network model and classifier, the method further includes: Constructing an objective function according to a preset evaluation index to evaluate the training results of the first neural network model, the second neural network model and the classifier; The preset evaluation index includes any combination of the following: precision, recall and accuracy.

9. The method for identifying diseased DNA according to claim 8, characterized in that: The precision, recall and accuracy are calculated according to the following formulas: Precision: TP / (TP+FP); Recall rate: TP / (TP+FN); Accuracy: (TP+TN) / (TP+FP+FN+TN); Among them, FN means that it is identified as non-pathological DNA, but it is actually pathological DNA; FP means that it is identified as containing pathological DNA, but it is actually non-pathological DNA; TN means that it is identified as a non-pathological DNA fragment, but it is actually non-pathological DNA; TP means that it is identified as pathological DNA, but it is actually pathological DNA.

10. The method for identifying diseased DNA according to claim 1, characterized in that: The BERT model is a neural network model based on a multi-head self-attention mechanism, and it has a 12-layer network structure, and its hidden layer has 768 dimensions.

11. A diseased DNA identification device, characterized in that: The device comprises: A preprocessing module, used for obtaining a DNA sequence to be tested and preprocessing the DNA sequence to be tested to obtain a plurality of DNA sequence fragments; A first feature vector generation module is used to input each DNA sequence fragment into a trained first neural network model to output a first feature vector corresponding to the DNA sequence fragment, wherein the first feature vector contains autocorrelation information between different parts of the DNA sequence fragment, including: performing embedded coding processing on the DNA sequence fragment to obtain a first coding vector, a second coding vector, and a third coding vector with a preset fixed dimension, wherein the first coding vector contains content coding information of the DNA sequence fragment, the second coding vector contains position coding information of the DNA sequence fragment, and the third coding vector contains segment coding information; adding the first coding vector, the second coding vector, and the third coding vector to obtain a merged coding vector; performing self-attention processing on the merged coding vector to obtain a corresponding self-attention vector; performing normalization and residual processing on the self-attention vector to obtain the first feature vector corresponding to the DNA sequence fragment; the first neural network model is a pre-trained BERT model; A second feature vector generation module is used for inputting each first feature vector into a trained second neural network model to output a second feature vector corresponding to the first feature vector, and adding the first feature vector and the second feature vector corresponding thereto to obtain a combined feature vector, wherein the second feature vector contains long-term and short-term dependency relationship information between different parts in the first feature vector; and the second neural network model is an LSTM model; The classification module is used to input each of the combined feature vectors into a trained classifier to obtain a classification result, wherein the classification result indicates whether the input DNA sequence to be tested is a diseased DNA; the classifier is a Softmax classifier.

12. The diseased DNA identification device according to claim 11, characterized in that: The device also includes: A training module, wherein the training module is used to: Collecting a data set for training, wherein the data set includes annotated diseased DNA sequences and non-lesioned DNA sequences; Dividing the data set into a training set and a test set according to a preset ratio, and using the training set and the test set to train and test the first neural network model, the second neural network model and the classifier, respectively; Based on the test results, the parameters of the first neural network model, the second neural network model and the classifier are adjusted until the preset accuracy requirements are reached to obtain the trained first neural network model, the second neural network model and the classifier.

13. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the diseased DNA identification method according to any one of claims 1 to 10 is implemented.

14. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the diseased DNA identification method according to any one of claims 1 to 10 is implemented.

Images