Method, device, equipment and storage medium for identifying lesion difference sites

Abstract

The present application discloses a method, device, equipment and storage medium for identifying differential lesion sites, the method comprising providing a plurality of DNA segmentation sequences to be tested to an attention network model; obtaining the corresponding attention scores of each segmentation and the feature vector of the segmentation sequence by implementing an attention network model configured to encode the segmentation in the sequence and perform self-attention processing to obtain the corresponding attention scores and feature vectors of the segmentation; providing the feature vector of the segmentation sequence to a classification recognition network model; obtaining the classification recognition result by implementing a classification recognition network model configured to perform lesion DNA and non-lesion DNA recognition based on the feature vector; comparing the corresponding attention scores of each segmentation in the DNA segmentation sequence to be tested and the classification recognition result to determine the differential lesion sites. The present application makes full use of DNA sequence information to find differential lesion sites, and utilizes the attention mechanism and neural network classification results to improve the recognition ability.

Description

Technical Field

[0001] The present application relates to the fields of bioinformatics and neural network application technology, and in particular to methods, devices, equipment and storage media for identifying lesion difference sites. Background Art

[0002] At present, the identification of pathological modification sites on DNA sequences (such as changes in methylation modification of CpG sites) is mainly through some statistical methods, such as calculating the methylation frequency of each CpG site on DNA in tumor and non-tumor tissues respectively, and calculating whether there is a significant difference in DNA methylation frequency between tumor and non-tumor tissues through statistical methods, and then identifying differentially methylated CpG (DMC). Studies have found that the methylation state of CpG sites is also regulated by the surrounding DNA sequences, but the existing statistical methods only consider the methylation differences of CpG sites, and do not consider the DNA sequences around CpG sites, so the recognition ability needs to be improved.

[0003] Therefore, it is necessary to propose a deep learning method that combines DNA sequence information to find differentially modified sites of lesions. Summary of the invention

[0004] The present application aims to provide a method, device, equipment and storage medium for identifying lesion difference sites in order to address the problem that existing methods for identifying lesion difference sites do not take into account the DNA sequence around the lesion site, resulting in poor recognition ability.

[0005] In a first aspect, an embodiment of the present application provides a method for identifying lesion difference sites, comprising:

[0006] Providing a plurality of DNA segmentation sequences to be tested to an attention network model; obtaining an attention score corresponding to each segmentation and a feature vector of the DNA segmentation sequence to be tested by implementing an attention network model configured to encode the segmentations in the segmentation sequence and perform self-attention processing to obtain attention scores and feature vectors corresponding to the segmentations;

[0007] Providing the feature vector of the DNA segmentation sequence to be tested to a classification recognition network model; obtaining a classification recognition result by implementing the classification recognition network model configured to perform diseased DNA and non-lesioned DNA recognition based on the feature vector;

[0008] The attention scores corresponding to each word in all the DNA sequences to be tested and the classification recognition results are compared to determine the differential lesion sites.

[0009] In one embodiment of the present application, the attention network model obtains the attention score corresponding to each word segment and the feature vector of the DNA word segment sequence to be tested, including:

[0010] Performing embedded coding processing on each word in the DNA word sequence to be tested to obtain a multi-dimensional coding vector corresponding to each word;

[0011] Perform self-attention processing on the multi-dimensional encoding vectors corresponding to all the segmented words in the DNA segmented word sequence to be tested to obtain the feature vector of the DNA segmented word sequence to be tested;

[0012] Linearly multiply the multi-dimensional encoding vector corresponding to each word segmentation by the three weight matrices to obtain three vectors K, Q, and V;

[0013] Based on the three vectors K, Q, and V, the attention score corresponding to each word segment is obtained through the attention calculation function; wherein the attention calculation function is expressed as follows:

[0014]

[0015] Among them, d k is the number of columns of vector Q or K; softmax represents the normalization function; K T is the transpose of K.

[0016] In one embodiment of the present application, after the attention network model obtains the attention score corresponding to each word segment, it also corrects the attention score according to the length of the DNA word segmentation sequence to be tested.

[0017] In one embodiment of the present application, the correcting of the attention score according to the length of the DNA word segmentation sequence to be tested includes: taking the product of the determined attention score and the correction coefficient as the corrected attention score, wherein the correction coefficient is determined according to the length of the DNA word segmentation sequence to be tested.

[0018] In one embodiment of the present application, the correction coefficient is expressed as follows:

[0019]

[0020] Where BP is the correction coefficient, lc is the length of the DNA segmentation sequence, and lr is the average length of all DNA segmentation sequences.

[0021] In one embodiment of the present application, the method further includes: training the attention network model and the classification recognition network model in the following manner:

[0022] Collecting DNA segmentation sequences marked as pathological DNA and non-pathological DNA segmentation sequences, dividing them into training sets and test sets according to a preset ratio, and using the training sets and test sets to train and test the attention network model and the classification recognition network model;

[0023] Based on the test results, the parameters of the attention network model and the classification recognition network model are adjusted until the preset accuracy requirements are reached to obtain the trained attention network model and the classification recognition network model.

[0024] In one embodiment of the present application, collecting DNA segmentation sequences annotated as pathological DNA segmentation sequences and non-pathological DNA segmentation sequences includes:

[0025] Obtaining DNA sequences annotated as diseased DNA sequences and non-lesioned DNA sequences, wherein the diseased DNA sequences and non-lesioned DNA sequences are published whole genome DNA sequences;

[0026] Replacing the lesion site in the DNA sequence with a conversion site that is different from each base in the DNA sequence to obtain a replaced DNA sequence;

[0027] After preprocessing the replaced DNA sequence, a DNA segmentation sequence is obtained;

[0028] Selecting a preset proportion of segmented words from the DNA segmented word sequence as a pre-selected segmented word set;

[0029] Select the segmented words in the pre-selected segmented word set for masking, and obtain the masked DNA segmented word sequence.

[0030] In one embodiment of the present application, selecting a segmentation in the preselected segmentation set for masking includes:

[0031] The pre-selected word segmentation set is divided into a word segmentation set containing conversion positions and a word segmentation set without conversion positions; and word segmentations are selected from the word segmentation set containing conversion positions and / or the word segmentation set without conversion positions for masking according to a preset ratio.

[0032] In one embodiment of the present application, selecting segmentations from a set of segmentations containing transition positions and / or a set of segmentations not containing transition positions for masking according to a preset ratio includes:

[0033] Select words from the word set containing the conversion position according to the first ratio, and use MASK to mask them; select words from the remaining part of the word set containing the conversion position according to the second ratio, and replace the words with random words; finally, the remaining words in the word set containing the conversion position are not replaced;

[0034] And / or, select segmentations from the segmentation set without conversion positions according to the third ratio, and mask them with MASK; select segmentations from the remaining part of the segmentation set without conversion positions according to the fourth ratio, and replace the segmentations with random segmentations; finally, the remaining segmentations in the segmentation set without conversion positions are not replaced.

[0035] In one embodiment of the present application, the attention scores corresponding to each word in all the DNA word sequences to be tested and the classification recognition results are compared to determine the lesion difference sites, including:

[0036] For the diseased DNA segmentation sequence and the non-pathological DNA segmentation sequence, the attention score corresponding to each base is determined based on the attention score of each segmentation, and a preset proportion of bases are selected from high to low according to the attention scores corresponding to each base to form an important site region;

[0037] The sites that appear in both the important site regions of the lesion DNA segmentation sequence and the important site regions of the non-lesion DNA segmentation sequence are counted. If these sites have lesion differences between the lesion DNA sequence and the non-lesion DNA sequence, they are lesion difference sites.

[0038] In a second aspect, the present application also provides a device for identifying lesion difference sites, including:

[0039] An attention network model is used to encode each word in multiple DNA word sequences to be tested and perform self-attention processing to obtain an attention score corresponding to the word and a feature vector of the DNA word sequence to be tested;

[0040] A classification recognition network model, used for performing diseased DNA and non-lesioned DNA recognition based on the feature vector of the DNA segmentation sequence to be tested to obtain a classification recognition result;

[0041] The processing module is used to compare the corresponding attention scores of each word in all the DNA word sequences to be tested and the classification recognition results to determine the difference sites of the lesions.

[0042] In one embodiment of the present application, the device further includes a training module, wherein the training module is used to:

[0043] Collecting DNA segmentation sequences marked as pathological DNA and non-pathological DNA segmentation sequences, dividing them into training sets and test sets according to a preset ratio, and using the training sets and test sets to train and test the attention network model and the classification recognition network model;

[0044] Based on the test results, the parameters of the attention network model and the classification recognition network model are adjusted until the preset accuracy requirements are reached to obtain the trained attention network model and the classification recognition network model.

[0045] In one embodiment of the present application, the training module is further used to obtain DNA sequences that have been annotated as diseased DNA sequences and non-lesioned DNA sequences, wherein the diseased DNA sequences and non-lesioned DNA sequences are published whole genome DNA sequences;

[0046] The site with the lesion in the DNA sequence is replaced with a conversion site that is different from each base in the DNA sequence to obtain a replaced DNA sequence;

[0047] After preprocessing the replaced DNA sequence, a DNA segmentation sequence is obtained;

[0048] Selecting a preset proportion of segmented words from the DNA segmented word sequence as a pre-selected segmented word set;

[0049] The segmented words in the pre-selected segmented word set are selected for masking to obtain the masked DNA segmented word sequence.

[0050] In a third aspect, the present application also provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for identifying lesion difference sites as described in any one of the first aspects is implemented.

[0051] In a fourth aspect, the present application further provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the method for identifying lesion difference sites as described in any one of the first aspects is implemented.

[0052] One of the above technical solutions has the following advantages or beneficial effects:

[0053] Compared with the prior art, this application makes full use of DNA sequence information to find disease mutation sites based on the DNA segmentation sequence after segmentation processing, and uses the attention mechanism and neural network classification results to improve the recognition ability;

[0054] In the process of training the attention network model and the classification and recognition network model, the present application masks the lesion sites and combines the attention processing mechanism to make the network pay more attention to the lesion sites and learn from them, further improving the ability to identify differential lesion sites.

[0055] In the process of calculating the attention score in this application, the attention score is corrected according to the length of the DNA segmentation sequence, eliminating the influence of the DNA length on the attention score, which can fully reflect the importance of each segmentation and is conducive to improving the recognition effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] The drawings described herein are for explanation purposes only and are not intended to limit the scope of the present application in any way. In addition, the shapes and proportional dimensions of the components in the drawings are only for illustration purposes and are used to help understand the present application. They do not specifically limit the shapes and proportional dimensions of the components of the present application. Under the guidance of the present application, those skilled in the art can select various possible shapes and proportional dimensions to implement the present application according to specific circumstances. In the drawings:

[0057] Figure 1 A schematic diagram of the process of the method for identifying lesion difference sites provided in the embodiment of the present application;

[0058] Figure 2 A schematic diagram of the structure of a device for identifying lesion difference sites provided in an embodiment of the present application;

[0059] Figure 3 The masking process adopted for existing technologies;

[0060] Figure 4 The masking process used in the lesion difference site identification method provided in the embodiment of the present application;

[0061] Figure 5 A schematic diagram of replacing a DNA sequence in the method for identifying differential lesion sites provided in an embodiment of the present application;

[0062] Figure 6 A schematic diagram of masking in the method for identifying lesion difference sites provided in an embodiment of the present application;

[0063] Figure 7 A schematic diagram of the attention scores of word segments obtained in the lesion difference site identification method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0064] In order to enable those skilled in the art to better understand the technical solutions in the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work should fall within the scope of protection of the present application.

[0065] To facilitate understanding of this application, the following is an introduction to the background technology.

[0066] The human genome DNA sequence data contains about 3 billion base pairs and can be obtained through first-generation or second-generation sequencing methods. DNA sequences can include adenine (A), thymine (T), cytosine (C), and guanine (G). Different bases can have different modifications (such as methylation of cytosine, which can be artificially represented by other unused letters). For example, a DNA sequence is: ATCGGCTTCATTGGCG.

[0067] The step of preprocessing the DNA sequence includes: deleting the fragment containing the unknown base N in the sequence, and randomly dividing the DNA sequence obtained after deleting the unknown base N into multiple DNA sequences of different lengths according to a preset length. If the measured DNA sequence is within the preset length range, there is no need to divide the DNA.

[0068] It should be noted that the lesions referred to in this application include: methylation (-CH3), formylation (-CHO), carboxylation (-COOH), etc. Among them, the site with methylation lesions is mainly on the CG site, and other lesions are on the corresponding sites.

[0069] CpG sites (also called CG sites) refer to a region of DNA where the base sequence is cytosine followed by guanine. "CpG" is the abbreviation of "—C—phospho—G—", which means that the phosphodiester bond connects cytosine and guanine, with C at the 5' end and G at the 3' end. Cytosine without methyl in CpG sites can be changed to 5-methylcytosine, or 5-methylcytosine can be changed to cytosine without methyl.

[0070] The existing technology uses statistical methods (such as t-test) to calculate whether there is a significant difference in the frequency of DNA lesions in tumor and non-tumor tissues, and then identify the differentially methylated regions (DMCs). t-test, also known as the T test, is mainly used for normal distribution data with unknown overall standard deviation σ.

[0071] Studies have found that the pathological status of the lesion site is also regulated by the surrounding DNA sequences, but the existing statistical methods only consider the lesion differences of the lesion site and do not consider the DNA sequences around the lesion site, so the recognition ability needs to be improved.

[0072] In order to solve this problem, the present application provides a method, device, equipment and storage medium for identifying lesion difference sites. The present application adopts an attention network model, calculates the attention score of each segmentation according to the input DNA segmentation sequence, and inputs the attention score into the classification recognition network model for classification recognition, and obtains whether the DNA segmentation sequence is lesion DNA or non-lesion DNA; based on the attention score of each segmentation and the classification recognition result, the lesion difference site is identified, and the present application can obtain a better recognition effect.

[0073] Combine the following Figure 1 to Figure 4 This application is described in detail.

[0074] Figure 1is a flow chart of a method for identifying lesion difference sites provided in an embodiment of the present application, the method comprising:

[0075] Step 101: Collect multiple DNA sequences to be tested, and perform word segmentation on the DNA sequences to obtain word segmentation sequences;

[0076] Step 102: providing a plurality of DNA word segmentation sequences to be tested to an attention network model; obtaining an attention score corresponding to each word segmentation and a feature vector of the DNA word segmentation sequence to be tested by implementing an attention network model configured to encode each word segmentation in the DNA word segmentation sequence to be tested and perform self-attention processing to obtain an attention score and a feature vector corresponding to the word segmentation;

[0077] Step 103: providing the feature vector of the DNA segmentation sequence to be tested to a classification recognition network model; obtaining a classification recognition result by implementing the classification recognition network model configured to perform diseased DNA and non-lesioned DNA recognition based on the feature vector;

[0078] Step 104: Compare the attention scores corresponding to each word in all the DNA word sequences to be tested and the classification recognition results to determine the lesion difference sites.

[0079] It should be noted that step 101 is not a necessary step. In some implementations, a DNA segmentation sequence that has been segmented can be directly obtained.

[0080] Exemplarily, the obtained DNA word segmentation sequence, for example: "CLS ATCGGCTTCATT GG CGSEP", CLS represents the start symbol of a DNA sequence fragment, SEP represents the end symbol of a DNA sequence fragment, one base is one word, and 16 words are obtained. Each word is represented by a 768-dimensional vector, such as word A is represented by a [0.001, 0.123, 0.234, ...] vector, which contains 768 floating point numbers. It should be noted that the number of bases in a word can be set arbitrarily, such as setting two bases per word, three bases per word, and five bases per word (token).

[0081] Exemplarily, in step 102, the attention network model obtains the attention score corresponding to each word segment and the feature vector of the DNA word segment sequence to be tested, including:

[0082] Perform embedding coding on each word in the DNA word sequence to be tested to obtain a multi-dimensional coding vector corresponding to each word;

[0083] Perform self-attention processing on the multi-dimensional encoding vectors corresponding to all the segmented words in the DNA segmented word sequence to be tested to obtain the feature vector of the DNA segmented word sequence to be tested;

[0084] Linearly multiply the multi-dimensional encoding vector corresponding to each word segmentation by the three weight matrices to obtain three vectors K, Q, and V;

[0085] Based on the three vectors K, Q, and V, the attention score corresponding to each word is obtained through the attention calculation function.

[0086] It should be noted that the existing technology can be used to obtain the multi-dimensional coding vector corresponding to each word segment. Optionally, the multi-dimensional coding vector is a merged coding vector, wherein the merged coding vector contains one or more of the content coding information of the word segment in the DNA word segmentation sequence, the position coding information of the word segment in the DNA word segmentation sequence, and the segmentation coding information of the word segment.

[0087] Optionally, in a specific embodiment, the method of merging multiple encoding vectors may be implemented by vector addition or the like, which is not limited thereto.

[0088] Through this step, the encoding vector corresponding to each word in the entire DNA word sequence can be obtained.

[0089] An embodiment of the present application provides a method for identifying differential lesion sites. The attention network model has 12 layers. The last layer extracts the attention score of each word in the word sequence. The level of the attention score can reflect the importance of each word. The sum of the attention scores of all words in a DNA word sequence is 1.

[0090] It should be noted that it is an existing technology to use a neural network model to perform self-attention processing on the multi-dimensional coding vectors corresponding to all the words in the DNA word segmentation sequence to be tested to obtain the feature vector of the DNA word segmentation sequence to be tested, which can optionally include obtaining the feature vector by normalizing and residual processing the coding vector.

[0091] Exemplarily, a scaled dot product attention mechanism can be used for self-attention processing, including dot product, scaling, masking (optional operation), and normalization operations on the input Q vector (Query vector) and K vector (i.e., Key vector), and dot product of the result of the normalization operation with the V vector (i.e., Value vector).

[0092] Exemplarily, the attention calculation function (used to calculate the attention score) is expressed as follows:

[0093]

[0094] Among them, Q, K, V are all vectors, d k Expressed as the number of columns of the Q or K matrix, i.e., the vector dimension, d k as a regulating factor; softmax represents a normalization function; T represents transposition. The input values ​​of Q, K, and V are multi-dimensional encoding vectors corresponding to the word segmentation.

[0095] Based on the above attention calculation function, the multi-dimensional encoding vector obtained after the same DNA segmentation sequence is processed by embedded coding is linearly transformed to obtain the Query vector, Key vector, and Value vector, and then the attention score is calculated (the attention score is Attention(Q, K, V) of the above function).

[0096] The specific operation is that the multi-dimensional encoding vector corresponding to each word segmentation must be linearly multiplied with three weight matrices Wq, Wk, and Wv (the weight matrix is ​​the neural network parameter, also known as the linear transformation matrix) to generate three vectors corresponding to each word (in the implementation method where one base corresponds to one word, that is, each base) Query (Q), Key (K), and Value (V). The following Embedding refers to the encoding vector.

[0097] For example: Query(Q)=Embedding*Wq; Key(K)=Embedding*Wq;

[0098] Value(V)=Embedding*Wq.

[0099] The multi-dimensional encoding vector corresponding to each word in the word sequence represents the relationship between words and the relationship between each word and the DNA sequence. Each word is composed of a preset number of bases.

[0100] Exemplarily, the attention network model is the BERT model, and the classification recognition network model is the Long Short-Term Memory network model (Long Short-Term Memory, LSTM for short). LSTM is a special RNN, which is mainly used to solve the gradient disappearance and gradient explosion problems in the long sequence training process. In short, compared with ordinary RNN, LSTM can perform better in longer sequences. Among them, the full name of the BERT model is Bidirectional Encoder Representations from Transformers. The BERT model is a 12-layer network structure with a hidden layer of 768 dimensions and a 12-head mode. The BERT model has the following advantages: First, it can use MLM (Masked Language Model) to pre-train bidirectional Transformers (which is a Seq2Seq model consisting of an encoder and a decoder) to generate deep bidirectional language representations. Second, after pre-training, the BERT model only needs to add an additional output layer for fine-tuning to achieve better performance in a variety of downstream tasks, and there is no need to perform task-specific structural modifications to the BERT model in this process.

[0101] Since the longer the sequence, the smaller the average attention score of each word, in order to eliminate the influence of DNA length on the attention score, the attention score is corrected using the above method in the embodiment, and the lesion difference site is identified based on the corrected attention score, with better identification effect.

[0102] Exemplarily, after the attention network model obtains the attention score corresponding to each word segment in step 102, the attention score is also corrected according to the length of the DNA word segmentation sequence to be tested.

[0103] Optionally, correcting the attention score according to the length of the DNA word segmentation sequence to be tested includes: taking the product of the determined attention score and the correction coefficient as the corrected attention score, wherein the correction coefficient is determined according to the length of the DNA word segmentation sequence to be tested.

[0104] One embodiment of the present application provides a method for identifying lesion difference sites, and correcting the attention score according to the length of the DNA segmentation sequence to be tested includes: taking the product of the determined attention score and the correction coefficient as the corrected attention score.

[0105] The expression of the correction coefficient is as follows:

[0106]

[0107] Where BP is the correction coefficient, lc is the length of the DNA segmentation sequence, and lr is the average length of all DNA segmentation sequences.

[0108] This application reduces the impact of sequence length on the attention score of each word, corrects the attention score according to the length, and improves the recognition effect of lesion difference sites.

[0109] In an embodiment of the present application, a method for identifying differential lesion sites is provided, step 104: comparing the attention scores corresponding to each word in the DNA word sequence to be tested and the classification recognition results to determine the differential lesion sites, including:

[0110] For the diseased DNA segmentation sequence and the non-pathological DNA segmentation sequence, the attention score corresponding to each base is determined based on the attention score of each segmentation, and a preset proportion of bases are selected from high to low according to the attention scores corresponding to each base to form an important site region;

[0111] The sites that appear in both the important site regions of the lesion DNA segmentation sequence and the important site regions of the non-lesion DNA segmentation sequence are counted. If these sites have lesion differences between the lesion DNA sequence and the non-lesion DNA sequence, they are lesion difference sites.

[0112] This application can determine which important sites on the DNA segmentation sequence the neural network uses to distinguish diseased DNA from non-lesioned DNA by the attention score. By counting the important sites that appear uniformly in the important site areas of all diseased DNA segmentation sequences and non-lesioned DNA segmentation sequences, if there are lesion differences in these sites on the diseased and non-lesioned DNA sequences, they are determined as important lesion difference sites. The existing statistical methods need to compare all lesion data in diseased DNA and non-lesioned DNA to find a batch of highly lesion sites with lesion tissue specificity. They are inefficient and do not use the DNA sequence information around the lesion sites. They may not be able to focus on the truly important sites. The method of this application can improve the recognition efficiency and recognition effect of lesion difference sites.

[0113] An embodiment of the present application provides a method for identifying lesion difference sites, further comprising: training an attention network model and a classification recognition network model in the following manner:

[0114] Collect the DNA segmentation sequences marked as pathological and non-pathological DNA segmentation sequences, divide them into training sets and test sets according to the preset ratio, and use the training sets and test sets to train and test the attention network model and the classification recognition network model;

[0115] Based on the test results, the parameters of the attention network model and the classification recognition network model are adjusted until the preset accuracy requirements are met to obtain the trained attention network model and the classification recognition network model.

[0116] Figure 3 The figure shows the random masking process commonly used in the current method. As shown in the figure, in the current learning method, the original DNA sequence is segmented to obtain a DNA segmentation sequence, a certain proportion of segmentations are selected as a pre-selected segmentation set, segmentations are selected from the pre-selected segmentation set according to a first set probability (such as 80%) for MASK (masking), and segmentations are selected from the remaining segmentations in the pre-selected segmentation set according to a second set probability (such as 10%) to replace with random segmentations, and the last remaining segmentations in the pre-selected segmentation set are not replaced.

[0117] If BERT directly uses the random masking process, because the proportion of lesion sites in the genome is small, there are relatively few lesion sites that are masked, so the model cannot learn the relationship between the lesion site and the surrounding sequence, because the model predicts the masked site through the sequence around the masked site. In order to make the model pay more attention to the lesion site, the random masking process of the model is improved in one embodiment of the present application.

[0118] In some embodiments, collecting DNA segmentation sequences annotated as diseased DNA segments and non-lesional DNA segmentation sequences includes:

[0119] Obtaining DNA sequences annotated as diseased DNA sequences and non-pathological DNA sequences, wherein the diseased DNA sequences and non-pathological DNA sequences are publicly available whole genome DNA sequences;

[0120] The CG site with a lesion in the DNA sequence (this embodiment is for methylation, and is replaced with a corresponding lesion site in other lesions) is replaced with a conversion site that is different from each base in the DNA sequence to obtain a replaced DNA sequence; the replaced DNA sequence is preprocessed to obtain a DNA segmentation sequence (such as Figure 5 shown);

[0121] Selecting a preset proportion of segmented words from the DNA segmented word sequence as a pre-selected segmented word set;

[0122] Select the segmented words in the pre-selected segmented word set for masking, and obtain the masked DNA segmented word sequence.

[0123] It should be noted that the transition position can be artificially represented by other unused letters, which must be different from the letters representing each base in the DNA sequence, such as adenine (A), thymine (T), cytosine (C), guanine (G), and unknown base (N). In one embodiment of the application, the transition position uses ML.

[0124] like Figure 4 , Figure 5 , Figure 6 As shown, selecting a word from the pre-selected word set for masking includes:

[0125] For the pre-selected word set, divide it into a word set containing conversion positions and a word set without conversion positions; select words from the word set containing conversion positions and / or the word set without conversion positions according to a preset ratio for MASK masking (such as Figure 6 shown).

[0126] like Figure 4 As shown, in some embodiments, according to the preset ratio ( Figure 4 In the example, 15% is used, and in other implementations, it can be set according to the situation.) Selecting segmentations from the segmentation set containing the conversion position and / or the segmentation set without the conversion position for masking includes:

[0127] According to a first ratio (such as 80%), from the conversion bit ( Figure 4 In the example, a word is selected from the word set with ML as the conversion position, and masked by MASK; a word is selected from the remaining part of the word set containing the conversion position according to a second ratio (such as 10%), and the word is replaced with a random word; finally, the remaining words in the word set containing the conversion position are not replaced;

[0128] And / or, select segmentations from the segmentation set without conversion positions according to a third ratio (such as 80%), and mask them with MASK; select segmentations from the remaining part of the segmentation set without conversion positions according to a fourth ratio (such as 10%), and replace the segmentations with random segmentations; finally, the remaining segmentations in the segmentation set without conversion positions are not replaced.

[0129] It should be noted that, in a specific embodiment, those skilled in the art may select the ratio according to actual needs, and this is not limited.

[0130] In the present application, masking is performed based on the replaced DNA sequence, which can mask more lesion sites, so that the model can learn more information about the lesions and further improve the recognition effect of lesion difference sites.

[0131] Figure 7 The following is an implementation of a word with three bases, and the attention scores of each word segment are obtained. After obtaining the attention score, the above-mentioned correction method can be selectively used to correct the attention score. After obtaining the attention score, the attention score corresponding to each base is determined according to the attention score of each word segment. For example, if the average attention score of the third base M is calculated, it is (0.01+0.1+0.001) / 3.

[0132] The following is a description of the lesion difference site identification device provided by the present application. The lesion difference site identification method described below can be referenced to the lesion difference site identification method described above.

[0133] Figure 2 It is a structural schematic diagram of a lesion difference site identification device provided by the present application, comprising: an attention network model 1, which is used to encode a plurality of DNA segmentation sequences to be tested and perform self-attention processing to obtain the attention scores corresponding to the segmentations and the feature vectors of the DNA segmentation sequences to be tested;

[0134] Classification and recognition network model 2, used for performing diseased DNA and non-lesioned DNA recognition based on the feature vector of the DNA segmentation sequence to be tested to obtain classification and recognition results;

[0135] Processing module 3 is used to compare the attention scores corresponding to each word in the DNA word sequence to be tested and the classification recognition results to determine the lesion difference site.

[0136] Exemplarily, the device further includes a training module, the training module being used to:

[0137] Collect the DNA segmentation sequences marked as pathological and non-pathological DNA segmentation sequences, divide them into training sets and test sets according to the preset ratio, and use the training sets and test sets to train and test the attention network model and the classification recognition network model;

[0138] Based on the test results, the parameters of the attention network model and the classification recognition network model are adjusted until the preset accuracy requirements are met to obtain the trained attention network model and the classification recognition network model.

[0139] Exemplarily, the training module collects DNA segmentation sequences that have been labeled as diseased DNA segmentation sequences and non-lesioned DNA segmentation sequences, including:

[0140] Obtaining DNA sequences annotated as diseased DNA sequences and non-pathological DNA sequences, wherein the diseased DNA sequences and non-pathological DNA sequences are publicly available whole genome DNA sequences;

[0141] The site with the lesion in the DNA sequence is replaced with a conversion site that is different from each base in the DNA sequence to obtain a replaced DNA sequence;

[0142] After preprocessing the replaced DNA sequence, a DNA segmentation sequence is obtained;

[0143] Selecting a preset proportion of segmented words from the DNA segmented word sequence as a pre-selected segmented word set;

[0144] The segmented words in the pre-selected segmented word set are selected for masking to obtain the masked DNA segmented word sequence.

[0145] In some embodiments, an electronic device is exemplified, which may include a processor, a communication module, a memory, and a communication bus. The processor / computer, the communication module, and the memory communicate with each other through the communication bus. The processor may call the logic instructions in the memory to execute a method for identifying lesion difference sites, which method includes:

[0146] Providing a plurality of DNA segmentation sequences to be tested to an attention network model; obtaining an attention score corresponding to each segmentation and a feature vector of the DNA segmentation sequence to be tested by implementing an attention network model configured to encode the segmentations in the segmentation sequence and perform self-attention processing to obtain attention scores and feature vectors corresponding to the segmentations;

[0147] Providing the feature vector of the DNA segmentation sequence to be tested to the classification recognition network model; obtaining the classification recognition result by implementing the classification recognition network model configured to perform diseased DNA and non-lesioned DNA recognition according to the feature vector;

[0148] The attention scores and classification recognition results of each word in all the DNA word sequences to be tested are compared to determine the differential lesion sites.

[0149] On the other hand, the present application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to perform the above-mentioned lesion difference site identification methods.

[0150] As an example only, a processor or computer may be implemented by one or more processing elements (such as a logic gate array, a controller and an arithmetic logic unit, a digital signal processor, a microcomputer, a programmable logic controller, a field programmable gate array, a programmable logic array, a microprocessor, or any other device or combination of devices configured to respond and execute instructions in a limited manner to achieve the desired result). In one example, a processor or computer includes or is connected to one or more memories storing instructions or software executed by a processor or computer. The hardware components implemented by a processor or computer can execute instructions or software (such as an operating system (OS) and one or more software applications running on the OS) to perform the operations described in this application. The hardware components can also access, manipulate, process, create and store data in response to the execution of instructions or software. For simplicity, the singular term "processor" or "computer" can be used for the description of the examples described in this application, but in other examples, multiple processors or computers can be used, or a processor or a computer can include multiple processing elements or multiple types of processing elements or both. For example, a single hardware component or two or more hardware components can be implemented by a single processor or two or more processors or a processor and a controller. One or more hardware components may be implemented by one or more processors or a processor and a controller, and one or more other hardware components may be implemented by one or more other processors or another processor and another controller. One or more processors or a processor and a controller may implement a single hardware component or two or more hardware components. The hardware components may have any one or more different processing configurations, examples of which include: a single processor, independent processors, parallel processors, single instruction single data (SISD) multiprocessing, single instruction multiple data (SIMD) multiprocessing, multiple instruction single data (MISD) multiprocessing, and multiple instruction multiple data (MIMD) multiprocessing.

[0151] The processes and methods for performing the operations described in the present application shown above are performed by computing hardware (e.g., by one or more processors or computers implemented to execute instructions or software as described above to perform the operations performed by the methods described in the present application). For example, a single operation or two or more operations may be performed by a single processor or two or more processors or a processor and a controller. One or more operations may be performed by one or more processors or a processor and a controller, and one or more other operations may be performed by one or more other processors or another processor and another controller. One or more processors or a processor and a controller may perform a single operation or two or more operations.

[0152] The instructions or software for controlling the processor or computer to implement the hardware components and perform the above method can be written as a computer program, a code segment, an instruction or any combination thereof, to individually or collectively indicate or configure the processor or computer as a machine or a special-purpose computer for performing the operations performed by the hardware components and the above method to operate. In one example, the instructions or software include machine code (such as, machine code generated by a compiler) directly executed by the processor or computer. In another example, the instructions or software include high-level code executed by the processor or computer using an interpreter. Any programming language can be used to write instructions or software based on the block diagrams and flow charts shown in the accompanying drawings and the corresponding descriptions in the specification, wherein the block diagrams and flow charts shown in the accompanying drawings and the corresponding descriptions in the specification disclose algorithms for performing the operations performed by the hardware components and the above method.

[0153] Instructions or software for controlling computing hardware (e.g., one or more processors or computers) to implement hardware components and perform the above methods, as well as any associated data, data files, and data structures, may be recorded, stored, or fixed in or on one or more non-transitory computer-readable storage media. Examples of non-transitory computer-readable storage media include: read-only memory (ROM), random access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-RLTH, BD-RE , Blu-ray or optical disk storage, hard disk drive (HDD), solid state drive (SSD), card-type storage such as a micro multimedia card or card (e.g., secure digital (SD) or extreme speed card (XD)), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid state disk, and any other device, wherein any other device is configured to: store instructions or software and any associated data, data files and data structures in a non-temporary manner, and provide instructions or software and any associated data, data files and data structures to one or more processors or computers so that one or more processors or computers can execute instructions. In one example, the instructions or software and any associated data, data files and data structures are distributed on a networked computer system so that the instructions and software and any associated data, data files and data structures are stored, accessed and executed in a distributed manner by one or more processors or computers.

[0154] It should be understood that the above description is for illustration and not for limitation. Many embodiments and many applications beyond the examples provided will be apparent to those skilled in the art upon reading the above description. Therefore, the scope of the present teachings should not be determined with reference to the above description, but rather with reference to the foregoing claims and the full scope of equivalents to which such claims are entitled. For the purpose of comprehensiveness, all articles and references, including disclosures of patent applications and publications, are incorporated herein by reference. The omission of any aspect of the subject matter disclosed herein in the foregoing claims is not intended to be a waiver of such subject matter, nor should it be considered that the applicant has not considered such subject matter to be part of the disclosed application subject matter.

Claims

1. A method for identifying differential lesion sites for non-disease diagnosis purposes, characterized in that: include: Providing a plurality of DNA word segmentation sequences to be tested to an attention network model; obtaining the attention score corresponding to each word segmentation and the feature vector of the DNA word segmentation sequence to be tested by implementing the attention network model configured to obtain the attention score and feature vector corresponding to the word segmentation by encoding the word segmentation in the word segmentation sequence and performing self-attention processing; Providing the feature vector of the DNA segmentation sequence to be tested to a classification recognition network model; obtaining a classification recognition result by implementing the classification recognition network model configured to perform diseased DNA and non-lesioned DNA recognition according to the feature vector; Compare the attention scores corresponding to each word in all the DNA word sequences to be tested and the classification and recognition results to determine the differential lesion sites; The step of comparing the attention scores corresponding to each word in all the DNA word sequences to be tested and the classification recognition results to determine the lesion difference sites includes: For the diseased DNA segmentation sequence and the non-pathological DNA segmentation sequence, the attention score corresponding to each base is determined based on the attention score of each segmentation, and a preset proportion of bases are selected from high to low according to the attention scores corresponding to each base to form an important site region; The sites that appear in both the important site regions of the lesion DNA segmentation sequence and the important site regions of the non-lesion DNA segmentation sequence are counted. If these sites have lesion differences in the lesion DNA sequence and the non-lesion DNA sequence, they are lesion difference sites. After the attention network model obtains the attention score corresponding to each word segment, the attention score is corrected according to the length of the DNA word segment sequence to be tested; The correcting of the attention score comprises: taking the product of the attention score corresponding to each word segment and the correction coefficient as the corrected attention score, wherein the correction coefficient is determined according to the length of the DNA word segment sequence to be tested; The expression of the correction coefficient is as follows: Where BP is the correction coefficient, lc is the length of the DNA segmentation sequence, and lr is the average length of all DNA segmentation sequences.

2. The method for identifying lesion difference sites according to claim 1, characterized in that: The step of obtaining the attention score corresponding to each word segment and the feature vector of the DNA word segment sequence to be tested includes: Performing embedded coding processing on each word in the DNA word sequence to be tested to obtain a multi-dimensional coding vector corresponding to each word; Perform self-attention processing on the multi-dimensional encoding vectors corresponding to all the segmented words in the DNA segmented word sequence to be tested to obtain the feature vector of the DNA segmented word sequence to be tested; Linearly multiply the multi-dimensional encoding vector corresponding to each word segmentation by the three weight matrices to obtain three vectors K, Q, and V; Based on the three vectors K, Q, and V, the attention score corresponding to each word segment is obtained through the attention calculation function; wherein the attention calculation function is expressed as follows: Among them, d k is the number of columns of vector Q or K; softmax represents the normalization function; K T is the transpose of K.

3. The method for identifying lesion difference sites according to claim 1, characterized in that: The method further includes: training the attention network model and the classification recognition network model in the following manner: Collecting DNA segmentation sequences marked as pathological DNA and non-pathological DNA segmentation sequences, dividing them into training sets and test sets according to a preset ratio, and using the training sets and test sets to train and test the attention network model and the classification recognition network model; Based on the test results, the parameters of the attention network model and the classification recognition network model are adjusted until the preset accuracy requirements are reached to obtain the trained attention network model and the classification recognition network model.

4. The method for identifying lesion difference sites according to claim 3, characterized in that: The method further includes: collecting DNA segmentation sequences that have been marked as pathological DNA segmentation sequences and non-pathological DNA segmentation sequences including: Obtaining DNA sequences annotated as diseased and non-lesioned DNA sequences, wherein the diseased and non-lesioned DNA sequences are published whole genome DNA sequences; Replacing the site with the lesion in the DNA sequence with a conversion site different from each base in the DNA sequence to obtain a replaced DNA sequence; Preprocessing the replaced DNA sequence to obtain a DNA segmentation sequence; Selecting a preset proportion of segmented words from the DNA segmented word sequence as a pre-selected segmented word set; The segmented words in the preselected segmented word set are selected for masking processing to obtain a masked segmented word sequence.

5. The method for identifying lesion difference sites according to claim 4, characterized in that: Selecting the segmented words in the pre-selected segmented word set for masking includes: The preselected word segmentation set is divided into a word segmentation set containing conversion positions and a word segmentation set without conversion positions; and word segmentations are selected from the word segmentation set containing conversion positions and / or the word segmentation set without conversion positions for masking according to a preset ratio.

6. The method for identifying lesion difference sites according to claim 5, characterized in that: Selecting segmentations for masking from the segmentation set containing transition positions and / or the segmentation set not containing transition positions according to a preset ratio respectively includes: Selecting words from the word set containing the conversion position according to the first ratio, and masking them with MASK; selecting words from the remaining part of the word set containing the conversion position according to the second ratio, and replacing the words with random words; finally, the remaining words in the word set containing the conversion position are not replaced; And / or, selecting participles from the participle set without conversion positions according to a third ratio, and masking them with MASK; selecting participles from the remaining part of the participle set without conversion positions according to a fourth ratio, and replacing the participles with random participles; finally, the remaining participles in the participle set without conversion positions are not replaced.

7. A device for identifying lesion difference sites for non-disease diagnosis purposes, characterized in that: include: An attention network model is used to encode each word in multiple DNA word sequences to be tested and perform self-attention processing to obtain an attention score corresponding to each word in the DNA word sequence to be tested and a feature vector of the DNA word sequence to be tested; A classification recognition network model is used to perform diseased DNA and non-lesioned DNA recognition based on the feature vector of the DNA segmentation sequence to be tested to obtain a classification recognition result; A processing module is used to compare the attention scores corresponding to each word of all DNA word sequences to be tested and the classification and recognition results to determine the difference sites of lesions; The processing module is specifically used for: For the diseased DNA segmentation sequence and the non-pathological DNA segmentation sequence, the attention score corresponding to each base is determined based on the attention score of each segmentation, and a preset proportion of bases are selected from high to low according to the attention scores corresponding to each base to form an important site region; The sites that appear in both the important site regions of the lesion DNA segmentation sequence and the important site regions of the non-lesion DNA segmentation sequence are counted. If these sites have lesion differences in the lesion DNA sequence and the non-lesion DNA sequence, they are lesion difference sites. After the attention network model obtains the attention score corresponding to each word segment, it is also used to correct the attention score according to the length of the DNA word segmentation sequence to be tested; The correcting of the attention score comprises: taking the product of the attention score corresponding to each word segment and the correction coefficient as the corrected attention score, wherein the correction coefficient is determined according to the length of the DNA word segment sequence to be tested; The expression of the correction coefficient is as follows: Where BP is the correction coefficient, lc is the length of the DNA segmentation sequence, and lr is the average length of all DNA segmentation sequences.

8. The lesion difference site identification device according to claim 7, characterized in that: The device also includes a training module, which is used to: Collecting DNA segmentation sequences marked as pathological DNA and non-pathological DNA segmentation sequences, dividing them into training sets and test sets according to a preset ratio, and using the training sets and test sets to train and test the attention network model and the classification recognition network model; Based on the test results, the parameters of the attention network model and the classification recognition network model are adjusted until the preset accuracy requirements are met to obtain a trained attention network model and classification recognition network model.

9. The lesion difference site identification device according to claim 8, characterized in that: The training module collects DNA segmentation sequences that have been labeled as pathological DNA segmentation sequences and non-pathological DNA segmentation sequences, including: Obtaining DNA sequences annotated as diseased DNA sequences and non-lesioned DNA sequences, wherein the diseased DNA sequences and non-lesioned DNA sequences are published whole genome DNA sequences; Replacing the site with the lesion in the DNA sequence with a conversion site that is different from each base in the DNA sequence to obtain a replaced DNA sequence; Preprocessing the replaced DNA sequence to obtain a DNA segmentation sequence; Selecting a preset proportion of segmented words from the DNA segmented word sequence as a pre-selected segmented word set; The segmented words in the preselected segmented word set are selected for masking to obtain a masked DNA segmented word sequence.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the lesion difference site identification method according to any one of claims 1 to 6 is implemented.

11. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for identifying lesion difference sites according to any one of claims 1 to 6 is implemented.

Images