A processing method and device for a methylation feature encoding model

Abstract

Embodiments of the present invention relate to a method and apparatus for processing a methylation feature encoding model. The method comprises: collecting a BS-seq read set; performing read splicing based on the read set to obtain a first sequence, performing a five-element sequence conversion on the first sequence to obtain a second sequence, and performing vocabulary learning based on the second sequence; constructing a methylation feature encoding model, a sequence decoding model, and a target source prediction model; forming a pre-training model framework comprising the methylation feature encoding model and the sequence decoding model, and forming a first task model framework comprising the methylation feature encoding model and the target source prediction model; training the pre-training model framework and the first task model framework; and, after training, processing a methylation sequence reconstruction task based on the pre-training model framework and a DNA source identification task based on the first task model framework. The present invention can improve the real-time performance of task processing and reduce task processing costs.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a method and device for processing a methylation feature coding model. Background Art

[0002] Epigenomics is a key field that studies chemical modifications of the genome and their regulatory mechanisms. Deoxyribonucleic acid (DNA) methylation is a key epigenetic modification, typically occurring as 5-methylcytosine (5-mC) at CpG sites. A CpG site is a region of DNA where a cytosine (C) followed by a guanine (G) base appears. "CpG" stands for "phosphodiester bond p-G." DNA methylation plays a crucial role in biological processes such as gene expression regulation, cell differentiation, and genomic imprinting. Abnormal patterns (e.g., abnormal methylation locations) are often closely associated with a variety of diseases, including a range of tumors (also known as cancers), cell-mediated leukemias, neurological disorders, and immune system diseases. Therefore, accurately predicting the global methylation status of DNA sequences is crucial for basic disease research.

[0003] Bisulfite sequencing (BS-seq), the recognized gold standard sequencing method in the field of DNA methylation, can be used to determine methylation sites corresponding to 5-mC in DNA sequences. The BS-seq process generally consists of two steps: 1) A pre-sequenced DNA sample is exposed to a sulfite treatment environment. This treatment converts unmethylated cytosine C in the sample into uracil (U), which is then replaced with thymine (T) during the subsequent DNA amplification process. This treatment prevents the conversion of the previously methylated cytosine C in the sample, known as 5-methylcytosine (5-mC). 2) The treated DNA sample is then sequenced using high-throughput sequencing technology to generate a new DNA sequence. By comparing the two sequences before and after treatment, the methylation status (yes / no) of each base can be determined, thereby providing the overall methylation status of the DNA sample. After obtaining methylation information from DNA samples using the BS-seq method, a series of bioinformatics tasks related to basic disease research can be performed, such as identifying the source of the sample based on its methylation information. The sample source mentioned here can be a type of tissue (such as the heart, liver, spleen, lung, stomach, duodenum, colon, etc.) or a type of cell (such as T cells, NK cells, monocytes, macrophages, granulocytes, and B cells). However, in practice, we have found that because the BS-seq method has a relatively cumbersome process and the various experimental instruments and sequencing equipment used in the process are also expensive, using conventional BS-seq methods to process DNA sample methylation sequence analysis tasks and bioinformatics identification tasks will face problems such as long task processing cycles and high processing costs. Summary of the Invention

[0004] The purpose of the present invention is to address the deficiencies of the prior art and provide a method, device, electronic device, and computer-readable storage medium for processing a methylation feature encoding model. The present invention sets a corresponding base marker M for 5-mC in the DNA sequence, and adjusts the four-base set of DNA [A, T, C, G] to a five-base set [A, T, C, G, M]; and performs big data collection on bisulfite sequencing reads of a certain species, and performs read segment splicing and methylation status marking based on the collected BS-seq read segment set to obtain a base sequence, and performs vocabulary learning based on the BPE algorithm and this base sequence; and then constructs an encoder model with the MosaicBERT model as the core, recorded as a methylation feature encoding model, and stores the learned vocabulary in the vocabulary within the encoding model. Storage module, and construct a decoder model for methylation sequence reconstruction recorded as a sequence decoding model, and construct a corresponding two-class prediction model based on a class of target source types recorded as a target source prediction model; then the methylation feature encoding model and the sequence decoding model form a pre-training model framework, and the methylation feature encoding model and the target source prediction model form a first task model framework; then, first train the pre-training model framework based on the above base sequence, and then train the first task model framework; finally, process the methylation sequence reconstruction task of the DNA sequence based on the pre-training model framework, and process a class of DNA sequence source identification tasks based on the first task model framework. The methylation feature encoding model provided by the present invention can learn the methylation features of DNA sequences. A pre-trained model framework with this methylation feature encoding model as the core encoder can be used to process methylation sequence analysis tasks. A first task model framework with this methylation feature encoding model as the core encoder can be used to process a type of target source identification task. Processing methylation sequence analysis tasks and target source identification tasks based on the methylation feature encoding model provided by the present invention and the corresponding two task model frameworks can not only simplify task processing steps, shorten task processing cycles, and improve the real-time performance of task processing, but also effectively reduce task processing costs.

[0005] To achieve the above-mentioned purpose, a first aspect of an embodiment of the present invention provides a method for processing a methylation feature coding model, the method comprising:

[0006] A corresponding extended marker is set for 5-methylcytosine in the DNA sequence as a base marker M, and the four-membered base set [A, T, C, G] of the DNA is adjusted to a corresponding five-membered base set [A, T, C, G, M]; big data is collected for bisulfite sequencing reads of the first species to form a corresponding BS-seq read set; reads are spliced ​​according to the BS-seq read set to obtain a corresponding first base sequence; and based on the five-membered base set [A, T, C, G, M] and the methylation state marker corresponding to each base marker C on the first base sequence, the first base sequence is converted into a five-element sequence to obtain a corresponding second base sequence; and vocabulary learning is performed based on the BPE algorithm and the second base sequence to obtain a corresponding first vocabulary;

[0007] A methylation feature encoding model is constructed with the MosaicBERT model as the core; the first vocabulary is stored in the vocabulary storage module of the methylation feature encoding model; a sequence decoding model for methylation sequence reconstruction is constructed; and a corresponding binary classification prediction model is constructed based on a target source type, which is specifically a tissue type or a cell type.

[0008] The methylation feature encoding model and the sequence decoding model form a pre-training model framework; and the pre-training model framework is trained with a first framework according to the second base sequence to obtain corresponding encoding model pre-training parameters;

[0009] A first task model framework is formed by the methylation feature encoding model and the target source prediction model; a model dataset is constructed based on the BS-seq read segment set and the target source type; and a second framework training is performed on the first task model framework based on the encoding model pre-training parameters and the model dataset;

[0010] After the training of the first and second frameworks is completed, the methylation sequence reconstruction task of the DNA sequence is processed based on the pre-trained model framework; and a type of DNA sequence source identification task corresponding to the target source type is processed based on the first task model framework.

[0011] Preferably, the base markers A, T, C, and G are markers for adenine, thymine, cytosine, and guanine bases, respectively;

[0012] The BS-seq read set is composed of multiple BS-seq read records; each of the BS-seq read records includes at least one BS-seq read and a corresponding read sample source; each of the BS-seq reads is a DNA sequence obtained by DNA sequencing of a DNA sample of the first species after bisulfite sequencing, and is sorted by base markers A, T, C, and G; each base marker C on each BS-seq read carries a corresponding methylation status marker; the methylation status marker is a binary status marker, including two states: unmethylated and methylated; the read sample source is a source type code corresponding to a current BS-seq read, and each source type code corresponds to a specific source type; the type range of the source type consists of multiple tissue types and / or multiple cell types of the first species;

[0013] The first base sequence is a marker sequence formed by sorting the base markers A, T, C, and G; the second base sequence is a marker sequence formed by sorting the base markers A, T, C, G, and M;

[0014] The first vocabulary table is composed of a plurality of first subword records; the first subword records include a first subword code, a first subword text, and a first subword frequency; the first subword text is a string formed by sorting one or more base markers in the five-element base set [A, T, C, G, M]; the first subword frequency is an integer value; all first subword texts in the first vocabulary table are different;

[0015] The model data set includes multiple first data records; the first data records include a first training sequence and a first label classification result; the first label classification result includes the target source type and the non-target source type.

[0016] Preferably, the step of collecting big data of bisulfite sequencing reads of the first species to form a corresponding BS-seq read set specifically includes:

[0017] Acquiring bisulfite sequencing reads of the DNA sequence of the first species through a plurality of preset data acquisition channels to obtain a plurality of BS-seq reads; the plurality of preset data acquisition channels at least comprising one or more public BS-seq read datasets and public technical literature containing BS-seq read information of the first species;

[0018] and setting a type range of the corresponding source type based on the sample source information corresponding to all the collected BS-seq reads; and setting a corresponding source type code for each source type in the type range of the source type; and using the source type code corresponding to each BS-seq read as the corresponding read sample source; and forming a corresponding BS-seq read record from each BS-seq read and the corresponding read sample source; and performing erroneous read deletion and duplicate read deduplication processing on all the obtained BS-seq read records;

[0019] The BS-seq read set is composed of all the remaining BS-seq read records.

[0020] Preferably, performing read segment splicing according to the BS-seq read segment set to obtain the corresponding first base sequence specifically includes:

[0021] All the BS-seq reads in the BS-seq read set are sequentially spliced ​​to obtain the corresponding first base sequence.

[0022] Preferably, performing five-element sequence conversion on the first base sequence based on the five-element base set [A, T, C, G, M] and the methylation status mark corresponding to each base marker C on the first base sequence to obtain the corresponding second base sequence specifically includes:

[0023] Perform sequence replication on the first base sequence to obtain a corresponding first replication sequence; and modify the status value of each current methylation status mark on the first replication sequence from the methylated base marker C to the corresponding base marker M; and use the first replication sequence after the marker modification as the corresponding second base sequence.

[0024] Preferably, the performing vocabulary learning based on the BPE algorithm and the second base sequence to obtain the corresponding first vocabulary specifically includes:

[0025] Step 61: Set an empty vocabulary as the corresponding first vocabulary; initialize the first vocabulary; set a threshold for the total number of records in the first vocabulary; and use the second base sequence as a corresponding first character string;

[0026] The initialized first vocabulary consists of six initial first subword records; the first subword texts of the six initial first subword records are character 'A', character 'T', character 'C', character 'G', character 'M' and character string 'CG' in sequence; the first subword word frequencies of the six initial first subword records are all initialized to 1;

[0027] Step 62: taking each of the first subword texts in the first vocabulary as a corresponding basic subword, and forming a corresponding current subword set from all the obtained basic subwords; and performing subword sequence splitting on the first character string according to the current subword set according to the subword sequence splitting method of the BPE algorithm to obtain a corresponding current subword sequence;

[0028] The current subword sequence is formed by sequentially sorting a plurality of the basic subwords; the basic subword pair is formed by sequentially sorting the first and last two basic subwords;

[0029] Step 63, according to the subword pair combination method of the BPE algorithm, each two adjacent basic subwords in the current subword sequence form a corresponding basic subword pair; and cluster the same basic subword pairs to obtain multiple cluster word pair sets; and use the total number of the basic subword pairs in each cluster word pair set as the corresponding first word pair number, and use the largest first word pair number as the corresponding high-frequency word pair number; and identify the high-frequency word pair number; if the high-frequency word pair number is 1, go to step 68; if the high-frequency word pair number is greater than 1, use the total number of the cluster word pair sets corresponding to the high-frequency word pair number as the corresponding high-frequency word pair number, and identify the high-frequency word pair number. If the high-frequency word pair number is 1, go to step 64; if the high-frequency word pair number is greater than 1, go to step 65;

[0030] Each of the clustered word pair sets is composed of one or more basic sub-word pairs, and all the basic sub-word pairs in each clustered word pair set are the same;

[0031] Step 64, taking the basic subword pair corresponding to the only one of the clustered word pair set corresponding to the total number of high-frequency word pairs as the corresponding final-selected subword pair; and then going to step 66;

[0032] Step 65: record all the clustered word pair sets corresponding to the total number of high-frequency word pairs as corresponding candidate sets; and record the basic subword pairs corresponding to each candidate set as the corresponding candidate subword pair; and record the character 'C', the character 'M', and the character string 'CG' as methylation-related characters; and record each candidate subword pair containing the methylation-related characters as a preferred subword pair; and identify the total number of the preferred subword pairs; if the total number of the preferred subword pairs is 0, select one from all the candidate subword pairs as the corresponding final subword pair; if the total number of the preferred subword pairs is greater than 0, select one from all the preferred subword pairs as the corresponding final subword pair;

[0033] Step 66: Add a new first subword record to the first vocabulary as the corresponding current newly added record, and set the first subword text of the current newly added record as the corresponding final subword pair; and after the record setting is completed, use each first subword text in the current first vocabulary as a new basic subword, and form the latest current subword set from all the newly obtained basic subwords; and according to the subword sequence splitting method of the BPE algorithm, perform subword sequence splitting on the first character string according to the current subword set to obtain the latest current subword sequence; and count the total number of each basic subword appearing in the current subword sequence and use the statistical result as the corresponding first word frequency, and set the first word frequency corresponding to each basic subword in the current subword set that does not appear in the current subword sequence to 0; and update the first subword frequency corresponding to each first subword text in the current first vocabulary to the corresponding first word frequency; and after the update is completed, delete the first subword record in the first vocabulary whose first subword frequency is 0;

[0034] Step 67: Count the total number of the first subword records in the current first vocabulary to obtain the corresponding total number of current records; and determine whether the total number of current records is less than the total number of records threshold; if so, return to step 63; if not, go to step 68;

[0035] Step 68: re-sort all the first sub-word records in the latest first vocabulary in descending order of the first sub-word frequency; and based on the preset sub-word encoding rules, set the first sub-word encoding corresponding to each first sub-word text in the sorted first vocabulary; and output the first vocabulary after the sub-word encoding setting as the vocabulary learning result of this time.

[0036] Preferably, the methylation feature encoding model is used to perform feature encoding on a first sequence X input to the model and output a corresponding first encoding vector Y; the first sequence X is composed of a plurality of first markers x i Sort by order; each of the first markers x i is a type of base marker in the five-element base set [A, T, C, G, M]; 1≤ marker index i≤L, L is the sequence length of the first sequence X; the first encoding vector Y is composed of multiple first word segmentation feature vectors y j Composition; 1≤ word segmentation index j≤W, W is the sequence length of the first word segmentation sequence S corresponding to the first sequence X;

[0037] The methylation feature encoding model is composed of a sequence word segmenter, the vocabulary storage module and the MosaicBERT model; the input end of the sequence word segmenter is connected to the model input end of the methylation feature encoding model, and the output end is connected to the input end of the MosaicBERT model; the sequence word segmenter is also connected to the vocabulary storage module; the output end of the MosaicBERT model is connected to the model output end of the methylation feature encoding model;

[0038] The sequence word segmenter is used to split the first sequence X into subwords according to the first vocabulary stored in the vocabulary storage module in a subword sequence splitting manner of the BPE algorithm to obtain a first subword sequence composed of multiple first subword texts; and add a preset classification subword text 'CLS' before the first first subword text in the first subword sequence; and use the sequence length of the added first subword sequence as the corresponding sequence length W; and use each subword text in the first subword sequence as a corresponding first segmentation s j , and consists of all the first participles s j The first segmentation sequence S is formed; and based on the first subword code corresponding to each first subword text in the first vocabulary and the classification subword code corresponding to the classification subword text 'CLS', each first segmentation s in the first segmentation sequence S is j The corresponding code is set to obtain the corresponding first segmentation code, and the corresponding first segmentation code sequence is composed of all the first segmentation codes obtained; and the built-in masking rule type is identified; if the masking rule type is no mask, the masking modification of the first segmentation code sequence is not performed; if the masking rule type is a type of random mask, the first segmentation code in the first segmentation code sequence is randomly masked according to the built-in type of random mask ratio and the preset segmentation mask code, and it is ensured that the total number of the masked first segmentation codes and the ratio of the sequence length of the first segmentation code sequence and the type of random mask ratio match; if the masking rule type is a type of second random mask, the first segmentation s containing methylation-related characters in the first segmentation sequence S are randomly masked. jRecorded as methylation-related participles, and the first participle codes corresponding to each of the methylation-related participles in the first participle code sequence are recorded as methylation-related codes, and the methylation-related codes in the first participle code sequence are randomly masked according to the built-in two-category random mask ratio and the participle mask code, and ensure that the ratio of the total number of masked methylation-related codes to the total number of methylation-related codes in the first participle code sequence and the two-category random mask ratio match; and based on the model input vector embedding coding rule of the MosaicBERT model, the first participle code sequence is embedded and coded to obtain the corresponding first embedded coding vector E and send it to the MosaicBERT model; the first participle sequence S consists of W first participles s j Composition, the first first participle s j＝1 For the classification subword text 'CLS', the second to last first participle s 2≤j≤W Each of the first participles s in j Corresponding to one of the first subword texts in the first vocabulary; the first embedding encoding vector E is composed of W first word embedding encoding vectors e j The first word segmentation embedding encoding vector e j With the first participle s j One-to-one correspondence;

[0039] The MosaicBERT model is used to perform feature encoding processing based on the first embedded coding vector E and output the corresponding first coding vector Y.

[0040] Preferably, the sequence decoding model is used to reconstruct the methylation sequence according to the first encoding vector Y input by the model and output the corresponding first decoding sequence Z; the first decoding sequence Z consists of L second markers z i The second marker z is sorted in sequence; i is a base marker in the five-base set [A, T, C, G, M]; the second marker z i With the first marker x i One-to-one correspondence;

[0041] The sequence decoding model consists of a first mapping layer, a first linear layer and a first Softmax layer; the first mapping layer is implemented based on a fully connected network; the first linear layer is based on another fully connected network;

[0042] The input end of the first mapping layer is connected to the model input end of the sequence decoding model, and the output end is connected to the input end of the first linear layer; the output end of the first linear layer is connected to the input end of the first Softmax layer; the output end of the first Softmax layer is connected to the model output end of the sequence decoding model;

[0043] The first mapping layer is used to convert the first word segmentation feature vectors y from the 2nd to the Wth in the first encoding vector Y 2≤j≤W Extracted to form a first extraction vector A of length W-1; and converted into a first mapping vector M of length L through linear transformation and sent to the first linear layer; the first extraction vector A is composed of W-1 first extraction sub-vectors a k Composition, 1≤ vector index k≤(W-1), each of the first extracted sub-vectors a k The first word segmentation feature vector y corresponding to j＝k+1 Matching; the first mapping vector M is composed of L first sub-mapping vectors m i composition;

[0044] The first linear layer is used to perform full connection calculation based on the first mapping vector M to obtain a corresponding first score vector B and send it to the first Softmax layer; the first score vector B is composed of L first score values ​​b i Composition, each of the first rating values ​​b i is a real number;

[0045] The first Softmax layer is used to perform five-category probability prediction based on the first score vector B to obtain the corresponding first prediction vector U; and i The base marker corresponding to the largest first predicted probability is used as the corresponding second marker z i ; and all the second markers z are obtained i Sequentially sorted to form the corresponding first decoding sequence Z; the first prediction vector U is composed of L first sub-vectors u i Each of the first subvectors u i It consists of five first prediction probabilities, each of which corresponds to a type of base marker in the five-element base set [A, T, C, G, M].

[0046] Preferably, the target source prediction model is used to perform binary classification prediction based on the first encoding vector Y input by the model and output a corresponding first prediction result R; the first prediction result R is a binary prediction result, consisting of two prediction results of the target source type and the non-target source type;

[0047] The target source prediction model consists of a first extraction module, a second linear layer, and a second Softmax layer; the second linear layer is based on another fully connected network;

[0048] The input end of the first extraction module is connected to the model input end of the target source prediction model, and the output end is connected to the input end of the second linear layer; the output end of the second linear layer is connected to the input end of the second Softmax layer; the output end of the second Softmax layer is connected to the model output end of the target source prediction model;

[0049] The first extraction module is used to extract the first first word segmentation feature vector y in the first encoding vector Y j＝1 The extracted vector is sent as the corresponding second extracted vector C to the second linear layer;

[0050] The second linear layer is used to perform a full connection calculation based on the second extraction vector C to obtain a corresponding second score vector D and send it to the second Softmax layer; the second score vector D is composed of two second scores d1 and d2;

[0051] The second Softmax layer is used to perform binary probability prediction based on the second scores d1 and d2 of the second scoring vector D to obtain the corresponding second prediction vector V; and identify the two second prediction probabilities v1 and v2 of the second prediction vector V; if the second prediction probability v1 is too large, the corresponding first prediction result R is set to the target source type; if the second prediction probability v2 is too large, the corresponding first prediction result R is set to the non-target source type; the second prediction vector V is composed of two second prediction probabilities v1 and v2, wherein the second prediction probability v1 is a true value probability and the second prediction probability v2 is a false value probability.

[0052] Preferably, performing first framework training on the pre-trained model framework according to the second base sequence to obtain corresponding encoding model pre-training parameters specifically includes:

[0053] Step 1001: Use the pre-trained model framework as the corresponding current model framework; use the methylation feature encoding model and the sequence decoding model of the current model framework as the corresponding current encoding model and current decoding model; set two positive integers as the corresponding first and second stage training times thresholds; set two ratio parameters with values ​​between 0 and 1 as the corresponding first and second ratio parameters; set the first and second type random mask ratios built into the sequence word segmenter of the current encoding model as the corresponding first and second ratio parameters; and set two counters initialized to 0 as the corresponding first and second counters;

[0054] Wherein, the first proportion parameter is less than the second proportion parameter, 10% is less than the first proportion parameter and less than 20%, and 90% is less than the second proportion parameter and less than 100%;

[0055] Step 1002: Take the second base sequence as the corresponding first sequence X; perform label vector conversion on the current first sequence X to obtain a corresponding label vector PG; and perform smoothed label vector conversion on the label vector PG to obtain a corresponding smoothed label vector PS;

[0056] Among them, the label vector PG consists of L sub-vectors pg i Composition; the subvector pg i and the first marker x of the first sequence X i One-to-one correspondence; each of the subvectors pg i Each is a one-hot encoding vector of length 5, consisting of 5 bases one-hot encoding pgA i 、pgT i 、pgC i 、pgG i 、pgM i Composition; each of the sub-vectors pg i The five base unique hot codes correspond one-to-one to the five types of base markers in the five-element base set [A, T, C, G, M]; each of the subvectors pg i Among the five base one-hot codes, only one is 1 and the other four are 0. The base marker corresponding to the base one-hot code of 1 is the same as the current sub-vector pg i The corresponding first marker x i match;

[0057] The smoothed label vector PS consists of L sub-vectors ps i Composition; the sub-vector ps i and the subvector pg of the label vector PG i One-to-one correspondence; each of the sub-vectors ps i The length of the vector is 5, and psA is encoded by 5 vectors i 、psT i 、psC i , psG i 、psM i Composition; each of the sub-vectors ps i The five vector codes correspond one-to-one to the five types of base markers in the five-element base set [A, T, C, G, M];

[0058] The subvector pg i With the subvector ps iThe conversion relationship is:

[0059]

[0060] α is a preset smoothing parameter, with a value between 0 and 1; I is a preset vector of all 1s with a length of 5; the first condition is: the subvector pg i The corresponding first marker x i is a base marker C or M; the second condition is: the subvector pg i The corresponding first marker x i is the base marker A, T or G;

[0061] Step 1003: Set the mask rule type built into the sequence word segmenter of the current coding model to a type of random mask;

[0062] Step 1004: input the first sequence X into the sequence word segmenter of the current encoding model for processing to obtain the corresponding first embedded code vector E;

[0063] Step 1005: Input the first embedded coding vector E into the MosaicBERT model of the current coding model for processing to obtain the corresponding first coding vector Y; input the first coding vector Y into the current decoding model for processing to obtain the corresponding first decoding sequence Z; and input the first subvectors u of the first prediction vector U generated during the current decoding model processing to obtain the first decoding sequence Z. i As a corresponding subvector pr i , and all the sub-vectors pr are obtained by i Form the corresponding prediction vector PR;

[0064] The prediction vector PR consists of L sub-vectors pr i Composition; the sub-vector pr i and the second marker z of the first decoding sequence Z i One-to-one correspondence; the sub-vector pr i The first predicted probability of the five is denoted as prA i prT i 、prC i , prG i 、prM i , which correspond one-to-one to the five types of base markers in the five-member base set [A, T, C, G, M];

[0065] Step 1006: Substitute the prediction vector PR and the smoothed label vector PS into the preset first model loss function L M1 Calculate and obtain the corresponding first loss value;

[0066] Among them, the first model loss function L M1 for:

[0067]

[0068] Step 1007, identifying whether the first loss value meets the preset first loss value range; if so, adding 1 to the first counter and going to step 1008; if not, moving towards making the first model loss function L based on the preset first model optimizer M1 Perform a round of modulation on the full model parameters of the current encoding model and the full model parameters of the current decoding model in the direction of reaching the minimum value, and return to step 1005 when this round of modulation is completed;

[0069] Wherein, the first model optimizer includes at least an Adam optimizer and an SGD optimizer;

[0070] Step 1008: Identify whether the first counter exceeds the threshold of the number of training times in one stage; if not, return to step 1004 to continue training; if exceeded, go to step 1009;

[0071] Step 1009: setting the mask rule type built into the sequence word segmenter of the current coding model to a second-class random mask;

[0072] Step 1010: Input the first sequence X into the sequence word segmenter of the current encoding model for processing to obtain the corresponding first embedded code vector E;

[0073] Step 1011: Input the first embedded coding vector E into the MosaicBERT model of the current coding model for processing to obtain the corresponding first coding vector Y; input the first coding vector Y into the current decoding model for processing to obtain the corresponding first decoding sequence Z; and input the first subvectors u of the first prediction vector U generated during the current decoding model processing to obtain the first decoding sequence Z. i As a corresponding subvector pr i , and all the sub-vectors pr are obtained by i Form the corresponding prediction vector PR;

[0074] Step 1012: Substitute the prediction vector PR and the smoothed label vector PS into the first model loss function L M1 Calculate and obtain the corresponding second loss value;

[0075] Step 1013, identifying whether the second loss value meets the preset second loss value range; if so, adding 1 to the second counter and going to step 1014; if not, moving towards making the first model loss function L based on the first model optimizer M1 Perform a round of modulation on the full model parameters of the current encoding model and the full model parameters of the current decoding model in the direction of reaching the minimum value, and return to step 1011 when this round of modulation is completed;

[0076] Step 1014: Identify whether the second counter exceeds the second-stage training times threshold; if not, return to step 1010 to continue training; if exceeded, go to step 1015;

[0077] Step 1015, confirming that the first framework training is completed, and saving the current model parameters of the current coding model as the corresponding pre-training parameters of the coding model; and solidifying the framework model parameters of the pre-training model framework.

[0078] Preferably, constructing a model dataset based on the BS-seq read set and the target source type specifically includes:

[0079] A corresponding positive sample set is formed by the BS-seq read records whose read sample sources in the BS-seq read set match the target source type, and a corresponding negative sample set is formed by the BS-seq read records whose read sample sources in the BS-seq read set do not match the target source type;

[0080] and taking the BS-seq reads of each of the BS-seq read records of the positive sample set as a corresponding first training sequence, and setting a first label classification result specifically set to the target source type; and forming a corresponding first data record by the first training sequence and the first label classification result corresponding to each of the BS-seq read records of the positive sample set;

[0081] and taking the BS-seq reads of each of the BS-seq read records of the negative sample set as a corresponding first training sequence, and setting a first label classification result specifically set to the non-target source type; and forming a corresponding first data record by the first training sequence and the first label classification result corresponding to each of the BS-seq read records of the negative sample set;

[0082] All the first data records obtained form the corresponding model data set.

[0083] Preferably, the performing second framework training on the first task model framework based on the encoding model pre-training parameters and the model data set specifically includes:

[0084] Step 1201: Use the first task model framework as the corresponding current model framework; use the methylation feature encoding model and the target source prediction model of the current model framework as the corresponding current encoding model and current prediction model; and set the mask rule type built into the sequence word segmenter of the current encoding model to no mask;

[0085] Step 1202: solidify the model parameters of the methylation feature encoding model into the corresponding pre-trained parameters of the encoding model; and based on the LoRA fine-tuning mechanism, load a corresponding LoRA adapter on one or more designated modules of the methylation feature encoding model; and form a corresponding adapter parameter set from the adapter parameters of all the LoRA adapters;

[0086] Among them, the solidified model parameters of each designated module are recorded as the parameters before fine-tuning W0, and the adapter parameters of the LoRA adapter corresponding to each designated module are composed of the matrix parameters of two low-rank matrices A and B; each LoRA adapter is used to fine-tune the parameters before fine-tuning W0 of the corresponding designated module to obtain the corresponding parameters after fine-tuning W1; the conversion relationship between the parameters after fine-tuning W1 and the parameters before fine-tuning W0 is: W1X in =W0X in +BAX in , X in is the input vector of the specified module, W0X in The current designated module is based on the input vector X when the corresponding LoRA adapter is not added in The output vector obtained by processing, W1X in The current specified module is based on the input vector X after adding the corresponding LoRA adapter in The output vector obtained by processing;

[0087] Step 1203: dividing the model dataset into two sub-datasets according to a preset first segmentation ratio and recording them as a corresponding first training set and a first evaluation set;

[0088] wherein both the first training set and the first evaluation set are composed of a plurality of the first data records; the ratio of the total number of records in the first training set to the total number of records in the first evaluation set satisfies the first segmentation ratio; and the ratio of positive and negative samples in the first training set and the first evaluation set are the same;

[0089] Step 1204: taking the first first data record of the first training set as the corresponding current training record;

[0090] Step 1205: setting a corresponding label vector TG based on the first label classification result of the current training record;

[0091] Wherein, the label vector TG is composed of two one-hot codes tg1 and tg2; the one-hot codes tg1 and tg2 correspond to the target source type and the non-target source type, respectively; if the first label classification result of the current training record is the target source type, the one-hot code tg1 is 1 and the one-hot code tg2 is 0; if the first label classification result of the current training record is the non-target source type, the one-hot code tg1 is 0 and the one-hot code tg2 is 1;

[0092] Step 1206: Input the first training sequence of the current training record as the corresponding first sequence X into the current model framework for processing to obtain the corresponding first prediction result R; and set a corresponding prediction vector TR based on the second prediction vector V generated by the current prediction model during this processing;

[0093] The prediction vector TR is composed of two vector codes tr1 and tr2; the vector code tr1 is the second prediction probability v1 of the second prediction vector V, and the vector code tr2 is the second prediction probability v2 of the second prediction vector V;

[0094] Step 1207: Substitute the prediction vector TR and the label vector TG into the preset second model loss function L M2 Calculate and obtain the corresponding third loss value;

[0095] Among them, the second model loss function L M2 Implemented based on L1 loss function, L2 loss function or cross entropy loss function;

[0096] Step 1208, whether the third loss value satisfies the preset third loss value range is identified; if the third loss value satisfies the third loss value range, whether the current training record is the last first data record of the first training set is identified, if so, go to step 1209, if not, the next first data record of the first training set is used as the new current training record and return to step 1205 to continue training; if the third loss value does not satisfy the third loss value range, then based on the preset second model optimizer, the second model loss function L is adjusted to the optimal value. M2Perform a round of modulation on the adapter parameter set and the full model parameters of the current prediction model in the direction of reaching the minimum value, and return to step 1206 when this round of modulation is completed;

[0097] Wherein, the second model optimizer includes at least an Adam optimizer and an SGD optimizer;

[0098] Step 1209: perform a round of traversal on all the first data records of the first evaluation set; and during this round of traversal, use the first data record currently traversed as the corresponding current evaluation record; and set a corresponding label vector TG based on the first label classification result of the current evaluation record; and input the first training sequence of the current evaluation record as the corresponding first sequence X into the current model framework for processing to obtain the corresponding first prediction result R, and set a corresponding prediction vector TR based on the second prediction vector V generated by the current prediction model during this processing; and form a corresponding prediction-label pair by the label vector TG and the prediction vector TR corresponding to the current evaluation record; and at the end of this round of traversal, bring all the obtained prediction-label pairs into the preset first model evaluation function for calculation to obtain the corresponding first evaluation value; and calculate the accuracy, precision, recall rate, and F1 score based on all the obtained prediction-label pairs to obtain the corresponding first accuracy, first precision, first recall rate, and first F1 score;

[0099] Wherein, the first model evaluation function is implemented based on the MSE function or the RMSE function;

[0100] Step 1210: Identify whether the first evaluation value, the first accuracy, the first precision, the first recall rate, and the first F1 score all satisfy the corresponding first evaluation value range, first accuracy range, first precision range, first recall rate range, and first F1 score range; if all satisfy, proceed to step 1211; if at least one of the first evaluation value, the first accuracy, the first precision, the first recall rate, and the first F1 score does not satisfy the corresponding range, return to step 1203 to continue training;

[0101] Step 1211, confirming that the second framework training is completed, and saving the encoding model pre-training parameters and the latest adapter parameter set as the corresponding fine-tuning encoding model parameter set; and solidifying the framework model parameters of the first task model framework.

[0102] Preferably, the methylation sequence reconstruction task of processing the DNA sequence based on the pre-trained model framework specifically includes:

[0103] The DNA sequence received as input by the user is recorded as the corresponding first DNA sequence; the first DNA sequence is input as the corresponding first sequence X into the pre-trained model framework for processing to obtain the corresponding first decoding sequence Z, and the current first decoding sequence Z is fed back to the current user as the corresponding first reconstructed sequence; wherein the first DNA sequence is a marker sequence formed by sorting base markers in the four-element base set [A, T, C, G]; and the first reconstructed sequence is a marker sequence formed by sorting base markers in the five-element base set [A, T, C, G, M].

[0104] Preferably, the processing of a DNA sequence source identification task corresponding to the target source type based on the first task model framework specifically includes:

[0105] The DNA sequence received from the user is recorded as the corresponding second DNA sequence; the second DNA sequence is input into the first task model framework as the corresponding first sequence X for processing to obtain the corresponding first prediction result R, and the current first prediction result R is fed back to the current user as the corresponding first recognition result; wherein the second DNA sequence is a marker sequence formed by sorting the base markers in the four-base set [A, T, C, G] or the five-base set [A, T, C, G, M].

[0106] A second aspect of an embodiment of the present invention provides a device for implementing the processing method of the methylation feature coding model described in the first aspect, the device comprising: a preprocessing module, a model building module, a first training module, a second training module, and a model application module;

[0107] The preprocessing module is used to set a corresponding extended marker for 5-methylcytosine in the DNA sequence as a base marker M, and adjust the four-membered base set [A, T, C, G] of the DNA to a corresponding five-membered base set [A, T, C, G, M]; and perform big data collection on the bisulfite sequencing reads of the first species to form a corresponding BS-seq read set; and perform read segment splicing based on the BS-seq read set to obtain a corresponding first base sequence; and perform five-element sequence conversion on the first base sequence based on the five-membered base set [A, T, C, G, M] and the methylation state marker corresponding to each base marker C on the first base sequence to obtain a corresponding second base sequence; and perform vocabulary learning based on the BPE algorithm and the second base sequence to obtain a corresponding first vocabulary;

[0108] The model construction module is used to construct a methylation feature encoding model with the MosaicBERT model as the core; store the first vocabulary in the vocabulary storage module of the methylation feature encoding model; and construct a sequence decoding model for methylation sequence reconstruction; and construct a corresponding binary classification prediction model based on a target source type, recorded as a target source prediction model; the target source type is specifically a tissue type or a cell type;

[0109] The first training module is used to form a pre-training model framework by using the methylation feature encoding model and the sequence decoding model; and perform first framework training on the pre-training model framework according to the second base sequence to obtain corresponding encoding model pre-training parameters;

[0110] The second training module is used to form a first task model framework by the methylation feature encoding model and the target source prediction model; and to construct a model dataset based on the BS-seq read segment set and the target source type; and to perform a second framework training on the first task model framework based on the encoding model pre-training parameters and the model dataset;

[0111] The model application module is used to process the methylation sequence reconstruction task of the DNA sequence based on the pre-trained model framework after the training of the first and second frameworks is completed; and to process a type of DNA sequence source identification task corresponding to the target source type based on the first task model framework.

[0112] A third aspect of an embodiment of the present invention provides an electronic device, including: a memory, a processor, and a transceiver;

[0113] The processor is configured to be coupled to the memory, read and execute instructions in the memory, so as to implement the method steps described in the first aspect above;

[0114] The transceiver is coupled to the processor, and the processor controls the transceiver to send and receive messages.

[0115] A fourth aspect of an embodiment of the present invention provides a computer-readable storage medium, which stores computer instructions. When the computer instructions are executed by a computer, the computer executes the instructions of the method described in the first aspect above.

[0116] The embodiment of the present invention provides a processing method, device, electronic device and computer-readable storage medium for a methylation feature coding model. As can be seen from the above content, the embodiment of the present invention sets a corresponding base marker M for 5-mC in the DNA sequence, and adjusts the four-base set [A, T, C, G] of DNA to a five-base set [A, T, C, G, M]; and performs big data collection on the bisulfite sequencing reads of a certain species, and performs read segment splicing and methylation status marking based on the collected BS-seq read segment set to obtain a base sequence, and performs vocabulary learning based on the BPE algorithm and this base sequence; and then constructs an encoder model with the MosaicBERT model as the core, recorded as the methylation feature coding model, and stores the learned vocabulary in the encoding model. The vocabulary storage module inside the model is constructed, and a decoder model for methylation sequence reconstruction is recorded as a sequence decoding model, and a corresponding two-class prediction model based on a class of target source types is constructed as a target source prediction model; then the methylation feature encoding model and the sequence decoding model form a pre-training model framework, and the methylation feature encoding model and the target source prediction model form a first task model framework; then, the pre-training model framework is trained based on the base sequence above, and then the first task model framework is trained; finally, the methylation sequence reconstruction task of the DNA sequence is processed based on the pre-training model framework, and the class of DNA sequence source identification task is processed based on the first task model framework. The methylation feature encoding model provided in the embodiment of the present invention can learn the methylation features of DNA sequences. The pre-trained model framework with this methylation feature encoding model as the core encoder can be used to process methylation sequence analysis tasks. The first task model framework with this methylation feature encoding model as the core encoder can be used to process a type of target source identification task. The methylation feature encoding model provided in the embodiment of the present invention and the corresponding two task model frameworks are used to process methylation sequence analysis tasks and target source identification tasks, which not only simplifies the task processing steps, shortens the task processing cycle, and improves the real-time performance of task processing, but also effectively reduces the task processing cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0117] Figure 1 A schematic diagram of a method for processing a methylation feature coding model provided in Example 1 of the present invention;

[0118] Figure 2 Module structure diagram of the pre-training model framework, first task model framework, methylation feature encoding model, sequence decoding model, and target source prediction model provided in Example 1 of the present invention;

[0119] Figure 3 A module structure diagram of a processing device for a methylation feature coding model provided in the second embodiment of the present invention;

[0120] Figure 4 This is a structural diagram of an electronic device provided in Example 3 of the present invention. DETAILED DESCRIPTION

[0121] To make the objectives, technical solutions, and advantages of the present invention more apparent, the present invention will be further described in detail below with reference to the accompanying drawings. It should be understood that the embodiments described herein are merely some, rather than all, of the present invention. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without inventive effort are intended to fall within the scope of protection of the present invention.

[0122] The first embodiment of the present invention provides a method for processing a methylation feature coding model, such as Figure 1 A schematic diagram of a method for processing a methylation feature coding model provided in Example 1 of the present invention is shown. The method mainly includes the following steps:

[0123] Step 1: Set a corresponding extended marker for 5-methylcytosine in the DNA sequence as the base marker M, and adjust the four-base set of DNA [A, T, C, G] to the corresponding five-base set [A, T, C, G, M]; and collect big data of the bisulfite sequencing reads of the first species to form a corresponding BS-seq read set; and perform read segment splicing based on the BS-seq read set to obtain the corresponding first base sequence; and perform five-element sequence conversion on the first base sequence based on the five-base set [A, T, C, G, M] and the methylation state marker corresponding to each base marker C on the first base sequence to obtain the corresponding second base sequence; and perform vocabulary learning based on the BPE algorithm and the second base sequence to obtain the corresponding first vocabulary;

[0124] Specifically, the method includes: step 11, setting a corresponding extended marker for 5-methylcytosine in the DNA sequence as a base marker M, and adjusting the DNA four-base set [A, T, C, G] to the corresponding five-base set [A, T, C, G, M];

[0125] Among them, the base markers A, T, C, and G are markers for adenine, thymine, cytosine, and guanine bases, respectively;

[0126] Step 12, performing big data collection on the bisulfite sequencing reads of the first species to form a corresponding BS-seq read set;

[0127] Specifically comprising: step 121, collecting data of bisulfite sequencing reads of a DNA sequence of a first species through a plurality of preset data collection channels to obtain a plurality of BS-seq reads;

[0128] Here, the first species in the embodiment of the present invention is human by default, and may be one of multiple species besides human, for example, monkey, orangutan, etc.; the multiple preset data collections include at least one or more public BS-seq read datasets and public technical literature containing BS-seq read information of the first species;

[0129] Step 122: Based on the sample source information corresponding to all collected BS-seq reads, a type range of corresponding source types is set; a corresponding source type code is set for each source type in the type range of source types; the source type code corresponding to each BS-seq read is used as the corresponding read sample source; each BS-seq read and the corresponding read sample source form a corresponding BS-seq read record; and all obtained BS-seq read records are processed to delete erroneous reads and deduplicate duplicate reads;

[0130] Step 123, forming a BS-seq read set from all remaining BS-seq read records;

[0131] Here, the BS-seq read set of the embodiment of the present invention is composed of multiple BS-seq read records; each BS-seq read record includes at least one BS-seq read and a corresponding read sample source; each BS-seq read is a DNA sequence obtained by DNA sequencing of a DNA sample of the first species after bisulfite sequencing, and is sorted by base markers A, T, C, and G; each base marker C on each BS-seq read carries a corresponding methylation status marker; the methylation status marker is a binary state marker, including two states: unmethylated and methylated; the read sample source is a source type code corresponding to a current BS-seq read, and each source type code corresponds to a specific source type; the type range of the source type consists of multiple tissue types and / or multiple cell types of the first species;

[0132] Step 13: performing read segment splicing based on the BS-seq read segment set to obtain the corresponding first base sequence;

[0133] Specifically comprising: sequentially splicing all BS-seq reads in the BS-seq read set to obtain the corresponding first base sequence;

[0134] Here, the first base sequence of the embodiment of the present invention is a marker sequence composed of base markers A, T, C, and G in sequence;

[0135] Step 14: performing a five-element sequence conversion on the first base sequence based on the five-element base set [A, T, C, G, M] and the methylation status mark corresponding to each base marker C on the first base sequence to obtain a corresponding second base sequence;

[0136] Specifically, the method includes: performing sequence replication on a first base sequence to obtain a corresponding first replication sequence; modifying the status value of each current methylation status mark on the first replication sequence from a methylated base marker C to a corresponding base marker M; and using the first replication sequence after the marker modification as the corresponding second base sequence;

[0137] Here, the second base sequence of the embodiment of the present invention is a marker sequence formed by sorting the base markers A, T, C, G, and M;

[0138] Step 15: performing vocabulary learning based on the BPE (Byte Pair Encoding) algorithm and the second base sequence to obtain a corresponding first vocabulary;

[0139] Here, the first vocabulary of the embodiment of the present invention is composed of multiple first subword records; the first subword record includes a first subword code, a first subword text, and a first subword frequency; the first subword text is a string formed by sorting one or more base markers in the five-element base set [A, T, C, G, M]; the first subword frequency is an integer value; all first subword texts in the first vocabulary are different;

[0140] Specifically, the steps include: step 151, setting an empty vocabulary as a corresponding first vocabulary; initializing the first vocabulary; setting a threshold for the total number of records in the first vocabulary; and taking the second base sequence as a corresponding first character string;

[0141] Here, the first vocabulary table initialized in the embodiment of the present invention is composed of six initial first subword records; the first subword texts of the six initial first subword records are the character 'A', the character 'T', the character 'C', the character 'G', the character 'M' and the string 'CG', and the first subword frequency of the six initial first subword records is initialized to 1;

[0142] Step 152: Taking each first subword text in the first vocabulary as a corresponding basic subword, and forming a corresponding current subword set from all the obtained basic subwords; and performing subword sequence splitting on the first character string according to the current subword set according to the subword sequence splitting method of the BPE algorithm to obtain a corresponding current subword sequence;

[0143] Here, the current subword sequence is formed by sorting multiple basic subwords in sequence; the basic subword pair is formed by sorting the first and last two basic subwords;

[0144] For example, the first initialized vocabulary contains 6 subword texts with non-zero word frequencies: 'A', 'T', 'C', 'G', 'M', and 'CG'; suppose the first string is 'TACCGTAACGCTGC', then the current subword sequence obtained by the subword sequence splitting method of the dark BPE algorithm is [T, A, C, CG, T, A, A, CG, C, T, G, C];

[0145] Step 153, according to the subword pair combination method of the BPE algorithm, a corresponding basic subword pair is formed from every two adjacent basic subwords in the current subword sequence; the same basic subword pairs are clustered to obtain multiple clustered word pair sets; the total number of basic subword pairs in each clustered word pair set is used as the corresponding first word pair count, and the largest first word pair count is used as the corresponding high-frequency word pair count; the high-frequency word pair count is identified; if the high-frequency word pair count is 1, go to step 158; if the high-frequency word pair count is greater than 1, the total number of clustered word pair sets corresponding to the high-frequency word pair count is used as the corresponding high-frequency word pair count, and the high-frequency word pair count is identified. If the high-frequency word pair count is 1, go to step 154; if the high-frequency word pair count is greater than 1, go to step 155;

[0146] Here, each cluster word pair set is composed of one or more basic subword pairs, and all basic subword pairs of each cluster word pair set are the same;

[0147] For example, if the current subword sequence is [T, A, C, CG, T, A, A, CG, C, T, G, C], then the basic subword pairs obtained by the subword pair combination method of the BPE algorithm are [TA, AC, CCG, CGT, TA, AA, ACG, CGC, CT, TG, GC]; thus, the corresponding relationship of [basic subword pair - first word pair number] is [TA-2], [AC-1], [CCG-1], [CGT-1], [AA-1], [ACG-1], [CGC-1], [CT-1], [TG-1], [GC-1]; among which the number of high-frequency word pairs = 2 corresponds to the basic subword pair 'TA', and the total number of high-frequency word pairs is 1;

[0148] It should be noted that the embodiment of the present invention sets two exit conditions when performing vocabulary learning: 1) when there are no repeated word pairs among all basic subword pairs obtained from the current subword sequence, that is, the first word pair count of all basic subword pairs is 1, the vocabulary learning process is exited. At this time, the number of high-frequency word pairs = 1, so the current step 153 will jump to step 158 when the number of high-frequency word pairs = 1; 2) when the vocabulary reaches the upper limit, that is, the total number of first subword records with non-zero word frequency in the first vocabulary exceeds the total number of records threshold, the vocabulary learning process is exited. As will be seen below, this will be determined in step 157;

[0149] Step 154: taking the basic subword pair corresponding to the only clustered word pair set corresponding to the total number of high-frequency word pairs as the corresponding final-selected subword pair; and then going to step 156;

[0150] For example, if the number of high-frequency word pairs obtained in step 153 = 2 corresponds to the basic sub-word pair 'TA', and the total number of high-frequency word pairs is 1, then the final sub-word pair should be 'TA';

[0151] Step 155: Record all clustered word pair sets corresponding to the total number of high-frequency word pairs as corresponding candidate sets; record the basic subword pairs corresponding to each candidate set as the corresponding candidate subword pairs; record the character 'C', the character 'M', and the character string 'CG' as methylation-related characters; record each candidate subword pair containing methylation-related characters as a preferred subword pair; and identify the total number of preferred subword pairs; if the total number of preferred subword pairs is 0, select one from all candidate subword pairs as the corresponding final subword pair; if the total number of preferred subword pairs is greater than 0, select one from all preferred subword pairs as the corresponding final subword pair;

[0152] Here, if the total number of high-frequency word pairs identified in step 153 is greater than 1, and the first word pair counts of two or more basic subword pairs are equal to the high-frequency word pair counts; then, in this embodiment of the present invention, the basic subword pairs whose first word pair counts are equal to the high-frequency word pair counts are regarded as candidate subword pairs, and the candidate subword pairs containing methylation-related characters (C, M, CG) are defaulted as preferred subword pairs and one of them is selected as the final selected subword pair. If there is no basic subword pair containing methylation-related characters (C, M, CG) among all the candidate subword pairs, one of them is selected as the final selected subword pair.

[0153] Step 156: Add a new first subword record to the first vocabulary as the corresponding current newly added record, and set the first subword text of the current newly added record as the corresponding final subword pair; and after the record setting is completed, use each first subword text in the current first vocabulary as a new basic subword, and form the latest current subword set from all the latest basic subwords; and according to the subword sequence splitting method of the BPE algorithm, perform subword sequence splitting on the first character string according to the current subword set to obtain the latest current subword sequence; and count the total number of each basic subword appearing in the current subword sequence and use the statistical result as the corresponding first word frequency, and set the first word frequency of each basic subword in the current subword set that does not appear in the current subword sequence to 0; and update the first subword frequency corresponding to each first subword text in the current first vocabulary to the corresponding first word frequency; and after the update is completed, delete the first subword record in the first vocabulary with a first subword frequency of 0;

[0154] Step 157: Count the total number of first subword records in the current first vocabulary to obtain the corresponding total number of current records; and determine whether the total number of current records is less than a record total threshold; if so, return to step 153; if not, go to step 158;

[0155] Step 158: re-sort all first sub-word records in the latest first vocabulary in descending order of first sub-word frequency; and based on a preset sub-word encoding rule, set encoding for the first sub-word encoding corresponding to each first sub-word text in the sorted first vocabulary; and output the first vocabulary after the sub-word encoding setting as the vocabulary learning result of this time.

[0156] Here, the subword encoding rules of the embodiment of the present invention can be customized according to application requirements; conventionally, they can be set using sequentially increasing integers.

[0157] Step 2: Build a methylation feature encoding model with the MosaicBERT model as the core; store the first vocabulary into the vocabulary storage module of the methylation feature encoding model; build a sequence decoding model for methylation sequence reconstruction; and build a corresponding two-class prediction model based on a class of target source types, recorded as the target source prediction model.

[0158] Here, the target source type in the embodiment of the present invention is specifically a type of tissue or a type of cell.

[0159] The methylation feature coding model of the embodiment of the present invention is used to perform feature coding on the first sequence X input by the model and output the corresponding first coding vector Y; wherein the first sequence X is composed of a plurality of first markers x i Sort by order; each first marker x i is a type of base marker in the five-element base set [A, T, C, G, M]; 1≤ marker index i≤L, L is the sequence length of the first sequence X; the first encoding vector Y is composed of multiple first word feature vectors y j Composition; 1≤ word segmentation index j≤W, W is the sequence length of the first word segmentation sequence S corresponding to the first sequence X.

[0160] like Figure 2 As shown, the methylation feature encoding model of the embodiment of the present invention is composed of a sequence word segmenter, a vocabulary storage module and a MosaicBERT model.

[0161] The connection relationship between the components of the methylation feature encoding model is as follows: the input end of the sequence segmenter is connected to the model input end of the methylation feature encoding model, and the output end is connected to the input end of the MosaicBERT model; the sequence segmenter is also connected to the vocabulary storage module; the output end of the MosaicBERT model is connected to the model output end of the methylation feature encoding model.

[0162] The functions of each component of the methylation feature encoding model are as follows:

[0163] 1) The sequence word segmenter is used to split the first sequence X into subwords according to the subword sequence splitting method of the BPE algorithm according to the first vocabulary stored in the vocabulary storage module to obtain a first subword sequence composed of multiple first subword texts; and add a preset classification subword text 'CLS' before the first first subword text of the first subword sequence; and use the sequence length of the added first subword sequence as the corresponding sequence length W; and use each subword text of the first subword sequence as a corresponding first segmentation s j , and by all first participles s j Form the corresponding first segmentation sequence S; and based on the first subword code corresponding to each first subword text in the first vocabulary and the classification subword code corresponding to the classification subword text 'CLS', each first segmentation s in the first segmentation sequence S j The corresponding code is set to obtain the corresponding first segmentation code, and the corresponding first segmentation code sequence is composed of all the obtained first segmentation codes; and the built-in masking rule type is identified; if the masking rule type is no masking, the first segmentation code sequence is not masked; if the masking rule type is a type of random masking, the first segmentation code in the first segmentation code sequence is randomly masked according to the built-in type of random masking ratio and the preset segmentation masking code, and it is ensured that the total number of masked first segmentation codes and the ratio of the sequence length of the first segmentation code sequence and the type of random masking ratio match; if the masking rule type is a type of random masking, the first segmentation s containing methylation-related characters in the first segmentation sequence S is randomly masked. j Record them as methylation-related segmentations, and record the first segmentation codes corresponding to each methylation-related segmentation in the first segmentation code sequence as methylation-related codes. Randomly mask the methylation-related codes in the first segmentation code sequence according to the built-in two-category random mask ratio and segmentation mask code, and ensure that the ratio of the total number of masked methylation-related codes to the total number of methylation-related codes in the first segmentation code sequence and the two-category random mask ratio match. Based on the model input vector embedding coding rule of the MosaicBERT model, embed the first segmentation code sequence to obtain the corresponding first embedded coding vector E and send it to the MosaicBERT model.

[0164] Here, the first word segmentation sequence S of the embodiment of the present invention is composed of W first word segmentations s j Composition, the first first participle s j＝1 For the classification subword text 'CLS', the second to last first participle s 2≤j≤W Each first participle s in j Corresponding to a first subword text in the first vocabulary; the first embedding encoding vector E is composed of W first word embedding encoding vectors e j Composition, the first word embedding encoding vector e j With the first participle s j One-to-one correspondence;

[0165] The sequence word segmenter has three built-in parameters: masking rule type, type one random masking ratio, and type two random masking ratio; the masking rule types include no masking, type one random masking, and type two random masking; 1) When the masking rule type is set to no masking, it means that there is no need to mask the first word segmentation coding sequence; 2) When the masking rule type is set to type one random masking, it means that the first word segmentation coding sequence needs to be randomly masked according to the type one random masking ratio. For example, the first word segmentation coding sequence contains 100 first word segmentation codes, and the type one random masking ratio is set to 15%. That means that any 100×15%=15 first word segmentation codes in the first word segmentation coding sequence need to be masked. Specifically, 15 first word segmentation codes are randomly selected from the first word segmentation coding sequence. word coding, and resetting the coding values ​​of these 15 first word segmentation codes to word segmentation mask codes, where the word segmentation mask code is a pre-selected coding value; 3) when the masking rule type is set to second-class random masking, it indicates that the methylation-related codes of the first word segmentation code sequence need to be randomly masked according to the second-class random masking ratio. For example, the total number of methylation-related codes of the first word segmentation code sequence is 100, and the second-class random masking ratio is set to 98%. That means that any 100×98%=98 methylation-related codes of these 100 methylation-related codes need to be masked. Specifically, 98 methylation-related codes are randomly selected from these 100 methylation-related codes, and the coding values ​​of these 98 methylation-related codes are reset to word segmentation mask codes.

[0166] 2) The vocabulary storage module is used to store the first vocabulary.

[0167] 3) The MosaicBERT model is used to perform feature encoding processing based on the first embedded coding vector E and output the corresponding first coding vector Y.

[0168] It should be noted that the model structure and component functions of the MosaicBERT model have been disclosed in the paper "MosaicBERT: A Bidirectional Encoder Optimized for Fast Pretraining", and the MosaicBERT model can be refined according to the disclosed content of the paper. The reason why the embodiment of the present invention chooses the MosaicBERT model as the core component of the methylation feature encoding model is that MosaicBERT has the following advantages over the conventional BERT model: 1) The ALiBi position encoding mechanism is used for position encoding, specifically the linear bias attention (Attention with Linear Biases) encoding method replaces the traditional position embedding encoding. This encoding mechanism supports variable length sequence input and improves the modeling ability of long sequences; 2) FlashAttention is used to optimize the attention calculation process and improve training efficiency; 3) The GeGLU activation function is used to replace the conventional GLU activation function, which enhances the nonlinear representation ability of the model.

[0169] The sequence decoding model of the embodiment of the present invention is used to reconstruct the methylation sequence according to the first encoding vector Y input by the model and output the corresponding first decoding sequence Z; wherein the first decoding sequence Z is composed of L second markers z i Sort by order; each second marker z i is a type of base marker in the five-base set [A, T, C, G, M]; the second marker z i With the first marker x i One to one correspondence.

[0170] like Figure 2 As shown, the sequence decoding model of the embodiment of the present invention consists of a first mapping layer, a first linear layer and a first Softmax layer; the first mapping layer is implemented based on a fully connected network; and the first linear layer is based on another fully connected network.

[0171] The connection relationship between the components of the sequence decoding model is as follows: the input end of the first mapping layer is connected to the model input end of the sequence decoding model, and the output end is connected to the input end of the first linear layer; the output end of the first linear layer is connected to the input end of the first Softmax layer; and the output end of the first Softmax layer is connected to the model output end of the sequence decoding model.

[0172] The functions of the components of the sequence decoding model are as follows:

[0173] 1) The first mapping layer is used to transform the first word segmentation feature vector y from the 2nd to the Wth in the first encoding vector Y 2≤j≤WExtract the first extracted vector A to form a first extracted vector A of length W-1; transform the first extracted vector A into a first mapped vector M of length L through linear transformation and send it to the first linear layer;

[0174] The first extraction vector A is composed of W-1 first extraction sub-vectors a k Composition, 1≤ vector index k≤(W-1), each first extracted sub-vector a k The first word segmentation feature vector y j＝k+1 Matching; the first mapping vector M is composed of L first sub-mapping vectors m i composition;

[0175] 2) The first linear layer is used to perform a full-connection calculation based on the first mapping vector M to obtain a corresponding first score vector B and send it to the first Softmax layer;

[0176] Among them, the first rating vector B consists of L first rating values ​​b i Composition, each first score value b i is a real number;

[0177] 3) The first Softmax layer is used to perform five-category probability prediction based on the first score vector B to obtain the corresponding first prediction vector U; and each first subvector u in the first prediction vector U is i The base marker corresponding to the largest first predicted probability is used as the corresponding second marker z i ; and all the second markers z are obtained i Sequentially sort to form a corresponding first decoding sequence Z;

[0178] Among them, the first prediction vector U is composed of L first sub-vectors u i Composition, each first subvector u i It consists of five first prediction probabilities, each of which corresponds to a type of base marker in the five-member base set [A, T, C, G, M].

[0179] The target source prediction model of an embodiment of the present invention is used to perform binary classification prediction based on the first coding vector Y input by the model and output the corresponding first prediction result R; wherein the first prediction result R is a binary prediction result, consisting of two prediction results: target source type and non-target source type.

[0180] like Figure 2 As shown, the target source prediction model of the embodiment of the present invention is composed of a first extraction module, a second linear layer and a second Softmax layer; the second linear layer is based on another fully connected network.

[0181] The connection relationship between the components of the target source prediction model is as follows: the input end of the first extraction module is connected to the model input end of the target source prediction model, and the output end is connected to the input end of the second linear layer; the output end of the second linear layer is connected to the input end of the second Softmax layer; the output end of the second Softmax layer is connected to the model output end of the target source prediction model.

[0182] The functions of the components of the target source prediction model are as follows:

[0183] 1) The first extraction module is used to extract the first first word segmentation feature vector y in the first encoding vector Y j＝1 The extracted vector is sent to the second linear layer as the corresponding second extracted vector C;

[0184] 2) The second linear layer is used to perform a full-connection calculation based on the second extracted vector C to obtain a corresponding second score vector D and send it to the second Softmax layer;

[0185] Among them, the second score vector D is composed of two second scores d1 and d2;

[0186] 3) The second Softmax layer is used to perform binary classification probability prediction based on the second scores d1 and d2 of the second score vector D to obtain the corresponding second prediction vector V; and identify the two second prediction probabilities v1 and v2 of the second prediction vector V; if the second prediction probability v1 is too large, the corresponding first prediction result R is set to the target source type; if the second prediction probability v2 is too large, the corresponding first prediction result R is set to the non-target source type;

[0187] The second prediction vector V is composed of two second prediction probabilities v1 and v2, where the second prediction probability v1 is a true value probability and the second prediction probability v2 is a false value probability.

[0188] Step 3, forming a pre-training model framework by the methylation feature encoding model and the sequence decoding model; and performing first framework training on the pre-training model framework according to the second base sequence to obtain corresponding encoding model pre-training parameters;

[0189] Specifically comprising: step 31, forming a pre-training model framework by a methylation feature encoding model and a sequence decoding model;

[0190] Here, the model connection relationship of the pre-trained model framework is as follows Figure 2 As shown;

[0191] Step 32, performing first framework training on the pre-trained model framework according to the second base sequence to obtain corresponding encoding model pre-training parameters;

[0192] Specifically, the method includes: step 3201, using the pre-trained model framework as the corresponding current model framework; using the methylation feature encoding model and sequence decoding model of the current model framework as the corresponding current encoding model and current decoding model; setting two positive integers as the corresponding first and second stage training times thresholds; setting two ratio parameters with values ​​between 0 and 1 as the corresponding first and second ratio parameters; setting the first and second type random mask ratios built into the sequence word segmenter of the current encoding model as the corresponding first and second ratio parameters; and setting two counters initialized to 0 as the corresponding first and second counters;

[0193] Here, in the embodiment of the present invention, the first proportion parameter is less than the second proportion parameter, 10% is less than the first proportion parameter and less than 20%, and 90% is less than the second proportion parameter and less than 100%;

[0194] Step 3202: The second base sequence is used as the corresponding first sequence X; label vector conversion is performed on the current first sequence X to obtain a corresponding label vector PG; and smooth label vector conversion is performed on the label vector PG to obtain a corresponding smoothed label vector PS.

[0195] Here, the label vector PG of the embodiment of the present invention is composed of L sub-vectors pg i Composition; subvector pg i With the first marker x of the first sequence X i One-to-one correspondence; each subvector pg i Each is a one-hot encoding vector of length 5, consisting of 5 bases one-hot encoding pgA i 、pgT i 、pgC i 、pgG i 、pgM i Composition; each subvector pg i The five base unique hot codes correspond one-to-one to the five base markers in the five-base set [A, T, C, G, M]; each subvector pg i Among the five base unique hot codes, only one is 1 and the other four are 0. The base marker corresponding to the base unique hot code of 1 is the same as the current sub-vector pg i The corresponding first marker x i match;

[0196] The smoothed label vector PS of the embodiment of the present invention is composed of L sub-vectors ps i Composition; sub-vector ps i and the subvector pg of the label vector PG i One-to-one correspondence; each sub-vector ps i The length of the vector is 5, and psA is encoded by 5 vectors i 、psT i、psC i , psG i 、psM i Composition; each sub-vector ps i The five vector codes of correspond one-to-one to the five types of base markers in the five-element base set [A, T, C, G, M];

[0197] The subvector pg of the embodiment of the present invention i With the subvector ps i The conversion relationship is:

[0198]

[0199] Among them, α is the preset smoothing parameter, which takes a value between 0 and 1; I is a preset all-1 vector with a length of 5; Condition 1 is: the subvector pg i The corresponding first marker x i is the base marker C or M; condition 2 is: subvector pg i The corresponding first marker x i is the base marker A, T or G;

[0200] Step 3203: Set the mask rule type built into the sequence word segmenter of the current encoding model to a type of random mask;

[0201] Step 3204: Input the current first sequence X into the sequence word segmenter of the current encoding model for processing to obtain the corresponding first embedded code vector E;

[0202] Step 3205: Input the first embedded coding vector E into the MosaicBERT model of the current coding model for processing to obtain the corresponding first coding vector Y; input the first coding vector Y into the current decoding model for processing to obtain the corresponding first decoding sequence Z; and input the first subvectors u of the first prediction vector U generated in the current decoding model processing process to obtain the first decoding sequence Z. i As a corresponding subvector pr i , and all the sub-vectors pr are obtained i Form the corresponding prediction vector PR;

[0203] Here, the prediction vector PR of the embodiment of the present invention is composed of L sub-vectors pr i Composition; sub-vector pr i The second marker z of the first decoding sequence Z i One-to-one correspondence; subvector pr i The five first predicted probabilities are denoted as prA i prT i 、prC i , prG i 、prM i, which correspond one-to-one to the five types of base markers in the five-member base set [A, T, C, G, M];

[0204] Step 3206: Substitute the prediction vector PR and the smoothed label vector PS into the preset first model loss function L M1 Calculate and obtain the corresponding first loss value;

[0205] Among them, the first model loss function L M1 for:

[0206]

[0207] Step 3207, identify whether the first loss value meets the preset first loss value range; if so, add 1 to the first counter and go to step 3208; if not, based on the preset first model optimizer, move towards making the first model loss function L M1 Perform a round of modulation on the full model parameters of the current encoding model and the full model parameters of the current decoding model in the direction where the minimum value is reached, and return to step 3205 when this round of modulation is completed;

[0208] Here, the first loss value range of the embodiment of the present invention is a preset loss value range; the first model optimizer includes at least an Adam optimizer and an SGD optimizer;

[0209] Step 3208: Identify whether the first counter exceeds the threshold of the number of training times in one stage; if not, return to step 3204 to continue training; if exceeded, go to step 3209;

[0210] Step 3209: Set the masking rule type built into the sequence word segmenter of the current encoding model to the second-class random mask;

[0211] Step 3210: Input the current first sequence X into the sequence word segmenter of the current encoding model for processing to obtain the corresponding first embedded code vector E;

[0212] Step 3211: Input the first embedded coding vector E into the MosaicBERT model of the current coding model for processing to obtain the corresponding first coding vector Y; input the first coding vector Y into the current decoding model for processing to obtain the corresponding first decoding sequence Z; and input the first subvectors u of the first prediction vector U generated in the current decoding model processing process to obtain the first decoding sequence Z. i As a corresponding subvector pr i , and all the sub-vectors pr are obtained i Form the corresponding prediction vector PR;

[0213] Step 3212: Substitute the prediction vector PR and the smoothed label vector PS into the first model loss function LM1 Calculate and obtain the corresponding second loss value;

[0214] Step 3213, identify whether the second loss value meets the preset second loss value range; if so, add 1 to the second counter and go to step 3214; if not, based on the first model optimizer, move towards making the first model loss function L M1 Perform a round of modulation on the full model parameters of the current encoding model and the full model parameters of the current decoding model in the direction where the minimum value is reached, and return to step 3211 when this round of modulation is completed;

[0215] Here, the second loss value range of the embodiment of the present invention is a preset loss value range;

[0216] Step 3214: Identify whether the second counter exceeds the second-stage training times threshold; if not, return to step 3210 to continue training; if exceeded, go to step 3215;

[0217] Step 3215, confirming that the first framework training is completed, and saving the current model parameters of the current coding model as the corresponding coding model pre-training parameters; and solidifying the framework model parameters of the pre-training model framework.

[0218] Step 4: The first task model framework is composed of the methylation feature encoding model and the target source prediction model; a model dataset is constructed based on the BS-seq read set and the target source type; and a second framework training is performed on the first task model framework based on the encoding model pre-training parameters and the model dataset;

[0219] Specifically, it includes: step 41, forming a first task model framework by a methylation feature encoding model and a target source prediction model;

[0220] Here, the model connection relationship of the first task model framework is as follows Figure 2 As shown;

[0221] Step 42, constructing a model dataset based on the BS-seq read set and the target source type;

[0222] The model data set includes a plurality of first data records; the first data record includes a first training sequence and a first label classification result; the first label classification result includes a target source type and a non-target source type;

[0223] Specifically comprising: step 421, forming a corresponding positive sample set from BS-seq read records whose read sample sources match the target source type in the BS-seq read set, and forming a corresponding negative sample set from BS-seq read records whose read sample sources do not match the target source type in the BS-seq read set;

[0224] Step 422, the BS-seq reads of each BS-seq read record of the positive sample set are used as a corresponding first training sequence, and a first label classification result specifically set to the target source type is set; and the first training sequence and the first label classification result corresponding to each BS-seq read record of the positive sample set are used to form a corresponding first data record;

[0225] In step 423, the BS-seq reads of each BS-seq read record in the negative sample set are used as a corresponding first training sequence, and a first label classification result specifically set to a non-target source type is set; and a corresponding first data record is formed by the first training sequence and the first label classification result corresponding to each BS-seq read record in the negative sample set;

[0226] Step 424, forming a corresponding model data set from all the obtained first data records;

[0227] Step 43, performing a second framework training on the first task model framework based on the encoding model pre-training parameters and the model dataset;

[0228] Specifically, the method includes: step 4301, using the first task model framework as the corresponding current model framework; using the methylation feature encoding model and the target source prediction model of the current model framework as the corresponding current encoding model and current prediction model; and setting the built-in masking rule type of the sequence segmenter of the current encoding model to no masking;

[0229] Step 4302: solidify the model parameters of the methylation feature encoding model into the corresponding encoding model pre-trained parameters; and based on the LoRA fine-tuning mechanism, load a corresponding LoRA adapter on one or more specified modules of the methylation feature encoding model; and form a corresponding adapter parameter set from the adapter parameters of all LoRA adapters;

[0230] Here, the fixed model parameters of each designated module of the methylation feature coding model of the embodiment of the present invention are recorded as the pre-fine-tuning parameter W0, and the adapter parameters of the LoRA adapter corresponding to each designated module are composed of the matrix parameters of two low-rank matrices A and B; each LoRA adapter is used to fine-tune the pre-fine-tuning parameter W0 of the corresponding designated module to obtain the corresponding post-fine-tuning parameter W1; the conversion relationship between the post-fine-tuning parameter W1 and the pre-fine-tuning parameter W0 is: W1× in =W0X in +BAX in , X in is the input vector of the specified module, W0X in For the current specified module, when no corresponding LoRA adapter is added, it is based on the input vector Xin The output vector obtained by processing, W1X in For the current specified module, after adding the corresponding LoRA adapter based on the input vector X in The output vector obtained by processing;

[0231] It should be noted that, in general, some or all of the self-attention modules in the methylation feature encoding model are used as designated modules;

[0232] Step 4303: Divide the model dataset into two sub-datasets according to a preset first segmentation ratio and record them as a corresponding first training set and a first evaluation set;

[0233] Here, the first split ratio is a preset ratio parameter, such as 8:2; the first training set and the first evaluation set both consist of multiple first data records; the ratio of the total number of records in the first training set to the total number of records in the first evaluation set satisfies the first split ratio; the ratio of positive to negative samples in the first training set and the first evaluation set are the same;

[0234] Step 4304: taking the first first data record of the first training set as the corresponding current training record;

[0235] Step 4305: Set a corresponding label vector TG based on the first label classification result of the current training record;

[0236] Here, the label vector TG of the embodiment of the present invention is composed of two one-hot codes tg1 and tg2; the one-hot codes tg1 and tg2 correspond to the target source type and the non-target source type, respectively; if the first label classification result of the current training record is the target source type, the one-hot code tg1 is 1 and the one-hot code tg2 is 0; if the first label classification result of the current training record is the non-target source type, the one-hot code tg1 is 0 and the one-hot code tg2 is 1;

[0237] Step 4306: Input the first training sequence of the current training record as the corresponding first sequence X into the current model framework for processing to obtain the corresponding first prediction result R; and set a corresponding prediction vector TR based on the second prediction vector V generated by the current prediction model during this processing;

[0238] Here, the prediction vector TR of the embodiment of the present invention is composed of two vector codes tr1 and tr2; the vector code tr1 is the second prediction probability v1 of the second prediction vector V, and the vector code tr2 is the second prediction probability v2 of the second prediction vector V;

[0239] Step 4307: Substitute the prediction vector TR and the label vector TG into the preset second model loss function L M2 Calculate and obtain the corresponding third loss value;

[0240] Here, the second model loss function L in the embodiment of the present invention is M2 Implemented based on L1 loss function, L2 loss function or cross entropy loss function;

[0241] Step 4308, whether the third loss value meets the preset third loss value range is identified; if the third loss value meets the third loss value range, whether the current training record is the last first data record of the first training set is identified, if so, go to step 4309, if not, the next first data record of the first training set is used as the new current training record and return to step 4305 to continue training; if the third loss value does not meet the third loss value range, then based on the preset second model optimizer, move towards making the second model loss function L M2 Perform a round of modulation on the adapter parameter set and the full model parameters of the current prediction model in the direction of reaching the minimum value, and return to step 4306 when this round of modulation is completed;

[0242] Here, the third loss value range of the embodiment of the present invention is a preset loss value range; the second model optimizer includes at least an Adam optimizer and an SGD optimizer;

[0243] Step 4309: Perform a round of traversal on all first data records of the first evaluation set; and during this round of traversal, use the currently traversed first data record as the corresponding current evaluation record; and set a corresponding label vector TG based on the first label classification result of the current evaluation record; and input the first training sequence of the current evaluation record as the corresponding first sequence X into the current model framework for processing to obtain the corresponding first prediction result R, and set a corresponding prediction vector TR based on the second prediction vector V generated by the current prediction model during this processing; and form a corresponding prediction-label pair by the label vector TG corresponding to the current evaluation record and the prediction vector TR; and at the end of this round of traversal, bring all the obtained prediction-label pairs into the preset first model evaluation function to calculate and obtain the corresponding first evaluation value; and calculate the accuracy, precision, recall rate, and F1 score based on all the obtained prediction-label pairs to obtain the corresponding first accuracy, first precision, first recall rate, and first F1 score;

[0244] Here, the first model evaluation function of the embodiment of the present invention is implemented based on the MSE function or the RMSE function;

[0245] Step 4310: Identify whether the first evaluation value, the first accuracy, the first precision, the first recall, and the first F1 score all meet the corresponding first evaluation value range, the first accuracy range, the first precision range, the first recall range, and the first F1 score range. If so, proceed to step 4311. If at least one of the first evaluation value, the first accuracy, the first precision, the first recall, and the first F1 score does not meet the corresponding range, return to step 4303 to continue training.

[0246] Here, the first evaluation value range, the first accuracy range, the first precision range, the first recall range, and the first F1 score range of the embodiment of the present invention are five preset value ranges;

[0247] Step 4311, confirm that the second framework training is completed, and save the encoding model pre-training parameters and the latest adapter parameter set as the corresponding fine-tuning encoding model parameter set; and solidify the framework model parameters of the first task model framework.

[0248] Step 5: After the first and second frameworks are trained, the methylation sequence reconstruction task of the DNA sequence is processed based on the pre-trained model framework; and a DNA sequence source identification task corresponding to the target source type is processed based on the first task model framework;

[0249] Specifically comprising: step 51, processing the methylation sequence reconstruction task of the DNA sequence based on the pre-trained model framework;

[0250] Specifically, the method includes: receiving a DNA sequence input by a user as a corresponding first DNA sequence; inputting the first DNA sequence as a corresponding first sequence X into a pre-trained model framework for processing to obtain a corresponding first decoding sequence Z; and feeding back the current first decoding sequence Z as a corresponding first reconstructed sequence to the current user;

[0251] The first DNA sequence is a marker sequence formed by sorting base markers in the four-base set [A, T, C, G]; the first reconstructed sequence is a marker sequence formed by sorting base markers in the five-base set [A, T, C, G, M];

[0252] Here, the user can identify the methylation sites on the current DNA sequence through the base marker M on the first reconstructed sequence;

[0253] Step 52, processing a type of DNA sequence source identification task corresponding to the target source type based on the first task model framework;

[0254] Specifically, the method includes: receiving a DNA sequence input by a user as a corresponding second DNA sequence; inputting the second DNA sequence as the corresponding first sequence X into a first task model framework for processing to obtain a corresponding first prediction result R; and feeding back the current first prediction result R as a corresponding first recognition result to the current user;

[0255] The second DNA sequence is a marker sequence formed by sorting base markers in the four-base set [A, T, C, G] or the five-base set [A, T, C, G, M].

[0256] From the above content, it can be seen that the embodiments of the present invention provide a base set upgrade scheme that regards 5-mC in the DNA sequence as a type of extended base marker M and thereby upgrades the conventional four-base set [A, T, C, G] to a five-base set [A, T, C, G, M]; provides a data acquisition scheme using BS-seq reads as training data; provides a vocabulary learning scheme that learns a class of special vocabulary with a large amount of methylation sub-word information based on BS-seq reads, the five-base set [A, T, C, G, M] and the BPE algorithm; provides an encoding model design scheme that uses the learned special vocabulary as the segmentation vocabulary and a methylation feature encoding model with the MosaicBERT model as the core, and also provides a processing scheme for pre-training and fine-tuning the encoding model based on two downstream tasks (referred to as the pre-training and fine-tuning scheme of the encoding model). It should be noted that the implementation principles of the above-mentioned series of technical solutions (base set upgrade solution, data acquisition solution, vocabulary learning solution, coding model design solution, coding model pre-training and fine-tuning solution) in the embodiments of the present invention are also applicable to other modification forms in the field of epigenetic modification, such as 6-methyladenine (6-mA) in RNA sequences, 5-hydroxymethylcytosine (5-hmC) in DNA sequences, and the like. Specifically, after determining a type of other modification form, for example, assuming that the current modification form is 5-hmC, a type of extended base marker H can be assigned to 5-hmC based on the implementation principle of the base set upgrade solution of the embodiment of the present invention, and a corresponding five-base set [A, T, C, G, H] or six-base set [A, T, C, G, M, H] can be expanded on the basis of the conventional four-base set [A, T, C, G] or the five-base set [A, T, C, G, M] in this case; and a sequencing technology type (such as oxBS-seq technology) that can detect 5-hmC is selected for the five-base set [A, T, C, G, H] or a six-base set [A, T, C, G, M, H] is selected. It can simultaneously detect 5-mC and 5-hmC sequencing technology types (such as EM-seq technology); and based on the implementation principle of the data collection scheme of the embodiment of the present invention, the sequencing reads corresponding to the current sequencing technology are used as training data for big data collection; and based on the implementation principle of the vocabulary learning scheme of the embodiment of the present invention, a special vocabulary is learned according to the currently expanded five-member base set [A, T, C, G, H] or six-member base set [A, T, C, G, M, H]; and based on the implementation principle of the encoding model design scheme of the embodiment of the present invention, a corresponding feature encoding model is designed for the 5-hmC modification form; and based on the implementation principle of the pre-training and fine-tuning scheme of the encoding model of the embodiment of the present invention, the current feature encoding model is pre-trained and fine-tuned.

[0257] Figure 3 This is a module structure diagram of a processing device for a methylation feature coding model provided in the second embodiment of the present invention. The device is a terminal device or server that implements the aforementioned method embodiment, or can be a device that enables the aforementioned terminal device or server to implement the aforementioned method embodiment. For example, the device can be a device or chip system of the aforementioned terminal device or server. Figure 3 As shown, the device includes: a preprocessing module 201, a model building module 202, a first training module 203, a second training module 204 and a model application module 205.

[0258] The preprocessing module 201 is used to set a corresponding extended marker for 5-methylcytosine in the DNA sequence as the base marker M, and adjust the four-base set of DNA [A, T, C, G] to the corresponding five-base set [A, T, C, G, M]; and perform big data collection on the bisulfite sequencing reads of the first species to form a corresponding BS-seq read set; and perform read segment splicing based on the BS-seq read set to obtain a corresponding first base sequence; and perform five-element sequence conversion on the first base sequence based on the five-base set [A, T, C, G, M] and the methylation status mark corresponding to each base marker C on the first base sequence to obtain a corresponding second base sequence; and perform vocabulary learning based on the BPE algorithm and the second base sequence to obtain a corresponding first vocabulary; the first species is defaulted to human, and in addition to human, it can also be one of multiple other species, such as monkeys, orangutans, etc.

[0259] The model construction module 202 is used to construct a methylation feature encoding model with the MosaicBERT model as the core; and store the first vocabulary in the vocabulary storage module of the methylation feature encoding model; and construct a sequence decoding model for methylation sequence reconstruction; and based on a class of target source types, construct a corresponding two-class prediction model recorded as a target source prediction model; the target source type is specifically a class of tissue type or a class of cell type.

[0260] The first training module 203 is used to form a pre-training model framework by a methylation feature encoding model and a sequence decoding model; and perform first framework training on the pre-training model framework according to the second base sequence to obtain corresponding encoding model pre-training parameters.

[0261] The second training module 204 is used to form a first task model framework consisting of a methylation feature encoding model and a target source prediction model; and to construct a model data set based on the BS-seq read segment set and the target source type; and to perform a second framework training on the first task model framework based on the encoding model pre-training parameters and the model data set.

[0262] The model application module 205 is used to process the methylation sequence reconstruction task of the DNA sequence based on the pre-trained model framework after the first and second framework training is completed; and to process a type of DNA sequence source identification task corresponding to the target source type based on the first task model framework.

[0263] An embodiment of the present invention provides a processing device for a methylation feature coding model, which can execute the method steps in the above method embodiment. Its implementation principles and technical effects are similar and will not be repeated here.

[0264] It should be noted that it should be understood that the division of the various modules of the above device is merely a division of logical functions. In actual implementation, they can be fully or partially integrated into a physical entity, or they can be physically separated. Moreover, these modules can all be implemented in the form of software called by a processing element; or they can all be implemented in the form of hardware; or some modules can be implemented in the form of software called by a processing element, and some modules can be implemented in the form of hardware. For example, the preprocessing module can be a separately established processing element, or it can be integrated into a chip of the above device. In addition, it can also be stored in the memory of the above device in the form of program code, and called by a processing element of the above device to perform the functions of the above-mentioned module. The implementation of other modules is similar. In addition, these modules can all or partly be integrated together, or they can be implemented independently. The processing element described here can be an integrated circuit with signal processing capabilities. During implementation, each step of the above method or each of the above modules can be completed by hardware integrated logic circuits in the processor element or instructions in the form of software.

[0265] For example, the above modules may be one or more integrated circuits configured to implement the above methods, such as one or more application-specific integrated circuits (ASICs), one or more digital signal processors (DSPs), or one or more field programmable gate arrays (FPGAs). For another example, when a module is implemented by scheduling program code through a processing element, the processing element may be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call program code. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0266] In the above embodiments, all or part of the embodiments may be implemented using software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments may be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the above method embodiments are generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The above-mentioned computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the above-mentioned computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, Bluetooth, microwave, etc.) means. The above-mentioned computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media. The above-mentioned available medium can be a magnetic medium (such as a floppy disk, hard disk, tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (SSD)), etc.

[0267] Figure 4 This is a schematic diagram of the structure of an electronic device provided in the third embodiment of the present invention. The electronic device can be a terminal device or server that implements the method of the aforementioned embodiment, or it can be a terminal device or server that implements the method of the aforementioned embodiment connected to the aforementioned terminal device or server. Figure 4 As shown, the electronic device may include: a processor 301 (such as a CPU), a memory 302, and a transceiver 303; the transceiver 303 is coupled to the processor 301, and the processor 301 controls the transceiver 303's transceiver actions. Various instructions may be stored in the memory 302 for completing various processing functions and implementing the processing steps described in the aforementioned embodiment method. Preferably, the electronic device involved in the embodiment of the present invention further includes: a power supply 304, a system bus 305, and a communication port 306. The system bus 305 is used to realize communication connections between components. The above-mentioned communication port 306 is used for connecting and communicating between the electronic device and other peripherals.

[0268] exist Figure 4The system bus 305 mentioned in the figure can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The system bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. The communication interface is used to realize communication between the database access device and other devices (such as clients, read-write libraries, and read-only libraries). The memory may include random access memory (RAM) and may also include non-volatile memory (Non-Volatile Memory), such as at least one disk storage.

[0269] The above-mentioned processors can be general-purpose processors, including central processing units (CPUs), network processors (NPs), graphics processing units (GPUs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0270] It should be noted that an embodiment of the present invention further provides a computer-readable storage medium, which stores instructions. When the computer-readable storage medium is run on a computer, it enables the computer to execute the methods and processing procedures provided in the above embodiments.

[0271] The embodiment of the present invention provides a processing method, device, electronic device and computer-readable storage medium for a methylation feature coding model. As can be seen from the above content, the embodiment of the present invention sets a corresponding base marker M for 5-mC in the DNA sequence, and adjusts the four-base set [A, T, C, G] of DNA to a five-base set [A, T, C, G, M]; and performs big data collection on the bisulfite sequencing reads of a certain species, and performs read segment splicing and methylation status marking based on the collected BS-seq read segment set to obtain a base sequence, and performs vocabulary learning based on the BPE algorithm and this base sequence; and then constructs an encoder model with the MosaicBERT model as the core, recorded as the methylation feature coding model, and stores the learned vocabulary in the encoding model. The vocabulary storage module inside the model is constructed, and a decoder model for methylation sequence reconstruction is recorded as a sequence decoding model, and a corresponding two-class prediction model based on a class of target source types is constructed as a target source prediction model; then the methylation feature encoding model and the sequence decoding model form a pre-training model framework, and the methylation feature encoding model and the target source prediction model form a first task model framework; then, the pre-training model framework is trained based on the base sequence above, and then the first task model framework is trained; finally, the methylation sequence reconstruction task of the DNA sequence is processed based on the pre-training model framework, and the class of DNA sequence source identification task is processed based on the first task model framework. The methylation feature encoding model provided in the embodiment of the present invention can learn the methylation features of DNA sequences. The pre-trained model framework with this methylation feature encoding model as the core encoder can be used to process methylation sequence analysis tasks. The first task model framework with this methylation feature encoding model as the core encoder can be used to process a type of target source identification task. The methylation feature encoding model provided in the embodiment of the present invention and the corresponding two task model frameworks are used to process methylation sequence analysis tasks and target source identification tasks, which not only simplifies the task processing steps, shortens the task processing cycle, and improves the real-time performance of task processing, but also effectively reduces the task processing cost.

[0272] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be implemented using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0273] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for processing a methylation feature coding model, characterized in that: The method comprises: A corresponding extended marker is set for 5-methylcytosine in the DNA sequence as a base marker M, and the four-membered base set [A, T, C, G] of the DNA is adjusted to a corresponding five-membered base set [A, T, C, G, M]; big data is collected for bisulfite sequencing reads of the first species to form a corresponding BS-seq read set; reads are spliced ​​according to the BS-seq read set to obtain a corresponding first base sequence; and based on the five-membered base set [A, T, C, G, M] and the methylation state marker corresponding to each base marker C on the first base sequence, the first base sequence is converted into a five-element sequence to obtain a corresponding second base sequence; and vocabulary learning is performed based on the BPE algorithm and the second base sequence to obtain a corresponding first vocabulary; A methylation feature encoding model is constructed with the MosaicBERT model as the core; the first vocabulary is stored in the vocabulary storage module of the methylation feature encoding model; a sequence decoding model for methylation sequence reconstruction is constructed; and a corresponding binary classification prediction model is constructed based on a target source type, which is specifically a tissue type or a cell type. The methylation feature encoding model and the sequence decoding model form a pre-training model framework; and the pre-training model framework is trained with a first framework according to the second base sequence to obtain corresponding encoding model pre-training parameters; A first task model framework is formed by the methylation feature encoding model and the target source prediction model; a model dataset is constructed based on the BS-seq read segment set and the target source type; and a second framework training is performed on the first task model framework based on the encoding model pre-training parameters and the model dataset; After the training of the first and second frameworks is completed, the methylation sequence reconstruction task of the DNA sequence is processed based on the pre-trained model framework; and a type of DNA sequence source identification task corresponding to the target source type is processed based on the first task model framework.

2. The method for processing the methylation feature coding model according to claim 1, wherein: The base markers A, T, C, and G are markers for adenine, thymine, cytosine, and guanine, respectively; The BS-seq read set is composed of multiple BS-seq read records; each of the BS-seq read records includes at least one BS-seq read and a corresponding read sample source; each of the BS-seq reads is a DNA sequence obtained by DNA sequencing of a DNA sample of the first species after bisulfite sequencing, and is sorted by base markers A, T, C, and G; each base marker C on each BS-seq read carries a corresponding methylation status marker; the methylation status marker is a binary status marker, including two states: unmethylated and methylated; the read sample source is a source type code corresponding to a current BS-seq read, and each source type code corresponds to a specific source type; the type range of the source type consists of multiple tissue types and / or multiple cell types of the first species; The first base sequence is a marker sequence formed by sorting the base markers A, T, C, and G; the second base sequence is a marker sequence formed by sorting the base markers A, T, C, G, and M; The first vocabulary table is composed of a plurality of first subword records; the first subword records include a first subword code, a first subword text, and a first subword frequency; the first subword text is a string formed by sorting one or more base markers in the five-element base set [A, T, C, G, M]; the first subword frequency is an integer value; all first subword texts in the first vocabulary table are different; The model data set includes multiple first data records; the first data records include a first training sequence and a first label classification result; the first label classification result includes the target source type and the non-target source type.

3. The method for processing the methylation feature coding model according to claim 2, wherein: The step of collecting big data of the bisulfite sequencing reads of the first species to form a corresponding BS-seq read set specifically includes: Acquiring bisulfite sequencing reads of the DNA sequence of the first species through a plurality of preset data acquisition channels to obtain a plurality of BS-seq reads; the plurality of preset data acquisition channels at least comprising one or more public BS-seq read datasets and public technical literature containing BS-seq read information of the first species; and setting a type range of the corresponding source type based on the sample source information corresponding to all the collected BS-seq reads; and setting a corresponding source type code for each source type in the type range of the source type; and using the source type code corresponding to each BS-seq read as the corresponding read sample source; and forming a corresponding BS-seq read record from each BS-seq read and the corresponding read sample source; and performing erroneous read deletion and duplicate read deduplication processing on all the obtained BS-seq read records; The BS-seq read set is composed of all the remaining BS-seq read records.

4. The method for processing the methylation feature coding model according to claim 2, wherein: The step of performing read segment splicing according to the BS-seq read segment set to obtain a corresponding first base sequence specifically includes: All the BS-seq reads in the BS-seq read set are sequentially spliced ​​to obtain the corresponding first base sequence.

5. The method for processing the methylation feature coding model according to claim 2, wherein: The step of performing five-element sequence conversion on the first base sequence based on the five-element base set [A, T, C, G, M] and the methylation status mark corresponding to each base marker C on the first base sequence to obtain the corresponding second base sequence specifically includes: Perform sequence replication on the first base sequence to obtain a corresponding first replication sequence; and modify the status value of each current methylation status mark on the first replication sequence from the methylated base marker C to the corresponding base marker M; and use the first replication sequence after the marker modification as the corresponding second base sequence.

6. The method for processing the methylation feature coding model according to claim 2, wherein: The performing vocabulary learning based on the BPE algorithm and the second base sequence to obtain the corresponding first vocabulary specifically includes: Step 61: Set an empty vocabulary as the corresponding first vocabulary; initialize the first vocabulary; set a threshold for the total number of records in the first vocabulary; and use the second base sequence as a corresponding first character string; The initialized first vocabulary consists of six initial first subword records; the first subword texts of the six initial first subword records are character 'A', character 'T', character 'C', character 'G', character 'M' and character string 'CG' in sequence; the first subword word frequencies of the six initial first subword records are all initialized to 1; Step 62: taking each of the first subword texts in the first vocabulary as a corresponding basic subword, and forming a corresponding current subword set from all the obtained basic subwords; and performing subword sequence splitting on the first character string according to the current subword set according to the subword sequence splitting method of the BPE algorithm to obtain a corresponding current subword sequence; The current subword sequence is formed by sequentially sorting a plurality of the basic subwords; the basic subword pair is formed by sequentially sorting the first and last two basic subwords; Step 63, according to the subword pair combination method of the BPE algorithm, each two adjacent basic subwords in the current subword sequence form a corresponding basic subword pair; and cluster the same basic subword pairs to obtain multiple cluster word pair sets; and use the total number of the basic subword pairs in each cluster word pair set as the corresponding first word pair number, and use the largest first word pair number as the corresponding high-frequency word pair number; and identify the high-frequency word pair number; if the high-frequency word pair number is 1, go to step 68; if the high-frequency word pair number is greater than 1, use the total number of the cluster word pair sets corresponding to the high-frequency word pair number as the corresponding high-frequency word pair number, and identify the high-frequency word pair number. If the high-frequency word pair number is 1, go to step 64; if the high-frequency word pair number is greater than 1, go to step 65; Each of the clustered word pair sets is composed of one or more basic sub-word pairs, and all the basic sub-word pairs in each clustered word pair set are the same; Step 64, taking the basic subword pair corresponding to the only one of the clustered word pair set corresponding to the total number of high-frequency word pairs as the corresponding final-selected subword pair; and then going to step 66; Step 65: record all the clustered word pair sets corresponding to the total number of high-frequency word pairs as corresponding candidate sets; and record the basic subword pairs corresponding to each candidate set as the corresponding candidate subword pair; and record the character 'C', the character 'M', and the character string 'CG' as methylation-related characters; and record each candidate subword pair containing the methylation-related characters as a preferred subword pair; and identify the total number of the preferred subword pairs; if the total number of the preferred subword pairs is 0, select one from all the candidate subword pairs as the corresponding final subword pair; if the total number of the preferred subword pairs is greater than 0, select one from all the preferred subword pairs as the corresponding final subword pair; Step 66: Add a new first subword record to the first vocabulary as the corresponding current newly added record, and set the first subword text of the current newly added record as the corresponding final subword pair; and after the record setting is completed, use each first subword text in the current first vocabulary as a new basic subword, and form the latest current subword set from all the newly obtained basic subwords; and according to the subword sequence splitting method of the BPE algorithm, perform subword sequence splitting on the first character string according to the current subword set to obtain the latest current subword sequence; and count the total number of each basic subword appearing in the current subword sequence and use the statistical result as the corresponding first word frequency, and set the first word frequency corresponding to each basic subword in the current subword set that does not appear in the current subword sequence to 0; and update the first subword frequency corresponding to each first subword text in the current first vocabulary to the corresponding first word frequency; and after the update is completed, delete the first subword record in the first vocabulary whose first subword frequency is 0; Step 67: Count the total number of the first subword records in the current first vocabulary to obtain the corresponding total number of current records; and determine whether the total number of current records is less than the total number of records threshold; if so, return to step 63; if not, go to step 68; Step 68: re-sort all the first sub-word records in the latest first vocabulary in descending order of the first sub-word frequency; and based on the preset sub-word encoding rules, set the first sub-word encoding corresponding to each first sub-word text in the sorted first vocabulary; and output the first vocabulary after the sub-word encoding setting as the vocabulary learning result of this time.

7. The method for processing the methylation feature coding model according to claim 2, wherein: The methylation feature encoding model is used to perform feature encoding on the first sequence X input by the model and output the corresponding first encoding vector Y; the first sequence X consists of multiple first markers x i Sort by order; each of the first markers x i is a type of base marker in the five-element base set [A, T, C, G, M]; 1≤ marker index i≤L, L is the sequence length of the first sequence X; the first encoding vector Y is composed of multiple first word segmentation feature vectors y j Composition; 1≤ word segmentation index j≤W, W is the sequence length of the first word segmentation sequence S corresponding to the first sequence X; The methylation feature coding model is composed of a sequence word segmenter, the word table storage module and the MosaicBERT model; the input end of the sequence word segmenter is connected to the model input end of the methylation feature coding model, and the output end is connected to the input end of the MosaicBERT model; the sequence word segmenter is also connected to the word table storage module; the output end of the MosaicBERT model is connected to the model output end of the methylation feature coding model; The sequence word segmenter is used to split the first sequence X into subwords according to the first vocabulary stored in the vocabulary storage module in a subword sequence splitting manner of the BPE algorithm to obtain a first subword sequence composed of multiple first subword texts; and add a preset classification subword text 'CLS' before the first first subword text in the first subword sequence; and use the sequence length of the added first subword sequence as the corresponding sequence length W; and use each subword text in the first subword sequence as a corresponding first segmentation s j , and consists of all the first participles s j The first segmentation sequence S is formed; and based on the first subword code corresponding to each first subword text in the first vocabulary and the classification subword code corresponding to the classification subword text 'CLS', each first segmentation s in the first segmentation sequence S is j The corresponding code is set to obtain the corresponding first segmentation code, and the corresponding first segmentation code sequence is composed of all the first segmentation codes obtained; and the built-in masking rule type is identified; if the masking rule type is no mask, the masking modification of the first segmentation code sequence is not performed; if the masking rule type is a type of random mask, the first segmentation code in the first segmentation code sequence is randomly masked according to the built-in type of random mask ratio and the preset segmentation mask code, and it is ensured that the total number of the masked first segmentation codes and the ratio of the sequence length of the first segmentation code sequence and the type of random mask ratio match; if the masking rule type is a type of second random mask, the first segmentation s containing methylation-related characters in the first segmentation sequence S are randomly masked. j Recorded as methylation-related participles, and the first participle codes corresponding to each of the methylation-related participles in the first participle code sequence are recorded as methylation-related codes, and the methylation-related codes in the first participle code sequence are randomly masked according to the built-in two-category random mask ratio and the participle mask code, and ensure that the ratio of the total number of masked methylation-related codes to the total number of methylation-related codes in the first participle code sequence and the two-category random mask ratio match; and based on the model input vector embedding coding rule of the MosaicBERT model, the first participle code sequence is embedded and coded to obtain the corresponding first embedded coding vector E and send it to the MosaicBERT model; the first participle sequence S consists of W first participles s j Composition, the first first participle s j＝1 For the classification subword text 'CLS', the second to last first participle s 2≤j≤W Each of the first participles s in j Corresponding to one of the first subword texts in the first vocabulary; the first embedding encoding vector E is composed of W first word embedding encoding vectors e j The first word segmentation embedding encoding vector e j With the first participle s j One-to-one correspondence; The MosaicBERT model is used to perform feature encoding processing based on the first embedded coding vector E and output the corresponding first coding vector Y.

8. The method for processing the methylation feature coding model according to claim 7, wherein: The sequence decoding model is used to reconstruct the methylation sequence according to the first encoding vector Y input by the model and output the corresponding first decoding sequence Z; the first decoding sequence Z consists of L second markers z i The second marker z is sorted in sequence; i is a base marker in the five-base set [A, T, C, G, M]; the second marker z i With the first marker x i One-to-one correspondence; The sequence decoding model consists of a first mapping layer, a first linear layer and a first Softmax layer; the first mapping layer is implemented based on a fully connected network; the first linear layer is based on another fully connected network; The input end of the first mapping layer is connected to the model input end of the sequence decoding model, and the output end is connected to the input end of the first linear layer; the output end of the first linear layer is connected to the input end of the first Softmax layer; the output end of the first Softmax layer is connected to the model output end of the sequence decoding model; The first mapping layer is used to convert the first word segmentation feature vectors y from the 2nd to the Wth in the first encoding vector Y 2≤j≤W Extracted to form a first extraction vector A of length W-1; and converted into a first mapping vector M of length L through linear transformation and sent to the first linear layer; the first extraction vector A is composed of W-1 first extraction sub-vectors a k Composition, 1≤ vector index k≤(W-1), each of the first extracted sub-vectors a k The first word segmentation feature vector y corresponding to j＝k+1 Matching; the first mapping vector M is composed of L first sub-mapping vectors m i composition; The first linear layer is used to perform a full connection calculation according to the first mapping vector M to obtain a corresponding first score vector B and send it to the first Softmax layer; The first rating vector B is composed of L first rating values ​​b i Composition, each of the first rating values ​​b i is a real number; The first Softmax layer is used to perform five-category probability prediction based on the first score vector B to obtain the corresponding first prediction vector U; and i The base marker corresponding to the largest first predicted probability is used as the corresponding second marker z i ; and all the second markers z are obtained i Sequentially sorted to form the corresponding first decoding sequence Z; The first prediction vector U is composed of L first sub-vectors u i Each of the first subvectors u i It consists of five first prediction probabilities, each of which corresponds to a type of base marker in the five-element base set [A, T, C, G, M].

9. The method for processing the methylation feature coding model according to claim 7, wherein: The target source prediction model is used to perform binary classification prediction based on the first encoding vector Y input by the model and output a corresponding first prediction result R; the first prediction result R is a binary prediction result, consisting of two prediction results of the target source type and the non-target source type; The target source prediction model consists of a first extraction module, a second linear layer, and a second Softmax layer; the second linear layer is based on another fully connected network; The input end of the first extraction module is connected to the model input end of the target source prediction model, and the output end is connected to the input end of the second linear layer; the output end of the second linear layer is connected to the input end of the second Softmax layer; the output end of the second Softmax layer is connected to the model output end of the target source prediction model; The first extraction module is used to extract the first first word segmentation feature vector y in the first encoding vector Y j＝1 The extracted vector is sent as the corresponding second extracted vector C to the second linear layer; The second linear layer is used to perform a full connection calculation based on the second extraction vector C to obtain a corresponding second score vector D and send it to the second Softmax layer; the second score vector D is composed of two second scores d1 and d2; The second Softmax layer is used to perform binary probability prediction based on the second scores d1 and d2 of the second scoring vector D to obtain the corresponding second prediction vector V; and identify the two second prediction probabilities v1 and v2 of the second prediction vector V; if the second prediction probability v1 is too large, the corresponding first prediction result R is set to the target source type; if the second prediction probability v2 is too large, the corresponding first prediction result R is set to the non-target source type; the second prediction vector V is composed of two second prediction probabilities v1 and v2, wherein the second prediction probability v1 is a true value probability and the second prediction probability v2 is a false value probability.

10. The method for processing the methylation feature coding model according to claim 8, characterized in that: The performing first framework training on the pre-trained model framework according to the second base sequence to obtain corresponding encoding model pre-training parameters specifically includes: Step 1001: Use the pre-trained model framework as the corresponding current model framework; use the methylation feature encoding model and the sequence decoding model of the current model framework as the corresponding current encoding model and current decoding model; set two positive integers as the corresponding first and second stage training times thresholds; set two ratio parameters with values ​​between 0 and 1 as the corresponding first and second ratio parameters; set the first and second type random mask ratios built into the sequence word segmenter of the current encoding model as the corresponding first and second ratio parameters; and set two counters initialized to 0 as the corresponding first and second counters; Wherein, the first proportion parameter is less than the second proportion parameter, 10% is less than the first proportion parameter and less than 20%, and 90% is less than the second proportion parameter and less than 100%; Step 1002: Take the second base sequence as the corresponding first sequence X; perform label vector conversion on the current first sequence X to obtain a corresponding label vector PG; and perform smoothed label vector conversion on the label vector PG to obtain a corresponding smoothed label vector PS; Among them, the label vector PG consists of L sub-vectors pg i Composition; the subvector pg i and the first marker x of the first sequence X i One-to-one correspondence; each of the subvectors pg i Each is a one-hot encoding vector of length 5, consisting of 5 bases one-hot encoding pgA i 、pgT i 、pgC i 、pgG i 、pgM i Composition; each of the sub-vectors pg i The five base unique hot codes correspond one-to-one to the five types of base markers in the five-element base set [A, T, C, G, M]; each of the subvectors pg i Among the five base one-hot codes, only one is 1 and the other four are 0. The base marker corresponding to the base one-hot code of 1 is the same as the current sub-vector pg i The corresponding first marker x i match; The smoothed label vector PS consists of L sub-vectors ps i Composition; the sub-vector ps i and the subvector pg of the label vector PG i One-to-one correspondence; each of the sub-vectors ps i The length of the vector is 5, and psA is encoded by 5 vectors i 、psT i 、psC i , psG i 、psM i Composition; each of the sub-vectors ps i The five vector codes correspond one-to-one to the five types of base markers in the five-element base set [A, T, C, G, M]; The subvector pg i With the subvector ps i The conversion relationship is: α is a preset smoothing parameter, with a value between 0 and 1; I is a preset vector of all 1s with a length of 5; Condition 1 is: the subvector pg i The corresponding first marker x i is the base marker C or M; condition 2 is: the subvector pg i The corresponding first marker x i is the base marker A, T or G; Step 1003: Set the mask rule type built into the sequence word segmenter of the current coding model to a type of random mask; Step 1004: input the first sequence X into the sequence word segmenter of the current encoding model for processing to obtain the corresponding first embedded code vector E; Step 1005: Input the first embedded coding vector E into the MosaicBERT model of the current coding model for processing to obtain the corresponding first coding vector Y; input the first coding vector Y into the current decoding model for processing to obtain the corresponding first decoding sequence Z; and input the first subvectors u of the first prediction vector U generated during the current decoding model processing to obtain the first decoding sequence Z. i As a corresponding subvector pr i , and all the sub-vectors pr are obtained by i Form the corresponding prediction vector PR; The prediction vector PR consists of L sub-vectors pr i Composition; the sub-vector pr i and the second marker z of the first decoding sequence Z i One-to-one correspondence; the sub-vector pr i The first predicted probability of the five is denoted as prA i prT i 、prC i , prG i 、prM i , which correspond one-to-one to the five types of base markers in the five-member base set [A, T, C, G, M]; Step 1006: Substitute the prediction vector PR and the smoothed label vector PS into the preset first model loss function L M1 Calculate and obtain the corresponding first loss value; Among them, the first model loss function L M1 for: Step 1007, identifying whether the first loss value meets the preset first loss value range; if so, adding 1 to the first counter and going to step 1008; if not, moving towards making the first model loss function L based on the preset first model optimizer M1 Perform a round of modulation on the full model parameters of the current encoding model and the full model parameters of the current decoding model in the direction of reaching the minimum value, and return to step 1005 when this round of modulation is completed; Wherein, the first model optimizer includes at least an Adam optimizer and an SGD optimizer; Step 1008: Identify whether the first counter exceeds a threshold of the number of training times in a stage; if not, return to step 1004 to continue training; if exceeded, go to step 1009; Step 1009: setting the mask rule type built into the sequence word segmenter of the current coding model to a second-class random mask; Step 1010: Input the first sequence X into the sequence word segmenter of the current encoding model for processing to obtain the corresponding first embedded code vector E; Step 1011: input the first embedded coding vector E into the MosaicBERT model of the current coding model for processing to obtain the corresponding first coding vector Y; input the first coding vector Y into the current decoding model for processing to obtain the corresponding first decoding sequence Z; and input the first subvectors u of the first prediction vector U generated in the current decoding model processing to obtain the first decoding sequence Z. i As a corresponding subvector pr i , and all the sub-vectors pr are obtained by i Form the corresponding prediction vector PR; Step 1012: Substitute the prediction vector PR and the smoothed label vector PS into the first model loss function L M1 Calculate and obtain the corresponding second loss value; Step 1013, identifying whether the second loss value meets the preset second loss value range; if so, adding 1 to the second counter and going to step 1014; if not, moving towards making the first model loss function L based on the first model optimizer M1 Perform a round of modulation on the full model parameters of the current encoding model and the full model parameters of the current decoding model in the direction of reaching the minimum value, and return to step 1011 when this round of modulation is completed; Step 1014: Identify whether the second counter exceeds the second-stage training times threshold; if not, return to step 1010 to continue training; if exceeded, go to step 1015; Step 1015, confirming that the first framework training is completed, and saving the current model parameters of the current coding model as the corresponding pre-training parameters of the coding model; and solidifying the framework model parameters of the pre-training model framework.

11. The method for processing the methylation feature coding model according to claim 2, characterized in that: The constructing of a model dataset based on the BS-seq read set and the target source type specifically includes: A corresponding positive sample set is formed by the BS-seq read records whose read sample sources in the BS-seq read set match the target source type, and a corresponding negative sample set is formed by the BS-seq read records whose read sample sources in the BS-seq read set do not match the target source type; and taking the BS-seq reads of each of the BS-seq read records of the positive sample set as a corresponding first training sequence, and setting a first label classification result specifically set to the target source type; and forming a corresponding first data record by the first training sequence and the first label classification result corresponding to each of the BS-seq read records of the positive sample set; and taking the BS-seq reads of each of the BS-seq read records of the negative sample set as a corresponding first training sequence, and setting a first label classification result specifically set to the non-target source type; and forming a corresponding first data record by the first training sequence and the first label classification result corresponding to each of the BS-seq read records of the negative sample set; All the first data records obtained form the corresponding model data set.

12. The method for processing the methylation feature coding model according to claim 9, characterized in that: The performing a second framework training on the first task model framework based on the encoding model pre-training parameters and the model data set specifically includes: Step 1201: Use the first task model framework as the corresponding current model framework; use the methylation feature encoding model and the target source prediction model of the current model framework as the corresponding current encoding model and current prediction model; and set the mask rule type built into the sequence word segmenter of the current encoding model to no mask; Step 1202: solidify the model parameters of the methylation feature encoding model into the corresponding pre-trained parameters of the encoding model; and based on the LoRA fine-tuning mechanism, load a corresponding LoRA adapter on one or more designated modules of the methylation feature encoding model; and form a corresponding adapter parameter set from the adapter parameters of all the LoRA adapters; Among them, the solidified model parameters of each designated module are recorded as the parameters before fine-tuning W0, and the adapter parameters of the LoRA adapter corresponding to each designated module are composed of the matrix parameters of two low-rank matrices A and B; each LoRA adapter is used to fine-tune the parameters before fine-tuning W0 of the corresponding designated module to obtain the corresponding parameters after fine-tuning W1; the conversion relationship between the parameters after fine-tuning W1 and the parameters before fine-tuning W0 is: W1X in =W0X in +BAX in , X in is the input vector of the specified module, W0X in The current designated module is based on the input vector X when the corresponding LoRA adapter is not added in The output vector obtained by processing, W1X in The current specified module is based on the input vector X after adding the corresponding LoRA adapter in The output vector obtained by processing; Step 1203: dividing the model dataset into two sub-datasets according to a preset first segmentation ratio and recording them as a corresponding first training set and a first evaluation set; wherein both the first training set and the first evaluation set are composed of a plurality of the first data records; the ratio of the total number of records in the first training set to the total number of records in the first evaluation set satisfies the first segmentation ratio; and the ratio of positive and negative samples in the first training set and the first evaluation set are the same; Step 1204: taking the first first data record of the first training set as the corresponding current training record; Step 1205: setting a corresponding label vector TG based on the first label classification result of the current training record; Wherein, the label vector TG is composed of two one-hot codes tg1 and tg2; the one-hot codes tg1 and tg2 correspond to the target source type and the non-target source type, respectively; if the first label classification result of the current training record is the target source type, the one-hot code tg1 is 1 and the one-hot code tg2 is 0; if the first label classification result of the current training record is the non-target source type, the one-hot code tg1 is 0 and the one-hot code tg2 is 1; Step 1206: Input the first training sequence of the current training record as the corresponding first sequence X into the current model framework for processing to obtain the corresponding first prediction result R; and set a corresponding prediction vector TR based on the second prediction vector V generated by the current prediction model during this processing; The prediction vector TR is composed of two vector codes tr1 and tr2; the vector code tr1 is the second prediction probability v1 of the second prediction vector V, and the vector code tr2 is the second prediction probability v2 of the second prediction vector V; Step 1207: Substitute the prediction vector TR and the label vector TG into the preset second model loss function L M2 Calculate and obtain the corresponding third loss value; Among them, the second model loss function L M2 Implemented based on L1 loss function, L2 loss function or cross entropy loss function; Step 1208, whether the third loss value satisfies the preset third loss value range is identified; if the third loss value satisfies the third loss value range, whether the current training record is the last first data record of the first training set is identified, if so, go to step 1209, if not, the next first data record of the first training set is used as the new current training record and return to step 1205 to continue training; if the third loss value does not satisfy the third loss value range, then based on the preset second model optimizer, the second model loss function L is adjusted to the optimal value. M2 Perform a round of modulation on the adapter parameter set and the full model parameters of the current prediction model in the direction of reaching the minimum value, and return to step 1206 when this round of modulation is completed; Wherein, the second model optimizer includes at least an Adam optimizer and an SGD optimizer; Step 1209: perform a round of traversal on all the first data records of the first evaluation set; and during this round of traversal, use the first data record currently traversed as the corresponding current evaluation record; and set a corresponding label vector TG based on the first label classification result of the current evaluation record; and input the first training sequence of the current evaluation record as the corresponding first sequence X into the current model framework for processing to obtain the corresponding first prediction result R, and set a corresponding prediction vector TR based on the second prediction vector V generated by the current prediction model during this processing; and form a corresponding prediction-label pair by the label vector TG and the prediction vector TR corresponding to the current evaluation record; and at the end of this round of traversal, bring all the obtained prediction-label pairs into the preset first model evaluation function for calculation to obtain the corresponding first evaluation value; and calculate the accuracy, precision, recall rate, and F1 score based on all the obtained prediction-label pairs to obtain the corresponding first accuracy, first precision, first recall rate, and first F1 score; Wherein, the first model evaluation function is implemented based on the MSE function or the RMSE function; Step 1210: Identify whether the first evaluation value, the first accuracy, the first precision, the first recall rate, and the first F1 score all satisfy the corresponding first evaluation value range, first accuracy range, first precision range, first recall rate range, and first F1 score range; if all satisfy, proceed to step 1211; if at least one of the first evaluation value, the first accuracy, the first precision, the first recall rate, and the first F1 score does not satisfy the corresponding range, return to step 1203 to continue training; Step 1211, confirming that the second framework training is completed, and saving the encoding model pre-training parameters and the latest adapter parameter set as the corresponding fine-tuning encoding model parameter set; and solidifying the framework model parameters of the first task model framework.

13. The method for processing the methylation feature coding model according to claim 8, characterized in that: The methylation sequence reconstruction task of processing the DNA sequence based on the pre-trained model framework specifically includes: The DNA sequence received as input by the user is recorded as the corresponding first DNA sequence; the first DNA sequence is input as the corresponding first sequence X into the pre-trained model framework for processing to obtain the corresponding first decoding sequence Z, and the current first decoding sequence Z is fed back to the current user as the corresponding first reconstructed sequence; wherein the first DNA sequence is a marker sequence formed by sorting base markers in the four-element base set [A, T, C, G]; and the first reconstructed sequence is a marker sequence formed by sorting base markers in the five-element base set [A, T, C, G, M].

14. The method for processing the methylation feature coding model according to claim 9, characterized in that: The processing of a DNA sequence source identification task corresponding to the target source type based on the first task model framework specifically includes: The DNA sequence received from the user is recorded as the corresponding second DNA sequence; the second DNA sequence is input into the first task model framework as the corresponding first sequence X for processing to obtain the corresponding first prediction result R, and the current first prediction result R is fed back to the current user as the corresponding first recognition result; wherein the second DNA sequence is a marker sequence formed by sorting the base markers in the four-base set [A, T, C, G] or the five-base set [A, T, C, G, M].

15. A device for executing the method for processing the methylation feature coding model according to any one of claims 1 to 14, characterized in that: The device includes: a pre-processing module, a model building module, a first training module, a second training module and a model application module; The preprocessing module is used to set a corresponding extended marker for 5-methylcytosine in the DNA sequence as a base marker M, and adjust the four-membered base set [A, T, C, G] of the DNA to a corresponding five-membered base set [A, T, C, G, M]; and perform big data collection on the bisulfite sequencing reads of the first species to form a corresponding BS-seq read set; and perform read segment splicing based on the BS-seq read set to obtain a corresponding first base sequence; and perform five-element sequence conversion on the first base sequence based on the five-membered base set [A, T, C, G, M] and the methylation state marker corresponding to each base marker C on the first base sequence to obtain a corresponding second base sequence; and perform vocabulary learning based on the BPE algorithm and the second base sequence to obtain a corresponding first vocabulary; The model construction module is used to construct a methylation feature encoding model with the MosaicBERT model as the core; and store the first vocabulary in the vocabulary storage module of the methylation feature encoding model; and construct a sequence decoding model for methylation sequence reconstruction; and construct a corresponding two-class prediction model based on a class of target source types, recorded as a target source prediction model; the target source type is specifically a class of tissue type or a class of cell type; The first training module is used to form a pre-training model framework by using the methylation feature encoding model and the sequence decoding model; and perform first framework training on the pre-training model framework according to the second base sequence to obtain corresponding encoding model pre-training parameters; The second training module is used to form a first task model framework by the methylation feature encoding model and the target source prediction model; and to construct a model dataset based on the BS-seq read segment set and the target source type; and to perform a second framework training on the first task model framework based on the encoding model pre-training parameters and the model dataset; The model application module is used to process the methylation sequence reconstruction task of the DNA sequence based on the pre-trained model framework after the training of the first and second frameworks is completed; and to process a type of DNA sequence source identification task corresponding to the target source type based on the first task model framework.

16. An electronic device, characterized in that: include: memory, processors, and transceivers; The processor is configured to be coupled to the memory, read and execute instructions in the memory, so as to implement the method according to any one of claims 1 to 14; The transceiver is coupled to the processor, and the processor controls the transceiver to send and receive messages.

17. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a computer, the computer is caused to execute the method according to any one of claims 1 to 14.

Images