Method and apparatus for training prediction model, and electronic device, computer-readable storage medium and computer program product

Abstract

Provided are a method and apparatus for training a prediction model, and an electronic device, a computer-readable storage medium and a computer program product. The method is applied to the electronic device, and comprises: pre-training a first prediction model on the basis of a plurality of first-sample whole-genome sequences so as to obtain a second prediction model; extracting target dinucleotide sites from a plurality of dinucleotide sites in each of the first-sample whole-genome sequences, and using, as second-sample whole-genome sequences, sequences within a preconfigured window range containing the target dinucleotide sites; annotating the actual methylation state for the target dinucleotide sites in each of the second-sample whole-genome sequences; and training the second prediction model on the basis of each of the annotated second-sample whole-genome sequences so as to obtain a trained third prediction model, wherein the third prediction model is used for predicting the methylation state of dinucleotide sites in whole-genome sequences to be detected.

Description

Method, device, electronic device, computer readable storage medium and computer program product for training a prediction model

[0001] Cross-reference to Related Applications

[0002] This application is based on and claims priority to Chinese patent application No. 2024111623457, filed on August 22, 2024, the entire contents of which are incorporated herein by reference. TECHNICAL FIELD

[0003] The present application relates to the field of artificial intelligence, and in particular, to a method and device for training a prediction model, an electronic device, a computer readable medium and a computer program product. BACKGROUND

[0004] Methylation refers to the process of catalytically transferring a methyl group from an active methyl compound to other compounds, mainly involving the addition of a methyl group (-CH3) to a Deoxyribo Nucleic Acid (DNA) molecule, usually occurring at the 5th carbon atom of Cytosine. Abnormal methylation status is closely related to various diseases, such as cancer, nervous system diseases, etc. By detecting the methylation status of CpG dinucleotide sites in DNA, the gene expression regulation mechanism can be deeply understood, and the epigenetic causes of diseases can be explored.

[0005] The methylation status detection of related technologies mainly relies on methylation-specific experiments, which require complex experimental procedures and high sequencing costs. At the same time, due to the existence of noise and technical bias, the related technology may have a high error rate when processing complex biological samples. In the related technology, there is no good way to improve the accuracy of methylation status prediction. SUMMARY

[0006] The embodiments of the present application provide a method and device for training a prediction model, an electronic device, a computer readable storage medium and a computer program product, which can improve the accuracy of predicting the methylation status of gene sites.

[0007] The technical solutions of the embodiments of the present application are implemented as follows:

[0008] The embodiments of the present application provide a method for training a prediction model, which is applied to an electronic device, and the method comprises:

[0009] Pre-training a first prediction model based on a plurality of first sample whole genome sequences to obtain a second prediction model;

[0010] extract a target dinucleotide site from a plurality of dinucleotide sites of each of the first sample whole genome sequences, and take a sequence within a preconfigured window range containing the target dinucleotide site as a second sample whole genome sequence;

[0011] label an actual methylation state of the target dinucleotide site in each of the second sample whole genome sequences to obtain a labeled second sample whole genome sequence;

[0012] train the second prediction model based on each of the labeled second sample whole genome sequences to obtain a trained third prediction model, wherein the third prediction model is used to predict a methylation state of a dinucleotide site in a to-be-detected whole genome sequence.

[0013] An embodiment of the present application provides a prediction model training device, comprising:

[0014] a pre-training module configured to pre-train a first prediction model based on a plurality of first sample whole genome sequences to obtain a second prediction model;

[0015] a sample acquisition module configured to extract a target dinucleotide site from a plurality of dinucleotide sites of each of the first sample whole genome sequences, and take a sequence within a preconfigured window range containing the target dinucleotide site as a second sample whole genome sequence;

[0016] the sample acquisition module is configured to label an actual methylation state of the target dinucleotide site in each of the second sample whole genome sequences to obtain a labeled second sample whole genome sequence;

[0017] a fine-tuning training module configured to train the second prediction model based on each of the labeled second sample whole genome sequences to obtain a trained third prediction model, wherein the third prediction model is used to predict a methylation state of a dinucleotide site in a to-be-detected whole genome sequence.

[0018] An embodiment of the present application provides an electronic device, comprising:

[0019] a memory configured to store computer executable instructions;

[0020] a processor configured to execute the computer executable instructions stored in the memory to implement a prediction model training method provided by an embodiment of the present application.

[0021] An embodiment of the present application provides a computer readable storage medium storing computer executable instructions, which are used to implement a prediction model training method provided by an embodiment of the present application when executed by a processor.

[0022] The embodiment of the present application provides a computer program product, comprising computer executable instructions, when the computer executable instructions are executed by a processor, a training method of a prediction model is realized.

[0023] The embodiment of the present application has the following beneficial effects:

[0024] Through pre-training, the prediction model can learn related features in the whole genome sequence, the whole genome sequence in a specific range is used in the training stage, and the prediction model is trained in a targeted manner to improve the accuracy of the prediction of the methylation state. The prediction model of the methylation state is trained. By limiting the range of the training data, the data complexity is reduced, the training efficiency is improved, and the computing resources required in the training process are saved. The completed prediction model is used for detecting the genome sequence to be detected. Compared with the methylation state detection in the related art, the complex experiment does not need to be performed, the noise and interference mixed in the experiment process are avoided, and the efficiency and accuracy of the methylation state detection are improved. BRIEF DESCRIPTION OF DRAWINGS

[0025] Fig. 1 is a schematic diagram of an application mode of the training method of the prediction model provided by the embodiment of the present application;

[0026] Fig. 2 is a schematic diagram of the structure of an electronic device provided by the embodiment of the present application;

[0027] Fig. 3A is a first flowchart of the training method of the prediction model provided by the embodiment of the present application;

[0028] Fig. 3B is a second flowchart of the training method of the prediction model provided by the embodiment of the present application;

[0029] Fig. 3C is a third flowchart of the training method of the prediction model provided by the embodiment of the present application;

[0030] Fig. 3D is a fourth flowchart of the training method of the prediction model provided by the embodiment of the present application;

[0031] Fig. 3E is a fifth flowchart of the training method of the prediction model provided by the embodiment of the present application;

[0032] Fig. 4 is a first schematic diagram of the structure of the prediction model provided by the embodiment of the present application;

[0033] Fig. 5 is a sixth flowchart of the training method of the prediction model provided by the embodiment of the present application;

[0034] Fig. 6A is a first training principle diagram provided by the embodiment of the present application;

[0035] Fig. 6B is a second training principle diagram provided by the embodiment of the present application;

[0036] FIG. 6C is a second structural schematic diagram of a prediction model according to an embodiment of the present application;

[0037] FIG. 6D is a data flow schematic diagram of a prediction model according to an embodiment of the present application.

[0038] It should be noted that the above-mentioned "first", "second" are only used to distinguish different schemes, and do not represent the advantages or disadvantages of the schemes or the priority in the implementation process. DETAILED DESCRIPTION

[0039] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application will be described in further detail below with reference to the drawings, and the described embodiments should not be regarded as limiting the present application. All other embodiments obtained by those of ordinary skill in the art without making creative labor fall within the scope of protection of the present application.

[0040] In the following description, "some embodiments" are referred to, which describe a subset of all possible embodiments, but it can be understood that "some embodiments" can be the same subset or different subset of all possible embodiments, and can be combined with each other without conflict.

[0041] In the following description, the terms "first\second\third" are only used to distinguish similar objects, and do not represent a specific order of the objects. It can be understood that "first\second\third" can interchange the specific order or sequence as allowed, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0042] In the present application, the biometric feature (for example: DNA sequence, biological sample) recognition technology involved in the above embodiments of the present application applied to specific products or technologies should comply with the requirements of relevant laws and regulations in the process of collecting, using and processing relevant data. Before collecting the biometric features, the information processing rules should be informed and the individual consent of the target object should be sought, and the biometric features should be processed in strict accordance with the legal requirements and personal information processing rules, and technical measures should be taken to ensure the safety of relevant data.

[0043] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or part of a computer program with a predetermined function, which works with other related parts to achieve a predetermined target, and can be implemented entirely or partially by using software, hardware (such as processing circuit or memory) or combination thereof. Similarly, one processor (or multiple processors or memory) can be used to implement one or more modules or units. In addition, each module or unit can be a part of an overall module or unit that includes the functions of the module or unit.

[0044] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the description herein is for describing particular embodiments only and is not intended to be limiting of the application.

[0045] Before the embodiments of the present application are explained in further detail, the nomenclature and terminology used in the embodiments of the present application are explained, which are applicable to the following explanations.

[0046] 1) Deoxyribo Nucleic Acid (DNA): is an important molecule in cells that carries the genetic information of an organism, hereinafter referred to as DNA, DNA is composed of two long chains that are intertwined in the form of a double helix. Each long chain is composed of many small nucleotide units, each of which contains a phosphate group, a sugar (deoxyribose), and a nitrogen-containing base. There are four types of nitrogen-containing bases in DNA: adenine (A), thymine (T), cytosine (C), and guanine (G). At corresponding positions on the two long chains, adenine always pairs with thymine, and cytosine always pairs with guanine. These base pairs are connected to each other by hydrogen bonds, forming the double helix structure of DNA.

[0047] 2) Cytosine: scientific name 4-amino-2-carbonyl pyrimidine, is an organic substance, molecular formula C4H5N3O. It is one of the main base components in nucleic acids (DNA and RNA).

[0048] 3) CpG dinucleotide sites: a special DNA sequence, also known as CpG sites, abbreviated as CG sites, in which the base sequence is cytosine (C) and guanine (G), which are connected by a phosphodiester bond.

[0049] 4) Methyl group: a chemical group, chemical formula -CH3 ("-" represents a single electron), English abbreviation -Me, composed of carbon and hydrogen elements, a neutral monovalent group left after removing one hydrogen atom from a methane molecule. As a chemical group, methyl is commonly found in various organic compounds and is the most common group, which can bind to certain specific sites on DNA. The process of methyl binding to DNA is called methylation, and conversely, the process of shedding from the compound is called demethylation.

[0050] 5) Methylation: refers to the process of catalyzing the transfer of methyl groups from active methyl compounds to other compounds, and the methylation process can form various methyl compounds, or chemical modification of certain proteins or nucleic acids to form methylated products. In biological systems, methylation is catalyzed by enzymes, and this methylation is related to heavy metal modification, regulation of gene expression, regulation of protein function, and ribonucleic acid processing. DNA methylation is a form of DNA chemical modification that can change genetic expression without changing DNA sequence. DNA methylation refers to the covalent bonding of a methyl group to the fifth carbon of cytosine (CpG) in the genome under the action of DNA methyltransferase. Methylation status includes several types: 1. Unmethylated: cytosine (C) does not add a methyl group (-CH3) to the fifth carbon atom, i.e. the original CG site. 2. Partially methylated: cytosine (C) partially adds a methyl group (-CH3) to the fifth carbon atom, i.e. some of the CG sites are converted to 5-methylcytosine (mC). 3. Completely methylated: cytosine (C) completely adds a methyl group (-CH3) to the fifth carbon atom, i.e. the CG site is completely converted to 5-methylcytosine (mC). 4. Hypermethylation: cytosine (C) adds too many methyl groups (-CH3) to the fifth carbon atom, resulting in a methylation level of CG sites that exceeds the normal level. 5. Hypomethylation: cytosine (C) reduces or completely removes the methyl group (-CH3) from the fifth carbon atom, resulting in a methylation level of CG sites that is lower than the normal level.

[0051] 6) Whole Genome Sequencing (WGS): A high-throughput sequencing technology used to obtain and study the entire genetic information of an organism, referred to below as WGS. Through WGS, the base sequence of all genes, non-coding sequences, and mitochondrial DNA of an organism can be determined.

[0052] 7) Whole Genome Bisulfite Methylation Sequencing (WGBS): A sequencing technology used to detect DNA methylation status in the whole genome. Bisulfite treatment is used to convert unmethylated cytosine (C) to uracil (U), while methylated cytosine (G) remains unchanged. The treated DNA is amplified by polymerase chain reaction (PCR) and sequenced, and the methylation status of individual C bases in the whole genome can be reconstructed by bioinformatics analysis of the original sequencing data.

[0053] 8) Transformer: Transformer is a deep neural network model based on self-attention mechanism, widely used in various tasks in natural language processing field, such as text classification, machine translation and question answering system, etc. The model can convert input sequence into output sequence while preserving important information in the input sequence. Because the transformer model performs well in processing long text, it has been widely used in the field of Chinese natural language processing. Compared with traditional recurrent neural network (RNN) and convolutional neural network (CNN), the transformer model can be calculated in parallel, speeding up the training. It has been widely used in natural language processing, speech recognition, image generation and other tasks.

[0054] In the related art, methylation is an epigenetic modification phenomenon, mainly involving the addition of a methyl group (-CH3) to a DNA molecule, usually occurring at the 5th carbon atom of cytosine (Cytosine). Abnormal methylation status is closely related to various diseases, such as cancer, nervous system diseases, etc. The related art determines the methylation status by whole genome bisulfite methylation sequencing method, but the data processing is complex when the experiment is running. Due to the existence of noise and technical bias, the whole genome bisulfite methylation sequencing method may have a high error rate when processing complex biological samples.

[0055] The embodiment of the present application provides a kind of training method of prediction model, the training device of prediction model, electronic equipment and computer readable storage medium and computer program product, can improve the accuracy of predicting gene site methylation status.

[0056] The following describes an exemplary application of the electronic device provided by the embodiments of the present application. The electronic device provided by the embodiments of the present application can implement terminal devices such as notebook computers, tablet computers, desktop computers, set-top boxes, smart televisions, vehicle-mounted terminals, virtual reality (VR) devices, augmented reality (AR) devices, and various types of terminals. It can also be implemented as a server. In the following, exemplary applications when the electronic device is implemented as a terminal device or a server will be described.

[0057] Referring to FIG. 1, FIG. 1 is an application mode schematic diagram of the training method of the prediction model provided by the embodiments of the present application; for example, FIG. 1 involves a server 200, a network 300, a terminal device 400, and a database 500. The terminal device 400 connects the server 200 through the network 300, and the network 300 can be a wide area network or a local area network, or a combination of the two.

[0058] For example, the user can be a technician, the terminal device 400 can be a computer used by the technician, the server 200 can be a server of a data processing platform, and the database 500 can store a large amount of genomic data.

[0059] For example, the user sends a training request to the server 200 through the terminal device 400, the server 200 extracts genomic data corresponding to the training request from the database 500 according to the training request, the server 200 trains the prediction model by calling the training method of the prediction model provided in the embodiments of the present application, obtains a trained prediction model, and stores the trained prediction model in the server 200. When the user sends genomic data to be detected through the terminal device 400, or the terminal device 400 sends a detection request to the server 200, the server 200 extracts the genomic data to be detected in the database 500 based on the data identifier carried in the detection request, the server 200 performs prediction processing on the genomic data to be detected by using the trained prediction model, obtains the methylation state of the dinucleotide site in the genomic data to be detected, and sends data related to the methylation state to the terminal device 400. The technician can analyze the methylation state of the genomic data to be detected.

[0060] In some embodiments, the training method of the prediction model of the embodiments of the present application can also be applied in the following application scenarios:

[0061] 1. Diagnosis of diseases, prediction of the methylation state of gene sites related to diseases, which can help scientists understand the pathogenesis of diseases, find biomarkers of diseases, and develop new treatment methods. The prediction model is trained by the training method of the prediction model provided in the embodiments of the present application, which is used to perform prediction processing on the DNA sequence in the blood sample of a patient, to obtain the methylation state of the dinucleotide site in the DNA sequence. Researchers can diagnose whether the patient has a related disease according to the position of the dinucleotide site in the DNA sequence and the methylation state of the dinucleotide site.

[0062] For example, in the diagnosis of colorectal cancer, researchers use the trained third prediction model to perform prediction processing on the fecal DNA sample of a patient. The model input is the whole genome sequence (extracting target dinucleotide sites, such as the CpG island of the SEPT9 gene region), and the output is the methylation state of these sites. If the model predicts that the high methylation rate of the key site exceeds the threshold value (such as >80%), it indicates a high risk of cancer, and the doctor can recommend the patient to undergo a colonoscopy accordingly. This method is more sensitive than traditional methods and can reduce the false negative rate, and has been applied in clinical research (reference source: Epigenetic Screening Guidelines, such as Liquid Biopsy related research).

[0063] 2. Drug development. In the process of drug development, predicting the methylation status of drug targets can help optimize drug design and improve drug efficacy. The prediction model is trained by the training method provided in the embodiments of the present application, which is used to predict the data of biological macromolecules to obtain the methylation status of the target points on the biological macromolecules which are specifically acted by drugs, therapeutic proteins or other interventions, and then researchers can design drug molecules that are easy to combine with the target points according to the methylation status of the target points.

[0064] As an example, in the treatment of leukemia, researchers train a third prediction model based on the labeled second sample whole genome sequence (from patient bone marrow cells). Then, the model is applied to predict the whole genome data of patients in clinical trials: the input is the sequence before / after drug treatment (pre-configured window covering the DNMT3A gene site), and the output is the methylation status change of the target dinucleotide site. If the model predicts that the methylation level of the key site is significantly reduced (such as positively correlated with drug response), researchers can screen out a high-response patient population and optimize drug dosage (such as using Azacitidine). This reduces the cost of trial and error and improves the success rate of treatment (reference source: Epigenetics applications in cancer drug development, such as Nature Reviews Drug Discovery).

[0065] 3. Agricultural breeding and crop improvement. In agricultural biotechnology, the model can be used to predict the methylation status of dinucleotide sites in the whole genome of crops or livestock to associate with environmental response traits (such as drought resistance, disease resistance). Methylation status can reflect epigenetic variation, affecting gene expression without changing the DNA sequence itself. The present scheme enhances the prediction accuracy of the model for specific trait sites by local window training, helping breeders quickly screen excellent individuals and shorten the breeding cycle.

[0066] As an example: In rice breeding, researchers use the third prediction model to predict the whole genome sequence of a large number of rice lines. The model input is the extracted target dinucleotide site (such as the OsWRKY gene window sequence related to rice blast resistance), and the output is the methylation status. Based on the prediction results (such as low methylation of certain sites positively correlated with disease resistance), breeders can preferentially select rice lines with these methylation patterns for hybridization tests. This is more efficient than traditional gene editing and can improve yield in drought or disease environments (reference source: Crop epigenetics breeding research, such as Plant Biotechnology Journal).

[0067] In some embodiments, the server 200 can be a stand-alone physical server, or a server cluster or distributed system composed of multiple physical servers. For example, the server 200 can be a server cluster or distributed system composed of a training server and a data prediction server. The training server is used to train a prediction model, and when the training of the prediction model is completed, the training server sends the trained prediction model to the data prediction server. The data prediction server is used to perform prediction processing on a to-be-detected gene sequence based on the stored prediction model, to obtain the methylation state of a dinucleotide site in the gene sequence.

[0068] In some embodiments, the server 200 can be a stand-alone physical server, or a server cluster or distributed system composed of multiple physical servers. The terminal and the server can be connected directly or indirectly through wired or wireless communication, which is not limited in the embodiments of the present application.

[0069] The embodiments of the present application can be implemented through database technology. In short, a database is an electronic file cabinet that stores electronic files. Users can perform operations such as adding, querying, updating, and deleting data in the files. A “database” is a collection of data stored together in a certain way, shared by multiple users, with as little redundancy as possible, and independent of application programs.

[0070] A database management system (DBMS) is a computer software system designed to manage a database. It generally has basic functions such as storage, interception, security assurance, backup, etc. Database management systems can be classified according to the database model they support, such as relational, XML (Extensible Markup Language); or according to the type of computer they support, such as server clusters, mobile phones; or according to the query language they use, such as structured query language (SQL), XQuery; or according to the performance focus, such as maximum size, maximum running speed; or other classification methods. Regardless of the classification method used, some DBMSs can cross categories, such as supporting multiple query languages at the same time.

[0071] Referring to FIG. 2, FIG. 2 is a structural schematic diagram of an electronic device provided in an embodiment of the present application. The electronic device can be the server 200 of FIG. 1. The server 200 shown in FIG. 2 includes at least one processor 410, a memory 450, and at least one network interface 420. The various components in the server 200 are coupled together by a bus system 440. It can be understood that the bus system 440 is used to realize the connection and communication between the components. The bus system 440 includes not only a data bus, but also a power supply bus, a control bus, and a status signal bus. However, for the purpose of clear illustration, all the buses are marked as the bus system 440 in FIG. 2.

[0072] The processor 410 can be an integrated circuit chip having a processing capability of a signal, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, and the like. The general-purpose processor can be a microprocessor or any conventional processor.

[0073] The memory 450 can be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drives, optical drives, and the like. The memory 450 optionally includes one or more storage devices remotely located from the processor 410 in a physical location.

[0074] The memory 450 includes volatile memory or non-volatile memory, and can also include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), and the volatile memory can be random access memory (RAM). The memory 450 described in the embodiments of the present application is intended to include any suitable type of memory.

[0075] In some embodiments, the memory 450 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, which are exemplarily illustrated below.

[0076] The operating system 451 includes system programs for processing various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, and the like, for implementing various basic services and processing hardware-based tasks.

[0077] The network communication module 452 is used to reach other electronic devices via one or more (wired or wireless) network interfaces 420. Exemplary network interfaces 420 include Bluetooth, wireless compatibility authentication (WiFi), and universal serial bus (USB), and the like.

[0078] In some embodiments, the device provided by the embodiments of the present application can be implemented in a software manner. FIG. 2 shows a training device 455 of a prediction model stored in a memory 450, which can be software in the form of programs and plug-ins, and includes the following software modules: a pre-training module 4551, a sample acquisition module 4552, and a fine-tuning training module 4553. These modules are logical, and thus can be combined or further split according to the implemented functions. The functions of the modules will be described below.

[0079] In some embodiments, a terminal device or a server can implement the prediction model training method provided by the embodiments of the present application by running a computer program. For example, the computer executable instructions can be microprogram level commands, machine instructions, or software instructions. The computer program can be a native program in an operating system or a software module; can be a native application (APP), i.e., a program that needs to be installed in an operating system to run, such as an instant messaging APP; or can be a small program that can be embedded into any APP, i.e., a program that only needs to be downloaded into a browser environment to run. In summary, the above computer executable instructions can be any form of instructions, and the above computer program can be any form of application program, module, or plug-in.

[0080] The prediction model training method provided by the embodiments of the present application will be described in combination with exemplary applications and implementations of a server or a terminal device provided by the embodiments of the present application.

[0081] The prediction model training method provided by the embodiments of the present application will be described below. As mentioned above, the electronic device implementing the prediction model training method of the embodiments of the present application can be a terminal device or a server, or a combination of the two. Therefore, the execution subject of each step will not be repeated in the following description.

[0082] It should be noted that in the following examples of methylation state prediction processing, the data of whole gene sequences are used as examples. Those skilled in the art can apply the prediction model training method provided by the embodiments of the present application to processing of other types of gene data, such as RNA sequencing data (RNA-seq) and chromatin immunoprecipitation sequencing data (ChIP-seq), according to the understanding of the following description.

[0083] Referring to FIG. 3A, which is a flowchart of the prediction model training method provided by the embodiments of the present application, the steps shown in FIG. 3A will be described in combination. The execution subject of the steps in FIG. 3A is the server 200 in FIG. 1.

[0084] For ease of understanding, the application scenario of the training method of the prediction model of the embodiments of the present application is explained. Methyl is a chemical group that can be combined at certain specific sites on DNA. Methylation refers to the process of catalyzing the transfer of methyl from active methyl compounds to other compounds. Methylation status includes unmethylation, partial methylation, complete methylation, hypermethylation, and demethylation. Dinucleotide sites (CG sites) include cytosine (C) and guanine (G), which are components of deoxyribonucleic acid (DNA). DNA methylation refers to the covalent bonding of a methyl group to the 5th carbon of cytosine in the dinucleotide (CpG) of the genome under the action of DNA methyltransferase. In the embodiments of the present application, the methylation status of the dinucleotide site is divided into unmethylation and methylation status, wherein the methylation status includes partial methylation, complete methylation, hypermethylation, and demethylation. Based on the above scenario, the training method of the prediction model provided in the embodiments of the present application is explained below.

[0085] In step 301, a first prediction model is pre-trained based on a plurality of first sample whole genome sequences to obtain a second prediction model.

[0086] For example, the first prediction model includes an embedding layer and a transformer block, a prediction head. The first prediction model and the second prediction model, and the third prediction model below have the same structure. The "first", "second", and "third" are names used to distinguish different stages of the same model. The whole genome sequence is the entire genetic information of an organism obtained by whole genome sequencing (WGS) processing of DNA. The source of DNA can be a biological sample, such as a blood sample, a hair sample.

[0087] For example, the pre-training process is used to train the function of the first prediction model for feature extraction of the first sample whole genome sequence to obtain the second prediction model. The pre-training process can be self-supervised training. Self-supervised training (Self-Supervised Training) is a machine learning training method that does not rely on externally labeled data for learning. In self-supervised training, the model learns the internal structure and patterns of the input data to obtain knowledge, and automatically generates a supervision signal.

[0088] In some embodiments, before step 301, a real whole genome sequence and a reference whole genome sequence are obtained to construct a first training data set. The first training data set can be obtained by: performing whole genome sequencing processing on a biological sample to obtain a plurality of real whole genome sequences; and using a preconfigured reference whole genome sequence and the plurality of real whole genome sequences as the first sample whole genome sequence.

[0089] By breaking the limitation of single reference, the embodiments of the present application construct a dynamic reference benchmark closer to the genetic diversity of the research population, thereby improving the accuracy of the pre-training of the first prediction model.

[0090] For example, the biological sample is, for example, a blood sample collected in cancer screening. In actual application scenarios, the biological sample is collected with the consent of the subject from which the biological sample is collected and under legal circumstances. The reference whole genome sequence can be a manually edited whole genome sequence or a sequenced whole genome sequence. The number of reference whole genome sequences is at least one.

[0091] For ease of understanding, an example of a whole genome sequence is given as follows: "... AACAACGGTAGATCAGGATTAGA...". Since DNA is composed of two long chains that are intertwined in the form of a double helix, the nitrogenous bases of DNA have four types: adenine (A), thymine (T), cytosine (C), and guanine (G). At corresponding positions on the two long chains, adenine always pairs with thymine, and cytosine always pairs with guanine. Therefore, by determining the base arrangement of one long chain of DNA, the base arrangement in the other long chain of DNA can be determined.

[0092] In some embodiments, the above whole genome sequencing process on the biological sample to obtain a plurality of real whole genome sequences can be achieved by the following technical solutions: fragmenting deoxyribonucleic acid and preparing the fragmentation results into text data for sequencing; sequencing the text data to obtain a plurality of fragment sequencing reads; determining the arrangement order of bases in the genome sequence based on the plurality of fragment sequencing reads; and reconstructing the genome sequence based on the arrangement order to obtain a plurality of real whole genome sequences.

[0093] The whole genome sequencing process can be achieved by the following methods: fragmenting the extracted DNA using a specific method and preparing the fragmented DNA into text data for sequencing, using a high-throughput sequencing platform (such as Illumina or SOLiD) to sequence the prepared text data to obtain a large number of short fragment sequencing reads, processing, aligning, variant detection, and annotation of the original data obtained by sequencing to obtain bases in the genome sequence, the arrangement order of the bases, and the entire genome sequence. The above process usually involves complex bioinformatics analysis processes, including quality control, mapping, assembly, variant detection, and functional annotation.

[0094] As an example, long strands of DNA are randomly broken into small pieces that fit the read length of the sequencing platform, the fragmented DNA is processed into a format recognizable by the sequencer (addition of adapters, amplification, purification, selection of fragments), the sequences at both ends of the library fragments are read on the sequencer, the mass of short reads, disordered, possibly containing errors, are mapped back to the reference genome or assembled into continuous sequences, the base at each position is determined, the complete, individual-specific whole genome sequence of each sample is obtained from the alignment or assembly results.

[0095] The embodiments of the present application combine mature experimental operations and powerful bioinformatics analysis processes, so that individualized and high-quality complete genome sequences can be obtained.

[0096] In some embodiments, a self-supervised training method is used, and reference is made to FIG. 3B, which is a second flowchart of a training method of a prediction model according to an embodiment of the present application. Step 301 can be implemented by steps 3011 to 3013 in FIG. 3B, which are described in detail below.

[0097] In step 3011, the first prediction model is called to perform feature extraction processing on the plurality of first sample whole genome sequences, to obtain a first feature vector of each first sample whole genome sequence.

[0098] For example, the first prediction model includes an embedding layer, which is used to map each first sample whole genome sequence into a feature vector in the form of a word embedding vector. For example, the data corresponding to each base in the first sample whole genome sequence is mapped into a word representation, and the word representations corresponding to each position are sequentially combined to obtain the first feature vector.

[0099] The purpose of step 3011 is to extract meaningful, low-dimensional numerical representations (i.e., first feature vectors) from the original genomic data using the existing first prediction model. The goal of this step is to convert the original, high-dimensional, complex whole genome sequence (a long string composed of A, T, C, and G bases) into a fixed-length numerical vector (first feature vector) that contains rich information. This provides an easier-to-process and more informative input for subsequent construction of the second prediction model.

[0100] The input is a plurality of first sample whole genome sequences, which is the original input data, and each sample represents a complete genome sequence (such as DNA from different individuals or cells).

[0101] The first prediction model is a trained model. It is usually a deep neural network (such as CNN, Transformer or its variants) whose core capability is to understand the contextual information, patterns (such as motifs) and potential structures of DNA sequences. This model is trained on the first training dataset, meaning it has learned to extract useful and general features from raw DNA sequences.

[0102] Inside the model, when a whole genome sequence is input, the model processes it layer by layer: the bottom layers can recognize local patterns (such as specific k-mer combinations), the middle layers can combine these local patterns to recognize more complex structures or contextual relationships, and before the final layer (or a specially designed output layer), an internal representation of high dimension and abstraction is formed inside the model for the input sequence. This internal representation (usually the activation values of a certain layer or a certain pooling layer of the model) is extracted as the “first feature vector” of the sample, and the first feature vector is output

[0103] For each input first sample whole genome sequence, after processing by the first prediction model, a corresponding fixed-length numerical vector is output. This vector is no longer the original base sequence, but a compressed and abstract representation of the sequence as a whole or key characteristics learned by the model. It contains information that the model considers useful for understanding the structure of the sequence and potential biological significance (even if it is not directly related to methylation at this time).

[0104] A corresponding fixed-dimension feature vector (first feature vector) is generated for each input whole genome sequence through step 3011. These vectors are more compact and information-intensive than the original sequences, providing a high-quality input basis for subsequent construction of a second prediction model that focuses on a specific task (such as predicting methylation). Essentially, this is a key preprocessing step to convert high-dimensional sequence data into a numerical representation that is easier for the model to learn.

[0105] In step 3012, based on a plurality of first feature vectors, a first loss function of the first prediction model is determined.

[0106] For example, in self-supervised training, the first loss function is a function for measuring the similarity between feature vectors of samples with the same content and the similarity between feature vectors of samples with different content, and the goal in the self-supervised training process is to increase the similarity between samples with the same content and reduce the similarity between samples with different content.

[0107] In some embodiments, step 3012 can be implemented by performing, for each first sample whole genome sequence, the following processing: determining a first similarity between the first feature vector of the first sample whole genome sequence and the first feature vector of a third sample whole genome sequence, and determining a second similarity between the first feature vector of the first sample whole genome sequence and the first feature vector of a fourth sample whole genome sequence, wherein the third sample whole genome sequence is another first sample whole genome sequence having the same base ordering as the first sample whole genome sequence, and the fourth sample whole genome sequence is another first sample whole genome sequence having a different base ordering from the first sample whole genome sequence; taking a first ratio between the first similarity and a first sum as a first sub-loss function corresponding to the first sample whole genome sequence, wherein the first sum is a sum of the first similarity and the second similarity; and taking a mean of the plurality of first sub-loss functions as a first loss function of the first prediction model.

[0108] For example, assuming that the first sample whole genome sequence i is any one of all the first sample whole genome sequences, a first sample whole genome sequence j having the same base ordering as the first sample whole genome sequence i is obtained, and a first sample whole genome sequence l having a different base ordering from the first sample whole genome sequence i is obtained. The first similarity and the second similarity are obtained respectively, and the first similarity and the second similarity can be cosine similarities between the feature vectors. The first ratio can be represented by the following formula: the first similarity / (the first similarity+the second similarity).

[0109] For example, the loss function used in self-supervised training can be a contrastive loss or an information regularization loss. The contrastive loss is a loss function used in self-supervised learning, which aims to learn a model that can distinguish positive sample pairs (i.e., related samples) from each other, while pulling negative sample pairs (i.e., unrelated samples) closer to each other. The application of contrastive loss in NLP is usually to learn the embedding representation of text, which can be used for subsequent tasks such as text classification, information retrieval, etc.

[0110] The information regularization loss is a loss function widely used in self-supervised learning, especially in the setting of contrastive learning. The purpose of the information regularization loss is to let the model learn to distinguish between positive sample pairs (similar samples) and negative sample pairs (dissimilar samples), while penalizing the model for excessively high similarity judgments of positive sample pairs to avoid overfitting.

[0111] Through the embodiments of the present application, by comparing the similarities of the same / different ordering samples, the feature vectors are forced to retain genetic information (such as haplotype, structural variation) sensitive to base ordering patterns and filter noise (such as sequencing errors, batch effects) irrelevant to genetic differences, and the training effect of supervised training is optimized as a whole.

[0112] In step 3013, the first prediction model is iteratively parameter updated based on the first loss function, to obtain a second prediction model.

[0113] For example, the second prediction model is used to extract a feature vector of the to-be-detected whole genome sequence. The number of iterations and the learning rate used in the parameter update can be set according to the actual application scenario. The parameter update process is a back propagation process.

[0114] In the embodiments of the present application, the pre-training process is realized in a self-supervised training manner, without the need for manual or machine additional labeling of sample data. Since the data used in self-supervised training is not labeled, the model needs to learn to automatically extract useful feature representations from the data, which helps to reduce the risk of overfitting. The prediction model learns more rich feature representations, which helps to improve the generalization ability of the prediction model.

[0115] In some embodiments, the first sample whole genome sequence includes a plurality of dinucleotide sites; referring to FIG. 3D, which is a fourth flowchart of a training method of a prediction model provided by the embodiments of the present application, before step 302, the actual methylation state of each dinucleotide site can be determined through steps 3061 to 3062 of FIG. 3D, which will be described in detail below.

[0116] In step 3061, each first sample whole genome sequence is subjected to bisulfite treatment to obtain a plurality of treated first sample whole genome sequences.

[0117] Here, the treated first sample whole genome sequence includes a plurality of treated dinucleotide sites.

[0118] For example, whole genome bisulfite methylation sequencing is a method for determining the methylation state of DNA, which is based on the principle that unmethylated cytosine (C) is converted to uracil (U) by bisulfite treatment. After bisulfite treatment, the dinucleotide sites with methylation state in the original DNA will appear as GpG, and the non-methylated dinucleotide sites will appear as CpG or UpG.

[0119] Bisulfite Treatment is the core experimental technique for detecting DNA methylation status. This step provides basic data for constructing the first sample genome sequence methylation "true label" in the technical scheme. The purpose of step 3061 is to distinguish between methylated and unmethylated cytosine (C). Bisulfite can convert unmethylated cytosine (C) in DNA into uracil (U), while methylated cytosine (5mC) remains unchanged. In subsequent PCR amplification or sequencing, uracil (U) is recognized as thymine (T). Therefore, C in the original sequence is displayed as T in sequencing, indicating that the site is unmethylated, and C in the original sequence is still C in sequencing, indicating that the site is methylated.

[0120] The steps of bisulfite treatment are as follows: DNA denaturation, double-stranded DNA samples are dissociated into single-stranded DNA under alkaline conditions to ensure that bisulfite can fully contact cytosine; bisulfite treatment, single-stranded DNA reacts with bisulfite (such as sodium bisulfite): unmethylated C is converted to sulfonated cytosine, which is deaminated under alkaline conditions to generate uracil (U), and methylated C (5mC) is basically unchanged due to extremely slow reaction rate; purification and desulfonation, removing residual chemicals and converting sulfonated intermediates to stable uracil (U); PCR amplification and sequencing, using a polymerase that is not biased towards uracil (U) to amplify DNA, and finally obtaining the converted sequence by sequencing.

[0121] Original sequence...ACG...(assuming the second C is unmethylated)→ becomes...ATG...after treatment (i.e. C→T conversion, while methylated C remains C).

[0122] Generating training labels: by comparing the sequence differences before and after treatment, the true methylation status of each cytosine site (especially CpG dinucleotide) can be directly labeled. These labeled data are used in the pre-training phase: to construct the feature vector of the first prediction model (such as self-supervised learning), and in the fine training phase: to provide a supervised signal for the second prediction model (such as labeling the methylation status of the target CpG site). The sequence after treatment constitutes the "gold standard" data for subsequent model training and verification, ensuring that the model prediction results have biological interpretability.

[0123] In step 3062, based on the dinucleotide site before treatment and the corresponding plurality of dinucleotide sites after treatment, the actual methylation status of each dinucleotide site is determined.

[0124] For example, based on the principle of bisulfite treatment, the dinucleotide sites before and after treatment can be compared to determine whether the original dinucleotide site has a methylation status.

[0125] In some embodiments, step 3062 can be implemented by: for each first sample whole genome sequence, aligning the pre-processed di-nucleotide site with the (corresponding) plurality of post-processed di-nucleotide sites to obtain a plurality of alignment results corresponding to each di-nucleotide site; determining that the methylation state of the di-nucleotide site is unmethylated when the alignment result is that the cytosine of the di-nucleotide site is converted to uracil or is not converted; determining that the methylation state of the di-nucleotide site is methylated when the alignment result is that the cytosine of the di-nucleotide site is converted to guanine; and determining the actual methylation state of the di-nucleotide site based on a second ratio between the number of methylations of the di-nucleotide site and the total number of occurrences of the di-nucleotide site, for each di-nucleotide site, wherein the second ratio is the probability of methylation of the di-nucleotide site. By aligning the pre-processed original site with the plurality of post-processed same sites (i.e. multiple rounds of sequencing or technical repeats) in the embodiments of the present application, the influence of single experiment errors (such as PCR amplification bias, sequencing errors) is significantly reduced.

[0126] For example, since the individual data is not generalizable, the results of a plurality of bisulfite treatments of the same di-nucleotide site are counted, and the number of methylations and the number of non-methylations are counted according to the alignment results, and the sum of the total number of occurrences of the di-nucleotide site at the same position, i.e. the number of methylations and the number of non-methylations, is calculated to represent the formula of the second ratio as follows wherein M count is the number of methylations, and NM count is the number of non-methylations.

[0127] In the embodiments of the present application, the actual methylation state is determined by experiment, which provides more accurate labeled data for subsequent training to improve the accuracy of training. The methylation state is determined by a plurality of data, and the methylation state is represented by a probability, which avoids the singularity of the data and improves the reliability of the training data.

[0128] With reference back to FIG. 3A, in step 302, target di-nucleotide sites are extracted from a plurality of di-nucleotide sites of each first sample whole genome sequence, and sequences within a pre-configured window range containing the target di-nucleotide sites are taken as second sample whole genome sequences.

[0129] For example, that is, the second sample whole genome sequence is a whole genome sequence cut from the complete first sample whole genome sequence based on the target di-nucleotide site as the reference site. The cutting range is the pre-configured window range, which can be determined according to the actual scene.

[0130] In some embodiments, referring to FIG. 3C, which is a third flowchart of a method for training a prediction model according to an embodiment of the present application, step 302 can be implemented by step 3021 in FIG. 3C, which is described in detail as follows.

[0131] For example, the dinucleotide sites are obtained by whole genome sequencing, which has been described above and will not be repeated here. The target dinucleotide site can be at least one of the plurality of dinucleotide sites, or all of them.

[0132] In step 3021, a sequence within a preconfigured window range is cut from the first sample whole genome sequence, as the second sample whole genome sequence, with the target dinucleotide site as the reference site.

[0133] Here, the reference site is located in the preconfigured window range in any of the following positions: the boundary of the preconfigured window range, the interior of the preconfigured window range.

[0134] For example, the target dinucleotide site is at a certain position in the first sample whole genome sequence, and a sequence of window width W is cut from the first sample whole genome sequence as the second sample whole genome sequence, with the position as the center.

[0135] In the embodiments of the present application, by limiting the length of the training sample, the computing resources consumed in the training process can be saved, and the pertinence of the prediction processing of the prediction model can be improved. By limiting the window width, the amount of data that needs to be processed by the prediction model can be reduced, thereby speeding up the training process and improving efficiency. A suitable window width can help the prediction model find a balance between remembering patterns in the genome sequence and generalizing to new data.

[0136] Continuing to refer to FIG. 3A, in step 303, the actual methylation state of the target dinucleotide site in each second sample whole genome sequence is labeled to obtain a labeled second sample whole genome sequence.

[0137] For example, the method for obtaining the actual methylation state has been described in steps 3061 to 3062 above, which will not be repeated here. For each second sample whole genome sequence, the position of the target dinucleotide site in the second sample whole genome sequence is labeled, and the actual methylation state of the target dinucleotide site is labeled. The actual methylation state is represented by a numerical value, and the value range is 0 to 1. The closer to 1, the more likely the target dinucleotide site is in the methylation state.

[0138] For example, for each target dinucleotide site, the actual methylation state and the second sample whole genome sequence are combined into a second training data set.

[0139] In some embodiments, the second training dataset is a training set of the second prediction model, and a validation set and a test set of the second prediction model can also be made based on the real data corresponding to the first training dataset. The validation set is a dataset used to adjust the hyperparameters of the model and evaluate the performance of the model. The test set is a dataset used to evaluate the performance of the trained model. After the model training is completed, the test set is used to verify the generalization ability of the model to unknown data. The performance of the model on the test set can help evaluate the accuracy and performance of the model.

[0140] In step 304, the second prediction model is trained based on each of the labeled second sample whole genome sequences, to obtain a trained third prediction model.

[0141] Here, the third prediction model is used to predict the methylation state of the dinucleotide site in the whole genome sequence to be detected.

[0142] For example, the training process is used to train the second prediction model to predict the methylation state of the dinucleotide site in the whole genome sequence to be detected. The training process can be iterative, and the loss function used can be cross-entropy loss or binary classification loss.

[0143] In some embodiments, referring to FIG. 3E, which is a fifth flowchart of a method for training a prediction model according to an embodiment of the present application, step 304 can be implemented by steps 3041 to 3044 in FIG. 3E, which are described in detail below.

[0144] In step 3041, the second prediction model is called to perform the following processing: performing feature extraction processing on each second sample whole genome sequence to obtain a second feature vector of each second sample whole genome sequence.

[0145] For example, the feature extraction processing can refer to step 3011 described above.

[0146] The purpose of step 3041 is to use the existing second prediction model to extract meaningful, low-dimensional numerical representations (i.e., second feature vectors) from the original genomic data. The goal of this step is to convert the original, high-dimensional, complex whole genome sequence (a long string composed of A, T, C, G bases) into a fixed-length numerical vector (second feature vector) that contains rich information. This provides an easier-to-process, more informative input for subsequent construction of the second prediction model.

[0147] The input is a plurality of second sample whole genome sequences, which is the original input data, and each sample represents a complete genomic sequence (such as DNA from different individuals or cells).

[0148] Within the model, when an input whole genome sequence is given, the model processes it layer by layer: the bottom layers can recognize local patterns (e.g. specific k-mer combinations), the middle layers can combine these local patterns to recognize more complex structures or contextual relationships, and before the final layer (or a specially designed output layer), an internal high-dimensional, abstract representation of the input sequence is formed within the model. This internal representation (usually the activations of the model’s penultimate layer or a certain pooling layer) is extracted as the “second feature vector” of the sample, and the second feature vector is finally outputted

[0149] For each input second sample whole genome sequence, after being processed by the second prediction model, a corresponding fixed-length numerical vector is outputted. This vector is no longer the original base sequence, but a compressed, abstract representation of the sequence as a whole or of its key characteristics, learned by the model. It contains information that the model considers useful for understanding the sequence’s structure and potential biological significance (even if it has not yet directly involved methylation).

[0150] In step 3041, a corresponding fixed-dimension feature vector (second feature vector) is generated for each input whole genome sequence. These vectors are more compact and information-dense than the original sequences, providing a high-quality input basis for subsequent construction of a second prediction model that focuses on a specific task (such as predicting methylation).

[0151] In step 3042, based on the second feature vector of each of the second sample whole genome sequences, a first prediction probability of the target dinucleotide site being in a methylation state is determined.

[0152] In some implementations, step 3042 can be implemented by performing the following processing for each of the second sample whole genome sequences: performing multiple levels of feature conversion processing on the second feature vector to obtain a converted second feature vector, wherein the feature conversion processing includes feature extraction based on a transformer block and column pooling processing; calling an attention mechanism based on the converted second feature vector to perform aggregation processing to obtain a global feature; and performing activation processing based on the global feature of the second sample whole genome sequence to obtain the first prediction probability of the target dinucleotide site being in a methylation state.

[0153] In some embodiments, the second prediction model includes an embedding layer, a plurality of transformer blocks, and a prediction head, wherein the embedding layer is used for feature extraction processing on the whole genome sequence, the plurality of transformer blocks are used to perform multiple levels of feature conversion processing, and the prediction module is used to perform aggregation processing and activation processing. The second prediction model, the first prediction model, and the third prediction model are names of different stages of the same model, and the first prediction model and the third prediction model have the same structure as the second prediction model.

[0154] For example, referring to FIG. 4, FIG. 4 is a first structural schematic diagram of a prediction model provided by an embodiment of the present application. The prediction model 401 includes an embedding layer 402, a transformer block 403, and a prediction head 404. The prediction head further integrates global information through a global attention mechanism. The global attention mechanism enhances the semantic understanding ability of the model by calculating the relevance between all positions. The prediction head contains multiple linear layers and activation functions, such as SiLU and Linear, for generating the final result.

[0155] Through the cascading design of local feature conversion (transformer + column pooling) -> global attention aggregation -> probability activation by the embodiments of the present application, high-precision methylation state prediction is achieved, and robust modeling is achieved in cross-organism and cross-species scenarios.

[0156] In step 3043, a second loss function of the second prediction model is determined based on the label probability corresponding to the actual methylation state of each target dinucleotide site and the first prediction probability.

[0157] For example, the second loss function is the mean of the sub-losses corresponding to each target dinucleotide site, and the sub-loss can be a cross-entropy loss or a binary classification loss. In the embodiments of the present application, the binary classification loss is taken as an example for illustration.

[0158] In some embodiments, step 3043 can be implemented in the following manner: for each target dinucleotide site, a binary classification loss between the label probability corresponding to the actual methylation state and the first prediction probability is determined; and the mean of the plurality of binary classification losses is taken as the second loss function of the second prediction model.

[0159] For example, the binary classification loss can be represented by the following formula (1):

[0160] L = - [ylogy` + (1 - y)log(1 - y')]

[0161] Wherein, L is the binary classification loss, y is the label probability corresponding to the actual methylation state, and y' is the first prediction probability.

[0162] Through the combination of the probabilistic label and the mean of the binary classification loss by the embodiments of the present application, the gradient update direction is strictly aligned with the biological significance, and the noise resistance is more robust: the mean aggregation shields local anomalies, and finally promotes the prediction accuracy of the methylation prediction model in complex scenarios.

[0163] In step 3044, the second prediction model is iteratively parameter updated based on the second loss function, to obtain a third prediction model.

[0164] For example, the third prediction model is used to determine the methylation state of a dinucleotide site in a whole genome sequence. The number of iterations and the learning rate used in parameter updating can be set according to the actual application scenario. The parameter updating process is a back propagation process.

[0165] In the embodiments of the present application, by limiting the range of training data, the data complexity is reduced, the training efficiency is improved, and the computing resources required in the training process are saved. By dividing the training into two stages of pre-training and fine-tuning, the generalization and accuracy of the model obtained by training can be improved.

[0166] In some embodiments, after step 304, the following processing is performed: obtaining a third training data set, wherein the third training data set includes: a plurality of third whole genome sequences and sample label probabilities of sample dinucleotide sites corresponding to the third whole genome sequences respectively; calling a third prediction model based on the plurality of third whole genome sequences to perform prediction processing, to obtain a second prediction probability of the sample dinucleotide sites in each third whole genome sequence being in a methylation state; determining a first difference value between the sample label probability and the second prediction probability (corresponding to the sample dinucleotide site), and determining a first number of first difference values less than a preconfigured difference value; in response to the accuracy of the third prediction model being less than a preconfigured accuracy, taking the third prediction model as a second prediction model, and returning to step 304.

[0167] For example, the accuracy is a ratio between the first number and the total number of sample dinucleotide sites. The first difference value is less than the preconfigured difference value, indicating that the second prediction probability predicted by the third prediction model is close to the sample label probability, and thus the first number of first difference values can be used to represent the accuracy of the third prediction model. By detecting the accuracy of the validation set, if the accuracy is low, the prediction model is retrained. The third training data set and the first training data set are extracted from the same data set at a specific ratio, and the specific ratio can be set according to the actual application scenario. The first training data set is used as a training set, and the third training data set is used as a validation set in the training process.

[0168] The embodiments of the present application innovatively combine probability continuity evaluation and dynamic model degradation, achieving three major breakthroughs: evaluation refinement: using probability difference instead of binary accuracy to reveal the true performance of the model in the confidence interval; iteration intelligence: automatically triggering model rollback based on quantitative indicators to avoid resource waste.

[0169] In some embodiments, after step 304, the following processing is performed: based on the to-be-detected whole genome sequence, a third prediction model is called to perform the following processing: performing feature extraction processing on the to-be-detected whole genome sequence to obtain a fourth feature vector of the to-be-detected whole genome sequence; based on the fourth feature vector, determining a third prediction probability that a to-be-detected dinucleotide site included in the second sample whole genome sequence is in a methylation state; and in response to the third prediction probability being greater than a preconfigured probability, determining that the to-be-detected dinucleotide site is in the methylation state.

[0170] For example, the preconfigured probability can be set according to an actual application scenario, and the preconfigured probability has a value range of a positive number less than 1. When the third prediction probability is greater than the preconfigured probability, it is indicated that the to-be-detected dinucleotide site is in the methylation state.

[0171] The setting of the preconfigured probability (decision threshold) needs to balance the prediction accuracy and the application scenario requirement, and its essence is to draw a dividing line between the continuous methylation probability (third prediction probability) output by the model and the binary classification decision (methylation / non-methylation). The performance of the verification set can be used for optimization: first, generate a probability distribution, input the third prediction model using an independent verification set (containing dinucleotide sites with known methylation states) to obtain the prediction probability (third prediction probability) of each site; then draw an evaluation curve, the ROC curve is used to calculate the true positive rate (TPR) and the false positive rate (FPR) under different thresholds, and the PR curve (suitable for sample imbalance scenarios) is used to calculate the precision (Precision) and the recall (Recall); select the optimal threshold to maximize the first indicator (TPR-FPR (the left upper corner point of the ROC curve)), and maximize the F1 score: F1=2×(Precision×Recall) / (Precision+Recall) (peak point of the PR curve); if there is a high specificity requirement (such as clinical diagnosis), a high threshold (such as 0.95) is selected to reduce false positives, and if there is a high sensitivity requirement (such as screening), a low threshold (such as 0.7) is selected to reduce false negatives.

[0172] In the embodiments of the present application, the trained model can be used to detect the methylation state of the whole genome sequence. Compared with the related art scheme of determining the methylation state of the whole genome sequence by experimental means, the trained model does not need to perform complex experiments, and the efficiency and accuracy of the methylation state detection are improved.

[0173] In some embodiments, the training data used in the training method of the prediction model provided in the embodiments of the present application includes but is not limited to RNA sequencing data (RNA-seq), chromatin immunoprecipitation sequencing data (ChIP-seq), and various biological data. By integrating various biological data, the prediction model can capture methylation information in the genome from different aspects, thereby improving the accuracy and coverage of the prediction.

[0174] In the embodiments of the present application, the pre-training enables the prediction model to learn the relevant features in the whole genome sequence. In the training stage, the whole genome sequence within a specific range is used for targeted training to improve the accuracy of the prediction model in predicting the methylation state. The prediction model of the methylation state is trained. By limiting the range of training data, the data complexity is reduced, the training efficiency is improved, and the computing resources required in the training process are saved. The trained prediction model is used to detect the genome sequence to be detected. Compared with the methylation state detection in the related art, the complex experiment is not required, the noise and interference mixed in the experiment process are avoided, and the efficiency and accuracy of the methylation state detection are improved.

[0175] In the following, an exemplary application of the training method of the prediction model in the embodiments of the present application in an actual application scenario will be described.

[0176] Methylation is an epigenetic modification phenomenon, mainly involving the addition of a methyl group (-CH3) to a DNA molecule, usually occurring at the 5th carbon atom of cytosine (Cytosine). Methylation is an important epigenetic regulation mechanism, which has important influence on biological processes such as gene expression, gene silencing, genome stability, chromatin structure and development. Abnormal methylation state is closely related to many diseases, such as cancer, nervous system diseases, etc. The existing whole genome bisulfite sequencing method (WGBS) can obtain methylation information in the whole genome range. The whole genome bisulfite sequencing method has high resolution and comprehensive coverage, but the experimental cost is high and the data processing is complex.

[0177] The related art solution can be divided into two categories: (1) prediction method based on sequence base pair arrangement pattern; (2) prediction method based on methylation site sequence sequencing.

[0178] For scheme (1), prediction methods based on sequence base pair arrangement patterns: These methods predict the methylation status by analyzing the base arrangement patterns of DNA sequences. By analyzing plasma samples from healthy individuals and cancer patients, the fragment patterns of cell-free DNA (cfDNA) are analyzed using whole genome sequencing (WGS) and whole genome bisulfite sequencing (WGBS). Studies have found that methylated dinucleotide sites (CpG dinucleotide sites), i.e., CG sites, have a higher probability of cutting at the CGN end and a lower probability of cutting at the NCG end. Therefore, at highly methylated dinucleotide sites, the CGN / NCG ratio is higher, while at lowly methylated dinucleotide sites, the CGN / NCG ratio is lower. By analyzing these ratios, the methylation level of the target sequence can be predicted. CGN-end break refers to the break point occurring at the dinucleotide site, i.e., the break occurring at the endpoint of cytosine or guanine. This break can affect the methylation status of the dinucleotide site, especially the methylation changes occurring near the break point. NCG-end break refers to the break point occurring at the non-dinucleotide sequence next to the dinucleotide site. This break may not directly affect the methylation status of the dinucleotide site, but it can affect the overall structure and stability of the genomic region where the dinucleotide site is located.

[0179] For scheme (2), prediction methods based on methylation site sequence sequencing: These methods predict the methylation status by directly sequencing the sequences of methylation sites in DNA samples. By collecting plasma samples from healthy individuals and various types of cancer patients, whole genome bisulfite sequencing (WGBS) and whole genome sequencing (WGS) are performed on cell-free DNA, and the FinaleMe model is created to predict the methylation status in the fragment pattern of cell-free DNA using non-homogeneous hidden Markov model (NHMM) and Gaussian mixture model (GMM). The FinaleMe model can accurately predict the methylation status and has verified its prediction ability in multiple cancer types, demonstrating its potential in non-invasive diagnosis of cancer.

[0180] However, the related art has the following problems: (1) High experimental complexity and cost: Related technologies, such as prediction methods based on sequence base pair arrangement patterns and prediction methods based on methylation site sequence sequencing, usually require complex experimental procedures and high sequencing costs. Especially whole genome bisulfite sequencing (WGBS), which requires bisulfite treatment and high-throughput sequencing of DNA, increases the complexity and cost of the experiment. (2) Complex data processing: The sequencing data processing of related technologies requires complex bioinformatics analysis, including data preprocessing, feature extraction and model training steps. These steps not only require high-performance computing resources, but also require professional bioinformatics knowledge and skills, which limits the popularization and promotion of these technologies in practical applications. (3) Limited prediction accuracy: Related technologies rely on sequence features or specific methylation sites, which may not fully capture the changes in methylation status across the genome, resulting in limited accuracy and coverage of the prediction. In addition, due to the existence of noise and technical bias, existing methods may have a high error rate when processing complex biological samples.

[0181] The embodiment of the present application proposes a training method of a prediction model, which pre-trains by using whole genome sequencing data (WGS) and fine-tunes by using paired whole genome sequencing data and the methylation status of CG sites (two nucleotide sites) to predict the methylation status of gene sites, which can solve the following problems of related technologies: (1) Reduce experimental complexity and cost: No methylation-specific experiment is required, only whole genome sequencing data is required to predict the methylation status, which significantly reduces the experimental complexity and cost. (2) Simplify the data processing process: By using genomics large models, the data processing process is simplified, the accuracy and efficiency of the prediction are improved, and the dependence on high-performance computing resources and professional skills is reduced. (3) Improve prediction accuracy and coverage: It can fully capture the changes in methylation status across the genome, and by integrating whole genome sequencing data and advanced machine learning techniques, the accuracy and coverage of methylation status prediction are significantly improved, effectively dealing with noise and technical bias in complex biological samples.

[0182] The training method of the prediction model provided by the embodiments of the present application will be explained and described below in combination with the accompanying drawings. Referring to FIG. 5, FIG. 5 is a sixth flowchart of the training method of the prediction model provided by the embodiments of the present application; the execution subject of FIG. 5 is the server 200 in FIG. 1.

[0183] In step 501, a data set of whole genome sequencing data is constructed.

[0184] For example, referring to FIG. 6A, which is a first training principle diagram provided by an embodiment of the present application; blood samples are collected from multiple cancer screenings and subjected to whole genome sequencing (WGS) to obtain whole genome sequence data of the samples, reference sequences (simulation data) are combined with WGS sequences (real data) of the samples to construct a training data set.

[0185] The whole genome sequencing data shown in FIG. 6A includes reference data and sample data. Taking the reference data as an example, the content of the reference data is “……AACAACGGTAGATCAGGATTAGA……”. DNA is composed of two long chains, which are intertwined in the form of a double helix. The nitrogenous bases of DNA have four types: adenine (A), thymine (T), cytosine (C) and guanine (G). At the corresponding positions of the two long chains, adenine is always paired with thymine, and cytosine is always paired with guanine, so only one long chain of DNA needs to be determined to determine the base arrangement in the other long chain of DNA. The reference data and the sample data are both represented in the form of one long chain, and the base arrangement of the other long chain can be determined according to the one long chain shown.

[0186] In some embodiments, bioinformatics methods are used to extract biological features in the whole genome sequencing data, such as CG site (dinucleotide site) content, repetitive sequences, genomic domains, etc. These features are used as input to train a machine learning model (such as a support vector machine or a random forest model) for methylation state prediction. Existing biological knowledge can be used to improve the interpretability of the prediction.

[0187] In an embodiment of the present application, whole genome sequencing data (WGS) is used to predict the methylation state of gene sites, improving the accuracy of the prediction. At the same time, the model is used to detect the whole genome sequencing data, without the need for methylation-specific experiments, and only the whole genome sequencing data is needed to predict the methylation state, thereby reducing the complexity and cost of the experiment.

[0188] In step 502, the prediction model is pre-trained based on the data set of the whole genome sequencing data, and a pre-trained prediction model is obtained.

[0189] For example, the training method of the prediction model provided by the embodiments of the present application includes two stages of pre-training and fine-tuning. In the pre-training stage, the preliminary training of the deep learning model is performed on a large amount of whole genome sequencing (WGS) data (characterized as sequence form), and potential features and patterns in the genomic sequence are extracted. In the fine-tuning stage, the model is further trained using specific methylation site data to optimize and improve the accuracy and reliability of the methylation state prediction. The purpose is to reduce the experimental complexity and cost, while providing an efficient and accurate DNA methylation state prediction solution.

[0190] With continued reference to FIG. 6A, the prediction model 601A is a deep learning architecture, which includes layers such as feed forward, column attention, row attention, and layer normalization (Layer Norm). The pre-training method can be self-supervised learning training to enable the model to generate a feature vector describing the input genomic sequence. During the model training process, the model implicitly learns the coverage information of the WGS sequence, the CGN / NCG ratio information of the CG site, the length information of each sequencing fragment, and the sequence information at both ends of the sequencing fragment.

[0191] In some embodiments, the prediction model 601A can be a combination of multiple different types of models, such as deep learning models, traditional machine learning models, and statistical models, etc., working together to make methylation state predictions. Ensemble learning can take full advantage of the strengths of different models, reduce prediction errors, and improve the stability and accuracy of predictions.

[0192] In step 503, a data set for fine-tuning training is constructed.

[0193] For example, the chromosome and position information of all CG sites are obtained by traversing the human reference genome, which is used for subsequent dataset construction. For example, blood samples are collected from multiple cancer screenings and subjected to whole genome sequencing (WGS) and whole genome bisulfite sequencing (WGBS), and CG sites in the reference genome are identified. Whole genome sequencing is a technology that can determine the entire DNA sequence of an individual's entire genome. By aligning the sequencing reads to the reference genome, the nucleotide sequence of each site can be determined. Whole genome bisulfite sequencing is a method for determining DNA methylation status. The principle is that unmethylated cytosine (C) is converted to uracil (U) by bisulfite treatment, and then sequencing is performed. The methylated dinucleotide site in the original DNA will appear as GpG, and the unmethylated dinucleotide site will appear as CpG or UpG. Then, the methylation status of the dinucleotide site can be determined by sequencing and alignment analysis. By aligning the sequence read to the reference genome, the methylation status of the dinucleotide site can be determined according to U (representing the original C in the dinucleotide site converted to U) and G (representing the original GpG site).

[0194] Referring to FIG. 6B, FIG. 6B is a second training principle diagram provided by the embodiments of the present application. Since the CG sites in each sequence can represent different methylation states, the individual data does not have generalizability, and then the methylation status of each CG site can be calculated according to multiple detection results, and the reference genome sequence of the CG site within the defined window size W is combined with the whole genome sequencing data to obtain a training sample, construct a training dataset, and the training label of the training sample is the methylation status of the CG site in the whole genome bisulfite sequencing result.

[0195] The training dataset can be constructed using the whole genome sequencing result of the CG site in the following manner: selecting the hyperparameter window width W (the window width defines the range of the genomic region to be analyzed, and a specific sequence length can be set as the window width according to the actual application scenario) to obtain the whole genome sequencing information around the CG site, while recording the corresponding chromosome, position, methylation status and other information. For each CG site, the methylation status of the site is calculated using the number of methylated and unmethylated CG sites in the genomic bisulfite sequencing report, and the specific calculation method is represented as wherein, M count is the number of methylated sites, and NM count is the number of unmethylated sites.

[0196] In some embodiments, the data sets of all samples are merged and divided according to the division criteria to construct a training set (Training Set), a validation set (Validation Set) and a test set (Test Set), the training set being the first training data set and the second training data set above, and the validation set being the third training data set above.

[0197] The training set is a data set used for training a machine learning model. During training, the model uses the samples in the training set to learn features and patterns in order to make predictions or classifications.

[0198] The validation set is a data set used for adjusting model hyperparameters and evaluating model performance. It is a separate set of samples set aside during the model training process, which can be used to adjust the model's hyperparameters and to make a preliminary evaluation of the model's ability. It is often used to verify the current model's generalization ability (accuracy, recall rate, etc.) when the model is iteratively trained, to decide whether to stop training. In neural networks, the validation data set is used to find the optimal network depth (number of hidden layers), or to determine the stopping point of the backpropagation algorithm, or to select the number of hidden layer neurons in the neural network; cross-validation (Cross Validation) commonly used in ordinary machine learning is to further subdivide the training data set into different validation data sets to train the model.

[0199] The test set is a data set used to evaluate the performance of the trained model. After the model training is completed, the test set is used to verify the model's generalization ability for unknown data. The model's performance on the test set can help evaluate the model's accuracy and performance.

[0200] In step 504, the pre-trained prediction model is trained based on the data set for fine-tuning to obtain a trained prediction model.

[0201] For example, the model is fine-tuned using paired whole genome sequencing data and methylation status of CG sites, focusing on learning and detecting the location of CG sites in whole genome sequencing sequences, the sequencing depth of whole genome sequencing sequences, the fragmentation pattern of whole genome sequencing sequences, and other features affecting the methylation status. The loss function used in the training process can be cross-entropy loss or binary classification loss. During the model training process, the model implicitly learns the coverage information of whole genome sequencing sequences, CGN / NCG ratio information of CG sites, fragment length information of each sequencing, sequence information at both ends of the sequencing fragment, etc., and uses these information to predict the methylation status of the site.

[0202] In an example, the sequencing depth of a whole genome sequencing sequence refers to the average number of times (frequency) that each base is detected in sequencing during the whole genome sequencing process. The depth information is usually calculated based on the reads generated by sequencing and the alignment results of the corresponding positions on the reference genome. The total coverage of each site or region can be calculated by adding the number of reads on the site or region and then dividing by the number of base pairs of the site or region. For example, a gene has 100 base pairs, and a total of 1000 reads cover the region, then the coverage depth of the gene is 10 (1000 reads / 100 bases). The breakage pattern of the whole genome sequencing sequence is used to describe large fragment deletion, insertion, inversion or repeat structural variations in the genome.

[0203] For ease of understanding, the following is explained and described in combination with the structure of the prediction model. Referring to FIG. 6C, FIG. 6C is a second structural schematic diagram of a prediction model provided by an embodiment of the present application. The input of the prediction model is a multiple sequence alignment (MSA) sequence, and the output is predicted methylation state information. The complete model structure is divided into five parts. The multiple sequence alignment sequence can be provided by a unified sequence model 601C (USM). The prediction model includes a token embedding module 610C, a plurality of transformer blocks (for example: a first transformer block 620C and an Nth transformer block, N is a positive integer), and a prediction head 630C. The original sequence data (MSA / sequence) input is processed by the token embedding of the token embedding module 610C, and each token in each sequence is mapped to a high-dimensional vector space. The converted input data passes through N transformer blocks.

[0204] Taking the first transformer block 620C as an example to explain the structure of the transformer block. According to the data flow direction, the first transformer block 620C includes: a root mean square normalization layer 621C, a column pooling layer 622C, a column pooling layer 623C, a root mean square normalization layer 624C, a multi-head attention mechanism 625C, a root mean square normalization layer 626C, and a SwiGLU activation function 627C.

[0205] The root mean square normalization layer 621C is used for root mean square normalization (RMS Norm) processing of input data, calculating the average of the gradient square of each feature, and then using this average as a normalization scale factor. Root mean square normalization can better handle the difference between different dimensions when using residual connection, because it does not depend on a specific network architecture. The process of root mean square normalization includes: 1. Calculate the gradient square of each feature, 2. Calculate the average of all feature gradient squares, 3. Normalization processing. Normalization is a common data preprocessing technique used to convert data to a uniform scale range to eliminate the dimensional difference between different variables. Through normalization, data can be mapped to a specific range, such as [0, 1] or [-1, 1].

[0206] In the column pooling layer, taking the column pooling layer 622C as an example, the column pooling layer 622C includes a transposer 6221, a multi-head attention mechanism 6222, and a transposer 6223. The column pooling layer is used to reduce the dimensionality of data while retaining important information by aggregating information in each column.

[0207] Multi-head attention: calculates the relevance between different parts of input data. Multi-head attention includes linear transformation, attention mechanism, rotary position encoding (RoPE), normalization function softmax, and linear transformation. Continuing to refer to FIG. 6C, the multi-head attention mechanism 625C is used to perform linear transformation 6251, rotary position encoding 6252, rotary position encoding 6253, normalization function 6254, and linear transformation 6255, etc. processing. Among them, rotary position encoding (RoPE) is a method of encoding position information. Absolute position encoding assigns a unique vector to each position, although simple but cannot be well extended and cannot effectively capture the relative position; relative position encoding focuses on the distance between tokens, enhancing the model's understanding of token relationships, but complicating the model architecture. Rotary position encoding combines the advantages of both. The way to allow the model to understand the absolute position of the token and its relative distance encodes the position information. This is achieved through a rotation mechanism, where each position in the sequence is represented by a rotation in the embedding space. The advantage of rotary position encoding is its simplicity and efficiency, which allows the model to better grasp the subtle differences in language grammar and semantics.

[0208] The SwiGLU activation function 627C in the first converter block 620C is used to perform the following processing: linear conversion 6271, linear conversion 6272, activation function 6273, linear conversion 6274. The Sigmoid Gated Linear Unit (SwiGLU) activation function can replace the traditional Linear rectification function (ReLU), increasing the expression ability of the model.

[0209] In some embodiments, the converter block further includes a Residual Connection module: adding input data to transformed data to prevent gradient vanishing (gradient becoming very small) or explosion (gradient becoming very large). The basic idea of residual connection is to introduce an additional connection between each layer of the network, so that the output of the network can be directly added to the input without going through nonlinear transformation. In this way, even if the nonlinear transformation causes the gradient to distort during propagation, the input data itself can be directly passed to the next layer, thereby preserving the gradient information to some extent.

[0210] In some embodiments, the converter block further includes a Layer Norm layer: normalizing the transformed data. Layer normalization calculates the mean and variance of each feature in each sample, and then standardizes the feature values of each sample to a distribution with a mean of 0 and a standard deviation of 1. Layer normalization helps stabilize training, especially in deep networks using residual connections, as it can reduce internal covariate shift. Layer normalization is usually applied to all features of each sample, so it considers both rescaling and recentering invariance.

[0211] The prediction head 630C is used to further integrate global information through a global attention mechanism. The global attention mechanism enhances the semantic understanding ability of the model by calculating the relevance between all positions. The prediction head 630C is also used to perform the final classification or regression task. The prediction head contains multiple linear layers and activation functions such as SiLU and Linear to generate the final result. The entire training process optimizes the model parameters through the backpropagation algorithm, so that the performance of the model on the training data gradually improves.

[0212] Referring to FIG. 6D, FIG. 6D is a data flow diagram of a prediction model according to an embodiment of the present application. The multiple sequence alignment sequence has low information density in the column direction, and column attention mechanism, column pooling operation, and multi-head attention mechanism are sequentially performed to obtain the methylation state.

[0213] In some embodiments, when training the prediction model, multiple methylation state related tasks can be considered simultaneously, such as predicting sequence length, sequence breakpoint features, and sequencing depth, etc. information. Through sharing part of the structure and parameters of the model, multi-task learning can improve the generalization ability of the model on each task, thereby improving the accuracy of methylation state prediction.

[0214] With reference back to FIG. 5, in step 505, the trained prediction model is used to make predictions on the gene data to be detected, obtaining predicted methylation states.

[0215] For example, the trained model can take WGS sequencing data containing CG sites as input and output the methylation state prediction value of the site. The methylation state prediction value is usually a probability value representing the probability that the given site is methylated (methylated state) or unmethylated (unmethylated state). For example, a prediction model may output a probability value between 0 and 1, where 0 represents complete unmethylation and 1 represents complete methylation.

[0216] For example, in practical applications, the data input into the trained model can also be RNA sequencing data (RNA-seq), chromatin immunoprecipitation sequencing data (ChIP-seq), which is biological data obtained by chromatin immunoprecipitation (ChIP) technology combined with high-throughput sequencing methods. This technology is mainly used to study the interaction between proteins (transcription factors, histone modification enzymes, etc.) and DNA. By integrating multi-modal data, the prediction model can capture methylation information in the genome from different levels, thereby improving the accuracy and coverage of the prediction.

[0217] The embodiments of the present application can be applied in the following application scenarios:

[0218] 1. Prediction of methylation state of specific gene sites is an important research content of epigenetics and molecular biology. Accurate prediction of methylation state is beneficial to technical personnel to deeply understand the gene expression regulation mechanism, explore the epigenetic causes of diseases, and develop corresponding diagnosis and treatment strategies. Predicting the methylation state of disease-related gene sites can help scientists understand the pathogenesis of diseases, find biomarkers of diseases, and develop new treatment methods. In addition, the embodiments of the present application can also be extended to the prediction of other epigenetic markers, such as histone modification state. Histone methylation is one of the key mechanisms for regulating gene expression. Predicting the methylation state of specific gene sites can help scientists understand how genes are activated or silenced, and how these changes affect cell function.

[0219] 2. Drug development. In the process of drug development, predicting the methylation status of drug targets can help optimize drug design and improve drug efficacy.

[0220] 3. Tumor research. The prediction of methylation status has great significance in the research and clinical application of cancer, nervous system diseases and other diseases. It can not only be used for early screening and diagnosis of diseases, achieving the purpose of early intervention and improving cure rate, but also can help identify epigenetic markers of specific diseases, providing scientific basis for precision medicine and personalized treatment plan. In tumor research, predicting the methylation status of tumor-related genes can help identify potential driver genes and prognostic markers of tumors, providing guidance for tumor diagnosis and treatment.

[0221] The embodiments of the present application have the following advantages. First, no methylation-specific experiment is needed, only whole genome sequencing data is needed to predict the methylation status, which significantly reduces the complexity and cost of the experiment. By using genomics large model, the data processing process is simplified, and the accuracy and efficiency of prediction are improved. In addition, it can comprehensively capture the methylation status changes in the genome range, improve the accuracy and coverage of prediction, and is suitable for complex biological samples. It has important significance in the research and clinical application of cancer, nervous system diseases and other diseases, and promotes the development of early screening and precision medicine of diseases. At the same time, it has wide scalability, not limited to the prediction of methylation status, but also can be applied to the prediction of other epigenetic markers, with broad application prospect.

[0222] The following continues to illustrate an exemplary structure of the implementation of the training device 455 of the prediction model provided by the embodiments of the present application as a software module. In some embodiments, as shown in FIG. 2, the software module stored in the training device 455 of the prediction model in the memory 450 can include: a pre-training module 4551 configured to pre-train a first prediction model based on a plurality of first sample whole genome sequences to obtain a second prediction model; a sample acquisition module 4552 configured to extract a target dinucleotide site from a plurality of dinucleotide sites of each of the first sample whole genome sequences, and take the sequence within a preconfigured window range containing the target dinucleotide site as a second sample whole genome sequence; the sample acquisition module 4552 is configured to label the actual methylation state of the target dinucleotide site in each of the second sample whole genome sequences to obtain a labeled second sample whole genome sequence; the sample acquisition module 4552 is configured to combine each of the labeled second sample whole genome sequences into a second training data set; a fine-tuning training module 4553 configured to train the second prediction model based on each of the labeled second sample whole genome sequences to obtain a trained third prediction model, wherein the third prediction model is used to predict the methylation state of the dinucleotide site in the whole genome sequence to be detected.

[0223] In some embodiments, the pre-training module 4551 is configured to call the first prediction model to perform feature extraction processing on a plurality of first sample whole genome sequences to obtain a first feature vector of each of the first sample whole genome sequences; determine a first loss function of the first prediction model based on a plurality of the first feature vectors; perform iterative parameter update processing on the first prediction model based on the first loss function to obtain the second prediction model, wherein the second prediction model is used to extract a feature vector of the whole genome sequence to be detected.

[0224] In some embodiments, the pre-training module 4551 is configured to, for each of the first sample whole genome sequences, perform the following processing: determine a first similarity between a first feature vector of the first sample whole genome sequence and a first feature vector of a third sample whole genome sequence, and determine a second similarity between the first feature vector of the first sample whole genome sequence and a first feature vector of a fourth sample whole genome sequence, wherein the third sample whole genome sequence is another first sample whole genome sequence having a same base order as the first sample whole genome sequence, and the fourth sample whole genome sequence is another first sample whole genome sequence having a different base order as the first sample whole genome sequence; take a first ratio between the first similarity and a first sum as a first sub-loss function corresponding to the first sample whole genome sequence, wherein the first sum is a sum of the first similarity and the second similarity; and take a mean of a plurality of the first sub-loss functions as a first loss function of the first prediction model.

[0225] In some embodiments, the first sample whole genome sequence comprises a plurality of dinucleotide sites; and the sample obtaining module 4552 is configured to, for each of the first sample whole genome sequences, perform the following processing: extract a target dinucleotide site from the plurality of dinucleotide sites of the first sample whole genome sequence; and take a sequence within a preconfigured window range from the first sample whole genome sequence as the second sample whole genome sequence, with the target dinucleotide site as a reference site, wherein the reference site is located at any one of the following positions in the preconfigured window range: a boundary in the preconfigured window range, and an interior of the preconfigured window range.

[0226] In some embodiments, the fine-tuning training module 4553 is configured to invoke the second prediction model based on the second training dataset to perform the following processing: perform feature extraction processing on each of the second sample whole genome sequences of the second training dataset to obtain a second feature vector of each of the second sample whole genome sequences; determine, based on the second feature vector of each of the second sample whole genome sequences, a first prediction probability that a target dinucleotide site included in the second sample whole genome sequence is in a methylation state; determine, based on a label probability corresponding to an actual methylation state of each of the target dinucleotide sites and the first prediction probability, a second loss function of the second prediction model; and perform iteratively parameter update processing on the second prediction model based on the second loss function to obtain the third prediction model, wherein the third prediction model is used to determine a methylation state of a dinucleotide site in a whole genome sequence.

[0227] In some embodiments, the fine-tuning training module 4553 is configured to, for each of the second sample whole genome sequences, perform the following processing: performing a plurality of levels of feature conversion processing on the second feature vectors to obtain converted second feature vectors; calling an attention mechanism based on the converted second feature vectors to perform aggregation processing to obtain global features; and performing activation processing based on the global features of the second sample whole genome sequences to obtain the first prediction probability of the methylation state of the target dinucleotide site.

[0228] In some embodiments, the fine-tuning training module 4553 is configured to, for each of the target dinucleotide sites, determine a binary classification loss between the label probability corresponding to the actual methylation state and the first prediction probability; and take the mean of a plurality of the binary classification losses as a second loss function of the second prediction model.

[0229] In some embodiments, before the sample acquisition module 4552 pre-trains the first prediction model based on a plurality of first sample whole genome sequences to obtain the second prediction model, the sample acquisition module 4552 is configured to perform whole genome sequencing processing on a biological sample to obtain a plurality of real whole genome sequences; and take a preconfigured reference whole genome sequence and the plurality of real whole genome sequences as the first sample whole genome sequences.

[0230] In some embodiments, the first sample whole genome sequences include a plurality of dinucleotide sites; and the sample acquisition module 4552 is configured to, before extracting a target dinucleotide site from the plurality of dinucleotide sites of each of the first sample whole genome sequences and taking the sequence within a preconfigured window range containing the target dinucleotide site as a second sample whole genome sequence, perform bisulfite treatment on each of the first sample whole genome sequences to obtain a plurality of processed first sample whole genome sequences, wherein the processed first sample whole genome sequences include a plurality of processed dinucleotide sites; and determine the actual methylation state of each of the dinucleotide sites based on the dinucleotide sites before processing and the corresponding plurality of processed dinucleotide sites.

[0231] In some embodiments, the sample acquisition module 4552 is configured to, for each of the first sample whole genome sequences, align the pre-processed dinucleotide site with the plurality of post-processed dinucleotide sites to obtain a plurality of alignment results corresponding to each of the dinucleotide sites; determine that the methylation state of the dinucleotide site is unmethylated when the alignment result is that the cytosine of the dinucleotide site is converted to uracil or is not converted; determine that the methylation state of the dinucleotide site is methylated when the alignment result is that the cytosine of the dinucleotide site is converted to guanine; and determine the actual methylation state of each of the dinucleotide sites based on a second ratio between the number of methylations of the dinucleotide site and the total number of occurrences of the dinucleotide site, wherein the second ratio is the probability of methylation of the dinucleotide site.

[0232] In some embodiments, the fine-tuning training module 4553 is configured to, after the training of the second prediction model based on each of the labeled second sample whole genome sequences to obtain the trained third prediction model, acquire a third training data set, wherein the third training data set includes a plurality of third whole genome sequences and sample label probabilities of sample dinucleotide sites corresponding to the third whole genome sequences respectively; call the third prediction model based on the plurality of third whole genome sequences to perform prediction processing to obtain a second prediction probability of the sample dinucleotide sites in each of the third whole genome sequences being in a methylation state; determine a first difference between the sample label probability and the second prediction probability, and determine a first number of the first difference that is less than a preconfigured difference value; and in response to an accuracy of the third prediction model being less than a preconfigured accuracy, take the third prediction model as the second prediction model and enter the step of training the second prediction model based on each of the labeled second sample whole genome sequences to obtain the trained third prediction model, wherein the accuracy is a ratio between the first number and a total number of the sample dinucleotide sites.

[0233] In some embodiments, the fine-tuning training module 4553 is configured to, after the training of the second prediction model based on each of the labeled second sample whole genome sequences to obtain the trained third prediction model, call the third prediction model based on the whole genome sequence to be detected to perform the following processing: perform feature extraction processing on the whole genome sequence to be detected to obtain a fourth feature vector of the whole genome sequence to be detected; determine a third prediction probability of the second sample whole genome sequence including a to-be-detected dinucleotide site being in a methylation state based on the fourth feature vector; and in response to the third prediction probability being greater than a preconfigured probability, determine that the to-be-detected dinucleotide site is in a methylation state.

[0234] The embodiment of the present application provides a computer program product, which comprises computer executable instructions stored in a computer readable storage medium. A processor of an electronic device reads the computer executable instructions from the computer readable storage medium, and the processor executes the computer executable instructions, so that the electronic device executes the training method of the prediction model provided by the embodiment of the present application.

[0235] The embodiment of the present application provides a computer readable storage medium storing computer executable instructions, wherein the computer executable instructions are stored in the computer readable storage medium, and when the computer executable instructions are executed by a processor, the processor executes the training method of the prediction model provided by the embodiment of the present application, for example, the training method of the prediction model shown in FIG. 3A.

[0236] In some embodiments, the computer readable storage medium can be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disc, or CD-ROM; and can also be various devices including one or any combination of the above memories.

[0237] In some embodiments, the computer executable instructions can be in the form of programs, software, software modules, scripts or codes, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and can be deployed in any form, including being deployed as independent programs or being deployed as modules, components, subroutines or other units suitable for use in a computing environment.

[0238] As an example, the computer executable instructions can but not necessarily correspond to files in a file system, can be stored in part of a file storing other programs or data, for example, stored in one or more scripts in a hyper text markup language (HTML, Hyper Text Markup Language) document, stored in a single file dedicated to the program in question, or stored in multiple cooperative files (for example, files storing one or more modules, subroutines or code portions).

[0239] As an example, the computer executable instructions can be deployed to be executed on one electronic device, or executed on multiple electronic devices located in one place, or executed on multiple electronic devices distributed in multiple places and interconnected through a communication network.

[0240] In summary, by pre-training, the prediction model can learn the relevant features in the whole genome sequence, and the training stage uses the whole genome sequence in a specific range to train and improve the accuracy of the prediction model in predicting the methylation state. By training the prediction model of the methylation state, by limiting the range of training data, the data complexity is reduced, the training efficiency is improved, and the computing resources required in the training process are saved. By using the trained prediction model to detect the genome sequence to be detected, compared with the methylation state detection of the related art, it is not necessary to perform complex experiments, avoiding the noise and interference mixed in the experimental process, and improving the efficiency and accuracy of the methylation state detection.

[0241] The above is only an embodiment of the present application, and is not used to limit the protection scope of the present application. Any modification, equivalent replacement and improvement made within the spirit and scope of the present application are included in the protection scope of the present application.

Claims

1. A method for training a prediction model, the method being applied to an electronic device, the method comprising: pre-training a first prediction model based on a plurality of first sample whole genome sequences to obtain a second prediction model; extracting a target dinucleotide site from a plurality of dinucleotide sites of each of the first sample whole genome sequences, and taking a sequence within a preconfigured window range containing the target dinucleotide site as a second sample whole genome sequence; labeling an actual methylation state of the target dinucleotide site in each of the second sample whole genome sequences to obtain a labeled second sample whole genome sequence; training the second prediction model based on each of the labeled second sample whole genome sequences to obtain a third prediction model after training, wherein the third prediction model is used to predict a methylation state of a dinucleotide site in a to-be-detected whole genome sequence.

2. The method of claim 1, wherein, The pre-training of the first prediction model based on a plurality of first sample whole genome sequences to obtain a second prediction model comprises: calling the first prediction model to perform feature extraction processing on a plurality of first sample whole genome sequences to obtain a first feature vector of each of the first sample whole genome sequences; determining a first loss function of the first prediction model based on a plurality of the first feature vectors; performing iterative parameter update processing on the first prediction model based on the first loss function to obtain the second prediction model, wherein the second prediction model is used to extract a feature vector of the to-be-detected whole genome sequence.

3. The method of claim 2, wherein, The determination of the first loss function of the first prediction model based on a plurality of the first feature vectors comprises: for each of the first sample whole genome sequences, the following processing is performed: determining a first similarity between the first feature vector of the first sample whole genome sequence and a first feature vector of a third sample whole genome sequence, and determining a second similarity between the first feature vector of the first sample whole genome sequence and a first feature vector of a fourth sample whole genome sequence, wherein the third sample whole genome sequence is another first sample whole genome sequence having the same base order as the first sample whole genome sequence, and the fourth sample whole genome sequence is another first sample whole genome sequence having a different base order from the first sample whole genome sequence; taking a first ratio between the first similarity and a first sum as a first sub-loss function corresponding to the first sample whole genome sequence, wherein the first sum is a sum of the first similarity and the second similarity; taking a mean of a plurality of the first sub-loss functions as the first loss function of the first prediction model.

4. The method of any one of claims 1 to 3, wherein, The taking of the sequence within the preconfigured window range containing the target dinucleotide site as a second sample whole genome sequence comprises: Taking the target dinucleotide site as a reference site, sequences in a preconfigured window range are cut from the first sample whole genome sequence as the second sample whole genome sequence, wherein the reference site is located at any one of the following positions in the preconfigured window range: a boundary of the preconfigured window range, and an interior of the preconfigured window range.

5. The method of any one of claims 1 to 4, wherein, The training of the second prediction model based on each of the labeled second sample whole genome sequences comprises: The second prediction model is called to perform the following processing: The second sample whole genome sequence is subjected to feature extraction processing to obtain a second feature vector of the second sample whole genome sequence; Based on the second feature vector of each of the second sample whole genome sequences, a first prediction probability of a target dinucleotide site included in each of the second sample whole genome sequences in a methylation state is determined; Based on the label probability corresponding to the actual methylation state of each of the target dinucleotide sites and the first prediction probability, a second loss function of the second prediction model is determined; The second prediction model is iteratively updated in parameters based on the second loss function to obtain the third prediction model, wherein the third prediction model is used to determine the methylation state of a dinucleotide site in a whole genome sequence.

6. The method of claim 5, wherein, The determination of the first prediction probability of the target dinucleotide site included in each of the second sample whole genome sequences in a methylation state based on the second feature vector of each of the second sample whole genome sequences comprises: For each of the second sample whole genome sequences, the following processing is performed: The second feature vector is subjected to multiple levels of feature conversion processing to obtain a converted second feature vector; Based on the converted second feature vector, an attention mechanism is called to perform aggregation processing to obtain a global feature; Based on the global feature of the second sample whole genome sequence, activation processing is performed to obtain the first prediction probability of the target dinucleotide site in a methylation state.

7. The method of claim 5, wherein, The determination of the second loss function of the second prediction model based on the label probability corresponding to the actual methylation state of each of the target dinucleotide sites and the first prediction probability comprises: For each of the target dinucleotide sites, a binary classification loss between the label probability corresponding to the actual methylation state and the first prediction probability is determined; The average of a plurality of the binary classification losses is taken as the second loss function of the second prediction model.

8. The method according to any one of claims 1 to 7, wherein, Before the pre-training of the first prediction model based on a plurality of first sample whole genome sequences to obtain the second prediction model, the method further comprises: Whole genome sequencing processing is performed on biological samples to obtain a plurality of real whole genome sequences; A preconfigured reference whole genome sequence and the plurality of real whole genome sequences are taken as first sample whole genome sequences.

9. The method of claim 8, wherein, The whole genome sequencing processing on biological samples to obtain a plurality of real whole genome sequences comprises: Deoxyribonucleic acid is fragmented, and the fragmentation result is prepared into text data for sequencing; sequencing the text data to obtain a plurality of fragment sequencing reads; determining an arrangement order of bases in a genome sequence based on the plurality of fragment sequencing reads; reconstructing the genome sequence based on the arrangement order to obtain a plurality of the real whole genome sequences.

10. The method according to any one of claims 1 to 9, wherein, The method further comprises: bisulfite treating each of the first sample whole genome sequences to obtain a plurality of treated first sample whole genome sequences, wherein the treated first sample whole genome sequence comprises a plurality of treated dinucleotide sites; determining the actual methylation state of each of the dinucleotide sites based on the pre-treatment dinucleotide site and the corresponding plurality of treated dinucleotide sites.

11. The method of claim 10, wherein, The determining the actual methylation state of each of the dinucleotide sites based on the pre-treatment dinucleotide site and the corresponding plurality of treated dinucleotide sites comprises: aligning the pre-treatment dinucleotide site with the plurality of treated dinucleotide sites for each of the first sample whole genome sequences to obtain a plurality of alignment results corresponding to each of the dinucleotide sites; when the alignment result is that the cytosine of the dinucleotide site is converted into uracil or is not converted, determining that the methylation state of the dinucleotide site is unmethylated; when the alignment result is that the cytosine of the dinucleotide site is converted into guanine, determining that the methylation state of the dinucleotide site is methylated; for each of the dinucleotide sites, determining the actual methylation state of the dinucleotide site based on a second ratio between the number of methylations of the dinucleotide site and the total number of occurrences of the dinucleotide site, wherein the second ratio is the probability of methylation of the dinucleotide site.

12. The method according to any one of claims 1 to 11, wherein, After the training of the second prediction model based on each of the labeled second sample whole genome sequences to obtain the trained third prediction model, the method further comprises: obtaining a third training data set, wherein the third training data set comprises a plurality of third whole genome sequences and sample label probabilities of sample dinucleotide sites corresponding to the third whole genome sequences respectively; calling the third prediction model based on the plurality of third whole genome sequences to obtain a second prediction probability of the sample dinucleotide site in the methylation state in each of the third whole genome sequences; determining a first difference between the sample label probability and the second prediction probability, and determining a first number of the first difference less than a preconfigured difference value; in response to the accuracy of the third prediction model being less than a preconfigured accuracy, taking the third prediction model as the second prediction model and entering the step of training the second prediction model based on each of the labeled second sample whole genome sequences to obtain the trained third prediction model, wherein the accuracy is a ratio between the first number and a total number of the sample dinucleotide sites.

13. The method according to any one of claims 1 to 12, wherein, After the training of the second prediction model based on each of the labeled second sample whole genome sequences is performed, the method further includes: Based on the to-be-detected whole genome sequence, the third prediction model is called to perform the following processing: The to-be-detected whole genome sequence is subjected to feature extraction processing to obtain a fourth feature vector of the to-be-detected whole genome sequence; Based on the fourth feature vector, a third prediction probability of the to-be-detected dinucleotide site included in the second sample whole genome sequence being in a methylation state is determined; In response to the third prediction probability being greater than a preconfigured probability, it is determined that the to-be-detected dinucleotide site is in a methylation state.

14. A training device of a prediction model, the device comprising: a pre-training module configured to pre-train a first prediction model based on a plurality of first sample whole genome sequences to obtain a second prediction model; a sample acquisition module configured to extract a target dinucleotide site from a plurality of dinucleotide sites of each of the first sample whole genome sequences, and to take a sequence within a preconfigured window range containing the target dinucleotide site as a second sample whole genome sequence; the sample acquisition module is configured to label an actual methylation state of the target dinucleotide site in each of the second sample whole genome sequences to obtain a labeled second sample whole genome sequence; a fine-tuning training module configured to train the second prediction model based on each of the labeled second sample whole genome sequences to obtain a trained third prediction model, wherein the third prediction model is used to predict a methylation state of a dinucleotide site in a to-be-detected whole genome sequence.

15. An electronic device, comprising: a memory for storing computer executable instructions; a processor for executing the computer executable instructions stored in the memory to implement the training method of the prediction model according to any one of claims 1 to 13.

16. A computer readable storage medium storing computer executable instructions, which, when executed by a processor, implement the training method of the prediction model according to any one of claims 1 to 13.

17. A computer program product comprising computer executable instructions, which, when executed by a processor, implement the training method of the prediction model according to any one of claims 1 to 13.

Images