Gene model training method, gene sequence classification method, equipment and product

Abstract

The embodiment of the invention provides a gene model training method, a gene sequence classification method, equipment and a product, which are applied to the technical field of computers, and the model training method comprises the following steps: obtaining a training set which comprises gene sequence information, comparing the gene sequence information with a standard gene sequence, and comparing the gene sequence information with the standard gene sequence; obtaining abnormal sequence information in the gene sequence information, wherein the abnormal sequence information is at least used for representing an abnormal position and an abnormal type of an abnormal gene in the gene sequence information; and based on the training set, training a gene model to obtain a trained gene model. According to the method provided by the embodiment of the invention, the generalization ability of the gene model trained on the basis of the training set in individual data can be improved, and then the complexity of applying the trained gene model to related downstream tasks can be reduced.

Description

Technical Field

[0001] The present application belongs to the field of computer technology. Specifically, the present application relates to a gene model training method, a gene sequence classification method, a device and a product. Background Art

[0002] With the popularity of large language models in recent years, there are more and more biological language models. The existing large genetic language models are mainly deoxyribonucleic acid (DNA) language models. This type of model uses human reference genes or reference genomes of multiple species to pre-train pure sequence language models, and then applies them to multiple DNA sequence-related downstream tasks, for example, to tasks related to gene variation, disease auxiliary diagnosis, and so on.

[0003] Since this type of model only uses reference genes, the model has weak generalization ability on individual data and is difficult to apply to related downstream tasks. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a method for training a gene model, a method, device and product for classifying gene sequences that can improve the generalization ability of the model in individual data and can be better applied to related downstream tasks.

[0005] In order to achieve the above purpose, the technical solutions provided by the embodiments of the present application are as follows: In a first aspect, a method for training a gene model is provided, the method comprising: Acquire a training set, the training set including gene sequence information, compare the gene sequence information with a standard gene sequence, and obtain abnormal sequence information in the gene sequence information, wherein the abnormal sequence information is at least used to characterize an abnormal position and an abnormal type of an abnormal gene in the gene sequence information; Based on the training set, the gene model is trained to obtain a trained first model.

[0006] In a possible implementation, training the gene model based on the training set to obtain the trained gene model includes: For each abnormal gene at an abnormal position, determining a target encoding method corresponding to the abnormal type of the abnormal gene, and encoding the abnormal position in the abnormal sequence information or the abnormal position in the standard gene sequence using the target encoding method to obtain encoded sequence information; The gene model is trained based on the encoded sequence information to obtain the trained gene model.

[0007] In another possible implementation, the method further includes: determining a target number of abnormal bases in the abnormal gene at each abnormal position; For each abnormal position, according to the abnormal type of the abnormal gene at the abnormal position, the target encoding method corresponding to the abnormal position is determined, including: For each abnormal position, a target encoding mode is determined according to the target number of abnormal bases in the abnormal gene at the abnormal position and the abnormal type of the abnormal gene at the abnormal position.

[0008] In another possible implementation, for each abnormal position, determining the target encoding mode according to the target number of abnormal bases in the abnormal gene at the abnormal position and the abnormal type of the abnormal gene at the abnormal position includes: If the target number is greater than a preset threshold number, at the abnormal position, a preset character string corresponding to the abnormal type at the abnormal position is used to replace the base at the abnormal position; If the target quantity is not greater than the preset threshold quantity, if the abnormal type is the target type, the character string corresponding to the target type is inserted into the abnormal gene sequence in the abnormal sequence information or the standard gene sequence corresponding to the abnormal gene sequence; if the abnormal type is a non-target type, the abnormal gene sequence in the abnormal sequence information or the standard gene sequence corresponding to the abnormal gene sequence is used as the encoded sequence information.

[0009] In another possible implementation, the target type includes a base insertion type or a base deletion type; If the abnormal type is the target type, a character string corresponding to the target type is inserted into the abnormal gene sequence in the abnormal sequence information or the standard gene sequence corresponding to the abnormal gene sequence, including: If the target type is a base insertion type, a preset character string corresponding to the base insertion type is used to insert a standard gene sequence corresponding to the abnormal gene sequence; If the target type is a base deletion type, a preset character string corresponding to the base deletion type is used to insert into the abnormal gene sequence.

[0010] In another possible implementation, the training set also includes: chromosome position information and base position information corresponding to the gene sequence information, each position information in the chromosome sequence information represents an identifier of the chromosome where each base in the gene sequence is located, and each position information in the base position sequence represents an identifier of the position where each base in the gene sequence is located in the chromosome; For each base, obtain the position identifier of the base in the chromosome, including: Determine the actual position information of the base in the chromosome; Determining a target position parameter, wherein the target position parameter represents the total number of bases contained in the chromosome; Based on the actual position information of the base in the chromosome and the target position parameter, the position identifier of the base in the chromosome is determined.

[0011] In another possible implementation, the gene sequence information includes a plurality of abnormal gene sequence information and at least one standard gene sequence information, wherein the plurality of abnormal gene sequence information includes abnormal gene sequence information of at least one individual object and abnormal gene sequence information of at least one population, each of the abnormal gene sequence information includes a standard gene sequence and an abnormal gene sequence of the standard gene sequence, and each standard gene sequence information includes two standard gene sequences; Among them, the amount of standard gene sequence information is greater than the amount of abnormal gene sequence information of individuals, and the amount of abnormal gene sequence information of individuals is greater than the amount of abnormal gene sequence information of the population.

[0012] In another possible implementation, the training of the gene model based on the training set to obtain the trained gene model includes: Masking at least one base of the abnormal gene sequence in the gene sequence information, and masking at least one base in the standard gene sequence corresponding to the abnormal gene sequence; Pre-training the gene model based on the masked gene sequence information to obtain a pre-trained mask prediction model as the trained gene model; wherein the gene model is used to predict the base corresponding to the mask in the gene sequence information based on the input masked gene sequence information; Among them, the target training loss of the mask prediction model includes an abnormal sequence prediction loss and a standard sequence prediction loss, wherein the abnormal sequence prediction loss is determined based on at least one gene in the abnormal gene sequence after mask processing and the corresponding gene prediction result, and the standard sequence training loss is determined based on at least one gene in the standard gene sequence after mask processing and the corresponding gene prediction result.

[0013] In a second aspect, a method for classifying gene sequences is provided, comprising: Acquiring gene sequence information to be classified, wherein the gene sequence information to be classified includes: the gene sequence to be classified; Classifying the gene sequence information to be classified through a target classification model to obtain a classification result corresponding to the gene sequence to be classified; Wherein, the target classification model is obtained by fine-tuning the trained gene model; Wherein, the trained gene model is obtained by the gene model training method described in any possible implementation manner of the first aspect.

[0014] In a possible implementation, the gene sequence information to be classified further includes: a chromosome identification sequence corresponding to the gene sequence to be classified, a position identification sequence corresponding to the gene sequence to be classified, and an object identification sequence corresponding to the gene sequence to be classified; the chromosome identification sequence includes an identification of the chromosome corresponding to each base in the gene sequence to be classified, and the position identification sequence includes: an identification of the position of each base in the gene sequence to be classified in the chromosome; The step of classifying the gene sequence information to be classified by using a target classification model to obtain a classification result corresponding to the gene sequence to be classified includes: splicing the gene sequence to be classified, the chromosome identification sequence corresponding to the gene sequence to be classified, the position identification sequence corresponding to the gene sequence to be classified, and the object identification sequence corresponding to the gene sequence to be classified to obtain a spliced ​​gene sequence; The gene sequence after the splicing process is classified by the target classification model to obtain a classification result corresponding to the gene sequence to be classified.

[0015] In a possible implementation, the trained gene model includes: a trained feature extraction sub-model and a trained mask output sub-model; The target classification model is obtained by fine-tuning the trained gene model in the following manner: Acquire a plurality of training samples with classification labels; wherein the training samples include at least one abnormal gene sequence and a standard gene reference sequence of the abnormal gene sequence; Based on the multiple training samples, the gene sequence classification model is fine-tuned to obtain a target classification model, and the target classification model is used to classify the variant gene sequence based on the input variant gene sequence and the corresponding standard gene sequence; wherein the gene sequence classification model includes: a trained feature extraction sub-model and a classification sub-model.

[0016] In a third aspect, a model training device is provided, the device comprising: An acquisition module is used to acquire a training set, the training set includes gene sequence information, compare the gene sequence information with a standard gene sequence, and obtain abnormal sequence information in the gene sequence information, the abnormal sequence information is used to at least characterize the abnormal position and abnormal type of the abnormal gene in the gene sequence information; The training model is used to train the gene model based on the training set to obtain a trained gene model.

[0017] In a possible implementation, the abnormal type includes the abnormal type of the abnormal gene at each abnormal position; When the training module trains the gene model based on the training set to obtain the trained gene model, the training module is specifically used to: For each abnormal gene at an abnormal position, determining a target encoding method corresponding to the abnormal type of the abnormal gene, and encoding the abnormal position in the abnormal sequence information or the abnormal position in the standard gene sequence using the target encoding method to obtain encoded sequence information; The gene model is trained based on the encoded sequence information to obtain the trained gene model.

[0018] In another possible implementation, the device further includes: a determination module, wherein: The determination module is used to determine the target number of abnormal bases in the abnormal gene at each abnormal position; Wherein, when the training module determines the target encoding method corresponding to each abnormal position according to the abnormal type of the abnormal gene at the abnormal position, the training module is specifically used to: For each abnormal position, a target encoding mode is determined according to the target number of abnormal bases in the abnormal gene at the abnormal position and the abnormal type of the abnormal gene at the abnormal position.

[0019] Wherein, when the training module determines the target encoding mode for each abnormal position according to the target number of abnormal bases in the abnormal gene at the abnormal position and the abnormal type of the abnormal gene at the abnormal position, the training module is specifically used to: If the target number is greater than a preset threshold number, at the abnormal position, a preset character string corresponding to the abnormal type at the abnormal position is used to replace the base at the abnormal position; If the target quantity is not greater than the preset threshold quantity, if the abnormal type is the target type, the character string corresponding to the target type is inserted into the abnormal gene sequence in the abnormal sequence information or the standard gene sequence corresponding to the abnormal gene sequence; if the abnormal type is a non-target type, the abnormal gene sequence in the abnormal sequence information or the standard gene sequence corresponding to the abnormal gene sequence is used as the encoded sequence information.

[0020] In another possible implementation, the target type includes a base insertion type or a base deletion type; if the abnormal type is the target type, the training module uses a character string corresponding to the target type to be inserted into the abnormal gene sequence in the abnormal sequence information or the standard gene sequence corresponding to the abnormal gene sequence, specifically for: If the target type is a base insertion type, a preset character string corresponding to the base insertion type is used to insert a standard gene sequence corresponding to the abnormal gene sequence; If the target type is a base deletion type, a preset character string corresponding to the base deletion type is used to insert into the abnormal gene sequence.

[0021] In another possible implementation, the training set also includes: chromosome position information and base position information corresponding to the gene sequence information, each position information in the chromosome sequence information represents an identifier of the chromosome where each base in the gene sequence is located, and each position information in the base position sequence represents an identifier of the position where each base in the gene sequence is located in the chromosome; When the acquisition module acquires the position identifier of each base in the chromosome, the acquisition module is specifically used to: Determine the actual position information of the base in the chromosome; Determining a target position parameter, wherein the target position parameter represents the total number of bases contained in the chromosome; Based on the actual position information of the base in the chromosome and the target position parameter, the position identifier of the base in the chromosome is determined.

[0022] In another possible implementation, the gene sequence information includes a plurality of abnormal gene sequence information and at least one standard gene sequence information, wherein the plurality of abnormal gene sequence information includes abnormal gene sequence information of at least one individual object and abnormal gene sequence information of at least one population, each of the abnormal gene sequence information includes a standard gene sequence and an abnormal gene sequence of the standard gene sequence, and each standard gene sequence information includes two standard gene sequences; Among them, the amount of standard gene sequence information is greater than the amount of abnormal gene sequence information of individuals, and the amount of abnormal gene sequence information of individuals is greater than the amount of abnormal gene sequence information of the population.

[0023] In another possible implementation, when the training module trains the gene model based on the training set to obtain the trained gene model, it is specifically used to: Masking at least one base of the abnormal gene sequence in the gene sequence information, and masking at least one base in the standard gene sequence corresponding to the abnormal gene sequence; Pre-training the gene model based on the masked gene sequence information to obtain a pre-trained mask prediction model as the trained gene model; wherein the gene model is used to predict the base corresponding to the mask in the gene sequence information based on the input masked gene sequence information; Among them, the target training loss of the mask prediction model includes an abnormal sequence prediction loss and a standard sequence prediction loss. The abnormal sequence prediction loss is determined based on at least one base in the abnormal gene sequence after mask processing and the corresponding base prediction result, and the standard sequence training loss is determined based on at least one base in the standard gene sequence after mask processing and the corresponding base prediction result.

[0024] In a fourth aspect, a device for classifying gene sequences is provided, comprising: A gene sequence information acquisition module, used to acquire gene sequence information to be classified, wherein the gene sequence information to be classified includes: the gene sequence to be classified; A classification module, used to classify the gene sequence information to be classified through a target classification model to obtain a classification result corresponding to the gene sequence to be classified; Wherein, the target classification model is obtained by fine-tuning the trained gene model; Wherein, the trained gene model is obtained by the gene model training method described in any possible implementation manner of the third aspect.

[0025] In a possible implementation, the gene sequence information to be classified further includes: a chromosome identification sequence corresponding to the gene sequence to be classified, a position identification sequence corresponding to the gene sequence to be classified, and an object identification sequence corresponding to the gene sequence to be classified; the chromosome identification sequence includes an identification of the chromosome corresponding to each base in the gene sequence to be classified, and the position identification sequence includes: an identification of the position of each base in the gene sequence to be classified in the chromosome; Wherein, when the classification module classifies the gene sequence information to be classified through the target classification model to obtain the classification result corresponding to the gene sequence to be classified, it is specifically used to: splicing the gene sequence to be classified, the chromosome identification sequence corresponding to the gene sequence to be classified, the position identification sequence corresponding to the gene sequence to be classified, and the object identification sequence corresponding to the gene sequence to be classified to obtain a spliced ​​gene sequence; The gene sequence after the splicing process is classified by the target classification model to obtain a classification result corresponding to the gene sequence to be classified.

[0026] In a possible implementation, the trained gene model includes: a trained feature extraction sub-model and a trained mask output sub-model; The target classification model is obtained by fine-tuning the trained gene model in the following manner: Acquire a plurality of training samples with classification labels; wherein the training samples include at least one abnormal gene sequence and a standard gene reference sequence of the abnormal gene sequence; Based on the multiple training samples, the gene sequence classification model is fine-tuned to obtain a target classification model, and the target classification model is used to classify the variant gene sequence based on the input variant gene sequence and the corresponding standard gene sequence; wherein the gene sequence classification model includes: a trained feature extraction sub-model and a classification sub-model.

[0027] In a fifth aspect, an embodiment of the present application further provides an electronic device, the payment verification device comprising a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the genetic model training method provided by any possible implementation method of the first aspect.

[0028] In a sixth aspect, an embodiment of the present application further provides an electronic device, the chargeback request processing device comprising a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the method for gene sequence classification provided in any possible implementation manner of the second aspect.

[0029] In the seventh aspect, an embodiment of the present application further provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, it implements the training method of the genetic model provided by any possible implementation method of the first aspect.

[0030] In an eighth aspect, an embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the method for classifying gene sequences provided by any possible implementation of the second aspect is implemented.

[0031] In the ninth aspect, an embodiment of the present application further provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the training method of the genetic model provided by any possible implementation method of the first aspect.

[0032] In the tenth aspect, an embodiment of the present application further provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the method for gene sequence classification provided by any possible implementation method of the second aspect.

[0033] The beneficial effects of the technical solution provided by the embodiment of the present application are as follows: The embodiments of the present application provide a method, apparatus, device, medium and program product for training a gene model. In the embodiments of the present application, in the process of training the gene model, the training set used includes gene sequence information, the gene sequence information includes abnormal sequence information, and the abnormal sequence information is obtained by comparing the gene sequence information with the standard gene sequence. The abnormal sequence information is at least used to characterize the abnormal position and abnormal type of the abnormal gene in the gene sequence information; that is, in the embodiment of the present application, in addition to using the human standard gene sequence or the standard gene sequence of multiple species, the abnormal gene sequence and the abnormal information corresponding to the abnormal gene sequence that can characterize the abnormal position and abnormal type of the abnormal gene are also used in the process of training the gene model, thereby improving the generalization ability of the gene model trained based on the training set in individual data, and further reducing the complexity of applying the trained gene model to related downstream tasks. The embodiments of the present application provide a method, apparatus, device, medium and program product for gene sequence classification. In the embodiments of the present application, a gene sequence to be classified is obtained, and classified by a target classification model to obtain a classification result corresponding to the gene sequence to be classified, wherein the target classification model is obtained by fine-tuning a trained gene model, and the training set used by the trained gene model not only uses human standard gene sequences or standard gene sequences of multiple species, but also uses abnormal gene sequences and abnormal information corresponding to the abnormal gene sequences that can characterize the abnormal position and abnormal type of the abnormal gene, thereby improving the generalization ability of the gene model trained based on the training set in individual data, thereby reducing the complexity of applying the trained gene model to related downstream tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in describing the embodiments of the present application.

[0035] Figure 1 This is an example diagram of an exemplary implementation environment in the embodiments of the present application; Figure 2 A schematic diagram of a training process of a gene model in an embodiment of the present application; Figure 3 A schematic diagram of the training process of another gene model in an embodiment of the present application; Figure 4 A schematic diagram of a long variant encoding in an embodiment of the present application; Figure 5 A schematic diagram of a short variant encoding in an embodiment of the present application; Figure 6 This is an overall example diagram of a gene encoding in an embodiment of the present application; Figure 7 This is a structural diagram of a UGM basic model in an embodiment of the present application; Figure 8 This is a schematic diagram of the design structure of a UGM Block in an embodiment of the present application; Figure 9a This is a schematic diagram of a method for classifying gene sequences in an embodiment of the present application; Figure 9b A schematic diagram of a method for classifying gene sequences in an embodiment of the present application; Fig.10 This is a schematic diagram of the structure of a training device for a gene model in an embodiment of the present application; Fig.11 This is a schematic diagram of the structure of a device for classifying gene sequences in an embodiment of the present application; Fig.12 It is a schematic diagram of the device structure of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0036] The embodiments of the present application are described below in conjunction with the drawings in the present application. It should be understood that the implementation methods described below in conjunction with the drawings are exemplary descriptions for explaining the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions of the embodiments of the present application.

[0037] It will be understood by those skilled in the art that, unless specifically stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the terms "including" and "comprising" used in the embodiments of the present application refer to that the corresponding features can be implemented as the presented features, information, data, steps, operations, elements and / or components, but do not exclude the implementation as other features, information, data, steps, operations, elements, components and / or combinations thereof supported by the technical field. It should be understood that when we say that an element is "connected" or "coupled" to another element, the one element may be directly connected or coupled to the other element, or it may refer to that the one element and the other element establish a connection relationship through an intermediate element. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein indicates at least one of the items defined by the term, for example, "A and / or B" may be implemented as "A", or as "B", or as "A and B". When describing multiple (two or more) items, if the relationship between the multiple items is not clearly defined, the multiple items may refer to one, multiple or all of the multiple items. For example, the description of "parameter A includes A1, A2, A3" can be implemented as parameter A including A1 or A2 or A3, and can also be implemented as parameter A including at least two of the three items A1, A2, A3.

[0038] With the development of gene sequencing technology, people have a deeper understanding of gene variation, which is of great significance in modern medicine and biological research. It can help us understand the genetic characteristics of individuals, assist in predicting disease risks and responses to drugs, and thus provide a basis for precision medicine. The human reference genome is a standardized genome sequence. It constructs a consensus sequence by sequencing and comparing the genomes of multiple individuals. Individual genetic variation refers to the change of an individual's DNA fragment relative to the reference gene. It is crucial to learn the correlation between variation or genetic abnormalities and diseases from a large amount of individual sequencing data, so as to assist medical staff in related treatment.

[0039] The relevant large gene language models are mainly DNA language models, such as DNABERT, NT, DNAGPT, HyenaDNA and other models. These models use human reference genes or multi-species reference genomes to pre-train pure sequence language models, and then apply them to multiple DNA sequence-related downstream tasks. These models are mainly language model structures such as BERT and GPT. Model training only uses DNA sequences, and there is no position encoding for genes. In addition, since only reference genes are used, the model has weak generalization ability on individual data and cannot be applied to tasks related to gene variation and disease auxiliary diagnosis.

[0040] In the field of auxiliary disease identification, related methods mainly normalize the patient's clinical information and variant information into tables. Use table learning methods, such as traditional machine learning methods such as random forests and decision trees, to train disease identification models. This type of method cannot utilize massive amounts of variant data, requires more expert knowledge than large language models, and is difficult to generalize to auxiliary disease identification tasks.

[0041] Furthermore, related gene pre-training models, such as DNABERT and NT, only learn DNA sequences. Since the changes of individuals relative to the reference genome are sparse, simply learning DNA sequences is very inefficient and time-consuming. Gene mutation requires a mutant gene sequence (also called an abnormal gene sequence) and a standard gene sequence (also called a reference sequence). If only one sequence is input, the model lacks information about the gene location and cannot be applied to the task of gene sequence classification. Since human gene sequences are very long, existing models cannot input complete individual information, cannot make judgments on gene sequence classification at the individual level, and are less likely to be able to assist in the judgment of diseases related to gene sequences.

[0042] Related models that classify gene sequences to assist in disease classification use table learning methods to collect statistics on a large amount of variation information. The model cannot input fine-grained sequence features, making it difficult to learn the correlation between variation and disease at the sequence level, and requires a lot of expert knowledge, resulting in poor versatility of the model.

[0043] In the variation coding method, the existing methods have the following problems: 1) Encoding gene name, losing sequence information; 2) Encode the number of mutations, but lose information such as the location of the mutation.

[0044] Based on the above-mentioned related technologies and technical problems of related technologies, the embodiments of the present application can be used to solve the following problems: (1) Through a unified genetic model, we can learn the distribution of population variation and individual variation from a large amount of data (learn the relationship between variations in the same individual. Due to genetic linkage disequilibrium, variations are not independent of each other); (2) In view of the diversity of gene variation types, a universal encoding method is used to effectively encode short variation (less than 50 BP) and long variation (greater than 50 BP) types, including insertion, deletion, single base variation, copy number, duplication, inversion and other common variation types.

[0045] (3) Use large language models to learn the correlation between variation and disease, reduce expert-designed features, and improve the generalization and versatility of the model.

[0046] Furthermore, in order to solve the above technical problems, the embodiment of the present application aims to establish a unified human gene basic model to learn the differences between individual genes and human reference genes, and provide a basic model for downstream pan-gene disease auxiliary diagnosis tasks such as cancer, heart disease, and herpes erythematosus. The embodiment of the present application has the following technical key points: 1) Through unsupervised learning, a universal human genetic model UGM (Unified Genomics Model) is trained using a large amount of computing resources, human reference genome and gene variation data.

[0047] 2) Propose a unified gene type coding method to effectively encode common gene variation types such as non-variation, short variation, and structural variation.

[0048] 3) A unified gene position coding method is proposed to effectively encode chromosome, position, sample and other information, which is more universal and effective than the existing gene name coding method.

[0049] 4) A new genetic model pre-training method is proposed, including genetic background learning (Context Learning), germline variant distribution learning (Germline Variant Learning), and individual variant correlation learning (Variant Correlation Learning), which effectively improves the generalization of the basic model.

[0050] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0051] UGM: Unified Genomics Model, unified genetic model, for example, DNA sequence and DNA variation language model; BERT: Bidirectional Encoder Representation from Transformers, a bidirectional encoding transformer model that uses random mask input to train the model; SNP: Single Nucleotide Polymorphism, single nucleotide variation; HLA: Linkage disequilibrium refers to the probability that alleles belonging to two or more gene loci appear on the same chromosome at the same time, which is higher than the random frequency. LORA: Low-Rank Adaptation of Large Language Models, a model fine-tuning method; In order to solve the above technical problems, the present application embodiment provides a gene model training method and a gene sequence classification method, which can be applied to Figure 1 In the exemplary implementation environment shown, the implementation environment may include: a terminal 101 and a server 102, wherein data communication is performed between the terminal 101 and the server 102 via a communication network, optionally, the communication network may be a wired network or a wireless network, and the communication network may be at least one of a local area network, a metropolitan area network, and a wide area network. The terminal 101 may be a mobile terminal such as a smart phone, a tablet computer, a laptop computer, or a desktop computer, a projection computer, or the like. The terminal 101 may obtain the gene sequence information to be classified of a certain object. The gene sequence information to be classified includes: the gene sequence to be classified and the standard gene sequence corresponding to the gene sequence to be classified. The terminal 101 may be a device for performing gene sequencing to obtain the gene sequence information to be classified of the certain object, or may obtain the gene sequence information to be classified of the object from other devices. The server 102 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. In the embodiment of the present application, the server 102 is a backend server corresponding to the terminal 101. A target classification model may be set in the server 102. The terminal 101 may classify the gene sequence information to be classified through the target classification model to obtain a classification result corresponding to the gene sequence to be classified, and transmit the obtained classification result corresponding to the gene sequence to be classified to the terminal 101 for display, wherein the target classification model is obtained after fine-tuning the gene sequence classification model. Fine-tuning the gene sequence classification model may be performed in the server 102 or in other electronic devices (not shown in the figure), which is not limited in the embodiment of the present application; Furthermore, the gene sequence classification model includes a trained feature extraction sub-model and a classification sub-model, wherein the trained feature extraction sub-model can be located in a pre-trained mask prediction model (trained gene model), wherein the trained mask prediction model also includes: a mask output sub-model, wherein the pre-trained mask prediction model is obtained after pre-training the first model, wherein the device for pre-training the gene model and the device for fine-tuning the gene sequence classification model may be the same device or may not be the same device, and no limitation is made in the embodiments of the present application.

[0052] It needs to be explained that Figure 1 This is only an example of a possible application environment and is not intended to limit the application environment to which the embodiments of the present application are applicable.

[0053] Based on the above application environment, the present application embodiment introduces a gene model training method and a gene sequence classification method. Specifically, the present application embodiment provides a gene model training method, such as Figure 2 As shown, it is executed by an electronic device. In the embodiment of the present application, the electronic device may be a terminal or a server. For example, it may be the above-mentioned Figure 1 As shown in the server 102, the method may include: Step S201: Obtain a training set.

[0054] The training set includes gene sequence information, and the gene sequence information is compared with a reference sequence to obtain abnormal sequence information in the gene sequence information. In an embodiment of the present application, the abnormal sequence information may include: an abnormal gene sequence and abnormal information of the abnormal gene sequence. The abnormal gene sequence may be referred to as a variant gene sequence.

[0055] The abnormal sequence information is at least used to characterize the abnormal position and abnormal type of the abnormal gene in the gene sequence information. That is to say, the abnormal information of the abnormal gene sequence at least characterizes the abnormal position (also referred to as the variation position) and abnormal type (also referred to as the variation type) of the abnormal gene in the gene sequence information.

[0056] Specifically, the gene sequence information may be sequencing data obtained through second-generation or third-generation gene sequencing. The sequencing data is aligned to a standard gene sequence (also referred to as a reference gene sequence) through conventional secondary analysis processes, such as quality control and deduplication, and then processed by variant calling. The variant detection tool is not limited to common variant detection software such as GATK to obtain abnormal sequence information.

[0057] The standard gene sequence may be obtained from a gene sequence database, uploaded by the target object, or obtained by other means, which is not limited in the embodiments of the present application.

[0058] Step S202: training the gene model based on the training set to obtain a trained gene model.

[0059] Specifically, after obtaining the training set through the above embodiment, the gene model is trained to obtain a trained gene model. In the embodiment of the present application, the gene model can be an initial model or a model after a certain training, which is not limited in the embodiment of the present application.

[0060] An embodiment of the present application provides a method for training a gene model. In the embodiment of the present application, in the process of training the gene model, the training set used includes gene sequence information, the gene sequence information includes abnormal sequence information, and the abnormal sequence information is obtained by comparing the gene sequence information with the standard gene sequence. The abnormal sequence information is at least used to characterize the abnormal position and abnormal type of the abnormal gene in the gene sequence information; that is, in the embodiment of the present application, in addition to using the human standard gene sequence or the standard gene sequence of multiple species, the abnormal gene sequence and the abnormal information corresponding to the abnormal gene sequence that can characterize the abnormal position and abnormal type of the abnormal gene are also used in the process of training the gene model, thereby improving the generalization ability of the gene model trained based on the training set in individual data, and further reducing the complexity of applying the trained gene model to related downstream tasks. Specifically, in step S202, the gene model is trained based on the training set to obtain the trained gene model, which may include: step S2021 and step S2022. Figure 3 As shown, Step S2021: mask at least one base in the abnormal gene sequence in the gene sequence information, and mask at least one base in the standard gene sequence corresponding to the abnormal gene sequence.

[0061] In this embodiment of the present application, random masking is performed on the abnormal gene sequence and the standard gene sequence corresponding thereto, that is, at least one base (at least one base) in the abnormal gene sequence and at least one base (at least one base) corresponding thereto in the standard gene sequence can be randomly masked.

[0062] In one possible implementation, a random mask ratio may be set, such as 5%, that is, 15% of the bases are randomly masked (for example, 5% of the bases in the abnormal gene sequence are masked and / or 5% of the bases in the corresponding standard gene sequence are randomly masked), for example, the masked bases in the sequence are replaced with [mask] or "M".

[0063] In another possible implementation, each gene sequence (including standard gene sequences and abnormal gene sequences) is divided into multiple tokens according to K-mer, and the tokens are randomly masked to obtain the masked gene sequence; k-mer is understood as dividing a long sequence into many short sequences of length k. k is selected according to the application requirements and can be 3, 4, 5, etc. Based on the k-mer division into multiple tokens, each token contains k bases. When k is 3, the gene sequence is divided into 3 bases per token, that is, each token contains 3 bases. After the token division, the token pairs are randomly masked. Set a random mask ratio, for example, 5%, that is, 5% of the tokens are randomly masked. Three masking methods are used in the masking process: randomly replacing some tokens with masks according to the first ratio, randomly assigning some tokens with random values ​​according to the second ratio, and randomly keeping some tokens with original values ​​according to the third ratio, wherein the sum of the first ratio, the second ratio and the third ratio is 1. For example, if the first ratio is 80%, the second ratio and the third ratio are 10%, then 5% of the tokens are masked in three ways according to the ratios of 80%, 10%, and 10%. After masking, the masked gene sequence is obtained.

[0064] Furthermore, it should be noted that the method of performing masking on at least one base of the abnormal gene sequence in the gene sequence information and the method of performing masking on at least one base in the standard gene sequence corresponding to the abnormal gene sequence may also adopt other masking methods, which are not limited in the embodiments of the present application.

[0065] Step S2022: pre-train the gene model based on the masked gene sequence information to obtain a pre-trained mask prediction model as a trained gene model.

[0066] The gene model is used to predict the bases corresponding to the masks in the gene sequence information based on the input masked gene sequence information, that is, to predict the bases corresponding to the masks in the abnormal gene sequence and / or to predict the bases corresponding to the masks in the corresponding standard gene sequence.

[0067] Among them, the target training loss of the mask prediction model includes the abnormal sequence prediction loss and the standard sequence prediction loss. The abnormal sequence prediction loss is determined based on at least one base in the abnormal gene sequence after mask processing and the corresponding base prediction result, and the standard sequence training loss is determined based on at least one base in the standard gene sequence after mask processing and the corresponding base prediction result.

[0068] The gene model includes: a feature extraction sub-model and a mask output sub-model. Further, pre-training the gene model based on the masked gene sequence information to obtain the pre-trained gene model may include: pre-training the feature extraction sub-model and the mask output sub-model based on the masked gene sequence information to obtain the pre-trained mask prediction model (the trained feature extraction sub-model and the trained mask output sub-model).

[0069] Furthermore, in view of the diversity of gene variation types, a universal encoding method is used for effective encoding to improve the speed and accuracy of model training.

[0070] Furthermore, the abnormal type includes the abnormal type of the abnormal gene at each abnormal position; in the embodiment of the present application, the abnormal type in the variant gene sequence can be obtained through variant calling processing.

[0071] Among them, the abnormal type includes the abnormal type of each mutated base, including insertion, deletion, single base variation, copy number, duplication, inversion and other common variation types.

[0072] Specifically, in step S202, the gene model is trained based on the training set to obtain a trained gene model, including: for each abnormal gene at an abnormal position, a target encoding method corresponding to the abnormal type of the abnormal gene is determined, and the abnormal position in the abnormal sequence information or the abnormal position in the standard gene sequence is encoded using the target encoding method to obtain the encoded sequence information; based on the encoded sequence information, the gene model is trained to obtain the trained gene model.

[0073] For the embodiments of the present application, the abnormal gene sequence is encoded based on the abnormal type of the abnormal gene at each abnormal position, or the standard gene sequence (the standard gene sequence corresponding to the abnormal gene sequence mentioned above) is encoded based on the abnormal type of the abnormality occurring in the abnormal gene sequence, or the abnormal gene sequence and the standard gene sequence of the abnormal gene sequence are encoded based on the abnormal type of the abnormality occurring in the abnormal gene sequence.

[0074] Furthermore, based on the encoded sequence information (ie, the gene sequence after encoding processing by the above embodiment), the gene model is trained to obtain a trained gene model.

[0075] Furthermore, the method further includes: determining the target number of abnormal bases in the abnormal gene at each abnormal position. In the embodiment of the present application, after obtaining the abnormal type of the abnormal gene in the abnormal gene sequence, it may include: determining the target number of abnormal bases in the abnormal gene at each abnormal position (or, after determining the abnormal position of the abnormal gene, determining the target number of abnormal bases in the abnormal gene at each abnormal position). In another possible implementation, it is also possible to obtain the target number of abnormal bases in the abnormal gene at the abnormal position at the same time or after obtaining the abnormal type of the abnormal gene in the variant gene sequence, that is, the target number of bases that continuously mutate at the position.

[0076] Specifically, if the target number is greater than the preset threshold number, the mutation occurring at this abnormal position is called a long mutation; if the target number is not greater than the preset threshold number, the mutation occurring at this abnormal position is called a short mutation.

[0077] In a possible implementation, the target quantity can also be characterized by the length of the abnormal bases that occur continuously at the abnormal position, and the preset threshold quantity can also be called the preset length threshold. For example, the preset length threshold can be 50BP, that is, if the length of the abnormal bases that occur continuously at the abnormal position is less than 50BP, then the abnormality or the variation that occurs at the abnormal position here can be called a short variation, otherwise it is called a long variation. Among them, short variations include SNP, Insertion, Deletion and compound variation, and long variations also refer to structural variations, including Insertion, Deletion, Duplication, Inversion, CNV (Copy Number Variation), etc.

[0078] Among them, for an abnormal gene sequence, only short mutations may occur in the abnormal gene sequence, or only long mutations may occur, or both short mutations and long mutations may occur. Alternatively, for an abnormal gene sequence, at least two types of mutations (such as insertion and deletion) may occur for short mutations, or at least two types of mutations (such as deletion and CNV) may occur for long mutations, or at least one type of mutation may occur for long mutations, and at least one type of mutation may occur for short mutations. This situation may be called a compound mutation.

[0079] Specifically, for each abnormal position, determining the target encoding method corresponding to the abnormal position according to the abnormal type of the abnormal gene at the abnormal position may specifically include: for each abnormal position, determining the target encoding method according to the target number of abnormal bases in the abnormal gene at the abnormal position and the abnormal type of the abnormal gene at the abnormal position.

[0080] Specifically, for a mutation position (abnormal position), the corresponding encoding method is different when it is a long mutation or a short mutation.

[0081] Specifically, based on the abnormal type and target number of abnormalities occurring in the abnormal gene sequence, that is, for a gene abnormal sequence, the number of abnormal bases (target number) of abnormalities at an abnormal position is greater than the preset threshold number, that is, it belongs to a long variation, then based on the abnormal type of abnormality occurring in the abnormal position of the abnormal gene sequence, the first encoding method is used for encoding, and for the number of abnormal bases of abnormalities occurring at an abnormal position is not greater than the preset threshold number, the second encoding method is used for encoding based on the abnormal type of abnormality occurring in the abnormal position in the abnormal gene sequence or the standard gene sequence. In the embodiment of the present application, for both long variations and short variations, it can be determined whether to encode the abnormal gene sequence or the standard gene sequence according to the abnormal type.

[0082] Specifically, for each abnormal position, the target encoding method is determined according to the target number of abnormal bases in the abnormal gene at the abnormal position and the abnormal type of the abnormal gene at the abnormal position, including: if the target number is greater than the preset threshold number, at the abnormal position, a preset character string corresponding to the abnormal type at the abnormal position is used to replace the base at the abnormal position; if the target number is not greater than the preset threshold number, if the abnormal type is the target type, the character string corresponding to the target type is inserted into the abnormal gene sequence in the abnormal sequence information or the standard gene sequence corresponding to the abnormal gene sequence; if the abnormal type is a non-target type, the abnormal gene sequence in the abnormal sequence information or the standard gene sequence corresponding to the abnormal gene sequence is used as the encoded sequence information. In the embodiment of the present application, for long coding variations, different variation types may correspond to different first preset character strings; since the variation sequence (abnormal sequence) is too long, it can be encoded using the starting base and the ending base, that is, the starting base and the ending base are used to represent the mutated base (the suggestion of the abnormality) to relieve memory pressure, and ultra-long structural variations are generally fatal, and simplifying the expression of structural variations is also conducive to the model inputting more variation sites. Figure 4 As shown, taking Deletion as an example, DELS indicates the start of deletion, and DELE indicates the end of deletion; taking CNV as an example, CNVS indicates the start of copy number variation, and CNVE indicates the end of copy number variation; taking DUP as an example, DUPS indicates the start of duplication variation, and DUPE indicates the end of copy number variation.

[0083] Specifically, if the target number is not greater than the preset threshold number, that is, the abnormal position belongs to a short mutation, and the abnormal type of the abnormality in the abnormal gene sequence includes: at least one of the base insertion type and the base deletion type; at this time, for the mutation position of the base insertion type, the target position corresponding to the mutation position in the standard gene sequence is determined, and the second preset character string is inserted at the target position; for the mutation position of the base deletion type, a third preset character string is inserted at the mutation position in the mutant gene sequence.

[0084] Among them, for the abnormal position of the base insertion type, a second preset character string is inserted at the corresponding target position in the standard gene sequence at the abnormal position, and the second preset character string can be represented by at least one "-"; "-" indicates that the base is deleted, and at least one "-" is inserted in the standard gene sequence to represent the presence of the inserted base in the abnormal gene sequence.

[0085] Wherein, for the abnormal position of base deletion type, a third preset character string is inserted at the abnormal position in the abnormal gene sequence. At least one "-" is inserted at the abnormal position of the abnormal gene sequence, indicating that the base at this position is deleted.

[0086] Taking DNA sequence as an example, the abnormal types of abnormalities occurring in abnormal gene sequences may also include: base change. The basic bases include: A, C, G, T, N. For example, the base at a certain position may change from C to G, which is a base change.

[0087] like Figure 5 As shown, a base in the mutant gene sequence (abnormal gene sequence) changes from T to A, which is a single nucleotide variation. Three G-type bases are inserted in the mutant gene sequence. At this time, three "-" are inserted at the target position of the standard gene sequence, which is an insertion variation. Three bases, "C", "T" and "A", are deleted from the mutant gene sequence, and three "-" are inserted at the insertion position, which is a deletion variation. A base in the mutant gene sequence changes from G to C, and three bases, "C", "T" and "A", are deleted from the mutant gene sequence, which are compound variations (or mixed variations).

[0088] In our abnormal type coding (variant type coding), all gene mutations are converted into reference sequence (standard gene sequence) and variant sequence (variant gene sequence). This coding is complete and fine-grained. In this way, all mutations are at the single-base level. By combining the reference base and the variant base, we can obtain the base mutation type, such as "AA" means that the reference base is A, the variant base is A, and the site has not mutated. "A-" means that the reference base is A, the variant base is a deletion, and the site is deleted. Similarly, "AINSS" means that the reference base is A and the site is the starting position of the insertion.

[0089] Further, based on the above embodiment, that is, after the abnormal gene sequence or the standard gene sequence is encoded in the above manner, if the abnormal gene sequence is encoded, then in step S2021, at least one base of the encoded abnormal gene sequence is masked. Similarly, if the standard gene sequence is encoded, then in step S2021, at least one base of the encoded standard gene sequence is masked.

[0090] The specific method of performing mask processing can be found in the above embodiment and will not be described in detail here.

[0091] Furthermore, the model lacks information about gene positions and cannot be applied to downstream tasks (for example, gene sequence classification to assist in disease diagnosis). Therefore, the training set also includes: chromosome position information and base position information corresponding to the gene sequence information, each position information in the chromosome sequence information represents the identifier of the chromosome where each base in the gene sequence is located, and each position information in the base position sequence represents the position identifier of each base in the gene sequence in the chromosome; that is, the training set may at least include the variant gene sequence and the standard gene sequence of the variant gene sequence, and may also include the chromosome position sequence and the base position sequence, and further, may also include the corresponding sample identification sequence; The location of the mutation is determined by the chromosome number (chromosome position sequence) and the position on the chromosome (base position sequence). For example, Figure 6 As shown, there are 3 variants (or 3 abnormalities), which are divided into Non-Variant at position 10 on chromosome 1; Compound variant type (SNP and Deletion) at positions 20 to 23 on chromosome 2; Insertion variant at positions 80 to 81 on chromosome 3, with CAAAA fragment inserted. There are 24 types of position codes including chromosome ID.

[0092] In addition to conventional chromosome and base positions, in some downstream tasks, such as auxiliary identification of family genetic diseases, multiple gene variants such as father, mother, and child need to be input at the same time. In this case, a sample ID is required to distinguish the sequencing of different individuals.

[0093] Since the longest chromosome 1 can be up to 249.25 Mb, directly encoding the value will cause value overflow. In order to reduce the value overflow caused by encoding the value, for each base, the position identifier of the base in the chromosome is obtained, which can specifically include: determining the actual position information of the base in the chromosome; determining the target position parameter, the target position parameter represents the total number of bases contained in the chromosome; based on the actual position information of the base in the chromosome and the target position parameter, determining the position identifier of the base in the chromosome.

[0094] For example, the base position is divided by 100000 to obtain a floating point position as the position identifier of the base in the chromosome.

[0095] Further, based on the above embodiment, the first model can be a UGM basic model such as Figure 7 As shown in the figure, the model inputs multiple variant sites (or abnormal sites), and all abnormal types (variant types) are converted into reference sequence, variant sequence, chromosome id, position, sample id, and then spliced ​​into complete reference sequence, variant sequence, first base position sequence, second base position sequence and sample identification sequence. The model includes 5 types of embeddings. Reference embedding, variant embedding, chromosome embedding and sample embedding use the standard Embedding layer. Position embedding uses the Linear layer because continuous coordinates need to be input. Among them, UGM, as a basic model, can replace the common DNABERT for DNA sequence-related tasks.

[0096] UGM Core is composed of multiple UGM Blocks stacked together. The design of UGM Block is as follows: Figure 8As shown. The UGM model is not limited to UGM Block, and other similar Transformers can also be used. After the UGM Core extracts features, the LM Head is used to predict the base variation type. The LM Head consists of Root Mean Square Normalization (RMSNorm) and Linear layers.

[0097] The model is pre-trained using randomly masked reference sequences and variant sequences. The loss function of the model is the standard cross entropy loss.

[0098] In another possible implementation of the embodiment of the present application, the gene sequence information includes multiple abnormal gene sequence information and at least one standard gene sequence information, wherein the multiple abnormal gene sequence information includes abnormal gene sequence information of at least one individual object and abnormal gene sequence information of at least one population, each abnormal gene sequence information includes a standard gene sequence and an abnormal gene sequence of the standard gene sequence, and each standard gene sequence information includes two standard gene sequences; Among them, the amount of standard gene sequence information is greater than the amount of abnormal gene sequence information of individuals, and the amount of abnormal gene sequence information of individuals is greater than the amount of abnormal gene sequence information of the population.

[0099] Specifically, the training set includes three parts, which are used for Context Learning (at least one standard gene sequence information), Germline Variant Learning (at least one abnormal gene sequence information of a population), and Variant Correlation Learning (at least one abnormal gene sequence information of an individual object). The ratio of these three parts of the data set can be 5:3:2.

[0100] Among them, Context Learning uses the human reference genome to construct the Non-Variant type. Reference sequence is equivalent to the context of Variant sequence. In the process of learning the reference genome, the basic model can continuously strengthen its understanding of the context of the real variant type. Context Learning and existing model types such as DNABERT and NT learn from the reference genome. In our solution, gene position learning is also added. In the process of Context Learning, the sample id defaults to 0, indicating that all sequences are population reference sequences.

[0101] Among them, Germline Variant Learning uses dbSNP and dbVar datasets to construct short variant and long variant types. These two datasets are human population variation distribution datasets, and all variations are known human variations. Adding population distribution datasets during pre-training allows the model to learn possible variations (or possible anomalies) at different gene locations. During the training process, the sample id defaults to 0, indicating that it is a variation of the human population, and there is a distinction between individual variation and population variation.

[0102] Variant Correlation Learning uses the real variation of individuals. The distribution of variation in the same individual on the gene is not random, and there is a phenomenon of gene linkage disequilibrium. In addition, the distribution of gene variation (genetic abnormality) of individuals with specific genetic diseases has a certain regularity. In the pre-training process, using real individual variation data can enable the model to learn the correlation between the disease and the variation point and the correlation between the variation points.

[0103] The present application embodiment provides a method for classifying gene sequences, which is performed by an electronic device, such as Figure 9a As shown, the method may include: Step S901: Obtain gene sequence information to be classified.

[0104] The gene sequence information to be classified includes: the gene sequence to be classified.

[0105] Furthermore, the gene sequence information to be classified may also include: a standard gene sequence corresponding to the gene sequence to be classified.

[0106] Furthermore, the gene sequence information to be classified also includes: a chromosome position sequence corresponding to the gene sequence to be classified, a base position sequence corresponding to the gene sequence to be classified, and an identification sequence corresponding to the gene sequence to be classified, wherein the identification sequence corresponding to the gene sequence to be classified can be used to identify the object to which the gene sequence information to be classified belongs.

[0107] In a possible implementation, the gene sequence information to be classified can be obtained by inputting a patient's variation detection file (such as a VCF, MAF, etc. format). In one case, the gene sequence to be classified can also be a standard gene sequence.

[0108] Step S902: classify the gene sequence information to be classified through the target classification model to obtain a classification result corresponding to the gene sequence to be classified.

[0109] Specifically, in an embodiment of the present application, the gene sequence information to be classified is classified by a target classification model to obtain the probability of the classification result corresponding to the gene sequence to be classified. In an embodiment of the present application, the features output by the target classification model select the features of the first Token or the average of the features of all Tokens for classification. In an embodiment of the present application, the bases in the gene sequence to be classified can be divided into multiple tokens, for example, 3 bases as one token.

[0110] Among them, the target classification model is obtained by fine-tuning the trained gene model; The trained gene model is obtained by the gene model training method shown in the above embodiment. In the embodiment of the present application, the specific method of training the gene model to obtain the trained gene model is detailed in the above embodiment and will not be repeated here.

[0111] An embodiment of the present application provides a method for classifying gene sequences. In the embodiment of the present application, a gene sequence to be classified is obtained, and classified by a target classification model to obtain a classification result corresponding to the gene sequence to be classified, wherein the target classification model is obtained by fine-tuning a trained gene model, and the training set used by the trained gene model not only uses human standard gene sequences or standard gene sequences of multiple species, but also uses abnormal gene sequences and abnormal information corresponding to the abnormal gene sequences that can characterize the abnormal position and abnormal type of the abnormal gene, thereby improving the generalization ability of the gene model trained based on the training set in individual data, and further reducing the complexity of applying the trained gene model to related downstream tasks.

[0112] Furthermore, the trained gene model includes: a trained feature extraction sub-model and a trained mask output sub-model; in an embodiment of the present application, the target classification model is obtained by fine-tuning the trained gene model in the following manner: obtaining multiple training samples with classification labels; wherein the training samples include at least one abnormal gene sequence and a standard gene reference sequence of the abnormal gene sequence; based on the multiple training samples, the gene sequence classification model is fine-tuned to obtain the target classification model.

[0113] Among them, in the embodiment of the present application, the training samples used for fine-tuning can be a part of the samples trained in the above-mentioned pre-training, or can be completely different from the samples trained in the pre-training, which is not limited in the embodiment of the present application.

[0114] The target classification model is used to classify the variant gene sequence based on the input variant gene sequence and the corresponding standard gene sequence; wherein the gene sequence classification model includes: a trained feature extraction sub-model and a classification sub-model. That is, based on multiple training samples, the classification sub-model and the trained feature extraction sub-model are fine-tuned to obtain the target classification model. In the embodiment of the present application, the fine-tuning method is not limited to LORA, full-scale fine-tuning and other methods.

[0115] Among them, the target classification model is used to classify the variant gene sequence based on the input variant gene sequence and the corresponding standard gene sequence.

[0116] Furthermore, the gene sequence information to be classified also includes: a chromosome identification sequence corresponding to the gene sequence to be classified, a position identification sequence corresponding to the gene sequence to be classified, and an object identification sequence corresponding to the gene sequence to be classified; the chromosome identification sequence includes the identification of the chromosome corresponding to each base in the gene sequence to be classified, and the position identification sequence includes: the position identification of each base in the gene sequence to be classified in the chromosome.

[0117] Among them, in step S902, the gene sequence information to be classified is classified through the target classification model to obtain the classification result corresponding to the gene sequence to be classified, which may specifically include: splicing the gene sequence to be classified, the standard gene sequence corresponding to the gene sequence to be classified, the chromosome identification sequence corresponding to the gene sequence to be classified, the position identification sequence corresponding to the gene sequence to be classified, and the object identification sequence corresponding to the gene sequence to be classified to obtain the spliced ​​gene sequence; classifying the spliced ​​gene sequence through the target classification model to obtain the classification result corresponding to the gene sequence to be classified.

[0118] In order to better understand and illustrate the practical value of the solution provided by the embodiment of the present application, the optional implementation methods of the present application are described below in conjunction with specific scenario embodiments. The abnormal gene sequence in this application scenario is also the abnormal gene sequence shown above.

[0119] 1. Training process: 1. Obtain each gene comparison sequence sample; a gene comparison series sample includes a variant gene sequence and a corresponding reference gene sequence (or called a standard gene sequence); 2. The mutation type and mutation position of each mutant gene sequence can be detected by other equipment; the mutation type is the mutation type of each mutation site; the mutation position is the position of the mutation site in the gene sequence (which specific position on which chromosome); 3. Obtain the mutation type and mutation position of each mutant gene sequence; 4. Encoding based on the variation type of each variant gene sequence to obtain an encoded gene sequence group; wherein an encoded gene sequence group includes an encoded variant gene sequence and a corresponding encoded reference gene sequence; For example, taking DNA sequence as an example, the basic base types include: A, C, G, T, N, INSS, INSE, DELS, DELE, INVS, INVE, DUPS, DUPE, CNVS, CNVE, -, M, P. Among them, "-" means the base is deleted, "M" means Masking base, and "P" means Padding base. Long mutation coding: A longer gene sequence mutates, and this longer gene sequence is represented by four bases. The two middle bases represent the starting base and the ending base of the mutation respectively; the other two bases represent the base adjacent to the starting base and the base adjacent to the ending base respectively.

[0120] 5. Encode the variant position to obtain a position sequence group corresponding to the encoded variant gene sequence; Wherein, the position sequence group includes: a chromosome position sequence (a first base position sequence), and a sequence located at the chromosome position (a second base position sequence); Among them, the chromosome position sequence (first base position sequence) represents on which chromosome each base in the encoded variant gene sequence is located; The sequence at the chromosome position (the second base position sequence) characterizes the position of each base in the encoded variant gene sequence on the chromosome; 6. Obtaining the sample number sequence to which the first gene coding sequence belongs, so as to indicate to which sample the gene sequence belongs; For example, different people can be characterized, different locations where gene sequences are obtained, and so on; 7. Splicing the first gene coding sequence, the second gene coding sequence, the first position sequence, the second position sequence and the sample coding sequence of the same sample to obtain a spliced ​​sample sequence; to obtain each spliced ​​sample sequence; wherein the first gene coding sequence and the second gene coding sequence of the same sample may be the variant gene sequence coding result and the corresponding standard gene sequence coding result shown above, or may be two standard gene sequence results; 8. Pre-train each concatenated sample sequence through the initial UGM to obtain a pre-trained UGM model; Among them, the UGM model is composed of UGM core and LM head; Among them, the UGM core is composed of multiple UGM blocks connected in series.

[0121] The step of pre-training each concatenated sample sequence through an initial UGM to obtain a pre-trained UGM model includes: Performing mask processing on the first gene coding sequence and the second gene coding sequence, wherein after the mask processing, some bases (bases at the same position) in the first gene coding sequence and the second gene coding sequence are replaced with masks; splicing the masked first gene coding sequence, the masked second gene coding sequence, the first position sequence, the second position sequence and the sample coding sequence to obtain a masked spliced ​​sample sequence; The masked spliced ​​sample sequence is passed through the initial UGM model to output the predicted first base (the base on the first gene coding sequence) and the predicted second base (the base on the second gene coding sequence); Determine a first prediction loss by using the predicted first base (the base on the first gene coding sequence) and the original first base; Determine a second prediction loss by using the predicted second base (the base on the coding sequence of the second gene) and the original second base; The initial UGM model is trained based on the first prediction loss and the second prediction loss.

[0122] 9. The pre-trained UGM core and classification head are combined into a new model, and the new model is fine-tuned to obtain the target classification model; Specifically, the new model of the gene disease discrimination model pre-trained in 8 is further fine-tuned on specific disease data. The discrimination model is different from the basic model, replacing the mask classification with the variant gene sequence classification. The features output by the model, the features of the first token or the average of the features of all tokens are selected for classification.

[0123] 2. Online application process: Figure 9b The following is an example of the scenario shown in the figure. First, sequencing data is obtained through second-generation or third-generation gene sequencing. The sequencing data goes through conventional secondary analysis processes, such as quality control and deduplication, to align the sequencing sequence to the reference genome (also called the standard gene sequence). After alignment, it is processed by variant calling and the gene mutation site is obtained. The site is then input into the fine-tuned target classification model to assist doctors in disease diagnosis and serve as a reference for doctors to issue clinical reports.

[0124] 1. Obtain a gene sequence to be classified; the gene sequence information to be classified includes: the gene sequence to be classified and the standard gene sequence corresponding to the gene sequence to be classified, the chromosome identification sequence corresponding to the gene sequence to be classified, the position identification sequence corresponding to the gene sequence to be classified, and the object identification sequence corresponding to the gene sequence to be classified; the chromosome identification sequence includes the identification of the chromosome corresponding to each base in the gene sequence to be classified, and the position identification sequence includes: the position identification of each base in the gene sequence to be classified in the chromosome.

[0125] 2. Splicing the gene sequence to be classified, the standard gene sequence corresponding to the gene sequence to be classified, the chromosome identification sequence corresponding to the gene sequence to be classified, the position identification sequence corresponding to the gene sequence to be classified, and the object identification sequence corresponding to the gene sequence to be classified to obtain a spliced ​​gene sequence; 3. Classify the spliced ​​gene sequence through the target classification model to obtain the classification result corresponding to the gene sequence to be classified.

[0126] Furthermore, in a more specific example, the method shown in the embodiment of the present application can be applied in the auxiliary diagnosis process of certain specific diseases, such as cancer, heart disease, herpes erythematosus and other pan-genetic disease auxiliary diagnosis tasks, wherein, taking the gene sequence of the test object A as an example, it is used to assist in detecting whether the test object A may have cancer, heart disease, herpes erythematosus and other pan-genetic diseases; the specific method can be as follows: First, the sequencing data is obtained by second-generation or third-generation gene sequencing of the test object A. The sequencing data is compared to the reference genome (also called the standard gene sequence) through conventional secondary analysis processes, such as quality control and deduplication. After comparison, it is processed by variant calling. After the gene mutation site is obtained, the site is input into the fine-tuned target classification model to assist doctors in disease diagnosis and serve as a reference for doctors to issue clinical reports. Specifically, 1. Obtain the gene sequence of the object A to be tested; the gene sequence information of the object A to be tested includes: the gene sequence of the object A to be tested and the corresponding standard gene sequence, the chromosome identification sequence corresponding to the gene sequence of the object A to be tested, the position identification sequence corresponding to the gene sequence of the object A to be tested, and the object identification sequence corresponding to the gene sequence of the object A to be tested; the chromosome identification sequence includes the identification of the chromosome corresponding to each base in the gene sequence of the object A to be tested, and the position identification sequence includes: the position identification of each base in the gene sequence of the object A to be tested in the chromosome.

[0127] 2. Splicing the gene sequence of the test object A, the standard gene sequence (human standard gene sequence) corresponding to the gene sequence of the test object A, the chromosome identification sequence corresponding to the gene sequence of the test object A, the position identification sequence corresponding to the gene sequence of the test object A, and the object identification sequence corresponding to the gene sequence of the test object A to obtain a spliced ​​gene sequence; 3. Classify the spliced ​​gene sequence through the target classification model to obtain the classification result corresponding to the gene sequence to be classified, so that medical personnel can be assisted in determining whether the subject A to be tested may have pan-gene diseases such as cancer, heart disease, lupus erythematosus, etc. based on the classification result.

[0128] The present application embodiment provides a training device for a gene model, such as Fig.10 As shown, the gene model training device 100 may include: an acquisition module 1001, a training model 1002, wherein: An acquisition module 1001 is used to acquire a training set, the training set includes gene sequence information, compare the gene sequence information with a standard gene sequence, and obtain abnormal sequence information in the gene sequence information, the abnormal sequence information is used to at least characterize an abnormal position and an abnormal type of an abnormal gene in the gene sequence information; The training model 1002 is used to train the gene model based on the training set to obtain a trained gene model.

[0129] In a possible implementation, the abnormal type includes the abnormal type of the abnormal gene at each abnormal position; The training module 1002 is specifically used to train the gene model based on the training set to obtain the trained gene model: For each abnormal gene at an abnormal position, determine a target encoding method corresponding to the abnormal type of the abnormal gene, and use the target encoding method to encode the abnormal position in the abnormal sequence information or the abnormal position in the standard gene sequence to obtain encoded sequence information; Based on the encoded sequence information, the gene model is trained to obtain a trained gene model.

[0130] In another possible implementation, the apparatus 100 further includes: a determination module, wherein: a determination module, for determining a target number of abnormal bases in the abnormal gene at each abnormal position; The training module 1002 is specifically used to determine the target encoding method corresponding to each abnormal position according to the abnormal type of the abnormal gene at the abnormal position: For each abnormal position, a target encoding mode is determined according to the target number of abnormal bases in the abnormal gene at the abnormal position and the abnormal type of the abnormal gene at the abnormal position.

[0131] The training module 1002 is specifically used to determine the target encoding mode for each abnormal position according to the target number of abnormal bases in the abnormal gene at the abnormal position and the abnormal type of the abnormal gene at the abnormal position: If the target number is greater than the preset threshold number, at the abnormal position, a preset character string corresponding to the abnormal type at the abnormal position is used to replace the base at the abnormal position; If the target quantity is not greater than the preset threshold quantity, if the abnormal type is the target type, the character string corresponding to the target type is inserted into the abnormal gene sequence in the abnormal sequence information or the standard gene sequence corresponding to the abnormal gene sequence; if the abnormal type is a non-target type, the abnormal gene sequence in the abnormal sequence information or the standard gene sequence corresponding to the abnormal gene sequence is used as the encoded sequence information.

[0132] In another possible implementation, the target type includes a base insertion type or a base deletion type; if the abnormal type is the target type, the training module 1002 uses a character string corresponding to the target type to be inserted into the abnormal gene sequence in the abnormal sequence information or the standard gene sequence corresponding to the abnormal gene sequence, specifically for: If the target type is a base insertion type, a preset character string corresponding to the base insertion type is used to insert a standard gene sequence corresponding to the abnormal gene sequence; If the target type is a base deletion type, a preset character string corresponding to the base deletion type is used and inserted into the abnormal gene sequence.

[0133] In another possible implementation, the training set also includes: chromosome position information and base position information corresponding to the gene sequence information, each position information in the chromosome sequence information represents the identifier of the chromosome where each base in the gene sequence is located, and each position information in the base position sequence represents the position identifier of each base in the gene sequence in the chromosome; When obtaining the position identifier of each base in the chromosome, the acquisition module 1001 is specifically used to: Determine the actual position information of the base in the chromosome; determining a target position parameter, wherein the target position parameter represents the total number of bases contained in the chromosome; Based on the actual position information of the base in the chromosome and the target position parameter, the position identifier of the base in the chromosome is determined.

[0134] In another possible implementation, the gene sequence information includes a plurality of abnormal gene sequence information and at least one standard gene sequence information, wherein the plurality of abnormal gene sequence information includes abnormal gene sequence information of at least one individual object and abnormal gene sequence information of at least one population, each piece of abnormal gene sequence information includes a standard gene sequence and an abnormal gene sequence of the standard gene sequence, and each piece of standard gene sequence information includes two standard gene sequences; Among them, the amount of standard gene sequence information is greater than the amount of abnormal gene sequence information of individuals, and the amount of abnormal gene sequence information of individuals is greater than the amount of abnormal gene sequence information of the population.

[0135] In another possible implementation, when the training module 1002 trains the gene model based on the training set to obtain the trained gene model, it is specifically used to: Masking at least one base of the abnormal gene sequence in the gene sequence information, and masking at least one base in the standard gene sequence corresponding to the abnormal gene sequence; Pre-training the gene model based on the masked gene sequence information to obtain a pre-trained mask prediction model as a trained gene model; wherein the gene model is used to predict the base corresponding to the mask in the gene sequence information based on the input masked gene sequence information; Among them, the target training loss of the mask prediction model includes the abnormal sequence prediction loss and the standard sequence prediction loss. The abnormal sequence prediction loss is determined based on at least one base in the abnormal gene sequence after mask processing and the corresponding base prediction result, and the standard sequence training loss is determined based on at least one base in the standard gene sequence after mask processing and the corresponding base prediction result.

[0136] The present application embodiment provides a device for classifying gene sequences, such as Fig.11 As shown, the gene sequence classification device 110 includes: a gene sequence information acquisition module 1101 and a classification module 1102, wherein: The gene sequence information acquisition module 1101 is used to acquire the gene sequence information to be classified, and the gene sequence information to be classified includes: the gene sequence to be classified; The classification module 1102 is used to classify the gene sequence information to be classified through the target classification model to obtain the classification result corresponding to the gene sequence to be classified; Among them, the target classification model is obtained by fine-tuning the trained gene model; The trained gene model is obtained through a gene model training method, which is described in detail in the above embodiment and will not be described again here.

[0137] In a possible implementation, the gene sequence information to be classified further includes: a chromosome identification sequence corresponding to the gene sequence to be classified, a position identification sequence corresponding to the gene sequence to be classified, and an object identification sequence corresponding to the gene sequence to be classified; the chromosome identification sequence includes an identification of the chromosome corresponding to each base in the gene sequence to be classified, and the position identification sequence includes: an identification of the position of each base in the gene sequence to be classified in the chromosome; The classification module 1102 is specifically used to classify the gene sequence information to be classified through the target classification model to obtain the classification result corresponding to the gene sequence to be classified: splicing the gene sequence to be classified, the chromosome identification sequence corresponding to the gene sequence to be classified, the position identification sequence corresponding to the gene sequence to be classified, and the object identification sequence corresponding to the gene sequence to be classified to obtain a spliced ​​gene sequence; The spliced ​​gene sequence is classified through the target classification model to obtain the classification result corresponding to the gene sequence to be classified.

[0138] In a possible implementation, the trained gene model includes: a trained feature extraction sub-model and a trained mask output sub-model; The target classification model is obtained by fine-tuning the trained gene model in the following ways: Acquire multiple training samples with classification labels; wherein the training samples include at least one abnormal gene sequence and a standard gene reference sequence of the abnormal gene sequence; Based on multiple training samples, the gene sequence classification model is fine-tuned to obtain a target classification model, which is used to classify the variant gene sequence based on the input variant gene sequence and the corresponding standard gene sequence; wherein the gene sequence classification model includes: a trained feature extraction sub-model and a classification sub-model.

[0139] Fig.12 A schematic diagram of the structure of an electronic device applicable to an embodiment of the present application is shown. Fig.12 As shown, the electronic device can be a server or a user terminal, and the electronic device can be used to implement the method provided in any embodiment of the present application. That is, the electronic device can be used to implement a model training method provided in any embodiment of the present application; the electronic device can also be used to implement a gene sequence classification method provided in any embodiment of the present application; like Fig.12 As shown in FIG. , the electronic device 2000 may mainly include at least one processor 2001 ( Fig.12), memory 2002, communication module 2003 and input / output interface 2004 and other components. Optionally, the components can be connected and communicated through bus 2005. It should be noted that Fig.12 The structure of the electronic device 2000 shown in the figure is merely illustrative and does not constitute a limitation on the electronic device to which the method provided in the embodiment of the present application is applicable.

[0140] The memory 2002 may be used to store operating systems and applications, etc. The application may include a computer program that implements the method shown in the embodiment of the present application when called by the processor 2001, and may also include a program for implementing other functions or services. The memory 2002 may be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and computer programs, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0141] The processor 2001 is connected to the memory 2002 via the bus 2005, and implements corresponding functions by calling the application program stored in the memory 2002. Among them, the processor 2001 can be a CPU (Central Processing Unit), a general processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof, which can implement or execute various exemplary logic blocks, modules and circuits described in combination with the disclosure of this application. The processor 2001 can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0142] The electronic device 2000 can be connected to the network through the communication module 2003 (which may include but is not limited to components such as a network interface) to communicate with other devices (such as a user terminal or a server, etc.) through the network to achieve data interaction, such as sending data to other devices or receiving data from other devices. Among them, the communication module 2003 may include a wired network interface and / or a wireless network interface, etc., that is, the communication module may include at least one of a wired communication module or a wireless communication module.

[0143] The electronic device 2000 can be connected to the required input / output devices, such as a keyboard, a display device, etc., through the input / output interface 2004. The electronic device 2000 itself can have a display device, and can also be connected to other display devices through the interface 2004. Optionally, a storage device, such as a hard disk, can also be connected through the interface 2004, so that data in the electronic device 2000 can be stored in the storage device, or data in the storage device can be read, and data in the storage device can also be stored in the memory 2002. It can be understood that the input / output interface 2004 can be a wired interface or a wireless interface. Depending on the actual application scenario, the device connected to the input / output interface 2004 can be a component of the electronic device 2000, or it can be an external device connected to the electronic device 2000 when needed.

[0144] The bus 2005 for connecting the components may include a path for transmitting information between the above components. The bus 2005 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. According to different functions, the bus 2005 may be divided into an address bus, a data bus, a control bus, etc.

[0145] Optionally, for the solution provided in the embodiments of the present application, the memory 2002 can be used to store a computer program for executing the solution of the present application, and run by the processor 2001. When the processor 2001 runs the computer program, the actions of the method or device provided in the embodiments of the present application are implemented.

[0146] Based on the same principle as the method provided in the embodiment of the present application, the embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the corresponding content of the aforementioned method embodiment can be implemented.

[0147] An embodiment of the present application also provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the corresponding content of the aforementioned method embodiment can be implemented.

[0148] It should be noted that the terms "first", "second", "third", "fourth", "1", "2", etc. (if any) in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the application described herein can be implemented in an order other than that shown in the figure or described in the text.

[0149] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program that has a predetermined function and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories), or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0150] It should be understood that, although each operation step is indicated by arrows in the flowchart of the embodiment of the present application, the implementation order of these steps is not limited to the order indicated by the arrows. Unless clearly stated herein, in some implementation scenarios of the embodiment of the present application, the implementation steps in each flowchart can be performed in other orders according to demand. In addition, some or all of the steps in each flowchart may include multiple sub-steps or multiple stages based on actual implementation scenarios. Some or all of these sub-steps or stages may be executed at the same time, and each sub-step or stage in these sub-steps or stages may also be executed at different times respectively. In different scenarios of execution time, the execution order of these sub-steps or stages may be flexibly configured according to demand, and the embodiment of the present application does not limit this.

[0151] The above is only an optional implementation method for some implementation scenarios of the present application. It should be pointed out that for ordinary technicians in this technical field, without departing from the technical concept of the solution of the present application, other similar implementation methods based on the technical ideas of the present application are also within the protection scope of the embodiments of the present application.

Claims

1. A method for training a gene model, characterized in that: The method comprises: Acquire a training set, the training set including gene sequence information, compare the gene sequence information with a standard gene sequence, and obtain abnormal sequence information in the gene sequence information, wherein the abnormal sequence information is at least used to characterize an abnormal position and an abnormal type of an abnormal gene in the gene sequence information; Based on the training set, the gene model is trained to obtain a trained gene model.

2. The method according to claim 1, characterized in that The abnormal type includes the abnormal type of the abnormal gene at each abnormal position; the gene model is trained based on the training set to obtain the trained gene model, including: For each abnormal gene at an abnormal position, determining a target encoding method corresponding to the abnormal type of the abnormal gene, and encoding the abnormal position in the abnormal sequence information or the abnormal position in the standard gene sequence using the target encoding method to obtain encoded sequence information; The gene model is trained based on the encoded sequence information to obtain the trained gene model.

3. The method according to claim 2, characterized in that The method further comprises: determining a target number of abnormal bases in the abnormal gene at each abnormal position; For each abnormal position, according to the abnormal type of the abnormal gene at the abnormal position, the target encoding method corresponding to the abnormal position is determined, including: For each abnormal position, a target encoding mode is determined according to the target number of abnormal bases in the abnormal gene at the abnormal position and the abnormal type of the abnormal gene at the abnormal position.

4. The method according to claim 3, characterized in that For each abnormal position, determining a target encoding method according to a target number of abnormal bases in the abnormal gene at the abnormal position and an abnormal type of the abnormal gene at the abnormal position includes: If the target number is greater than a preset threshold number, at the abnormal position, a preset character string corresponding to the abnormal type at the abnormal position is used to replace the base at the abnormal position; If the target quantity is not greater than the preset threshold quantity, if the abnormal type is the target type, the character string corresponding to the target type is inserted into the abnormal gene sequence in the abnormal sequence information or the standard gene sequence corresponding to the abnormal gene sequence; if the abnormal type is a non-target type, the abnormal gene sequence in the abnormal sequence information or the standard gene sequence corresponding to the abnormal gene sequence is used as the encoded sequence information.

5. The method according to claim 4, characterized in that The target type includes a base insertion type or a base deletion type; If the abnormal type is the target type, a character string corresponding to the target type is inserted into the abnormal gene sequence in the abnormal sequence information or the standard gene sequence corresponding to the abnormal gene sequence, including: If the target type is a base insertion type, a preset character string corresponding to the base insertion type is used to insert a standard gene sequence corresponding to the abnormal gene sequence; If the target type is a base deletion type, a preset character string corresponding to the base deletion type is used to insert into the abnormal gene sequence.

6. The method according to claim 1, characterized in that The training set also includes: chromosome position information and base position information corresponding to the gene sequence information, each position information in the chromosome sequence information represents the identifier of the chromosome where each base in the gene sequence is located, and each position information in the base position sequence represents the position identifier of each base in the gene sequence in the chromosome; For each base, obtain the position identifier of the base in the chromosome, including: Determine the actual position information of the base in the chromosome; Determining a target position parameter, wherein the target position parameter represents the total number of bases contained in the chromosome; Based on the actual position information of the base in the chromosome and the target position parameter, the position identifier of the base in the chromosome is determined.

7. The method according to claim 1, characterized in that The gene sequence information includes a plurality of abnormal gene sequence information and at least one standard gene sequence information, wherein the plurality of abnormal gene sequence information includes abnormal gene sequence information of at least one individual subject and abnormal gene sequence information of at least one population, each of the abnormal gene sequence information includes a standard gene sequence and an abnormal gene sequence of the standard gene sequence, and each standard gene sequence information includes two standard gene sequences; Among them, the amount of standard gene sequence information is greater than the amount of abnormal gene sequence information of individuals, and the amount of abnormal gene sequence information of individuals is greater than the amount of abnormal gene sequence information of the population.

8. The method according to claim 1 or 2, characterized in that: The step of training the gene model based on the training set to obtain the trained gene model comprises: Masking at least one base of the abnormal gene sequence in the gene sequence information, and masking at least one base in the standard gene sequence corresponding to the abnormal gene sequence; Pre-training the gene model based on the masked gene sequence information to obtain a pre-trained mask prediction model as the trained gene model; wherein the gene model is used to predict the base corresponding to the mask in the gene sequence information based on the input masked gene sequence information; Among them, the target training loss of the mask prediction model includes an abnormal sequence prediction loss and a standard sequence prediction loss. The abnormal sequence prediction loss is determined based on at least one base in the abnormal gene sequence after mask processing and the corresponding base prediction result, and the standard sequence training loss is determined based on at least one base in the standard gene sequence after mask processing and the corresponding base prediction result.

9. A method for classifying gene sequences, characterized in that: include: Acquiring gene sequence information to be classified, wherein the gene sequence information to be classified includes the gene sequence to be classified; Classifying the gene sequence information to be classified through a target classification model to obtain a classification result corresponding to the gene sequence to be classified; Wherein, the target classification model is obtained by fine-tuning the trained gene model; Wherein, the trained gene model is obtained by the gene model training method according to any one of claims 1-8.

10. The method according to claim 9, characterized in that The gene sequence information to be classified also includes: a chromosome identification sequence corresponding to the gene sequence to be classified, a position identification sequence corresponding to the gene sequence to be classified, and an object identification sequence corresponding to the gene sequence to be classified; the chromosome identification sequence includes the identification of the chromosome corresponding to each base in the gene sequence to be classified, and the position identification sequence includes: the position identification of each base in the gene sequence to be classified in the chromosome; The step of classifying the gene sequence information to be classified by using a target classification model to obtain a classification result corresponding to the gene sequence to be classified includes: splicing the gene sequence to be classified, the chromosome identification sequence corresponding to the gene sequence to be classified, the position identification sequence corresponding to the gene sequence to be classified, and the object identification sequence corresponding to the gene sequence to be classified to obtain a spliced ​​gene sequence; The gene sequence after the splicing process is classified by the target classification model to obtain a classification result corresponding to the gene sequence to be classified.

11. The method according to claim 9, characterized in that The trained gene model includes: a trained feature extraction sub-model and a trained mask output sub-model; The target classification model is obtained by fine-tuning the trained gene model in the following manner: Acquire a plurality of training samples with classification labels; wherein the training samples include at least one abnormal gene sequence and a standard gene reference sequence of the abnormal gene sequence; Based on the multiple training samples, the gene sequence classification model is fine-tuned to obtain a target classification model, and the target classification model is used to classify the variant gene sequence based on the input variant gene sequence and the corresponding standard gene sequence; wherein the gene sequence classification model includes: a trained feature extraction sub-model and a classification sub-model.

12. A gene model training device, characterized in that: The device comprises: An acquisition module is used to acquire a training set, the training set includes gene sequence information, compare the gene sequence information with a standard gene sequence, and obtain abnormal sequence information in the gene sequence information, the abnormal sequence information is used to at least characterize the abnormal position and abnormal type of the abnormal gene in the gene sequence information; The training model is used to train the gene model based on the training set to obtain a trained gene model.

13. A device for classifying gene sequences, characterized in that: include: A gene sequence information acquisition module, used to acquire gene sequence information to be classified, wherein the gene sequence information to be classified includes the gene sequence to be classified; A classification module, used to classify the gene sequence information to be classified through a target classification model to obtain a classification result corresponding to the gene sequence to be classified; Wherein, the target classification model is obtained by fine-tuning the trained gene model; Wherein, the trained gene model is obtained by the gene model training device described in claim 12.

14. An electronic device, characterized in that: The payment verification device comprises a memory and a processor, wherein a computer program is stored in the memory, and the processor executes the method according to any one of claims 1 to 11 when running the computer program.

15. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 11 is implemented.

16. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 11 is implemented.

Images