Genome analsis apparatus, and generating method for training data

Abstract

The present invention relates to a method for generating training data necessary for constructing a transfer learning-based general-purpose gene language model.

Description

Technology Field

[0001] The present invention relates to a method for generating training data necessary for constructing a large general-purpose genomic language model based on transfer learning. Background Technology

[0002] With the recent advancement of Next Generation Sequencing (NGS) technology, the volume of accumulating genomic data is increasing exponentially. In particular, the emergence of single-cell sequencing technology has enabled precise genomic analysis at the cellular level, making personalized precision medicine a reality.

[0003] In this regard, personal genome analysis aims to identify genes associated with disease or disease induction, discover drug target proteins, and predict drug sensitivity for each patient.

[0004] However, conventional genomic analysis relies on traditional statistical analysis or gene interaction network analysis because the application of artificial intelligence technology, which requires large amounts of refined training data, is difficult due to the insufficient number of genomic data samples per patient group, resulting in limitations in prediction accuracy.

[0005] Meanwhile, in this regard, recent self-supervised transfer learning technology based on general-purpose language models is enabling the construction of high-accuracy artificial intelligence models specialized for specific tasks in the fields of text, image, and speech, using only a small amount of training data.

[0006] Accordingly, the present invention proposes a method for generating training data (input sentence data) that can effectively construct a transfer learning-based large general-purpose genomic language model. The problem to be solved

[0007] The present invention was created in consideration of the above-mentioned circumstances, and the objective of the present invention is to generate training data necessary for constructing a transfer learning-based general-purpose gene language model using accumulated large-scale genomic data. means of solving the problem

[0008] A genome analysis device according to one embodiment of the present invention for achieving the above objective is characterized by comprising: a preprocessing unit that maps genome data of a target disease to a gene interaction network; and an encoding unit that encodes genetic characteristic information read from the gene interaction network into a sentence and outputs the encoded sentence as training data required to build a general-purpose gene language model.

[0009] Specifically, the gene interaction network may include gene nodes that branch into a hierarchical structure starting from a node of a ROOT gene according to gene information identified from the genomic data, and edges that interconnect gene nodes between adjacent layers.

[0010] Specifically, the training data includes a token embedding in which the gene ID of each gene node serves as a token, and the encoding unit can encode the token embedding by listing the gene IDs of the gene nodes in each hierarchy according to a depth-first search order starting from the node of the ROOT gene.

[0011] Specifically, the training data further includes an expression pattern embedding that encodes the RNA expression state of each gene node according to the order listed in the token embedding, and the encoding unit can encode the expression pattern embedding based on the RNA relative expression level read from the relative size of each gene node on the gene interaction network.

[0012] Specifically, the training data further includes genetic variation embeddings that encode the genetic variation state of each gene node according to the order listed in the token embeddings, and the encoding unit can encode the genetic variation embeddings based on whether there is a genetic variation and the type of variation read from the color of each gene node on the gene interaction network.

[0013] Specifically, the training data further includes a branch state embedding that encodes the branch state of each gene node according to the order listed in the token embedding, and the encoding unit can encode the branch state embedding based on the number of edges each gene node has for connection with a lower layer on the gene interaction network.

[0014] A method for generating training data performed in a genome analysis device according to an embodiment of the present invention for achieving the above objective comprises: a preprocessing step of mapping genome data of a target disease to a gene interaction network; and an encoding step of encoding genetic characteristic information read from the gene interaction network into a sentence and outputting the encoded sentence as training data required to build a general-purpose gene language model.

[0015] Specifically, the gene interaction network may include gene nodes that branch into a hierarchical structure starting from a node of a ROOT gene according to gene information identified from the genomic data, and edges that interconnect gene nodes between adjacent layers.

[0016] Specifically, the above preprocessing step can map RNA expression information of genes using the relative size of each gene node on the gene interaction network.

[0017] Specifically, the above preprocessing step can map genetic variation information of genes using the color of each gene node on the gene interaction network. Effects of the invention

[0018] Accordingly, in the genome analysis device and training data generation method of the present invention, training data necessary for constructing a transfer learning-based general-purpose gene language model is generated using accumulated large-scale genome data, thereby enabling the construction of large-scale general-purpose genome language models of various structures, and a high-accuracy final artificial intelligence model for various genome analysis tasks, such as identifying disease-causing genes, discovering drug target genes, and predicting drug sensitivity, can be constructed by transfer learning a pre-trained language model using only a small number of patient sample genome data. Brief explanation of the drawing

[0019] FIG. 1 is an illustrative diagram for explaining a genome analysis environment according to one embodiment of the present invention. FIG. 2 is a configuration diagram of a genome analysis device according to one embodiment of the present invention. FIG. 3 is an illustrative diagram for explaining a gene interaction network according to an embodiment of the present invention. FIG. 4 is an illustrative diagram for explaining training data (embedding) according to an embodiment of the present invention. FIG. 5 is an illustrative diagram for explaining a gene vocabulary dictionary according to one embodiment of the present invention. FIG. 6 is a flowchart illustrating a method for generating training data according to an embodiment of the present invention. Specific details for implementing the invention

[0020] Hereinafter, preferred embodiments of the present invention will be described with reference to the attached drawings.

[0021] In one embodiment of the present invention, a technology regarding language model-based genomic analysis is described.

[0022] With the recent advancement of Next Generation Sequencing (NGS) technology, the volume of accumulating genomic data is increasing exponentially. In particular, the emergence of single-cell sequencing technology has enabled precise genomic analysis at the cellular level, making personalized precision medicine a reality.

[0023] In this regard, the purpose of personal genome analysis is to identify genes associated with diseases or disease development, discover drug target proteins, and predict drug sensitivity for each patient.

[0024] However, conventional genomic analysis relies on traditional statistical analysis or gene interaction network analysis because the application of artificial intelligence technology, which requires large amounts of refined training data, is difficult due to the insufficient number of genomic data samples per patient group, resulting in limitations in prediction accuracy.

[0025] Meanwhile, in this regard, recent self-supervised transfer learning technology based on general-purpose language models is enabling the construction of high-accuracy artificial intelligence models specialized for specific tasks in the fields of text, image, and speech, using only a small amount of training data.

[0026] Accordingly, in one embodiment of the present invention, a new method is proposed to generate training data necessary for constructing a transfer learning-based general-purpose gene language model using accumulated large-scale genomic data.

[0027] In this regard, FIG. 1 illustrates an exemplary genome analysis environment according to one embodiment of the present invention.

[0028] As illustrated in FIG. 1, a genome analysis environment according to one embodiment of the present invention includes a genome analysis device (100) that generates training data necessary to build a general-purpose gene language model.

[0029] The genome analysis device (100) can output training data necessary to build a universal gene language model through the analysis of accumulated genome data for a target disease in the form of input sentence data.

[0030] Such a genome analysis device (100) can be implemented in the form of a computing device (e.g., PC, smartphone) or a server on which software (e.g., an application) is installed.

[0031] If the genome analysis device (100) is implemented in the form of a server, for example, it may be implemented in the form of a web server, a database server, a proxy server, etc., and one or more of various software that enables a network load balancing mechanism or service device to operate on the Internet or another network may be installed, and thus it may also be implemented as a computerized system.

[0032] In the above, in a genome analysis environment according to one embodiment of the present invention, learning data capable of effectively constructing a general-purpose gene language model based on transfer learning can be generated through the aforementioned configuration. Below, the configuration of a genome analysis device (100) for realizing this will be described in more detail.

[0033] FIG. 2 schematically shows the configuration of a genome analysis device (100) according to one embodiment of the present invention.

[0034] As illustrated in FIG. 2, a genome analysis device (100) according to one embodiment of the present invention may have a configuration including a preprocessing unit (110) and an encoding unit (120).

[0035] The entire configuration or at least some configuration of the genome analysis device (100) including the above preprocessing unit (110) and encoding unit (120) may be implemented in the form of a hardware module or a software module, or in a combined form of a hardware module and a software module.

[0036] Here, a software module can be understood as, for example, an instruction executed by a processor that controls operations within a genome analysis device (100), and such instruction may have a form loaded in memory within the genome analysis device (100).

[0037] Here, a software module can be understood as an instruction executed by a processor that controls operations within the genome analysis device (100), such as an API (Application Programming Interface), and such instruction may have a form loaded in memory within the genome analysis device (100).

[0038] Meanwhile, the genome analysis device (100) according to one embodiment of the present invention can be implemented in the form of an API (Application Programming Interface) by adopting the configuration of the software module as described above.

[0039] The genome analysis device (100) according to one embodiment of the present invention can generate learning data necessary for constructing a transfer learning-based general-purpose gene language model through the aforementioned configuration. Below, a more detailed explanation of the configuration within the genome analysis device (100) for realizing this will be provided.

[0040] The preprocessing unit (110) is responsible for the function of mapping genomic data.

[0041] More specifically, the preprocessing unit (110) maps the large volume of genomic data accumulated for the target disease to a gene interaction network to form an initial training data set.

[0042] Here, the genomic data can identify not only information regarding protein-protein interactions (PPI) but also genetic information such as genetic variation information and RNA expression information for each gene.

[0043] In this regard, the preprocessing unit (110) can construct a gene interaction network with gene nodes (① ~ ⑩) branching into a hierarchical structure starting from the node (①) of the ROOT gene according to gene information identified from the genome data, for example as in FIG. 3, and edges (E: Edge) connecting gene nodes between adjacent layers.

[0044] In particular, the preprocessing unit (110) maps RNA expression information of genes using the relative size (size of a circle) of each gene node on the gene interaction network, and can also map genetic variation information of genes using the color of each gene node.

[0045] Here, RNA expression information expressed by the relative size (size of the circle) of each gene node represents the RNA expression amount of each gene, and genetic variation information expressed by the color of each gene node can be understood as the presence of genetic variation and the type of variation.

[0046] The encoding unit (120) is responsible for the function of outputting training data based on a general-purpose genetic language model.

[0047] More specifically, when an initial learning data set is configured in which genomic data is mapped to a gene interaction network, the encoding unit (120) outputs learning data that reflects genetic characteristic information read from the gene interaction network.

[0048] To this end, the encoding unit (120) can encode genetic characteristic information read from the gene interaction network into a sentence and output the encoded sentence as training data required to build a general-purpose gene language model.

[0049] As such, in one embodiment of the present invention, the training data output for constructing a general-purpose gene language model may include, for example, a token embedding in which the gene ID of each gene node is used as a token, as shown in FIG. 4 (a).

[0050] In this regard, the encoding unit (120) can encode token embeddings by listing the gene IDs of the gene nodes in each hierarchy according to a depth-first search order starting from the node of the ROOT gene.

[0051] Here, the gene ID of each gene node may refer to a gene vocabulary that pre-assigns a unique ID to each gene, such as in Fig. 5.

[0052] In addition, in the training data according to one embodiment of the present invention, expression pattern embeddings encoding the RNA expression state of each gene node according to the order listed in the aforementioned token embeddings, as shown in (b) of Fig. 4 above, may be added.

[0053] In this regard, the encoding unit (120) can encode an expression pattern embedding based on the RNA relative expression level read from the relative size of each gene node on the gene interaction network.

[0054] In addition, in the training data according to one embodiment of the present invention, a variant embedding that encodes the genetic variation state of each gene node according to the order listed in the aforementioned token embedding as shown in (c) of FIG. 4, which was exemplified above, may be added.

[0055] In this regard, the encoding unit (120) can encode genetic variation embeddings based on whether there is a genetic variation and the type of variation read from the color of each gene node on the gene interaction network.

[0056] For reference, these user mutation embeddings can distinguish between mutation states where a change in amino acids occurs at the protein level (1: Non-Synonymous Mutation), states where a mutation has occurred but there is no change in amino acids (0.5: Synonymous Mutation), and states where there is no mutation (0: No Mutation).

[0057] In addition, in the training data according to one embodiment of the present invention, an edge degree embedding that encodes the branching state of each gene node according to the order listed in the aforementioned token embedding, as shown in (d) of FIG. 4, which was exemplified above, may be added.

[0058] In this regard, the encoding unit (120) can encode branched state embeddings based on the number of edges each gene node has for connection with lower layers on the gene interaction network.

[0059] In summary, the encoding unit (120) can generate final training data for constructing a general-purpose gene language model by adding an expression pattern embedding that encodes the RNA expression state of each gene node, a variant embedding that encodes the genetic variation state of each gene node, and an edge degree embedding that encodes the branch state of each gene node to a token embedding that uses the gene ID of each gene node as a token.

[0060] As described above, according to the configuration of the genome analysis device (100) according to one embodiment of the present invention, accumulated genome data for a target disease is mapped to a gene interaction network, and genetic characteristic information read from the gene interaction network is converted into a sentence form and output as training data required to build a transfer learning-based general-purpose genome language model. Through this, a large general-purpose genome language model of various structures can be built, and a high-accuracy final artificial intelligence model for various genome analysis tasks, such as identifying disease-causing genes, discovering drug target genes, and predicting drug sensitivity, can be built by transfer learning the pre-trained language model using only a small number of patient sample genome data.

[0061] Hereinafter, a method for generating training data according to an embodiment of the present invention will be described with reference to FIG. 6.

[0062] The explanation will continue on the premise that the operating entity of the learning data generation method according to one embodiment of the present invention is the genome analysis device (100) described above with reference to FIG. 2.

[0063] First, the genome analysis device (100) maps the large volume of accumulated genome data for the target disease to a gene interaction network to form an initial training data set (S110-S120).

[0064] Here, the genomic data can identify not only information regarding protein-protein interactions (PPI) but also genetic information such as genetic variation information and RNA expression information for each gene.

[0065] In this regard, the genome analysis device (100) can construct a gene interaction network with gene nodes (① ~ ⑩) branching into a hierarchical structure starting from the node (①) of the ROOT gene according to gene information identified from genome data as illustrated in FIG. 3, as previously illustrated, and edges (E: Edge) connecting gene nodes between adjacent layers.

[0066] In particular, the genome analysis device (100) maps RNA expression information of genes using the relative size (size of a circle) of each gene node on a gene interaction network, and can also map genetic variation information of genes using the color of each gene node.

[0067] Here, RNA expression information expressed by the relative size (size of the circle) of each gene node represents the RNA expression amount of each gene, and genetic variation information expressed by the color of each gene node can be understood as the presence of genetic variation and the type of variation.

[0068] Afterwards, when the genome analysis device (100) configures an initial learning data set in which genome data is mapped to a gene interaction network, it outputs learning data that reflects genetic characteristic information read from the gene interaction network (S130-S150).

[0069] To this end, the genome analysis device (100) can encode genetic characteristic information read from a gene interaction network into a sentence and output the encoded sentence as training data required to build a general-purpose gene language model.

[0070] As such, in one embodiment of the present invention, the training data output for constructing a general-purpose gene language model may include a token embedding in which the gene ID of each gene node is used as a token, as shown in (a) of FIG. 4, which was previously exemplified.

[0071] In this regard, the genome analysis device (100) can encode token embeddings by listing the gene IDs of the gene nodes in a hierarchy according to a depth-first search order starting from the node of the ROOT gene.

[0072] Here, the gene ID of each gene node may refer to a gene vocabulary in which a unique ID is pre-assigned to each gene, such as in Figure 5, which was previously exemplified.

[0073] In addition, in the training data according to one embodiment of the present invention, expression pattern embeddings encoding the RNA expression state of each gene node according to the order listed in the aforementioned token embeddings, as shown in (b) of Fig. 4 above, may be added.

[0074] In this regard, the genome analysis device (100) can encode an expression pattern embedding based on the RNA relative expression level read from the relative size of each gene node on the gene interaction network.

[0075] In addition, in the training data according to one embodiment of the present invention, a variant embedding that encodes the genetic variation state of each gene node according to the order listed in the aforementioned token embedding as shown in (c) of FIG. 4, which was exemplified above, may be added.

[0076] In this regard, the genome analysis device (100) can encode genetic variation embeddings based on whether there is a genetic variation and the type of variation read from the color of each gene node on the gene interaction network.

[0077] For reference, these user mutation embeddings can distinguish between mutation states where a change in amino acids occurs at the protein level (1: Non-Synonymous Mutation), states where a mutation has occurred but there is no change in amino acids (0.5: Synonymous Mutation), and states where there is no mutation (0: No Mutation).

[0078] In addition, in the training data according to one embodiment of the present invention, an edge degree embedding that encodes the branching state of each gene node according to the order listed in the aforementioned token embedding, as shown in (d) of FIG. 4, which was exemplified above, may be added.

[0079] In this regard, the genome analysis device (100) can encode branched state embeddings based on the number of edges each gene node has for connection with lower layers on a gene interaction network.

[0080] In summary, the genome analysis device (100) can generate final training data for constructing a general-purpose gene language model by adding an expression pattern embedding that encodes the RNA expression state of each gene node, a variant embedding that encodes the genetic variation state of each gene node, and an edge degree embedding that encodes the branching state of each gene node to a token embedding that uses the gene ID of each gene node as a token.

[0081] As described above, according to the method for generating training data according to one embodiment of the present invention, genomic data accumulated for a target disease is mapped to a gene interaction network, and genetic characteristic information read from the gene interaction network is converted into a sentence form and output as training data necessary for constructing a transfer learning-based general-purpose gene language model. Through this, a large general-purpose genomic language model of various structures can be constructed, and a high-accuracy final artificial intelligence model for various genomic analysis tasks, such as identifying disease-causing genes, discovering drug target genes, and predicting drug sensitivity, can be constructed by transfer learning the pre-trained language model using only a small number of patient sample genomic data.

[0082] Meanwhile, the functional operations and embodiments of the subject described herein may be implemented as digital electronic circuits, or as computer software, firmware, or hardware comprising the structures and structural equivalents disclosed herein, or as a combination of one or more of these. The embodiments of the subject described herein may be implemented as one or more computer program products, that is, as one or more modules relating to computer program instructions encoded on a tangible program storage medium for processing the operation of a processing system or for execution by means of such processing.

[0083] A computer-readable medium may be a machine-readable storage device, a machine-readable storage substrate, a memory device, or a combination of one or more of these.

[0084] In this specification, the terms “system” or “device” encompass all apparatuses, devices, and machines for processing data, including, for example, programmable processors, computers, or multiple processors or computers. A processing system may, in addition to hardware, include code that forms an execution environment for a computer program upon request, such as, for example, code constituting processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of these.

[0085] A computer program (also known as a program, software, software application, script, or code) may be written in any form of a programming language, including compiled or interpreted languages, or a priori or procedural languages, and may be deployed in any form, including a standalone program, a module, a component, a subroutine, or other unit suitable for use in a computer environment. A computer program does not necessarily correspond to a file in a file system. A program may be stored in a single file provided to the requested program, in multiple interacting files (e.g., a file storing one or more modules, subprograms, or parts of code), or in a part of a file containing other programs or data (e.g., one or more scripts stored within a markup language document). A computer program may be deployed to be executed on multiple computers or a single computer located at a single site or distributed across multiple sites and interconnected by a communication network.

[0086] Meanwhile, computer-readable media suitable for storing computer program instructions and data may include all forms of non-volatile memory, media, and memory devices, such as semiconductor memory devices like EPROM, EEPROM, and flash memory devices, magnetic disks like internal hard disks or external disks, magneto-optical disks, and CD-ROM and DVD-ROM disks. Processors and memory may be supplemented by or integrated with special-purpose logic circuits.

[0087] An embodiment of the subject described herein may be implemented in a computing system comprising, for example, a backend component such as a data server, for example, a middleware component such as an application server, for example, a frontend component such as a client computer having a web browser or a graphical user interface through which a user can interact with the embodiment of the subject described herein, or any combination of one or more such backend, middleware, or frontend components. Components of the system may be interconnected by any form or medium of digital data communication, such as a communication network.

[0088] Although this specification contains details of a number of specific embodiments, they should not be understood as limiting the scope of any invention or claimables, but rather as descriptions of features that may be specific to a particular embodiment of a particular invention. Likewise, specific features described in this specification in the context of individual embodiments may be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented in multiple embodiments individually or in any appropriate sub-combination. Furthermore, while features may operate in a specific combination and be described as initially claimed, one or more features from the claimed combination may be excluded from the combination in some cases, and the claimed combination may be changed to a sub-combination or a variation of the sub-combination.

[0089] Furthermore, although operations are described in the drawings in a specific order in this specification, it should not be understood that such operations must be performed in that specific or sequential order depicted to obtain a desirable result, or that all depicted operations must be performed. In certain cases, multitasking and parallel processing may be advantageous. Additionally, the separation of the various system components of the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together into a single software product or packaged into multiple software products.

[0090] As such, this specification is not intended to limit the invention to the specific terms presented. Accordingly, although the invention has been described in detail with reference to the examples described above, those skilled in the art may make modifications, changes, and variations to these examples without departing from the scope of the invention. The scope of the invention is defined by the claims set forth below rather than by the detailed description above, and all modifications or variations derived from the meaning and scope of the claims and equivalents thereof should be interpreted as being included within the scope of the invention. Industrial applicability

[0091] According to the genome analysis device and training data generation method of the present invention, since training data necessary for constructing a transfer learning-based general-purpose gene language model can be generated using accumulated large-scale genome data, the invention overcomes the limitations of existing technology; as such, it is an invention with industrial applicability, as there is sufficient possibility for the commercialization or business of the applied device, not just the use of related technology, but also a degree that is clearly and practically feasible. Explanation of the symbols

[0092] 100: Genome analysis device 110: Preprocessing unit 120: Encoding unit

Claims

Claim 1 A preprocessing unit that maps genomic data of a target disease to a gene interaction network; The system includes an encoding unit that encodes genetic characteristic information read from the gene interaction network into a sentence and outputs the encoded sentence as training data required to build a general-purpose gene language model. The gene interaction network includes gene nodes that branch into a hierarchical structure starting from a node of the ROOT gene according to gene information identified from the genomic data, and edges that interconnect gene nodes between adjacent layers. The preprocessing unit maps RNA expression information of the gene using the relative size of each gene node and maps genetic variation information of the gene using the color of each gene node. The encoding unit includes: a token embedding that encodes the gene IDs of the gene nodes by layer according to a depth-first search order starting from the node of the ROOT gene, using the gene ID of each gene node as a token; an expression pattern embedding that encodes the RNA expression state of each gene node based on the RNA relative expression level read from the relative size of each gene node according to the order listed in the token embedding; and the presence of genetic variation and variation read from the color of each gene node according to the order listed in the token embedding. A genome analysis device characterized by outputting genetic characteristic information, which is converted into a sentence form and output as training data required to construct a general-purpose gene language model, wherein the genetic characteristic information is sequentially combined with a genetic variation embedding that encodes the genetic variation state of each gene node based on a type and a branch state embedding that encodes the branch state of each gene node based on the number of edges each gene node has for connection with a lower layer on the gene interaction network according to the order listed in the token embedding. Claim 2 delete Claim 3 delete Claim 4 delete Claim 5 delete Claim 6 delete Claim 7 A method for generating training data performed in a genome analysis device, comprising: a preprocessing step of mapping genomic data of a target disease to a gene interaction network; The method includes an encoding step for encoding genetic characteristic information read from the gene interaction network into a sentence and outputting the encoded sentence as training data required to build a general-purpose gene language model, wherein the gene interaction network includes gene nodes branching into a hierarchical structure starting from a ROOT gene node according to gene information identified from the genomic data, and edges interconnecting gene nodes between adjacent layers, wherein the preprocessing step maps RNA expression information of the gene using the relative size of each gene node and maps genetic variation information of the gene using the color of each gene node, and the encoding step includes: a token embedding encoded by listing the gene IDs of the gene nodes by layer according to a depth-first search order starting from the ROOT gene node using the gene ID of each gene node as a token; an expression pattern embedding encoded by encoding the RNA expression state of each gene node based on the RNA relative expression level read from the relative size of each gene node according to the order listed in the token embedding; and whether or not genetic variation is read from the color of each gene node according to the order listed in the token embedding. A method for generating training data characterized by: a genetic variation embedding that encodes the genetic variation state of each gene node based on the type of variation; and a branch state embedding that encodes the branch state of each gene node based on the number of edges each gene node has for connection with a lower layer on the gene interaction network according to the order listed in the token embedding, sequentially combining genetic characteristic information in the form of a sentence to output training data required to build a general-purpose gene language model. Claim 8 delete Claim 9 delete Claim 10 delete

Images