Method, computing device and computer program for predicting patient information by using tokenized electrocardiogram data

Abstract

A method, a computing device and a computer program for predicting patient information by using tokenized electrocardiogram data are provided. The method by which the computing device predicts patient information by using tokenized electrocardiogram data, according to various embodiments of the present disclosure, comprises the steps of: acquiring biometric data of a patient; tokenizing the acquired biometric data so as to generate input data; and inputting the generated input data into a pre-trained deep learning-based patient information prediction model so as to predict patient information about the patient.

Description

Method for predicting patient information using tokenized electrocardiogram data, computing device and computer program

[0001] Various embodiments of the present disclosure relate to a method, a computing device, and a computer program for predicting patient information using tokenized electrocardiogram data.

[0002] Electrocardiogram (ECG) data is a crucial biosignal for assessing cardiac activity and plays a key role in the early diagnosis and prevention of cardiovascular disease. Cardiovascular disease is a leading cause of death worldwide, highlighting the need for effective prediction and prevention strategies. Such predictive technology can not only save patients' lives but also contribute to reducing healthcare costs.

[0003] Electrocardiogram (ECG) data has diverse formats and complex characteristics, making analysis difficult. For example, ECG data is provided in various formats, such as 1-lead, 6-lead, and 12-lead, depending on the measurement method, and each lead represents electrical signals from a different part of the heart. This data can vary greatly in duration depending on the patient's condition and recording environment, and its size and quality are often inconsistent. Consequently, building a model capable of comprehensively analyzing this data presents significant challenges.

[0004] Conventional patient information prediction methods often rely on limited analysis based on specific lead data or on time-consuming, resource-intensive manual analysis. These methods suffer from limitations, such as their inability to effectively handle data diversity and the complexities of large-scale data processing. In particular, analyses utilizing multi-lead data can fail to accurately reflect correlations between leads or result in information loss during data integration. This negatively impacts the accuracy and reliability of patient information predictions.

[0005] Furthermore, the lack of standardization of ECG data is being pointed out as a major issue. Data formats and measurement standards vary across medical institutions and equipment manufacturers, creating additional complexity in integrating and analyzing data from diverse sources. This can lead to predictive systems utilizing advanced artificial intelligence (AI) models failing to function properly or resulting in results optimized for specific data sets.

[0006] The background technology described above is something that the inventor possessed or acquired in the process of deriving the contents of the present disclosure, and cannot necessarily be said to be a publicly known technology disclosed to the general public prior to the present application.

[0007] The problem to be solved by the present disclosure is to provide a method for predicting patient information using tokenized electrocardiogram data, a computing device and a computer program capable of predicting patient information more accurately regardless of the number of leads and time length of the electrocardiogram data by generating a token set including one or more tokens corresponding to input data in the form of tokens, that is, electrocardiogram information included in the electrocardiogram data, by tokenizing electrocardiogram data in various input forms to have a finite combination for the purpose of solving the conventional problems described above, and predicting patient information of a patient based on the tokenized electrocardiogram data.

[0008] The problems to be solved by the present disclosure are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the description below.

[0009] A method for predicting patient information using tokenized electrocardiogram data according to one embodiment of the present disclosure for solving the above-described problem may be performed by a computing device. In various embodiments, the method for predicting patient information using tokenized electrocardiogram data may include the steps of: acquiring patient biometric data; generating normalized input data in the form of tokens by tokenizing the acquired biometric data; and inputting the generated input data into a pre-trained deep learning-based patient information prediction model to predict patient information about the patient.

[0010] In various embodiments, the acquired biometric data may be N-lead electrocardiogram data measured through an electrocardiogram measuring device including N electrodes.

[0011] In various embodiments, the step of generating the input data includes a step of converting the acquired biometric data into one or more tokens by tokenizing the acquired biometric data through a pre-learned tokenization module, wherein the pre-learned tokenization module may include an encoder for extracting features from the biometric data, a codebook for storing a plurality of tokens each corresponding to a plurality of features that can be extracted from the biometric data, and a vector quantization unit for converting features extracted through the encoder into one or more tokens in a vector form based on the plurality of tokens stored in the codebook.

[0012] In various embodiments, the step of generating the input data may include, when the acquired biometric data is electrocardiogram data, a step of generating the input data by converting the electrocardiogram data into one or more tokens in proportion to the length of electrocardiogram information included in the electrocardiogram data.

[0013] In various embodiments, the step of generating the input data may include a step of generating a plurality of unit biometric data by dividing the biometric signals included in the acquired biometric data so that at least some areas overlap when the time length of the acquired biometric data exceeds a reference time length, a step of tokenizing each of the generated plurality of unit biometric data, and a step of generating one input data by collating the tokenized plurality of unit biometric data.

[0014] In various embodiments, the step of generating the input data includes a step of preprocessing the acquired biometric data so that the number of channels of the biometric data becomes the number of reference channels when the number of channels of the acquired biometric data is different from the number of reference channels, and a step of generating the input data by tokenizing the biometric data including the masking value, and the step of preprocessing the acquired biometric data may include a step of comparing the number of channels of the acquired biometric data with the number of reference channels, and adding one or more channels filled with the masking value by the number of channels that is short of the number of reference channels to the acquired biometric data.

[0015] In various embodiments, the step of predicting the patient information may include a step of predicting clinical test results for the patient as patient information by inputting the generated input data into the patient information prediction model.

[0016] In various embodiments, the step of predicting the patient information may include a step of predicting the likelihood of occurrence of each of a plurality of different heart diseases as patient information by inputting the generated input data into the patient information prediction model.

[0017] According to another embodiment of the present disclosure for solving the above-described problem, a method for generating a tokenization module for generating tokenized electrocardiogram data for predicting patient information may be performed by a computing device. In various embodiments, the method for generating a tokenization module for generating tokenized electrocardiogram data for predicting patient information includes: generating training data by masking at least a portion of biometric data; generating a training AI model for restoring original biometric data corresponding to at least a portion of the masked data based on the generated training data by training a non-trained AI model using the generated training data; and generating a tokenization module using the generated training AI model, wherein the generated tokenization module may be a module for generating input data in the form of a token set including one or more tokens by tokenizing input biometric data.

[0018] In various embodiments, the non-learning artificial intelligence model includes an encoder for extracting features from input biometric data, a codebook for storing a plurality of tokens defining features extractable from the biometric data, a vector quantization unit for converting features extracted by the encoder into one or more tokens in a vector form based on the plurality of tokens stored in the codebook, and a decoder for restoring original biometric data based on the one or more tokens converted by the vector quantization unit, wherein the step of generating the learning artificial intelligence model may include a step of performing pre-learning on the codebook and vector quantization unit of the non-learning artificial intelligence model using the generated learning data, and a step of performing fine tuning on the encoder and decoder of the non-learning artificial intelligence model and the pre-learned vector quantization unit using the generated learning data while fixing a plurality of tokens included in the pre-learned codebook.

[0019] In various embodiments, the step of performing the pre-training includes the steps of initializing a codebook of the non-learning artificial intelligence model, converting features extracted from an encoder of the non-learning artificial intelligence model into one or more tokens based on tokens included in the initialized codebook through a vector quantization unit of the non-learning artificial intelligence model, and pre-training the initialized codebook based on one or more tokens converted from the vector quantization unit of the non-learning artificial intelligence model so as to minimize a first loss function, and pre-training the vector quantization unit of the non-learning artificial intelligence model so as to minimize a second loss function based on one or more tokens converted from the vector quantization unit of the non-learning artificial intelligence model, wherein the first loss function may be a function that calculates a degree of imbalance in a distribution of tokens used by the vector quantization unit of the non-learning artificial intelligence model for conversion, and the second loss function may be a function that calculates an error in token conversion performed by the vector quantization unit of the non-learning artificial intelligence model.

[0020] In various embodiments, the step of performing the fine tuning includes the steps of extracting features from the generated training data through an encoder of the non-learning artificial intelligence model, converting the extracted features into one or more tokens based on a plurality of tokens included in the pre-learned codebook through the pre-learned vector quantization unit, restoring biometric data based on one or more tokens converted from the pre-learned vector quantization unit through a decoder included in the non-learning artificial intelligence model, and fine-tuning the encoder and decoder of the non-learning artificial intelligence model and the pre-learned vector quantization unit based on the restored biometric data so that a third loss function is minimized, wherein the third loss function may be a function that calculates an error between original biometric data corresponding to the generated training data and the restored biometric data.

[0021] In various embodiments, the generated learning artificial intelligence model includes the fine-tuned encoder, the pre-learned codebook, the fine-tuned vector quantization unit, and the fine-tuned decoder, and the step of generating the tokenization module may include the step of generating the tokenization module by modularizing the fine-tuned encoder, the pre-learned codebook, and the fine-tuned vector quantization unit.

[0022] In various embodiments, the step of generating the learning data may include, when the number of channels of the biometric data is less than the number of reference channels, the step of preprocessing the biometric data by adding one or more channels filled with a preset masking value to the biometric data so that the number of channels of the biometric data becomes the number of reference channels, and the step of generating the learning data using the preprocessed biometric data.

[0023] In various embodiments, the step of generating the learning data may include the step of performing preprocessing on the biometric data so that the time length of the biometric data becomes a multiple of a reference time length by adjusting the time length of the biometric data, and the step of generating learning data using the preprocessed biometric data.

[0024] According to another embodiment of the present disclosure for solving the above-described problem, a computing device includes a processor, a network interface, a memory, and a computer program loaded into the memory and executed by the processor, wherein the processor can perform methods according to various embodiments of the present disclosure by executing one or more instructions included in the computer program.

[0025] A computer program according to another embodiment of the present disclosure for solving the above-described problem can be stored in a recording medium readable by a computing device to execute methods according to various embodiments of the present disclosure.

[0026] Other specific details of the present disclosure are included in the detailed description and drawings.

[0027] According to various embodiments of the present disclosure, by tokenizing electrocardiogram data, input data in the form of tokens, i.e., a token set including one or more tokens corresponding to electrocardiogram information included in the electrocardiogram data, can be generated, thereby standardizing various types of electrocardiogram data into a finite type of token combination, and by predicting patient information of a patient based on the tokenized data, there is an advantage in that patient information of a patient can be predicted more accurately regardless of the number of leads and time length of the electrocardiogram data.

[0028] The effects of the present disclosure are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description below.

[0029] The following drawings attached to this specification illustrate preferred embodiments of the present disclosure and, together with the detailed description of the invention, serve to further understand the technical idea of ​​the present disclosure, and therefore, the present disclosure should not be interpreted as being limited to matters described in such drawings.

[0030] FIG. 1 is a diagram illustrating a hardware configuration of a computing device according to one embodiment of the present disclosure.

[0031] FIG. 2 is a flowchart of a method for generating a tokenization module for generating tokenized electrocardiogram data for predicting patient information according to another embodiment of the present disclosure.

[0032] FIG. 3 is a diagram illustrating a tokenization module creation process in various embodiments.

[0033] FIG. 4 is a flowchart of a method for predicting patient information using tokenized electrocardiogram data according to another embodiment of the present disclosure.

[0034] FIG. 5 is a diagram illustrating a patient information prediction process in various embodiments.

[0035] FIG. 6 is a flowchart of a method for learning a patient information prediction model for predicting heart disease according to another embodiment of the present disclosure.

[0036] FIG. 7 is a diagram illustrating a classification process of electrocardiogram data in various embodiments.

[0037] FIG. 8 is a diagram illustrating a process of segmenting electrocardiogram data into individual heart beats in various embodiments.

[0038] Figures 9A to 9C are diagrams showing types of patient information prediction models applicable to various embodiments.

[0039] FIG. 10 is a diagram illustrating a learning step of a patient information prediction model in various embodiments.

[0040] FIG. 11 is a flowchart of a method for predicting the likelihood of occurrence of heart disease as patient information using a learned patient information prediction model according to another embodiment of the present disclosure.

[0041] FIG. 12 is a diagram illustrating a step of predicting the likelihood of occurrence of heart disease using a patient information prediction model in various embodiments.

[0042] The advantages and features of the present disclosure, and methods for achieving them, will become clearer with reference to the embodiments described below in detail with the accompanying drawings. However, the present disclosure is not limited to the embodiments disclosed below and may be implemented in various different forms. These embodiments are provided solely to ensure that the disclosure is complete and to fully inform those skilled in the art of the scope of the present disclosure, and the present disclosure is defined solely by the scope of the claims.

[0043] The terminology used herein is for the purpose of describing embodiments and is not intended to limit the present disclosure. In this specification, singular forms also include plural forms, unless specifically stated otherwise. As used herein, the terms "comprises" and / or "comprising" do not exclude the presence or addition of one or more other components in addition to the components mentioned.

[0044] Throughout this specification, the same reference numerals refer to the same elements, and the term "and / or" includes each and every combination of the elements mentioned. Although terms such as "first," "second," etc. are used to describe various elements, these elements are not limited by these terms. These terms are merely used to distinguish one element from another. Accordingly, it should be understood that a first element mentioned below may also be a second element within the technical scope of the present disclosure.

[0045] The term "component" or "module" as used herein refers to a software or hardware component such as an FPGA or ASIC, and the "component" or "module" performs certain functions. However, the "component" or "module" is not limited to software or hardware. The "component" or "module" may be configured to reside on an addressable storage medium and may be configured to execute one or more processors. Thus, by way of example, the "component" or "module" includes components such as software components, object-oriented software components, class components, and task components, as well as processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays, and variables. The functionality provided within the components and "components" or "modules" may be combined into a smaller number of components and "components" or "modules" or further separated into additional components and "components" or "modules."

[0046] Spatially relative terms such as "below," "beneath," "lower," "above," and "upper" can be used to easily describe the relationship between one component and other components as depicted in the drawings. Spatially relative terms should be understood to include different orientations of the components during use or operation in addition to the orientations depicted in the drawings. For example, if a component depicted in the drawings were flipped over, a component described as "below" or "beneath" another component could end up "above" the other component. Thus, the exemplary term "below" can include both the above and below orientations. Components can also be oriented in other directions, and thus spatially relative terms can be interpreted accordingly.

[0047] As used herein, the expressions “first,” “second,” or “first,” “second,” etc., unless the context indicates otherwise, are used to refer to multiple similar objects and to distinguish one object from another, and do not limit the order or importance among the objects.

[0048] As used herein, the expressions "A, B, and C," "A, B, or C," "A, B, and / or C," or "at least one of A, B, and C," "at least one of A, B, or C," "at least one of A, B, and / or C," "at least one selected from A, B, and C," "at least one selected from A, B, or C," "at least one selected from A, B, and / or C," and the like can mean each listed item or all possible combinations of the listed items. For example, "at least one selected from A and B" can refer to (1) A, (2) at least one of A, (3) B, (4) at least one of B, (5) at least one of A and at least one of B, (6) at least one of A and B, (7) at least one of B and A, and (8) both A and B.

[0049] The expression "based on" as used herein is used to describe one or more factors that influence a decision, act of judgment, or action described in a phrase or sentence containing the expression, and this expression does not exclude additional factors that influence the decision, act of judgment, or action.

[0050] As used herein, the expression that a component (e.g., a first component) is “connected” or “connected” to another component (e.g., a second component) may mean that the component is directly connected or connected to the other component, as well as connected or connected via a new other component (e.g., a third component).

[0051] The expression "configured to" used herein may have the meanings of "set to", "having the ability to", "modified to", "made to", "capable of", etc., depending on the context. The expression is not limited to the meaning of "specifically designed in hardware", and for example, a processor configured to perform a specific operation may mean a generic-purpose processor that can perform the specific operation by executing software.

[0052] Unless otherwise defined, all terms (including technical and scientific terms) used herein may be used in their common sense to those of ordinary skill in the art to which this disclosure pertains. Furthermore, terms defined in commonly used dictionaries are not to be interpreted ideally or excessively unless explicitly and specifically defined otherwise.

[0053] In this specification, the term "computer" refers to any type of hardware device including at least one processor, and may also be understood to encompass software components operating on the hardware device, depending on the embodiment. For example, the term "computer" may be understood to encompass, but is not limited to, smartphones, tablet PCs, desktops, laptops, and all user clients and applications running on each device.

[0054] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the attached drawings.

[0055] Although each step described in this specification is described as being performed by a computer, the subject of each step is not limited thereto, and at least some of each step may be performed by different devices depending on the embodiment.

[0056]

[0057] FIG. 1 is a diagram illustrating a hardware configuration of a computing device according to one embodiment of the present disclosure.

[0058] Referring to FIG. 1, a computing device (100) according to another embodiment of the present disclosure may include one or more processors (110), a memory (120) for loading a computer program (151) executed by the processor (110), a bus (130), a communication interface (140), and a storage (150) for storing the computer program (151). Here, only components related to the embodiment of the present disclosure are illustrated in FIG. 1. Therefore, a person skilled in the art to which the present disclosure pertains may understand that other general components may be included in addition to the components illustrated in FIG. 1.

[0059] The processor (110) controls the overall operation of each component of the computing device (100). The processor (110) may be configured to include a CPU (Central Processing Unit), an MPU (Micro Processor Unit), an MCU (Micro Controller Unit), a GPU (Graphics Processing Unit), or any other type of processor well known in the art of the present disclosure.

[0060] Additionally, the processor (110) may perform operations for at least one application or program for executing a method according to embodiments of the present disclosure, and the computing device (100) may have one or more processors.

[0061] In various embodiments, the processor (110) may further include a Random Access Memory (RAM) (not shown) and a Read-Only Memory (ROM) (not shown) that temporarily and / or permanently store signals (or data) processed within the processor (110). In addition, the processor (110) may be implemented in the form of a System on Chip (SoC) that includes at least one of a graphics processing unit, RAM, and ROM.

[0062] The memory (120) stores various data, commands, and / or information. The memory (120) can load a computer program (151) from the storage (150) to execute methods / operations according to various embodiments of the present disclosure. When the computer program (151) is loaded into the memory (120), the processor (110) can perform the method / operation by executing one or more instructions constituting the computer program (151). The memory (120) may be implemented as a volatile memory such as RAM, but the technical scope of the present disclosure is not limited thereto.

[0063] The bus (130) provides a communication function between components of the computing device (100). The bus (130) may be implemented as various types of buses, such as an address bus, a data bus, and a control bus.

[0064] The communication interface (140) supports wired and wireless Internet communication of the computing device (100). Furthermore, the communication interface (140) may support various communication methods other than Internet communication. To this end, the communication interface (140) may be configured to include a communication module well known in the technical field of the present disclosure. In some embodiments, the communication interface (140) may be omitted.

[0065] Storage (150) can non-temporarily store a computer program (151). When performing a patient information prediction process using tokenized electrocardiogram data through a computing device (100), storage (150) can store various information necessary to provide a patient information prediction process using tokenized electrocardiogram data.

[0066] Storage (150) may be configured to include non-volatile memory such as ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), flash memory, a hard disk, a removable disk, or any form of computer-readable recording medium well known in the art to which the present disclosure pertains.

[0067] The computer program (151) may include one or more instructions that, when loaded into the memory (120), cause the processor (110) to perform a method / operation according to various embodiments of the present disclosure. That is, the processor (110) may perform the method / operation according to various embodiments of the present disclosure by executing the one or more instructions.

[0068] In one embodiment, the computer program (151) may include one or more instructions for performing a method for predicting patient information using tokenized electrocardiogram data, including the steps of obtaining patient biometric data, generating input data by tokenizing the obtained biometric data, and inputting the generated input data into a pre-trained deep learning-based patient information prediction model to predict patient information about the patient.

[0069] In addition, the computer program (151) may include one or more instructions for performing a method for generating a tokenization module for generating tokenized electrocardiogram data for predicting patient information, including a step of generating learning data by masking at least a portion of the biometric data, a step of generating a learning artificial intelligence model that restores original biometric data based on the generated learning data by training a non-learning artificial intelligence model using the generated learning data, and a step of generating a tokenization module using the generated learning artificial intelligence model.

[0070] In one embodiment, the execution screen of the computer program (151) may be displayed through the display (160). In the case of FIG. 1, the display (160) is represented as a separate device connected to the computing device (100). However, in the case of a computing device (100) such as a terminal that a user can carry, such as a smartphone or tablet, the display (160) may be a component of the computing device (100). The screen displayed on the display (160) may be a result of inputting information into the program or an execution of the program. Hereinafter, methods performed by the computing device (100) will be described with reference to FIGS. 2 to 12.

[0071]

[0072] FIG. 2 is a flowchart of a method for generating a tokenization module for generating tokenized electrocardiogram data for predicting patient information according to another embodiment of the present disclosure, and FIG. 3 is a diagram illustrating a tokenization module generation process in various embodiments.

[0073] Here, the method to be described with reference to FIGS. 2 and 3 can be performed by the computing device (100) illustrated in FIG. 1, but is not limited thereto.

[0074] Referring to FIGS. 2 and 3, in step S110, the computing device (100) can generate learning data based on the patient's biometric data.

[0075] In various embodiments, the computing device (100) can generate at least partially obscured biometric data as learning data by masking at least a portion of the patient's biometric data.

[0076] Here, the patient's biometric data may be electrocardiogram data, which may be N-lead electrocardiogram data measured by an electrocardiogram measuring device including N electrodes. For example, the electrocardiogram data may be, but is not limited to, 1-lead electrocardiogram data collected for a first period through a smartwatch worn by the user, 6-lead electrocardiogram data collected by wearing electrodes on both the limbs and the chest, and / or 12-lead electrocardiogram data collected through a 24-hour Holter monitor or a home electrocardiogram monitor.

[0077] Additionally, here, the patient's bio-data may have different time lengths, but is not limited thereto.

[0078] In various embodiments, when the number of channels of biometric data is less than the reference number of channels, the computing device (100) may preprocess the biometric data by adding one or more channels filled with preset masking values ​​to the biometric data so that the number of channels of the biometric data becomes the reference number of channels, and may generate learning data using the preprocessed biometric data.

[0079] Here, the reference channel number refers to the number of channels of standard biometric data. For example, the standard biometric data may be 12-lead electrocardiogram data, and accordingly, the reference channel number may be 12, which is the number of leads.

[0080] For example, when the biometric data is electrocardiogram data and the reference number of channels (reference number of leads) is 12, and the electrocardiogram data for which learning data is to be generated is 6-lead electrocardiogram data, the computing device (100) can preprocess the electrocardiogram data into 12-lead electrocardiogram data by adding 6 leads including 0 using a zero padding method.

[0081] In various embodiments, the computing device (100) may perform preprocessing on the biometric data by adjusting the time length of the biometric data so that the time length of the biometric data becomes the reference time length when the time length of the biometric data is different from the reference time length, and may generate learning data using the preprocessed biometric data.

[0082] Here, the reference time length refers to the time length of standard biometric data. For example, the standard biometric data may be 10 seconds of electrocardiogram data, and thus the reference time length may be 10 seconds. However, the reference time length may be the time length of biometric data that can be used as input to the model.

[0083] For example, if the time length of the biometric data is 72 hours of electrocardiogram data and the reference time length is 10 seconds, the computing device (100) can preprocess the electrocardiogram data into 10 seconds of electrocardiogram data by adjusting the 72 hours of electrocardiogram data to 10 seconds in length.

[0084] Here, adjusting the time length may be, but is not limited to, truncating the biometric data by a preset rule and / or arbitrarily by a standard time length.

[0085] In various embodiments, the computing device (100) may perform preprocessing on the biometric data by adjusting the time length of the biometric data so that the time length of the biometric data becomes a multiple of a reference time length, and may generate learning data using the preprocessed biometric data.

[0086] In step S120, the computing device (100) can train a non-learning artificial intelligence model using the learning data generated through step S110.

[0087] In various embodiments, the computing device (100) can generate a learning artificial intelligence model that restores original biometric data corresponding to at least a portion of the masked biometric data based on the learning data generated by training a non-learning artificial intelligence model using at least a portion of the masked biometric data as learning data.

[0088] Here, the non-learning AI model and the learning AI model are described as separate components, but this is to more easily and clearly distinguish between the pre- and post-learning AI models, and the non-learning AI model and the learning AI model can be one and the same model.

[0089] In other words, a non-learning AI model may refer to an initial AI model that has not been trained (pre-learning AI model), and a learning AI model may refer to a model that has trained an initial AI model based on training data (post-learning AI model).

[0090] An AI model (e.g., a non-learning AI model and a learning AI model) is a neural network composed of one or more network functions. Each network function may be composed of a set of interconnected computational units, generally referred to as "nodes." These "nodes" may also be referred to as "neurons." Each network function comprises at least one node. The nodes (or neurons) constituting each network function may be interconnected by one or more "links."

[0091] Within an AI model, one or more nodes connected via links can form a relationship between input nodes and output nodes. The concepts of input nodes and output nodes are relative, meaning that any node in an output node relationship with one node can also be in an input node relationship with another node, and vice versa. As described above, input node-to-output node relationships can be created around links. One input node can be connected to one or more output nodes via links, and vice versa.

[0092] In a relationship between input nodes and output nodes connected through a single link, the value of the output node can be determined based on the data input to the input node. Here, the node interconnecting the input nodes and output nodes can have a weight. The weight can be variable and can be varied by a user or an algorithm so that the artificial intelligence model can perform a desired function. For example, when one or more input nodes are interconnected to one output node through each link, the output node can determine the output node value based on the values ​​input to the input nodes connected to the output node and the weight set for the link corresponding to each input node.

[0093] As described above, an AI model consists of one or more nodes interconnected through one or more links, forming input and output node relationships within the AI ​​model. The characteristics of an AI model can be determined based on the number of nodes and links, the relationships between nodes and links, and the weights assigned to each link. For example, if two AI models exist with the same number of nodes and links but different weight values ​​between the links, the two AI models may be perceived as different from each other.

[0094] Some of the nodes that make up an AI model can form a layer based on their distances from the initial input node. For example, a set of nodes that are n distances from the initial input node can form n layers. The distance from the initial input node can be defined by the minimum number of links that must be passed to reach the node from the initial input node. However, this definition of a layer is arbitrary for illustrative purposes, and the order of layers within an AI model can be defined in a different way than described above. For example, the layer of nodes can also be defined by their distance from the final output node.

[0095] The initial input node may refer to one or more nodes in an AI model into which data is directly input without going through a link in its relationship with other nodes. Alternatively, in an AI model network, in terms of the relationship between nodes based on links, it may refer to nodes that do not have other input nodes connected by links. Similarly, the final output node may refer to one or more nodes in an AI model that do not have an output node in its relationship with other nodes. In addition, a hidden node may refer to nodes that constitute an AI model other than the initial input node and the final output node. An AI model according to one embodiment of the present disclosure may be an AI model in which the number of nodes in the input layer may be greater than that in the hidden layer closer to the output layer, and the number of nodes decreases as it progresses from the input layer to the hidden layer.

[0096] An AI model may include one or more hidden layers. Hidden nodes in a hidden layer can receive the output of the previous layer and the output of surrounding hidden nodes as input. The number of hidden nodes in each hidden layer may be the same or different. The number of nodes in the input layer may be determined based on the number of data fields in the input data and may be the same as or different from the number of hidden nodes. Input data entered into the input layer may be computed by the hidden nodes in the hidden layer and output by the output layer, a fully connected layer (FCL).

[0097] In various embodiments, the artificial intelligence model may be a deep learning model.

[0098] A deep learning model (e.g., a deep neural network (DNN)) can refer to an artificial intelligence model that includes multiple hidden layers in addition to input and output layers. Using a deep neural network, it is possible to identify latent structures in data. That is, it is possible to identify latent structures in photos, text, videos, voices, and music (e.g., what objects are in the photo, what the content and emotion of the text are, what the content and emotion of the voice are, etc.).

[0099] Deep neural networks may include, but are not limited to, convolutional neural networks (CNNs), recurrent neural networks (RNNs), autoencoders, generative adversarial networks (GANs), restricted boltzmann machines (RBMs), deep belief networks (DBNs), Q networks, U networks, Siamese networks, etc.

[0100] In various embodiments, the network function may include an autoencoder, which may be a type of artificial neural network that outputs output data similar to input data.

[0101] An autoencoder may include at least one hidden layer, and an odd number of hidden layers may be positioned between the input and output layers. The number of nodes in each layer may be reduced from the number of nodes in the input layer to an intermediate layer called the bottleneck layer (encoding), and then expanded symmetrically from the bottleneck layer to the output layer (symmetrical to the input layer). The nodes of the dimensionality reduction layer and the dimensionality restoration layer may or may not be symmetrical. In addition, the autoencoder may perform nonlinear dimensionality reduction. The number of input and output layers may correspond to the number of sensors remaining after preprocessing the input data. In the autoencoder structure, the number of nodes in the hidden layer included in the encoder may have a structure that decreases as it moves away from the input layer. The number of nodes in the bottleneck layer (the layer with the fewest nodes located between the encoder and the decoder) may be maintained above a certain number (e.g., more than half of the number of nodes in the input layer) because too small a number may not convey sufficient information.

[0102] In various embodiments, the artificial intelligence model may be a VQ-VAE (Vector Quantized Variational Autoencoder) model including an encoder that extracts features from input biometric data that may include missing values, a codebook that stores a plurality of tokens defining features that can be extracted from the biometric data, a vector quantizer that converts features extracted through the encoder into one or more tokens in vector form based on the plurality of tokens stored in the codebook, and a decoder that restores original biometric data based on the tokens converted through the vector quantizer.

[0103] In various embodiments, by performing pre-training on the codebook and vector quantization unit of a non-learning artificial intelligence model using learning data of a computing device (100), a codebook including a plurality of tokens can be constructed, and based on this, fine tuning can be performed on the encoder and decoder of the non-learning artificial intelligence model and the pre-trained vector quantization unit.

[0104] More specifically, first, the computing device (100) can initialize a codebook of a non-learning artificial intelligence model.

[0105] Thereafter, the computing device (100) can extract features from the training data through the encoder of the non-learning artificial intelligence model. In addition, the computing device (100) can convert the features extracted from the encoder of the non-learning artificial intelligence model into one or more tokens based on the tokens included in the initialized codebook through the vector quantization unit of the non-learning artificial intelligence model.

[0106] Thereafter, the computing device (100) can pre-train the initialized codebook so that the first loss function is minimized based on one or more tokens converted from the vector quantization unit of the non-learning artificial intelligence model.

[0107] Here, the first loss function may be a function that calculates the degree of imbalance in the distribution of tokens used for transformation by the vector quantization unit of the non-learning artificial intelligence model (e.g., diversity loss (KL divergence loss)), i.e., a function that evaluates how many and how diverse the number of tokens used by the vector quantization unit were.

[0108] Additionally, the computing device (100) can pre-train the vector quantization unit of the non-learning artificial intelligence model so that the second loss function is minimized based on one or more tokens converted from the vector quantization unit of the non-learning artificial intelligence model.

[0109] Here, the second loss function may be a function that calculates the error of the token conversion performed by the vector quantization unit of the non-learning artificial intelligence model (e.g., confidence loss), i.e., a function that evaluates how confident the probability distribution of the token used through the vector quantization unit was.

[0110] Thereafter, the computing device (100) can extract features from the training data through the encoder of the non-learning artificial intelligence model. In addition, the computing device (100) can convert the features extracted from the encoder of the non-learning artificial intelligence model into one or more tokens based on a plurality of tokens included in the pre-learned codebook through the pre-learned vector quantization unit.

[0111] Thereafter, the computing device (100) can restore the biometric data based on one or more tokens converted from the pre-learned vector quantization unit through a decoder included in the non-learning artificial intelligence model.

[0112] Thereafter, the computing device (100) can fine-tune the encoder and decoder of the non-learning artificial intelligence model and the pre-learned vector quantization unit based on the restored biometric data so that the third loss function is minimized.

[0113] Here, the third loss function may be a function that calculates the error between the original biometric data corresponding to the training data and the restored biometric data (e.g., Reconstruction Loss), i.e., a function that evaluates how well the masked portion of the training data was restored through the decoder.

[0114] The learning artificial intelligence model generated through the above process may include a pre-learned codebook based on learning data, a fine-tuned encoder and decoder, and a pre-learned and fine-tuned vector quantization unit.

[0115] At step S130, the computing device (100) can generate a tokenization module by modularizing the encoder, codebook, and vector quantization unit included in the learning artificial intelligence model, i.e., the fine-tuned encoder, the pre-learned codebook, and the pre-learned and fine-tuned vector quantization unit.

[0116] The tokenization module generated in this way may be a module that analyzes input biometric data through an encoder to extract features, and converts the features extracted from the encoder into one or more tokens through a vector quantization unit, thereby deriving a token set including one or more tokens corresponding to features extracted from the input biometric data.

[0117]

[0118] FIG. 4 is a flowchart of a method for predicting patient information using tokenized electrocardiogram data according to another embodiment of the present disclosure, and FIG. 5 is a diagram illustrating a patient information prediction process in various embodiments.

[0119] Here, the method to be described with reference to FIGS. 4 and 5 can be performed by the computing device (100) illustrated in FIG. 1, but is not limited thereto.

[0120] Referring to FIGS. 4 and 5, in step S210, the computing device (100) can obtain biometric data from a patient whose patient information is to be predicted.

[0121] Here, the biometric data may be, but is not limited to, N-lead electrocardiogram data measured through an electrocardiogram measuring device including N electrodes.

[0122] For example, the computing device (100) may be connected to a wearable device worn by a patient, and may acquire 1-lead electrocardiogram data acquired through a sensor included in the wearable device as the patient's biometric data, but is not limited thereto.

[0123] Here, the biometric data may be electrocardiogram data having a relatively short time length of about 10 seconds, but is not limited thereto, and may also be electrocardiogram data having a long time length of 24 hours or more.

[0124] In step S220, the computing device (100) can generate input data by tokenizing the biometric data acquired through step S210.

[0125] Here, generating input data by tokenizing biometric data may mean generating normalized data in the form of tokens by converting biometric data into a finite type of tokens defined in advance.

[0126] In various embodiments, the computing device (100) may generate input data in the form of a token set including one or more tokens by tokenizing biometric data through a tokenization module. For example, the computing device (100) may extract one or more features from the biometric data through an encoder of the tokenization module, and may convert one or more features extracted from the encoder into one or more tokens based on a plurality of tokens stored in a codebook through a vector quantization unit of the tokenization module. Through this, the computing device (100) may generate a token set including one or more tokens as input data by converting the biometric data into one or more tokens corresponding to one or more features extracted from the encoder.

[0127] In various embodiments, when the number of channels of the patient's biometric data is different from the number of reference channels, the computing device (100) may preprocess the biometric data by adding one or more channels filled with a preset masking value to the biometric data so that the number of channels of the biometric data becomes the number of reference channels, and may generate input data by tokenizing the preprocessed biometric data. For example, when the biometric data is electrocardiogram data and the number of reference channels (number of reference leads) is 12, and the patient's electrocardiogram data is 1-lead electrocardiogram data acquired from a wearable device, the computing device (100) may preprocess the electrocardiogram data into 12-lead electrocardiogram data by adding 11 leads including 0 in a zero padding manner, and may generate input data by tokenizing the same.

[0128] In various embodiments, when the number of channels of first biometric data acquired for a patient is different from the number of reference channels, the computing device (100) may generate second biometric data having a reference number of channels based on a first token set generated by tokenizing the first biometric data, and may generate a second token set as input data by tokenizing the second biometric data. For example, when the first biometric data acquired for a patient is 6-lead electrocardiogram data and the number of reference channels is 12, the computing device (100) may generate a first token set based on the 6-lead electrocardiogram data, and may generate 12-lead electrocardiogram data based on the first token set in a latent space through a biometric data restoration module.

[0129] Here, the biometric data restoration module may be an artificial intelligence model separately provided for biometric data restoration, such as a latent diffusion model, but is not limited thereto.

[0130] In various embodiments, when the biometric data is electrocardiogram data, the computing device (100) may generate input data by converting the electrocardiogram data into one or more tokens in proportion to the length of the electrocardiogram information included in the electrocardiogram data.

[0131] In various embodiments, the computing device (100) may generate a plurality of unit biometric data by dividing the biometric signals included in the biometric data so that at least some areas overlap when the time length of the biometric data exceeds a reference time length, tokenize each of the plurality of unit biometric data, and generate one input data by combining the tokenized plurality of unit biometric data.

[0132] At step S230, the computing device (100) can predict patient information about the patient by inputting the input data generated through step S220 into a pre-learned deep learning-based patient information prediction model.

[0133] Here, patient information may be lab findings about the patient and / or the likelihood of developing each of multiple different heart diseases.

[0134] To this end, the deep learning-based heart disease prediction model may be a model trained using a plurality of token sets generated in response to each of a plurality of biometric data as input data, and patient information (clinical test results (lab findings)) and / or presence or absence of heart disease (e.g., possibility of disease occurrence such as arrhythmia) corresponding to each of the plurality of biometric data as correct answer data, and may be a model that receives specific biometric data as input and predicts clinical test results and / or possibility of occurrence of each of a plurality of heart diseases as patient information.

[0135] Here, the results of clinical tests include blood cell counts (e.g., White blood cells, Hemoglobin, Platelets), liver and kidney function (e.g., Total bilirubin, AST / ALT, ALP, GGT, Cholesterol, Albumin, Blood urea nitrogen, Creatinine), electrolyte levels (e.g., Na (sodium), K (potassium), Cl (chloride), TCO2 (bicarbonate), Calcium (calcium) / Phosphate), metabolic indicators (e.g., Cholesterol, HbA1c, Lactate), other physiological status (e.g., CRP (C-reactive protein), ESR (erythrocyte sedimentation rate), PT (prothrombin time), aPTT (activated partial prothrombin time), ABGA (pH, pO2, pCO2)) and additional items (e.g., Age, Sex, Body Mass Index (BMI)).

[0136] As described above, according to the method for predicting patient information using tokenized electrocardiogram data of the present disclosure, by tokenizing electrocardiogram data collected from a patient, a token set including tokens corresponding to meaningful electrocardiogram information can be generated, and patient information about the patient can be predicted based on the token set, thereby predicting patient information more accurately regardless of the number of leads of the electrocardiogram data.

[0137] In particular, the method for predicting patient information using tokenized electrocardiogram data of the present disclosure enables highly reliable prediction of patient information regardless of the number of leads, so that even when analyzing 1-lead electrocardiogram data obtained from a wearable device worn by a patient, prediction at the same level as analyzing 12-lead electrocardiogram data is possible, thereby enabling real-time prediction of patient information for a patient based on a wearable device.

[0138] Here, the method for predicting patient information using tokenized electrocardiogram data of the present disclosure is described as predicting patient information by tokenizing electrocardiogram data and directly inputting the tokenized electrocardiogram data into a patient information prediction model for the purpose of predicting patient information regardless of the number of leads of the electrocardiogram data, but is not limited thereto, and in some cases, in various embodiments, patient information may be predicted by standardizing electrocardiogram data based on tokenized electrocardiogram data and inputting the standardized electrocardiogram data into a patient information prediction model.

[0139] More specifically, first, the computing device (100) can standardize the patient's biometric data to generate standard biometric data. For example, when biometric data is acquired from a patient, the computing device (100) can tokenize the biometric data and input the tokenized biometric data into a biometric data restoration module (e.g., a latent diffusion model) to generate standard biometric data.

[0140] Here, when the biometric data is electrocardiogram data, the standard biometric data may be 12-lead electrocardiogram data of 10 seconds in length.

[0141] Additionally, here, if the number of channels of biometric data matches the number of channels of standard biometric data, the biometric data standardization process may be omitted.

[0142] Thereafter, the computing device (100) can predict patient information by inputting standard biometric data (12-lead electrocardiogram data) generated through the above process into a patient information prediction model.

[0143] That is, by standardizing the electrocardiogram data obtained from the patient into standard electrocardiogram data and analyzing it to predict patient information, patient information can be predicted more accurately regardless of the number of leads of the electrocardiogram data.

[0144] In various embodiments, the computing device (100) can predict patient information by extracting individual heartbeats from the electrocardiogram data when the standard biometric data is electrocardiogram data and individually inputting the individual heartbeats into a patient information prediction model.

[0145] For example, the computing device (100) can obtain individual heartbeats from electrocardiogram data, and by individually inputting each individual heartbeat into a patient information prediction model, can predict multiple unit clinical test results, and can derive a single clinical test result by combining the predicted multiple unit clinical test results.

[0146] As another example, the computing device (100) can obtain individual heartbeats from electrocardiogram data, and individually input each of the individual heartbeats into a patient information prediction model, thereby deriving a plurality of probability values ​​corresponding to the likelihood of occurrence of each of the plurality of heart diseases, and deriving the likelihood of occurrence of each of the plurality of heart diseases based on the plurality of probability values. Hereinafter, with reference to FIGS. 6 to 12, a method for predicting the likelihood of occurrence of a heart disease (e.g., likelihood of occurrence of arrhythmia) as patient information using individual heartbeats will be exemplarily described.

[0147]

[0148] FIG. 6 is a flowchart of a method for learning a patient information prediction model for predicting heart disease according to another embodiment of the present disclosure.

[0149] Here, the method according to FIG. 6 can be performed by the computing device (100) illustrated in FIG. 1.

[0150] Referring to FIG. 6, in step S310, the computing device (100) can classify electrocardiogram data acquired from a plurality of patients into a training data set, a verification data set, and a test data set. First, the electrocardiogram data acquired from a plurality of patients can be classified into groups of true-normal sinus rhythm (T-NSR), atrial fibrillation-normal sinus rhythm (AF-NSR), and clinically important arrhythmia-normal sinus rhythm (CIA-NSR). At this time, the electrocardiogram data acquired from a plurality of patients may be 10-second 12-lead electrocardiogram data, but the type of such electrocardiogram data is only one example and is not limited to the above example.

[0151] Meanwhile, clinically important arrhythmia (CIA) may include atrial arrhythmia, ventricular arrhythmia, atrial fibrillation, and bundle branch block (BBB). Atrial arrhythmia refers to an arrhythmia originating in the atrium that is sustained for more than 30 seconds or non-sustained for less than 30 seconds, and may include atrial premature complex, or sustained / non-sustained atrial rhythm. Ventricular arrhythmia refers to an arrhythmia originating in the ventricle that is sustained for more than 30 seconds or non-sustained for less than 30 seconds, and may include ventricular premature complex, or sustained / non-sustained ventricular arrhythmia. Atrial fibrillation can refer to an arrhythmia characterized by the absence of regular electrical signals and contractions in the atria, resulting in irregular ventricular contractions. Finally, BBB can refer to an arrhythmia characterized by a characteristic electrocardiographic pattern due to blockage of signal transmission in the right or left bundle branch of the heart, which transmits cardiac signals through the ventricles.

[0152] The T-NSR group could be comprised of ECGs from patients with no history of atrial fibrillation or arrhythmia and at least three normal sinus rhythm ECGs per year. The AF-NSR group could be comprised of ECGs from patients with a normal sinus rhythm ECG paired with an atrial fibrillation or atrial flutter ECG that occurred within 14 days of the corresponding normal sinus rhythm ECG. Similarly, the CIA-NSR group could be comprised of ECGs from patients with a normal sinus rhythm ECG paired with an arrhythmia ECG that occurred within 14 days of the corresponding normal sinus rhythm ECG.

[0153] The computing device (100) can classify electrocardiogram data belonging to the T-NSR group, AF-NSR group, and CIA-NSR group, which are thus classified, into a training data set, a verification data set, and a test data set applicable to a deep learning-based patient information prediction model for predicting heart disease.

[0154] FIG. 7 is a diagram illustrating a classification process of electrocardiogram data in various embodiments. As illustrated in FIG. 7, the computing device (100) can classify electrocardiogram data belonging to the T-NSR group, the AF-NSR group, and the CIA-NSR group according to the heart disease to be predicted. For example, the computing device (100) can classify the electrocardiogram data into the T-NSR group and the AF-NSR group for predicting atrial fibrillation, and can classify the electrocardiogram data into the T-NSR group and the CIA-NSR group for predicting arrhythmia.

[0155] Thereafter, the computing device (100) can classify the electrocardiogram data classified by each heart disease into a training data set, a verification data set, and a test data set based on the arbitrary date on which the electrocardiogram data was generated.

[0156] For example, if the electrocardiogram data as shown in FIG. 7 are acquired between 2017-05-23 and 2022-05-23, the computing device (100) can classify the electrocardiogram data before a certain date (e.g., 2021-06-11) into a training data set and a verification data set according to a certain ratio, and classify the electrocardiogram data after the date (including the date) into a test data set. In the example of FIG. 7, the electrocardiogram data are classified into a training data set (60%), a verification data set (20%), and a test data set (20%), but such classification ratio is only one example and is not limited to the above example.

[0157] In step S320, the computing device (100) can obtain individual heartbeats from each of the electrocardiogram data classified into a training data set, a validation data set, and a test data set. The computing device (100) can perform preprocessing on the 10-second 12-lead electrocardiogram data classified into the training data set, the validation data set, and the test data set to obtain accurate and reliable data.

[0158] More specifically, the computing device (100) can receive electrocardiogram data classified into a training data set, a verification data set, and a test data set in the form of an XML (eXtensible Markup Language) file. The computing device (100) can parse the input data into a structured data portion, such as the patient's name, age, and gender, and an unstructured data portion composed of continuous signals. Thereafter, the computing device (100) can perform noise removal preprocessing on the unstructured data portion composed of continuous signals and distinguish key markers of heartbeats. The computing device (100) can obtain discontinuous individual heartbeats from the unstructured data composed of continuous signals through the key markers of the heartbeats distinguished in this way.

[0159] FIG. 8 is a diagram illustrating a process for segmenting electrocardiogram data into individual heartbeats in various embodiments. As illustrated in FIG. 8, a computing device (100) may decode 10-second 12-lead electrocardiogram data using Base64 encryption, and then pass the data through an IIR Butterworth SOS filter with a moving average kernel and a power-line noise filter for noise removal and cleansing. The computing device (100) may then segment the de-noised 10-second 12-lead electrocardiogram data into individual heartbeats using a QRS peak detection algorithm.

[0160] As a result, the computing device (100) can obtain multiple discontinuous individual heartbeats from a single 10-second 12-lead electrocardiogram data as shown in FIG. 7, and the individual heartbeats obtained in this way can be used to learn a patient information prediction model for more accurate heart disease prediction.

[0161] In step S330, the computing device (100) can train the patient information prediction model to predict the heart disease class of each of the first individual heart beats in response to inputting the first individual heart beats obtained from the electrocardiogram data included in the training data set into the patient information prediction model for predicting the heart disease of the patient.

[0162] At this time, the computing device (100) can be trained to predict the heart disease class of each of the first individual heartbeats using any one of the deep learning models of ResNet-18, Conv1D including LSTM (Long Short-Term Memory), and Conv1D including a transformer.

[0163] Figures 9A to 9C are diagrams showing types of patient information prediction models applicable to various embodiments.

[0164] For example, ResNet-18, as shown in Figure 9A, is a deep learning model that can extract essential features from inputs using convolutional operations similar to those used in various convolutional neural networks (CNNs). To address the vanishing gradient problem of CNN architectures, ResNet-18 can perform residual learning through skip connections, as shown in Figure 9A, where input data bypasses multiple layers of the network and is directly connected to the output layer.

[0165] Since the deep learning model of ResNet-18 requires a fixed-length input, the computing device (100) can fix the length of individual heartbeats to the average length of all individual heartbeats. For example, if the average length of all individual heartbeats is 700, the computing device (100) can perform slicing on individual heartbeats having a heartbeat length longer than 700 and zero-padding on individual heartbeats having a heartbeat length shorter than 700 to fix the length to 700.

[0166] As another example, Conv1D with LSTM, as shown in Figure 9B, can capture both local temporal patterns and long-range temporal patterns in sequential data. In this case, the Conv1D layer is adept at detecting local temporal patterns, while the LSTM layer excels at modeling long-range dependencies.

[0167] As another example, Conv1D with a transformer, as shown in Figure 9C, can capture both local patterns and global dependencies in the input data. In this case, the transformer layer is suitable for modeling global dependencies, while the Conv1D layer can be effective for detecting local patterns. Unlike ResNet-18, which has a fixed input length, Conv1D with a transformer has the advantage of accommodating various input sizes.

[0168] FIG. 10 is a diagram illustrating a learning step of a patient information prediction model in various embodiments. Referring to FIG. 10, a computing device (100) can optimize the parameters of a deep learning model using binary cross entropy with log loss and an AdamW optimizer with an initial learning rate of 0.0001. At this time, the binary cross entropy is a loss function that reduces the difference between the predicted result and the actual correct answer in the learning of the deep learning model, and the AdamW optimizer is an algorithm involved in updating the actual deep learning model based on this loss function. The computing device (100) can obtain a probability value for the heart disease class of each of the first individual heartbeats in the range of 0 to 1 by applying a sigmoid function to the output of the deep learning model optimized with the binary cross entropy and the AdamW optimizer.

[0169] In step S340, the computing device (100) can apply the second individual heartbeats obtained from the electrocardiogram data included in the verification data set to the learned patient information prediction model to determine a threshold for distinguishing the heart disease class.

[0170] More specifically, the computing device (100) collects probability values ​​for each of the individual heartbeats separated from the same electrocardiogram data for each of the second individual heartbeats, as shown in FIG. 10, derives a final probability score by averaging the probability values ​​for all the collected second individual heartbeats, and then fine-tunes an optimal threshold for distinguishing a heart disease class based on the derived final probability score.

[0171] At this time, the optimal threshold can be obtained by applying a threshold value between 0 and 1 in increments of 0.01 to achieve the best F1 score on the validation data set for the heart disease class, i.e., the T-NSR and AF-NSR classes or the T-NSR and CIA-NSR classes.

[0172] At this time, the F1 score can be defined as in Equation 1 below.

[0173] <Formula 1>

[0174]

[0175] Here, silver And, silver am.

[0176] Here, (Precision) represents the proportion of correctly classified predictions, which can indicate how accurate the detected results are, i.e. how many of the detected results contain actual objects. (Recall) represents the proportion of true positives that are correctly classified, and is an indicator of how well the model captures real-world objects without missing them. In addition, and means the samples classified as true positive and true negative, respectively, and refers to samples classified as false positive and false negative, respectively.

[0177] The computing device (100) can store the optimal threshold for such a heart disease class together with the weights of the patient information prediction model.

[0178] In various embodiments, the computing device (100) may analyze biometric data from a past predetermined period of time to generate a patient information prediction model that determines whether a heart disease will occur during a future predetermined period of time.

[0179] Here, the biometric data for a given period of time may be data collected through a sensor included in a wearable device, for example, 1-lead electrocardiogram data collected for a first period of time through a smartwatch worn by the user, but is not limited thereto, and various forms of biometric data such as 6-lead electrocardiogram data (e.g., electrocardiogram data collected by wearing electrodes on both the limbs and the chest) or 12-lead electrocardiogram data (e.g., electrocardiogram data collected through a 24-hour Holter or home electrocardiogram machine) may be applied.

[0180] More specifically, first, the computing device (100) can generate learning data. For example, the computing device (100) can generate learning data including biometric data for a predetermined period of time in the past (e.g., 1 hour) and biometric data for a predetermined period of time in the future (e.g., 23 hours), as well as a label indicating whether or not cardiac arrest has occurred.

[0181] At this time, the computing device (100) may perform preprocessing, such as reducing noise through filtering, adjusting data to a certain range through normalization, or adjusting the size of data by adjusting the sampling rate as needed, taking into account that electrocardiogram data contains a lot of noise.

[0182] Thereafter, the computing device (100) can learn a patient information prediction model using the above learning data, thereby analyzing biometric data for a predetermined period of time in the past and constructing a model for determining and predicting whether a heart disease will occur for a predetermined period of time in the future.

[0183] Here, the patient information prediction model may be a model based on at least one of LSTM, GRU (Gated Recurrent Unit), and Transformer, but is not limited thereto.

[0184] For example, if the patient information prediction model is LSTM, it can be implemented to convert 1-lead electrocardiogram data for 1 hour into a sequence format, receive it as input, pass it through an LSTM layer, and then use the output from the last sequence to configure a classifier that predicts whether or not an arrhythmia will occur for the next 23 hours.

[0185] As another example, if the patient information prediction model is GRU, it can be built in a similar way to LSTM, but using GRU layers instead of LSTM layers.

[0186] As another example, in the case of a Transformer model, a patient information prediction model can be implemented by using 1-lead electrocardiogram data for 1 hour as input, extracting features through the encoder structure of the Transformer, and then passing them to a classifier to predict whether or not an arrhythmia will occur for the next 23 hours.

[0187]

[0188] FIG. 11 is a flowchart of a method for predicting the likelihood of occurrence of heart disease as patient information using a learned patient information prediction model according to another embodiment of the present disclosure.

[0189] Here, the method according to FIG. 11 can be performed by the computing device (100) illustrated in FIG. 1.

[0190] Referring to FIG. 11, at step S410, the computing device (100) may receive electrocardiogram data of a patient whose heart disease class is to be predicted, and obtain individual heartbeats from the electrocardiogram data. At this time, the received electrocardiogram data and the obtained individual heartbeats may correspond to the test data set disclosed in FIG. 7.

[0191] In step S420, the computing device (100) can input the acquired individual heartbeats into a patient information prediction model for predicting a patient's heart disease to derive a probability value for each individual heartbeat to be predicted as a specific heart disease class.

[0192] At step S430, the computing device (100) can predict the patient's heart disease using the probability values ​​derived for each individual heartbeat. More specifically, the computing device (100) can classify the patient's heart disease class by comparing the average of the probability values ​​derived for each individual heartbeat with a threshold value.

[0193] FIG. 12 is a diagram illustrating a heart disease prediction step using a patient information prediction model in various embodiments. Referring to FIG. 12 , the computing device (100) can load the weights and threshold values ​​of the learned patient information prediction model and then derive probability values ​​for each individual heartbeat. The computing device (100) can then obtain an average value of the probability values ​​of all individual heartbeats derived in this manner.

[0194] For example, the computing device (100) determines that the T-NSR Logit value, which is the average of the probability values ​​to be judged as T-NSR, is a first threshold ( ) is greater than the second threshold ( CIA-NSR Logit value, which is the average of the probability values ​​to be judged by CIA-NSR). ) is less than that, the patient's heart disease class can be judged as normal, i.e. T-NSR.

[0195] Or the computing device (100) has a T-NSR Logit value of the first threshold ( ) is greater than the CIA-NSR Logit value of the second threshold ( ) is greater than, the T-NSR Logit value and the CIA-NSR Logit value can be compared to select the larger value as the final prediction. For example, if the CIA-NSR Logit value is greater than the T-NSR Logit value, the patient's heart disease class can be determined as CIA-NSR, which is likely to cause arrhythmia.

[0196] Additionally, the computing device (100) has a T-NSR Logit value of a first threshold ( ) is smaller than the CIA-NSR Logit value of the second threshold ( ) is smaller than , the T-NSR Logit value and the CIA-NSR Logit value can be compared and the larger value can be selected as the final prediction.

[0197] Finally, the computing device (100) determines whether the T-NSR Logit value is greater than the first threshold ( ) is smaller than the CIA-NSR Logit value of the second threshold ( ) is greater than that, the patient's cardiac disease class can be judged as CIA-NSR, which is likely to cause arrhythmia.

[0198] The four methods for distinguishing these heart disease classes can equally be applied to distinguishing T-NSR and AF-NSR.

[0199] In various embodiments, the computing device (100) may extract individual heartbeats from the electrocardiogram data when the biometric data obtained from the patient is electrocardiogram data including individual heartbeats of the patient, and may generate a plurality of unit input data by tokenizing each of the individual heartbeats.

[0200] Thereafter, the computing device (100) can input each of the plurality of unit input data generated by tokenizing each of the individual heartbeats into the patient information prediction model, thereby deriving a plurality of probability values ​​corresponding to each of the plurality of unit input data, and can predict the patient's heart disease based on the derived plurality of probability values.

[0201] In various embodiments, the computing device (100) can analyze biometric data for a predetermined period of time in the past and use a patient information prediction model that determines and predicts whether a heart disease will occur during a predetermined period of time in the future to determine / predict whether a heart disease will occur during a predetermined period of time in the future by analyzing biometric data for a predetermined period of time in the past.

[0202] More specifically, the computing device (100) can input 1-lead electrocardiogram data collected through the user's wearable device for 1 hour into a patient information prediction model, thereby extracting one or more features from the 1-lead electrocardiogram data through the patient information prediction model, and based on the extracted one or more features, can determine / predict whether a heart disease will occur for the next 23 hours.

[0203] Here, the one or more features may include, but are not limited to, at least one of an R-peak, a QRS complex, and an ST segment.

[0204]

[0205] The above-described method may be provided as a computer program stored on a computer-readable recording medium for execution on a computer. The medium may be one that continuously stores a computer-executable program or one that temporarily stores it for execution or download. In addition, the medium may be various recording means or storage means in the form of a single or multiple hardware combinations, and is not limited to a medium directly connected to a computer system, but may also be distributed over a network. Examples of the medium may include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and those configured to store program instructions, including ROM, RAM, and flash memory. In addition, examples of other media may include recording or storage media managed by app stores that distribute applications, sites that supply or distribute various software, servers, etc.

[0206] The methods, operations, or techniques of the present disclosure may be implemented by various means. For example, these techniques may be implemented in hardware, firmware, software, or a combination thereof. Those skilled in the art will appreciate that the various exemplary logical blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein may be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability of hardware and software, various exemplary components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software will depend on the particular application and the design requirements imposed on the overall system. Those skilled in the art may implement the described functionality in various ways for each particular application, but such implementations should not be construed as departing from the scope of the present disclosure.

[0207] In a hardware implementation, the processing units used to perform the techniques may be implemented within one or more ASICs, DSPs, digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, electronic devices, other electronic units designed to perform the functions described herein, a computer, or a combination thereof.

[0208] Accordingly, the various exemplary logical blocks, modules, and circuits described in connection with the present disclosure may be implemented or performed by any combination of a general-purpose processor, a DSP, an ASIC, an FPGA or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or those designed to perform the functions described herein. A general-purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration.

[0209] In a firmware and / or software implementation, the techniques may be implemented as instructions stored on a computer-readable medium, such as random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable PROM (EEPROM), flash memory, a compact disc (CD), a magnetic or optical data storage device, etc. The instructions may be executable by one or more processors and may cause the processor(s) to perform certain aspects of the functionality described herein.

[0210] When implemented in software, the techniques described above may be stored on or transmitted as one or more instructions or code on a computer-readable medium. Computer-readable media includes both computer storage media and communication media, including any medium that facilitates transfer of a computer program from one place to another. Storage media may be any available media that can be accessed by a computer. By way of example, and not limitation, such computer-readable media can include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer. Also, any connection is properly termed a computer-readable medium.

[0211] For example, if the software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, digital subscriber line, or wireless technologies such as infrared, radio, and microwave are included within the definition of media. Disk and disc, as used herein, includes compact discs, laser discs, optical discs, digital versatile discs (DVDs), floppy disks, and Blu-ray discs, where disks usually reproduce data magnetically, whereas discs reproduce data optically using lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0212] A software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium may be coupled to the processor such that the processor can read information from, and write information to, the storage medium. Alternatively, the storage medium may be integral to the processor. The processor and the storage medium may reside in an ASIC. The ASIC may reside in a user terminal. Alternatively, the processor and the storage medium may reside as discrete components in the user terminal.

[0213] While the embodiments described above have been described as utilizing aspects of the presently disclosed subject matter in one or more standalone computer systems, the present disclosure is not limited thereto and may be implemented in conjunction with any computing environment, such as a network or distributed computing environment. Furthermore, aspects of the present disclosure may be implemented in multiple processing chips or devices, and storage may be similarly affected across multiple devices. Such devices may include personal computers, network servers, and portable devices.

[0214] While the present disclosure has been described in connection with certain embodiments herein, various modifications and variations may be made without departing from the scope of the present disclosure, which would be apparent to those skilled in the art. Furthermore, such modifications and variations are intended to fall within the scope of the claims appended to this specification.

Claims

1. A method for predicting patient information using tokenized electrocardiogram data performed by a computing device, Step of obtaining patient's bio-data; A step of generating normalized input data in the form of tokens by tokenizing the acquired biometric data; and A step of inputting the generated input data into a pre-trained deep learning-based patient information prediction model to predict patient information about the patient, A method for predicting patient information using tokenized electrocardiogram data.

2. In paragraph 1, The above acquired biometric data is, N-lead electrocardiogram data measured by an electrocardiogram measuring device containing N electrodes, A method for predicting patient information using tokenized electrocardiogram data.

3. In paragraph 1, The steps for generating the above input data are: A step of converting the acquired biometric data into one or more tokens by tokenizing the acquired biometric data through a pre-learned tokenization module, The above-mentioned pre-trained tokenization module, An encoder that extracts features from biometric data; A codebook storing a plurality of tokens defining features that can be extracted from biometric data; and A vector quantization unit that converts features extracted through the encoder based on multiple tokens stored in the codebook into one or more tokens in vector form. A method for predicting patient information using tokenized electrocardiogram data.

4. In paragraph 1, The steps for generating the above input data are: If the acquired biometric data is electrocardiogram data, a step of generating input data by converting the electrocardiogram data into one or more tokens in proportion to the length of electrocardiogram information included in the electrocardiogram data is included. A method for predicting patient information using tokenized electrocardiogram data.

5. In paragraph 1, The steps for generating the above input data are: A step of generating a plurality of unit biometric data by dividing the biometric signals included in the acquired biometric data so that at least some areas overlap, when the time length of the acquired biometric data exceeds the reference time length; A step of tokenizing each of the plurality of generated unit biometric data; and A step of generating one input data by collating the above tokenized plurality of unit biometric data, A method for predicting patient information using tokenized electrocardiogram data.

6. In paragraph 1, The steps for generating the above input data are: If the number of channels of the acquired biometric data is different from the reference channel number, a step of preprocessing the acquired biometric data so that the number of channels of the acquired biometric data becomes the reference channel number; and A step of generating input data by tokenizing the above preprocessed biometric data is included. The step of preprocessing the acquired biometric data is as follows: Comprising a step of comparing the number of channels of the acquired biometric data with the number of reference channels, and adding one or more channels filled with masking values equal to the number of channels lacking compared to the number of reference channels to the acquired biometric data. A method for predicting patient information using tokenized electrocardiogram data.

7. In paragraph 1, The step of predicting the above patient information is: By inputting the generated input data into the patient information prediction model, a step of predicting the clinical test results for the patient as patient information is included. A method for predicting patient information using tokenized electrocardiogram data.

8. In paragraph 1, The step of predicting the above patient information is: By inputting the generated input data into the patient information prediction model, a step of predicting the possibility of occurrence of each of a plurality of different heart diseases as patient information is included. A method for predicting patient information using tokenized electrocardiogram data.

9. A method for generating a tokenization module for generating tokenized electrocardiogram data for predicting patient information performed by a computing device, A step of generating training data by masking at least a portion of the biometric data; A step of generating a learning artificial intelligence model that restores original biometric data corresponding to at least a portion of the masked data based on the generated learning data by training a non-learning artificial intelligence model using the generated learning data; and A step of generating a tokenization module using the above-mentioned generated learning artificial intelligence model is included. The tokenization module generated above is: A module that generates input data in the form of a token set containing one or more tokens by tokenizing input biometric data. A method for creating a tokenization module that generates tokenized electrocardiogram data for predicting patient information.

10. In paragraph 9, The above non-learning artificial intelligence model is, An encoder that extracts features from input biometric data; A codebook storing multiple tokens defining features that can be extracted from biometric data; A vector quantization unit that converts features extracted through the encoder into one or more tokens in vector form based on multiple tokens stored in the codebook; and A decoder for restoring original biometric data based on one or more tokens converted through the vector quantization unit; The steps for creating the above learning artificial intelligence model are: A step of performing pre-learning on the codebook and vector quantization unit of the non-learning artificial intelligence model using the generated learning data; and A step of performing fine tuning on the encoder and decoder of the non-learning artificial intelligence model and the pre-learned vector quantization unit using the generated learning data while fixing a plurality of tokens included in the pre-learned codebook. A method for creating a tokenization module that generates tokenized electrocardiogram data for predicting patient information.

11. In paragraph 10, The steps for performing the above pre-learning are: A step of initializing the codebook of the above non-learning artificial intelligence model; A step of converting features extracted from an encoder of the non-learning artificial intelligence model into one or more tokens based on tokens included in the initialized codebook through a vector quantization unit of the non-learning artificial intelligence model; and A step of pre-training the initialized codebook so that a first loss function is minimized based on one or more tokens converted from the vector quantization unit of the non-learning artificial intelligence model, and a step of pre-training the vector quantization unit of the non-learning artificial intelligence model so that a second loss function is minimized based on one or more tokens converted from the vector quantization unit of the non-learning artificial intelligence model, The above first loss function is, The vector quantization part of the above non-learning artificial intelligence model is a function that calculates the degree of imbalance in the distribution of tokens used for conversion. The second loss function is, A function that calculates the error of token conversion performed by the vector quantization part of the above non-learning artificial intelligence model. A method for creating a tokenization module that generates tokenized electrocardiogram data for predicting patient information.

12. In paragraph 11, The steps for performing the above fine tuning are: A step of extracting features from the generated learning data through an encoder of the above non-learning artificial intelligence model; A step of converting the extracted feature into one or more tokens based on a plurality of tokens included in the pre-learned codebook through the pre-learned vector quantization unit; A step of restoring biometric data based on one or more tokens converted from the pre-learned vector quantization unit through a decoder included in the above non-learning artificial intelligence model; and Based on the above restored biometric data, a step of fine-tuning the encoder and decoder of the non-learning artificial intelligence model and the pre-learned vector quantizer so that the third loss function is minimized, The third loss function is, A function that calculates the error between the original biometric data corresponding to the generated learning data and the restored biometric data, A method for creating a tokenization module that generates tokenized electrocardiogram data for predicting patient information.

13. In paragraph 10, The above generated learning artificial intelligence model is, comprising the fine-tuned encoder, the pre-learned codebook, the fine-tuned vector quantizer and the fine-tuned decoder, The steps for creating the above tokenization module are: A step of generating a tokenization module by modularizing the fine-tuned encoder, the pre-learned codebook and the fine-tuned vector quantizer, A method for creating a tokenization module that generates tokenized electrocardiogram data for predicting patient information.

14. In paragraph 9, The steps for generating the above learning data are: If the number of channels of the biometric data is less than the reference number of channels, a step of preprocessing the biometric data by adding one or more channels filled with a preset value to the biometric data so that the number of channels of the biometric data becomes the reference number of channels; A step of generating learning data using the above preprocessed biometric data is included. A method for creating a tokenization module that generates tokenized electrocardiogram data for predicting patient information.

15. In paragraph 9, The steps for generating the above learning data are: A step of performing preprocessing on the biometric data by adjusting the time length of the biometric data so that the time length of the biometric data becomes a multiple of the reference time length; and A step of generating learning data using the above preprocessed biometric data is included. A method for creating a tokenization module that generates tokenized electrocardiogram data for predicting patient information.

16. Processor; network interface; memory; and A computer program loaded into the above memory and executed by the above processor, The above processor, A computing device that performs the method of claim 1 or 9 by executing one or more instructions included in the computer program.

17. Combined with a computing device, A computer program stored on a recording medium readable by a computing device to execute the method of claim 1 or 9.

Images