Method, device, and program for generating diagnostic information about user by analyzing biometric data

Abstract

The present invention provides a method, a computer device, and a program for generating diagnostic information about a user by analyzing biometric data. A method according to one embodiment disclosed herein comprises the steps of: extracting feature information from acquired biometric data about a user, and identifying the state of the user on the basis of the feature information; acquiring context data for assisting in explaining the state of the user from a pre-constructed database on the basis of the feature information and the state of the user; and generating a prompt on the basis of the feature information, the state of the user, and the context data, and providing the prompt to a language model to generate diagnostic information about the state of the user.

Description

Method, device and program for generating user diagnostic information by analyzing biometric data

[0001] The present disclosure relates to deep learning technology in the medical field, and more specifically, to a method, device, and program for analyzing biometric data to generate diagnostic information for a user.

[0002] With the advancement of electronic communication, the medical field has continuously developed technologies that assess a user's health status based on biosignals and automatically analyze the presence of disease. Electrocardiogram (ECG) data, in particular, is a key indicator for the early detection of heart disease and is widely utilized in various clinical settings. However, existing analysis technologies have focused on quantifying biosignals or simply classifying them into specific states based on established criteria. This has made it difficult to fully reflect the complex context required in clinical practice, and has limited the ability to provide explanations of analysis results in natural language.

[0003] To overcome these limitations, active efforts have recently been made to utilize large language models (LLMs) in diagnostic applications. LLMs possess the ability to understand and generate diverse text data formats, making them an effective tool for enhancing the explainability of biosignal analysis results by generating diagnostic explanations in natural language.

[0004] One approach to utilizing LLMs has been to fine-tune them to generate explanations tailored to specific disease groups or diagnostic tasks. However, fine-tuning LLMs requires large, high-quality medical datasets, and building and managing such datasets presents significant challenges. Furthermore, fine-tuned models are often over-optimized for specific purposes, making them difficult to apply to other diseases or clinical situations. Therefore, a new approach is needed that leverages pre-trained, large-scale language models to efficiently generate natural language diagnostic explanations without the need for separate fine-tuning.

[0005] The present disclosure has been made in response to the aforementioned background technology, and aims to provide a method, device, and program for generating user diagnostic information by analyzing biometric data.

[0006] However, the problems to be solved in this disclosure are not limited to the problems mentioned above, and other problems not mentioned can be clearly understood based on the description below.

[0007] According to an embodiment of the present disclosure for realizing the task described above, a method for generating diagnostic information of a user by analyzing biometric data, performed by a computing device including at least one processor, includes the steps of extracting feature information from acquired biometric data of the user and determining a state of the user based on the feature information, obtaining context data that assists in explaining the state of the user from a pre-built database based on the feature information and the state of the user, and generating a prompt based on the feature information, the state of the user, and the context data, and providing the prompt to a language model to generate diagnostic information regarding the state of the user.

[0008] Alternatively, the above-mentioned pre-built database includes a plurality of first text information regarding each of a plurality of characteristic information extractable from the biometric data and a plurality of second text information generated by an expert regarding a plurality of states that can be identified through the biometric data, wherein the first text information and the second text information can each be generated by an expert.

[0009] Alternatively, the step of obtaining context data for assisting in describing the user's status from the above-mentioned pre-built database may include the step of selecting third text information corresponding to the feature information from among the plurality of first text information based on a similarity between the text information corresponding to the feature information and each of the first text information, the step of selecting fourth text information corresponding to the user's status from among the plurality of second text information based on a similarity between the text information corresponding to the user's status and each of the second text information, and the step of obtaining the third text information and the fourth text information as the context data.

[0010] Alternatively, the step of selecting the third text information may include a step of identifying a similarity between the text information corresponding to the feature information and each of the first text information based on a distance between a first vector obtained by embedding the text information corresponding to the feature information and a plurality of second vectors obtained by embedding each of the first text information.

[0011] Alternatively, the step of selecting the fourth text information may include a step of identifying a similarity between the text information corresponding to the user's state and each of the second text information based on a distance between a third vector obtained by embedding the text information corresponding to the user's state and a plurality of fourth vectors obtained by embedding each of the second text information.

[0012] Alternatively, the third text information may be composed of a plurality of texts each describing the plurality of characteristic information, and the fourth text information may be composed of a plurality of texts each describing the plurality of states of the user.

[0013] Alternatively, the step of obtaining context data to assist in describing the user's status may include the step of identifying first characteristic information that determines the user's status among the characteristic information, and the step of obtaining first context data corresponding to the first characteristic information and second context data corresponding to the user's status from the constructed database.

[0014] Alternatively, the step of generating diagnostic information about the user's status may include the step of generating a prompt based on the first characteristic information, the user's status, the first context data, and the second context data.

[0015] Alternatively, the step of determining the user's status may include a step of inputting the biometric data into a pre-learned neural network model to extract the feature information from the biometric data, and a step of determining the user's status based on the feature information using a rule-based model.

[0016] Alternatively, the biometric data may include electrocardiogram data, and the step of extracting the feature information may include the step of extracting lead-specific feature information of the electrocardiogram data and global feature information of the electrocardiogram data, and the step of obtaining the context data may include the step of obtaining context data that assists in explaining the user's status from a pre-built database using at least one of the lead-specific feature information and the global feature information as a query.

[0017] According to an embodiment of the present disclosure for realizing the task described above, a computing device for generating user diagnostic information by analyzing biometric data includes a processor including at least one core and a memory including program codes executable by the processor, wherein the processor acquires user biometric data, extracts feature information from the acquired user biometric data, determines the user's status based on the feature information, acquires context data that assists in explaining the user's status from a pre-built database based on the feature information and the user's status, generates a prompt based on the feature information, the user's status, and the context data, and provides the prompt to a language model, thereby generating diagnostic information regarding the user's status.

[0018] According to an embodiment of the present disclosure for realizing the above-described task, a computer program stored in a computer-readable storage medium, wherein the computer program, when executed on one or more processors, performs operations for analyzing biometric data to generate user diagnostic information, wherein the operations may include an operation for extracting feature information from acquired user biometric data and determining a user's status based on the feature information, an operation for obtaining context data that assists in explaining the user's status from a pre-built database based on the feature information and the user's status, and an operation for generating a prompt based on the feature information, the user's status, and the context data, and providing the prompt to a language model, thereby generating diagnostic information regarding the user's status.

[0019] According to the method of generating user diagnostic information by analyzing biometric data of the present disclosure, a prompt is generated based on feature information extracted from the biometric data and user status information, and this is provided to a Large Language Model (LLM) to generate diagnostic information in natural language. This allows for the provision of descriptive diagnostic information that is easy for medical professionals to understand and utilize.

[0020] In addition, since the present invention can generate diagnostic explanations by directly utilizing pre-learned LLM, it can flexibly respond to various clinical situations without separate fine-tuning, thereby reducing the burden of securing medical data and the cost of maintaining and operating the model.

[0021] FIG. 1 is a block diagram of a computing device that analyzes biometric data to generate user diagnostic information according to one embodiment of the present disclosure.

[0022] FIG. 2 is a flowchart schematically illustrating a method for generating user diagnostic information by analyzing biometric data according to one embodiment of the present disclosure.

[0023] FIG. 3 is an exemplary diagram illustrating a method for obtaining context data based on feature information and state information according to one embodiment of the present disclosure.

[0024] FIG. 4 is an exemplary diagram illustrating a method for obtaining context data based on feature information and state information according to one embodiment of the present disclosure.

[0025] FIG. 5 is a detailed configuration diagram of a computing device according to another embodiment of the present disclosure.

[0026] Below, embodiments of the present disclosure are described in detail with reference to the attached drawings so that those skilled in the art can easily implement the present disclosure. The embodiments presented in this disclosure are provided to enable those skilled in the art to utilize or implement the contents of the present disclosure. Accordingly, various modifications to the embodiments of the present disclosure will be apparent to those skilled in the art. That is, the present disclosure may be implemented in various different forms and is not limited to the embodiments described below.

[0027] Throughout the specification of this disclosure, identical or similar drawing numbers refer to identical or similar components. Furthermore, for the purpose of clearly describing the disclosure, drawing numbers for parts in the drawings that are not relevant to the description of the disclosure may be omitted.

[0028] The term "or" as used herein is intended to mean an inclusive "or" rather than an exclusive "or." That is, unless otherwise specified herein or clear from context, "X employs A or B" should be understood to mean either of the natural inclusive permutations. For example, unless otherwise specified herein or clear from context, "X employs A or B" can be interpreted to mean either X employs A, X employs B, or X employs both A and B.

[0029] The term "and / or" as used herein should be understood to refer to and include all possible combinations of one or more of the related concepts listed.

[0030] The terms "comprises" and / or "comprising" as used herein should be understood to mean the presence of certain features and / or components. However, it should be understood that the terms "comprises" and / or "comprising" do not exclude the presence or addition of one or more other features, other components, and / or combinations thereof.

[0031] Unless otherwise specified in this disclosure or unless the context makes it clear that the singular form is intended to be referred to, the singular should generally be construed to include “one or more.”

[0032] The term "Nth (N is a natural number)" used in the present disclosure can be understood as an expression used to mutually distinguish components of the present disclosure based on a predetermined standard such as a functional perspective, a structural perspective, or convenience of explanation. For example, components performing different functional roles in the present disclosure can be distinguished as a first component or a second component. However, components that are substantially the same within the technical spirit of the present disclosure but must be distinguished for convenience of explanation may also be distinguished as a first component or a second component.

[0033] The term "acquisition" as used in this disclosure may be understood to mean not only receiving data through a wired or wireless communication network with an external device or system, but also generating data in an on-device form.

[0034] Meanwhile, the term "module" or "unit" used in the present disclosure can be understood as a term referring to an independent functional unit that processes computing resources, such as a computer-related entity, firmware, software or a part thereof, hardware or a part thereof, or a combination of software and hardware. At this time, the "module" or "unit" may be a unit composed of a single element, or a unit expressed as a combination or set of multiple elements. For example, as a narrow concept, a "module" or "unit" may refer to a hardware element of a computing device or a set thereof, an application program that performs a specific function of software, a processing process implemented through software execution, or a set of instructions for program execution, etc. In addition, as a broad concept, a "module" or "unit" may refer to the computing device itself that constitutes the system, or an application running on the computing device, etc. However, since the above-described concept is only an example, the concept of “module” or “part” may be defined in various ways within a range understandable to those skilled in the art based on the contents of the present disclosure.

[0035] The term "model" as used herein may be understood as a system implemented using mathematical concepts and language to solve a specific problem, a set of software units to solve a specific problem, or an abstract model of a processing process to solve a specific problem. For example, a neural network "model" may refer to the entire system implemented as a neural network that has problem-solving capabilities through learning. In this case, the neural network can have problem-solving capabilities by optimizing the parameters connecting nodes or neurons through learning. A neural network "model" may include a single neural network or a set of neural networks that are a combination of multiple neural networks.

[0036] The term "data" used in this disclosure may include "image," "signal," and the like. The term "image" used in this disclosure may refer to multidimensional data composed of discrete image elements. In other words, "image" may be understood as a term referring to a digital representation of an object visible to the human eye. For example, "image" may refer to multidimensional data composed of elements corresponding to pixels in a two-dimensional image. "Image" may refer to multidimensional data composed of elements corresponding to voxels in a three-dimensional image.

[0037] The explanation of the above terms is intended to aid understanding of the present disclosure. Therefore, unless explicitly stated as limiting the contents of the present disclosure, it should be noted that the above terms are not intended to limit the technical ideas of the contents of the present disclosure.

[0038] FIG. 1 is a block diagram of a computing device that analyzes biometric data to generate user diagnostic information according to one embodiment of the present disclosure.

[0039] A computing device (100) according to an embodiment of the present disclosure may be a hardware device or a part of a hardware device that performs comprehensive processing and calculation of data, or may be a software-based computing environment connected to a communication network. For example, the computing device (100) may be a server that performs intensive data processing functions and shares resources, or may be a client that shares resources through interaction with a server. In addition, the computing device (100) may be a cloud system in which multiple servers and clients interact to comprehensively process data. Since the above description is only one example related to the type of computing device (100), the type of computing device (100) may be configured in various ways within a category understandable to those skilled in the art based on the contents of the present disclosure. For example, the computing device (100) may be implemented as various electronic devices such as a server, a desktop, a laptop, a smartphone, a smart watch, a smart ring, etc.

[0040] Referring to FIG. 1, a computing device (100) according to an embodiment of the present disclosure may include a processor (110), a memory (120), and a communication interface (unit) (130). However, FIG. 1 is merely an example, and the computing device (100) may include other components for implementing a computing environment. In addition, only some of the disclosed components may be included in the computing device (100).

[0041] A processor (110) according to an embodiment of the present disclosure may be understood as a configuration unit including hardware and / or software for performing computing operations. For example, the processor (110) may read a computer program to perform data processing for machine learning. The processor (110) may process computational processes such as processing input data for machine learning, feature extraction for machine learning, and error calculation based on backpropagation. The processor (110) for performing such data processing may include a central processing unit (CPU), a general purpose graphics processing unit (GPGPU), a tensor processing unit (TPU), an application specific integrated circuit (ASIC), or a field programmable gate array (FPGA). The above-described type of processor (110) is only one example, and thus, the type of processor (110) may be configured in various ways within a range understandable to those skilled in the art based on the contents of the present disclosure.

[0042] The processor (110) is connected to other components of the computing device (100) (i.e., memory (120) and communication interface (130)) and controls the overall operation of the computing device (100).

[0043] The memory (120) according to one embodiment of the present disclosure may be understood as a configuration unit including hardware and / or software for storing and managing data processed in the computing device (100). That is, the memory (120) may store any type of data generated or determined by the processor (110) and any type of data received by the communication interface (130). For example, the memory (120) may include at least one type of storage medium among a flash memory type, a hard disk type, a multimedia card micro type, a card type memory, a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, and an optical disk. In addition, the memory (120) may also include a database system that controls and manages data in a predetermined system. The type of memory (120) described above is only one example, and thus the type of memory (120) can be configured in various ways within a range understandable to those skilled in the art based on the contents of the present disclosure.

[0044] The memory (120) can structure and organize and manage data, combinations of data, and program codes executable by the processor (110) required for the processor (110) to perform operations. For example, the memory (120) can store bio-data (aortic blood pressure data, brain wave data, and electrocardiogram data) received through a communication interface (130) to be described later. In addition, the memory (120) can store a neural network model learned to determine the severity of a patient or a neural network model learned to determine the responsiveness of a patient to a possibility of a disease, and can store program codes that operate to perform learning of each neural network model, program codes that operate the neural network model to receive bio-data (aortic blood pressure data, brain wave data, and electrocardiogram data, etc.) and perform inference according to the purpose of use of the computing device (100), and processed data generated as the program codes are executed.

[0045] A communication interface (130) according to an embodiment of the present disclosure may be understood as a component that transmits and receives data through any known wired or wireless communication system. For example, the communication interface (130) may perform data transmission and reception using a wired or wireless communication system such as a local area network (LAN), wideband code division multiple access (WCDMA), long term evolution (LTE), wireless broadband internet (WiBro), fifth generation mobile communication (5G), ultrawide-band, ZigBee, radio frequency (RF) communication, wireless LAN, wireless fidelity, near field communication (NFC), or Bluetooth. Since the above-described communication systems are only examples, the wired and wireless communication system for data transmission and reception of the communication interface (130) may be applied in various ways other than the above-described examples.

[0046] The communication interface (130) can receive data necessary for the processor (110) to perform calculations through wired or wireless communication with any system or any client, etc. In addition, the communication interface (130) can transmit data generated through calculations of the processor (110) through wired or wireless communication with any system or any client, etc. For example, the communication interface (130) can receive a patient's biometric data through communication with a cloud server that performs tasks such as standardization of databases and medical data in a hospital environment, or a computing device, etc. The communication interface (130) can transmit output data of a neural network model, intermediate data derived from the calculation process of the processor (110), processed data, etc. through communication with the aforementioned database, server, or computing device, etc.

[0047] FIG. 2 is a flowchart schematically illustrating a method for generating user diagnostic information by analyzing biometric data according to one embodiment of the present disclosure.

[0048] Referring to FIG. 2, according to one embodiment of the present disclosure, the processor extracts characteristic information from the acquired user's biometric data and determines the user's status based on the characteristic information (S210). Specifically, the processor (110) may acquire the user's biometric data from a biometric signal measuring device (e.g., a smartwatch worn by the user) that is connected to the computing device (100) via the communication interface of the computing device.

[0049] In addition, the processor (110) may directly obtain biometric data from the user through a sensing unit included in the computing device. For example, the biometric data may include electrocardiogram data. In this case, the processor may detect an electrocardiogram signal from the user as multiple electrodes included in the sensing unit are attached to the user's body, and digitally process the detected electrocardiogram signal (e.g., A / D conversion, etc.) to obtain the user's electrocardiogram data. Depending on the measurement method using the multiple electrodes, the processor (110) may obtain various forms of electrocardiogram data, such as 1 lead, 3 leads, 6 leads, and 12 leads.

[0050] Meanwhile, the processor (110) is not limited to the above-described examples, and can obtain various biometric data from the user in addition to electrocardiogram data. For example, the processor (110) can obtain various biometric data such as the user's body temperature, photoreceptor blood flow, heart rate, and electroencephalogram data.

[0051] The processor (110) can extract feature information from biometric data. Here, the feature information may include quantitative or standard information that can be identified from a biometric signal corresponding to the biometric data. In addition, the feature information may include potential unique information (patterns, waveform trends, etc.) of the biometric data. For example, when the biometric data is electrocardiogram data, the feature information may include at least one feature point identifiable on the electrocardiogram signal and a parameter calculated through the feature point. Specifically, the feature information may include the amplitude of the P wave, the time length of the QRS complex, the potential change of the ST segment, the shape of the T wave, the RR interval, etc.

[0052] In addition, the processor (110) can analyze the extracted feature information to determine the user's condition. The processor (110) can determine the user's condition by obtaining a comparison result (e.g., amount of change, etc.) with the value of the extracted feature information or the biometric data acquired at a previous time (specifically, the user's past biometric data acquired at a previous time and stored in memory or feature information extracted from the past biometric data). Here, the user's condition may include the presence or absence of a specific disease (e.g., left ventricular systolic dysfunction, myocardial infarction, etc.), the degree of progression of the disease, etc., and in addition to the disease, may include general health conditions such as stress, fatigue, sleep status, exercise responsiveness, etc.

[0053] According to one embodiment of the present disclosure, the processor may input biometric data into a pre-trained neural network model to extract feature information from the biometric data in order to determine the user's status, and may determine the user's status based on the feature information using a rule-based model.

[0054] The processor may utilize a pre-trained neural network model to extract feature information contained or latent in biometric data. Here, the pre-trained neural network model may be a model trained to extract feature information from biometric data. The neural network model may be trained to identify and extract feature information contained in each biometric data based on a training data set composed of multiple biometric data of the same type. To this end, the training data set may include multiple label data in which feature information contained in each biometric data is labeled. The processor (110) may input training data (multiple biometric data included in the training data set) into a first neural network model, and may calculate a loss function based on the difference between the output value of the first neural network model during the training process and the label data corresponding to each biometric data. The loss function may be defined as a cross entropy loss or an objective function that optimizes the balance between precision and recall. Based on the calculated loss function, the processor (110) can adjust the weights of the model through backpropagation. By repeating this process, the processor (110) can improve the classification performance of the neural network model with respect to feature information in the biometric data, and ultimately obtain a first neural network model trained to extract feature information from electrocardiogram data. Meanwhile, the feature information may include multiple feature types that are defined differently and parameters or numerical values ​​for each feature type. That is, when the biometric data is electrocardiogram data, the neural network model can produce multiple values ​​for multiple feature types, such as the amplitude value of the P wave, the time length value of the QRS complex, etc.

[0055] Meanwhile, the biosignals corresponding to the biodata included in the learning data used to train the neural network model may have a commonly preset length. Accordingly, when the processor (110) acquires biodata from a patient, the processor (110) may divide the electrocardiogram signal corresponding to the bioelectrocardiogram data into a preset length or divide the electrocardiogram signal into multiple segments by applying a window of a preset length on the time axis. For example, in the case of electrocardiogram data, the electrocardiogram signal may be a signal measured at a length of 10 seconds through 12 leads, each measured at 500 points per second. Meanwhile, the electrocardiogram signal may be a limb lead electrocardiogram signal and a single lead electrocardiogram signal.

[0056] Neural network models can be implemented as multi-layer perceptrons (MLPs), convolutional neural networks (CNNs), recurrent neural networks (RNNs), and generative adversarial networks (GANs).

[0057] Meanwhile, the neural network model may include multiple first sub-neural network models classified according to the type of biometric data. For example, the neural network model may include a first-first sub-neural network model that extracts feature information from electrocardiogram (ECG) data, a first-second sub-neural network model that extracts feature information from electroencephalogram (EEG) data, and the like.

[0058] After extracting at least one feature information from biometric data using a pre-trained neural network model, the processor can identify the user's status based on the at least one feature information extracted using a rule-based model. Here, the rule-based model may be a model that classifies the user's status by comparing the extracted feature information with one or more pre-defined judgment criteria or thresholds corresponding to the status type, according to one or more pre-defined user status types. For example, the rule-based model may include a conditional judgment structure (If-Then rule), a threshold-based comparison, a rule-based classification structure such as a decision tree, or a rule-based ensemble structure that performs a judgment by combining multiple rules.

[0059] At this time, for each condition type defined in the rule-based model, the type and number of feature information compared with the judgment criteria may be different. In addition, the processor can simultaneously identify multiple condition types of the user by inputting multiple feature information extracted into the acquired rule-based model. For example, if the feature information is extracted from an electrocardiogram signal, the rule-based model can simultaneously identify conditions such as atrial premature contraction, complete atrioventricular block, and atrial fibrillation by determining whether values ​​such as the PR interval, QRS width, and QT interval exceed or fall below a specific standard. The processor can input quantitative feature information derived from the neural network model, compare it with a set of conditions defined in the rule-based model, and determine that the user has a specific disease or specific health condition if the conditions are satisfied.

[0060] In this way, by combining the feature information extraction step through a neural network model and the condition judgment step through a rule-based model, the processor can simultaneously secure quantitative accuracy and medical interpretability, and provide users with the basis and logic of the diagnostic process beyond simple classification results.

[0061] According to one embodiment of the present disclosure, a processor may extract lead-specific feature information of electrocardiogram data and global feature information of electrocardiogram data. According to one embodiment of the present disclosure, the processor may extract lead-specific feature information of electrocardiogram data and global feature information of electrocardiogram data. Specifically, lead-specific feature information is a feature value independently calculated for each of 12 electrocardiogram leads, and may include information such as PR interval, width of QRS complex, potential of ST segment, amplitude of R wave and T wave, and presence or absence of T wave inversion. Global feature information is an index calculated by synthesizing the entire electrocardiogram signal or the entire lead, and may include numerical information such as heart rate, average RR interval, QT interval or corrected QT interval (QTc), potential average of the entire signal, and the like. By extracting such lead-specific feature information and global feature information together, the processor enables precise analysis and multifaceted judgment of the user's status.

[0062] Here, the lead-specific feature information may be feature information independently extracted from the electrocardiogram data corresponding to each individual lead when electrocardiogram data is acquired through multiple leads (e.g., 12 leads). For example, the lead-specific feature information may be a quantitative index calculated based on the time-voltage waveform of each lead. The lead-specific feature information may include the PR interval, the time length of the QRS complex, the amplitude of the R wave, the presence or absence of T wave inversion, and the change in the potential of the ST segment. The global feature information may be a quantitative index calculated by integrating and analyzing multiple electrocardiogram data corresponding to all leads when electrocardiogram data is acquired through multiple leads. For example, it may be a feature reflecting the overall physiological state of the patient's electrocardiogram signal. The global feature information may include a numerical value calculated based on the interrelationship between leads or the time series pattern across the entire lead, such as the heart rate, the mean RR interval, and the QT interval. Since the nature of these feature informations is different, the processor may analyze the lead-specific feature information and the global feature information.

[0063] According to one embodiment of the present disclosure, a processor may extract lead-specific feature information and global feature information through different neural network models. To this end, lead-specific feature information may be learned based on a training data set composed of electrocardiogram data corresponding to each lead, and global feature information may be learned based on a training data set composed of multiple electrocardiogram data as a single input data. In this regard, the description of the above-described neural network model training method may be equally applied. In addition, the neural network model for extracting lead-specific feature information may include a plurality of second sub-neural network models corresponding to each lead. In this case, each second sub-neural network model may be individually trained based on a training data set composed of electrocardiogram data acquired through the corresponding lead.

[0064] Additionally, the processor can configure separate processing paths for each type of feature information within a single integrated neural network model. For example, lead-specific feature information can be extracted through a neural network (e.g., a convolutional neural network (CNN)) independently applied to the ECG data of each lead. In this case, temporal characteristics of the waveforms of each lead can be detected to individually calculate values ​​such as QRS width and R wave amplitude. Global feature information can be extracted through a neural network (e.g., a recurrent neural network (RNN), a long short-term memory (LSTM) neural network, or a Transformer-based architecture) that reflects the temporal relationships and overall patterns across all leads. These models learn the temporal and spatial correlations of the entire signal, thereby supporting the precise derivation of global physiological indicators such as heart rate and QTc.

[0065] FIG. 3 and FIG. 4 are exemplary diagrams showing a method for obtaining context data based on feature information and state information according to one embodiment of the present disclosure.

[0066] Referring back to FIG. 2, according to one embodiment of the present disclosure, the processor (110) may extract feature information and then, based on the extracted feature information and the user's status, obtain context data that assists in explaining the user's status from the established database (300) (S220). Here, the context data is information related to the feature information or status, and may be reference information for supplementing or assisting in the explanation or judgment of the user's feature information and status. For example, the context data may include descriptive text of biometric data (10) written by a medical domain expert (doctor, medical field expert, etc.), descriptive text of each type of feature information, descriptive text of each type of condition, a document containing diagnostic criteria or interpretation guidelines for a specific disease, etc. In addition to text, the context data may also include images or diagrams containing medical descriptions of feature information and condition types, or visual examples of electrocardiogram waveforms.

[0067] Meanwhile, according to an embodiment of the present disclosure, a database (300) constructed may include a plurality of text information (hereinafter, first text information) regarding each of a plurality of characteristic information extractable from biometric data (10) and a plurality of text information (hereinafter, second text information) generated by an expert regarding a plurality of states that can be identified through the biometric data (10). In this case, the first text information and the second text information may each be generated by an expert. The first and second text information may include tags, metadata, or text information defining an image, such as an image, a diagram, or a visual example.

[0068] Referring to FIG. 3, the processor (110) may obtain context data as reference information (or auxiliary information) for generating diagnostic information about the user based on the characteristic information and the user's status. In particular, the processor (110) may use at least one of the characteristic information and the user's status to search for context data that assists in explaining the characteristic information extracted from the user's biometric data (10) within the database (300), or may search for context data that assists in explaining the identified user's status.

[0069] In particular, the processor (110) can obtain context data that assists in explaining the user's status from a pre-built database (300) using feature information (e.g., lead-specific feature information and global feature information, etc.) and status as queries. For example, referring to FIG. 4, the processor (110) uses a pre-trained neural network model (20) to determine, based on the user's feature information, that the width of the QRS complex is extended to 130 ms, the QTc interval exceeds a reference value, and a left bundle branch block (LBBB) pattern is detected in leads V1 to V6, and then, using a rule-based model (30), the processor can determine the user's status as Left Ventricular Systolic Dysfunction (LVSD). At this time, the processor (110) can obtain keywords (e.g., QRS width extension, LBBB pattern, QTc interval extension) corresponding to the feature information based on the feature information. These keywords may be converted according to predefined judgment criteria or templates, or corresponding descriptive phrases may be selected based on a mapping table. Alternatively, the processor (110) may obtain keywords corresponding to the feature information using a natural language generation model. The processor (110) may use keywords of the generated feature information as queries to search for text information (hereinafter, third text information) defining or explaining each feature information among a plurality of first text information within the pre-built database (300). In addition, the processor (110) may use names of simultaneously identified conditions (e.g., left ventricular systolic dysfunction or LVSD) as queries to search for text information (hereinafter, fourth text information) defining the corresponding disease or including explanations such as diagnostic criteria or medical opinions among a plurality of second text information within the pre-built database (300).

[0070] Meanwhile, according to one embodiment of the present disclosure, the processor (110) may select third text information corresponding to the feature information from among a plurality of first text information based on the similarity between the text information corresponding to the feature information and each first text information, and may select fourth text information corresponding to the user's state from among a plurality of second text information based on the similarity between the text information corresponding to the user's state and each second text information. In addition, the processor (110) may obtain the third text information and the fourth text information as context data.

[0071] Specifically, the similarity may be an indicator indicating the degree of semantic or contextual similarity between keywords of feature information or status information used as a query (e.g., “QRS interval prolongation” and “left ventricular systolic dysfunction”) and a plurality of text information (specifically, a plurality of first and second text information) stored in the database (300). For example, when the keyword “QTc interval prolongation” is used as a query, the processor (110) may search for sentences or documents containing content semantically similar to the keyword among the first text information. At this time, the similarity may be determined not only by whether words are directly identical, but also by including expressions that have clinically similar meanings.

[0072] Meanwhile, according to one embodiment of the present disclosure, the processor (110) may identify the similarity between the text information corresponding to the feature information and each of the first text information based on the distance between the first vector obtained by embedding text information corresponding to the feature information and the plurality of second vectors obtained by embedding each of the first text information in order to select the third text information. In addition, the processor (110) may identify the similarity between the text information corresponding to the user's state and each of the second text information based on the distance between the third vector obtained by embedding text information corresponding to the user's state and the plurality of fourth vectors obtained by embedding each of the second text information in order to select the fourth text information. The embedding process may be performed through a vectorization technique based on natural language processing, and for example, a pre-trained model such as BERT, Sentence-BERT, or Word2Vec may be used. The processor (110) may vectorize the first text information and the second text information using the natural language processing-based embedding technique and store them in a preset vector space. The processor (110) can generate a first vector and a third vector by embedding a query (e.g., a keyword) corresponding to feature information and state information in a preset vector space, and can calculate a similarity by comparing the first vector and the fourth vector with a plurality of second vectors and a plurality of fourth vectors corresponding to a plurality of first text information and a plurality of second text information, respectively. The processor (110) can calculate a cosine similarity or a Euclidean distance based on the distance or angle between these vectors, and can select an item with the highest similarity among the first text information as the third text information and an item with the highest similarity among the second text information as the fourth text information based on the similarity.The selected third and fourth text information can be utilized as context data to supplement the interpretation or explanation of the feature information and status information.

[0073] Meanwhile, according to one embodiment of the present disclosure, the processor (110) may identify first feature information that determines the user's status from among a plurality of feature information acquired through a pre-learned neural network model (20), and may acquire first context data corresponding to the identified first feature information and second context data corresponding to the user's status from the constructed database (300). Specifically, when a plurality of feature information are extracted from the biometric data (10) through the pre-learned neural network model (20), the processor (110) may selectively acquire first context data corresponding to one or more feature information (i.e., the first feature information) that substantially contributed to determining the user's status. At this time, the user's status may be a status type determined as abnormal compared to a normal status from among a plurality of pre-defined status types, and the processor (110) may acquire second context data that supports an explanation or interpretation of the corresponding abnormal status.

[0074] Referring again to FIG. 2, according to one embodiment of the present disclosure, the processor (110) may generate a prompt based on feature information, the user's status, and context data, and provide the prompt to a language model (40) to generate diagnostic information about the user's status (S230).

[0075] Specifically, the language model (40) may be a large language model (LLM) (Large Language Model, LLM), which may be a pre-trained text generation model capable of understanding and generating various types of natural language data. For example, the language model (40) may include OpenAI's GPT, Google's PaLM, Meta's LLaMA series, etc. The processor (110) may input a prompt to the language model (40) to output a diagnostic description or report for the user containing medical meaning regarding the user's condition in natural language form. In particular, the processor (110) may input a prompt to the language model (40) to obtain an output in the form of a diagnostic report that interprets or explains the user's condition. The prompt may include extracted feature information, determined condition information, and context data related thereto (e.g., third and fourth text information), and may be configured so that the language model (40) may generate appropriate diagnostic information. The processor (110) may include instructions to guide the language model (40) to clinically interpret the prompt. For example, an instruction such as "Based on the following ECG characteristics, diagnostic results, and contextual information, write a medical description of the patient's condition" may be included at the top of the prompt, with structured input information arranged below it.

[0076] According to one embodiment of the present disclosure, the processor (110) may additionally include instructions regarding the output format within the prompt to clearly specify the response format of the language model (40). For example, the processor may request that the diagnostic results be returned in JSON format so that the response generated by the language model (40) can be utilized in a structured form, or a template including specific fields (e.g., diagnosis name, definition, ECG basis, clinical implications, etc.) may be pre-specified. Through this, the generated diagnostic report can be provided not only as a human-understandable natural language description, but also in a structured data format suitable for machine reading and subsequent processing.

[0077] FIG. 5 is a detailed configuration diagram of a computing device (600) according to another embodiment of the present disclosure. Referring to FIG. 5, a computing device (600) according to an embodiment of the present disclosure includes a processor (610), a memory (620), a communication interface (630), a display (640), a user interface (650), a sensing unit (660), a camera (670), a speaker (680), and a microphone (690). Among the configurations illustrated in FIG. 6, a detailed description of configurations that overlap with those illustrated in FIG. 1 will be omitted.

[0078] The display (640) can display various images and texts. Here, the images include both still images and moving images. The display (640) can also output electrocardiogram data and electrocardiogram signals corresponding to the electrocardiogram data (e.g., electrocardiogram graphs), and can also output diagnostic information acquired through a language model. The display (640) can be implemented as various types of displays, such as an LCD (Liquid Crystal Display Panel), an OLED (Organic Light Emitting Diodes), an LCoS (Liquid Crystal on Silicon), a DLP (Digital Light Processing), etc. In addition, the display (640) can also include a driving circuit, a backlight unit, etc., which can be implemented in a form such as an a-si TFT, an LTPS (low temperature poly silicon) TFT, an OTFT (organic TFT), etc. Meanwhile, the display (640) can be implemented as a touch screen by being combined with a touch panel.

[0079] The display (640) can function not only as an output interface that outputs images via a touch screen, but also as an input interface that receives a user's touch input. The display (640) can display extracted feature information, identified user status, generated prompts, etc.

[0080] The user interface (650) is a configuration used by the computing device (600) to perform interaction with the user, and may include, but is not limited to, at least one of a touch sensor, a motion sensor, a button, a jog dial, and a switch.

[0081] The sensing unit (660) senses the patient's bio-signals to acquire bio-data. For example, the sensing unit (660) may detect the electrical signals of the patient's heartbeat through multiple electrodes, thereby acquiring the patient's electrocardiogram (ECG) data. Alternatively, the sensing unit (660) may include an optical sensor, and may acquire the patient's optical blood flow data through the optical sensor.

[0082] The camera (670) can capture images of the user (1) by photographing objects around the user. Based on the captured images, the processor (610) can determine the user's biological information (height, age, gender, etc.) and include it in the prompt. The camera (670) can be implemented with an imaging device such as an imaging device having a CMOS structure (CIS, CMOS Image Sensor) or an imaging device having a CCD structure (Charge Coupled Device). However, the present invention is not limited thereto, and the camera (670) can be implemented with a camera module having various resolutions capable of capturing an object. Meanwhile, the camera (670) can be implemented with a depth camera (e.g., an IR depth camera), a stereo camera, an RGB camera, etc.

[0083] The speaker (680) is a component that outputs various audio data that have undergone various processing operations, such as decoding, amplification, and noise filtering, by an audio processing unit (not shown). The speaker (680) can output various notification sounds or voice messages. According to one embodiment of the present disclosure, the processor (610) can convert an electrical signal into a user's (1) voice and output it through the speaker (680). As an example, the speaker (680) can output diagnostic information in audio form.

[0084] The microphone (690) can receive the user's voice. The processor (610) can acquire the user's voice to modify or generate a prompt through the microphone (690).

[0085] Meanwhile, according to one embodiment of the present disclosure, a non-transitory computer-readable medium storing a program that performs a method of generating user diagnostic information by analyzing the aforementioned biometric data may be provided. Here, the non-transitory computer-readable medium refers to a medium that semi-permanently stores data and can be read by a device, rather than a medium that stores data for a short period of time, such as a register, cache, or memory. Specifically, the various applications or programs described above may be stored and provided on a non-transitory computer-readable medium, such as a CD, DVD, hard disk, Blu-ray disc, USB, memory card, or ROM.

[0086] The various embodiments of the present disclosure described above can be combined with additional embodiments and modified within a scope that can be understood by those skilled in the art in light of the detailed description set forth above. It should be understood that the embodiments of the present disclosure are illustrative in all respects and not restrictive. For example, each component described as being single may be implemented in a distributed manner, and similarly, components described as being distributed may be implemented in a combined form. Accordingly, all changes or modifications derived from the meaning, scope, and equivalent concepts of the claims of the present disclosure should be interpreted as being included within the scope of the present disclosure.

Claims

1. A method for generating user diagnostic information by analyzing biometric data, performed by a computing device including at least one processor, A step of extracting characteristic information from the acquired user's biometric data and determining the user's status based on the characteristic information; A step of obtaining context data that assists in explaining the user's status from a pre-built database based on the above characteristic information and the user's status; and A step of generating a prompt based on the above characteristic information, the user's status, and the context data, and providing the prompt to a language model to generate diagnostic information about the user's status; method.

2. In paragraph 1, The above-mentioned database is, Includes a plurality of first text information regarding each of a plurality of feature information extractable from the biometric data and a plurality of second text information generated by an expert regarding a plurality of states that can be identified through the biometric data, The first text information and the second text information are each generated based on text written by a domain expert. method.

3. In paragraph 2, The step of obtaining context data to assist in explaining the user's status from the above-mentioned pre-built database is as follows: A step of selecting third text information corresponding to the feature information from among the plurality of first text information based on the similarity between the text information corresponding to the feature information and each first text information; A step of selecting fourth text information corresponding to the user's status from among the plurality of second text information based on the similarity between the text information corresponding to the user's status and each second text information; and A step of obtaining the third text information and the fourth text information as the context data; including; method.

4. In paragraph 3, The third step of selecting text information is: A step of identifying the similarity between the text information corresponding to the feature information and each of the first text information based on the distance between the first vector obtained by embedding the text information corresponding to the feature information and the plurality of second vectors obtained by embedding each of the first text information; including; method.

5. In paragraph 3, The fourth step of selecting text information is: A step of identifying the similarity between the text information corresponding to the user's state and each of the second text information based on the distance between the third vector obtained by embedding the text information corresponding to the user's state and the plurality of fourth vectors obtained by embedding the respective second text information; including; method.

6. In paragraph 3, The third text information is composed of a plurality of texts each describing the plurality of characteristic information, and the fourth text information is composed of a plurality of texts each describing the plurality of states of the user. method.

7. In paragraph 1, The step of obtaining context data to assist in describing the status of the above user is as follows: A step of identifying first characteristic information that determines the user's status among the above characteristic information, and a step of obtaining first context data corresponding to the first characteristic information and second context data corresponding to the user's status from the constructed database, method.

8. In paragraph 7, The step of generating diagnostic information about the status of the above user is: A step of generating a prompt based on the first feature information, the user's status, the first context data, and the second context data; method.

9. In paragraph 1, The steps to determine the status of the above user are: A step of inputting the biometric data into a pre-learned neural network model to extract the feature information from the biometric data; and A step of identifying the user's status based on the characteristic information using a rule-based model; method.

10. In paragraph 9, The above biometric data is, Contains electrocardiogram data, The step of extracting the above feature information is: A step of extracting lead-specific feature information of the electrocardiogram data and global feature information of the electrocardiogram data; The step of obtaining the above context data is: A step of obtaining context data that assists in explaining the user's status from a pre-built database using at least one of the lead-specific feature information and global feature information as a query, method.

11. In a computing device that analyzes biometric data to generate user diagnostic information, a processor comprising at least one core; and a memory including program codes executable by the processor; The above processor, Obtaining a user's biometric data, extracting feature information from the acquired user's biometric data, identifying the user's status based on the feature information, obtaining context data that assists in explaining the user's status from a pre-built database based on the feature information and the user's status, generating a prompt based on the feature information, the user's status, and the context data, and providing the prompt to a language model to generate diagnostic information regarding the user's status. Computing device.

12. A computer program stored in a computer-readable storage medium, wherein the computer program, when executed on one or more processors, performs operations for analyzing biometric data to generate user diagnostic information. The above actions are, An operation of extracting characteristic information from the acquired user's biometric data and determining the user's status based on the characteristic information; An operation of obtaining context data that assists in explaining the user's status from a pre-built database based on the above characteristic information and the user's status; and An operation of generating a prompt based on the above characteristic information, the user's status, and the context data, and providing the prompt to a language model to generate diagnostic information about the user's status; Computer program.

Images