Device, method, and program for acquiring neural network model trained to identify state of user on basis of plurality of training data sets

Abstract

The present disclosure provides a device, a method, and a program by which a computing device including at least one processor acquires a neural network model trained to identify the state of a user on the basis of a plurality of training data sets. The method according to one embodiment of the present disclosure comprises the steps of: acquiring a first training data set and a second training data set on the basis of a plurality of pieces of biometric data; and training a neural network model on the basis of the first training data set and the second training data set so as to identify the state of a user on the basis of the biometric data.

Description

Device, method and program for acquiring a neural network model trained to identify a user's status based on multiple learning data sets

[0001] The present disclosure relates to deep learning technology in the medical field, and to a device, method, and program for acquiring a neural network model trained to identify a user's condition based on a plurality of learning data sets.

[0002] Myocardial infarction is a cardiovascular disease that causes the sudden interruption of blood flow to the heart, resulting in the death of the heart muscle. If prompt action is not taken, myocardial infarction can lead to death. Therefore, early detection of the possibility of myocardial infarction or the early prediction of potential myocardial infarction is essential for improving patient survival and prognosis. An electrocardiogram (ECG) is a biosignal that records the electrical activity of the heart, and its analysis can detect or predict the onset of myocardial infarction. However, conventional ECG analysis relies primarily on the interpretation and interpretation of specialists or skilled personnel, limiting its efficiency and accuracy. In particular, analysis results can vary depending on the skill level of the personnel, making it difficult to ensure consistent accuracy. Furthermore, in situations requiring rapid diagnosis, timely and appropriate responses are hindered.

[0003] Recent advancements in information and communication technology (ICT) and artificial intelligence (AI) have led to the active use of AI to analyze various medical data, including electrocardiograms (ECGs). Analyzing users' ECG data using AI models can automatically determine or predict the onset of myocardial infarction, which is expected to improve efficiency and speed compared to existing methods that rely on human expertise. However, AI-based ECG analysis methods still have limitations in terms of accuracy, and in some cases, they fail to provide sufficient performance for reliable prediction or early detection of myocardial infarction. Therefore, there is a growing need for a more accurate and reliable ECG-based method for predicting and analyzing myocardial infarction.

[0004] The present disclosure is conceived in response to the aforementioned background technology, and aims to provide a device, method, and program for acquiring a neural network model trained to identify a user's status based on a plurality of learning data sets.

[0005] However, the problems to be solved in this disclosure are not limited to the problems mentioned above, and other problems not mentioned can be clearly understood based on the description below.

[0006] A method for obtaining a neural network model trained to identify a user's status based on a plurality of learning data sets, which is performed by a computing device including at least one processor for solving a task as described above, includes the steps of obtaining a first learning data set and a second learning data set based on a plurality of biometric data, and the step of training a neural network model based on the first learning data set and the second learning data set to identify a user's status based on the biometric data.

[0007] Alternatively, the step of obtaining the first learning data set and the second learning data set may include the step of obtaining the first learning data set by assigning a first label to the plurality of biometric data according to whether the subject of the plurality of biometric data has the first disease, and the step of obtaining the second learning data set by assigning a second label to the plurality of biometric data according to the second disease of the subject of the plurality of biometric data.

[0008] Alternatively, the second disease may be a first type of the first disease.

[0009] Alternatively, the neural network model may include a first encoder that extracts first feature information corresponding to a first disease from the plurality of biometric data, and a second encoder that extracts second feature information corresponding to a first type of the first disease based on the first feature information extracted from the first encoder.

[0010] Alternatively, the neural network model may further include a classifier directly connected to the second encoder and connected to the first encoder via a skip connection.

[0011] Alternatively, the step of training the neural network model may include a step of training the neural network model based on the first learning data set, and when training based on the first learning data set is completed, a step of retraining the neural network model based on the second learning data set.

[0012] Alternatively, the biometric data may include electrocardiogram data, the first disease may include myocardial infarction, and the first type of the first disease may include ST-segment elevation myocardial infarction (STEMI).

[0013] A computing device for acquiring a neural network model trained to identify a user's status based on a plurality of learning data sets for solving a task as described above includes a processor including at least one core and a memory including program codes executable in the processor, wherein the processor acquires a first learning data set and a second learning data set based on a plurality of biometric data, and trains a neural network model based on the first learning data set and the second learning data set to identify a user's status based on the biometric data.

[0014] A computer program stored in a computer-readable storage medium for solving the aforementioned problem, wherein the computer program, when executed on one or more processors, performs an operation of acquiring a neural network model trained to identify a user's status based on a plurality of learning data sets, the operation including an operation of acquiring a first learning data set and a second learning data set based on a plurality of biometric data, and an operation of training a neural network model based on the first learning data set and the second learning data set to identify a user's status based on the biometric data.

[0015] According to one embodiment of the present disclosure, the accuracy of disease prediction can be improved by stepwise training a neural network model to extract general disease-related features and features specific to a specific type of disease based on biometric data.

[0016] FIG. 1 is an exemplary diagram of a method for identifying a user's status using a neural network model according to an embodiment of the present disclosure.

[0017] FIG. 2 is a block diagram of a computing device that acquires a neural network model trained to identify a user's status based on a plurality of learning data sets according to an embodiment of the present disclosure.

[0018] FIG. 3 is a flowchart of a method for obtaining a neural network model (500) trained to identify a user's status based on multiple learning data sets according to an embodiment of the present disclosure.

[0019] FIG. 4 is an exemplary diagram of a method for obtaining multiple learning data sets according to one embodiment of the present disclosure.

[0020] FIG. 5 is a diagram illustrating the structure of a neural network model according to an embodiment of the present disclosure.

[0021] FIG. 6 is a block diagram of a computing device according to another embodiment of the present disclosure.

[0022] Below, embodiments of the present disclosure are described in detail with reference to the attached drawings so that those skilled in the art can easily implement the present disclosure. The embodiments presented in this disclosure are provided to enable those skilled in the art to utilize or implement the contents of the present disclosure. Accordingly, various modifications to the embodiments of the present disclosure will be apparent to those skilled in the art. That is, the present disclosure may be implemented in various different forms and is not limited to the embodiments described below.

[0023] Throughout the specification of this disclosure, identical or similar drawing numbers refer to identical or similar components. Furthermore, for clarity in the description of this disclosure, drawing numbers for parts unrelated to the description of this disclosure may be omitted in the drawings.

[0024] The term "or" as used herein is intended to mean an inclusive "or" rather than an exclusive "or." That is, unless otherwise specified herein or clear from context, "X employs A or B" should be understood to mean either of its natural inclusive permutations. For example, unless otherwise specified herein or clear from context, "X employs A or B" can be interpreted to mean either X employs A, X employs B, or X employs both A and B.

[0025] The term "and / or" as used herein should be understood to refer to and include all possible combinations of one or more of the related concepts listed.

[0026] The terms "comprises" and / or "comprising" as used herein should be understood to mean the presence of certain features and / or components. However, it should be understood that the terms "comprises" and / or "comprising" do not exclude the presence or addition of one or more other features, other components, and / or combinations thereof.

[0027] Unless otherwise specified in this disclosure or unless the context makes it clear that the singular form is being referred to, the singular should generally be construed to include “one or more.”

[0028] The term "Nth (N is a natural number)" used in this disclosure can be understood as an expression used to mutually distinguish components of this disclosure based on a predetermined standard such as a functional perspective, a structural perspective, or convenience of explanation. For example, components performing different functional roles in this disclosure can be distinguished as a first component or a second component. However, components that are substantially the same within the technical spirit of this disclosure but must be distinguished for convenience of explanation may also be distinguished as a first component or a second component.

[0029] The term "acquisition" as used in the present disclosure may be understood to mean not only receiving data through a wired or wireless communication network with an external device or system, but also generating data in an on-device form.

[0030] Meanwhile, the term "module" or "unit" used in the present disclosure can be understood as a term referring to an independent functional unit that processes computing resources, such as a computer-related entity, firmware, software or a part thereof, hardware or a part thereof, or a combination of software and hardware. At this time, the "module" or "unit" may be a unit composed of a single element, or a unit expressed as a combination or set of multiple elements. For example, as a narrow concept, a "module" or "unit" may refer to a hardware element or a set thereof of a computing device, an application program that performs a specific function of software, a processing process implemented through software execution, or a set of instructions for program execution, etc. In addition, as a broad concept, a "module" or "unit" may refer to the computing device itself that constitutes the system, or an application running on the computing device, etc. However, since the above-described concept is only an example, the concept of “module” or “part” may be defined in various ways within a range understandable to those skilled in the art based on the contents of the present disclosure.

[0031] The term "model" as used herein may be understood as a system implemented using mathematical concepts and language to solve a specific problem, a set of software units to solve a specific problem, or an abstract model of a processing process to solve a specific problem. For example, a neural network "model" may refer to the entire system implemented as a neural network that has problem-solving capabilities through learning. In this case, the neural network can have problem-solving capabilities by optimizing parameters connecting nodes or neurons through learning. A neural network "model" may include a single neural network or a set of neural networks that are a combination of multiple neural networks.

[0032] The term "data" used in this disclosure may include "images," signals, and the like. The term "image" used in this disclosure may refer to multidimensional data composed of discrete image elements. In other words, "image" may be understood as a term referring to a digital representation of an object visible to the human eye. For example, "image" may refer to multidimensional data composed of elements corresponding to pixels in a two-dimensional image. "Image" may refer to multidimensional data composed of elements corresponding to voxels in a three-dimensional image.

[0033] The explanation of the above terms is intended to aid understanding of the present disclosure. Therefore, unless explicitly stated as limiting the contents of the present disclosure, it should be noted that the above terms are not intended to limit the technical ideas of the contents of the present disclosure.

[0034] FIG. 1 is an exemplary diagram of a method for identifying a user's status using a neural network model according to an embodiment of the present disclosure.

[0035] A computing device (100) according to an embodiment of the present disclosure may be a hardware device or a part of a hardware device that performs comprehensive processing and calculation of data, or may be a software-based computing environment connected via a communication interface. For example, the computing device (100) may be a server that performs intensive data processing functions and shares resources, or may be a client that shares resources through interaction with a server. In addition, the computing device (100) may be a cloud system in which a plurality of servers and clients interact to comprehensively process data. For example, the computing device (100) may be a cloud server device that is connected to a plurality of clients and processes biometric data collected from the plurality of clients, or integrates and analyzes calculation results performed by the clients. Alternatively, the computing device (100) may be a device (e.g., a smartphone, a smartwatch, a laptop, a tablet PC, a desktop, etc.) that obtains biometric data of a user (1) and analyzes it on-device or can be linked to an external cloud system for analysis.

[0036] Since the above description is only one example related to the type of computing device (100), the type of computing device (100) can be configured in various ways within a range understandable to a person skilled in the art based on the contents of the present disclosure.

[0037] According to one embodiment of the present disclosure, a computing device (100) may obtain biometric data from a user (1) and analyze the obtained biometric data using a pre-trained neural network model (500) to determine the state of the user (1). The state of the user (1) may include the possibility of the user's (1) disease (particularly, the possibility of heart disease), stress index, or the possibility of an abnormal health symptom. The computing device (100) may obtain the biometric data of the user (1) by directly measuring it, or may receive biometric data obtained by an external biometric signal measuring device (200).

[0038] Biometric data may include electrocardiography (ECG) data, photoplethysmography (PPG) data, electroencephalogram (EEG) data, electromyogram (EMG) data, photoplethysmography (PPG), etc.

[0039] A computing device (100) can input biometric data obtained from a user (1) into a pre-trained neural network model (500) to obtain a score corresponding to the state of the user (1). The neural network model (500) can be implemented in a multi-layer perceptron (MLP), convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks, or residual neural networks (ResNet) structure.

[0040] The computing device (100) can obtain a score corresponding to the likelihood of a specific disease from a pre-trained neural network model (500). The score corresponding to the likelihood of a disease can be interpreted as an indicator indicating the likelihood of the user (1) having a disease or the likelihood of the disease developing in the future. The computing device (100) can determine the status of the user (1) regarding a specific disease by comparing the obtained score with a reference value. In this case, the higher the score, the higher the likelihood of the user (1) having a disease.

[0041] The computing device (100) can identify the risk group of the user (1) with respect to a specific disease by comparing the acquired score with a reference value. Specifically, the computing device (100) can classify the status of the user (1) with respect to the possibility of the user's (1) disease into a preset grade by comparing the acquired score with the reference value. Here, the preset grade may be a classification of the severity of the user's (1) disease. For example, in the case of the heart disease being myocardial infarction, if the acquired score is less than a first reference value, the status of the user (1) is determined to be normal, if the acquired score is equal to or greater than the first reference value and less than a second reference value, the status of the user (1) is determined to be low-risk, and if the acquired score is equal to or greater than the second reference value, the status of the user (1) is determined to be high-risk.

[0042] To this end, the computing device (100) can pre-train the neural network model (500) based on a learning data set consisting of a plurality of biometric data. In particular, the computing device (100) can train the neural network model (500) to analyze biometric data using the learning data set to predict the likelihood of a specific disease of the user (1) and to produce a score corresponding to the likelihood of the specific disease.

[0043] Hereinafter, embodiments of the present disclosure related thereto will be described.

[0044] FIG. 2 is a block diagram of a computing device (100) that acquires a neural network model (500) trained to identify a state of a user (1) based on a plurality of learning data sets according to an embodiment of the present disclosure.

[0045] Referring to FIG. 2, a computing device (100) includes a processor (110) (hereinafter, processor (110)) including at least one core and a memory (120). However, FIG. 2 is only an example, and thus, the computing device (100) may further include other configurations for implementing a computing environment.

[0046] A processor (110) according to an embodiment of the present disclosure may be understood as a configuration unit including hardware and / or software for performing computing operations. For example, the processor (110) may read a computer program to perform data processing for machine learning. The processor (110) may process computational processes such as processing input data for machine learning, feature extraction for machine learning, and error calculation based on backpropagation. The processor (110) for performing such data processing may include a central processing unit (CPU), a general purpose graphics processing unit (GPGPU), a tensor processing unit (TPU), an application specific integrated circuit (ASIC), or a field programmable gate array (FPGA). The above-described type of processor (110) is only one example, and thus, the type of processor (110) may be configured in various ways within a range understandable to those skilled in the art based on the contents of the present disclosure.

[0047] The processor (110) can acquire training data required for training the neural network model (500). Specifically, the computing device (100) can acquire biometric data from a plurality of subjects to configure the training data. As described above, the biometric data may include electrocardiography (ECG) data, photoplethysmography (PPG) data, electroencephalogram (EEG) data, electromyogram (EMG) data, etc. However, for the convenience of explanation of the present disclosure, the biometric data will be described below assuming electrocardiogram data (10). After acquiring training data, the processor (110) can train the neural network model (500) based on the acquired training data.

[0048] The processor (110) is electrically connected to other components of the computing device (100) (e.g., memory (120), etc.) and controls the overall operation of the computing device (100).

[0049] The memory (120) according to one embodiment of the present disclosure may be understood as a configuration unit including hardware and / or software for storing and managing data processed in the computing device (100). That is, the memory (120) may store any type of data generated or determined by the processor (110) and any type of data received by the network unit of the computing device (100). For example, the memory (120) may include at least one type of storage medium among a flash memory type, a hard disk type, a multimedia card micro type, a card type memory, a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory (120), a magnetic disk, and an optical disk. Additionally, the memory (120) may include a database system that controls and manages data in a predetermined manner. The types of memory (120) described above are merely examples, and thus, the types of memory (120) may be configured in various ways within a range understandable to those skilled in the art based on the contents of the present disclosure.

[0050] The memory (120) can structure and organize and manage data, combinations of data, and program codes executable in the processor (110) required for the processor (110) to perform operations. For example, the memory (120) can store a neural network model (500) and a learning data set used to train the neural network model (500), which will be described later. In addition, the memory (120) can store program codes for performing supervised learning on the neural network model (500) based on the learning data set or for operating the neural network model (500) for which learning has been completed, program codes for operating the neural network model (500) to receive electrocardiogram data (10) and perform inference according to the purpose of use of the computing device (100) as the program codes are executed, and processed data generated as the program codes are executed.

[0051] FIG. 3 is a flowchart of a method for obtaining a neural network model (500) trained to identify a state of a user (1) based on a plurality of learning data sets according to an embodiment of the present disclosure.

[0052] Referring to FIG. 3, the processor obtains a first learning data set and a second learning data set based on a plurality of biometric data (S310).

[0053] The processor (110) can acquire multiple biometric data. As described above, the biometric data can be acquired as electrocardiogram data (10). The processor (110) can acquire multiple electrocardiogram data (10) measured from multiple subjects.

[0054] Specifically, the processor (110) may acquire a plurality of electrocardiogram data (10) acquired by another external electronic device (e.g., a biosignal measuring device (200)) through a communication interface or may acquire a plurality of electrocardiogram data (10) stored in a memory (120). Here, the plurality of electrocardiogram data (10) includes a plurality of electrocardiogram data (10) acquired from different objects. The plurality of electrocardiogram data (10) may be electrocardiogram data (10) that is not assigned a label, such as specific class information.

[0055] At this time, according to one embodiment of the present disclosure, the processor (110) may obtain a first learning data set by assigning a first label to the plurality of biometric data depending on whether the subject of the plurality of biometric data has a first disease, and may obtain a second learning data set by assigning a second label to the plurality of biometric data depending on whether the subject has a second disease. The first disease and the second disease may be heart diseases, and may include myocardial infarction, left ventricular systolic dysfunction, left ventricular diastolic dysfunction, angina pectoris, heart failure, arrhythmia, atrial fibrillation, ischemic heart disease, etc.

[0056] The first and second diseases can each be defined as different diseases. Specifically, the second disease can be a subtype of the first disease. The subtypes can be categorized based on the severity or presentation of the first disease. For example, if the first disease is myocardial infarction, the second disease can be ST-elevation myocardial infarction (STEMI). Alternatively, if the first disease is left ventricular systolic dysfunction (LVSD), the second disease can be one of the stage-specific subtypes, such as mild systolic dysfunction or severe systolic dysfunction. Additionally, if the first disease is left ventricular diastolic dysfunction, the second disease can be any of Grade I (Impaired Relaxation), Grade II (Pseudonormal Filling), or Grade III (Restrictive Filling).

[0057] The processor (110) can obtain a plurality of learning data sets by assigning labels set according to different criteria to each of the acquired plurality of biometric data. The processor (110) can determine whether a subject has a specific disease (i.e., a first disease) and assign a first label set according to the presence or absence of the specific disease to obtain a first learning data set. In addition, the processor (110) can determine whether the subject corresponds to a specific disease subtype (i.e., a first type of the first disease) and assign a second label set according to the presence or absence of the specific disease subtype to obtain a second learning data set. That is, the processor (110) can determine whether the subject has a first disease of the first type and assign a second label. In this way, the processor (110) assigns a label to the first learning data set based on the occurrence of the disease in relation to the same disease, and assigns a label to the second learning data set based on the occurrence of a specific type of disease, thereby performing labeling from different viewpoints on the same biometric data, and can generate multiple learning data sets (the first learning data set and the second learning data set) distinguished according to the criteria for each labeling.

[0058] FIG. 4 is an exemplary diagram of a method for obtaining multiple learning data sets according to one embodiment of the present disclosure.

[0059] According to one embodiment of the present disclosure, the first disease includes myocardial infarction, and a first type of the first disease may include ST-segment elevation myocardial infarction (STEMI).

[0060] Specifically, referring to FIG. 4, the processor (110) can obtain a first learning data set by determining whether the subject has suffered from myocardial infarction for each of a plurality of electrocardiogram data (10') and assigning a first label according to the determination result. In addition, the processor (110) can additionally determine whether the subject has suffered from ST-segment elevation myocardial infarction (STEMI) among the electrocardiogram data (10') assigned a first label, and assign a second label according to the determination result to form a second learning data set. For example, in the case of the first learning data set, the electrocardiogram data (10') of the subject with myocardial infarction can be assigned a label 1, and the electrocardiogram data (10') of the subject without myocardial infarction, i.e., a normal subject, can be assigned a label 0. On the other hand, in the case of the second learning data set, the electrocardiogram data (10) of a subject with ST-segment elevation myocardial infarction may be assigned a label 1, and the electrocardiogram data (10) of a subject without ST-segment elevation myocardial infarction may be assigned a label 0. Here, in the case of no ST-segment elevation myocardial infarction, not only the electrocardiogram data (10') of a normal subject, but also the electrocardiogram data (10') of a subject with another type of myocardial infarction without ST-segment elevation may be included.

[0061] And, the processor (110) trains the neural network model (500) to identify the status of the user (1) based on the biometric data based on the first learning data set and the second learning data set (S320).

[0062] For example, the processor (110) may train a neural network model (500) to predict the likelihood of a first disease based on a first learning data set, and then train the neural network model (500) to predict the likelihood of a first type of the first disease based on a second learning data set. Specifically, the processor (110) may perform training to determine the likelihood of the first disease of a user (1) by extracting latent feature information from biometric data included in the first learning data set using the first learning data set for the same neural network model (500). The processor (110) may compare a predicted value output through a classifier (530) with a label assigned to the electrocardiogram data (10') (i.e., a label regarding the presence or absence of myocardial infarction) to calculate a loss, obtain a gradient for the loss, and then perform backpropagation to update the parameters of the neural network model (500). And, the processor (110) can perform learning to determine the possibility of the first type of the first disease of the user (1) by extracting latent feature information from the biometric data included in the second learning data set using the second learning data set for the same neural network model (500). The processor (110) can compare the output predicted value with the label (i.e., the label regarding whether the first type of the first disease is present) assigned to the electrocardiogram data (10') included in the second learning data set, calculate a loss, derive a gradient for the loss, and then perform backpropagation to update the parameters of the neural network model (500) again.

[0063] Meanwhile, this is not limited to the above, and the order of the learning process based on the first learning data set and the learning process based on the second learning data set may be changed and may be performed simultaneously.

[0064] FIG. 5 is a diagram showing the structure of a neural network model (500) according to one embodiment of the present disclosure.

[0065] Meanwhile, when the learning of the neural network model (500) is completed based on the first learning data set and the second learning data set, the processor (110) can analyze the electrocardiogram data (10) of the user (1) using the learned neural network model (500) and obtain a score corresponding to the possibility of the first disease of the user (1). The score may be a predicted value corresponding to the possibility of the first disease calculated from the neural network model (500), or may be calculated as a normalized or corrected value by applying a weight to the predicted value or through a post-processing process.

[0066] Specifically, the processor (110) may obtain electrocardiogram data from the user (1) and input the obtained electrocardiogram data into a neural network model (500) that has completed learning based on a first learning data set and a second learning data set. At this time, referring to FIG. 5, the neural network model (500) may include a first encoder (510) that extracts potential first feature information of the biometric data from the biometric data and a second encoder (520) that extracts high-dimensional second feature information based on the first feature information extracted from the first encoder (510). In addition, the neural network model (500) may include a classifier (530) directly connected to the second encoder (520). At this time, the classifier (530) may also be connected to the first encoder (510) (more specifically, the output terminal of the first encoder (510)) through a skip-connection structure. For example, when electrocardiogram data (10) is input, the first encoder (510) can extract a latent representation vector (i.e., first feature information) related to the first disease from the electrocardiogram data (10). To this end, the first encoder (510) (and the second encoder (520)) can be implemented as a neural network layer such as a convolutional neural network (CNN), a recurrent neural network (RNN), a transformer, etc., and can generate a latent representation vector that effectively reflects the time-series and frequency characteristics of the electrocardiogram data (10). The second encoder (520) can additionally extract a latent representation vector (i.e., second feature information) related to the first disease from the input electrocardiogram data (10). The second feature information can be a deep representation generated through a nonlinear transformation and a high-dimensional feature extraction process based on the first feature information.And, the classifier (530) can determine the possibility of the first disease of the user (1) based on the first feature information and the second feature information transmitted to the first encoder (510) (specifically, the output terminal of the first encoder (510). That is, the classifier (530) can produce an output value corresponding to the possibility of the first disease of the user (1) based on the first feature information and the second feature information. For example, the classifier (530) can comprehensively analyze the first feature information and the second feature information by concatenating or combining (e.g., adding) the first feature information and the second feature information. And, the processor (110) can determine the status of the user (1) with respect to the first disease by comparing the obtained score with a preset reference value.

[0067] The processor (110) can guide actions for the user (1) based on the user's (1) status, for example, based on the determined risk group. For example, if the user is classified as a high-risk group, the processor (110) can output a warning message (and signal) through a user terminal device, speaker, etc., and transmit the corresponding information in real time to a nearby hospital or designated medical staff based on the identified user's (1) location information. If the user is classified as a low-risk group, the processor (110) can guide regular follow-up examinations or output customized notifications for improving lifestyle habits.

[0068] FIG. 6 is a block diagram of a computing device (600) according to another embodiment of the present disclosure.

[0069] Referring to FIG. 6, a computing device (600) according to an embodiment of the present disclosure includes a processor (610), a memory (620), a communication interface (630), a sensing unit (640), a display (650), a user interface (660), a camera (670), and a speaker (680). Among the configurations illustrated in FIG. 6, the processor (610) and the memory (620) correspond to the configurations of the processor (110) and the memory (120) of the computing device (100) illustrated in FIG. 2, and thus a detailed description thereof will be omitted.

[0070] A communication interface (630) according to an embodiment of the present disclosure may be understood as a component that transmits and receives data through any known wired or wireless communication system. For example, the communication interface (630) may perform data transmission and reception using a wired or wireless communication system such as a local area network (LAN), wideband code division multiple access (WCDMA), long term evolution (LTE), wireless broadband internet (WiBro), fifth generation mobile communication (5G), ultrawide-band, ZigBee, radio frequency (RF) communication, wireless LAN, wireless fidelity, near field communication (NFC), or Bluetooth. Since the above-described communication systems are only examples, the wired and wireless communication system for data transmission and reception of the communication interface (630) may be applied in various ways other than the above-described examples.

[0071] The communication interface (630) can receive data required for the processor (610) to perform calculations through wired or wireless communication with any system or any client, etc. In addition, the communication interface (630) can transmit data generated through calculations of the processor (610) through wired or wireless communication with any system or any client, etc. For example, the communication interface (630) can receive a plurality of electrocardiogram data through communication with a cloud server that performs tasks such as databases in a hospital environment, standardization of medical data, or a computing device (600). In addition, the communication interface (630) can receive electrocardiogram data measured from a user (1) through an external biosignal device. The communication interface (630) can transmit output data of the neural network model (500), intermediate data, processed data, etc. derived from the calculation process of the processor (610), etc. through communication with the aforementioned database, server, or other computing device, etc. For example, the processor (610) can receive a neural network model (500) from an external computing device (e.g., an external server device (a computing device within a hospital), a user terminal device) via a communication interface (630).

[0072] The sensing unit (640) can directly obtain biometric data of the subject. For example, the sensing unit (640) may include a plurality of electrodes. In this case, the processor (610) can obtain an electrocardiogram signal of the user (1) as biometric data through at least one electrode. In addition, the sensing unit (640) may include an image sensor or an optical sensor. In this case, the processor (610) can obtain a photoplethysmographic blood flow signal of the user (1) as biometric data through the image sensor (or optical sensor).

[0073] The display (650) can display various images. Here, the images include both still images and moving images. The display (650) can output guide information regarding activities generated based on the user (1) status. The display (650) can be implemented as various types of displays, such as an LCD (Liquid Crystal Display Panel), an OLED (Organic Light Emitting Diodes), an LCoS (Liquid Crystal on Silicon), a DLP (Digital Light Processing), etc. In addition, the display (650) can also include a driving circuit, a backlight unit, etc., which can be implemented in a form such as an A-si TFT, an LTPS (Low temperature poly silicon) TFT, an OTFT (organic TFT), etc.

[0074] Meanwhile, the display (650) may be implemented as a touch screen by being combined with a touch panel, and in this case, the display (650) may not only function as an output interface that outputs an image through the touch screen, but also as an input interface that receives a touch input from the user (1). The display (650) may display the results of a judgment on the possibility of a disease predicted through a pre-learned neural network model, predicted electrocardiogram data, a treatment plan, and guidance information for the user (1).

[0075] The user interface (660) is a configuration used by the computing device (600) to perform interaction with the user (1), and may include at least one of a touch sensor, a motion sensor, a button, a jog dial, and a switch, but is not limited thereto. The processor (610) may receive biological information (occupation, age, gender, etc.) of the user (1) through the user interface (660).

[0076] The camera (670) captures an image of an object around the user (1) by taking a picture of the object. Specifically, the camera (670) can capture an image of food consumed by the user (1). At this time, the processor (610) can determine the nutritional status of the user (1) based on the status of the user (1) and the image of the food consumed by the user (1) and provide recommended diet information related to heart disease as guide information. To this end, the camera (670) can be implemented as an imaging device such as an imaging device having a CMOS structure (CIS, CMOS Image Sensor) or an imaging device (Charge Coupled Device) having a CCD structure.

[0077] However, the present invention is not limited thereto, and the camera (670) may be implemented as a camera module with various resolutions capable of photographing a subject. Meanwhile, the camera (670) may be implemented as a depth camera (e.g., an IR depth camera), a stereo camera, or an RGB camera.

[0078] The speaker (680) is a component that outputs various audio data on which various processing operations such as decoding, amplification, and noise filtering have been performed by an audio processing unit (not shown). The speaker (680) can output various notification sounds or voice messages. According to one embodiment of the present disclosure, the processor (610) can convert an electrical signal received from an external device into a user voice and output it through the speaker (680). For example, the speaker (680) can output a voice message warning of or suggesting diagnosis of a heart disease based on the judgment result on the possibility of a heart disease identified through a task-tailored neural network model (500).

[0079] The various embodiments of the present disclosure described above can be combined with additional embodiments and modified within the scope understood by those skilled in the art in light of the detailed description above. It should be understood that the embodiments of the present disclosure are illustrative in all respects and not restrictive. For example, each component described as a single component may be implemented in a distributed manner, and likewise, components described as distributed may be implemented in a combined manner. Accordingly, all changes or modifications derived from the meaning, scope, and equivalent concepts of the claims of the present disclosure should be construed as being included within the scope of the present disclosure.

Claims

1. A method for obtaining a neural network model trained to identify a user's status based on a plurality of training data sets performed by a computing device including at least one processor, A step of obtaining a first learning data set and a second learning data set based on a plurality of biometric data; and A step of training a neural network model to identify a user's status based on the biometric data based on the first learning data set and the second learning data set; comprising; method.

2. In paragraph 1, The steps for obtaining the first and second learning data sets are: A step of obtaining the first learning data set by assigning a first label to the plurality of biometric data according to whether the subject of the plurality of biometric data has a first disease; and A step of obtaining the second learning data set by assigning a second label to the plurality of biometric data according to whether the subject of the plurality of biometric data has a second disease; including; method.

3. In paragraph 2, The second disease above is, The first type of the above first disease, method.

4. In paragraph 3, The above neural network model is, A first encoder for extracting first feature information from the plurality of biometric data and a second encoder for extracting second feature information corresponding to the first type of the first disease based on the first feature information extracted from the first encoder, method.

5. In paragraph 4, The above neural network model is, Further comprising a classifier directly connected to the second encoder and connected to the first encoder through a skip connection, method.

6. In paragraph 5, The step of training the above neural network model is: A step of training the neural network model based on the first learning data set; and A step of training the neural network model based on the second learning data set; method.

7. In paragraph 3, The above biometric data is, Contains electrocardiogram data, The first disease above is, Including myocardial infarction, The first type of the above first disease is, including ST elevation myocardial infarction (STEMI), method.

8. A computing device for acquiring a neural network model trained to determine the state of a user based on multiple training data sets, A processor comprising at least one core; and Memory containing program codes executable on the above processor; and The above processor, Obtaining a first learning data set and a second learning data set based on a plurality of biometric data, and training a neural network model to identify a user's status based on the biometric data based on the first learning data set and the second learning data set. Computing device.

9. A computer program stored in a computer-readable storage medium, wherein the computer program, when executed on one or more processors, performs an operation of acquiring a neural network model trained to identify a user's status based on a plurality of learning data sets. The above action is, The operation of acquiring a first training data set and a second training data set based on a plurality of biometric data; and An operation of training a neural network model to identify a user's status based on the biometric data based on the first learning data set and the second learning data set; including; Computer program.

Images