Method, device and computer program for predicting possibility of syncope on basis of user biometric data by using neural network model

Abstract

The present disclosure provides a device, a method, and a computer program for predicting a possibility of syncope on the basis of user biometric data by using at least a neural network model. The method performed by a computing device including at least one processor, according to one embodiment of the present disclosure, comprises the steps of: acquiring user biometric data; and predicting a syncope-related state of the user on the basis of the biometric data by using a pre-trained neural network model.

Description

Method, device, and computer program for predicting the likelihood of fainting based on user biometric data through a neural network model

[0001] The present disclosure relates to deep learning technology in the medical field, and more specifically, to a method, apparatus, and computer program for predicting the likelihood of fainting based on a user's biometric data through a neural network model.

[0002] With the recent rapid development of information and communication technology, artificial intelligence (AI) is being actively utilized in various fields. In particular, continuous attempts are being made to integrate AI into the medical sector, which requires high expertise and accuracy. Traditionally, determining a patient's health status or the presence of disease relied on the knowledge and experience of specialized personnel such as doctors and researchers; however, utilizing AI technology offers the advantage of enabling faster and more consistent judgments by quantitatively analyzing patients' biological data.

[0003] In particular, electrocardiogram (ECG) data records electrical signals generated by the heart in a time-series format. As it contains various signs of cardiac abnormalities, such as heart rhythm, electrical conduction abnormalities, and arrhythmias, it is widely used for the early diagnosis of heart disease or the identification of at-risk groups. Time-series biological data like ECGs has the advantage of being able to precisely capture changes in a patient's health status or potential abnormal signals compared to static data from a single point in time, as it can continuously reflect physiological changes over time.

[0004] Recently, there has been a growing need for technologies that not only enable the early diagnosis of specific diseases but also predict critical clinical outcomes, such as patient survival probability or mortality. This is because deaths or serious adverse events are closely linked to complex changes in various biomarkers rather than a single cause. Therefore, a technological approach is required that can precisely analyze potential features contained in time-series biological data and make predictions based on this analysis.

[0005] Nevertheless, there is still a lack of effective predictive technology for sudden events such as syncope. Since syncope can occur suddenly during daily life and lead to secondary risks such as trauma or traffic accidents as the patient collapses, there is a need for measures that enable preventive action by predicting it before it occurs, rather than simply diagnosing it after it happens.

[0006] The present disclosure is devised in response to the aforementioned background technology and aims to provide a method, apparatus, and computer program for predicting the likelihood of fainting based on a user's biometric data through a neural network model.

[0007] However, the problems to be solved in this disclosure are not limited to those mentioned above, and other unmentioned problems may be clearly understood based on the description below.

[0008] A method for predicting the likelihood of fainting based on a user's biometric data through a neural network model performed by a computing device including at least one processor according to one embodiment of the present disclosure to solve the above-mentioned problem comprises the steps of acquiring the user's biometric data and predicting the user's fainting-related state based on the biometric data through a pre-trained neural network model.

[0009] Alternatively, the step of predicting the user's fainting-related state includes the step of inputting the biometric data into the previously trained first neural network model to obtain a first score corresponding to the user's fainting probability, and the step of predicting the user's fainting probability based on the user's fainting-related state based on the obtained score.

[0010] Alternatively, the step of predicting the user's fainting-related state includes the step of inputting the biometric data into the previously trained second neural network model to obtain a second score corresponding to the remaining time until the user faints, and the step of calculating the user's fainting-related state and the user's fainting-predicted time based on the obtained second score.

[0011] Alternatively, the above biological data may include electrocardiogram data, and the previously trained second neural network model may be trained based on a training data set that includes labels corresponding to the time from when the subject fainted until when the electrocardiogram data corresponding to the subject was acquired, for electrocardiogram data acquired from each of the multiple subjects who fainted.

[0012] Alternatively, the electrocardiogram data can be obtained through a wearable electronic device.

[0013] Alternatively, the biometric data may include electrocardiogram data and PPG (Photoplethysmography) data, and the pre-trained neural network model may include a first neural network model trained to predict the likelihood of the user fainting based on the electrocardiogram data and a third neural network model trained to predict the likelihood of the user fainting based on the PPG data.

[0014] Alternatively, the step of predicting the likelihood of fainting of the user may include the step of inputting the electrocardiogram data into the first neural network model to obtain a first score corresponding to the likelihood of fainting of the user; the step of inputting the PPG data into the third neural network model to obtain a third score corresponding to the likelihood of fainting of the user when it is determined that the first score is greater than or equal to a preset value; and the step of finally predicting the likelihood of fainting of the user based on the third score.

[0015] Alternatively, it may include a step of providing guidance information based on the predicted result of the user's likelihood of fainting.

[0016] Alternatively, the method includes a step of analyzing the cause of fainting of the user based on the prediction result of the user's likelihood of fainting and the user's context information, wherein the user's context information may include at least one of the user's location information and the user's posture information.

[0017] To solve the above-mentioned problem, a computing device for predicting the possibility of fainting based on a user's biometric data through a neural network model according to one embodiment of the present disclosure comprises a processor including at least one core and a memory including program codes executable on said processor, and said processor acquires the user's biometric data and can predict the user's fainting-related state based on said biometric data through a previously trained neural network model.

[0018] In order to solve the above-mentioned problem, a computer program stored on a computer-readable storage medium according to one embodiment of the present disclosure is configured such that, when executed on one or more processors, the computer program performs operations to predict the likelihood of fainting based on a user's biometric data through a neural network model, and said operations may include an operation to acquire the user's biometric data and an operation to predict the user's fainting-related state based on said biometric data through a pre-trained neural network model.

[0019] According to one embodiment of the present disclosure, the possibility of a user fainting can be predicted in advance by analyzing the user's biometric data through an artificial intelligence model. Accordingly, secondary risks such as falls, trauma, and traffic accidents caused by sudden fainting can be prevented, and the user's safety can be ensured by enabling appropriate responses, such as taking a rest in advance or asking for help from those around.

[0020] FIG. 1 is an exemplary diagram of a method for predicting the likelihood of fainting based on a user's biometric data through a neural network model according to one embodiment of the present disclosure.

[0021] FIG. 2 is a block diagram of a computing device that predicts the likelihood of fainting based on a user's biometric data through a neural network model according to one embodiment of the present disclosure.

[0022] FIG. 3 is a flowchart of a method for predicting a fainting-related state based on a user's biometric data through a neural network model according to one embodiment of the present disclosure.

[0023] FIG. 4 is an exemplary diagram of a method for predicting the likelihood of fainting based on a user's biometric data through a first neural network model according to one embodiment of the present disclosure.

[0024] FIG. 5 is an exemplary diagram illustrating a method for labeling a plurality of electrocardiogram data obtained from the same object according to one embodiment of the present disclosure.

[0025] FIG. 6 is an exemplary diagram illustrating a method for determining a user's predicted execution time through a second neural network model according to one embodiment of the present disclosure.

[0026] FIG. 7 is an example diagram for determining the time to predict fainting based on a plurality of biological data according to one embodiment of the present disclosure.

[0027] FIG. 8 is a detailed block diagram of a computing device according to another embodiment of the present disclosure.

[0028] Embodiments of the present disclosure are described below with reference to the attached drawings so that those skilled in the art (hereinafter, those skilled in the art) can easily implement them. The embodiments presented in the present disclosure are provided to enable those skilled in the art to use or implement the contents of the present disclosure. Accordingly, various modifications to the embodiments of the present disclosure will be apparent to those skilled in the art. That is, the present disclosure may be embodied in various different forms and is not limited to the embodiments below.

[0029] Throughout the specification of the present disclosure, identical or similar reference numerals refer to identical or similar components. Additionally, to clearly explain the present disclosure, reference numerals in the drawings that are unrelated to the description of the present disclosure may be omitted.

[0030] The term “or” as used in this disclosure is intended to mean an implicit “or” rather than an exclusive “or.” That is, unless otherwise specified in this disclosure or its meaning is unclear from the context, “X uses A or B” should be understood to mean one of the natural implicit substitutions. For example, unless otherwise specified in this disclosure or its meaning is unclear from the context, “X uses A or B” may be interpreted as X using A, X using B, or X using both A and B.

[0031] The term “and / or” as used in this disclosure should be understood to refer to and include all possible combinations of one or more of the enumerated related concepts.

[0032] The terms “comprising” and / or “comprising” as used in this disclosure should be understood to mean the presence of certain features and / or components. However, the terms “comprising” and / or “comprising” should be understood not to exclude the presence or addition of one or more other features, other components and / or combinations thereof.

[0033] Where not otherwise specified in the present disclosure or where it is not clear from the context that the singular form indicates, the singular should generally be interpreted as including “one or more.”

[0034] The term "the N (N is a natural number)" used in this disclosure may be understood as an expression used to distinguish the components of this disclosure from one another according to certain criteria, such as functional perspectives, structural perspectives, or convenience of explanation. For example, components performing different functional roles in this disclosure may be distinguished as a first component or a second component. However, components that are substantially identical within the technical scope of this disclosure but must be distinguished for the convenience of explanation may also be distinguished as a first component or a second component.

[0035] The term “acquisition” as used in this disclosure can be understood to mean not only receiving data through a wired or wireless communication network with an external device or system, but also generating data in an on-device form.

[0036] Meanwhile, the terms "module" or "unit" used in this disclosure may be understood as referring to an independent functional unit that processes computing resources, such as a computer-related entity, firmware, software or a part thereof, hardware or a part thereof, or a combination of software and hardware. In this case, "module" or "unit" may be a unit composed of a single element, or a unit expressed as a combination or set of multiple elements. For example, in a narrow sense, "module" or "unit" may refer to a hardware element of a computing device or a set thereof, an application program that performs a specific function of software, a procedure implemented through software execution, or a set of instructions for program execution. Furthermore, in a broad sense, "module" or "unit" may refer to the computing device itself that constitutes the system, or an application executed on the computing device. However, since the above-described concept is merely an example, the concepts of "module" or "part" may be defined in various ways within the scope understandable to those skilled in the art based on the contents of this disclosure.

[0037] As used in this disclosure, the term "model" may be understood as a system implemented using mathematical concepts and language to solve a specific problem, a set of software units to solve a specific problem, or an abstract model regarding a processing process to solve a specific problem. For example, a neural network "model" may refer to an overall system implemented as a neural network that possesses problem-solving capabilities through learning. In this case, the neural network may possess problem-solving capabilities by optimizing parameters connecting nodes or neurons through learning. A neural network "model" may include a single neural network or a set of neural networks composed of multiple neural networks.

[0038] The term "data" as used in this disclosure may include "image," "signal," etc. The term "image" as used in this disclosure may refer to multidimensional data composed of discrete image elements. In other words, "image" may be understood as a term referring to a digital representation of an object visible to the human eye. For example, "image" may refer to multidimensional data composed of elements corresponding to pixels in a two-dimensional image. "Image" may refer to multidimensional data composed of elements corresponding to voxels in a three-dimensional image.

[0039] As used in this disclosure, "object" refers to an object or entity from which data is collected or analyzed. For example, in the context of medical data processing, "object" may include a person (patient or subject) from whom biometric data or image data is acquired. Furthermore, in cases involving image processing or object recognition, "object" may be extended to specific objects or structures, and may include biological entities such as animals in addition to humans. In other words, "object" may encompass various forms of physical or biological objects depending on the purpose of analysis and refers to an entity that acquires data through biosignal measurement devices or sensors.

[0040] The explanation of the foregoing terms is intended to aid in understanding the present disclosure. Accordingly, it should be noted that unless a foregoing term is explicitly stated as a matter limiting the content of the present disclosure, it is not to be used in the sense of limiting the technical concept of the content of the present disclosure.

[0041] FIG. 1 is an example diagram of a method for predicting the possibility of fainting based on the biometric data (20) of a user (1) through a neural network model (400) according to one embodiment of the present disclosure.

[0042] A computing device (100) according to one embodiment of the present disclosure may be a hardware device or part of a hardware device that performs comprehensive processing and computation of data, or it may be a software-based computing environment connected to a communication network. For example, the computing device (100) may be a server that performs intensive data processing functions and is an entity that shares resources, or it may be a client that shares resources through interaction with the server. Additionally, the computing device (100) may be a cloud system in which multiple servers and clients interact to comprehensively process data. Since the description above is merely one example regarding the type of computing device (100), the type of computing device (100) may be configured in various ways within a range understandable to those skilled in the art based on the contents of the present disclosure. For example, the computing device (100) may be implemented as various electronic devices such as a server, a cloud-based virtual server, a desktop, a laptop, a smartphone, a tablet PC, a smart watch, a smart ring, etc.

[0043] A computing device (100) according to one embodiment of the present disclosure can obtain biometric data (20) from a user (1) and predict a state related to fainting of the user (1) by analyzing the biometric data (20) based on a previously learned neural network model (400). Specifically, the computing device (100) can obtain various biometric data (20) from the user (1). For example, the computing device (100) can obtain biometric data (20), such as electrocardiogram data, PPG data, and blood pressure, from the user (1). The biometric data (20) can be obtained from an external biometric signal measuring device (e.g., the user's smart watch (200), etc.) that detects and measures the user's (1) biometric signal, or it can be obtained directly through a sensing unit of the computing device (100).

[0044] A computing device (100) can identify potential feature information related to fainting included in acquired biometric data (20) using a pre-trained neural network model (400). The neural network model (400) can be pre-trained to learn the correlation between fainting and biometric data (20). Furthermore, the computing device (100) can identify the state related to fainting of the user (1) based on the potential feature information. Here, the state of the user (1) related to fainting may include the possibility of fainting occurring or the remaining time until fainting (i.e., the fainting prediction time). That is, by analyzing the biometric data (20) using the pre-trained neural network model (400), the computing device (100) can identify whether there is a possibility that the user will faint within a pre-set time or the remaining time until the user's expected fainting. Furthermore, the computing device (100) can support the user (1) in taking preventive measures before reaching a dangerous situation, that is, before fainting.

[0045] Hereinafter, embodiments of the present disclosure related thereto will be described with reference to FIGS. 2 to 8.

[0046] FIG. 2 is a block diagram of a computing device that predicts the possibility of fainting based on the biometric data (20) of a user (1) through a neural network model (400) according to one embodiment of the present disclosure.

[0047] Referring to FIG. 2, a computing device (100) according to one embodiment of the present disclosure may include a processor (110), a memory (120), and a communication interface (130). However, since FIG. 2 is merely an example, the computing device (100) may include other components for implementing a computing environment.

[0048] A processor (110) according to one embodiment of the present disclosure may be understood as a constituent unit comprising hardware and / or software for performing computing operations. For example, the processor (110) may read a computer program and perform data processing for machine learning. The processor (110) may process computational processes such as processing input data for machine learning, extracting features for machine learning, and calculating errors based on backpropagation. A processor (110) for performing such data processing may include a central processing unit (CPU), a general purpose graphics processing unit (GPGPU), a tensor processing unit (TPU), an application specific integrated circuit (ASIC), or a field programmable gate array (FPGA). Since the above-described types of processors (110) are merely examples, the types of processors (110) may be configured in various ways within a range understandable to those skilled in the art based on the contents of the present disclosure. For example, the computing device (100) can be implemented as various electronic devices such as a server, a cloud-based virtual server, a desktop, a laptop, a smartphone, a tablet PC, a smart watch, a smart ring, etc.

[0049] The processor (110) is connected to other components of the computing device (100) (e.g., memory (120) and communication interface (130)) to control the overall operation of the computing device (100).

[0050] A memory (120) according to one embodiment of the present disclosure may be understood as a unit comprising hardware and / or software for storing and managing data processed by a computing device (100). That is, the memory (120) may store data of any form generated or determined by a processor (110) and data of any form received by a communication interface (130) of the computing device (100). To this end, the memory (120) may include at least one type of storage medium among a flash memory type, a hard disk type, a multimedia card micro type, a card type memory, RAM (random access memory), SRAM (static random access memory), ROM (read-only memory), EEPROM (electrically erasable programmable read-only memory), PROM (programmable read-only memory), magnetic memory, a magnetic disk, and an optical disk. Additionally, the memory (120) may include a database system that controls and manages data in a predetermined system. Since the above-described type of memory (120) is merely an example, the type of memory (120) may be configured in various ways within a range understandable to those skilled in the art based on the contents of this disclosure.

[0051] The memory (120) can structure and organize data, combinations of data, and program code executable by the processor (110) that are necessary for the processor (110) to perform calculations. For example, the memory (120) may store biometric data (20) received through the communication interface (130), learning data composed of biometric data (20), and information of an object corresponding to the biometric data (20). Additionally, the memory (120) may store a neural network model (400), program code that operates the neural network model (400) to perform learning, program code that operates the neural network model (400) to receive biometric data (20) as input and perform inference according to the purpose of use of the computing device (100) (i.e., prediction of the user's state related to fainting), and output data or processed data generated as the program code is executed.

[0052] A communication interface (130) according to one embodiment of the present disclosure may be understood as a configuration unit that transmits and receives data through any known form of wired or wireless communication system. For example, the communication interface (130) may perform data transmission and reception using wired or wireless communication systems such as a local area network (LAN), wideband code division multiple access (WCDMA), long term evolution (LTE), wireless broadband internet (WiBro), 5th generation mobile communication (6G), ultrawide-band wireless communication, ZigBee, radio frequency (RF) communication, wireless LAN, wireless fidelity (Wi-Fi), near field communication (NFC), or Bluetooth. Since the communication systems described above are merely examples, wired or wireless communication systems for data transmission and reception of the communication interface (130) may be applied in various ways other than those described above.

[0053] The communication interface (130) can receive data necessary for the processor (110) to perform calculations through wired or wireless communication with any system or any client. Additionally, the communication interface (130) can transmit data generated through the calculations of the processor (110) through wired or wireless communication with any system or any client. For example, the communication interface (130) can receive bio-data (20) through communication with a database within a hospital environment, a cloud server performing tasks such as standardization of medical data, or an external computing device (e.g., a bio-signal measuring device). The communication interface (130) can transmit output data of each model (400), and intermediate data and processed data derived during the calculation process of the processor (110), through communication with the aforementioned database, server, or computing device. For example, the processor (110) can transmit user state information related to the user (1)'s fainting (i.e., possibility of fainting, predicted time of occurrence, etc.) obtained through the neural network model (400) via the communication interface (130) to a user terminal device (e.g., smartphone, tablet PC, smart watch, etc.).

[0054] FIG. 3 is a flowchart of a method for predicting the possibility of fainting based on the biometric data (20) of a user (1) through a neural network model (400) according to one embodiment of the present disclosure.

[0055] Referring to FIG. 3, the processor (110) acquires the biometric data (20) of the user (1) (S310). Specifically, the processor (110) can acquire the biometric data (20) of the user (1) from a biometric signal measuring device connected via a communication interface (130).

[0056] Specifically, the processor (110) can obtain biometric data (20) of the user (1) from an external biometric signal measuring device connected to the computing device (100) via a communication interface (130). The biometric data (20) may include data corresponding to various biometric signals measurable from the subject (i.e., user (1)), such as body temperature, blood pressure, heart rate, electrocardiogram, electroencephalogram (EEG), photoplethysmogram (PPG), etc.

[0057] In particular, biometric data (20) can be obtained through a wearable electronic device. For example, electrocardiogram data, blood pressure, and PPG data can be obtained through a smart watch, smart ring, etc. worn by the user (1). The processor (110) can obtain biometric data (20) obtained from the wearable electronic device through a communication interface (130).

[0058] The processor (110) may acquire biometric data (20) through a sensing unit included in the computing device (100). For example, the processor (110) may acquire electrocardiogram data from the user (1) through a plurality of electrodes included in the sensing unit. Alternatively, the processor (110) may acquire the user's (1) PPG data through an optical sensor included in the sensing unit.

[0059] Meanwhile, the processor (110) can periodically acquire the user's (1) biometric data (20). That is, the processor (110) can acquire the user's (1) biometric data (20) at preset intervals without a separate control command and then predict the likelihood of fainting. For example, the processor (110) can acquire electrocardiogram data from the user (1) every 10 minutes and predict the likelihood of fainting of the user (1) based on the acquired electrocardiogram data.

[0060] According to one embodiment of the present disclosure, the processor (110) can predict the state of the user (1) related to fainting based on the biometric data (20) through a previously trained neural network model (400) after acquiring biometric data (20) (S320).

[0061] FIG. 4 is an example diagram of a method for predicting the possibility of fainting based on the biometric data (20) of a user (1) through a first neural network model (400-1) according to one embodiment of the present disclosure.

[0062] According to one embodiment of the present disclosure, the neural network model (400) may include a first neural network model (400-1) trained to predict the likelihood of a user (1) fainting based on biometric data (20). Referring to FIG. 4, the processor (110) can predict the likelihood of a user (1) fainting by inputting biometric data (20) into the first neural network model (400-1).

[0063] According to one embodiment of the present disclosure, the processor inputs biometric data (20) into a pre-trained first neural network model (400-1) to obtain a score corresponding to the possibility of fainting of the user (1), and can predict the possibility of fainting of the user (1) based on the obtained score (S320). The processor (110) can predict that the higher the score obtained from the first neural network model (400-1), the higher the possibility of fainting of the user (1).

[0064] To this end, the processor (110) can pre-train a first neural network model (400-1) using a training data set composed of biometric data (20) obtained from a fainting subject. Here, the subject refers to the entity providing the biometric data (20) during the training phase, and may be a concept distinct from the user (1) who is evaluated for the possibility of fainting during the actual prediction phase. For example, the processor (110) may prepare a training data set by configuring electrocardiogram data obtained from multiple subjects as input data and configuring whether the subject faints as a label. As an example, the electrocardiogram data of the input data may consist of data obtained by measuring within a preset time (e.g., 10 minutes) from a fainting subject. Then, the processor (110) inputs multiple electrocardiogram data constituting the input data into the first neural network model (400-1) and can calculate a loss function based on the difference between the output value for each electrocardiogram data of the first neural network model (400-1) and the label corresponding to each electrocardiogram data. The loss function can be defined as a cross-entropy loss or an objective function that optimizes the balance between precision and recall. Based on the calculated loss function, the processor (110) can adjust the weights of the first neural network model (400-1) through backpropagation. By repeating this process, the processor (110) can improve the classification performance regarding fainting of the first neural network model (400-1) and finally obtain a neural network model (400) trained to determine the possibility of fainting of the user (1) based on the electrocardiogram data. The processor (110) can obtain a score corresponding to the possibility of fainting of the user (1) from the first neural network model (400-1). Here, the score may be a probability value for the possibility of fainting obtained as an output value from the first neural network model (400-1), or a value obtained by applying a pre-set weight to the probability value.

[0065] The first neural network model (400-1) can be implemented as a multi-layer perceptron (MLP), convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network, or residual neural network (ResNet) structure.

[0066] The processor (110) can determine the risk (or severity) of the possibility of fainting of the user (1) by comparing the score obtained from the first neural network model (400-1) with a preset value. If the score is greater than or equal to a preset first value, the processor (110) determines the risk as high, indicating a high possibility of fainting; if the score is less than the preset first value and greater than or equal to a second value, it determines the risk as low, indicating a low possibility of fainting; and if the score is less than the preset second value, the processor (110) determines the user's fainting-related state as normal, indicating no possibility of fainting. However, this is not limited thereto, and the risk regarding the possibility of fainting of the user may be classified into various grades.

[0067] Meanwhile, the processor (110) can adjust the acquisition cycle of biometric data (20) based on the score obtained from the first neural network model (400-1). If the processor (110) identifies that the acquired score is high (i.e., identifies that the user (1) has a high probability of fainting), it can set the acquisition cycle of biometric data (20) shorter to identify the user (1)'s probability of fainting at a shorter cycle. For example, if the processor (110) identifies that the score obtained through electrocardiogram data acquired every 30 minutes is greater than or equal to a preset value and identifies that the user (1) has a very high probability of fainting, it can adjust the acquisition cycle of electrocardiogram data to 10 minutes and predict the probability of fainting.

[0068] According to an embodiment of the present disclosure, the processor (110) can calculate the predicted time of fainting for the user (1) based on the acquired score. Specifically, the processor (110) defines the predicted time of fainting corresponding to the score interval as a mapping table or a regression function, and can derive the predicted time of fainting for the user (1) using the interval to which the score acquired from the neural network model (400) belongs or the calculated function value. At this time, the higher the score, the more the processor (110) calculates the predicted time of fainting imminently, and the lower the score, the more it can predict that fainting will occur after a relatively long time. In addition, the processor (110) may calculate the predicted time of fainting for the user (1) by applying weights to the acquired score. The user (1) can prepare more effectively for dangerous situations by being provided not only with the possibility of fainting but also with the remaining time until fainting occurs.

[0069] FIG. 5 is an exemplary diagram illustrating a method of assigning labels to a plurality of electrocardiogram data obtained from the same object according to one embodiment of the present disclosure. FIG. 6 is an exemplary diagram illustrating a method of determining a user's predicted execution time through a second neural network model according to one embodiment of the present disclosure.

[0070] According to one embodiment of the present disclosure, the neural network model (400) may include a second neural network model (400-2) trained to predict the remaining time until fainting of the user (1) based on biometric data (20).

[0071] In this regard, the processor (110) can construct a label of a learning data set corresponding to the time from when the subject fainted to when the electrocardiogram data corresponding to the subject was acquired, for electrocardiogram data acquired from each of the multiple subjects. Specifically, the processor (110) can identify the time of the subject's fainting and assign a label to the electrocardiogram data acquired prior to fainting, corresponding to the time from when the fainting occurred to when the electrocardiogram data was acquired. For example, the processor (110) can calculate the time from when the electrocardiogram data was acquired from the subject to when the subject fainted, and assign a label corresponding to the calculated period (i.e., a label value set corresponding to the calculated period) to the electrocardiogram data. Then, the processor (110) can acquire a learning data set containing the electrocardiogram data to which the label has been assigned.

[0072] In particular, the label value set corresponding to the time from when the biometric data (20) was acquired until fainting may be proportional to the period. That is, the longer the time, the larger the label value may be. However, it is not limited to this, and the label value set corresponding to the time from when the biometric data (20) was acquired until fainting may be set inversely proportional to the period. That is, the label value set corresponding to the period from when the biometric data (20) was acquired until the day of fainting may include multiple label values ​​depending on the period.

[0073] When multiple biometric data (20) are obtained from a subject at different first and second times, the processor (110) may assign a first label corresponding to a first period from the first time until the subject faints to the first biometric data (20) obtained at the first time among the multiple biometric data (20), and assign a second label corresponding to a second period from the second time until the subject faints to the second biometric data (20) obtained at the second time among the multiple biometric data (20). For example, referring to FIG. 3, if there is electrocardiogram data obtained by measuring on August 8, 2021 (hereinafter, first electrocardiogram data (20-1)), electrocardiogram data obtained by measuring on July 15, 2021 (hereinafter, second electrocardiogram data (20-2)), and electrocardiogram data obtained by measuring on May 15, 2021 (hereinafter, third electrocardiogram data (20-3)) from a first patient who fainted on August 15, 2021, the processor (110) can assign a label value to each electrocardiogram data that corresponds to the period from the date the electrocardiogram data was measured to the time of the first patient's fainting. Specifically, the processor (110) can label the first electrocardiogram data (20-1) with a1 corresponding to the period (7 days) between the first patient's syncope date (August 15, 2021) and the date the first electrocardiogram data (20-1) was measured (August 8, 2021). Likewise, the processor (110) may label the second electrocardiogram data (20-2) with a2 (a value greater than a1) corresponding to the period (30 days) between the first patient's syncope date (August 15, 2021) and the date the second electrocardiogram data was measured (July 15, 2021), and label the third electrocardiogram data (20-3) with a3 (a value greater than a2) corresponding to the period (92 days) between the first patient's syncope date (August 15, 2021) and the date the third electrocardiogram data (20-3) was measured (May 15, 2021).Therefore, even if electrocardiogram data is obtained from the same subject, different labels may be assigned depending on the time of acquisition of each data. Meanwhile, according to one embodiment of the present disclosure, the processor (110) may divide the time from the time when the bio-data (20) is obtained until the time when the subject faints into multiple intervals, and set the same label for each interval.

[0074] The processor (110) can prepare a learning data set composed of biometric data (20) by preparing biometric data (20) and a label assigned based on the time of fainting of the subject as input data for each subject. The learning data set includes multiple biometric data (20) obtained from multiple subjects. At this time, each biometric data (20) may be assigned a label corresponding to a period calculated according to the time of fainting of the subject of each biometric data (20). Therefore, even for the same subject, the label values ​​between the biometric data (20) may differ, and in the case of biometric data (20) of different subjects, the same label value may be assigned according to the calculated period.

[0075] Referring to FIG. 6, the processor (110) can train a second neural network model (400-2) based on the training data set prepared in this manner. In this regard, the description related to the neural network model of the present disclosure described above applies equally, so a detailed description is omitted. At this time, the processor (110) can determine the possibility of fainting of the user (1) and the predicted time of fainting based on the score obtained through the second neural network model (400-2). That is, the score corresponding to the possibility of fainting can be interpreted as a value containing information on the time remaining until the user (1) faints, and through this, it can provide not only the simple possibility of occurrence but also whether fainting is imminent. The processor (110) can provide appropriate guidance information (e.g., remaining time information, etc.) to the user (1) according to the predicted time of fainting.

[0076] The second neural network model (400-2) can be implemented with a Multi-Layer Perceptron (MLP), Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN), Generative Adversarial Network, or Residual Neural Network (ResNet) structure.

[0077] The processor (110) may construct a plurality of neural network models (400) that predict the likelihood of fainting and the time of fainting, respectively. That is, the neural network model may include a neural network model trained to predict the likelihood of fainting and calculate a score corresponding to the likelihood of fainting (i.e., a first neural network model (400-1)) and a neural network model trained to calculate the time of fainting (i.e., a second neural network model (400-2)). However, it is not limited thereto, and the first neural network model (400-1) and the second neural network model (400-2) may share a neural network (e.g., an encoder) trained to extract first feature information related to fainting from electrocardiogram data (20), and may be implemented with a first neural network (e.g., a first encoder, etc.) that calculates the likelihood of fainting based on the first feature information output from the said neural network, a second neural network (e.g., a second encoder, etc.) that calculates the time of fainting, and respective independent prediction heads.

[0078] Meanwhile, according to one embodiment of the present disclosure, the processor (110) may prepare a learning data set through a tilt table test for a plurality of subjects. Specifically, the processor (110) monitors the biological data (20) of each subject undergoing a tilt table test, identifies the subject that has fainted, and labels the biological data (20) before and after the time of fainting as fainting occurrence data. Additionally, the biological data (20) of the subject that has not fainted may be labeled as non-fainting data to form a learning data set. By assigning the time difference between the time of fainting occurrence and the time when the corresponding biological data (20) was acquired as a label value, the learning data set may be expanded to calculate the fainting prediction time.

[0079] According to one embodiment of the present disclosure, the processor (110) may predict the likelihood of fainting of the user (1) using a plurality of biometric data (20) (i.e., biometric data of different types (20)). Specifically, when the biometric data (20) includes electrocardiogram data (20) and PPG (Photoplethysmography) data, the processor (110) may input the electrocardiogram data (20) and the PPG data (30) into a neural network model (400), respectively, to obtain a plurality of scores corresponding to the likelihood of fainting. Then, the processor (110) may predict the likelihood of fainting of the user (1) based on the plurality of scores. To this end, the previously trained neural network model (400) may include a first neural network model (400-1) trained to predict the likelihood of fainting of the user (1) based on electrocardiogram data (20) and a third neural network model (400-3) trained to predict the likelihood of fainting of the user (1) based on PPG data (30).

[0080] For example, the processor (110) can finally predict the user (1)'s likelihood of fainting by summing the score regarding the likelihood of fainting corresponding to the electrocardiogram data (20) (hereinafter, the first score) and the score regarding the likelihood of fainting corresponding to the PPG data (30) (hereinafter, the third score), or by calculating an average value. More specifically, the likelihood of fainting can be identified by classifying it into a risk level by comparing the summed value (or average value) with a plurality of preset reference values. At this time, the processor (110) can input the electrocardiogram data (20) into the first neural network model (400) to obtain the score corresponding to the user (1)'s likelihood of fainting (hereinafter, the first score), and if it is determined that the first score is less than the preset first value and greater than or equal to the second value, it can input the PPG data (30) into the third neural network model (400-3) to obtain the third score corresponding to the user (1)'s likelihood of fainting. And, the processor (110) can finally predict the likelihood of fainting of the user (1) based on the third score. That is, if the processor (110) determines that the risk of fainting of the user (1) is low based on the first score obtained based on the electrocardiogram data (20), it can obtain PPG data (30) and verify the prediction result of the likelihood of fainting based on the PPG data (30).

[0081] FIG. 7 is an example diagram for determining the time to predict fainting based on a plurality of biological data (20) according to one embodiment of the present disclosure.

[0082] Additionally, the processor (110) can determine the predicted time of fainting of the user (1) based on the score (hereinafter, the second score) obtained from the second neural network model (400-2), and can adjust the predicted time of fainting of the user (1) determined based on the third score. Specifically, the third score is obtained as the output of the third neural network model (400-3) learned based on PPG data (30), and is calculated based on feature information reflecting changes in blood flow or pulse wave variability. If the predicted time of fainting calculated through the second score is biased to be excessively short or long, the processor (110) can calculate the final predicted time of fainting by applying a correction coefficient or weight while considering the third score together. For example, if the predicted time of fainting calculated by the second score based on electrocardiogram data (20) is calculated to be 5 minutes, but the third score based on PPG data (30) at the same time indicates a low risk, the processor (110) can adjust the final predicted time of fainting to be longer. Conversely, even if the prediction time based on electrocardiogram data (20) is calculated to be 15 minutes, if the third score based on PPG data (30) indicates a high risk, the final syncope prediction time can be adjusted to be shorter. Thus, through a mutual correction process based on multiple biosignals, a more reliable syncope prediction result can be provided compared to a prediction based on a single signal.

[0083] Meanwhile, regarding the third neural network model (400-3), the description of the neural network model (400) of the present disclosure described above can be applied in the same way, except that the type of biometric data is changed to PPG data (30).

[0084] According to one embodiment of the present disclosure, the processor (110) may provide guidance information based on the prediction result of the likelihood of fainting of the user (1). Specifically, if the processor (110) determines that there is a high likelihood of fainting, it may send an alarm signal to a user terminal device or a wearable device worn by the user via the communication interface (130) to guide the user (1) to immediately assume a safe posture, such as sitting or lying down. That is, the processor (110) may enable a safe response by providing the prediction result as an alarm signal to the user terminal or wearable device, or by sending an emergency warning message to a guardian and medical staff. Through this, the computing device can ensure the safety of the user (1) who is at high risk of fainting and prevent secondary accidents caused by fainting. For example, if the processor (110) predicts that there is a high likelihood of fainting, it may provide the user (1) with location information of a nearby medical institution, such as a hospital or public health center, or make an emergency report via a smart watch (200), a smartphone, etc. Or,

[0085] Additionally, if the predicted time for fainting is calculated to be imminent, the processor (110) can enable a rapid response by outputting an emergency notification to people around the user (1) or by providing a warning message including the user's (1) location information to registered guardians or medical staff. On the other hand, if the likelihood of fainting is predicted to be low, the processor (110) can provide guidance information recommending periodic monitoring while allowing the user (1) to continue their activities. For example, the processor (110) can provide information on the remaining time until fainting, provide location information of medical institutions such as hospitals or health centers located at a distance corresponding to the remaining time, recommend rest to the user based on the remaining time, or perform the aforementioned emergency report.

[0086] According to one embodiment of the present disclosure, the processor (110) can analyze the cause of fainting of the user (1) based on the prediction result of the possibility of fainting of the user (1) and the context information of the user (1). At this time, the context information of the user (1) may include at least one of the location information of the user (1) and the posture information of the user (1). Specifically, the processor (110) may analyze that the user (1) is associated with a change in body position, such as suddenly switching from a lying or sitting position to a standing position, and that if the possibility of fainting is predicted to be high at this time, it is caused by orthostatic hypotension.

[0087] In addition, if the user (1) is predicted to have a high probability of fainting while standing for a long time, it can be analyzed as vasovagal syncope caused by blood stagnation in the lower extremity veins and reduced venous return, and if the user is predicted to have a high probability of fainting without any change in posture while standing or sitting in a stable standing or sitting position, it can be analyzed as fainting caused by cardiac arrhythmia or neurological factors. Furthermore, if the location information is a place associated with psychological tension, anxiety, or fear, such as a restroom, a performance hall, a subway platform, or a conference room, the processor (110) can analyze it as vasovagal syncope caused by emotional shock or parasympathetic overexcitation if the probability of fainting is predicted to be high in that situation. For example, in a restroom, it can be identified as fainting due to increased abdominal pressure and vagal nerve stimulation during defecation, and in a performance hall or a crowded place, it can be identified as fainting due to tension, hyperventilation, anxiety, etc.

[0088] Meanwhile, the processor (110) may analyze it as cardiogenic syncope by additionally considering major biosignals such as electrocardiogram data (20) when the likelihood of syncope is repeatedly predicted to be high even though the posture information and location information of the user (1) do not match the situation of occurrence of orthostatic hypotension or vasovagal syncope. For example, if the likelihood of syncope is repeatedly predicted to be high even in a stable standing or sitting state without any significant change in posture or external stimulation, it may be determined to be syncope caused by cardiac factors such as arrhythmia, myocardial infarction, or structural heart disease.

[0089] The processor (110) may also consider the context data of the user (1) when providing guidance information. For example, if the location information is identified as an environment with a high risk of secondary accidents, such as a road, stairs, a swimming pool, or near a construction site, it may recommend immediate rest or send a warning notification to a guardian to ensure safety, even if the probability of fainting is predicted to be relatively low. Conversely, if the location information is identified as a safe space, such as a living room or bedroom within the home, it may provide guidance information at a relatively milder level even with the same prediction result. Additionally, the processor (110) may also consider the user's (1) posture information, and if a dangerous posture is detected, such as driving or working at a height, it may prioritize recommending a forced stop notification or emergency response procedure, even if the risk of fainting is low.

[0090] FIG. 8 is a detailed block diagram of a computing device according to another embodiment of the present disclosure.

[0091] FIG. 8 is a configuration diagram of a computing device (800) according to an embodiment of the present disclosure. Referring to FIG. 8, the computing device (800) according to an embodiment of the present disclosure includes a processor (810), memory (820), a communication interface (830), a display (840), a user interface (850), a sensing unit (860), a camera (870), a speaker (880), and a microphone (890). A detailed description of configurations shown in FIG. 8 that overlap with configurations shown in FIG. 2 will be omitted.

[0092] According to another embodiment of the present disclosure, a computing device (800) can directly acquire and analyze biometric data. To this end, the computing device (800) can be implemented as various electronic devices such as a desktop, laptop, smartphone, tablet PC, smart watch, smart ring, etc.

[0093] The display (840) can display various images and text. Here, images include both still images and videos. The display (840) may also output the probability of fainting, the predicted time of fainting, or the time remaining until the predicted time of fainting, etc., predicted through the learned neural network model (400). The display (840) can be implemented as various types of displays such as LCD (Liquid Crystal Display Panel), OLED (Organic Light Emitting Diodes), LCoS (Liquid Crystal on Silicon), DLP (Digital Light Processing), etc. Additionally, the display (840) may also include a driving circuit, a backlight unit, etc., which can be implemented in forms such as a-si TFT, LTPS (low temperature poly silicon) TFT, OTFT (organic TFT), etc.

[0094] Meanwhile, the display (840) may be implemented as a touch screen by combining it with a touch panel. The display (840) can perform the function of an output interface that outputs images through the touch screen, as well as an input interface that receives touch input from the user (1). The display (840) can display extracted feature information, identified user (1) status, generated prompts, etc.

[0095] The user interface (850) is a configuration used by the computing device (800) to perform interaction with the user (1), and may include at least one of a touch sensor, a motion sensor, a button, a jog dial, and a switch, but is not limited thereto. The computing device (800) may receive biological information of the user (1) (e.g., age, gender, etc.) through the user interface (850).

[0096] The sensing unit (860) senses the user's (1) biosignal to obtain the user's (1) time-series biosignal data (20). For example, the user's (1) electrocardiogram data (20) can be obtained by detecting the user's (1) heartbeat electrical signal through a plurality of electrodes included in the sensing unit (860). Alternatively, the sensing unit (860) may include an optical sensor and obtain the user's (1) optical blood flow data through the optical sensor.

[0097] The camera (870) can capture objects around the user (1) to obtain an image of the user (1). Based on the obtained image, the processor (810) can determine the biological information (height, age, gender, etc.) of the user (1). The camera (870) can be implemented as an image sensor having a CMOS structure (CIS, CMOS Image Sensor), an image sensor having a CCD structure (Charge Coupled Device), etc. However, it is not limited thereto, and the camera (870) can be implemented as a camera module of various resolutions capable of capturing a subject. Meanwhile, the camera (870) can be implemented as a depth camera (e.g., IR depth camera, etc.), a stereo camera, or an RGB camera, etc.

[0098] The speaker (880) is configured to output various audio data, for which various processing operations such as decoding, amplification, and noise filtering have been performed by an audio processing unit (not shown). The speaker (880) can output various notification sounds or voice messages. According to one embodiment of the present disclosure, the processor (810) can convert an electrical signal into the voice of the user (1) and output it through the speaker (880). For example, the speaker (880) can output voice messages regarding the possibility of fainting, the predicted time of fainting, or the time remaining until the predicted time of fainting.

[0099] The microphone (890) can receive the voice of the user (1). The processor (810) can obtain the voice of the user (1) controlling the computing device (100) through the microphone (890). To this end, the computing device (800) may include a model (400) for voice recognition. Specifically, the computing device (800) may include an Automatic Speech Recognition (ASR) model (400) for converting the user (1) voice into text, and may also include a Natural Language Understanding (NLU) model (400).

[0100] Meanwhile, according to one embodiment of the present disclosure, a non-transitory computer-readable medium may be provided that stores a program for performing a method of predicting the possibility of fainting based on the biometric data (20) of a user (1) through the neural network model (400) described above. Here, a non-transitory computer-readable medium refers to a medium that stores data semi-permanently and is readable by a device, rather than a medium that stores data for a short period of time, such as a register, cache, or memory. Specifically, the various applications or programs described above may be provided by being stored on a non-transitory computer-readable medium such as a CD, DVD, hard disk, Blu-ray disk, USB, memory card, ROM, etc.

[0101] The various embodiments of the present disclosure described above may be combined with additional embodiments and modified to the extent understandable to those skilled in the art in light of the detailed description above. The embodiments of the present disclosure are illustrative in all respects and should be understood as not restrictive. For example, each component described as a single unit may be implemented in a distributed manner, and components described as distributed may likewise be implemented in a combined form. Accordingly, all modifications or variations derived from the meaning, scope, and equivalents of the claims of the present disclosure should be interpreted as being included within the scope of the present disclosure.

Claims

1. A method for predicting the likelihood of fainting based on a user's biometric data through a neural network model performed by a computing device comprising at least one processor, Step of acquiring the user's biometric data; and A step of predicting the user's fainting-related state based on the biometric data through a previously trained neural network model; comprising method.

2. In Paragraph 1, The step of predicting the fainting-related state of the above-mentioned user is, A step of inputting the above biometric data into the above-mentioned first neural network model to obtain a first score corresponding to the user's likelihood of fainting; and A step of predicting the likelihood of fainting of the user based on the fainting-related state of the user based on the score obtained above; comprising method.

3. In Paragraph 1, The step of predicting the fainting-related state of the above-mentioned user is, A step of inputting the above biometric data into the above-mentioned second neural network model to obtain a second score corresponding to the remaining time until the user faints; and A step of calculating the user's predicted time to faint based on the user's fainting-related state based on the second score obtained above; comprising method.

4. In Paragraph 3, The above biometric data is, Includes electrocardiogram data, The aforementioned previously trained second neural network model is, For electrocardiogram data acquired from a plurality of fainting subjects, learned based on a training dataset including labels corresponding to the time from the time the subject fainted until the time the electrocardiogram data corresponding to the subject was acquired, method.

5. In Paragraph 4, The above electrocardiogram data is, Acquired through wearable electronic devices, method.

6. In Paragraph 2, The above biometric data is, Includes electrocardiogram data and PPG (Photoplethysmography) data, The above-mentioned previously trained neural network model is, A first neural network model trained to predict the likelihood of fainting of the user based on the electrocardiogram data, and a third neural network model trained to predict the likelihood of fainting of the user based on the PPG data, method.

7. In Paragraph 6, The step of predicting the possibility of the above-mentioned user fainting is, A step of inputting the electrocardiogram data into the first neural network model to obtain a first score corresponding to the user's likelihood of fainting; If it is determined that the first score is greater than or equal to a preset value, the step of inputting the PPG data into the third neural network model to obtain a third score corresponding to the possibility of the user fainting; and A step of finally predicting the likelihood of the user fainting based on the third score; comprising method.

8. In Paragraph 1, A step of providing guidance information based on the prediction result of the user's likelihood of fainting; comprising method.

9. In Paragraph 1, The method includes the step of analyzing the cause of fainting of the user based on the prediction result of the user's likelihood of fainting and the user's context information; The above user's context information is, including at least one of the user's location information and the user's posture information, method.

10. In a computing device that predicts the likelihood of fainting based on a user's biometric data through a neural network model, A processor comprising at least one core; and memory containing program codes executable on the above processor; and The above processor is, Acquiring a user's biometric data and predicting the user's fainting-related state based on the biometric data through a pre-trained neural network model, Computing device.

11. A computer program stored on a computer-readable storage medium, wherein the computer program, when executed on one or more processors, performs actions to predict the likelihood of fainting based on the user's biometric data through a neural network model, and The above operations are, The operation of acquiring the user's biometric data; and The operation of predicting the user's fainting-related state based on the biometric data through a previously trained neural network model; Computer program.

Images