Device, method, and program for acquiring user-customized neural network model for predicting biometric data of user on basis of dual buffer

Abstract

The present disclosure provides a device, a method, and a program for acquiring a user-customized neural network model for predicting biometric data of a user on the basis of a dual buffer. The method according to an embodiment of the present disclosure comprises the steps of: acquiring a neural network model trained to predict second biometric data of a user on the basis of first biometric data of the user; and updating the neural network model on the basis of at least one of the first biometric data stored in a first buffer and the second biometric data stored in a second buffer.

Description

Device, method and program for obtaining a user-customized neural network model for predicting a user's biometric data based on a dual buffer

[0001] The present disclosure relates to deep learning technology in the medical field, and more particularly, to a device, method, and program for obtaining a user-customized neural network model that predicts a user's biometric data based on a dual buffer.

[0002] Recent advancements in information and communication technology have led to the active use of artificial intelligence (AI) in various fields. In particular, even in the medical field, where specialists like doctors and researchers were once reliant on diagnosing patients' illnesses, AI technologies are increasingly being used to automatically analyze patients' conditions or predict disease. A representative example is a technology that automatically analyzes patients' conditions by inputting electrocardiogram data acquired in real time into a pre-trained deep learning model.

[0003] However, most of these deep learning-based medical analysis technologies are trained using common models based on predefined training data. Therefore, in practice, variations in biometric data characteristics across users can lead to reduced predictive performance. Even for users with the same disease or condition, the waveforms and values ​​of biometric data can vary depending on biological characteristics or environmental factors. Therefore, there is a need for technologies that can continuously calibrate or update deep learning models to reflect the characteristics of biometric data acquired from individual users, thereby ensuring personalized diagnostic performance.

[0004] The present disclosure is conceived in response to the aforementioned background technology, and aims to provide a device, method, and program for obtaining a user-customized neural network model that predicts a user's biometric data based on a dual buffer.

[0005] However, the problems to be solved in this disclosure are not limited to the problems mentioned above, and other problems not mentioned can be clearly understood based on the description below.

[0006] In order to achieve the above-described task, a method for obtaining a user-customized neural network model based on a dual buffer, which is performed by a computing device including at least one processor according to an embodiment of the present disclosure, comprises the steps of obtaining a neural network model trained to predict second biometric data of the user based on first biometric data of the user, and the step of updating the neural network model based on at least one of first biometric data stored in a first buffer and second biometric data stored in a second buffer.

[0007] Alternatively, the method comprises the step of, when first biometric data is acquired from the user, storing the acquired first biometric data in the first buffer.

[0008] Alternatively, the step of storing in the first buffer includes, when first biometric data is acquired from the user, a step of identifying whether second biometric data corresponding to the acquired first biometric data has been acquired, a step of storing the acquired first biometric data in the first buffer and the second buffer, respectively, when the second biometric data corresponding to the acquired first biometric data is identified as having been acquired, and a step of storing the first biometric data only in the first buffer when the second biometric data corresponding to the acquired first biometric data is identified as not having been acquired.

[0009] Alternatively, the method comprises the step of, when second biometric data is acquired from the user, storing the acquired second biometric data in the second buffer.

[0010] Alternatively, the method includes, when second biometric data is acquired from the user, a step of identifying whether first biometric data corresponding to the acquired second biometric data has been acquired; when it is identified that the first biometric data corresponding to the acquired second biometric data has been acquired, a step of matching the acquired first biometric data with the acquired second biometric data and storing the matched first biometric data in the second buffer; and when it is identified that the first biometric data corresponding to the acquired second biometric data has not been acquired, a step of requesting acquisition of the first biometric data corresponding to the second biometric data.

[0011] Alternatively, the step of updating the neural network model includes the step of updating the neural network model by performing self-supervised learning on the neural network model based on the first biometric data stored in the first buffer.

[0012] Alternatively, the step of updating the neural network model by performing the self-supervised learning includes the step of inputting the first biometric data into the neural network model and performing self-supervised learning on the neural network model to restore the first biometric data.

[0013] Alternatively, the step of updating the neural network model includes the step of updating the neural network model by performing supervised learning on the neural network model based on the second biometric data stored in the second buffer.

[0014] Alternatively, the method includes the step of extracting a plurality of first biometric data from the first buffer and a plurality of second biometric data from the second buffer at a preset ratio, and the step of updating the neural network model based on the extracted plurality of first biometric data and the extracted plurality of second biometric data.

[0015] Alternatively, the extracting step includes extracting the first biometric data in a most recent time order stored in the first buffer.

[0016] Alternatively, the first biometric data includes electrocardiogram data, and the second biometric data includes blood pressure data.

[0017] According to one embodiment of the present disclosure for achieving the above-described task, a computer program stored in a computer-readable storage medium, wherein the computer program, when executed on one or more processors, performs an operation of obtaining a user-customized neural network model for predicting a user's biometric data based on a dual buffer, the operation including an operation of obtaining a neural network model trained to predict a user's second biometric data based on the user's first biometric data and an operation of updating the neural network model based on at least one of the first biometric data stored in the first buffer and the second biometric data stored in the second buffer.

[0018] A computing device for obtaining a user-customized neural network model for predicting a user's biometric data based on a dual buffer according to an embodiment of the present disclosure for realizing the task as described above, comprises a memory including a first buffer and a second buffer, and a processor for obtaining a neural network model trained to predict a user's second biometric data based on the user's first biometric data, and updating the neural network model based on at least one of the first biometric data stored in the first buffer and the second biometric data stored in the second buffer.

[0019] According to a method for obtaining a user-customized neural network model for predicting a user's biometric data based on a dual buffer according to an embodiment of the present disclosure, learning data including label data and learning data excluding label data are distinguished and stored in different separate buffers, and supervised learning and self-supervised learning are performed in parallel based on the learning data stored in each buffer, thereby obtaining a user-customized neural network model that can actively adapt to changes in distribution or changes in individual user biometric signal characteristics.

[0020] FIG. 1 is an exemplary diagram of a method for obtaining a user-customized neural network model for predicting a user's biometric data based on a dual buffer according to an embodiment of the present disclosure.

[0021] FIG. 2 is a block diagram of a computing device that obtains a user-customized neural network model for predicting a user's biometric data based on a dual buffer according to an embodiment of the present disclosure.

[0022] FIG. 3 is a schematic flowchart of a method for obtaining a user-customized neural network model for predicting a user's biometric data based on a dual buffer according to an embodiment of the present disclosure.

[0023] FIGS. 4A and 4B are exemplary diagrams illustrating updating a neural network model based on learning data stored in a dual buffer according to an embodiment of the present disclosure.

[0024] FIG. 5 is a block diagram of a computing device according to another embodiment of the present disclosure.

[0025] Below, embodiments of the present disclosure are described in detail with reference to the attached drawings so that those skilled in the art can easily implement the present disclosure. The embodiments presented in this disclosure are provided to enable those skilled in the art to utilize or implement the contents of the present disclosure. Accordingly, various modifications to the embodiments of the present disclosure will be apparent to those skilled in the art. That is, the present disclosure may be implemented in various different forms and is not limited to the embodiments described below.

[0026] Throughout the specification of this disclosure, identical or similar drawing numbers refer to identical or similar components. Furthermore, for clarity in the description of this disclosure, drawing numbers for parts unrelated to the description of this disclosure may be omitted in the drawings.

[0027] The term "or" as used herein is intended to mean an inclusive "or" rather than an exclusive "or." That is, unless otherwise specified herein or clear from context, "X employs A or B" should be understood to mean either of its natural inclusive permutations. For example, unless otherwise specified herein or clear from context, "X employs A or B" can be interpreted to mean either X employs A, X employs B, or X employs both A and B.

[0028] The term "and / or" as used herein should be understood to refer to and include all possible combinations of one or more of the related concepts listed.

[0029] The terms "comprises" and / or "comprising" as used herein should be understood to mean the presence of certain features and / or components. However, it should be understood that the terms "comprises" and / or "comprising" do not exclude the presence or addition of one or more other features, other components, and / or combinations thereof.

[0030] Unless otherwise specified in this disclosure or unless the context makes it clear that the singular form is intended to be referred to, the singular should generally be construed to include “one or more.”

[0031] The term "Nth (N is a natural number)" used in the present disclosure can be understood as an expression used to mutually distinguish components of the present disclosure based on a predetermined standard such as a functional perspective, a structural perspective, or convenience of explanation. For example, components performing different functional roles in the present disclosure can be distinguished as a first component or a second component. However, components that are substantially the same within the technical spirit of the present disclosure but must be distinguished for convenience of explanation may also be distinguished as a first component or a second component.

[0032] The term "acquisition" as used in this disclosure may be understood to mean not only receiving data through a wired or wireless communication network with an external device or system, but also generating data in an on-device form.

[0033] Meanwhile, the term "module" or "unit" used in the present disclosure can be understood as a term referring to an independent functional unit that processes computing resources, such as a computer-related entity, firmware, software or a part thereof, hardware or a part thereof, or a combination of software and hardware. At this time, the "module" or "unit" may be a unit composed of a single element, or a unit expressed as a combination or set of multiple elements. For example, as a narrow concept, a "module" or "unit" may refer to a hardware element of a computing device or a set thereof, an application program that performs a specific function of software, a processing process implemented through software execution, or a set of instructions for program execution, etc. In addition, as a broad concept, a "module" or "unit" may refer to the computing device itself that constitutes the system, or an application running on the computing device, etc. However, since the above-described concept is only an example, the concept of “module” or “part” may be defined in various ways within a range understandable to those skilled in the art based on the contents of the present disclosure.

[0034] The term "model" as used herein may be understood as a system implemented using mathematical concepts and language to solve a specific problem, a set of software units to solve a specific problem, or an abstract model of a processing process to solve a specific problem. For example, a neural network "model" may refer to the entire system implemented as a neural network that has problem-solving capabilities through learning. In this case, the neural network can have problem-solving capabilities by optimizing the parameters connecting nodes or neurons through learning. A neural network "model" may include a single neural network or a set of neural networks that are a combination of multiple neural networks.

[0035] The term "data" used in this disclosure may include "image," "signal," and the like. The term "image" used in this disclosure may refer to multidimensional data composed of discrete image elements. In other words, "image" may be understood as a term referring to a digital representation of an object visible to the human eye. For example, "image" may refer to multidimensional data composed of elements corresponding to pixels in a two-dimensional image. "Image" may refer to multidimensional data composed of elements corresponding to voxels in a three-dimensional image.

[0036] The explanation of the above terms is intended to aid understanding of the present disclosure. Therefore, unless explicitly stated as limiting the contents of the present disclosure, it should be noted that the above terms are not intended to limit the technical ideas of the contents of the present disclosure.

[0037] FIG. 1 is an exemplary diagram of a method for obtaining a user-customized neural network model for predicting a user's biometric data based on a dual buffer according to an embodiment of the present disclosure.

[0038] A computing device (100) according to an embodiment of the present disclosure may be a hardware device or a part of a hardware device that performs comprehensive processing and calculation of data, or may be a software-based computing environment connected to a communication network. For example, the computing device (100) may be a server that performs intensive data processing functions and shares resources, or may be a client that shares resources through interaction with a server. In addition, the computing device (100) may be a cloud system in which multiple servers and clients interact to comprehensively process data. Since the above description is only one example related to the type of computing device (100), the type of computing device (100) may be configured in various ways within a category understandable to those skilled in the art based on the contents of the present disclosure. For example, the computing device (100) may be implemented as a server, a cloud-based virtual server, a desktop, a laptop, a smartphone, a tablet PC, a smart watch, a smart ring, and various other electronic devices.

[0039] A computing device (100) may store a neural network model (400). At this time, the computing device (100) may be trained in advance to identify another bio-data of the user (1) based on the bio-data obtained from the user (1). Here, the bio-data may include various bio-signals obtained from the user (1) (e.g., electrocardiogram (ECG), electroencephalogram (EEG), photoplethysmogram (PPG), arterial blood pressure (ABP), central venous pressure (CVP), blood sugar level (fasting blood glucose (FBG), etc.), and glycated hemoglobin (HbA1c).

[0040] Hereinafter, for the convenience of explanation of the present disclosure, biometric data acquired by a computing device (100) and input to a neural network model (400) will be referred to as first biometric data (10), and data (i.e., output data) output through a neural network model (400) using the acquired biometric signal as input data will be referred to as second biometric data (20).

[0041] According to an embodiment of the present disclosure, the first biometric data (10) and the second biometric data (20) may be different types of biometric data. The first biometric data (10) may include biometric data acquired from the user (1) based on a non-invasive measurement method. For example, the first biometric data (10) may include data corresponding to an electrocardiogram signal, an electroencephalogram signal, a photosensitive pulse wave, etc. On the other hand, the second biometric data (20) may include biometric data acquired from the user (1) based on an invasive measurement method or biometric data acquired by direct measurement by a medical professional. For example, the second biometric data (20) may include body temperature, blood pressure (e.g., arterial blood pressure, central venous blood pressure, etc.), left ventricular ejection fraction, blood sugar level (fasting blood glucose (FBG), etc.), glycated hemoglobin (HbA1c), etc. That is, the neural network model (400) can be trained to identify second biometric data (20) obtained based on an invasive method that requires intervention by a medical professional or is generally difficult to measure, from first biometric data (10) obtained based on a non-invasive method that can be easily measured from a user (1).

[0042] However, the present invention is not limited thereto, and the neural network model (400) may be pre-trained to predict various user states as well as biometric data. Here, the user's state may include health status, likelihood of disease onset, stress level, etc. In this case, the neural network model (400) may be trained to output information about the user's state when biometric data (i.e., first biometric data (10)) is input. For example, when electrocardiogram data is input, the neural network model (400) may produce a score corresponding to the likelihood of the user's specific disease (specifically, heart disease such as left ventricular systolic dysfunction, left ventricular diastolic dysfunction, acute heart failure, etc.).

[0043] The computing device (100) can update a pre-trained neural network model (400) to suit the user. Here, updating the neural network model (400) may mean adjusting parameters included in the neural network model (400) by retraining the neural network model (400) based on biometric data directly acquired from the user. This is to enable the neural network model (400) to derive prediction results specialized for a specific user by reflecting individual differences in data due to factors such as physiological characteristics, lifestyle habits, and health conditions that are different for each user.

[0044] A computing device (100) can obtain first biometric data (10) and second biometric data (20) from a user. At this time, the computing device (100) can construct different learning data sets based on the first biometric data (10) and the second biometric data (20). Specifically, the computing device (100) can construct a learning data set (hereinafter, “first learning data set”) consisting only of the first biometric data (10) and a learning data set (hereinafter, “second learning data set”) including the first biometric data (10) and the second biometric data (20) as a pair. Since the second biometric data (20) corresponds to the output value of the neural network model (400), it can be utilized as label data in the second learning data set. And, the computing device (100) can update the neural network model (400) by storing different learning data sets (i.e., the first and second learning data sets) in separate buffers and performing re-learning based on the learning data sets (i.e., the first and second learning data sets) stored in each buffer. The first biometric data (10) acquired through a non-invasive method can be acquired relatively more frequently than the second biometric data (20) acquired through an invasive method. Therefore, when the first learning data set and the second learning data set are stored together in the same buffer, the second biometric data (20) constituting the second learning data set by the first biometric data (10) acquired relatively frequently can be deleted early according to a FIFO (First-In First-Out) structure. Accordingly, one embodiment of the present disclosure stores a second learning data set composed of second biometric data (20) as label data in a separate buffer from the first learning data set, thereby enabling a plurality of second biometric data (20) acquired over a sufficient period of time to be utilized for re-learning of a neural network model (400), thereby enabling highly reliable supervised learning to be performed stably.

[0045] According to one embodiment of the present disclosure, the learning methods based on the learning data sets (i.e., the first and second learning data sets) stored in each buffer may be different.

[0046] According to one embodiment of the present disclosure, the computing device (100) may store multiple neural network models corresponding to each user. Accordingly, when biometric data is acquired, the computing device (100) may identify the user who is the subject of the acquired biometric data, select a neural network model corresponding to the user, and update the selected neural network model based on the acquired biometric data.

[0047] Hereinafter, embodiments of the present disclosure will be described in detail with reference to FIGS. 2 to 5. Meanwhile, the description of the present disclosure described above can be equally applied to embodiments of the present disclosure described below.

[0048] FIG. 2 is a block diagram of a computing device that obtains a user-customized neural network model for predicting a user's biometric data based on a dual buffer according to an embodiment of the present disclosure.

[0049] Referring to FIG. 2, a computing device (100) according to an embodiment of the present disclosure may include a processor (110), a memory (120), and a communication interface (communication unit) (130). However, FIG. 2 is merely an example, and the computing device (100) may include other components for implementing a computing environment. In addition, only some of the disclosed components may be included in the computing device (100).

[0050] A processor (110) according to an embodiment of the present disclosure may be understood as a configuration unit including hardware and / or software for performing computing operations. For example, the processor (110) may read a computer program to perform data processing for machine learning. The processor (110) may process computational processes such as processing input data for machine learning, feature extraction for machine learning, and error calculation based on backpropagation. The processor (110) for performing such data processing may include a central processing unit (CPU), a general purpose graphics processing unit (GPGPU), a tensor processing unit (TPU), an application specific integrated circuit (ASIC), or a field programmable gate array (FPGA). The above-described type of processor (110) is only one example, and thus, the type of processor (110) may be configured in various ways within a range understandable to those skilled in the art based on the contents of the present disclosure.

[0051] The processor (110) is connected to other components of the computing device (100) (i.e., memory (120) and communication interface (130)) and controls the overall operation of the computing device (100).

[0052] The memory (120) according to one embodiment of the present disclosure may be understood as a configuration unit including hardware and / or software for storing and managing data processed in the computing device (100). That is, the memory (120) may store any type of data generated or determined by the processor (110) and any type of data received by the communication interface (130). For example, the memory (120) may include at least one type of storage medium among a flash memory type, a hard disk type, a multimedia card micro type, a card type memory, a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, and an optical disk. In addition, the memory (120) may also include a database system that controls and manages data in a predetermined system. The type of memory (120) described above is only one example, and thus the type of memory (120) can be configured in various ways within a range understandable to those skilled in the art based on the contents of the present disclosure.

[0053] The memory (120) may include a first buffer and a second buffer that store a first learning data set and a second learning data set, respectively. The buffers (i.e., the first and second buffers) are memory spaces used to coordinate data flow between components with different processing speeds or acquisition cycles, and enable the acquired biometric data to be temporarily stored and sequentially processed. In particular, according to one embodiment of the present disclosure, the memory (120) prevents data loss by storing the first biometric data, which is frequently acquired in a non-invasive manner, and the second biometric data, which is relatively less frequently acquired, in separate first and second buffers, respectively, in order to stably manage the data.

[0054] The memory (120) can structure and organize and manage data, combinations of data, and program codes executable by the processor (110) required for the processor (110) to perform operations. For example, the memory (120) can store first biometric data (e.g., electrocardiogram data, optical pulse data, etc.) and second biometric data (e.g., blood pressure data, etc.) received through a communication interface (130) to be described later. In addition, the memory (120) can store a pre-learned neural network model, and can store program codes that operate the neural network model to perform learning or re-learning, program codes that operate the neural network model to receive first biometric data and perform inference (i.e., infer second biometric data) according to the purpose of use of the computing device (100), and processed data generated as the program code is executed.

[0055] A communication interface (130) according to an embodiment of the present disclosure may be understood as a component that transmits and receives data through any known wired or wireless communication system. For example, the communication interface (130) may perform data transmission and reception using a wired or wireless communication system such as a local area network (LAN), wideband code division multiple access (WCDMA), long term evolution (LTE), wireless broadband internet (WiBro), fifth generation mobile communication (5G), ultrawide-band, ZigBee, radio frequency (RF) communication, wireless LAN, wireless fidelity, near field communication (NFC), or Bluetooth. Since the above-described communication systems are only examples, the wired and wireless communication system for data transmission and reception of the communication interface (130) may be applied in various ways other than the above-described examples.

[0056] The communication interface (130) can receive data necessary for the processor (110) to perform calculations through wired or wireless communication with any system or any client, etc. In addition, the communication interface (130) can transmit data generated through calculations of the processor (110) through wired or wireless communication with any system or any client, etc. For example, the communication interface (130) can receive the first and second biometric data of the patient through communication with a database in a hospital environment, a cloud server that performs tasks such as standardization of medical data, or an external computing device (e.g., a biometric signal measuring device). The communication interface (130) can transmit output data of each model, intermediate data derived from the calculation process of the processor (110), processed data, etc. through communication with the aforementioned database, server, or computing device, etc.

[0057] FIG. 3 is a schematic flowchart of a method for obtaining a user (1) customized neural network model (400) that predicts biometric data of a user (1) based on a dual buffer according to an embodiment of the present disclosure.

[0058] FIGS. 4a and 4b are exemplary diagrams showing updating a neural network model (400) to suit a user (1) based on learning data stored in a dual buffer according to one embodiment of the present disclosure.

[0059] Referring to FIG. 3, according to one embodiment of the present disclosure, a processor (110) can obtain a neural network model (400) trained to predict second biometric data (20) based on first biometric data (10) (S310).

[0060] Specifically, the processor (110) may obtain a neural network model (400) trained to estimate second biometric data (20) from the first biometric data (10) based on a learning data set that includes input data composed of first biometric data (10) and corresponding second biometric data (20) as label data. To this end, the processor (110) may train the neural network model (400) to predict second biometric data (20) based on the first biometric data (10) using the learning data set composed of the first biometric data (10) and the second biometric data (20).

[0061] According to one embodiment of the present disclosure, the processor (110) may perform self-supervised learning for a neural network model (400) based on a learning data set (third learning data set) composed of a plurality of first biometric data (10) acquired from a plurality of subjects. In addition, the processor (110) may perform supervised learning for the neural network model (400) based on a learning data set (fourth learning data set) composed of a plurality of first biometric data (10) acquired from a plurality of subjects and a plurality of second biometric data (20) corresponding to the plurality of first biometric data (10).

[0062] In particular, referring to FIG. 4A, the processor (110) can perform self-supervised learning to restore the first biometric data (10) inputted to the neural network model (400) based on the third learning data set. For example, the processor (110) can input a plurality of electrocardiogram data (Xsrcn, n is a natural number greater than or equal to 1) constituting the third learning data set into the neural network model (400), and train the neural network model (400) to restore each of the input electrocardiogram data. The SS processor (110) can compare the electrocardiogram data restored through the neural network model (400) with the input electrocardiogram data (i.e., original data) to calculate the loss, and repeatedly train the neural network model (400) to minimize the loss. The loss function can use a contrastive loss function or an info NCE loss function. Such self-supervised learning can improve the generalization performance of the neural network model (400) during subsequent supervised learning by having the neural network model (400) learn the time series pattern and latent feature information of the first biometric data (10) in advance.

[0063] In addition, referring to FIG. 4A, the processor (110) may perform supervised learning to generate second biometric data (20) based on the first biometric data (10) input to the neural network model (400) based on the fourth learning data set. For example, the processor (110) may input a plurality of electrocardiogram data (Xsrcn, n is a natural number greater than or equal to 1) constituting the input data of the fourth learning data set into the neural network model (400), and train the neural network model (400) to generate blood pressure data (Ysrcn, n is a natural number greater than or equal to 1) corresponding to each electrocardiogram data. At this time, the label data of the fourth learning data set may be blood pressure data of a subject of each electrocardiogram data. Alternatively, the processor (110) may input a plurality of electrocardiogram data constituting the fourth learning data set into the neural network model (400), and train the neural network model (400) to calculate the disease probability (or a numerical value representing the disease probability) of the user (1). At this time, the label data of the fourth learning data set may be information on the presence or absence of a disease (e.g., left ventricular systolic dysfunction, left ventricular systolic dysfunction, acute heart failure, etc.) of the subject of each electrocardiogram data. The processor (110) may divide the first biometric data (10) of the fourth learning data set into patch units and input the first biometric data (10) into the neural network model (400) as in the self-supervised learning process. The processor (110) can compare blood pressure data corresponding to each electrocardiogram data (i.e., blood pressure data directly measured from the subject) with blood pressure data obtained from the neural network model (400) to calculate loss, and repeatedly train the neural network model (400) to minimize the loss.The loss function may be defined as a mean squared error (MSE), a mean absolute error (MAE), etc., to quantify the difference between the output data (i.e., blood pressure data obtained from the neural network model (400)) and the label (i.e., blood pressure data directly measured from the subject). The processor (110) may perform backpropagation based on the calculated loss function, thereby repeatedly adjusting the weights of the neural network model (400), thereby performing learning.

[0064] Referring to FIG. 4A, supervised learning may be performed after or in parallel with the self-supervised learning based on the third learning data set described above. For example, the processor (110) may first perform self-supervised learning on the neural network model (400) based on the third learning data set to initialize the model (400) parameters, and then perform fine-tuning on the neural network model (400) based on the third learning data set and the fourth learning data set. At this time, when the fine-tuning is completed, the processor (110) may distribute the neural network model (400) to each user (1) or set the neural network model (400) corresponding to each user (1), and when new biometric data (i.e., the first and second biometric data (10 and 20)) measured from each user (1) are acquired, the neural network model (400) may be updated to be personalized for each user (1). This will be explained in detail in step S320.

[0065] According to one embodiment of the present disclosure, a neural network model (400) may include an encoder (410) and a plurality of heads. Specifically, the encoder (410) may compress input first biometric data (10) into a high-dimensional latent space and extract temporal patterns based on time series and latent representations within the first biometric data (10). The encoder (410) receives first biometric data (10), such as electrocardiogram data, in units of patches divided into certain sections along the time axis, and analyzes the temporal relationship between each patch and the overall pattern. The encoder (410) may be implemented with a convolutional neural network (CNN) or transformer-based structure. The latent representations output from the encoder (410) may be individually processed by the plurality of heads according to each learning purpose. For example, the first head (421) may operate as a head for self-supervised learning to restore masked input data, and the second head (422) may operate as a head for supervised learning to predict physiological indicators such as blood pressure and left ventricular ejection fraction based on biometric data. The first head (421) may include a decoder, and the second head (422) may include a regressor and a classifier.

[0066] According to one embodiment of the present disclosure, the processor (110) can obtain (S310) a pre-learned neural network model (400) and then predict the second biometric data (20) or other status of the user (1) using the neural network model (400). To explain again with the above-described example, when electrocardiogram data is obtained from the user (1), the processor (110) can input the electrocardiogram data into the pre-learned neural network model (400) to determine the blood pressure data of the user (1). Through this, the processor (110) can determine the second biometric data (20) of the user (1) without the intervention of medical staff for an invasive measurement method.

[0067] Meanwhile, according to one embodiment of the present disclosure, the processor (110) may update the neural network model (400) based on at least one of the first biometric data (10) of the user (1) stored in the first buffer (121) and the second biometric data (20) of the user (1) stored in the second buffer (122) (S320). Specifically, when the learning of the neural network model (400) is completed, the processor (110) may update the neural network model (400) based on newly acquired biometric data from the user (1). For the update, the processor (110) may stop the inference process using the neural network model (400). The newly acquired biometric data may include first biometric data (10) and second biometric data (20) directly measured from the user (1).

[0068] In particular, the processor (110) can obtain first biometric data (10) and second biometric data (20) from a biometric signal measuring device through a communication interface. For example, the processor (110) can obtain electrocardiogram data, which is the first biometric data (10), from an electrocardiogram measuring device (e.g., an electrocardiogram measuring device, a smart watch, etc.) or obtain blood pressure data, which is the second biometric data (20), from a blood pressure measuring device (e.g., a blood pressure monitor, etc.). In addition, the processor (110) can directly obtain the first biometric data (10) and second biometric data (20) of the user (1) through a sensing unit of a computing device, or can receive values ​​of the first biometric data (10) and second biometric data (20) through an input interface.

[0069] When new biometric data of the user (1) is acquired, the processor (110) can store the acquired new biometric data in the first buffer (121) and the second buffer (122).

[0070] According to one embodiment of the present disclosure, when first biometric data (10) is acquired from a user (1), the processor (110) may store the acquired first biometric data (10) in a first buffer (121). The first biometric data (10) may be acquired relatively more frequently than the second biometric data (20) described below. For example, the first biometric data (10) and the second biometric data (20) may be acquired periodically. The first biometric data (10) may be acquired every first cycle, and the second biometric data (20) may be acquired every second cycle that is longer than the first cycle.

[0071] As described above, a first learning data set consisting only of first biometric data (10), which is input data, can be stored in the first buffer (121). In particular, the processor (110) can update the first learning data set by storing the acquired first biometric data (10) in the first buffer (121) and adding it to the first learning data set whenever the first biometric data (10) is acquired from the user (1).

[0072] When second biometric data (20) corresponding to the acquired first biometric data (10) is acquired, the processor (110) may also store the first biometric data (10) in the second buffer (122). Specifically, when the processor (110) acquires the first biometric data (10) from the user (1), it may identify whether second biometric data (20) corresponding to the acquired first biometric data (10) has been acquired. Here, the second biometric data (20) corresponding to the first biometric data (10) may be second biometric data (20) measured (or acquired) within a preset time range based on the time at which the first biometric data (10) was acquired.

[0073] For example, if electrocardiogram data measured at 13:00 on June 31, 2025 is acquired, the processor (110) can identify whether blood pressure data was measured or acquired from the user (1) within 10 seconds before or after the time at which the electrocardiogram data was acquired. At this time, if the processor (110) identifies that second biometric data (20) corresponding to the acquired first biometric data (10) has been acquired, the processor (110) can store the acquired first biometric data (10) together with the second biometric data (20) corresponding to the first biometric data (10) in a second buffer (122). At this time, the first biometric data (10) stored in the second buffer (122) can constitute a second learning data set as input data and the second biometric data (20) can constitute label data corresponding to the input data. Meanwhile, if the processor (110) determines that the second biometric data (20) corresponding to the acquired first biometric data (10) has not been acquired, the processor (110) can store the first biometric data (10) only in the first buffer (121). That is, if the processor (110) determines that only the first biometric data (10) has been acquired, the processor (110) can store the acquired first biometric data (10) only in the first buffer (121).

[0074] Meanwhile, according to one embodiment of the present disclosure, when second biometric data (20) is acquired from a user (1), the processor (110) may store the acquired second biometric data (20) in a second buffer (122). At this time, a second learning data set, in which the first biometric data (10) as input data and the second biometric data (20) as label data are paired as described above, is stored in the second buffer (122). That is, when the second biometric data (20) is acquired, the processor (110) may store the acquired second biometric data (20) and the first biometric data (10) corresponding to the second biometric data (20) together in the second buffer (122). Here, the first biometric data (10) corresponding to the second biometric data (20) may be the first biometric data (10) measured (or acquired) within a preset time range based on the time at which the second biometric data (20) is acquired. Since the first biometric data (10) is acquired relatively more frequently than the second biometric data, when the second biometric data (20) is acquired, the processor (110) can select the first biometric data (10) corresponding to the second biometric data (20) from among the plurality of first biometric data acquired and store it in the second buffer (122) together with the second biometric data. For example, the processor (110) can select first biometric data corresponding to second biometric data (20) from the first buffer (121) (i.e., measured (or acquired) within a preset time range based on the time point at which the second biometric data (20) was acquired) and store the selected first biometric data and second biometric data together in the second buffer (122). In this way, the processor (110) can update the second learning data set by storing the acquired first biometric data (10) and the acquired second biometric data (20) in the second buffer (122) each time the second biometric data (20) is acquired from the user (1) and adding them to the second learning data set.

[0075] Meanwhile, when second biometric data (20) is acquired from a user (1), the processor (110) identifies whether first biometric data (10) corresponding to the acquired second biometric data (20) has been acquired, and when it is identified that the first biometric data (10) corresponding to the acquired second biometric data (20) has been acquired, the processor (110) can match the acquired first biometric data (10) and the acquired second biometric data (20) and store them in a second buffer (122). That is, the processor can match the first biometric data (10) as input data and the second biometric data (20) as label data and store them in the second buffer (122). On the other hand, when it is identified that the first biometric data (10) corresponding to the acquired second biometric data (20) has not been acquired, the processor (110) can request to acquire the first biometric data (10) corresponding to the second biometric data (20). Specifically, when the processor (110) determines that the first biometric data (10) has not been measured within a preset time range based on the time at which the second biometric data (20) was acquired, the processor (110) may request the biometric signal measuring device (or the user's (1) terminal device (e.g., a smart phone)) that acquires the first biometric data (10) to acquire the first biometric data (10) or display a message requesting the acquisition of the first biometric data (10) through the display of the computing device.

[0076] In this way, since the first biometric data (10) and the second biometric data (20) correspond to different types and the means by which the processor (110) acquires them are different, such as different biometric signal measuring devices, the processor (110) can acquire the first biometric data (10) and the second biometric data (20) separately. In addition, the processor (110) can perform a process of selecting other biometric data corresponding to the acquired biometric data or identifying whether other biometric data has been acquired.

[0077] The processor (110) can update the neural network model (400) by performing self-supervised learning on the neural network model (400) based on the first biometric data (Xn, n is a natural number greater than or equal to 1) (10) stored in the first buffer (121). That is, the processor (110) can update the neural network model (400) by performing self-supervised learning on the neural network model (400) using the first learning data set constructed based on the first biometric data (10) stored in the first buffer (121). At this time, the processor (110) can input the first biometric data (10) into the neural network model (400) and perform self-supervised learning on the neural network model (400) to restore the first biometric data (10). In this regard, the description of the self-supervised learning described above applies equally.

[0078] Meanwhile, the processor (110) may update the neural network model (400) by performing supervised learning on the neural network model (400) based on the second biometric data (Xn and Yn, n is a natural number greater than or equal to 1) (20) stored in the second buffer (122). That is, the processor (110) may update the neural network model (400) by performing supervised learning on the neural network model (400) using the second learning data set constructed based on the second biometric data (20) (and the first biometric data (10)) stored in the second buffer (122). At this time, the processor (110) may input the first biometric data (10) into the neural network model (400) and perform supervised learning on the neural network model (400) to produce the second biometric data (20). In this regard, the description of the supervised learning described above applies equally.

[0079] The processor (110) may simultaneously perform self-supervised learning based on the first learning data set and supervised learning based on the second learning data set. For example, since the first biometric data is acquired relatively more frequently than the second biometric data, the processor (110) may repeatedly perform self-supervised learning for the neural network model (400) based on the first learning data set, and when the second biometric data is sufficiently stored in the second buffer (122), supervised learning for the neural network model (400) may be simultaneously performed based on the second learning data set.

[0080] Meanwhile, the processor (110) can repeatedly update the neural network model (400) using the first biometric data (10) and the second biometric data (20) stored in the first buffer (121) and the second buffer (122).

[0081] In this regard, the processor (110) may extract a plurality of first biometric data (10) from the first buffer (121) and a plurality of second biometric data (20) from the second buffer (122) at a preset ratio, and update the neural network model (400) based on the extracted plurality of first biometric data (10) and the extracted plurality of second biometric data (20). For example, the processor (110) may extract 80% of the total learning data used for updating the neural network model (400) from the first buffer (121) and the remaining 20% ​​from the second buffer (122). Accordingly, 80% of the total learning data may be composed of the first biometric data (10) extracted from the first buffer (121), and 20% may be composed of pairs of the first biometric data (10) and the second biometric data (20) extracted from the second buffer (122). For this purpose, the total number of learning data used for updates can also be set in advance.

[0082] Meanwhile, the processor (110) identifies the number of pairs of first biometric data (10) and second biometric data (20) stored in the second buffer (122), and if it is determined that the number of pairs of first biometric data (10) and second biometric data (20) can be extracted at a preset ratio, the processor (110) can extract pairs of first biometric data (10) and first biometric data (10) and second biometric data (20) from the first buffer (121) and the second buffer (122), respectively, at the preset ratio.

[0083] At this time, the processor (110) can extract the first biometric data (10) in the most recent chronological order stored in the first buffer (121). Specifically, the processor (110) can identify the plurality of first biometric data (10) stored in the first buffer (121) in chronological order, and extract the first biometric data (10) used for updating the neural network model (400) in reverse order starting from the most recent data by a predetermined number. For example, the neural network model (400) can be trained by extracting the most recent 10 pieces of electrocardiogram data based on the time point at which the most recent electrocardiogram data stored in the first buffer (121) was measured (or stored). This can also be applied equally to pairs of first biometric data (10) and second biometric data (20) stored in the second buffer (122).

[0084] FIG. 5 is a block diagram of a computing device according to another embodiment of the present disclosure.

[0085] FIG. 5 is a block diagram of a computing device (500) according to another embodiment of the present disclosure. Referring to FIG. 5, the computing device (500) according to one embodiment of the present disclosure includes a processor (510), a memory (520), a communication interface (530), a display (540), a user interface (550), a sensing unit (560), a camera (570), a speaker (580), and a microphone (590). Among the components illustrated in FIG. 5, a detailed description of components that overlap with those illustrated in FIG. 2 will be omitted.

[0086] The display (540) can display various images and texts. Here, the images include both still images and moving images. The display (540) can also output electrocardiogram data and electrocardiogram signals corresponding to the electrocardiogram data (e.g., electrocardiogram graphs), and can also output information of second biometric data output from a neural network model. The display (540) can be implemented as various types of displays, such as an LCD (Liquid Crystal Display Panel), an OLED (Organic Light Emitting Diodes), an LCoS (Liquid Crystal on Silicon), a DLP (Digital Light Processing), etc. In addition, the display (540) can also include a driving circuit, a backlight unit, etc., which can be implemented in a form such as an a-si TFT, an LTPS (low temperature poly silicon) TFT, an OTFT (organic TFT), etc.

[0087] Meanwhile, the display (540) may be implemented as a touch screen by combining with a touch panel. The display (540) may function not only as an output interface that outputs images via the touch screen, but also as an input interface that receives a user's touch input. The display (540) may display extracted feature information, identified user status, generated prompts, and the like.

[0088] The user interface (550) is a component used by the computing device (500) to perform interaction with the user, and may include, but is not limited to, at least one of a touch sensor, a motion sensor, a button, a jog dial, and a switch. The computing device (500) may receive the user's first biometric data or second biometric data through the user interface (550).

[0089] The sensing unit (560) senses the patient's bio-signals to acquire bio-data. For example, the sensing unit (560) may detect the electrical signals of the patient's heartbeat through multiple electrodes, thereby acquiring the patient's electrocardiogram (ECG) data. Alternatively, the sensing unit (560) may include an optical sensor, and may acquire the patient's optical blood flow data through the optical sensor.

[0090] The camera (570) can capture images of the user by photographing objects around the user. Based on the captured images, the processor (510) can determine the user's biological information (height, age, gender, etc.). The camera (570) can be implemented with an imaging device such as a CMOS image sensor (CIS) having a CMOS structure, a charge coupled device (CCD) having a CCD structure, etc. However, the present invention is not limited thereto, and the camera (570) can be implemented with a camera module of various resolutions capable of capturing an object. Meanwhile, the camera (570) can be implemented with a depth camera (e.g., an IR depth camera), a stereo camera, an RGB camera, etc.

[0091] The speaker (580) is a component that outputs various audio data that have undergone various processing operations, such as decoding, amplification, and noise filtering, by an audio processing unit (not shown). The speaker (580) can output various notification sounds or voice messages. According to one embodiment of the present disclosure, the processor (510) can convert an electrical signal into a user's voice and output it through the speaker (580). As an example, the speaker (580) can output a voice message requesting acquisition of first biometric data.

[0092] The microphone (590) can receive a user's voice. The processor (510) can acquire the user's voice for controlling the computing device (100) through the microphone (590). To this end, the computing device (500) may include a model for voice recognition. Specifically, the computing device (500) may include an Automatic Speech Recognition (ASR) model for converting the user's voice into text, and may also include a Natural Language Understanding (NLU) model.

[0093] Meanwhile, according to one embodiment of the present disclosure, a non-transitory computer-readable medium storing a program for performing a method of obtaining a user-customized neural network model for predicting a user's biometric data based on the dual buffer described above may be provided. Here, the non-transitory computer-readable medium refers to a medium that semi-permanently stores data and can be read by a device, rather than a medium that stores data for a short period of time, such as a register, cache, or memory. Specifically, the various applications or programs described above may be stored and provided on a non-transitory computer-readable medium, such as a CD, DVD, hard disk, Blu-ray disk, USB, memory card, or ROM.

[0094] The various embodiments of the present disclosure described above can be combined with additional embodiments and modified within a scope that can be understood by those skilled in the art in light of the detailed description set forth above. It should be understood that the embodiments of the present disclosure are illustrative in all respects and not restrictive. For example, each component described as being single may be implemented in a distributed manner, and similarly, components described as being distributed may be implemented in a combined form. Accordingly, all changes or modifications derived from the meaning, scope, and equivalent concepts of the claims of the present disclosure should be interpreted as being included within the scope of the present disclosure.

Claims

1. A method for obtaining a user-customized neural network model for predicting a user's biometric data based on a dual buffer performed by a computing device including at least one processor, A step of obtaining a neural network model trained to predict second biometric data based on first biometric data; and A step of updating the neural network model based on at least one of the user's first biometric data stored in the first buffer and the user's second biometric data stored in the second buffer; method.

2. In paragraph 1, When first biometric data is obtained from the user, a step of storing the obtained first biometric data in the first buffer is included; method.

3. In paragraph 1, The step of storing in the above first buffer is: When first biometric data is acquired from the user, a step of identifying whether second biometric data corresponding to the acquired first biometric data has been acquired; When it is identified that second biometric data corresponding to the acquired first biometric data has been acquired, a step of storing the acquired first biometric data in the first buffer and the second buffer, respectively; and If it is determined that second biometric data corresponding to the acquired first biometric data has not been acquired, a step of storing the first biometric data only in the first buffer is included; method.

4. In paragraph 1, When second biometric data is obtained from the user, a step of storing the obtained second biometric data in the second buffer is included; method.

5. In paragraph 4, When second biometric data is acquired from the user, a step of identifying whether first biometric data corresponding to the acquired second biometric data has been acquired; When it is identified that first biometric data corresponding to the acquired second biometric data has been acquired, a step of matching the acquired first biometric data and the acquired second biometric data and storing them in the second buffer; and a step of requesting acquisition of first biometric data corresponding to the second biometric data when it is determined that the first biometric data corresponding to the acquired second biometric data has not been acquired; method.

6. In paragraph 1, The step of updating the above neural network model is: A step of updating the neural network model by performing self-supervised learning on the neural network model based on the first biometric data stored in the first buffer; method.

7. In paragraph 6, The step of updating the neural network model by performing the above self-supervised learning is: A step of inputting the first biometric data into the neural network model and performing self-supervised learning on the neural network model to restore the first biometric data; comprising; method.

8. In paragraph 6, The step of updating the above neural network model is: A step of updating the neural network model by performing supervised learning on the neural network model based on the second biometric data stored in the second buffer; method.

9. In paragraph 1, A step of extracting a plurality of first biometric data from the first buffer and a plurality of second biometric data from the second buffer at a preset ratio; and A step of updating the neural network model based on the extracted plurality of first biometric data and the extracted plurality of second biometric data; method.

10. In paragraph 9, The above extraction step is, A step of extracting the first biometric data in the most recent time order stored in the first buffer; method.

11. In paragraph 1, The above first biometric data is, Contains electrocardiogram data, The above second biometric data is, Contains blood pressure data, method.

12. A computer program stored in a computer-readable storage medium, wherein the computer program, when executed on one or more processors, performs an operation of obtaining a user-customized neural network model that predicts the user's biometric data based on a dual buffer, the operation comprising: An operation of obtaining a neural network model trained to predict second biometric data of a user based on first biometric data of the user; and An operation of updating the neural network model based on at least one of first biometric data stored in a first buffer and second biometric data stored in a second buffer; Computer program.

13. A computing device that obtains a user-customized neural network model that predicts the user's biometric data based on a dual buffer, Memory including a first buffer and a second buffer; and A processor comprising: a neural network model trained to predict second biometric data of a user based on first biometric data of the user; and a processor that updates the neural network model based on at least one of the first biometric data stored in the first buffer and the second biometric data stored in the second buffer. Computing device.

Images