Device, method and computer program for training neural network model on basis of training data acquired through self-labeling

Abstract

The present disclosure provides a device, a method and a computer program for training a neural network model on the basis of training data acquired through self-labeling. The method according to one embodiment of the present disclosure comprises the steps of: assigning labels to biometric data according to the self-labeling technique, and acquiring the training data based on the biometric data to which the labels have been assigned; and training the neural network model on the basis of the acquired training data.

Description

Device, method and computer program for training a neural network model based on training data acquired through a self-labeling method

[0001] The present disclosure relates to deep learning technology in the medical field, and more particularly, to a device, method, and computer program for training a neural network model based on learning data acquired through a self-labeling method.

[0002] Recent advancements in information and communication technology and artificial intelligence are enabling the analysis and utilization of diverse medical data. In particular, the electrocardiogram (ECG), a representative biosignal recording the electrical activity of the heart, is essential for the diagnosis and monitoring of heart disease. While ECG data analysis plays a crucial role in determining the type of heart disease, existing analytical methods rely on the interpretation of skilled personnel, such as specialist physicians and researchers, limiting efficiency and speed. Furthermore, the accuracy of analysis results can vary depending on the skill level of the individual.

[0003] Accordingly, AI-based automated electrocardiogram analysis technology has been introduced, contributing to improved diagnosis accuracy and interpretation speed for certain diseases. To effectively apply this AI technology, a large volume of labeled training data is essential. However, due to the nature of medical data, manual labeling by experts is required, resulting in excessive time and cost, making it difficult to secure large-scale training data. In particular, when quantitative labels are required for various features, it is difficult to maintain consistency in label definition criteria and assignment methods, which can hinder the learning accuracy and reproducibility of AI models.

[0004] The present disclosure has been made in response to the aforementioned background technology, and aims to provide a device, method, and computer program for training a neural network model based on training data acquired through a self-labeling method.

[0005] However, the problems to be solved in this disclosure are not limited to the problems mentioned above, and other problems not mentioned can be clearly understood based on the description below.

[0006] A method for training a neural network model based on learning data acquired through a self-labeling method performed by a computing device including at least one processor for solving a task as described above includes a step of labeling biometric data according to the self-labeling method, acquiring learning data based on the labeled biometric data, and a step of training a neural network model based on the acquired learning data.

[0007] Alternatively, the step of obtaining the learning data includes the step of extracting at least one feature information included in the biometric data according to the self-labeling method and assigning a label corresponding to the at least one feature information.

[0008] Alternatively, the at least one feature information includes feature information that can be derived from the biometric data.

[0009] Alternatively, the step of training the neural network model includes the step of training the neural network model to predict the at least one feature information based on the acquired training data.

[0010] Alternatively, when the at least one feature information is plural, the neural network model includes one of a regressor and a classifier corresponding to each feature information, depending on the type of the plurality of feature information.

[0011] Alternatively, once the training of the neural network model is completed, a step of fine-tuning the neural network model according to the task type is included.

[0012] A computing device for training a neural network model based on training data acquired through a self-labeling method for solving the aforementioned problem includes a processor including at least one core and a memory including program codes executable by the processor, wherein the processor labels biometric data according to the self-labeling method, acquires training data based on the labeled biometric data, and trains a neural network model based on the acquired training data.

[0013] A computer program stored in a computer-readable storage medium for solving the aforementioned problem, wherein the computer program, when executed on one or more processors, performs an operation of training a neural network model based on training data acquired through a self-labeling method, the operation including an operation of assigning a label to biometric data according to the self-labeling method, acquiring training data based on the labeled biometric data, and an operation of training a neural network model based on the acquired training data.

[0014] According to one embodiment of the present disclosure, by utilizing automatically derived feature information from biometric data as labels, a neural network model can be effectively trained without the need for manual expert intervention. This reduces labeling costs and time, while also building a high-quality foundation model that reflects diverse feature information.

[0015] FIG. 1 is a block diagram of a computing device that trains a neural network model based on learning data obtained through a self-labeling method according to an embodiment of the present disclosure.

[0016] FIG. 2 is a flowchart of a method for training a neural network model based on learning data obtained through a self-labeling method according to an embodiment of the present disclosure.

[0017] FIG. 3 is an exemplary diagram of a method for training a neural network model based on learning data obtained through a self-labeling method according to an embodiment of the present disclosure.

[0018] FIG. 4 is an exemplary diagram of a method for obtaining electrocardiogram data corresponding to a first point in time based on a plurality of electrocardiogram data obtained before a first point in time according to one embodiment of the present disclosure.

[0019] FIG. 5 is an exemplary diagram illustrating fine-tuning a neural network model (400) according to a task according to one embodiment of the present disclosure.

[0020] FIG. 6 is a block diagram of a computing device according to another embodiment of the present disclosure.

[0021] Below, embodiments of the present disclosure are described in detail with reference to the attached drawings so that those skilled in the art can easily implement the present disclosure. The embodiments presented in this disclosure are provided to enable those skilled in the art to utilize or implement the contents of the present disclosure. Accordingly, various modifications to the embodiments of the present disclosure will be apparent to those skilled in the art. That is, the present disclosure may be implemented in various different forms and is not limited to the embodiments described below.

[0022] Throughout the specification of this disclosure, identical or similar drawing numbers refer to identical or similar components. Furthermore, for the purpose of clearly describing the disclosure, drawing numbers for parts in the drawings that are not relevant to the description of the disclosure may be omitted.

[0023] The term "or" as used herein is intended to mean an inclusive "or" rather than an exclusive "or." That is, unless otherwise specified herein or clear from context, "X employs A or B" should be understood to mean either of its natural inclusive permutations. For example, unless otherwise specified herein or clear from context, "X employs A or B" can be interpreted to mean either X employs A, X employs B, or X employs both A and B.

[0024] The term "and / or" as used herein should be understood to refer to and include all possible combinations of one or more of the related concepts listed.

[0025] The terms "comprises" and / or "comprising" as used herein should be understood to mean the presence of certain features and / or components. However, it should be understood that the terms "comprises" and / or "comprising" do not exclude the presence or addition of one or more other features, other components, and / or combinations thereof.

[0026] Unless otherwise specified in this disclosure or unless the context makes it clear that the singular form is being referred to, the singular should generally be construed to include “one or more.”

[0027] The term "Nth (N is a natural number)" used in this disclosure can be understood as an expression used to distinguish components of this disclosure from each other based on a predetermined standard such as a functional perspective, a structural perspective, or convenience of explanation. For example, components performing different functional roles in this disclosure can be distinguished as a first component or a second component. However, components that are substantially the same within the technical spirit of this disclosure but must be distinguished for convenience of explanation may also be distinguished as a first component or a second component.

[0028] The term "acquisition" as used in this disclosure may be understood to mean not only receiving data through a wired or wireless communication network with an external device or system, but also generating data in an on-device form.

[0029] Meanwhile, the term "module" or "unit" used in the present disclosure can be understood as a term referring to an independent functional unit that processes computing resources, such as a computer-related entity, firmware, software or a part thereof, hardware or a part thereof, or a combination of software and hardware. At this time, the "module" or "unit" may be a unit composed of a single element, or a unit expressed as a combination or set of multiple elements. For example, as a narrow concept, a "module" or "unit" may refer to a hardware element of a computing device or a set thereof, an application program that performs a specific function of software, a processing process implemented through software execution, or a set of instructions for program execution, etc. In addition, as a broad concept, a "module" or "unit" may refer to the computing device itself that constitutes the system, or an application running on the computing device, etc. However, since the above-described concept is only an example, the concept of “module” or “part” may be defined in various ways within a range understandable to those skilled in the art based on the contents of the present disclosure.

[0030] The term "model" as used herein may be understood as a system implemented using mathematical concepts and language to solve a specific problem, a set of software units to solve a specific problem, or an abstract model of a processing process to solve a specific problem. For example, a neural network "model" may refer to the entire system implemented as a neural network that has problem-solving capabilities through learning. In this case, the neural network can have problem-solving capabilities by optimizing the parameters connecting nodes or neurons through learning. A neural network "model" may include a single neural network or a set of neural networks that are a combination of multiple neural networks.

[0031] The term "data" used in this disclosure may include "images," signals, and the like. The term "image" used in this disclosure may refer to multidimensional data composed of discrete image elements. In other words, "image" may be understood as a term referring to a digital representation of an object visible to the human eye. For example, "image" may refer to multidimensional data composed of elements corresponding to pixels in a two-dimensional image. "Image" may refer to multidimensional data composed of elements corresponding to voxels in a three-dimensional image.

[0032] The explanation of the above terms is intended to aid understanding of the present disclosure. Therefore, unless explicitly stated as limiting the contents of the present disclosure, it should be noted that the above terms are not intended to limit the technical ideas of the present disclosure.

[0033] FIG. 1 is a block diagram of a computing device (100) that trains a neural network model based on learning data acquired through a self-labeling method according to one embodiment of the present disclosure.

[0034] A computing device (100) according to an embodiment of the present disclosure may be a hardware device or a part of a hardware device that performs comprehensive processing and calculation of data, or may be a software-based computing environment connected to a communication interface. For example, the computing device (100) may be a server that performs intensive data processing functions and shares resources, or may be a client that shares resources through interaction with a server. In addition, the computing device (100) may be a cloud system in which a plurality of servers and clients interact to comprehensively process data. Since the above description is only one example related to the type of computing device (100), the type of computing device (100) may be configured in various ways within a range that can be understood by those skilled in the art based on the contents of the present disclosure.

[0035] Referring to FIG. 1, a computing device (100) includes a processor (110) (hereinafter, referred to as the processor (110)) including at least one core and a memory (120). However, FIG. 1 is merely an example, and the computing device (100) may further include other configurations for implementing a computing environment. In addition, only some of the disclosed configurations may be included in the computing device (100).

[0036] A processor (110) according to an embodiment of the present disclosure may be understood as a configuration unit including hardware and / or software for performing computing operations. For example, the processor (110) may read a computer program to perform data processing for machine learning. The processor (110) may process computational processes such as processing input data for machine learning, feature extraction for machine learning, and error calculation based on backpropagation. The processor (110) for performing such data processing may include a central processing unit (CPU), a general purpose graphics processing unit (GPGPU), a tensor processing unit (TPU), an application specific integrated circuit (ASIC), or a field programmable gate array (FPGA). The above-described type of processor (110) is only one example, and thus, the type of processor (110) may be configured in various ways within a range understandable to those skilled in the art based on the contents of the present disclosure.

[0037] The processor (110) can acquire training data required for training a neural network model. Specifically, the computing device (100) can acquire biometric data from a plurality of subjects to prepare a training data set required for training a neural network model. The biometric data may include electrocardiography (ECG) data, photoplethysmography (PPG) data, electroencephalogram (EEG) data, electromyogram (EMG) data, etc. However, for the convenience of explanation of the present disclosure, the biometric data will be described below assuming electrocardiogram data. After acquiring training data, the processor (110) can train a neural network model based on the acquired training data.

[0038] The processor (110) is electrically connected to other components of the computing device (100) (e.g., memory (120), etc.) and controls the overall operation of the computing device (100).

[0039] The memory (120) according to one embodiment of the present disclosure may be understood as a configuration unit including hardware and / or software for storing and managing data processed in the computing device (100). That is, the memory (120) may store any type of data generated or determined by the processor (110) and any type of data received by the network unit of the computing device (100). For example, the memory (120) may include at least one type of storage medium among a flash memory type, a hard disk type, a multimedia card micro type, a card type memory, a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory (120), a magnetic disk, and an optical disk. Additionally, the memory (120) may include a database system that controls and manages data in a predetermined system. The types of memory (120) described above are merely examples, and thus, the types of memory (120) may be configured in various ways within a range understandable to those skilled in the art based on the contents of the present disclosure.

[0040] The memory (120) can structure and organize and manage data, combinations of data, and program codes executable by the processor (110) required for the processor (110) to perform operations. For example, the memory (120) can store a neural network model and a learning data set used to train the neural network model, which will be described later. In addition, the memory (120) can store program codes for performing supervised learning on the neural network model based on the learning data set or for operating the neural network model for which learning has been completed, program codes for operating the neural network model to receive electrocardiogram data and perform inference according to the purpose of use of the computing device (100) as the program codes are executed, and processed data generated as the program codes are executed. In addition, the memory (120) can store an algorithm or program for performing self-labeling on biometric data.

[0041] FIG. 2 is a flowchart illustrating a method for training a neural network model (400) based on learning data acquired through a self-labeling method according to an embodiment of the present disclosure. FIG. 3 is an exemplary diagram illustrating a method for training a neural network model (400) based on learning data acquired through a self-labeling method according to an embodiment of the present disclosure.

[0042] Referring to FIG. 2, the processor (110) can label biometric data according to a self-labeling method and obtain learning data based on the labeled biometric data (S210).

[0043] The processor (110) may first acquire a plurality of biometric data. As described above, the biometric data may be acquired as electrocardiogram data. The processor (110) may acquire a plurality of electrocardiogram data measured from a plurality of subjects. Specifically, the processor (110) may acquire a plurality of electrocardiogram data acquired by another external electronic device (e.g., a biometric signal measuring device) through a communication interface or may acquire a plurality of electrocardiogram data stored in the memory (120). Here, the plurality of electrocardiogram data includes a plurality of electrocardiogram data acquired from different subjects. The plurality of electrocardiogram data may be electrocardiogram data that is not assigned a label, such as specific class information.

[0044] Meanwhile, the present invention is not limited thereto, and the processor (110) may directly measure electrocardiogram data from multiple objects using the sensing unit of the computing device (100).

[0045] In addition, the processor (110) can assign labels to each of the acquired electrocardiogram data based on a self-labeling (Sel-Labeling) method. Self-labeling is a method of automatically extracting meaningful feature information from biometric data without the intervention of a domain expert or separate diagnostic results, and utilizing the extracted feature information as a label for learning data.

[0046] The processor (110) may extract at least one feature information included in biometric data according to a self-labeling method and assign a label corresponding to the at least one feature information. Specifically, the processor (110) may extract feature information included in each electrocardiogram data using a mathematical formula, algorithm, or rule-based model set for the electrocardiogram data, and assign the extracted feature information as a label for each electrocardiogram data. The algorithm or rule-based model may be stored in the memory (120). Meanwhile, labels regarding the same type of feature information may be assigned to multiple electrocardiogram data.

[0047] At this time, at least one feature information may include feature information that can be derived from the biometric data. Specifically, the feature information may be a quantitative index derived based on structural or statistical characteristics inherent in the signal waveform of the biometric data. Accordingly, the feature information may be automatically extracted based on a mathematical formula, an algorithm, or a rule-based model. For example, referring to FIG. 4, in the case of electrocardiogram data, information such as the presence and height of the P wave, the width of the QRS complex (QRS duration), the RR interval, the heart rate (HR), and the heart rate variability (HRV) may be used as feature information. Since such information is defined based on objective criteria such as the start and end points of a specific waveform within the given electrocardiogram data (or the electrocardiogram signal corresponding to the electrocardiogram data), the time difference between peak locations, and the rate of change in potential within the interval, it can be consistently derived according to the self-labeling method without a separate domain expert or diagnostic result.

[0048] In addition, the processor (110) can train a neural network model (400) based on the acquired learning data (10) (S320). Specifically, the processor (110) can prepare learning data (10) using a plurality of labeled electrocardiogram data according to a self-labeling method, and train the neural network model (400) using a supervised learning method using the learning data (10).

[0049] In particular, the processor (110) may train the neural network model (400) to predict at least one feature information based on the acquired learning data (10). For example, the processor (110) may input a plurality of electrocardiogram data included in the input data into the neural network model (400), and may calculate a loss function based on the difference between the output value of the neural network model (400) and the corresponding label data. The loss function may be defined as at least one of the mean squared error (MSE), the mean absolute error (MAE), or the cross-entropy loss to quantify the difference between the output value and the label. The processor (110) may perform backpropagation based on the calculated loss function, and may perform learning by iteratively adjusting the weights of the neural network model (400) through this. By repeating this process, the processor (110) can train the neural network model (400) to stably predict at least one feature information from the input electrocardiogram data.

[0050] At this time, the neural network model (400) can be implemented with a multi-layer perceptron (MLP), convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks, or residual neural networks (ResNet) structure.

[0051] FIG. 4 is an exemplary diagram for explaining the structure of a neural network model (400) according to one embodiment of the present disclosure.

[0052] At this time, when there is a plurality of at least one feature information according to an embodiment of the present disclosure, the neural network model (400) may include any one of a regressor and a classifier corresponding to each feature information according to the type of the plurality of feature information. Specifically, when the feature information is expressed as a continuous numerical value, the output layer corresponding to the feature information may be configured as a regressor, and for example, referring to FIG. 4, feature information such as QRS duration, PR interval, RR interval, heart rate (HR), and heart rate variability (HRV) may be produced through regressors (420-1 to 420-n, hereinafter referred to as 420). Meanwhile, although not clearly illustrated in the drawing, when the feature information is expressed as a discrete category value, the output layer corresponding to the feature information may include a classifier. For example, feature information such as the presence or absence of a P wave, whether the rhythm is irregular, and whether the QRS waveform is normal or abnormal may be produced through the classifier.

[0053] In addition, the neural network model (400) may be configured with a multi-task learning structure for predicting multiple feature information in parallel. Specifically, the neural network model (400) may be configured to process electrocardiogram data input through a common encoder (410) and simultaneously predict each feature information through multiple output layers based on the output of the encoder (410). Based on this structure, the neural network model (400) can effectively reflect the inherent correlation between multiple feature information while individually improving the prediction accuracy of the output corresponding to each feature information.

[0054] Meanwhile, according to one embodiment of the present disclosure, when there are multiple pieces of at least one piece of feature information, the neural network model (400) may include multiple sub-neural network models corresponding to each piece of feature information. Specifically, each sub-neural network model may be configured with an individual neural network structure designed to predict at least one piece of specific feature information, and each sub-neural network model may independently perform learning and inference based on the same electrocardiogram data. For example, feature information expressed as continuous numerical values, such as QRS duration, PR interval, RR interval, heart rate (HR), heart rate variability (HRV), etc., may be respectively calculated through multiple regression-based sub-neural network models. Alternatively, feature information expressed as discrete categorical values, such as the presence or absence of a P wave, whether the rhythm is irregular, and whether the QRS waveform is normal or abnormal, may be respectively calculated through multiple classification-based sub-neural network models. Based on this structure, each sub-neural network model can be individually trained with a structure optimized for the characteristics of specific feature information, thereby minimizing interference between feature information while improving prediction accuracy for each feature information. By independently training and executing each of these multiple sub-neural network models, the processor (110) can effectively predict multiple feature information for electrocardiogram data.

[0055] FIG. 5 is an exemplary diagram illustrating fine-tuning a neural network model (400) according to a task according to one embodiment of the present disclosure.

[0056] According to one embodiment of the present disclosure, when training of the neural network model (400) is completed, the processor (110) may fine-tune the neural network model (400) according to a preset task type. At this time, the processor (110) may fine-tune the neural network model (400) to optimize it for the task. That is, the processor (110) can obtain a task-tailored neural network model (500) (hereinafter, referred to as the second neural network model (500)) by selectively learning all or part of the layers of the foundation model (400). During fine tuning, the processor (110) can replace only the output layer of the existing foundation model (400) or perform retraining including the intermediate layers, depending on the complexity of the task and the amount of learning data. The processor (110) can quickly and efficiently obtain a second neural network model (500) optimized for a specific task while maintaining the generalized expressiveness of the existing foundation model (400).

[0057] Specifically, referring to FIG. 5, the processor (110) may utilize a pre-trained neural network model (400) as a foundation model (400) based on learning data prepared through self-labeling. At this time, the processor (110) may extract an encoder (410) included in the foundation model (400), and combine a separate classifier (520) corresponding to a task type defined according to the user's purpose or environment with the output of the encoder (410) to configure a new second neural network model (500). In addition, the processor (110) may obtain learning data composed of electrocardiogram data (20) to which labels optimized for the corresponding task are assigned, and perform re-training on the second neural network model (500) including the classifier (520) combined with the encoder (410) based on the learning data. At this time, the task type may include, for example, prediction of a user's disease probability, judgment of health status, prediction of biometric data, patient-tailored diagnosis, etc.

[0058] In addition, when the foundation model (400) includes a plurality of sub-neural network models, the processor (110) can select at least one sub-neural network model suitable for the task, extract an encoder of the selected sub-neural network model, and then combine a classifier corresponding to the task to build a second neural network model (500). At this time, the processor (110) can select a plurality of sub-neural network models that produce feature values ​​necessary for classifying a preset task from among a plurality of sub-neural network models trained to produce different feature values, and extract and combine encoders of the selected sub-neural network models, thereby obtaining a second neural network model (500) suitable for the preset task. In addition, the processor (110) can perform fine-tuning on the second neural network model (500) using learning data to which labels corresponding to the task are assigned, thereby obtaining a second neural network model (500) optimized for the corresponding task.

[0059] FIG. 6 is a block diagram of a computing device (600) according to another embodiment of the present disclosure.

[0060] Referring to FIG. 6, a computing device (600) according to an embodiment of the present disclosure includes a processor (610), a memory (620), a communication interface (630), a sensing unit (640), a display (650), a user interface (660), a camera (670), and a speaker (680). Among the configurations illustrated in FIG. 6, the processor (610) and the memory (620) correspond to the configurations of the processor (110) and the memory (120) of the computing device (100) illustrated in FIG. 1, and thus a detailed description thereof will be omitted.

[0061] A communication interface (630) according to an embodiment of the present disclosure may be understood as a component that transmits and receives data through any known wired or wireless communication system. For example, the communication interface (630) may perform data transmission and reception using a wired or wireless communication system such as a local area network (LAN), wideband code division multiple access (WCDMA), long term evolution (LTE), wireless broadband internet (WiBro), fifth generation mobile communication (5G), ultrawide-band, ZigBee, radio frequency (RF) communication, wireless LAN, wireless fidelity, near field communication (NFC), or Bluetooth. Since the above-described communication systems are only examples, the wired and wireless communication system for data transmission and reception of the communication interface (630) may be applied in various ways other than the above-described examples.

[0062] The communication interface (630) can receive data required for the processor (610) to perform calculations through wired or wireless communication with any system or any client, etc. In addition, the communication interface (630) can transmit data generated through calculations of the processor (610) through wired or wireless communication with any system or any client, etc. For example, the communication interface (630) can receive a plurality of electrocardiogram data through communication with a cloud server that performs tasks such as databases in a hospital environment, standardization of medical data, or a computing device (600). In addition, the communication interface (630) can receive electrocardiogram data measured from a plurality of subjects from an external biosignal device. The communication interface (630) can transmit output data of the neural network model (foundation model (400) and task-tailored neural network model (500)), intermediate data derived from the calculation process of the processor (610), processed data, etc. through communication with the aforementioned database, server, or other computing device, etc. For example, the processor (610) can transmit the foundation model (400) or the task-tailored neural network model (500) to an external computing device (e.g., an external server device (a computing device within a hospital), a user terminal device) via a communication interface (630).

[0063] The sensing unit (640) can directly obtain electrocardiogram data for the subject. For example, the sensing unit (640) may include a plurality of electrodes. At this time, the processor (610) can measure the electrocardiogram signal of the subject through the plurality of electrodes and digitally process it to obtain the electrocardiogram. In addition, the sensing unit (640) may include an image sensor or an optical sensor. At this time, the processor (610) can obtain the user's optical blood flow signal as biometric data through the image sensor (or optical sensor).

[0064] The display (650) can display various images. Here, the images include both still images and moving images. The display (650) can display output information (e.g., disease possibility information) of the task-customized neural network model (500). The display (650) can be implemented as various types of displays, such as an LCD (Liquid Crystal Display Panel), an OLED (Organic Light Emitting Diodes), an LCoS (Liquid Crystal on Silicon), a DLP (Digital Light Processing), etc. In addition, the display (650) can also include a driving circuit, a backlight unit, etc., which can be implemented in a form such as an a-si TFT, an LTPS (low temperature poly silicon) TFT, an OTFT (organic TFT), etc.

[0065] Meanwhile, the display (650) may be implemented as a touch screen by being combined with a touch panel. In this case, the display (650) may not only function as an output interface that outputs images through the touch screen, but also as an input interface that receives a user's touch input. The display (650) may display the results of a judgment on the possibility of a disease predicted through a task-tailored neural network model, predicted electrocardiogram data, a treatment plan, and user guidance information.

[0066] The user interface (660) is a component used by the computing device (600) to perform interaction with the user, and may include at least one of a touch sensor, a motion sensor, a button, a jog dial, and a switch, but is not limited thereto. The processor (610) may receive the user's biological information (age, weight, height, gender, etc.) through the user interface (660).

[0067] The camera (670) captures images of objects surrounding the user. Specifically, the camera (670) may capture images of the user or images of food consumed by the user. At this time, the processor (610) may determine the user's nutritional status based on the user's condition (e.g., disease potential) identified through the task-tailored neural network model (500) and the images of food consumed by the user, and may provide recommended diet information related to the disease as guidance information.

[0068] For this purpose, the camera (670) may be implemented with an imaging element such as an imaging element having a CMOS structure (CIS, CMOS Image Sensor) or an imaging element having a CCD structure (Charge Coupled Device). However, the present invention is not limited thereto, and the camera (670) may be implemented with a camera module having various resolutions capable of photographing a subject. Meanwhile, the camera (670) may be implemented with a depth camera (e.g., an IR depth camera, etc.), a stereo camera, or an RGB camera.

[0069] The speaker (680) is a component that outputs various audio data that have undergone various processing operations, such as decoding, amplification, and noise filtering, by an audio processing unit (not shown). The speaker (680) can output various notification sounds or voice messages. The processor (610) can convert an electrical signal received from an external device into a user's voice and output it through the speaker (680). For example, the speaker (680) can output a voice message warning of or suggesting a diagnosis of a heart disease based on the judgment result on the possibility of a heart disease identified through a task-tailored neural network model (500).

[0070] Meanwhile, according to one embodiment of the present disclosure, a non-transitory computer-readable medium storing a program for performing a method of training a neural network model based on training data acquired through a self-labeling method for the biometric data described above may be provided. Here, the non-transitory computer-readable medium refers to a medium that permanently stores data and can be read by a device, rather than a medium that stores data for a short period of time, such as a register, cache, or memory. Specifically, the various applications or programs described above may be stored and provided on a non-transitory computer-readable medium, such as a CD, DVD, hard disk, Blu-ray disk, USB, memory card, or ROM.

[0071] The various embodiments of the present disclosure described above can be combined with additional embodiments and modified within the scope understood by those skilled in the art in light of the detailed description above. It should be understood that the embodiments of the present disclosure are illustrative in all respects and not restrictive. For example, each component described as a single component may be implemented in a distributed manner, and likewise, components described as distributed may be implemented in a combined manner. Accordingly, all changes or modifications derived from the meaning, scope, and equivalent concepts of the claims of the present disclosure should be construed as being included within the scope of the present disclosure.

Claims

1. A method for training a neural network model based on training data obtained through a self-labeling method performed by a computing device including at least one processor, A step of labeling biometric data according to a self-labeling method and obtaining learning data based on the labeled biometric data; and A step of training a neural network model based on the acquired learning data; including; method.

2. In paragraph 1, The step of acquiring the above learning data is: A step of extracting at least one feature information included in the biometric data according to the self-labeling method and assigning a label corresponding to the at least one feature information; method.

3. In paragraph 2, At least one of the above characteristic information, Including feature information that can be derived from the above biometric data, method.

4. In paragraph 2, The step of training the above neural network model is: A step of training the neural network model to predict the at least one feature information based on the acquired learning data; method.

5. In paragraph 4, If there is at least one feature information above, The above neural network model is, Depending on the type of multiple feature information, one of the regressors and classifiers corresponding to each feature information is included. method.

6. In paragraph 1, When the learning of the neural network model is completed, a step of fine-tuning the neural network model according to the task type is included. method.

7. A computing device that trains a neural network model based on learning data acquired through a self-labeling method, a processor comprising at least one core; and a memory including program codes executable by the processor; The above processor, Labeling biometric data according to a self-labeling method, obtaining learning data based on the labeled biometric data, and training a neural network model based on the obtained learning data. Computing device.

8. A computer program stored in a computer-readable storage medium, wherein the computer program, when executed on one or more processors, performs an operation of training a neural network model based on training data acquired through a self-labeling method. The above action is, An operation of labeling biometric data according to a self-labeling method and obtaining learning data based on the labeled biometric data; and An operation of training a neural network model based on the acquired learning data; including; Computer program.

Images