Methods, programs, and apparatus for quantifying the quality of biological signals.

Abstract

According to an embodiment of the present disclosure, a method, a program, and an apparatus for quantifying quality of a biological signal are disclosed, which may include obtaining at least one of a first electrocardiogram data set labeled based on morphological features of the electrocardiogram signal or a second electrocardiogram data set labeled based on a domain expert's analysis of an interpretation of the electrocardiogram signal, and generating a machine learning model for quantifying quality of the electrocardiogram signal based on the at least one of the first electrocardiogram data set or the second electrocardiogram data set.

Description

Technical Field

[0001] The present disclosure relates to deep learning technology in the medical field, and specifically to a method and apparatus capable of quantifying by reflecting the readability of a biological signal in quality analysis.

Background Art

[0002] In order to utilize the data of a signal base in an application field, it is important to evaluate the quality of the collected signal. Therefore, metrics for evaluating the quality of signals have been studied in various application fields such as acoustics, communication, optics, and medicine. In this way, the metrics for evaluating the quality of signals studied in various application fields are collectively referred to as the signal quality index (SQI).

[0003] Most of the previously studied signal quality indexes reflect the degree of noise and artifacts generated in the signal itself in the signal quality. However, in the medical field, even if there is a lot of noise and artifacts in the signal, there are often many cases where it is reasonable to use it for clinical judgments such as diagnosing or predicting diseases. That is, even if the quality is evaluated as poor due to noise and artifacts by the previously studied signal quality index, if it sufficiently contains the information necessary for clinical reading, it needs to be evaluated as data that can be utilized in the medical field. Therefore, in the medical field, when evaluating the quality of a signal, it is necessary to reflect whether the signal is clinically readable.

[0004] For example, if we assume that we are evaluating the quality of an electrocardiogram (ECG) signal in order to use it to diagnose cardiac disease A, even if an ECG signal is judged to be of poor quality based on an existing signal quality index, it may still contain all the information necessary to diagnose cardiac disease A. Therefore, it is not appropriate to evaluate a signal as unusable simply because it lacks the information necessary to interpret cardiac disease A, even if it is judged to be of poor quality based on a reference signal quality index. In other words, signal quality must be evaluated according to the purpose for which the signal is intended to be used, so it is necessary to have a signal quality index that is optimized according to the purpose of use in the medical field. [Overview of the project] [Problems that the invention aims to solve]

[0005] This disclosure aims to provide a method and apparatus for quantifying new quality that can reflect the clinical readability of biological signals. Furthermore, this disclosure aims to provide a method and apparatus capable of generating a reliable machine learning model for the aforementioned quality quantification.

[0006] However, the issues that this disclosure aims to address are not limited to those mentioned above, and other issues not mentioned can be clearly understood from the following description. [Means for solving the problem]

[0007] One embodiment of the present disclosure for achieving the aforementioned problems discloses a method for quantifying the quality of a biological signal performed by a computing device. The method includes the steps of: obtaining at least one of a first electrocardiogram dataset labeled based on the morphological characteristics of the electrocardiogram signals, or a second electrocardiogram dataset labeled based on an analysis of the reading of the electrocardiogram signals by a domain expert; and generating a machine learning model for quantifying the quality of the electrocardiogram signals based on at least one of the first or second electrocardiogram datasets.

[0008] Alternatively, the first electrocardiogram dataset may be labeled into a first class, indicating that the electrocardiogram signals are readable, or a second class, indicating that the electrocardiogram signals are unreadable, based on noise identified based on the morphological characteristics of the electrocardiogram signals.

[0009] Alternatively, the second class may correspond to at least one of the following: a noise present in a predetermined ratio or greater that makes it impossible to identify at least one of the start or end points of the electrocardiogram signal waveform, or a noise present in a predetermined ratio or greater that makes it impossible to identify the R peak of the electrocardiogram signal.

[0010] Alternatively, the second electrocardiogram dataset may be labeled based on the mode of the domain expert's analysis regarding whether noise present in the electrocardiogram signal affects disease interpretation.

[0011] Alternatively, one of the first electrocardiogram dataset and the second electrocardiogram dataset may be divided into a training dataset, a validation dataset, and a test dataset for use in generating the machine learning model. The other of the first electrocardiogram dataset and the second electrocardiogram dataset may be used as a test dataset for generating the machine learning model.

[0012] Alternatively, the machine learning model may include a first model based on a neural network that estimates the readability of electrocardiogram signals based on an electrocardiogram dataset, and a second model based on regression analysis that estimates the readability of electrocardiogram signals based on an electrocardiogram dataset.

[0013] Alternatively, the step of generating a machine learning model for quantifying the quality of an electrocardiogram signal based on at least one of the first electrocardiogram dataset or the second electrocardiogram dataset may include the steps of: training a candidate model of the first model based on a first training dataset included in the first electrocardiogram dataset; verifying the performance of the trained candidate model based on a first validation dataset included in the first electrocardiogram dataset; evaluating the performance of at least one candidate model selected by the validation based on a first test dataset and a second test dataset which is the second electrocardiogram dataset included in the first electrocardiogram dataset; and generating the first model based on the specification of the candidate model identified by the evaluation.

[0014] Alternatively, the step of generating a machine learning model for quantifying the quality of an electrocardiogram signal based on at least one of the first electrocardiogram dataset or the second electrocardiogram dataset may include the steps of: extracting information about a signal quality index (SQI) used to evaluate noise in the electrocardiogram signal from the first electrocardiogram dataset to generate an index dataset; training a candidate model of the second model based on a third training dataset included in the generated index dataset; verifying the performance of the trained candidate model based on a third validation dataset included in the generated index dataset; evaluating the performance of at least one candidate model selected by the validation based on a third test dataset included in the generated index dataset; and generating the second model based on the specifications of the candidate model identified by the evaluation.

[0015] A method for quantifying the quality of a biological signal, performed by a computer device, is disclosed in one embodiment of the present disclosure to address the aforementioned issues. The method may include the steps of acquiring data to be read and inputting the acquired data to be read into a machine learning model to calculate a score indicating the readability of the data to be read. The machine learning model may be generated based on at least one of a first electrocardiogram dataset labeled based on the morphological characteristics of the electrocardiogram signals, or a second electrocardiogram dataset labeled based on domain expert analysis of the reading of the electrocardiogram signals.

[0016] Alternatively, the method may further include the steps of: estimating whether or not there are signal missings in the acquired data to be read; and determining whether or not the acquired data to be read is readable data by combining the calculated score and the estimated presence or absence of signal missings.

[0017] Alternatively, the presence or absence of signal leakage can be estimated by whether the signal values ​​included in the acquired data to be read are blank at a predetermined ratio or more, or whether the waveform of the signal included in the acquired data to be read has a flat shape.

[0018] As an alternative, the step of determining whether the acquired data to be read is readable by combining the calculated score and the estimated presence or absence of signal leakage may include the steps of determining whether noise exists in the signal contained in the acquired data to be read by comparing the calculated score with a critical value, and determining whether the acquired data to be read is readable based on at least one of the presence or absence of signal noise determined based on the calculated score, or the presence or absence of signal leakage estimated.

[0019] Alternatively, the step of comparing the calculated score with a critical value to determine whether noise exists in the signal contained in the acquired data to be read may include a step of determining that noise exists in the signal of a particular read if the score calculated based on that particular read is greater than or equal to the critical value.

[0020] Alternatively, the step of determining whether the acquired data to be read is readable, based on at least one of the presence or absence of noise in the signal determined based on the calculated score, or the presence or absence of signal leakage estimated, may include the step of determining that the signal of a particular read is unreadable if noise is determined to be present based on the calculated score, or if signal leakage is estimated.

[0021] An embodiment of the present disclosure for achieving the aforementioned problems discloses a computer program stored on a computer-readable storage medium. The computer program, when executed on one or more processors, is configured to perform operations for quantifying the quality of a biological signal. The operations may include acquiring at least one of a first electrocardiogram dataset labeled based on the morphological characteristics of the electrocardiogram signal, or a second electrocardiogram dataset labeled based on the empirical judgment of a domain expert related to reading the electrocardiogram signal, and generating a machine learning model for quantifying the quality of the electrocardiogram signal based on at least one of the acquired first or second electrocardiogram datasets.

[0022] A computing device for quantifying the quality of a biological signal is disclosed according to an embodiment of the present disclosure for realizing the problems described above. The device may include a processor including at least one core, a memory including program code executable by the processor, and a network unit for obtaining at least one of a first electrocardiogram dataset labeled based on morphological features of an electrocardiogram signal or a second electrocardiogram dataset labeled by the empirical judgment of a domain expert related to the reading of an electrocardiogram signal. Here, the processor can generate a machine learning model for quantifying the quality of an electrocardiogram signal based on at least one of the obtained first electrocardiogram dataset or the obtained second electrocardiogram dataset.

Advantages of the Invention

[0023] The present disclosure attempts to provide a quality quantification method and device that can reflect the readability of biological signals. In addition, the present disclosure attempts to provide a method and device that can generate a reliable machine learning model for the above-described quality quantification.

Brief Description of the Drawings

[0024] [Figure 1] It is a block diagram of a computing device according to an embodiment of the present disclosure. [Figure 2] It is a block diagram showing the generation process of a machine learning model according to an embodiment of the present disclosure. [Figure 3] It is a block diagram showing the generation process of a machine learning model according to an alternative embodiment of the present disclosure. [Figure 4] It is a block diagram showing the process of quantifying the quality of a biological signal of a computer device according to an embodiment of the present disclosure. [Figure 5] It is a flowchart showing a method for generating a machine learning model for quantifying the quality of a biological signal according to an embodiment of the present disclosure. [Figure 6] It is a flowchart showing a method for quantifying the quality of a biological signal according to an embodiment of the present disclosure. [Figure 7] It is a flowchart summarizing the process of quantifying the quality of a biological signal according to an embodiment of the present disclosure.

BEST MODE FOR CARRYING OUT THE INVENTION

[0025] Hereinafter, embodiments of the present disclosure will be described in detail so that those having ordinary knowledge in the technical field of the present disclosure (hereinafter referred to as those skilled in the art) can easily implement them with reference to the accompanying drawings. The embodiments presented in the present disclosure are provided so that those skilled in the art can use or implement the content of the present disclosure. Therefore, various modifications to the embodiments of the present disclosure will be apparent to those skilled in the art. That is, the present disclosure can be embodied in various different forms and is not limited to the following embodiments.

[0026] Throughout the specification of the present disclosure, the same or similar reference numerals refer to the same or similar components. Also, in order to clearly explain the present disclosure, the reference numerals of the parts not related to the description of the present disclosure can be omitted from the drawings.

[0027] The term "or" used in the present disclosure is intended to mean an inclusive "or" rather than an exclusive "or". That is, in the present disclosure, unless otherwise specified or the meaning is not clear in the context, "X uses A or B" should be understood to mean one of the natural inclusive substitutions. For example, in the present disclosure, unless otherwise specified or the meaning is not clear in the context, "X uses A or B" can be interpreted as either X uses A, X uses B, or X uses both A and B.

[0028] The term "and / or" used in the present disclosure should be understood to include while indicating all possible combinations of one or more of the related concepts listed.

[0029] The terms “contains” and / or “contains” as used in this disclosure should be understood to mean the presence of certain features and / or components. However, the terms “contains” and / or “contains” should be understood not to exclude the presence or addition of one or more other features, other components and / or combinations thereof.

[0030] Wherever the context does not clearly indicate otherwise or singular form in this disclosure, singular should generally be interpreted as including "one or more."

[0031] The term “nth (where n is a natural number)” as used in this disclosure can be understood as an expression used to distinguish components of this disclosure from one another based on predetermined criteria such as functional, structural, or ease of explanation. For example, components in this disclosure that perform different functional roles may be distinguished as either a first component or a second component. However, components that are substantially identical within the technical concept of this disclosure but must be distinguished for ease of explanation may also be distinguished as either a first component or a second component.

[0032] As used in this disclosure, the term “acquisition” can be understood to mean not only receiving data via a wireless communication network with an external device or system, but also generating or receiving data in an on-device form.

[0033] On the other hand, the terms "module" or "unit" as used in this disclosure can be understood as referring to an independent functional unit that processes computing resources, such as a computer-related entity, firmware, software or a part thereof, hardware or a part thereof, or a combination of software and hardware. Here, a "module" or "unit" may be a unit composed of a single element, or it may be a unit expressed as a combination or set of multiple elements. For example, as a concept, a "module" or "unit" may refer to a hardware element or set thereof of a computing device, an application program that achieves a specific function of software, a processing procedure embodied by the execution of software, or a set of instructions for executing a program. Furthermore, as a broader concept, a "module" or "unit" may refer to the computing device itself that constitutes a system, or an application executed on a computing device. However, the above concepts are merely examples, and the concepts of "module" or "unit" can be defined in various ways to the extent that a person skilled in the art can understand them based on the content of this disclosure.

[0034] As used in this disclosure, the term “model” can be understood as a system embodied using mathematical concepts and language to solve a particular problem, a set of software units to solve a particular problem, or an abstract model of a processing process to solve a particular problem. For example, a machine learning “model” may refer to any system that performs calculations based on a machine learning algorithm. Here, the machine learning algorithm may include classification algorithms such as naive Bayes and decision trees, regression analysis algorithms such as linear regression and logistic regression, and deep learning algorithms such as convolutional neural networks. The types of machine learning algorithms in this disclosure are not limited to the examples given above and can be configured in a variety of ways within the scope understandable to a person skilled in the art based on the examples given above.

[0035] As used in this disclosure, the term "image" refers to multidimensional data composed of discrete image elements. In other words, "image" can be understood as a digital representation of an object that can be seen by the human eye. For example, "image" can refer to multidimensional data composed of elements corresponding to pixels in a two-dimensional image. "Image" can also refer to multidimensional data composed of elements corresponding to voxels in a three-dimensional image.

[0036] The explanations of the terms provided above are intended to aid in understanding this disclosure. Therefore, unless the terms are explicitly stated to limit the content of this disclosure, it should be noted that they are not intended to limit the technical ideas contained herein.

[0037] Figure 1 is a block diagram of a computing device according to one embodiment of the present disclosure.

[0038] A computing device 100 according to one embodiment of this disclosure may be a hardware device or part of a hardware device that performs comprehensive data processing and calculations, or it may be a software-based computing environment connected via a communication network. For example, the computing device 100 may be a server that performs intensive data processing functions and is the entity that shares resources, or it may be a client that shares resources through interaction with a server. Furthermore, the computing device 100 may be a cloud system that enables multiple servers and clients to interact with each other to comprehensively process data. The above description is merely an example related to the type of computing device 100, and the type of computing device 100 can be configured in a variety of ways within the scope that can be understood by a person skilled in the art based on the content of this disclosure.

[0039] Referring to Figure 1, a computing device 100 according to one embodiment of the present disclosure may include a processor 110, memory 120, and a network unit 130. However, since Figure 1 is merely an example, the computing device 100 may include other configurations to embody a computer environment. Furthermore, the computing device 100 may include only a portion of the disclosed configurations.

[0040] A processor 110 according to one embodiment of the present disclosure can be understood as a component unit including hardware and / or software for performing computing operations. For example, the processor 110 can read a computer program and perform data processing for machine learning. The processor 110 can perform computational processes such as preprocessing input data for machine learning and error calculation based on backpropagation. A processor 110 for performing such data processing may include a central processing unit (CPU), a general-purpose graphics processing unit (GPGPU), a tensor processing unit (TPU), an application-specific integrated circuit (ASIC), or a field-programmable gate array (FPGA). The types of processors 110 described above are merely examples, and the types of processors 110 can be configured in a variety of ways within the scope understandable to those skilled in the art based on the content of the present disclosure.

[0041] The processor 110 can generate a machine learning model for quantifying the quality of electrocardiogram (ECG) signals based on ECG data. The processor 110 can generate a machine learning model for quantifying the quality of ECG signals using an ECG dataset labeled by factors that influence the reading of ECG signals that form the basis for clinical judgments. Here, the ECG dataset used to generate the machine learning model may include at least one of the following: an ECG set labeled based on morphological features of the waveform that influence the reading of the ECG signal, or an ECG dataset labeled by domain expert analysis of the reading of the ECG signal. A domain expert can be understood as a group or member of such a group that can interpret ECG signals to make clinical judgments, such as diagnosing a particular disease. That is, the processor 110 can leverage together the features identifiable in the waveform of the ECG signal and the empirical evidence and judgments used to read the ECG signal to generate a machine learning model that can provide a quantitative index of whether the ECG signal is of clinically usable quality.

[0042] The processor 110 can estimate the quality of the electrocardiogram data to be read using the machine learning model generated as described above. Here, the quality of the electrocardiogram data indicates whether the electrocardiogram data is clinically readable. Based on the quality of the electrocardiogram data to be read, the processor 110 can determine whether the electrocardiogram data to be read is clinically readable. Specifically, the processor 110 can input the electrocardiogram data to be read into the machine learning model to generate quantitative indicators for the signal quality of each lead of the electrocardiogram data to be read. The processor 110 can also analyze the waveform of the electrocardiogram data to determine whether there is signal leakage for each lead of the electrocardiogram data to be read. Then, the processor 110 can combine the quantitative indicators generated by the machine learning model and the results of the judgment on the presence or absence of signal leakage for each lead generated by the waveform analysis to determine whether the data is clinically readable for each lead of the electrocardiogram data to be read.

[0043] A memory 120 according to one embodiment of the present disclosure can be understood as a component unit including hardware and / or software for storing and managing data processed by the computing device 100. That is, the memory 120 can store any form of data generated or determined by the processor 110 and any form of data received by the network unit 130. For example, the memory 120 may include at least one type of storage medium from among flash memory type, hard disk type, multimedia card micro type, card type memory, RAM (random access memory), SRAM (static random access memory), ROM (read-only memory), EEPROM (electrically erasable programmable read-only memory), PROM (programmable read-only memory), magnetic memory, magnetic disk, or optical disk. The memory 120 may also include a database system for controlling and managing data in a predetermined manner. The types of memory 130 described above are merely examples, and the types of memory 120 can be configured in a variety of ways within the scope understandable to those skilled in the art based on the content of the present disclosure.

[0044] Memory 120 can structure and organize and manage data, data combinations, and program code executable by the processor 110 that are necessary for the processor 110 to perform calculations. For example, memory 120 can store electrocardiogram data acquired via the network unit 130, which will be described later. Memory 120 can store program code that causes the processor 110 to generate a machine learning model, program code that causes the processor 110 to estimate the quality of the electrocardiogram data using the generated machine learning model, and various data calculated by the execution of the program code.

[0045] A network unit 130 according to one embodiment of this disclosure can be understood as a component that transmits and receives data via any known wired wireless communication system. For example, the network unit 130 can transmit and receive data using a wired wireless communication system such as a local area network (LAN), wideband code division multiple access (WCDMA), LTE (long term evolution), WiBro (wireless broadband internet), 5G, ultra-wide-band wireless communication, ZigBee, radio frequency (RF) communication, wireless LAN, Wi-Fi (wireless fidelity), near field communication (NFC), or Bluetooth®. The above-mentioned communication systems are merely examples, and a wide variety of wired wireless communication systems for data transmission and reception of the network unit 130 are applicable beyond the examples given above.

[0046] The network unit 130 can receive data necessary for the processor 110 to perform calculations via wired or wireless communication with any system, server, or client. The network unit 130 can also transmit data generated by the processor 110's calculations via wired or wireless communication with any system, server, or client. For example, the network unit 130 can receive electrocardiogram data sets via wired or wireless communication with an electrocardiogram sensor or a medical environment database. The network unit 130 can also transmit various types of data generated by the processor 110's calculations based on the electrocardiogram data via wired or wireless communication with the electrocardiogram sensor or the medical environment database.

[0047] Figure 2 is a block diagram showing the process of generating a machine learning model according to one embodiment of the present disclosure.

[0048] Referring to Figure 2, a computer device 100 according to one embodiment of the present disclosure can generate a machine learning model 200 for quantifying the quality of electrocardiogram signals based on a first electrocardiogram dataset 10 labeled based on the morphological characteristics of the electrocardiogram signals, and a second electrocardiogram dataset 20 labeled based on domain expert analysis of the readability of the electrocardiogram signals. The computer device 100 can generate a machine learning model 200 that estimates the clinical readability of the electrocardiogram signals and provides a quantitative index by learning a machine learning-based candidate model based on the first electrocardiogram dataset 10 and the second electrocardiogram dataset 20, and tuning and evaluating the hyperparameters of the candidate model. Here, the first electrocardiogram dataset 10 may be a dataset labeled based on whether the electrocardiogram signals are readable, analyzed by noise grasped based on the morphological characteristics of the electrocardiogram signals. Furthermore, the second electrocardiogram dataset 20 may be a dataset labeled based on the results of an analysis by clinical domain experts to determine whether electrocardiogram signals can be used to interpret specific diseases.

[0049] For example, the first electrocardiogram dataset 10 may include labels based on noise distinguishable by the morphological features of the electrocardiogram signal waveform. The labels included in the first electrocardiogram dataset 10 may include a first class indicating that the electrocardiogram signal is readable due to noise perceived based on the morphological features of the electrocardiogram signal, and a second class indicating that the electrocardiogram signal is unreadable due to noise perceived based on the morphological features of the electrocardiogram signal. Here, the first or second class can be distinguished by whether the noise is present at a level that makes it possible to identify feature points used for clinical reading of the electrocardiogram signal. Specifically, if there is noise present at a predetermined ratio or higher that makes it impossible to identify at least one of the start or end points of the electrocardiogram signal waveform, or if there is noise present at a predetermined ratio or higher that makes it impossible to identify the R peak of the electrocardiogram signal, then the data containing the electrocardiogram signal may be labeled as the second class. Here, the predetermined ratio may be a ratio set by the producer or user of the computer device 100 to achieve the purpose of quantifying quality. Furthermore, if the waveform of the electrocardiogram signal is flat, the data containing that electrocardiogram signal may be labeled as Class 2. If none of the three conditions described above are met, the data containing that electrocardiogram signal may be labeled as Class 1. In this way, the electrocardiogram data included in the first electrocardiogram dataset 10 may be labeled based on how the noise, which is understood based on the morphological characteristics of the electrocardiogram signal, affects the clinical readability of the electrocardiogram signal.

[0050] The second electrocardiogram dataset 20 may include labels based on expert analysis of how noise in the electrocardiogram signals affects clinical readability. Labels in the second electrocardiogram dataset 20 may include a third class indicating signals readable by clinical domain expert analysis, and a fourth class indicating signals unreadable by clinical domain expert analysis. Since the third or fourth class is distinguished by the empirical and intuitive analysis of domain experts, the second electrocardiogram dataset 20 may be labeled based on the mode of domain expert analysis regarding the impact of noise in the electrocardiogram signals on disease interpretation. In other words, the electrocardiogram data in the second electrocardiogram dataset 20 may be labeled into the third or fourth class based on the mode that appears in the big data, which corresponds to the set of domain expert analysis results, in order to increase the reliability of the labels.

[0051] The computer device 100 can generate a high-quality model by using datasets labeled in different ways for training, validation, and testing to generate a machine learning model 200. Referring to Figure 2, the computer device 100 can use either the first electrocardiogram dataset 10 or the second electrocardiogram dataset 20 for training, validation, and testing of the machine learning model 200. The computer device 100 can then use the other of the first electrocardiogram dataset 10 or the second electrocardiogram dataset 20 for testing the machine learning model 200. In other words, by using the first electrocardiogram dataset 10 and the second electrocardiogram dataset 20 in combination to generate the machine learning model 200, the computer device 100 can generate a model whose performance is validated by various criteria and which is optimized for quantifying the quality of electrocardiogram signals.

[0052] Specifically, the computer device 100 can divide the first electrocardiogram dataset 10 into a first training dataset 11, a first validation dataset 15, and a first test dataset 19 for use. The computer device 100 can then use the second electrocardiogram dataset 20 as a second test dataset 25 to generate a machine learning model 200. That is, the computer device 100 can use a portion of the first electrocardiogram dataset 10 for training and validation of candidate models to generate the machine learning model 200, and can use the second electrocardiogram dataset 20 together with the remaining portion of the first electrocardiogram dataset 10 for evaluation of at least one candidate model selected through training and validation. The computer device 100 can then generate the machine learning model 200 based on the evaluation results of at least one candidate model performed using the remaining portion of the first electrocardiogram dataset 10 and the second electrocardiogram dataset 20.

[0053] For example, the computer device 100 can divide the first electrocardiogram dataset 10 such that the ratio of the first training dataset 11:first validation dataset 15:first test dataset 19 is 8:1:1 and use it to generate a machine learning model 200. Based on the first training dataset 11 included in the first electrocardiogram dataset 10, the computer device 100 can train candidate models designed with various parameters for generating a machine learning model. Here, the candidate model may be a model that receives electrocardiogram data lead by lead and estimates the readability of the lead-specific electrocardiogram signals included in the electrocardiogram data. Based on the first validation dataset 15 included in the first electrocardiogram dataset 10, the computer device 100 can perform performance validation on the trained candidate model and select at least one candidate model by performance validation. The computer device 100 can use the second electrocardiogram dataset 20 as a second test dataset 25 along with the first test dataset 19 included in the first electrocardiogram dataset 10 to evaluate the performance of at least one candidate model selected by performance validation. The computer device 100 can then select candidate models whose evaluation values ​​are above a predetermined standard as the final candidate models for generating the machine learning model 200. The computer device 100 can then generate the machine learning model 200 based on the specifications of the final candidate model, such as its structure and parameters. Here, the machine learning model 200 may be a model that receives electrocardiogram data lead by lead and outputs an electrocardiogram score 30 that indicates the readability of the lead-specific signals contained in the electrocardiogram data. The electrocardiogram score 30 is an index that quantitatively shows the noise evaluation result that reflects the probability value of the readability of the electrocardiogram signal, and can be expressed as a number, a symbol, etc. On the other hand, the ratio of the dataset described above is merely an example, and this disclosure is not limited thereto.

[0054] Thus, the computer device 100 can use the entire second electrocardiogram dataset 20, along with a portion of the first electrocardiogram dataset 10, to evaluate a model generated based on a portion of the first electrocardiogram dataset 10, in order to reflect the impact of signal noise on clinical interpretations such as disease diagnosis in quantifying signal quality. In other words, by using the entire second electrocardiogram dataset 20 to evaluate a model generated based on the first electrocardiogram dataset 10, the computer device 100 can ensure that the signal quality estimated by the machine learning model 200 reflects whether the electrocardiogram signal is clinically readable. By generating such a machine learning model 200, the computer device 100 can quantify signal quality so that it does not simply evaluate signal quality based on signal noise or artifacts, but rather the signal quality can indicate the clinical readability of the signal.

[0055] Figure 3 is a block diagram showing the process of generating a machine learning model according to an alternative embodiment of the present disclosure.

[0056] Referring to Figure 3, an alternative embodiment of the present disclosure may include a first model 210 based on a neural network for estimating the readability of electrocardiogram signals based on an electrocardiogram dataset, and a second model 220 based on regression analysis for estimating the readability of electrocardiogram signals based on an electrocardiogram dataset. Specifically, according to an alternative embodiment of the present disclosure, a computer device 100 can generate a machine learning model by ensembling the first model 210 based on a neural network and the second model 220 based on regression analysis.

[0057] For example, the first model 210 may include a convolutional neural network based on ResNet. Such a first model 210 can be generated by the following process. First, a candidate model of the first model 210 can be trained using the first training dataset 42. Here, training can be performed by supervised learning using labels based on the morphological features of electrocardiogram signals included in the first training dataset 42. Once the training of the candidate model of the first model 210 is complete, the candidate model of the first model 210 can be validated using the first validation dataset 43. The candidate models of the first model 210 selected through performance validation can be evaluated for performance using the second test dataset 55 along with the first test dataset 44. Then, based on the specifications of the candidate models that perform well in both the first test dataset 44 and the second test dataset 55 as a result of the performance evaluation, the first model 210 can be generated. Here, the first model 210 may be a model that receives electrocardiogram data and outputs a first electrocardiogram score 61 indicating the readability of individual lead signals included in the electrocardiogram data. The first electrocardiogram score 61 may be a concept corresponding to the electrocardiogram score 30 in Figure 2 described above.

[0058] The second model 220 may include a model based on logistic regression analysis. Such a second model 220 can be generated by the following process. First, a candidate model of the second model 220 can use an index dataset 41 extracted from the first electrocardiogram dataset 40 as input. The index dataset 41 may be a dataset generated by extracting information about signal quality indices (zerocrossSQI, minSQI, maxSQI, powerSQI, q1SQI, q3SQI, sSQI, kSQI, highfreqSQI, baseSQI, pSQI, etc.) that can be used to evaluate noise in the electrocardiogram signal. That is, the candidate model of the second model 220 can be trained by receiving a third training dataset 45 of the index dataset 41, which contains signal features that can be mathematically expressed by signal quality indices for noise evaluation. Once the training of the candidate model of the second model 220 is complete, the candidate model of the second model 220 can undergo performance validation by receiving a third validation dataset 46. Candidate models for the second model 220, selected through performance verification, can undergo performance evaluation using the third test dataset 47. Based on this performance evaluation, the second model 220 can be generated using the specifications of the candidate models that perform well on the third test dataset 47. Here, the second model 220 may be a model that receives index data extracted from electrocardiogram data and outputs a second electrocardiogram score 65 indicating the readability of individual lead signals included in the electrocardiogram signal. The second electrocardiogram score 65 may be a concept corresponding to the electrocardiogram score 30 in Figure 2 described above.

[0059] Thus, the computer device 100 can be expected to achieve generalized performance on external data other than the dataset on which training and evaluation were performed, by using a second model 220 based on regression analysis together with the first model 210, which is based on a neural network, in order to minimize the overfitting problem, which is a limitation of the first model 210. Furthermore, since the second model 220 receives features extracted by clear mathematical formulas, and the decision process and rationale for what criteria were used to make what decisions can be verified, the computer device 100 can overcome the limitation of the first model 210, which makes it difficult to verify the rationale for decisions, by utilizing the second model 220 together with it.

[0060] Furthermore, the first model 210, based on a neural network, is considered unreliable because its reasoning is unclear. The computer device 100 can provide a minimum level of safety in the reliability problem by ensembling the first model 210 and the second model 220 to generate a machine learning model. Typically, the first model 210, based on a neural network, is highly accurate but has low rationale, while the second model 220, based on regression analysis, is less accurate but has clear rationale. Therefore, the computer device 100 can customize and generate the final result value according to the user's desired purpose by combining the first model 210 and the second model 220.

[0061] Figure 4 is a block diagram showing the process of quantifying the quality of a biological signal using a computer device according to one embodiment of the present disclosure.

[0062] Referring to Figure 4, a computer device 100 according to one embodiment of the present disclosure can input the data to be read 70 into a first model 210 included in a machine learning model to calculate a first electrocardiogram score 81 indicating readability for each read of the data to be read 70. Here, the first model 210 can be generated from at least one of the following: an electrocardiogram dataset labeled based on the morphological characteristics of the electrocardiogram signal, or an electrocardiogram dataset labeled by domain expert analysis of the reading of the electrocardiogram signal. Although not shown in Figure 4, the computer device 100 can also extract index data from the data to be read 70 that includes information about the signal quality index against noise in the electrocardiogram signal. The computer device 100 can then input the index data extracted from the data to be read 70 into a second model 220 included in a machine learning model to calculate a second electrocardiogram score 85 indicating readability for each read of the data to be read 70. The second model 220 can be generated based on an index dataset extracted from an electrocardiogram dataset labeled based on the morphological characteristics of the electrocardiogram signals.

[0063] The computer device 100 can then estimate whether or not there are signal leaks (missing signals) in the data to be read 70. The computer device 100 can estimate whether or not there are signal leaks for all reads included in the data to be read 70 and generate a leak estimation result 89 for each read. Specifically, the presence or absence of signal leaks can be estimated by whether the signal value for each read of the data to be read 70 is missing by a predetermined ratio or more, or whether the waveform of the signal for each read of the data to be read 70 has a flat shape. Here, the predetermined ratio may be a ratio set by the producer or user of the computer device 100 to achieve the purpose of quantifying quality.

[0064] The computer device 100 can determine whether the data to be read 70 is readable by combining the first electrocardiogram score 81, the second electrocardiogram score 85, and the leakage presence / absence estimation result 89. The computer device 100 can generate a determination result 90 regarding whether each lead of the data to be read is readable by comprehensively analyzing the output value of the machine learning model and the estimation results for leakage presence / absence of the signal for each lead of the data to be read 70. For example, the computer device 100 can compare the first electrocardiogram score 81 with the first critical value and determine that noise is present in the signal of leads above the first critical value. The computer device 100 can also compare the second electrocardiogram score 85 with the second critical value and determine that noise is present in the signal of leads above the second critical value. Here, the first critical value and the second critical value are values ​​predetermined by the producer or user of the computer device 100 according to the purpose, and may be the same value or may be different values. Furthermore, the computer device 100 can determine that the signal of a lead determined to have leaked based on the lead-specific leak detection result 89 contains noise, and that the signal of a lead determined not to have leaked contains no noise. If the computer device 100 determines that the signal of a lead contains noise based on at least one of the determination results based on the first electrocardiogram score 81, the second electrocardiogram score 85, or the lead-specific leak detection result 89, it can determine that the signal of that lead is unreadable data. In this way, the computer device 100 can provide highly reliable quantitative information about the quality of signals by using the analysis results that can be used to determine readability in a complementary manner.

[0065] Figure 5 is a flowchart showing a method for generating a machine learning model to quantify the quality of biological signals according to one embodiment of the present disclosure.

[0066] Referring to Figure 5, a computer device 100 according to one embodiment of the present disclosure can acquire at least one of a first electrocardiogram dataset labeled based on the morphological characteristics of the electrocardiogram signal or a second electrocardiogram dataset labeled based on an analysis by a domain expert on the reading of the electrocardiogram signal (S110). Here, the first electrocardiogram dataset may be labeled into a first class indicating that the electrocardiogram signal is a readable signal or a second class indicating that the electrocardiogram signal is an unreadable signal, based on noise perceived by the morphological characteristics of the electrocardiogram signal. The second class may correspond to at least one of the following: the presence of noise at a predetermined ratio or higher that makes it impossible to identify at least one of the start or end points of the waveform of the electrocardiogram signal, or the presence of noise at a predetermined ratio or higher that makes it impossible to identify the R peak of the electrocardiogram signal. The second electrocardiogram dataset may be labeled based on the mode of an analysis by a domain expert on whether the noise present in the electrocardiogram signal affects disease reading. For example, the computer device 100 can obtain at least one of the first electrocardiogram dataset or the second electrocardiogram dataset via wireless communication with a client for labeling electrocardiogram data. The computer device 100 can also directly perform labeling on electrocardiogram data using an input / output unit and generate at least one of the first electrocardiogram dataset or the second electrocardiogram dataset.

[0067] The computer device 100 can generate a machine learning model for quantifying the quality of electrocardiogram signals based on at least one of the first electrocardiogram dataset or the second electrocardiogram dataset acquired in step S110 (S120). The computer device 100 can use the first electrocardiogram dataset by dividing it into a training dataset, a validation dataset, and a test dataset in order to generate the machine learning model. The computer device 100 can also use the second electrocardiogram dataset as a test dataset in order to generate the machine learning model. Here, the machine learning model is a model that outputs an index that quantifies the readability of the electrocardiogram signal based on the lead-specific electrocardiogram dataset, and may include a first model based on a neural network and a second model based on regression analysis.

[0068] For example, the computer device 100 can train a candidate model of the first model based on a first training dataset included in the first electrocardiogram dataset. The computer device 100 can verify the performance of the trained candidate model based on a first validation dataset included in the first electrocardiogram dataset. The computer device 100 can evaluate the performance of at least one candidate model selected by validation based on a first test dataset and a second test dataset, which is the second electrocardiogram dataset, both included in the first electrocardiogram dataset. By utilizing both the first and second test datasets in the evaluation, the computer device 100 can identify models that perform well on data labeled with different features and criteria. Here, the model identified by the computer device 100 as having good performance may be the model with the highest performance evaluation index, or it may be a model that exceeds a certain threshold value. The computer device 100 can generate the first model based on the specifications of the candidate model identified by the evaluation. Here, the model specifications are information about the parameters for constructing the neural network, and may include kernel size, depth, width, learning rate, etc.

[0069] The computer device 100 can extract information about the signal quality index used to evaluate the noise in the electrocardiogram signal from the first electrocardiogram dataset and generate an index dataset. That is, the computer device 100 can generate an index dataset based on features in the first electrocardiogram dataset that indicate the signal quality index due to noise in the electrocardiogram signal. The computer device 100 can train a candidate model of the second model based on the third training dataset included in the generated index dataset. The computer device 100 can verify the performance of the trained candidate model based on the third validation dataset included in the generated index dataset. The computer device 100 can evaluate the performance of at least one candidate model selected by validation based on the third test dataset included in the generated index dataset. The computer device 100 can identify a model with good performance based on the third test dataset. Here, the model with good performance identified by the computer device 100 may be the model with the highest performance evaluation index, or it may be a model above a certain threshold value. The computer device 100 can generate a second model based on the specifications of the candidate model identified by evaluation. Here, the model specifications may be information about the model parameters for performing logistic regression analysis.

[0070] Figure 6 is a flowchart showing a method for quantifying the quality of a biological signal according to one embodiment of the present disclosure.

[0071] Referring to Figure 6, a computer device 100 according to one embodiment of the present disclosure can acquire data to be read (S210). The data to be read can be understood as electrocardiogram data generated for use in clinical interpretation, such as the diagnosis or prediction of a specific disease. For example, the computer device 100 can receive data to be read generated by an electrocardiogram sensor via wireless communication with the electrocardiogram sensor.

[0072] The computer device 100 can input the data to be read acquired in step S210 into a machine learning model to calculate a score indicating the readability of the data (S220). Here, the machine learning model may be a model generated based on an electrocardiogram dataset labeled based on the morphological features of the electrocardiogram signals as shown in Figure 5 above, and an electrocardiogram dataset labeled based on the analysis of domain experts' interpretations of the electrocardiogram signals. For example, the computer device 100 can input the data to be read into a first model based on a neural network included in the machine learning model to calculate a first score indicating noise that reflects readability. The computer device 100 can also input the data to be read into a second model based on regression analysis included in the machine learning model to calculate a second score indicating noise that reflects readability.

[0073] On the other hand, the computer device 100 can estimate whether or not there is signal leakage in the data to be read acquired in step S210. Signal leakage estimation can be performed in parallel with step S220, in which a score is calculated via machine learning. For example, if the signal value is blank for 50% or more of a specific read of the data to be read, the computer device 100 can estimate that the signal for that read is leaked. Also, if the waveform of the signal is flat for a specific read of the data to be read, the computer device 100 can estimate that the signal for that read is leaked. The value of 50 mentioned above is merely an example, so the ratio used to determine blanks can be changed according to the purpose of use of the computer device 100.

[0074] The computer device 100 can determine whether the data to be read is readable by combining the score calculated in step S220 and the presence or absence of signal leakage estimated in the process described above. The computer device 100 can determine whether noise exists in the signal contained in the data to be read by comparing the score calculated in step S220 with a critical value. Then, the computer device 100 can determine whether the data to be read is readable based on at least one of the presence or absence of signal noise or the presence or absence of signal leakage determined based on the score. For example, the computer device 100 can determine whether noise exists in the read-specific signal of the data to be read by comparing the first score generated via a first model based on a neural network with a critical value. The computer device 100 can determine whether noise exists in the read-specific signal of the data to be read by comparing the second score generated via a second model based on regression analysis with a critical value. Then, the computer device 100 can determine whether noise exists in the read-specific signal by checking for the presence or absence of leakage in the read-specific signal. The computer device 100 can take a comprehensive look at the results of each judgment regarding the presence or absence of noise and determine whether clinical reading is possible for each read of the data to be read.

[0075] Figure 7 is a flowchart summarizing the process of quantifying the quality of a biological signal according to one embodiment of the present disclosure. Steps S310 and S320 in Figure 7 correspond to the steps in Figure 6 described above, so their explanation will be omitted below.

[0076] Referring to Figure 7, a computer device 100 according to one embodiment of the present disclosure can determine whether the score calculated by the machine learning model for the data to be read is above a critical value (S330). If the score is below the critical value with respect to a specific read, the computer device 100 can determine that there is no noise in the signal of that read (S341). Conversely, if the score is above the critical value with respect to a specific read, the computer device 100 can determine that there is noise in the signal of that read (S345). If the machine learning model includes a first model based on a neural network and a second model based on regression analysis, the computer device 100 can individually compare the score of each model with a critical value to determine the presence or absence of noise. Here, the critical value compared with the score calculated by the first model and the critical value compared with the score calculated by the second model may be the same or may be different from each other.

[0077] The computer device 100 can determine whether there is signal leakage based on whether the signals of each read of the data to be read are blank at a predetermined ratio or more, or whether the waveform of the signals of each read is flat (S350). If the signals of a particular read of the data to be read are blank at a predetermined ratio or more, or if the waveform of the signals of a particular read is flat, the computer device 100 can determine that the read has leaked signal (S361). The computer device 100 can then determine that noise is present in the signal of the read from which the signal leaked. Conversely, if the signals of a particular read of the data to be read are blank at a ratio less than the predetermined ratio, or if the waveform of the signals of a particular read is not flat, the computer device 100 can determine that the read from which the signal did not leak did not (S365). The computer device 100 can then determine that there is no noise in the signal of the read from which the signal did not leak.

[0078] If, during the process described above, it is determined that noise is present in a particular lead, the computer device 100 can determine that the signal of that particular lead is clinically unreadable (S370). That is, if the score is estimated to be above a critical value based on a particular lead and it is determined that noise is present (S345), or if it is determined that a signal has leaked (S361), the computer device 100 can determine that the signal of that lead is unreadable (S370). Conversely, if, during the process described above, it is determined that there is no noise in a particular lead, the computer device 100 can determine that the signal of that lead is clinically readable (S380). That is, if the score is estimated to be below a critical value based on a particular lead and it is determined that there is no noise (S341), and if it is determined that no signal has leaked (S365), the computer device 100 can determine that the signal of that lead is readable (S380).

[0079] The various embodiments of this disclosure described above can be combined with additional embodiments and modified within the scope understandable to those skilled in the art based on the detailed description above. It should be understood that the embodiments of this disclosure are illustrative in all respects and not limiting. For example, each component described as a single type can be implemented in a distributed manner, and similarly, components described as distributed can be implemented in a combined manner. Accordingly, all forms of modification or alteration derived from the meaning, scope and equivalent concepts of the claims of this disclosure should be construed as being within the scope of this disclosure.

Claims

1. A method for quantifying the quality of a biological signal, performed by a computing device including at least one processor, A step of obtaining at least one of the following: a first electrocardiogram dataset labeled based on the morphological characteristics of the electrocardiogram signals, or a second electrocardiogram dataset labeled based on the analysis of a domain expert's interpretation of the electrocardiogram signals; A step of generating a machine learning model for quantifying the quality of an electrocardiogram signal based on at least one of the first electrocardiogram dataset or the second electrocardiogram dataset, Methods that include...

2. The method according to claim 1, wherein the first electrocardiogram dataset is labeled into a first class indicating that the electrocardiogram signal is a readable signal, or a second class indicating that the electrocardiogram signal is an unreadable signal, based on noise identified based on the morphological characteristics of the electrocardiogram signal.

3. The aforementioned second class is, If noise is present at a predetermined ratio or higher that makes it impossible to identify at least one of the start or end points of the electrocardiogram signal waveform, or If noise exists at a certain ratio or higher that prevents the R peak of the electrocardiogram signal from being identified, The method according to claim 2, which corresponds to at least one of the following.

4. The method according to claim 1, wherein the second electrocardiogram dataset is labeled based on the mode of the domain expert's analysis of whether noise present in the electrocardiogram signal affects disease interpretation.

5. One of the first electrocardiogram dataset and the second electrocardiogram dataset is divided into a training dataset, a validation dataset, and a test dataset for use in order to generate the machine learning model. The method according to claim 1, wherein the other of the first electrocardiogram dataset and the second electrocardiogram dataset is used as a test dataset to generate the machine learning model.

6. The aforementioned machine learning model, A first model based on a neural network that estimates the readability of electrocardiogram signals based on an electrocardiogram dataset, A second model based on regression analysis to estimate the readability of electrocardiogram signals based on an electrocardiogram dataset, The method according to claim 1, including the method described in claim 1.

7. The step of generating a machine learning model for quantifying the quality of electrocardiogram signals based on at least one of the first electrocardiogram dataset or the second electrocardiogram dataset is: A step of training a candidate model of the first model based on the first training dataset included in the first electrocardiogram dataset, The steps include verifying the performance of the trained candidate model based on the first validation dataset included in the first electrocardiogram dataset, A step of evaluating the performance of at least one candidate model selected by the verification based on the first test dataset included in the first electrocardiogram dataset and the second test dataset which is the second electrocardiogram dataset, A step of generating the first model based on the specifications of the candidate models identified by the evaluation, The method according to claim 6, including the method described in claim 6.

8. The step of generating a machine learning model for quantifying the quality of electrocardiogram signals based on at least one of the first electrocardiogram dataset or the second electrocardiogram dataset is: The first electrocardiogram dataset is used to generate an index dataset by extracting information about the signal quality index (SQI) used to evaluate the noise in the electrocardiogram signal. The steps include training a candidate model of the second model based on a third training dataset included in the generated index dataset, The steps include verifying the performance of the trained candidate model based on the third validation dataset included in the generated index dataset, A step of evaluating the performance of at least one candidate model selected by the validation based on a third test dataset included in the generated index dataset, A step of generating the second model based on the specifications of the candidate model identified by the evaluation, The method according to claim 6, including the method described in claim 6.

9. A method for quantifying the quality of a biological signal, performed by a computer device including at least one processor, The stage of acquiring the data to be read, The steps include: inputting the acquired data to be read into a machine learning model to calculate a score indicating the readability of the data to be read; including, The machine learning model is generated based on at least one of the following: a first electrocardiogram dataset labeled based on the morphological features of the electrocardiogram signals, or a second electrocardiogram dataset labeled based on the analysis of domain experts' interpretations of the electrocardiogram signals. method.

10. The steps include: estimating whether or not there are signal leaks (missing signals) in the acquired data to be read; The steps include determining whether the acquired data to be read is readable data by combining the calculated score and the estimated presence or absence of signal leakage, The method according to claim 9, further comprising:

11. The method according to claim 10, wherein the presence or absence of signal leakage is estimated by whether the signal values ​​included in the acquired data to be read are blank at a predetermined ratio or more, or whether the waveform of the signal included in the acquired data to be read has a flat shape.

12. The step of determining whether the acquired data to be read is readable data by combining the calculated score and the estimated presence or absence of signal leakage is as follows: The steps include comparing the calculated score with the critical value to determine whether noise exists in the signal included in the acquired data to be read, A step of determining whether the acquired data to be read is readable data, based on at least one of the presence or absence of noise in the signal determined based on the calculated score, or the presence or absence of leakage in the signal estimated above. The method according to claim 10, including the method described in claim 10.

13. The step of comparing the calculated score with the critical value to determine whether noise exists in the signal included in the acquired data to be read is as follows: If the score calculated based on a specific lead exceeds a critical value, it is determined that noise exists in the signal of the said specific lead. The method according to claim 12, including the method described in claim 12.

14. The step of determining whether the acquired data to be read is readable data, based on at least one of the presence or absence of noise in the signal determined based on the calculated score, or the presence or absence of leakage in the signal estimated, is as follows: Based on the calculated score, the stage in which it is determined that noise is present, or that the signal of a specific read from which a signal leak is presumed to be present is unreadable data, The method according to claim 12, including the method described in claim 12.

15. A computer program stored on a computer-readable storage medium, wherein, when executed by one or more processors, the computer program performs an operation for quantifying the quality of a biological signal. The aforementioned operation is, The process involves obtaining at least one of the following: a first electrocardiogram dataset labeled based on the morphological characteristics of the electrocardiogram signals, or a second electrocardiogram dataset labeled based on the empirical judgment of a domain expert related to the reading of electrocardiogram signals. The operation of generating a machine learning model for quantifying the quality of electrocardiogram signals based on at least one of the acquired first electrocardiogram dataset or the acquired second electrocardiogram dataset, A computer program that includes [this].

16. A computing device for quantifying the quality of biological signals, A processor including at least one core, Memory containing program code executable by the aforementioned processor, A network unit for obtaining at least one of either a first electrocardiogram dataset labeled based on the morphological characteristics of the electrocardiogram signal, or a second electrocardiogram dataset labeled based on the empirical judgment of a domain expert related to the reading of the electrocardiogram signal, Includes, The processor generates a machine learning model for quantifying the quality of the electrocardiogram signal based on at least one of the acquired first electrocardiogram dataset or the acquired second electrocardiogram dataset. Device.

Images