Method, program, and apparatus for providing electrocardiogram readout service

Abstract

According to one embodiment of the present disclosure, a method, program, and apparatus for providing an electrocardiogram reading service performed by a computing device are disclosed. The method may include: a step of analyzing a reading generated using a first machine learning model pre-trained to diagnose a disease based on an electrocardiogram to generate first additional information regarding a reading result included in the reading; a step of generating information related to the first machine learning model used to generate the reading as second additional information; and a step of providing at least one of the first additional information or the second additional information through a user interface.

Description

Technology Field

[0001] The content of the present disclosure relates to data processing technology in the medical field, and specifically to a method for processing and additionally providing information that can supplement the results of electrocardiogram readings using artificial intelligence. Background Technology

[0003] Existing automated electrocardiogram (ECG) interpretations merely indicate the possibility of a specific disease based on the ECG. However, information regarding potential diseases alone is not helpful to the physician reviewing the results. Since physicians utilizing ECG interpretation services (i.e., those who requested the service) lack specialized knowledge regarding ECG analysis, they often do not know what tests or treatments to prescribe to treat the condition, even if they confirm a diagnosis suggesting a disease. Therefore, it is necessary to inform these clients (non-specialists) not only that disease A exists but also what additional tests are required and what prescriptions should be issued.

[0004] Furthermore, existing automated ECG interpretations fail to explain the underlying reasons for the results, making it difficult to externally verify their reliability. Consequently, there is a possibility that doctors using ECG interpretation services may distrust the results and choose not to use them. Prior art literature

[0006] Korean Patent Publication No. 10-2272448 (July 2, 2021) The problem to be solved

[0007] The present disclosure aims to provide a method that not only diagnoses a disease based on an electrocardiogram reading but also provides additional information that can supplement the diagnosis, such as additional tests or measures to be taken, and explain the reason for the reading result.

[0008] However, the problems to be solved in this disclosure are not limited to those mentioned above, and other unmentioned problems may be clearly understood based on the description below. means of solving the problem

[0010] A method for providing an electrocardiogram reading service, performed by a computing device according to an embodiment of the present disclosure for realizing the aforementioned objectives, is disclosed. The method may include: a step of analyzing a reading statement generated using a first machine learning model pre-trained to diagnose a disease based on an electrocardiogram, thereby generating first additional information regarding the reading result included in the reading statement; a step of generating information related to the first machine learning model used to generate the reading statement as second additional information; and a step of providing at least one of the first additional information or the second additional information through a user interface.

[0011] Alternatively, the step of generating first additional information regarding the reading results included in the reading by analyzing a reading generated using a first machine learning model pre-trained to diagnose a disease based on the electrocardiogram may include: a step of extracting at least one of the disease name or electrocardiogram feature included in the reading as a keyword; and a step of generating first additional information including at least one of a prescription, test, or follow-up action related to the extracted keyword using a pre-established keyword database or a second machine learning model which is a pre-trained generative model.

[0012] Alternatively, the previously established keyword database may manage data in a data structure comprising, as components, key data corresponding to a disease name, sub-key data corresponding to at least one of a name regarding a prescription, test, or follow-up measure associated with the disease name, and content data corresponding to the sub-key data.

[0013] Alternatively, the step of generating first additional information including at least one of a prescription, inspection, or follow-up action related to the extracted keyword using a second machine learning model, which is a pre-established keyword database or a pre-trained generative model, may include: a step of deriving key data matching the extracted keyword from the pre-established keyword database; and a step of generating the first additional information by extracting sub-key data and content data based on the derived key data.

[0014] Alternatively, the second machine learning model may be pre-trained to generate information regarding prescriptions, tests, or follow-up measures related to keywords included in the training data, based on training data comprising at least one of diagnostic data written or reviewed by medical staff or medical electronic record data.

[0015] Alternatively, the first machine learning model may be pre-trained to receive an electrocardiogram as input and output a score indicating the likelihood of disease onset or progression.

[0016] Alternatively, the second additional information may include at least one of a cutoff value corresponding to a reading criterion for a score output from the first machine learning model, the prevalence of a disease analyzed by the first machine learning model, the sensitivity of the first machine learning model, the specificity of the first machine learning model, information regarding the training data of the first machine learning model, or information regarding the validation data of the first machine learning model.

[0017] Alternatively, the above cutoff value or the above prevalence rate may be determined based on environmental information in which electrocardiogram data input into the first machine learning model is measured.

[0018] Alternatively, the sensitivity or specificity may be determined based on the determined cutoff value.

[0019] Alternatively, the step of providing at least one of the first additional information or the second additional information through a user interface may include the step of visualizing at least one of the first additional information or the second additional information together with the reading statement in accordance with a second user request entered through the user interface.

[0020] According to one embodiment of the present disclosure for realizing the aforementioned objectives, a computer program stored on a computer-readable storage medium is disclosed. When the computer program is executed on one or more processors, it performs operations for providing an electrocardiogram reading service. At this time, the operations may include: an operation to generate a reading statement by analyzing a reading statement generated using a first machine learning model pre-trained to diagnose a disease based on an electrocardiogram, and generating a first additional information regarding the reading result included in the reading statement; an operation to generate information related to the first machine learning model used to generate the reading statement as a second additional information; and an operation to provide at least one of the first additional information or the second additional information through a user interface.

[0021] According to one embodiment of the present disclosure for realizing the aforementioned objectives, a computing device for providing an electrocardiogram reading service is disclosed. The device may include a processor comprising at least one core; and a memory comprising program codes executable on the processor. In this case, the processor may analyze a reading generated using a first machine learning model pre-trained to diagnose a disease based on an electrocardiogram, generate first additional information regarding the reading result included in the reading, generate second additional information related to the first machine learning model used to generate the reading, and provide at least one of the first additional information or the second additional information through a user interface. Effects of the invention

[0023] The present disclosure goes beyond merely diagnosing a disease based on electrocardiogram readings; by providing additional information that can supplement the diagnosis, such as additional tests or measures to be taken, and explain the reasons for the reading results, it can increase the reliability of the reading results and help customers accurately understand the reading results. Brief explanation of the drawing

[0025] FIG. 1 is a block diagram of a computing device according to one embodiment of the present disclosure. FIG. 2 is a block diagram illustrating a process of providing an electrocardiogram reading service according to one embodiment of the present disclosure. FIG. 3 is a flowchart illustrating a method for providing an electrocardiogram reading service according to one embodiment of the present disclosure. Specific details for implementing the invention

[0026] Embodiments of the present disclosure are described below with reference to the attached drawings so that those skilled in the art (hereinafter, those skilled in the art) can easily implement them. The embodiments presented in the present disclosure are provided to enable those skilled in the art to use or implement the contents of the present disclosure. Accordingly, various modifications to the embodiments of the present disclosure will be apparent to those skilled in the art. That is, the present disclosure may be embodied in various different forms and is not limited to the embodiments below.

[0027] Throughout the specification of the present disclosure, identical or similar reference numerals refer to identical or similar components. Additionally, to clearly explain the present disclosure, reference numerals in the drawings that are unrelated to the description of the present disclosure may be omitted.

[0028] The term “or” as used in this disclosure is intended to mean an implicit “or” rather than an exclusive “or.” That is, unless otherwise specified in this disclosure or its meaning is not clear from the context, “X uses A or B” should be understood to mean one of the natural implicit substitutions. For example, unless otherwise specified in this disclosure or its meaning is not clear from the context, “X uses A or B” may be interpreted as X using A, X using B, or X using both A and B.

[0029] The term “and / or” as used in this disclosure should be understood to refer to and include all possible combinations of one or more of the enumerated related concepts.

[0030] The terms “comprising” and / or “comprising” as used in this disclosure should be understood to mean the presence of certain features and / or components. However, the terms “comprising” and / or “comprising” should be understood not to exclude the presence or addition of one or more other features, other components and / or combinations thereof.

[0031] Where not otherwise specified in the present disclosure or where it is not clear from the context that the singular form indicates, the singular should generally be interpreted as including “one or more.”

[0032] The term “the N (N is a natural number)” used in this disclosure may be understood as an expression used to distinguish the components of this disclosure from one another according to certain criteria, such as functional perspectives, structural perspectives, or convenience of explanation. For example, components performing different functional roles in this disclosure may be distinguished as a first component or a second component. However, components that are substantially identical within the technical scope of this disclosure but must be distinguished for the convenience of explanation may also be distinguished as a first component or a second component.

[0033] The term “acquisition” as used in this disclosure may be understood to mean not only receiving data through a wired or wireless communication network with an external device or system, but also generating data in an on-device form.

[0034] Meanwhile, the terms "module" or "unit" used in this disclosure may be understood as referring to an independent functional unit that processes computing resources, such as a computer-related entity, firmware, software or a part thereof, hardware or a part thereof, or a combination of software and hardware. In this case, "module" or "unit" may be a unit composed of a single element, or a unit expressed as a combination or set of multiple elements. For example, in a narrow sense, "module" or "unit" may refer to a hardware element of a computing device or a set thereof, an application program that performs a specific function of software, a procedure implemented through software execution, or a set of instructions for program execution. Furthermore, in a broad sense, "module" or "unit" may refer to the computing device itself that constitutes the system, or an application executed on the computing device. However, since the above-described concept is merely an example, the concepts of "module" or "part" may be defined in various ways within the scope understandable to those skilled in the art based on the contents of this disclosure.

[0035] As used in this disclosure, the term "model" may be understood as a system implemented using mathematical concepts and language to solve a specific problem, a set of software units to solve a specific problem, or an abstract model regarding a processing process to solve a specific problem. For example, a neural network "model" may refer to an overall system implemented as a neural network that possesses problem-solving capabilities through learning. In this case, the neural network may possess problem-solving capabilities by optimizing parameters connecting nodes or neurons through learning. A neural network "model" may include a single neural network or a set of neural networks composed of multiple neural networks.

[0036] The explanation of the foregoing terms is intended to aid in understanding the present disclosure. Accordingly, it should be noted that unless a foregoing term is explicitly stated as a matter limiting the content of the present disclosure, it is not to be used in the sense of limiting the technical concept of the content of the present disclosure.

[0037] FIG. 1 is a block diagram of a computing device according to one embodiment of the present disclosure.

[0038] A computing device (100) according to one embodiment of the present disclosure may be a hardware device or part of a hardware device that performs comprehensive processing and computation of data, or it may be a software-based computing environment connected to a communication network. For example, the computing device (100) may be a server that performs intensive data processing functions and is an entity that shares resources, or it may be a client that shares resources through interaction with the server. Additionally, the computing device (100) may be a cloud system in which a plurality of servers and clients interact to comprehensively process data. Since the above description is merely one example regarding the type of computing device (100), the type of computing device (100) may be configured in various ways within a range understandable to those skilled in the art based on the contents of the present disclosure.

[0039] Referring to FIG. 1, a computing device (100) according to one embodiment of the present disclosure may include a processor (110), a memory (120), and a network unit (130). However, since FIG. 1 is merely an example, the computing device (100) may include other configurations for implementing a computing environment. Additionally, only some of the disclosed configurations may be included in the computing device (100).

[0040] A processor (110) according to one embodiment of the present disclosure may be understood as a constituent unit comprising hardware and / or software for performing computing operations. For example, the processor (110) may read a computer program and perform data processing. The processor (110) may process computational processes such as processing input data for machine learning, extracting features for machine learning, and calculating errors based on backpropagation. A processor (110) for performing such data processing may include a central processing unit (CPU), a general purpose graphics processing unit (GPGPU), a tensor processing unit (TPU), an application specific integrated circuit (ASIC), or a field programmable gate array (FPGA). Since the above-described types of processors (110) are merely examples, the types of processors (110) may be configured in various ways within a range understandable to those skilled in the art based on the contents of the present disclosure.

[0041] The processor (110) can analyze a reading generated based on artificial intelligence to generate first additional information regarding the reading results included in the reading. The processor (110) can analyze the text included in the reading and derive information matching the analyzed text using a pre-established database or machine learning model. At this time, the first additional information can be understood as information that can help in the diagnosis of a disease, such as prescriptions, tests, or follow-up measures related to the disease derived from the reading result. For example, if the reading generated based on artificial intelligence includes content simply stating that a disease may be present, such as "atrial fibrillation is suspected," the processor (110) can analyze the text of the reading and extract the keyword "atrial fibrillation." Then, the processor (110) can extract information regarding additional tests and treatments related to "atrial fibrillation" through a pre-established database or machine learning model. The processor (110) organizes the extracted information into text and can generate first additional information in the form of, "If this is a newly discovered finding, consultation with a cardiologist is required. An additional 24-hour electrocardiogram is required, and administration of an antithrombotic agent is required. Prescribe aspirin and recommend referral to a hospital with a cardiologist." At this time, the database from which the processor (110) extracts information may be a set of data in which supplementary information such as a disease and prescriptions, tests, or follow-up measures related to the disease is integrated and managed. And, the machine learning model may be a model pre-trained to generate supplementary information such as prescriptions, tests, or follow-up measures related to keywords based on keywords derived by analyzing the text of the diagnosis report.

[0042] As described above, through the first additional information generated by the processor (110), the customer who requested the electrocardiogram reading can smoothly perform the diagnosis of the disease or additional measures without performing additional medical judgment or information collection. In particular, since non-specialists using the electrocardiogram service often do not know what additional tests or treatments to perform when a disease is suspected through electrocardiogram analysis, the first additional information can provide practical assistance in the diagnosis of the disease and immediate response.

[0043] The processor (110) may generate second additional information related to the artificial intelligence model used to analyze the text in order to provide not only the reading result but also the reason why the reading result was derived through the service. When the text is generated, the processor (110) can identify the artificial intelligence model used to analyze the text. Then, the processor (110) may generate second additional information corresponding to the basis for the derived reading result by analyzing the type of data used in the development of the artificial intelligence model, indicators related to the output of the artificial intelligence model, and processing methods for the output of the artificial intelligence model. At this time, the analysis of the artificial intelligence model performed to generate the second additional information may be performed based on a database in which information regarding the data used when developing or updating the artificial intelligence model, information related to the data input into the artificial intelligence model for reading, and information related to the output of the artificial intelligence model are managed. For example, the processor (110) may perform an analysis of the specification of the machine learning model used to generate the text, the data itself used for reading or derived as a reading result, or information related to the data. Specifically, the processor (110) may generate at least one of the following as second additional information: data used to develop the machine learning model used for reading, the accuracy of the model's output, information about the hospital that verified the model's output accuracy, the sensitivity of the model, the specificity of the model, or statistical information such as the prevalence of the disease analyzed by the model.

[0044] As described above, through the second additional information generated by the processor (110), the customer who requested the electrocardiogram reading can obtain not only the reading result but also information regarding the basis from which the reading result was derived, such as what data the model used for the reading was created from, what research results it was verified through, and what the prevalence of the disease derived from the reading result is. In addition, the second additional information may include not only the probability value that the model determines by analyzing the data that there is a disease, but also the certainty (uncertainty) of that determination, thereby increasing the reliability of the reading result and helping the customer accurately understand the meaning of the reading result.

[0045] A memory (120) according to one embodiment of the present disclosure may be understood as a configuration unit comprising hardware and / or software for storing and managing data processed by a computing device (100). That is, the memory (120) may store data of any form generated or determined by a processor (110) and data of any form received by a network unit (130). For example, the memory (120) may include at least one type of storage medium among a flash memory type, a hard disk type, a multimedia card micro type, a card type memory, RAM (random access memory), SRAM (static random access memory), ROM (read-only memory), EEPROM (electrically erasable programmable read-only memory), PROM (programmable read-only memory), magnetic memory, a magnetic disk, and an optical disk. Additionally, the memory (120) may include a database system that controls and manages data in a predetermined system. Since the above-described type of memory (120) is merely an example, the type of memory (120) can be configured in various ways within a range understandable to those skilled in the art based on the contents of the present disclosure.

[0046] The memory (120) can structure and organize data, combinations of data, and program code executable by the processor (110) that are necessary for the processor (110) to perform calculations. For example, the memory (120) can store medical data received through the network unit (130) described later. The memory (120) can store program code that stores rules for processing medical data, program code that causes a neural network model to receive medical data and perform learning, program code that causes a neural network model to receive medical data and perform inference according to the purpose of use of the computing device (100), and processed data generated as the program code is executed.

[0047] A network unit (130) according to one embodiment of the present disclosure may be understood as a configuration unit that transmits and receives data through any known form of wired or wireless communication system. For example, the network unit (130) may perform data transmission and reception using wired or wireless communication systems such as a local area network (LAN), wideband code division multiple access (WCDMA), long term evolution (LTE), wireless broadband internet (WiBro), 5th generation mobile communication (5G), ultra-wide-band wireless communication, ZigBee, radio frequency (RF) communication, wireless LAN, wireless fidelity (Wi-Fi), near field communication (NFC), or Bluetooth. Since the communication systems described above are merely examples, wired or wireless communication systems for data transmission and reception of the network unit (130) may be applied in various ways other than those described above.

[0048] The network unit (130) can receive data necessary for the processor (110) to perform calculations through wired or wireless communication with any system or any client. Additionally, the network unit (130) can transmit data generated through the calculations of the processor (110) through wired or wireless communication with any system or any client. For example, the network unit (130) can receive medical data through communication with a database within a hospital environment, a cloud server performing tasks such as standardization of medical data, a client such as a smart watch, or a medical computing device. The network unit (130) can transmit output data of a neural network model, intermediate data derived during the calculation process of the processor (110), processed data, etc., through communication with the aforementioned database, server, client, or computing device.

[0049] FIG. 2 is a block diagram illustrating a process of providing an electrocardiogram reading service according to one embodiment of the present disclosure.

[0050] A computing device (100) according to one embodiment of the present disclosure can input electrocardiogram data (10) into a first machine learning model (200) to calculate a disease score (20) indicating the likelihood of disease onset or progression. At this time, the disease score (20) may be expressed as a numerical value from 0 to 100 points. The closer the disease score (20) is to 0 points, the lower the likelihood of disease onset and the healthier the state, and the closer it is to 100 points, the higher the likelihood of disease onset or the greater the severity of the disease, indicating a dangerous state.

[0051] Meanwhile, the first machine learning model (200) may be a model trained based on training data that includes electrocardiogram data and labels indicating the likelihood of disease onset or progression. For example, the first machine learning model (200) may receive electrocardiogram data included in the training data as input and extract electrocardiogram features. At this time, electrocardiogram features can be understood as features identified from the waveform of an electrocardiogram signal, such as P waves, QRS complexes, and T waves. The first machine learning model (200) may calculate a disease score based on the extracted electrocardiogram features. Then, the first machine learning model (200) may update neural network parameters by comparing the calculated disease score with the labels. At this time, the comparison between the disease score and the labels may be performed based on a loss function. When an error between the disease score and the labels is calculated through the loss function, the first machine learning model (200) may repeat the process of updating neural network parameters until the error satisfies a minimum criterion. In addition to supervised learning as described above, the first machine learning model (200) may also perform learning through self-supervised learning by including a neural network that includes an attention layer.

[0052] The computing device (100) can determine whether a disease has occurred based on the disease score (20). The computing device (100) can determine whether a disease has developed in the subject whose electrocardiogram data (10) was measured by comparing the disease score (20) with a cutoff value. For example, if the disease score (20) is greater than or equal to the cutoff value, the computing device (100) can determine that a disease has developed in the subject whose electrocardiogram data (10) was measured. If the disease score (20) is less than the cutoff value, the computing device (100) can determine that a disease has not developed in the subject whose electrocardiogram data (10) was measured. At this time, the cutoff value can be understood as a value that changes dynamically depending on the environment in which the electrocardiogram data (10) was measured. If the measurement environment is a health checkup center with a low level of urgency among medical environments, the cutoff value may increase, and if the measurement environment is an intensive care unit with a high level of urgency among medical environments, the cutoff value may decrease.

[0053] The computing device (100) can generate a reading statement by organizing the reading results into text form through a reading statement generation unit (300) implemented through a processor (110). For example, the reading statement may be in the form of reading information listed as text, such as "CONSIDER RIGHT VENTRICULAR HYPERTROPHY ... large R or R' V1 / V2", "SUPRAVENTRICULAR TACHYCARDIA, RATE 231 ... V-rate>(220-age) or 150". At this time, the reading information listed in the reading statement may include the name of the disease, electrocardiogram characteristics of the disease, etc.

[0054] A computing device (100) can extract keywords from a reading result in the form of text included in a reading report through an additional information generating unit (400) implemented through a processor (110). The computing device (100) can generate first additional information (30) including at least one of a prescription, examination, or follow-up action for a disease related to the keywords extracted through the additional information generating unit (400). At this time, the first additional information (30) can be generated through a pre-established keyword database or a second machine learning model, which is a pre-trained generative model.

[0055] For example, the additional information generation unit (400) can extract at least one of a disease name or an electrocardiogram feature as a keyword from text-based reading information included in a reading report. The additional information generation unit (400) can generate the first additional information (30) by searching a keyword database in which information regarding at least one of a prescription, test, or follow-up action for a disease that matches the extracted keyword is stored. At this time, the keyword database may manage data in a data structure that includes key data corresponding to the disease name, sub-key data corresponding to at least one of the names regarding a prescription, test, or follow-up action associated with the disease name, and content data corresponding to the sub-key data as components. The additional information generation unit (400) can search for key data that matches the keyword corresponding to at least one of the disease name or electrocardiogram feature in a keyword database in which data is structured in the form of [key data - sub-key data - content data]. Then, the additional information generation unit (400) can generate the first additional information (30) by extracting the sub-key data and content data associated with the key data from the keyword database.

[0056] Additionally, the additional information generation unit (400) may generate the first additional information (30) through a second machine learning model that generates information regarding at least one of a prescription, examination, or follow-up measure related to a keyword extracted from a reading. The additional information generation unit (400) may input at least one of a disease name or electrocardiogram feature in text form extracted from a reading into the second machine learning model to generate information regarding at least one of a prescription, examination, or follow-up measure related to a keyword as text-form data. At this time, the data form input into the second machine learning model may be the keyword itself extracted from the reading, or a sentence containing the keyword. Assuming that the disease name extracted from the report is atrial fibrillation, the additional information generation unit (400) inputs the word "atrial fibrillation" or the sentence "atrial fibrillation is suspected" into the second machine learning model to generate information regarding at least one of the prescriptions, tests, or follow-up measures related to atrial fibrillation, such as "If atrial fibrillation is a newly discovered finding, consultation with a cardiologist is required. An additional 24-hour electrocardiogram is required, and antithrombotic medication is required. Prescribe aspirin and recommend referral to a hospital with a cardiologist." as the first additional information (30).

[0057] Meanwhile, the second machine learning model may be a model pre-trained to generate information regarding prescriptions, tests, or follow-up measures related to keywords included in the training data, based on training data that includes at least one of diagnostic report data written or reviewed by medical staff or medical electronic record data. For example, the second machine learning model may include a neural network capable of processing sequential data, such as a recurrent neural network (RNN), a long short term memory (LSTM) neural network, or a transformer. Furthermore, the second machine learning model may be trained based on supervised learning through labels included in the training data, or it may be trained through self-supervised learning.

[0058] The computing device (100) can generate information related to the first machine learning model (200) used to generate a reading statement as second additional information (40) through an additional information generating unit (400) implemented through a processor (110). At this time, the second additional information (40) may include at least one of a cutoff value corresponding to a reading criterion for a score output from the first machine learning model (200), the prevalence of a disease analyzed by the first machine learning model (200), the sensitivity of the first machine learning model (200), the specificity of the first machine learning model (200), the training data of the first machine learning model (200), or the verification data of the first machine learning model (200). The cutoff value or the prevalence may be determined based on environmental information in which the electrocardiogram data (10) input to the first machine learning model (200) was measured. And, the sensitivity or the specificity above can be determined based on the determined cutoff value.

[0059] For example, when acquiring electrocardiogram data (10), the computing device (100) may acquire environmental information where the electrocardiogram data (10) was measured. Environmental information may include information about where the electrocardiogram data (10) was measured, such as a health checkup center, a general ward, or an intensive care unit. The computing device (100) may use environmental information to determine a cutoff value used to determine whether a disease has occurred based on a disease score (20). If the electrocardiogram data (10) is measured in a routine medical environment, such as a home or a health checkup center, the computing device (100) may determine a relatively high cutoff value, and if it is measured in a non-routine medical environment, such as an intensive care unit, the computing device (100) may determine a relatively low cutoff value. The computing device (100) may thus flexibly determine the cutoff value according to the environmental information. The computing device (100) may search a database that stores information regarding the relationship between the environment and the prevalence rate to determine the prevalence rate of the environment identified in the environmental information. The computing device (100) can calculate the sensitivity and specificity of the first machine learning model (200) according to a cutoff value determined based on environmental information. Additionally, the computing device (100) can search a database in which information regarding data used for the development or update of the first machine learning model (200) is stored, and collect information regarding data used for the training or verification of the first machine learning model (200). The computing device (100) can combine the cutoff value, prevalence rate, sensitivity, specificity, and information regarding data used for training or verification gathered in this way to generate second additional information (40).

[0060] The computing device (100) may generate a user interface for providing a reading statement. Additionally, the computing device (100) may provide at least one of a first additional information (30) or a second additional information (40) through the user interface for providing the reading statement. For example, when the reading of electrocardiogram data (10) is completed, the computing device (100) may provide the reading statement to the terminal of the customer who requested the electrocardiogram reading through the user interface to visualize it first. At this time, when a user request for viewing additional information is entered through the user interface, the computing device (100) may provide at least one of the first additional information (30) or the second additional information (40) to the customer's terminal through the user interface to visualize it together with the reading statement. At this time, at least one of the first additional information (30) or the second additional information (40) may be visualized by being overlaid on the reading statement, or the area may be separated from the reading statement and visualized. The computing device (100) may simultaneously visualize the reading and at least one of the first additional information (30) or the second additional information (40) by providing them to the terminal of the customer who requested the electrocardiogram reading through the user interface.

[0061] FIG. 3 is a flowchart illustrating a method for providing an electrocardiogram reading service according to one embodiment of the present disclosure.

[0062] Referring to FIG. 3, a computing device (100) according to one embodiment of the present disclosure can generate first additional information regarding the reading results included in the reading by analyzing a reading statement generated using a first machine learning model that is pre-trained to diagnose a disease based on an electrocardiogram (S100). The computing device (100) can extract at least one of the disease name or electrocardiogram feature included in the reading statement as a keyword. The computing device (100) can generate first additional information including at least one of a prescription, test, or follow-up action related to the extracted keyword by using a pre-established keyword database or a second machine learning model, which is a pre-trained generative model. Specifically, the computing device (100) can derive key data matching the extracted keyword from the pre-established keyword database. The computing device (100) can generate first additional information by extracting sub-key data and content data based on the derived key data. At this time, the extraction of the first additional information may be performed by a user request through a user interface. When a user requests to view the first additional information through a user interface, the computing device (100) can generate the first additional information by verifying key data, sub-key data, and content data. However, since a user request for the extraction of the first additional information is not necessarily required, the computing device (100) can generate the first additional information immediately when a readout is generated even without a user request. The computing device (100) may also generate the first additional information in the form of text by inputting keywords into a second machine learning model, which includes at least one of a prescription, inspection, or follow-up action related to the keywords.At this time, the second machine learning model may be a model that has been pre-trained to generate information regarding prescriptions, tests, or follow-up measures related to keywords included in the training data, based on training data that includes at least one of diagnostic report data written or reviewed by medical staff or medical electronic record data.

[0063] The computing device (100) may generate information related to the first machine learning model used to generate the reading as second additional information (S200). Since the electrocardiogram reading is performed using an artificial intelligence model such as the first machine learning model, verifying on what basis and how the reading based on the output of the first machine learning model was performed is an important part of the reliability of the reading. Accordingly, the computing device (100) may generate information regarding what data was used in the development and update of the first machine learning model, and how the performance of the first machine learning model is evaluated according to various indicators, as second additional information. For example, the second additional information may include at least one of a cutoff value corresponding to the reading criteria for the score output from the first machine learning model, the prevalence of the disease analyzed by the first machine learning model, the sensitivity of the first machine learning model, the specificity of the first machine learning model, information regarding the training data of the first machine learning model, or information regarding the validation data of the first machine learning model. At this time, the cutoff value or prevalence rate can be determined based on environmental information in which electrocardiogram data input into the first machine learning model is measured. And, sensitivity and specificity can be determined by mathematical calculation based on the determined cutoff value.

[0064] The computing device (100) may provide at least one of the first additional information or the second additional information through a user interface (S300). The computing device (100) may visualize at least one of the first additional information or the second additional information together with the reading and provide it to the terminal of the customer who requested the electrocardiogram reading through the user interface. At this time, at least one of the first additional information or the second additional information may be visualized simultaneously with the reading and exposed on the display of the terminal, or may be selectively visualized and exposed on the display of the terminal according to a user request.

[0065] For example, the computing device (100) can visualize the reading document in the form of text by including the first additional information and the second additional information together as follows.

[0066] Based on the interpretation of the electrocardiogram (ECG) requested by OO, the probability of myocardial infarction is 3.7 points. The prevalence of myocardial infarction at health checkup centers is 0.5%, and our recommended cutoff is 40 points. When applying a cutoff of 40 points, the AI ​​accuracy is 80%, sensitivity is 70%, and specificity is 65%. This AI model was trained on data measured at health checkup centers for OO adults from 2018 to 2022, validated based on ECGs measured at Hospital A from 2013 to 2015, and achieved an accuracy of OO%.

[0067] Additionally, the computing device (100) can visualize the reading in text form independently without the first additional information and the second additional information. At this time, when a customer performs a mouseover action by placing a mouse cursor over a keyword, such as a disease name or electrocardiogram characteristic, among the text included in the reading, at least one of the first additional information or the second additional information related to the selected keyword can be visualized in text form through an action such as a pop-up.

[0068] The various embodiments of the present disclosure described above may be combined with additional embodiments and modified to the extent understandable to those skilled in the art in light of the detailed description above. The embodiments of the present disclosure are illustrative in all respects and should be understood as not restrictive. For example, each component described as a single unit may be implemented in a distributed manner, and components described as distributed may likewise be implemented in a combined form. Accordingly, all modifications or variations derived from the meaning, scope, and equivalents of the claims of the present disclosure should be interpreted as being included within the scope of the present disclosure.

Claims

Claim 1 A method for providing an electrocardiogram reading service, performed by a computing device comprising at least one processor, comprising: a step of analyzing a reading statement generated using a first machine learning model pre-trained to diagnose a disease based on an electrocardiogram, and generating first additional information regarding the reading result included in the reading statement; a step of generating second additional information as information related to data used for training or verification of the first machine learning model used to generate the reading statement or the specifications of the first machine learning model; and a step of providing at least one of the first additional information or the second additional information through a user interface; wherein the first machine learning model is pre-trained to receive an electrocardiogram as input and output a score indicating the likelihood of onset or progression of the disease, and a cutoff value corresponding to the reading criteria for the score output from the first machine learning model is determined based on environmental information in which the electrocardiogram data input to the first machine learning model was measured. Claim 2 A method according to claim 1, wherein the step of generating first additional information regarding a reading result included in the reading by analyzing a reading generated using a first machine learning model pre-trained to diagnose a disease based on the electrocardiogram comprises: a step of extracting at least one of a disease name or an electrocardiogram feature included in the reading as a keyword; and a step of generating first additional information including at least one of a prescription, test, or follow-up action related to the extracted keyword using a pre-established keyword database or a second machine learning model which is a pre-trained generative model. Claim 3 A method according to claim 2, wherein the previously established keyword database manages data in a data structure comprising, as components, key data corresponding to a disease name, sub-key data corresponding to at least one of a name regarding a prescription, test, or follow-up measure associated with the disease name, and content data corresponding to the sub-key data. Claim 4 In claim 3, the step of generating first additional information including at least one of a prescription, inspection, or follow-up action related to the extracted keyword using a second machine learning model, which is a pre-established keyword database or a pre-trained generative model, comprises: a step of deriving key data matching the extracted keyword from the pre-established keyword database; and a step of generating the first additional information by extracting sub-key data and content data based on the derived key data. Claim 5 A method according to claim 2, wherein the second machine learning model is pre-trained to generate information regarding prescriptions, tests, or follow-up measures related to keywords included in the training data, based on training data comprising at least one of diagnostic data written or verified by medical personnel or medical electronic record data. Claim 6 delete Claim 7 A method according to claim 1, wherein the second additional information comprises at least one of the cutoff value, the prevalence of the disease analyzed by the first machine learning model, the sensitivity of the first machine learning model, the specificity of the first machine learning model, information regarding the training data of the first machine learning model, or information regarding the validation data of the first machine learning model. Claim 8 In claim 7, the prevalence rate is determined based on environmental information in which electrocardiogram data input into the first machine learning model is measured. Claim 9 In claim 7, the method wherein the sensitivity or specificity is determined based on the determined cutoff value. Claim 10 A method according to claim 1, wherein the step of providing at least one of the first additional information or the second additional information through a user interface comprises the step of visualizing at least one of the first additional information or the second additional information together with the reading statement in accordance with a user request entered through the user interface. Claim 11 A computer program stored on a computer-readable storage medium, wherein the computer program performs operations for providing an electrocardiogram reading service when executed on one or more processors, the operations include: an operation of generating a reading statement by analyzing a reading statement generated using a first machine learning model pre-trained to diagnose a disease based on an electrocardiogram, and generating a first additional information regarding the reading result included in the reading statement; an operation of generating a second additional information as information related to data used for training or verifying the first machine learning model used to generate the reading statement or the specifications of the first machine learning model; and an operation of providing at least one of the first additional information or the second additional information through a user interface; wherein the first machine learning model is pre-trained to receive an electrocardiogram as input and output a score indicating the likelihood of onset or progression of the disease, and a cutoff value corresponding to the reading criteria for the score output from the first machine learning model is determined based on environmental information in which the electrocardiogram data input to the first machine learning model was measured. Claim 12 A computing device for providing an electrocardiogram reading service, comprising: a processor including at least one core; and a memory including program codes executable on the processor; wherein the processor analyzes a reading statement generated using a first machine learning model pre-trained to diagnose a disease based on an electrocardiogram, generates first additional information regarding the reading result included in the reading statement, generates second additional information regarding data used for training or verification of the first machine learning model used to generate the reading statement or information related to the specifications of the first machine learning model, and provides at least one of the first additional information or the second additional information through a user interface; wherein the first machine learning model is pre-trained to receive an electrocardiogram input and output a score indicating the likelihood of disease onset or progression, and a cutoff value corresponding to the reading criteria for the score output from the first machine learning model is determined based on environmental information in which the electrocardiogram data input to the first machine learning model was measured.

Images