Method, program and apparatus for providing electrocardiogram interpretation service

Abstract

According to an embodiment of the present disclosure, there is disclosed a method, a program, and an apparatus for providing an electrocardiogram interpretation service, which are performed by a computing device. The method may include analyzing an interpretation sentence generated using a first machine learning model previously trained to diagnose diseases based on an electrocardiogram, and generating first additional information for the interpretation result included in the interpretation sentence, generating information related to the first machine learning model used to generate the interpretation sentence as second additional information, and providing at least one of the first additional information or the second additional information via a user interface.

Description

[Technical Field]

[0001] The present disclosure relates to data processing technology in the medical field, and more particularly to a method for processing and providing additional information that can supplement the results of electrocardiogram interpretation using artificial intelligence. [Background technology]

[0002] Existing automated ECG interpretation services simply notify the possibility of a certain disease based on the ECG. However, this information alone is not useful to the doctor reviewing the ECG interpretation results. Doctors who use ECG interpretation services (i.e., doctors who request ECG interpretation services) lack specialized knowledge of ECG interpretation, so even if they confirm that an ECG reading suggests a certain disease, they are unsure of what tests or treatments to perform to treat it. Therefore, it is necessary to inform such clients (non-specialists) not only that they have disease A, but also which additional tests to perform and what prescriptions to prescribe.

[0003] In addition, existing automated ECG interpretation services cannot explain the reasons for the interpretation results, making it difficult to externally verify the reliability of the interpretation results. As a result, doctors who use ECG interpretation services may not use them because they cannot trust the interpretation results. Summary of the Invention [Problem to be solved by the invention]

[0004] The present disclosure aims to provide a method that goes beyond diagnosing a disease based on electrocardiogram interpretation results, and also provides additional information that can supplement the diagnostic results, such as additional tests or measures that must be performed, and that can explain why the interpretation results were derived.

[0005] However, the problems to be solved by this disclosure are not limited to those mentioned above, and other problems not mentioned will be clearly understood from the description below. [Means for solving the problem]

[0006] To achieve the above object, one embodiment of the present disclosure discloses a method for providing an electrocardiogram interpretation service, performed by a computing device, which includes: analyzing an interpretation sentence generated using a first machine learning model previously trained to diagnose diseases based on an electrocardiogram, and generating first additional information for the interpretation result included in the interpretation sentence, generating information related to the first machine learning model used to generate the interpretation sentence as second additional information, and providing at least one of the first additional information or the second additional information via a user interface.

[0007] Alternatively, the step of analyzing the interpretation sentence generated using a first machine learning model pre-trained to diagnose a disease based on the electrocardiogram and generating first additional information for the interpretation result included in the interpretation sentence may include the steps of extracting at least one of the disease name or electrocardiogram features included in the interpretation sentence as a keyword, and generating first additional information including at least one of prescription, examination, or follow-up measure related to the extracted keyword using a pre-established keyword database or a second machine learning model which is a pre-trained generative model.

[0008] Alternatively, the already constructed keyword database may manage data in a data structure including, as components, key data corresponding to a disease name, subkey data corresponding to at least one of the names of prescriptions, tests, or follow-up measures related to the disease name, and content data corresponding to the subkey data.

[0009] Alternatively, the step of generating first additional information including at least one of prescription, examination, or follow-up action related to the extracted keywords using a pre-established keyword database or a second machine learning model that is a pre-trained generative model may include the steps of: deriving key data matching the extracted keywords from the pre-established keyword database; and extracting sub-key data and content data based on the derived key data to generate the first additional information.

[0010] Alternatively, the second machine learning model may be pre-trained based on training data including at least one of interpreted text data created or accepted by medical staff or electronic medical record data to generate information about prescriptions, tests, or follow-up measures related to keywords included in the training data.

[0011] Alternatively, the first machine learning model may be pre-trained to receive an electrocardiogram as input and output a score indicative of the likelihood of disease onset or progression.

[0012] Alternatively, the second additional information may include at least one of a cutoff value corresponding to an interpretation standard for the score output from the first machine learning model, a prevalence of the disease analyzed by the first machine learning model, a sensitivity of the first machine learning model, a specificity of the first machine learning model, information about the training data of the first machine learning model, or information about validation data of the first machine learning model.

[0013] Alternatively, the cut-off value or the prevalence rate may be determined based on environmental information measured from electrocardiogram data input into the first machine learning model.

[0014] Alternatively, the sensitivity or the specificity may be determined based on the determined cut-off value.

[0015] Alternatively, the step of providing at least one of the first additional information or the second additional information via a user interface may include the step of visualizing at least one of the first additional information or the second additional information together with the interpreted text in response to a user request input via the user interface.

[0016] According to one embodiment of the present disclosure, there is provided a computer program stored on a computer-readable storage medium. The computer program, when executed by one or more processors, performs operations for providing an electrocardiogram interpretation service. The operations may include: analyzing an interpretation sentence generated using a first machine learning model previously trained to diagnose diseases based on an electrocardiogram, and generating first additional information for the interpretation result included in the interpretation sentence; generating information related to the first machine learning model used to generate the interpretation sentence as second additional information; and providing at least one of the first additional information or the second additional information via a user interface.

[0017] According to one embodiment of the present disclosure, there is disclosed a computing device for providing an electrocardiogram interpretation service. The device may include a processor including at least one core and a memory including program code executable by the processor. The processor may analyze an interpretation sentence generated using a first machine learning model previously trained to diagnose a disease based on an electrocardiogram, generate first additional information for the interpretation result included in the interpretation sentence, generate information related to the first machine learning model used to generate the interpretation sentence as second additional information, and provide at least one of the first additional information or the second additional information via a user interface. [Effects of the Invention]

[0018] The present disclosure is not limited to diagnosing diseases based on electrocardiogram interpretation results, but can also supplement the diagnostic results with additional information such as additional tests and measures that must be performed, and can explain why the interpretation results were derived, thereby increasing the reliability of the interpretation results and helping customers to accurately understand the interpretation results. [Brief explanation of the drawings]

[0019] [Figure 1] FIG. 1 is a block diagram of a computing device according to one embodiment of the present disclosure. [Figure 2] 1 is a block diagram illustrating a process for providing an electrocardiogram interpretation service according to one embodiment of the present disclosure. [Figure 3] 1 is a flowchart illustrating a method for providing electrocardiogram interpretation services according to one embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0020] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily implement the present disclosure. The embodiments presented in this disclosure are provided to enable those skilled in the art to use or practice the contents of the present disclosure. Therefore, various modifications to the embodiments of the present disclosure will be apparent to those skilled in the art. That is, the present disclosure may be embodied in various different forms and is not limited to the following embodiments.

[0021] Throughout the specification of the present disclosure, the same or similar reference numerals refer to the same or similar components. In addition, in order to clearly explain the present disclosure, reference numerals of parts that are not relevant to the explanation of the present disclosure may be omitted from the drawings.

[0022] The term "or" as used in this disclosure is intended to mean an inclusive "or" rather than an exclusive "or." That is, unless otherwise specified or otherwise clear from the context in this disclosure, "X uses A or B" should be understood to mean one of the natural inclusive permutations. For example, unless otherwise specified or otherwise clear from the context in this disclosure, "X uses A or B" can be interpreted as either X uses A, X uses B, or X uses both A and B.

[0023] The term "and / or" as used in this disclosure must be understood to indicate and include all possible combinations of one or more of the associated listed concepts.

[0024] The terms "comprises" and / or "comprising" as used in this disclosure should be understood to mean that the specified features and / or components are present. However, the terms "comprises" and / or "comprising" should not be understood to exclude the presence or addition of one or more other features, other components and / or combinations thereof.

[0025] In this disclosure, unless otherwise specified or clear from the context as referring to the singular form, the singular should generally be construed as including "one or more."

[0026] The term "nth (n is a natural number)" used in this disclosure can be understood as an expression used to distinguish components of the present disclosure from one another based on a predetermined criterion, such as functional, structural, or convenience of description. For example, in this disclosure, components that perform different functional roles can be classified as a first component or a second component. However, components that are substantially identical within the technical concept of the present disclosure but must be distinguished for convenience of description can also be classified as a first component or a second component.

[0027] The term "acquire" as used in this disclosure may be understood to mean not only receiving data from an external device or system via a wired or wireless communication network, but also generating data in an on-device form.

[0028] Meanwhile, the terms "module" or "unit" used in this disclosure may be understood to refer to an independent functional unit that processes computing resources, such as a computer-related entity, firmware, software or a portion thereof, hardware or a portion thereof, or a combination of software and hardware. Here, a "module" or "unit" may refer to a unit composed of a single element or a unit expressed as a combination or collection of multiple elements. For example, as a concept of connotation, a "module" or "unit" may refer to a hardware element or a collection of hardware elements of a computing device, an application program that performs a specific software function, a processing procedure implemented by executing software, or a collection of instructions for executing a program. Furthermore, as a broad concept, a "module" or "unit" may refer to a computing device itself that constitutes a system, or an application executed on a computing device. However, the above concepts are merely examples, and the concepts of "module" and "unit" may be defined in various ways within the scope of understanding of those skilled in the art based on the contents of this disclosure.

[0029] The term "model" as used in this disclosure may be understood as a system implemented using mathematical concepts and language to solve a specific problem, a collection of software units for solving a specific problem, or an abstract model of a processing process for solving a specific problem. For example, a neural network "model" may refer to a system implemented as a neural network that has problem-solving capabilities through learning. Here, a neural network may have problem-solving capabilities by optimizing parameters connecting nodes or neurons through learning. A neural network "model" may include a single neural network or a neural network ensemble in which multiple neural networks are combined.

[0030] The explanations of the above terms are intended to aid in understanding the present disclosure. Therefore, unless the above terms are explicitly stated as matters that limit the contents of the present disclosure, care should be taken not to use them in a way that limits the technical ideas of the contents of the present disclosure.

[0031] FIG. 1 is a block diagram of a computing device according to one embodiment of the present disclosure.

[0032] The computing device 100 according to an embodiment of the present disclosure may be a hardware device or part of a hardware device that performs comprehensive data processing and calculations, or may be a software-based computing environment connected via a communication network. For example, the computing device 100 may be a server that performs intensive data processing functions and shares resources, or a client that shares resources by interacting with the server. The computing device 100 may also be a cloud system in which multiple servers and clients interact with each other to comprehensively process data. The above description is merely an example of a type of computing device 100, and various types of computing device 100 may be configured within the scope of what one skilled in the art would understand based on the contents of this disclosure.

[0033] 1, a computing device 100 according to an embodiment of the present disclosure may include a processor 110, a memory 120, and a network unit 130. However, since FIG. 1 is merely an example, the computing device 100 may include other components for implementing a computer environment. Also, the computing device 100 may include only some of the disclosed components.

[0034] The processor 110 according to an embodiment of the present disclosure may be understood as a component including hardware and / or software for performing computing operations. For example, the processor 110 may read a computer program to perform data processing. The processor 110 may process operations such as input data processing for machine learning, feature extraction for machine learning, and error calculation based on backpropagation. The processor 110 for performing such data processing may include a central processing unit (CPU), a general-purpose graphics processing unit (GPGPU), a tensor processing unit (TPU), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), etc. The types of processor 110 described above are merely examples, and various types of processor 110 may be configured within the scope of what one skilled in the art would understand based on the present disclosure.

[0035] The processor 110 may analyze the interpretation text generated by the artificial intelligence and generate first additional information for the interpretation result included in the interpretation text. The processor 110 may analyze the text included in the interpretation text and derive information matching the analyzed text using a pre-established database or machine learning model. Here, the first additional information may be understood as information that can be useful for diagnosing a disease, such as a prescription, examination, or follow-up measure related to the disease derived as the interpretation result. For example, if the interpretation text generated by the artificial intelligence includes content that simply indicates the possibility of a certain disease, such as "atrial fibrillation is suspected," the processor 110 may analyze the text of the interpretation text and extract the keyword "atrial fibrillation." Then, the processor 110 may extract information regarding additional examinations or treatments related to "atrial fibrillation" using a pre-established database or machine learning model. The processor 110 may organize the extracted information as text and generate first additional information in the form of, for example, "If this is the first finding, a consultation with a cardiologist is required. An additional 24-hour electrocardiogram is required, and antithrombotic medication is required. Aspirin is prescribed, and a cardiologist recommends referring the patient to a certain hospital." Here, the database from which the processor 110 extracts information may be a collection of data in which supplementary information such as diseases and prescriptions, examinations, or follow-up measures related to the diseases are integrated and managed. The machine learning model may be a model pre-trained to generate supplementary information such as prescriptions, examinations, or follow-up measures related to keywords based on keywords derived by analyzing the text of the interpreted sentence.

[0036] As described above, the first additional information generated by the processor 110 allows a customer who has requested an electrocardiogram reading to smoothly diagnose a disease or take additional measures without making additional medical judgments or gathering information. In particular, since non-specialist physicians who use electrocardiogram services often do not know what additional tests or treatments to perform when they analyze an electrocardiogram and suspect a disease, the first additional information can be substantially helpful in diagnosing a disease and taking immediate action.

[0037] The processor 110 may generate second additional information related to the artificial intelligence model used to analyze the deciphered text to provide not only the interpretation result but also the reason why the interpretation result was derived via the service. Once the deciphered text is generated, the processor 110 may ascertain the artificial intelligence model used to analyze the deciphered text. The processor 110 may then analyze the type of data used to develop the AI ​​model, indicators related to the output of the AI ​​model, and a processing method for the output of the AI ​​model to generate second additional information corresponding to the basis on which the interpretation result was derived. Here, the analysis of the AI ​​model performed to generate the second additional information may be performed based on a database that manages information on data used when developing or updating the AI ​​model, information related to data input into the AI ​​model for interpretation, information related to the output of the AI ​​model, etc. For example, the processor 110 may perform an analysis of the specifications of the machine learning model used to generate the deciphered text, the data itself or information related to the data used for interpretation or derived as an interpretation result. Specifically, the processor 110 may generate as the second additional information at least one of statistical information such as the data used to develop the machine learning model used in the interpretation, the model's output accuracy, information about the hospitals that verified the model's output accuracy, the model's sensitivity, the model's specificity, or the prevalence of the disease analyzed by the model.

[0038] As described above, the second additional information generated by the processor 110 allows a customer who has requested an electrocardiogram interpretation to obtain not only the interpretation result but also information on the basis on which the interpretation result was derived, such as the data from which the model used for the interpretation was created, the research results by which it was verified, and the prevalence rate of the disease derived as the interpretation result. Furthermore, the second additional information can include not only the probability value that the model will analyze the data and determine that there is a disease, but also the degree of confidence in that determination (certainty, uncertainty), thereby increasing the reliability of the interpretation result and helping the customer accurately understand the meaning of the interpretation result.

[0039] The memory 120 according to an embodiment of the present disclosure may be understood as a component including hardware and / or software for storing and managing data processed by the computing device 100. That is, the memory 120 may store any type of data generated or determined by the processor 110 and any type of data received by the network unit 130. For example, the memory 120 may include at least one type of storage medium selected from the group consisting of flash memory, hard disk, multimedia card micro, card-type memory, random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, and optical disk. The memory 120 may also include a database system that manages data in a predetermined manner. The types of memory 120 described above are merely examples, and various configurations of the memory 120 are possible within the scope of what would be understood by one skilled in the art based on the present disclosure.

[0040] The memory 120 may structure and organize and manage data, a combination of data, and program code executable by the processor 110, required for the processor 110 to perform calculations. For example, the memory 120 may store medical data received via the network unit 130 (described below). The memory 120 may store program code storing rules for processing medical data, program code for operating a neural network model to receive medical data and perform learning, program code for operating the neural network model to receive medical data and perform inference according to the intended use of the computing device 100, and processed data generated by executing the program code.

[0041] The network unit 130 according to an embodiment of the present disclosure may be understood as a component that transmits and receives data via any type of known wired or wireless communication system. For example, the network unit 130 may transmit and receive data using a wired or wireless communication system such as a local area network (LAN), wideband code division multiple access (WCDMA), long term evolution (LTE), wireless broadband internet (WiBro), 5th generation mobile communication (5G), ultra wide-band, ZigBee, radio frequency (RF) communication, wireless LAN, wireless fidelity (WiFi), near field communication (NFC), or Bluetooth. The above-described communication systems are merely examples, and various wired or wireless communication systems for transmitting and receiving data by the network unit 130 may be applied in addition to the above examples.

[0042] The network unit 130 may receive data necessary for the processor 110 to perform calculations via wired or wireless communication with any system or any client. The network unit 130 may also transmit data generated by calculations by the processor 110 via wired or wireless communication with any system or any client. For example, the network unit 130 may receive medical data by communicating with a database in a hospital environment, a cloud server that performs tasks such as medical data standardization, a client such as a smart watch, or a medical computing device. The network unit 130 may transmit output data of the neural network model, and intermediate data and processed data derived during the calculation process of the processor 110, via communication with the database, server, client, or computing device.

[0043] FIG. 2 is a block diagram illustrating a process for providing an electrocardiogram interpretation service according to one embodiment of the present disclosure.

[0044] A computing device 100 according to an embodiment of the present disclosure can input electrocardiogram data 10 into a first machine learning model 200 to calculate a disease score 20 indicating the likelihood of disease onset or progression. Here, the disease score 20 can be expressed as a numerical value ranging from 0 to 100. The closer the disease score 20 is to 0, the lower the likelihood of disease onset and the healthier the patient is, whereas the closer the disease score is to 100, the higher the likelihood of disease onset and the more severe the disease is, indicating a dangerous state.

[0045] Meanwhile, the first machine learning model 200 may be a model trained based on training data including electrocardiogram data and labels indicating the likelihood of disease onset or progression. For example, the first machine learning model 200 may extract electrocardiogram features from the electrocardiogram data included in the training data. Here, the electrocardiogram features may be understood as features identified from the waveform of an electrocardiogram signal, such as a P wave, a QRS complex, or a T wave. The first machine learning model 200 may calculate a disease score based on the extracted electrocardiogram features. The first machine learning model 200 may then compare the calculated disease score with the label and update neural network parameters. Here, the comparison between the disease score and the label may be performed based on a loss function. Once the error between the disease score and the label is calculated using the loss function, the first machine learning model 200 may repeatedly update the neural network parameters until the error meets a minimum criterion. In addition to supervised learning, such as the examples above, the first machine learning model 200 can also perform learning via self-supervised learning, including using a neural network that includes an attention layer.

[0046] The computing device 100 can determine whether a disease has occurred based on the disease score 20. The computing device 100 can compare the disease score 20 with a cutoff value to determine whether the subject whose electrocardiogram data 10 was measured has developed a disease. For example, if the disease score 20 is equal to or greater than the cutoff value, the computing device 100 can determine that the subject whose electrocardiogram data 10 was measured has developed a disease. If the disease score 20 is less than the cutoff value, the computing device 100 can determine that the subject whose electrocardiogram data 10 was measured has not developed a disease. Here, the cutoff value can be understood as a value that dynamically changes depending on the environment in which the electrocardiogram data 10 was measured. If the measurement environment is a health checkup center, which is a medical environment with a low level of emergency response, the cutoff value can be high. If the measurement environment is a critical care unit, which is a medical environment with a high level of emergency response, the cutoff value can be low.

[0047] The computing device 100 may generate an interpretation sentence by arranging the interpretation results in a text format through the interpretation sentence generator 300 implemented by the processor 110. For example, the interpretation sentence may be in the form of a list of interpretation information as text, such as "CONSIDER RIGHT VENTRICULAR HYPERTROPHY ... large R or R' V1 / V2" or "SUPRAVENTRICULAR TACHYCARDIA, RATE 231 ... V-rate>(220-age) or 150." Here, the interpretation information listed in the interpretation sentence may include the name of a disease, electrocardiogram characteristics of the disease, etc.

[0048] The computing device 100 may extract keywords from the interpretation result in the form of text included in the interpretation sentence via the additional information generating unit 400 implemented by the processor 110. The computing device 100 may generate first additional information 30 including at least one of prescription, examination, or follow-up measures for a disease related to the keywords extracted by the additional information generating unit 400. Here, the first additional information 30 may be generated by a pre-established keyword database or a second machine learning model, which is a pre-trained generative model.

[0049] For example, the additional information generating unit 400 may extract at least one of a disease name or electrocardiogram characteristics as a keyword from the text-type interpretation information included in the interpretation sentence. The additional information generating unit 400 may generate the first additional information 30 by searching a keyword database storing information on at least one of prescriptions, examinations, or follow-up measures for the disease that matches the extracted keyword. Here, the keyword database may manage data in a data structure including, as components, key data corresponding to the disease name, subkey data corresponding to at least one of names of prescriptions, examinations, or follow-up measures related to the disease name, and content data corresponding to the subkey data. The additional information generating unit 400 may search for key data matching a keyword corresponding to at least one of the disease name or electrocardiogram characteristics in a keyword database that structures data in the form of [key data-subkey data-content data]. Then, the additional information generating unit 400 may generate the first additional information 30 by extracting subkey data and content data related to the key data from the keyword database.

[0050] The additional information generator 400 may also generate the first additional information 30 using a second machine learning model that generates information about at least one of a prescription, examination, or follow-up measure related to a keyword extracted from the deciphered text. The additional information generator 400 may input at least one of a disease name or electrocardiogram feature in text form extracted from the deciphered text to the second machine learning model, and generate information about at least one of a prescription, examination, or follow-up measure related to the keyword as text data. Here, the data form input to the second machine learning model may be the keyword itself extracted from the deciphered text, or a sentence including the keyword. Assuming that the disease name extracted from the interpreted text is atrial fibrillation, the additional information generation unit 400 can input the word "atrial fibrillation" or the sentence "atrial fibrillation is suspected" into the second machine learning model and generate, as the first additional information 30, information on at least one of prescriptions, tests, or follow-up measures related to atrial fibrillation, such as "If atrial fibrillation is the first finding, consultation with a cardiologist is required. An additional 24-hour electrocardiogram is required, and antithrombotic medication is required. Aspirin is prescribed, and a cardiologist recommends referral to a certain hospital."

[0051] Meanwhile, the second machine learning model may be a model pre-trained to generate information about prescriptions, examinations, or follow-up measures related to keywords included in training data, based on training data including at least one of interpreted text data created or accepted by medical staff or electronic medical record data. For example, the second machine learning model may include a neural network capable of processing sequential data, such as a recurrent neural network (RNN), a long short-term memory (LSTM) neural network, or a transformer. The second machine learning model may be trained by supervised learning using labels included in the training data, or by self-supervised learning.

[0052] The computing device 100 may generate, via the additional information generator 400 embodied by the processor 110, information related to the first machine-learning model 200 used to generate the interpretation sentence as second additional information 40. Here, the second additional information 40 may include at least one of a cutoff value corresponding to an interpretation standard for the score output from the first machine-learning model 200, a prevalence of a disease analyzed by the first machine-learning model 200, a sensitivity of the first machine-learning model 200, a specificity of the first machine-learning model 200, training data of the first machine-learning model 200, or validation data of the first machine-learning model 200. The cutoff value or the prevalence may be determined based on environmental information measured from the electrocardiogram data 10 input to the first machine-learning model 200. The sensitivity or the specificity may then be determined based on the determined cutoff value.

[0053] For example, when acquiring the electrocardiogram data 10, the computing device 100 may acquire environmental information about the environment in which the electrocardiogram data 10 was measured. The environmental information may include information about where the electrocardiogram data 10 was measured, such as a health screening center, a general ward, or an intensive care unit. The computing device 100 may use the environmental information to determine a cutoff value used to determine the presence or absence of a disease based on the disease score 20. If the electrocardiogram data 10 is measured in a routine medical environment, such as at home or a health screening center, the computing device 100 may determine a relatively high cutoff value. If the electrocardiogram data 10 is measured in an unroutine medical environment, such as an intensive care unit, the computing device 100 may determine a relatively low cutoff value. In this way, the computing device 100 may dynamically determine the cutoff value in accordance with the environmental information. The computing device 100 may search a database that stores information about the relationship between environments and prevalence rates and determine the prevalence rate of the environment identified from the environmental information. The computing device 100 may calculate the sensitivity and specificity of the first machine learning model 200 using the cutoff value determined based on the environmental information. The computing device 100 may then search a database storing information about data used in the development or update of the first machine learning model 200 to collect information about data used in training or validation of the first machine learning model 200. The computing device 100 may generate second additional information 40 by combining the thus-organized cutoff value, prevalence, sensitivity, specificity, and information about the data used in training or validation.

[0054] The computing device 100 may generate a user interface for providing the interpretation text. The computing device 100 may additionally provide at least one of first additional information 30 and second additional information 40 through the user interface for providing the interpretation text. For example, once interpretation of the electrocardiogram data 10 is completed, the computing device 100 may provide the interpretation text to the terminal of the customer who requested the electrocardiogram interpretation through the user interface and visualize it preferentially. Here, if a user request for viewing additional information is input through the user interface, the computing device 100 may provide at least one of the first additional information 30 and second additional information 40 to the terminal of the customer through the user interface and visualize it together with the interpretation text. Here, at least one of the first additional information 30 and second additional information 40 may be visualized by overlaying it on the interpretation text, or may be visualized as a separate area from the interpretation text. The computing device 100 may also provide at least one of the interpretation text, the first additional information 30, or the second additional information 40 to the terminal of the customer who requested the electrocardiogram interpretation via the user interface and visualize them simultaneously.

[0055] FIG. 3 is a flowchart illustrating a method for providing an electrocardiogram interpretation service according to one embodiment of the present disclosure.

[0056] Referring to FIG. 3, a computing device 100 according to an embodiment of the present disclosure may analyze a generated interpretation sentence using a first machine learning model previously trained to diagnose a disease based on an electrocardiogram (ECG) and generate first additional information about the interpretation result included in the interpretation sentence (S100). The computing device 100 may extract at least one of a disease name or an ECG feature included in the interpretation sentence as a keyword. The computing device 100 may generate the first additional information including at least one of a prescription, examination, or follow-up measure associated with the extracted keyword using a pre-established keyword database or a pre-trained second machine learning model, which is a generative model. Specifically, the computing device 100 may derive key data matching the extracted keyword from a pre-established keyword database. The computing device 100 may generate the first additional information by extracting subkey data and content data based on the derived key data. Here, the extraction of the first additional information may be performed in response to a user request via a user interface. When a user requests to view the first additional information through a user interface, the computing device 100 may generate the first additional information by checking the key data, subkey data, and content data. However, a user request for extracting the first additional information is not necessarily required. Therefore, the computing device 100 may generate the first additional information immediately once the interpretation text is generated, even without a user request. The computing device 100 may input keywords into a second machine learning model to generate the first additional information in text form, including at least one of a prescription, examination, or follow-up action associated with the keywords. Here, the second machine learning model may be a model pre-trained to generate information about a prescription, examination, or follow-up action associated with keywords included in training data, based on training data including at least one of interpretation text data created or accepted by medical staff or electronic medical record data.

[0057] The computing device 100 may generate, as second additional information, information related to the first machine learning model used to generate the interpretation sentence (S200). Because electrocardiogram interpretation is performed using an artificial intelligence model such as the first machine learning model, determining on what basis and how the interpretation based on the output of the first machine learning model was made plays an important role in determining the reliability of the interpretation. Therefore, the computing device 100 may generate, as second additional information, information regarding what data was used to develop and update the first machine learning model, how the performance of the first machine learning model is evaluated using various indicators, etc. For example, the second additional information may include at least one of a cutoff value corresponding to the interpretation standard for the score output from the first machine learning model, the prevalence of the disease analyzed by the first machine learning model, the sensitivity of the first machine learning model, the specificity of the first machine learning model, information about the training data of the first machine learning model, or information about the validation data of the first machine learning model. Here, the cutoff value or prevalence can be determined based on environmental information measured from electrocardiogram data input into the first machine learning model, and the sensitivity and specificity can be determined by mathematical calculation based on the determined cutoff value.

[0058] The computing device 100 may provide at least one of the first additional information or the second additional information through a user interface (S300). The computing device 100 may visualize at least one of the first additional information or the second additional information together with the interpretation text and provide it to the terminal of the customer who requested the electrocardiogram interpretation through the user interface. Here, at least one of the first additional information or the second additional information may be visualized together with the interpretation text and displayed on the terminal display, or may be selectively visualized and displayed on the terminal display at the user's request.

[0059] For example, the computing device 100 may visualize the deciphered sentence in text form by including the first additional information and the second additional information together as follows:

[0060] "Based on the electrocardiogram reading requested by Mr. / Ms. XX, the probability of myocardial infarction is 3.7 points. The prevalence of myocardial infarction at the health screening center is 0.5%, and our recommended cutoff point is 40 points. When a cutoff point of 40 points is applied, the AI ​​has an accuracy of 80%, a sensitivity of 70%, and a specificity of 65%. This AI model was trained based on data measured at the health screening center on XX adults from 2018 to 2022, and verified based on electrocardiograms measured at Hospital A from 2013 to 2015. The accuracy was XX%."

[0061] In addition, the computing device 100 may visualize the interpretation sentence in a text form alone without the first additional information and the second additional information. Here, when a customer performs a mouse-over operation of placing a mouse cursor on a keyword, such as a disease name or an electrocardiogram characteristic, in the text included in the interpretation sentence, at least one of the first additional information or the second additional information related to the selected keyword may be visualized in a text form by an operation such as a pop-up.

[0062] The various embodiments of the present disclosure described above can be combined with additional embodiments and can be modified within the scope that can be understood by those skilled in the art from the above detailed description. The embodiments of the present disclosure are illustrative in all respects and should not be construed as limiting. For example, each component described as a single type can also be implemented in a distributed form, and similarly, components described as distributed can also be implemented in a combined form. Therefore, all modifications and variations derived from the meaning, scope, and equivalent concepts of the claims of the present disclosure should be construed as being within the scope of the present disclosure.

Claims

1. 1. A method of providing electrocardiogram interpretation services performed by a computing device including at least one processor, comprising: Analyzing the generated interpretation sentence using a first machine learning model previously trained to diagnose a disease based on an electrocardiogram, and generating first additional information for the interpretation result included in the interpretation sentence; generating second additional information related to a first machine learning model used to generate the deciphered sentence; providing at least one of the first additional information or the second additional information through a user interface; A method comprising:

2. The step of analyzing the generated interpretation sentence using a first machine learning model previously trained to diagnose a disease based on the electrocardiogram and generating first additional information for the interpretation result included in the interpretation sentence includes: extracting at least one of a disease name or an electrocardiogram feature included in the interpretation text as a keyword; generating first additional information including at least one of a prescription, an examination, or a follow-up action related to the extracted keywords using a second machine learning model, which is a pre-constructed keyword database or a pre-trained generative model; The method of claim 1 , comprising:

3. The method according to claim 2, wherein the already constructed keyword database manages data in a data structure including, as components, key data corresponding to a disease name, subkey data corresponding to at least one of the names of prescriptions, tests, or follow-up measures related to the disease name, and content data corresponding to the subkey data.

4. generating first additional information including at least one of a prescription, an examination, or a follow-up measure related to the extracted keyword using a second machine learning model that is a pre-constructed keyword database or a pre-trained generative model; deriving key data matching the extracted keywords from the pre-constructed keyword database; extracting subkey data and content data based on the derived key data to generate the first additional information; The method of claim 3, comprising:

5. 3. The method of claim 2, wherein the second machine learning model is pre-trained to generate information about prescriptions, tests, or follow-up measures related to keywords included in training data, based on training data including at least one of interpreted text data created or accepted by medical staff or electronic medical record data.

6. The method of claim 1 , wherein the first machine learning model is pre-trained to receive an electrocardiogram as input and output a score indicating the likelihood of disease onset or progression.

7. 7. The method of claim 6, wherein the second additional information includes at least one of a cutoff value corresponding to an interpretation standard for the score output from the first machine learning model, a prevalence of the disease analyzed by the first machine learning model, a sensitivity of the first machine learning model, a specificity of the first machine learning model, information about training data of the first machine learning model, or information about validation data of the first machine learning model.

8. The method of claim 7 , wherein the cutoff value or the prevalence is determined based on environmental information measured from electrocardiogram data input into the first machine learning model.

9. The method of claim 7 , wherein the sensitivity or specificity is determined based on the determined cutoff value.

10. The step of providing at least one of the first additional information or the second additional information through a user interface includes: The method of claim 1 , further comprising visualizing at least one of the first additional information or the second additional information together with the interpreted text in response to a user request input via the user interface.

11. A computer program stored on a computer-readable storage medium, the computer program performing operations for providing an electrocardiogram interpretation service when executed by one or more processors; The operation is Analyzing the generated interpretation sentence using a first machine learning model previously trained to diagnose a disease based on an electrocardiogram, and generating first additional information for the interpretation result included in the interpretation sentence; generating second additional information related to a first machine learning model used to generate the deciphered sentence; providing at least one of the first additional information or the second additional information through a user interface; a computer program comprising:

12. 1. A computing device for providing electrocardiogram interpretation services, comprising: a processor including at least one core; a memory containing program code executable by the processor; Including, The processor: Analyzing the generated interpretation sentence using a first machine learning model previously trained to diagnose a disease based on an electrocardiogram, and generating first additional information for the interpretation result included in the interpretation sentence; generating second additional information related to the first machine learning model used to generate the deciphered sentence; The apparatus provides at least one of the first additional information or the second additional information via a user interface.

Images