Method and apparatus for providing user interface for labeling medical artificial intelligence

Abstract

Disclosed are a method, a program, and a device for providing a user interface for labeling medical artificial intelligence according to an embodiment of the present invention. The method includes steps of: providing a first user interface for listing labelable ECG features on an ECG signal; and providing a second user interface for visually displaying the ECG signal.

Description

Method and device for providing a user interface for labeling medical artificial intelligence

[0001] The present disclosure relates to user interface technology, and more particularly, to a user interface that provides a labeling function for learning artificial intelligence used in analyzing medical data.

[0002]

[0003] Electrocardiogram signals can be broadly categorized into P waves, QRS waves, and T waves. When assessing health status or disease using an electrocardiogram, the electrocardiogram signal is not analyzed as a whole, but is segmented into P waves, QRS waves, and T waves and then analyzed. For example, the PR interval, which refers to the period from the beginning of the P wave to the beginning of the QRS complex, is normally considered to be no longer than 0.2 seconds, the QRS complex is considered to be within 0.1 seconds, and the QT interval, which refers to the period from the beginning of the Q wave to the end of the T wave, is considered to be around 0.4 seconds. The signal waveforms and intervals that influence the assessment of health status or disease are generally referred to as electrocardiogram features.

[0004] Accurately distinguishing P, QRS, and T waves in an ECG signal is a critical factor in improving analysis accuracy. When analyzing ECG signals using AI, inputting accurate P, QRS, and T wave information can further improve accuracy. In other words, if AI can accurately identify ECG characteristics during signal analysis, it can make highly accurate assessments of health status and the presence of disease.

[0005] However, for AI to accurately identify ECG features on its own, it needs accurate information on how these features are distinguished from the ECG signal. These answers are provided through labeling by experts in the field of ECG interpretation. For example, experts can identify the onset, peak, and offset of ECG features (P wave, QRS complex, and T wave) in the ECG signal and provide AI with a recognizable answer. Because this task is time-consuming and costly, it is crucial to create an environment where it can be performed as efficiently as possible.

[0006]

[0007] The present disclosure aims to provide a method for providing a user interface that enables an expert to efficiently perform labeling of electrocardiogram signals with simple access and operation.

[0008] However, the problems to be solved in this disclosure are not limited to the problems mentioned above, and other problems not mentioned can be clearly understood based on the description below.

[0009]

[0010] According to one embodiment of the present disclosure, a method for providing a user interface for labeling medical artificial intelligence is disclosed to achieve the aforementioned objectives. The method may include the steps of providing a first user interface that lists labelable electrocardiogram features on an electrocardiogram signal; and the steps of providing a second user interface that visually displays the electrocardiogram signal.

[0011] Alternatively, the first user interface may include a first control graphic that dynamically changes a visual representation and state in response to a first command for selecting an electrocardiogram feature to be labeled from among the labelable electrocardiogram features on the electrocardiogram signal.

[0012] Alternatively, the second user interface may include a second control graphic displayed on the electrocardiogram signal in response to a second command to label the electrocardiogram feature selected in accordance with the first command on the electrocardiogram signal.

[0013] Alternatively, when a plurality of second control graphics exist on the ECG signal, a second control graphic that exists in a waveform related to an ECG characteristic corresponding to a first control graphic that is activated in response to the first command among the plurality of second control graphics may be activated on the ECG signal.

[0014] Alternatively, the second user interface may include a third control graphic displayed on the electrocardiogram signal based on a label for a predicted electrocardiogram feature by inputting the electrocardiogram signal without the second control graphic displayed or the electrocardiogram signal with the second control graphic displayed into a pre-trained artificial intelligence model.

[0015] Alternatively, the third control graphic may specify the location of the predicted electrocardiogram feature on the electrocardiogram signal to guide the input of the second command, or may express error information of the electrocardiogram feature specified through the second control graphic, calculated based on the predicted label.

[0016] Alternatively, the second user interface may include a fourth control graphic representing location information of a point at which the second command is input on the electrocardiogram signal or a guide line to assist labeling on the electrocardiogram signal.

[0017] Alternatively, the second user interface may dynamically change the visual representation of the electrocardiogram signal in response to a third command selecting either a state in which noise present in the electrocardiogram signal is removed or a state in which the noise is not removed.

[0018] Alternatively, the method may further comprise the step of providing a third user interface that lists electrocardiogram features defined by a user command without labeling the electrocardiogram signal.

[0019] Alternatively, the third user interface may include a fifth control graphic that dynamically changes its visual representation and state in response to a fourth command that determines the presence or absence of electrocardiogram features listed in the third user interface.

[0020] Alternatively, the fifth control graphic may be a graphic for determining the presence or absence of electrocardiogram features listed in the third user interface according to the fourth command.

[0021] Alternatively, the method may further comprise the step of providing a fourth user interface that displays the labeling status performed by the user.

[0022] Alternatively, the fourth user interface may include a sixth control graphic indicating whether at least one electrocardiogram feature per lead is labeled.

[0023] According to one embodiment of the present disclosure for achieving the above-described task, a computer program stored on a computer-readable storage medium is disclosed. When the computer program is executed on one or more processors, the computer program performs operations for providing a user interface for labeling medical artificial intelligence, the operations including: providing a first user interface for listing labelable electrocardiogram features on an electrocardiogram signal; and providing a second user interface for visually displaying the electrocardiogram signal.

[0024] According to one embodiment of the present disclosure for achieving the aforementioned task, a computing device providing a user interface for labeling medical artificial intelligence is disclosed. The device may include a processor including at least one core; a memory including program codes executable by the processor; and an input / output unit providing a first user interface listing labelable electrocardiogram features on an electrocardiogram signal and a second user interface visually displaying the electrocardiogram signal.

[0025]

[0026] The present disclosure provides a computing environment in which labeling work can be performed accurately and efficiently through a user interface that allows an expert to perform labeling of electrocardiogram signals through simple operations according to features required for electrocardiogram interpretation and to intuitively understand the progress of labeling.

[0027]

[0028] FIG. 1 is a block diagram of a computing device according to one embodiment of the present disclosure.

[0029] FIG. 2 and FIG. 3 are conceptual diagrams showing screens implementing a user interface according to one embodiment of the present disclosure.

[0030] FIG. 4 is a flowchart illustrating a method for providing a user interface according to one embodiment of the present disclosure.

[0031] FIG. 5 is a block diagram showing a hardware environment according to one embodiment of the present disclosure.

[0032]

[0033] Below, embodiments of the present disclosure are described in detail with reference to the attached drawings so that those skilled in the art can easily implement the present disclosure. The embodiments presented in this disclosure are provided to enable those skilled in the art to utilize or implement the contents of the present disclosure. Accordingly, various modifications to the embodiments of the present disclosure will be apparent to those skilled in the art. That is, the present disclosure may be implemented in various different forms and is not limited to the embodiments described below.

[0034] Throughout the specification of this disclosure, identical or similar drawing numbers refer to identical or similar components. Furthermore, for the purpose of clearly describing the disclosure, drawing numbers for parts in the drawings that are not relevant to the description of the disclosure may be omitted.

[0035] The term "or" as used herein is intended to mean an inclusive "or" rather than an exclusive "or." That is, unless otherwise specified herein or clear from context, "X employs A or B" should be understood to mean either of the natural inclusive permutations. For example, unless otherwise specified herein or clear from context, "X employs A or B" can be interpreted to mean either X employs A, X employs B, or X employs both A and B.

[0036] The term "and / or" as used herein should be understood to refer to and include all possible combinations of one or more of the related concepts listed.

[0037] The terms "comprises" and / or "comprising" as used herein should be understood to mean the presence of certain features and / or components. However, it should be understood that the terms "comprises" and / or "comprising" do not exclude the presence or addition of one or more other features, other components, and / or combinations thereof.

[0038] Unless otherwise specified in this disclosure or unless the context makes it clear that the singular form is intended to be referred to, the singular should generally be construed to include “one or more.”

[0039] The term "Nth (N is a natural number)" used in the present disclosure can be understood as an expression used to mutually distinguish components of the present disclosure based on a predetermined standard such as a functional perspective, a structural perspective, or convenience of explanation. For example, components performing different functional roles in the present disclosure can be distinguished as a first component or a second component. However, components that are substantially the same within the technical spirit of the present disclosure but must be distinguished for convenience of explanation may also be distinguished as a first component or a second component.

[0040] The term "acquisition" as used in this disclosure may be understood to mean not only receiving data through a wired or wireless communication network with an external device or system, but also generating data in an on-device form.

[0041] Meanwhile, the term "module" or "unit" used in the present disclosure can be understood as a term referring to an independent functional unit that processes computing resources, such as a computer-related entity, firmware, software or a part thereof, hardware or a part thereof, or a combination of software and hardware. At this time, the "module" or "unit" may be a unit composed of a single element, or a unit expressed as a combination or set of multiple elements. For example, as a narrow concept, a "module" or "unit" may refer to a hardware element of a computing device or a set thereof, an application program that performs a specific function of software, a processing process implemented through software execution, or a set of instructions for program execution, etc. In addition, as a broad concept, a "module" or "unit" may refer to the computing device itself that constitutes the system, or an application running on the computing device, etc. However, since the above-described concept is only an example, the concept of “module” or “part” may be defined in various ways within a range understandable to those skilled in the art based on the contents of the present disclosure.

[0042] The term "model" as used herein may be understood as a system implemented using mathematical concepts and language to solve a specific problem, a set of software units to solve a specific problem, or an abstract model of a processing process to solve a specific problem. For example, a neural network "model" may refer to the entire system implemented as a neural network that has problem-solving capabilities through learning. In this case, the neural network can have problem-solving capabilities by optimizing the parameters connecting nodes or neurons through learning. A neural network "model" may include a single neural network or a set of neural networks that are a combination of multiple neural networks.

[0043] The term "control graphic" used in this disclosure can be understood as a graphical object that executes a command and produces a visible result when manipulating a user interface. In other words, a "control graphic" can be understood as a basic unit that constitutes a user interface, a user interface element that displays content to be provided to a user or enables manipulation by the user.

[0044] The explanation of the above terms is intended to aid understanding of the present disclosure. Therefore, unless explicitly stated as limiting the contents of the present disclosure, it should be noted that the above terms are not intended to limit the technical ideas of the contents of the present disclosure.

[0045] FIG. 1 is a block diagram of a computing device according to one embodiment of the present disclosure.

[0046] A computing device (100) according to one embodiment of the present disclosure may be a hardware device or a part of a hardware device that performs comprehensive processing and calculation of data, or may be a software-based computing environment connected to a communication network. For example, the computing device (100) may be a server that performs intensive data processing functions and shares resources, or may be a client that shares resources through interaction with a server. In addition, the computing device (100) may be a cloud system in which multiple servers and clients interact to comprehensively process data. Since the above description is only one example related to the type of computing device (100), the type of computing device (100) may be configured in various ways within a range understandable to those skilled in the art based on the contents of the present disclosure.

[0047] Referring to FIG. 1, a computing device (100) according to one embodiment of the present disclosure may include a processor (110), a memory (120), a network unit (130), and an input / output unit (140). However, FIG. 1 is merely an example, and the computing device (100) may include other components for implementing a computing environment. In addition, only some of the disclosed components may be included in the computing device (100).

[0048] The processor (110) according to one embodiment of the present disclosure may be understood as a configuration unit including hardware and / or software for performing computing operations. For example, the processor (110) may process commands generated as a result of user interaction through a user interface. In addition, the processor (110) may read a computer program to perform data processing for machine learning. The processor (110) may process computational processes such as processing of input data for machine learning, feature extraction for machine learning, and error calculation based on backpropagation. The processor (110) for performing such data processing and operations may include a central processing unit (CPU), a general purpose graphics processing unit (GPGPU), a tensor processing unit (TPU), an application specific integrated circuit (ASIC), or a field programmable gate array (FPGA). The type of processor (110) described above is only one example, and thus the type of processor (110) can be configured in various ways within a range understandable to those skilled in the art based on the contents of the present disclosure.

[0049] The processor (110) can generate a user interface that an electrocardiogram analysis expert can use for labeling. The user interface can include control graphics for outputting information necessary for performing labeling, receiving input based on user operations, or implementing functions necessary for labeling. For example, the user interface can include control graphics that visualize data necessary for labeling, such as electrocardiogram features and electrocardiogram signals. In addition, the user interface can include control graphics for inputting commands by the user for labeling the electrocardiogram signals, and control graphics that prompt the user to input commands. Such control graphics can have visual representations or states that can be changed by user operations.

[0050] The memory (120) according to one embodiment of the present disclosure may be understood as a configuration unit including hardware and / or software for storing and managing data processed in the computing device (100). That is, the memory (120) may store any type of data generated or determined by the processor (110) and any type of data received by the network unit (130). For example, the memory (120) may include at least one type of storage medium among a flash memory type, a hard disk type, a multimedia card micro type, a card type memory, a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, and an optical disk. In addition, the memory (120) may also include a database system that controls and manages data in a predetermined system. The type of memory (120) described above is only one example, and thus the type of memory (120) can be configured in various ways within a range understandable to those skilled in the art based on the contents of the present disclosure.

[0051] The memory (120) can structure and organize and manage data, combinations of data, and program codes executable by the processor (110) required for the processor (110) to perform operations. For example, the memory (120) can store medical data received through the network unit (130) to be described later. The memory (120) can store program codes that store rules for processing medical data, program codes that operate a neural network model to perform learning by receiving medical data as input, program codes that operate a neural network model to perform inference according to the purpose of use of the computing device (100) by receiving medical data as input, and processed data generated as the program codes are executed.

[0052] The network unit (130) according to one embodiment of the present disclosure may be understood as a component that transmits and receives data through any type of known wired or wireless communication system. For example, the network unit (130) may perform data transmission and reception using a wired or wireless communication system such as a local area network (LAN), wideband code division multiple access (WCDMA), long term evolution (LTE), wireless broadband internet (WiBro), fifth generation mobile communication (5G), ultra wide-band, ZigBee, radio frequency (RF) communication, wireless LAN, wireless fidelity, near field communication (NFC), or Bluetooth. Since the above-described communication systems are only examples, the wired and wireless communication system for data transmission and reception of the network unit (130) may be applied in various ways other than the above-described examples.

[0053] The network unit (130) can receive data necessary for the processor (110) to perform calculations through wired or wireless communication with any system or any client, etc. In addition, the network unit (130) can transmit data generated through calculations of the processor (110) through wired or wireless communication with any system or any client, etc. For example, the network unit (130) can receive biometric data through communication with a cloud server that performs tasks such as databases in a hospital environment, standardization of medical data, clients such as smart watches, or medical computing devices, etc. The network unit (130) can transmit output data of a neural network model, intermediate data derived from the calculation process of the processor (110), processed data, etc. through communication with the aforementioned database, server, client, or computing device, etc.

[0054] The input / output unit (140) according to one embodiment of the present disclosure may be understood as a configuration unit including hardware and / or software for implementing a user interface. That is, the input / output unit (140) may visualize and output any type of data generated or determined by the processor (110) and any type of data received by the network unit (130). In addition, the input / output unit (140) may receive a user input that generates a command to be transmitted to any system or any client connected to the processor (110) or the computing device (100) via wired or wireless communication. For example, the input / output unit (140) may include a display module that can output visualized information or implement a touch screen, such as a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED), a flexible display, a 3D display, etc. In addition, the input / output unit (140) may include an input module capable of recognizing user actions such as motion and voice, such as a camera, microphone, keyboard, mouse, etc. The modules described above are merely examples, and thus, the modules included in the input / output unit (140) may be configured in various ways within a range understandable to those skilled in the art based on the contents of the present disclosure, in addition to the examples described above.

[0055] The input / output unit (140) can implement a user interface, output graphics generated by the processor (110), or receive user commands generated by user operation and transmit them to the processor (110). For example, the input / output unit (140) can output graphics for receiving user commands regarding specific functions, graphics for providing information, etc., in a specific area of ​​the user interface. In addition, the input / output unit (140) can receive user commands for graphics output in a specific area of ​​the user interface. In this case, the user command for a specific graphic can be understood as an input signal generated by a user operation for the specific graphic. In addition, the user operation for the specific graphic can mean an action that the user can perform through the input / output unit (140), such as a click, double-click, hover, flick, pinch, or spread for the specific graphic. The input / output unit (140) can receive user commands and transmit them to the processor (110), thereby enabling operations for providing a user interface based on the user's control to be performed.

[0056] FIG. 2 and FIG. 3 are conceptual diagrams showing screens implementing a user interface according to one embodiment of the present disclosure.

[0057] According to one embodiment of the present disclosure, the first user interface (210) may configure an area for listing labelable electrocardiogram features on an electrocardiogram signal. The electrocardiogram features listed in the first user interface (210) may be information necessary for electrocardiogram interpretation, such as electrocardiogram waveforms or intervals. For example, as shown in FIG. 2, the electrocardiogram features in the first user interface (210) may be expressed by dividing them into two layers. Electrocardiogram features such as P waves, QRS waves, T waves, F waves, noise, and pacing spikes may be expressed as an upper layer in the first user interface (210). In addition, detailed features such as starting points, peaks, ending points, Q waves, R waves, and S waves in the upper layer may be expressed as a lower layer in the first user interface (210).

[0058] The first user interface (210) may include a first control graphic (211) that dynamically changes a visual representation and state in response to a first command for selecting an electrocardiogram feature to be labeled among electrocardiogram features that can be labeled on an electrocardiogram signal. The first control graphic (211) may be configured for each electrocardiogram feature listed in the first user interface (210). The first control graphic (211) may be a button-type graphic that can accept a user command for selecting a specific electrocardiogram feature. For example, the first control graphic (211) may be included in a lower layer for an upper layer such as a P wave, a QRS wave, a T wave, an F wave, etc. If there is no first command for selecting a specific feature to be labeled among the electrocardiogram features listed in the first user interface (210), the first control graphic (211) may remain in an inactive state without a color change. When a first command is input, the first control graphic (211) may change color and switch to an activated state. At this time, the activated state may be understood as a state in which a specific feature selected by a user command from among the electrocardiogram features listed in the first user interface (210) may be labeled on the electrocardiogram signal. Meanwhile, the first command for the first control graphic (211) may be input by a user operation, such as an action of clicking the first control graphic (211), an action of double-clicking, or an action of selecting a shortcut key assigned to the first control graphic (211).

[0059] The first user interface (210) can display electrocardiogram features as in the example described above, thereby enabling the user to intuitively understand the electrocardiogram features that are the target of labeling and easily determine the target of labeling among the electrocardiogram features. In addition, the first user interface (210) can provide an environment in which the user can easily select or replace the target of labeling through the first control graphic (211) as in the example described above. This improvement in usability can prevent unnecessary user manipulation, thereby improving the efficiency of computing resources.

[0060] The second user interface (220) can configure an area for visually displaying electrocardiogram signals. When electrocardiogram data is acquired by a user command, the second user interface (220) can upload the electrocardiogram signals included in the electrocardiogram data and express them in a graph form. In the second user interface (220), the electrocardiogram signals are aligned over time, and the position or size at which they are displayed can be changed in response to a user command. At this time, the user command is generated by a user operation performed within the area where the electrocardiogram signal is expressed, and the user operation can include actions such as dragging or scrolling.

[0061] The second user interface (220) may include a second control graphic (221) displayed on the electrocardiogram signal in response to a second command for labeling an electrocardiogram feature selected according to a first command in the first user interface (210) on the electrocardiogram signal. When a specific feature is selected in the first user interface (210), a ready state for labeling the specific feature on the electrocardiogram signal displayed in the second user interface (220) may be activated. At this time, when a second command for selecting a specific point on the electrocardiogram signal displayed in the second user interface (220) is input, the second control graphic (221) may be generated at the point where the second command is input. For example, after the first control graphic (211) is activated by the first command, when a cursor for inputting a user command is positioned on the electrocardiogram signal displayed in the second user interface (220), the shape and color of the cursor may change. When the shape and color of the cursor are changed, the user can label the feature selected by the first command on the electrocardiogram signal through the second command. Specifically, when the cursor, whose shape and color have been changed by the user operation, is positioned at point A on the electrocardiogram signal and the second command is input, the second control graphic (221) can be displayed at point A on the electrocardiogram signal. At this time, the second command can be input by a user operation such as an action of clicking point A, an action of double-clicking point A, or an action of selecting a shortcut key assigned to the first control graphic (211).

[0062] Meanwhile, when a plurality of second control graphics exist on the electrocardiogram signal, a second control graphic that exists in a waveform related to an electrocardiogram feature corresponding to a first control graphic that is activated in response to a first command among the plurality of second control graphics may be activated on the electrocardiogram signal. For example, when second control graphics for the P wave, QRS wave, and T wave exist on the electrocardiogram signal, and the first control graphic for the starting point of the P wave is activated by the first command, only the second control graphic for the P wave may be activated, and the second control graphics for the remaining QRS waves and T waves may be deactivated. In this case, the activation of the second control graphic may be understood as a state that a user can visually recognize on the electrocardiogram signal. Conversely, the deactivation of the second control graphic may be understood as a state that exists on the electrocardiogram signal but cannot be visually recognized by the user. That is, when labeling is performed on an electrocardiogram signal for multiple features, the second user interface (220) can selectively visualize the second control graphic (221) depending on the activation state of the first control graphic (211) so that the user can smoothly check the labeling status for each electrocardiogram feature. This function allows the user to smoothly distinguish and perform labeling on the labels existing on the electrocardiogram signal for each feature, thereby reducing unnecessary operations by the user and improving efficiency of computing resources.

[0063] The second user interface (220) may include a third control graphic displayed on the ECG signal based on a label for an ECG feature predicted by inputting an ECG signal without the second control graphic (221) displayed or an ECG signal with the second control graphic (221) displayed into a pre-trained artificial intelligence model. In order to enable a user to perform more accurate labeling, the second user interface (220) may display a third control graphic for suggesting or recommending a label estimated using artificial intelligence on the ECG signal. Specifically, when an ECG signal without the second control graphic (221) displayed is input into the artificial intelligence model and labels for ECG features are derived from the ECG signal, the third control graphic may be generated on the ECG signal based on the labels, which are predicted values ​​of the artificial intelligence model. In this case, the label for the ECG feature, which is the predicted value of the artificial intelligence model, may be information on the location of the ECG feature to be displayed on the ECG signal. Accordingly, the third control graphic can specify and express the location of the electrocardiogram feature predicted by the artificial intelligence model on the electrocardiogram signal in order to guide the input of the second command. In other words, the third control graphic can display the predicted value of the artificial intelligence model at a specific point on the electrocardiogram signal in a form corresponding to the second control graphic (221). At this time, the second control graphic (221) and the third control graphic can be visually distinguished from each other based on differences in color, brightness, saturation, etc. In addition, the third control graphic can also express error information of the electrocardiogram feature specified through the second control graphic (221) based on the label predicted by the artificial intelligence model.That is, if there is a label predicted by an artificial intelligence model based on a specific electrocardiogram feature and a second control graphic (221) by a second command, the third control graphic can represent error information generated as a result of comparing the position based on the label and the position of the second control graphic (221). In this way, the user can perform labeling more accurately and easily through the third control graphic that visualizes the label predicted by artificial intelligence or expresses the error of the user's label. The improvement of labeling accuracy through the third control graphic can reduce unnecessary work that may occur due to incorrect labeling during the repetitive labeling process, thereby improving the efficiency of computing resources.

[0064] Meanwhile, the artificial intelligence model used to generate the third control graphic may be pre-trained to receive an electrocardiogram signal as input and extract electrocardiogram features present in the electrocardiogram signal. For example, the artificial intelligence model may use training data including electrocardiogram signals with labeled electrocardiogram features for training. The artificial intelligence model may receive the electrocardiogram signal included in the training data as input, extract the electrocardiogram features, and generate output data including information on the type and location of the electrocardiogram features. The artificial intelligence model may perform training by adjusting the parameters that constitute the model through an optimizer based on the error between the output data produced using a loss function and the labels included in the training data. In this case, the artificial intelligence model may be a neural network model based on a convolutional neural network, a transformer, or the like. The types and learning methods of the artificial intelligence model described above are merely examples, and the artificial intelligence model of the present disclosure may be configured in various ways within a range understandable to those skilled in the art based on the contents of the present disclosure.

[0065] The second user interface (220) may include a fourth control graphic that represents location information of a point where a second command is input on the electrocardiogram signal or a guide line to assist labeling on the electrocardiogram signal. If a user performs labeling solely based on the shape of the electrocardiogram signal, the likelihood of errors increases. Therefore, to reduce such errors, the fourth control graphic may provide auxiliary information that the user can utilize when performing labeling. For example, when a cursor for inputting the second command is positioned on the electrocardiogram signal, the fourth control graphic may represent a guide line representing information or a visual reference for the corresponding location on the electrocardiogram signal. The user can perform labeling more accurately and easily through the fourth control graphic that provides auxiliary information to assist labeling in this way. The improvement in labeling accuracy through the fourth control graphic may reduce unnecessary work that may occur due to incorrect labeling during repetitive labeling processes, thereby improving computing resource efficiency.

[0066] The second user interface (220) can dynamically change the visual representation of the electrocardiogram signal in response to a third command that selects either a state in which noise is removed from the electrocardiogram signal or a state in which noise is not removed. Depending on the situation, the user may need to check the raw signal before noise is removed. For example, among the electrocardiogram features, pacing spikes, which are indicated by a machine that artificially generates electrical signals for heartbeats within the heart, may disappear due to noise removal. Therefore, if labeling is performed based on the noise-removed signal, the user may not be able to label the pacing spikes. To address this issue, the second user interface (220) can selectively visualize the signal with noise removed or the signal without noise removed through a third command. In this case, the third command may be input by a user operation, such as clicking an additional control graphic provided in the second user interface or another user interface, or selecting a shortcut key assigned to the second user interface.

[0067] Meanwhile, ECG features may overlap with each other in the ECG signal displayed on the second user interface (220). Therefore, in order to accurately label the point where multiple features overlap, a shortcut key that performs a function of overlapping ECG features may be assigned to the second user interface (220). The user may input a command to the second user interface (220) by selecting a shortcut key that performs the function of overlapping ECG features. Then, the second user interface (220) may display the multiple features by overlapping them in the ECG signal in response to the command. For example, the R wave endpoint and the S wave starting point may overlap and exist in the ECG signal. In this case, when the user selects the shortcut key assigned to the second user interface (220), the second user interface (220) may label the S wave starting point at the same location as the already labeled R wave ending point in response to the user command.

[0068] The third user interface (230) can configure an area that lists electrocardiogram features defined by user commands without labeling the electrocardiogram signal. Among the electrocardiogram features, there are features that can be defined as specific points on the electrocardiogram signal, such as the features listed in the first user interface (210), while there are also features that cannot be defined as specific points on the electrocardiogram signal. For example, features that cannot be defined as specific points on the electrocardiogram signal include notched or slurred R, which is used to identify left bundle branch block (LBBB), and delta waves, which are mainly found in a disease called Wolff-Parkinson-White syndrome (WPW). Since these features are variant forms that appear in specific waveforms, they cannot be defined as specific points on the electrocardiogram signal. Therefore, the third user interface (230) can visualize these features so that they can be separately defined by user commands. In addition, the third user interface (230) can list the electrocardiogram features by categorization by lead so that they can be defined by lead. The electrocardiogram features listed in the third user interface (230) may appear in different cases depending on the type of lead, and may not appear in all beats in a single lead. Therefore, the electrocardiogram features listed in the third user interface can be displayed by distinguishing them by lead so that the presence or absence of each lead can be labeled.

[0069] The third user interface (230) may include a fifth control graphic (231) that dynamically changes a visual representation and state in response to a fourth command that determines the presence or absence of electrocardiogram features listed in the third user interface (230). The fifth control graphic (231) may be a graphic for determining the presence or absence of electrocardiogram features listed in the third user interface (230) in accordance with the fourth command. The fifth control graphic (231) may be configured for each electrocardiogram feature listed in the third user interface (230). The third control graphic (231) may be a button-type graphic that can accept a user command for determining the presence or absence of a specific electrocardiogram feature. For example, the third control graphic (231) may include sub-graphics that enable the presence or absence to be individually determined. That is, the third control graphic (231) may include a sub-graphic defining a state that is not a labeling target (“-”), a sub-graphic defining the presence of an electrocardiogram feature (“O” in FIG. 2), and a sub-graphic indicating the absence of an electrocardiogram feature (“X” in FIG. 2). Each of the sub-graphics may remain in an inactive state without a color change if there is no fourth command. If the fourth command is input for any one of the three sub-graphics, the sub-graphic to which the fourth command is input may have its color changed and may be switched to an active state. Depending on the state defined by the sub-graphic switched to an active state, the presence or absence of an electrocardiogram feature corresponding to the third control graphic (230) may be determined. That is, if the fourth command is input for a sub-graphic defining the presence of a delta wave, the delta wave may be labeled as existing on the electrocardiogram signal.Meanwhile, the fourth command for the third control graphic (231) can be input by a user operation such as an action of clicking the third control graphic (231), an action of double-clicking, or an action of selecting a shortcut key assigned to the third control graphic (231).

[0070] The third user interface (230) can provide an environment in which the user can conveniently label various electrocardiogram features that can be used for electrocardiogram analysis in addition to the basic electrocardiogram features by listing electrocardiogram features that are not displayed in the first user interface (210). In addition, the third user interface (230) can provide an environment in which the user can easily select or replace a labeling target through the third control graphic (231) as in the example described above. This improvement in usability can prevent unnecessary user manipulation and thus improve the efficiency of computing resources.

[0071] The fourth user interface (240) may configure an area that displays the labeling status performed by the user. In other words, the fourth user interface (240) may configure a dashboard area that organizes the labeling status performed up to the current point so that the user can grasp at a glance. In addition, the fourth user interface (240) may include a sixth control graphic (241) that expresses whether at least one electrocardiogram feature has been labeled for each lead. The sixth control graphic (241) may be a graphic that outputs information by reflecting the type and number of labels present on the electrocardiogram signal displayed in the second user interface (220). For example, as shown in FIG. 3, the fourth user interface (240) may output a table in which the electrocardiogram features and the standard 12 leads are organized into rows and columns, respectively. In addition, the sixth control graphic (241) may be displayed for each cell organized into rows and columns. If at least one electrocardiogram feature A is labeled in Lead Ⅰ, the sixth control graphic existing in the cell where Lead Ⅰ and the electrocardiogram feature A intersect can be displayed in a shape that intuitively indicates its presence, such as an "O". If at least one electrocardiogram feature A is not labeled in Lead Ⅰ, the sixth control graphic existing in the cell where Lead Ⅰ and the electrocardiogram feature A intersect can be displayed in a shape that intuitively indicates its absence, such as an "X". That is, the sixth control graphic (241) can dynamically change its visual expression and color to reflect the presence or absence of a label. If the sixth control graphic (241) does not intuitively express the presence or absence in this way and instead shows the number of labels, the problem arises that not only is the number of bits different for each electrocardiogram, but the total number cannot be determined, making it meaningless for the user to determine the labeling status.Therefore, in order to make it easy for the user to intuitively grasp the entire labeling status, the sixth control graphic (241) provides information in a way that expresses the presence or absence of a label.

[0072] FIG. 4 is a flowchart illustrating a method for providing a user interface according to one embodiment of the present disclosure.

[0073] A computing device (100) according to one embodiment of the present disclosure can acquire an electrocardiogram signal according to a user command (S100). For example, when a user command for selecting electrocardiogram data including an electrocardiogram signal to be labeled is input, the computing device (100) can receive the electrocardiogram data to be labeled through communication with a medical database or an electrocardiogram signal measuring device. If an electrocardiogram signal is already stored in the memory (120) of the computing device (100), the computing device (100) can also perform a task of transferring the electrocardiogram data stored in the memory (120) to the processor (110).

[0074] The computing device (100) may provide a first user interface that lists electrocardiogram features that can be labeled on the electrocardiogram signal (S200). The computing device (100) may provide a second user interface that visually displays the electrocardiogram signal (S300). In addition, the computing device (100) may provide a third user interface that lists electrocardiogram features defined by a user command without labeling the electrocardiogram signal (S400). As shown in FIG. 3, the first user interface, the second user interface, and the third user interface may be combined and provided on a single screen configured through the input / output unit (140) of the computing device (100). The labeling process may proceed as follows through the user interfaces provided in this manner.

[0075] Once an ECG signal is acquired, the user can select any one of multiple features displayed through the first user interface. For example, to specify the start, peak, and end points of the P wave, the user can select the "(Q)P" button graphic at the top of the first user interface using the cursor. The "(Q)P" button graphic will then turn blue and be activated. The user can also select the "(Q)P" button graphic by pressing the "Q" key on the keyboard, which is a shortcut key that activates the button.

[0076] After the button graphic of the first user interface is activated, when the user places the cursor over the electrocardiogram signal displayed on the second user interface, the shape of the cursor may change. For example, when the button graphic of the first user interface is not activated, the cursor may be shaped like an arrow. However, when the button graphic of the first user interface is activated and the cursor is positioned over the electrocardiogram signal, the cursor may change from an arrow shape to a red cross shape.

[0077] When the button graphic of the first user interface is activated and the user places the cursor on the ECG signal, additional information, such as the x-axis value or y-axis value at the corresponding position, is displayed next to the cursor in the second user interface. In addition, when the user selects a specific point on the ECG signal using the cursor or performs an action of pressing a shortcut key on the specific point on the ECG signal, the ECG feature corresponding to the activated button graphic of the first user interface can be labeled. For example, to label the onset of the P wave, the user can place the cursor on the corresponding position on the ECG signal and then press the shortcut key "1" assigned to the onset to store the location information of the P wave onset. To label the peak of the P wave, the user can place the cursor on the corresponding position on the ECG signal and then press the shortcut key "2" assigned to the peak to store the location information of the P wave peak. The designated position can be displayed as a circular dot on the ECG signal according to each matching color.

[0078] Meanwhile, since features listed through the third user interface, such as delta waves, are not displayed on the ECG signal, their presence or absence can be labeled through button graphics of the third user interface. For example, a user can label the presence or absence of an ECG feature in the ECG signal by selecting any one of the button graphics represented as "O" (present), "X" (absent), or "-" (no specified value) on the third user interface.

[0079] The computing device (100) may provide a fourth user interface that displays the labeling status performed by the user (S500). The fourth user interface may not be provided simultaneously with the first user interface, the second user interface, and the third user interface, but may be selectively provided according to a specific user command. For example, when the user selects a button graphic displayed in the middle of a screen where the first, second, and third user interfaces are provided using a cursor, the fourth user interface may be displayed on the screen through a dropdown, pop-up, or the like. Through the fourth user interface selectively displayed in this way, the user can grasp the labeling status for each electrocardiogram feature and lead at a glance.

[0080] FIG. 5 is a block diagram showing a hardware environment according to one embodiment of the present disclosure.

[0081] Referring to FIG. 5, a system according to one embodiment of the present disclosure may include a client (300) that provides a user interface to a user, and a server (400) that processes information obtained through the user interface or information provided through the user interface. For example, the client (300) may correspond to the computing device (100) of FIG. 1. The client (300) may control graphics implemented in the user interface using at least one of a command or data obtained through interaction with a user. In addition, the client (300) may transmit at least one of a user command or data obtained through the user interface to the server (400). The server (400) may process information to be provided through the user interface using at least one of the command or data obtained through the user interface. Specifically, the server (400) may generate visualized information to be provided to the user through the user interface using an artificial intelligence model. The server (400) may transmit the processed information to the client (300), and the client (300) may provide the processed information to the user through the user interface.

[0082] The various embodiments of the present disclosure described above can be combined with additional embodiments and modified within the scope understood by those skilled in the art in light of the detailed description above. It should be understood that the embodiments of the present disclosure are illustrative in all respects and not restrictive. For example, each component described as a single component may be implemented in a distributed manner, and likewise, components described as distributed may be implemented in a combined manner. Accordingly, all changes or modifications derived from the meaning, scope, and equivalent concepts of the claims of the present disclosure should be construed as being included within the scope of the present disclosure.

Claims

1. A method for providing a user interface for labeling medical artificial intelligence, Providing a first user interface listing labelable electrocardiogram features on an electrocardiogram signal; and A step of providing a second user interface for visually displaying the electrocardiogram signal; including, method.

2. In paragraph 1, The above first user interface is, A first control graphic that dynamically changes a visual representation and state in response to a first command for selecting an electrocardiogram feature to be labeled among electrocardiogram features that can be labeled on the electrocardiogram signal, method.

3. In paragraph 2, The second user interface is, Including a second control graphic displayed on the electrocardiogram signal in response to a second command for labeling the electrocardiogram feature selected according to the first command on the electrocardiogram signal. method.

4. In paragraph 3, If there are multiple second control graphics on the above ECG signal, Among the plurality of second control graphics, a second control graphic present in a waveform related to an electrocardiogram characteristic corresponding to a first control graphic that is activated in response to the first command is activated on the electrocardiogram signal. method.

5. In paragraph 3, The second user interface is, A third control graphic is included that is displayed on the electrocardiogram signal based on a label for the predicted electrocardiogram feature by inputting the electrocardiogram signal without the second control graphic displayed or the electrocardiogram signal with the second control graphic displayed into a pre-trained artificial intelligence model. method.

6. In paragraph 5, The above third control graphic is, To guide the input of the second command, the location of the predicted electrocardiogram feature is specified and expressed on the electrocardiogram signal, or Expressing error information of a specified electrocardiogram feature through the second control graphic based on the predicted label, method.

7. In paragraph 3, The second user interface is, Including a fourth control graphic representing location information of a point where the second command is input on the electrocardiogram signal or a guide line to assist labeling on the electrocardiogram signal. method.

8. In paragraph 1, The second user interface is, In response to a third command selecting either a state in which noise present in the electrocardiogram signal is removed or a state in which the noise is not removed, dynamically changing the visual representation of the electrocardiogram signal. method.

9. In paragraph 1, Providing a third user interface that lists electrocardiogram features defined by a user command without labeling the electrocardiogram signal; including more, method.

10. In paragraph 9, The third user interface is: In response to a fourth command determining the presence or absence of electrocardiogram features listed in the third user interface, a fifth control graphic dynamically changing its visual representation and state, method.

11. In paragraph 10, The above fifth control graphic is, A graphic for determining the presence or absence of electrocardiogram features listed in the third user interface according to the fourth command, method.

12. In paragraph 1, A step of providing a fourth user interface that displays the labeling status performed by the user; including more, method.

13. In paragraph 12, The above fourth user interface is, Including a sixth control graphic representing whether at least one electrocardiogram feature per lead is labeled; method.

14. A computer program stored on a computer-readable storage medium, wherein the computer program, when executed on one or more processors, performs operations that provide a user interface for labeling of medical artificial intelligence. The above actions are, An operation of providing a first user interface listing labelable electrocardiogram features on an electrocardiogram signal; and An operation of providing a second user interface for visually displaying the electrocardiogram signal; including, Computer program.

15. A computing device providing a user interface for labeling medical artificial intelligence, A processor comprising at least one core; A memory containing program codes executable by the processor; and An input / output unit providing a first user interface listing labelable electrocardiogram features on an electrocardiogram signal and a second user interface visually displaying the electrocardiogram signal; Including device.

Images