Method and device for generating digital electrocardiogram signal

Abstract

The present disclosure relates to a method and a device for generating a digital electrocardiogram signal on the basis of an electrocardiogram record. According to an embodiment of the present disclosure, a method for generating a digital electrocardiogram signal on the basis of an electrocardiogram record is provided, the method including the steps of: performing preprocess to extract first electrocardiogram information on the basis of a first electrocardiogram record; training an artificial intelligence segmentation model on the basis of the first electrocardiogram record and the first electrocardiogram information; segmenting a second electrocardiogram record into second electrocardiogram information by using the second electrocardiogram record as an input of the trained artificial intelligence segmentation model; and converting the segmented second electrocardiogram information into a digital electrocardiogram signal.

Description

Digital electrocardiogram signal generation method and device

[0001] The present invention relates to the field of technology for processing heart-related data, and more specifically, to a method and apparatus for generating a digital electrocardiogram (ECG) signal from a non-digital ECG recording. The present invention relates to enhancing the usability of medical data through digital conversion by utilizing existing paper-based ECG data and converting it into a digital signal.

[0002] The electrocardiogram (ECG) records the electrical activity of the heart and serves as an important tool for diagnosing various heart diseases. It provides essential information for assessing the condition of the heart, including heart rate, rhythm, and abnormalities in electrical conduction. It plays a crucial role in the early diagnosis of various heart conditions, such as myocardial infarction, arrhythmia, and cardiomyopathy, and is utilized in diverse settings, including hospitals, emergency rooms, and telemedicine environments.

[0003] Electrocardiogram (ECG) data is used to infer heart disease by monitoring a patient's condition in real time or by comparing and analyzing historical data through records. In particular, digitized ECG data contributes to enhancing the accuracy and reliability of diagnoses by enabling quantitative analyses, such as heart rate variability analysis, T-wave abnormality detection, and PR interval measurement. This data is being utilized by an increasing number of medical institutions and plays a crucial role in automated heart disease prediction and early warning systems when combined with artificial intelligence (AI)-based algorithms.

[0004] However, in regions where digitized ECG data has not yet become widespread, paper-based non-digital ECG recordings are still widely used. These recordings rely on manual analysis by medical professionals, making it difficult to benefit from the latest medical technologies without conversion to digital data. Consequently, there is a growing demand for technology to effectively digitize non-digital ECG recordings and, through this, expand the utilization of digital data in the global healthcare environment.

[0005] The aforementioned background technology is technical information that the inventor possessed for the derivation of the present invention or acquired during the process of deriving the present invention, and it cannot be considered as prior art disclosed to the general public prior to the filing of the present invention.

[0006] The present invention has been devised in consideration of the above-mentioned purpose, and the objective of the present invention is to provide a method and apparatus for generating a digital electrocardiogram signal.

[0007] In addition, the objective of the present invention is to provide a method and apparatus for training a segmented artificial intelligence model based on electrocardiogram information.

[0008] In addition, the objective of the present invention is to provide a method and apparatus for generating a digital signal and simultaneously inferring heart disease information by utilizing an artificial intelligence model that extracts electrocardiogram information based on a non-digital electrocardiogram recording.

[0009] The problems to be solved in the embodiments of the present invention are not limited thereto, and may also include objectives or effects that can be identified from the means for solving the problems or embodiments described below.

[0010] A method for generating a digital electrocardiogram signal based on an electrocardiogram recording to solve the problem presented in the present invention may include: a step of preprocessing to extract first electrocardiogram information based on a first electrocardiogram recording; a step of training an artificial intelligence segmentation model based on the first electrocardiogram recording and the first electrocardiogram information; a step of segmenting a second electrocardiogram recording into second electrocardiogram information using the trained artificial intelligence segmentation model as input; and a step of converting the segmented second electrocardiogram information into a digital electrocardiogram signal.

[0011] The present invention can convert a non-digital electrocardiogram recording into a digital electrocardiogram by providing a method and apparatus capable of generating a digital electrocardiogram signal based on a non-digital electrocardiogram recording. In particular, the present invention can convert an electrocardiogram reading recorded on paper into a digital electrocardiogram signal.

[0012] In addition, generating digital electrocardiogram signals from non-digital electrocardiogram recordings can provide various medical and technical effects.

[0013] In addition, by training an artificial intelligence model that generates digital electrocardiogram signals based on non-digital electrocardiogram recordings, it is possible to generate digital electrocardiogram signals and simultaneously infer heart disease from the electrocardiogram readings.

[0014] FIG. 1 is a conceptual diagram showing a system in which various aspects of a method for generating a digital electrocardiogram signal based on an electrocardiogram recording related to one embodiment of the present invention can be implemented.

[0015] FIG. 2 is a block diagram illustrating the configuration of a computing device for generating a digital electrocardiogram signal based on an electrocardiogram recording related to an embodiment of the present invention.

[0016] FIG. 3 is a diagram illustrating electrocardiogram information extracted from an electrocardiogram recording of the present invention.

[0017] FIG. 4 is a flowchart illustrating a preprocessing method for extracting electrocardiogram information according to an embodiment of the present invention.

[0018] FIG. 5 is a diagram illustrating a system for generating a digital electrocardiogram signal based on an electrocardiogram recording according to an embodiment of the present invention.

[0019] FIG. 6 is a flowchart illustrating a method for learning and segmenting an artificial intelligence model that processes electrocardiogram information based on electrocardiogram information according to an embodiment of the present invention.

[0020] FIG. 7 is a flowchart illustrating a method for converting electrocardiogram information into a digital electrocardiogram signal based on subdivided electrocardiogram information according to an embodiment of the present invention.

[0021] FIG. 8 is a flowchart illustrating a method for generating a digital electrocardiogram signal based on an electrocardiogram recording according to an embodiment of the present invention.

[0022] FIGS. 9a and 9b are drawings illustrating experimental results obtained through a method for generating a digital electrocardiogram signal based on an electrocardiogram recording according to an embodiment of the present invention.

[0023] A method for generating a digital electrocardiogram signal based on an electrocardiogram recording to solve the problem presented in the present invention may include: a step of preprocessing to extract first electrocardiogram information based on a first electrocardiogram recording; a step of training an artificial intelligence segmentation model based on the first electrocardiogram recording and the first electrocardiogram information; a step of segmenting a second electrocardiogram recording into second electrocardiogram information using the trained artificial intelligence segmentation model as input; and a step of converting the segmented second electrocardiogram information into a digital electrocardiogram signal.

[0024] Additionally, the preprocessing step may include a step of converting the color space of the first electrocardiogram recording; and a step of extracting the first electrocardiogram information from the first electrocardiogram recording in which the color space has been converted.

[0025] And, the above color space can be converted from the RGB color space to the HSV color space.

[0026] In addition, the preprocessing step may further include a step of generating an electrocardiogram image using an image generation tool for the first electrocardiogram recording.

[0027] And, the first electrocardiogram information and the second electrocardiogram information may be a 3-channel binary mask.

[0028] In addition, the above artificial intelligence segmentation model may be a U-Net-based artificial intelligence model.

[0029] And, the artificial intelligence segmentation model includes an encoding unit and a decoding unit, the encoding unit includes a plurality of layers, and each of the plurality of layers of the encoding unit reduces the dimensionality as processing proceeds, and the decoding unit includes a plurality of layers, and each of the plurality of layers of the decoding unit may increase the dimensionality as processing proceeds.

[0030] Additionally, the method may further include the step of training the artificial intelligence segmentation model to classify heart disease; and the step of inputting the second electrocardiogram recording into the artificial intelligence segmentation model to perform classification of heart disease.

[0031] And, the step of converting into the digital electrocardiogram signal may include: a step of estimating the number and location of waveforms based on at least one of waveform information, text, and grid lines included in the second electrocardiogram information; and a step of obtaining a location index based on the estimated number and location of waveforms.

[0032] Additionally, the step of converting into a digital electrocardiogram signal may further include the step of dividing the second electrocardiogram information based on the position index.

[0033] And, the step of converting into the digital electrocardiogram signal may further include a step of correcting the slope based on a text index included in the second electrocardiogram information.

[0034] And, the step of converting into a digital electrocardiogram signal may include: a step of converting waveform information included in the second electrocardiogram information into a digital signal; a step of obtaining an amplitude by normalizing the waveform information of the second electrocardiogram information converted into a digital signal; and a step of generating the digital electrocardiogram signal based on text included in the second electrocardiogram information.

[0035] An apparatus for generating a digital electrocardiogram signal based on an electrocardiogram recording to solve the problem presented in the present invention comprises at least one memory; and at least one processor; wherein the processor preprocesses to extract first electrocardiogram information based on a first electrocardiogram recording, trains an artificial intelligence segmentation model based on the first electrocardiogram recording and the first electrocardiogram information, segments a second electrocardiogram recording into second electrocardiogram information using the trained artificial intelligence segmentation model as input, and converts the segmented second electrocardiogram information into a digital electrocardiogram signal.

[0036] The means for solving the above-mentioned problem describe a method and apparatus for generating a digital electrocardiogram signal based on an electrocardiogram recording according to one embodiment of the present invention, and it should be understood that additions, deletions, and changes are possible within the spirit and scope of the invention other than those described above.

[0037] The following detailed description of the invention refers to the accompanying drawings, which illustrate specific embodiments in which the invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention. It should be understood that various embodiments of the invention are different but need not be mutually exclusive. For example, specific shapes, structures, and characteristics described herein may be implemented in other embodiments without departing from the spirit and scope of the invention in relation to one embodiment.

[0038] Terms containing ordinal numbers, such as first, second, etc., may be used to describe various components, but the components are not limited by these terms. Terms may be used for the purpose of distinguishing one component from another. For example, it should be understood that a first component may be named a second component, and conversely, a second component may be named a first component.

[0039] The terms used in this application are used merely to describe specific embodiments and are not intended to limit the invention. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, terms such as "comprising" or "having" are intended to specify the presence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0040] Furthermore, it should be understood that the location or arrangement of individual components within each disclosed embodiment may be changed without departing from the spirit and scope of the invention. Accordingly, the following detailed description is not intended to be taken in a limiting sense, and the scope of the invention is limited only by the appended claims, including all equivalents thereof, provided appropriately described. Similar reference numerals in the drawings refer to the same or similar functions across various aspects.

[0041] FIG. 1 is a conceptual diagram showing a system in which various aspects of a method for generating a digital electrocardiogram signal based on an electrocardiogram recording related to one embodiment of the present invention can be implemented.

[0042] Referring to FIG. 1, a system in which various aspects of a method for generating a digital electrocardiogram signal based on an electrocardiogram recording related to one embodiment of the present invention may be implemented may include at least one of a computing device (10), an external server (20), and a user terminal (30), and each device may mutually transmit and receive data for the system through a network.

[0043] According to one embodiment of the present invention, a computing device (10) or an external server (20) may be a server that provides cloud computing services. More specifically, the computing device (10) or the external server (20) may be a server that provides cloud computing services, which are a type of internet-based computing, where information is processed by another computer connected to the internet rather than the user's computer. A cloud computing service may be a service that stores data on the internet and allows users to access necessary data or programs anytime and anywhere via internet access without installing them on their own computers, and allows data stored on the internet to be easily shared and transmitted through simple operations and clicks. Furthermore, a cloud computing service may not only simply store data on a server on the internet but also allow users to perform desired tasks using the functions of applications provided on the web without installing separate programs, and may be a service that allows multiple people to work while simultaneously sharing documents. Additionally, a cloud computing service may be implemented in at least one form among IaaS (Infrastructure as a Service), PaaS (Platform as a Service), SaaS (Software as a Service), a virtual machine-based cloud server, and a container-based cloud server. That is, the computing device (10) or external server (20) of the present invention may be implemented in at least one form of the cloud computing service described above. The specific description of the cloud computing service described above is merely an example and may include any platform for establishing the cloud computing environment of the present invention.

[0044] According to one embodiment of the present invention, a user terminal (30) may refer to a terminal possessed by a user that can receive or provide electrocardiogram recording images, electrocardiogram information, or electrocardiogram-related data through information exchange with a computing device (10) or an external server (20). For example, the user terminal (30) may be an electronic device for transmitting the user's electrocardiogram recording images, electrocardiogram information, or electrocardiogram-related data to the computing device (10), or an electronic device for receiving electrocardiogram recording images, electrocardiogram information, or electrocardiogram-related data processed by the computing device (10).

[0045] According to one embodiment of the present invention, the external server (20) may be a server that stores data for learning deep learning or artificial intelligence. Alternatively, the external server (20) may be a digital device equipped with a processor and memory to have computational capabilities.

[0046] In addition, an external server (20) according to one embodiment of the present invention may have an artificial intelligence, deep learning model, or large language model for digital conversion or heart disease inference based on an electrocardiogram recording and electrocardiogram information. For example, when an electrocardiogram recording image or electrocardiogram information or data related to an electrocardiogram obtained from a user terminal (30) is transmitted to the external server (20), the external server (20) can generate a digital electrocardiogram signal or infer heart disease through an implemented artificial intelligence model.

[0047] Additionally, according to one embodiment of the present invention, the computing device (10), external device (20), and user terminal (30) that construct the system may be named interchangeably, and may perform distributed computation for generating digital electrocardiogram signals or inferring heart disease, or perform one or more operations separately.

[0048] FIG. 2 is a block diagram illustrating the configuration of a computing device for generating a digital electrocardiogram signal based on an electrocardiogram recording related to an embodiment of the present invention.

[0049] According to one embodiment of the present invention, as illustrated in FIG. 1, a computing device (100) for generating a digital electrocardiogram signal based on an electrocardiogram recording may include a network unit (110), a memory (120), a sensor unit (130), and a processor (140). The components of the computing device (100) described above are exemplary, and the scope of the present invention is not limited to the components described above. That is, it should be understood that additional components may be included or some of the components described above may be omitted depending on the implementation mode of the embodiments of the present invention.

[0050] According to one embodiment of the present invention, a computing device (100) for generating a digital electrocardiogram signal based on an electrocardiogram recording may include a network unit (110) capable of transmitting and receiving data with the aforementioned external server (20) and user terminal (30). For example, the network unit (110) may transmit data to the external server (20) for computational processing required by the computing device (100), and may receive data from the external server (20) or the user terminal (30).

[0051] According to one embodiment of the present invention, the memory (120) may store a computer program for performing a method for generating a digital electrocardiogram signal based on an electrocardiogram recording, and the stored computer program may be read and executed by a processor (130). Additionally, the memory (120) may store any form of information and / or instructions generated or determined by the processor (130) and any form of information received by the network unit (110).

[0052] According to one embodiment of the present invention, the processor (130) can read a computer program stored in memory (120) to generate a digital electrocardiogram signal based on an electrocardiogram recording for learning a deep learning or artificial intelligence model and perform data processing for heart disease inference.

[0053] FIG. 3 is a diagram illustrating electrocardiogram information extracted from an electrocardiogram recording of the present invention.

[0054] The electrocardiogram (ECG) is important medical data that records the electrical activity of the heart and is used to diagnose various heart diseases, including heart rate and rhythm, as well as myocardial infarction, arrhythmia, and conduction abnormalities. In clinical practice, ECGs are utilized to monitor heart conditions in real time or to compare and analyze past records, establishing themselves as an essential tool in diverse environments such as hospitals, emergency situations, and telemedicine. Recently, digitized ECG data combined with artificial intelligence technology is significantly improving the accuracy of heart disease diagnosis and enhancing the quality of medical services through automated analysis and early warning systems.

[0055] Electrocardiogram (ECG) data can primarily be utilized for analyzing heart rate variability, measuring PR intervals and QRS widths, and detecting T-wave abnormalities. ECG recordings converted into digital signals are stored in databases and can be used to track a patient's condition over time or analyze the progression of heart disease. Such applications provide medical professionals with quantitative and objective diagnostic data, thereby enabling more reliable diagnoses and treatment planning.

[0056] Recently, electrocardiograms are measured using digital devices and are immediately acquired as digital electrocardiogram signals. With the recent advancement of data processing technologies, such as artificial intelligence, digitized electrocardiograms have enabled more precise analysis and prediction. However, conventional technology involved acquiring electrocardiogram recordings and having medical professionals directly analyze the data. Therefore, for non-digital electrocardiogram recordings, specifically those obtained from electrocardiogram reports, it is necessary to extract electrocardiogram information from the recordings to convert them into digital signals.

[0057] A preprocessing method for an electrocardiogram (ECG) recording according to an embodiment of the present invention can extract ECG information including "waveform information," "grid lines," and "text" from a non-digital ECG recording. ECG images obtained by scanning a paper-based ECG recording or capturing it with a digital camera can be preprocessed and extracted as ECG information. "Waveform information" is information representing the electrical activity of the heart, which can be used to generate a digital ECG signal. "Grid lines" are lines for alignment when recording an ECG record (e.g., an ECG report) on paper or the like; they serve as a standard for data alignment and size adjustment, and enable accurate measurement of the relative position and size of each waveform. Finally, "text" information includes additional information such as patient personal details, recording date, and medical staff notes; by including the "text" written on the ECG record (e.g., an ECG report) in a database, management efficiency is enhanced, and the data can be utilized in the process of digital conversion following preprocessing and segmentation. The present invention can automatically identify and extract the above information.

[0058] Additionally, electrocardiogram information according to one embodiment of the present invention may be extracted from an electrocardiogram recording or an electrocardiogram image. The electrocardiogram information may include at least one of "waveform information," "grid lines," and "text." However, it should be understood that the electrocardiogram information is not limited to what is described above and may include characteristic information contained in the electrocardiogram recording or electrocardiogram image.

[0059] FIG. 4 is a flowchart illustrating a preprocessing method for extracting electrocardiogram information according to an embodiment of the present invention.

[0060] Referring to FIG. 4, a preprocessing method for extracting electrocardiogram information according to one embodiment of the present invention may include the step of generating an electrocardiogram image using an image generation tool (S110), the step of converting the color space of the generated electrocardiogram image (S120), the step of extracting electrocardiogram information from the converted color space image (S130), and the step of combining the extracted electrocardiogram information (S140).

[0061] The step (S110) of generating an electrocardiogram image using an image generation tool can generate an electrocardiogram image by inputting an electrocardiogram recording or an electrocardiogram database into the image generation tool.

[0062] An image generation tool according to one embodiment of the present invention may be implemented using the ECG-Image-Kit program, but refers to an image generation tool related to an electrocardiogram, and is not limited thereto.

[0063] An image generation tool according to one embodiment of the present invention can generate an electrocardiogram image (e.g., an electrocardiogram reading image).

[0064] In addition, the image generation tool can generate electrocardiogram images (e.g., electrocardiogram readout images) and annotation information based on electrocardiogram signals from a pre-prepared electrocardiogram dataset.

[0065] For example, an image generation tool according to one embodiment of the present invention can generate an electrocardiogram reading image with a height of 1728 and a width of 2208 based on the ECG-Image-Kit program.

[0066] An electrocardiogram image generated by an image generation tool according to an embodiment of the present invention may include all elements necessary to visually represent the electrical activity of the electrocardiogram and may be used for conversion into a digital signal and data analysis in a subsequent step.

[0067] In addition, an electrocardiogram image generated by an image generation tool according to an embodiment of the present invention can be designed so that waveform information, grid lines, and text information are clearly distinguished, which can contribute to increasing the accuracy and reliability of information extraction.

[0068] The step (S120) of converting the color space of the generated electrocardiogram image can be performed to increase image processing efficiency and convert it into a format suitable for data analysis.

[0069] ECG image data typically uses the RGB (Red, Green, Blue) color space. While the RGB color space is suitable for representing colors, it can be inefficient for data analysis and computational processing.

[0070] Accordingly, the step of converting the color space (S120) according to one embodiment of the present invention may include the step of converting the RGB color space of the electrocardiogram image into the HSV (Hue, Saturation, Value) color space.

[0071] The HSV color space separates hue, saturation, and value, enabling more effective identification and analysis of specific elements within an image. Converting to the HSV color space enhances the contrast between waveform information and grid lines, and increases the accuracy of automated data extraction and digital signal conversion processes. The converted electrocardiogram images in the HSV color space can be utilized to process and extract electrocardiogram information more precisely in subsequent steps.

[0072] The step (S130) of extracting electrocardiogram information from a converted color space image can extract electrocardiogram information (e.g., waveform information, text, grid lines, etc.) from an electrocardiogram image in a converted color space.

[0073] Specifically, the step (S130) of extracting electrocardiogram information from a converted color space image can be performed based on a threshold-based technique and an algorithm that limits or cuts data within a specified boundary.

[0074] A threshold-based technique according to one embodiment of the present invention may be, for example, a thresholding technique. Specifically, the thresholding technique is a method of performing binary processing based on whether an unknown pixel value exceeds or falls below a specific threshold, and can rapidly separate a region of interest within an image.

[0075] An algorithm for restricting or clipping data within a specified boundary according to one embodiment of the present invention may be a clipping algorithm. Specifically, a clipping algorithm is an algorithm for restricting or clipping data within a specified boundary and is suitable for accurately extracting fixed structures such as text.

[0076] However, it should be understood that the thresholding technique and clipping algorithm are techniques according to one embodiment for extracting electrocardiogram information from an image in a converted color space, and that a person skilled in the art may appropriately adopt an extraction method to facilitate the processing of image segmentation described below.

[0077] For example, in the step (S130) of extracting electrocardiogram information from a converted color space image, if the converted color space of the electrocardiogram image is HSV, the electrocardiogram waveform region can be extracted by applying a thresholding technique to the Value (brightness) channel, and the text region can be extracted by applying a thresholding technique combined with a clipping algorithm.

[0078] Additionally, the step (S130) of extracting electrocardiogram information from a converted color space image can extract grid lines of an electrocardiogram image (e.g., an electrocardiogram report image) by applying a thresholding technique to the saturation channel when the converted color space of the electrocardiogram image is HSV.

[0079] The step of combining extracted electrocardiogram information (S140) can combine the extracted electrocardiogram information (e.g., waveform information, text, grid lines, etc.) to generate a mask having three channels. Additionally, the mask having three channels may be binary and may be referred to as a three-channel binary mask in this specification.

[0080] In addition, a 3-channel binary mask generated by combining extracted electrocardiogram information can be used as training data for an artificial intelligence segmentation model, and the artificial intelligence segmentation model can proceed with training by using the 3-channel binary mask as the correct answer data for training.

[0081] In addition, a 3-channel binary mask according to one embodiment of the present invention independently represents extracted electrocardiogram information, thereby enabling an artificial intelligence model to easily identify the features and detailed structure of the electrocardiogram image.

[0082] FIG. 5 is a diagram illustrating a system for generating a digital electrocardiogram signal based on an electrocardiogram recording according to an embodiment of the present invention.

[0083] A system (200) for generating a digital electrocardiogram signal based on an electrocardiogram recording according to one embodiment of the present invention may include a preprocessing module (210), a segmentation model (220), and a digitization module (230).

[0084] A preprocessing module (210) according to one embodiment of the present invention may include an electrocardiogram data set (212), an image generation tool (214), an electrocardiogram reading (or electrocardiogram image) (216), and extracted electrocardiogram information (218). Additionally, the preprocessing module (210) may perform the aforementioned preprocessing to generate a digital electrocardiogram signal containing accurate information from an electrocardiogram recording or an electrocardiogram image (216).

[0085] The electrocardiogram data set (212) is a database containing existing non-digital electrocardiogram records and may include scanned electrocardiogram reading images or non-digitized electrocardiogram data. The data included in the electrocardiogram data set (212) may be input into an image generation tool (214), and electrocardiogram information (218) may be extracted directly using the preprocessing method described above.

[0086] The image generation tool (214) is a tool for generating an electrocardiogram reading sheet or an electrocardiogram image (216), and can generate an image in the form of a reading sheet by visually representing electrocardiogram data. The image generation tool (214) of one embodiment of the present invention may be an ECG-Image-Kit.

[0087] The electrocardiogram reading sheet or electrocardiogram image (216) can be used as an input for extracting electrocardiogram information (218) by the preprocessing method described above, or as an input to the segmentation model (220).

[0088] A segmentation model (220) according to one embodiment of the present invention may be an artificial intelligence model for the segmentation of medical data. Specifically, it may be an artificial intelligence used to distinguish specific structures or diseases, etc., in medical images (e.g., CT, MRI, ultrasound, ECG reports, etc.) in medical data.

[0089] For example, the segmentation model (220) may include at least one of U-Net, V-Net, ResUNet, Attention U-Net, Dense U-Net, SegNet, and 3D U-Net, but is not limited thereto.

[0090] A segmentation model (220) according to one embodiment of the present invention may include an encoder unit (222) and a decoder unit (224).

[0091] Specifically, the encoder unit (222) can extract high-level features by progressively compressing the input electrocardiogram information through multiple layers of convolution layers and pooling operations. Through this, key information (e.g., waveform information, grid lines, text) of the electrocardiogram recording (e.g., electrocardiogram reading image) can be effectively learned.

[0092] Additionally, specifically, the decoder unit (224) can restore detailed information based on high-level features extracted from the encoder unit, along with information received from the encoder unit, and generate an output image close to the original resolution.

[0093] Additionally, the segmentation model (220) according to one embodiment of the present invention may be a U-Net-based artificial intelligence model. For example, the encoder unit (222) may be a U-Net-based artificial intelligence model using EfficienNet-B0 and the decoder unit (224) may be a U-Net-based artificial intelligence model using a U-Net Decoder.

[0094] Additionally, the segmentation model (220) according to one embodiment of the present invention can perform data augmentation to respond to variations, as the actual electrocardiogram reading may include information on geometric features, paper texture, and color that are more complex and diverse than the generated data.

[0095] In addition, a segmentation model (220) according to one embodiment of the present invention can perform the data augmentation described above, apply a domain adaptation technique to learn the distribution of various paper documents, and prevent overfitting of the model.

[0096] In addition, the segmentation model (220) according to one embodiment of the present invention may additionally perform learning to classify diseases based on an input electrocardiogram record (e.g., an electrocardiogram image or an electrocardiogram reading image).

[0097] Specifically, heart disease classification can be learned by generating a logit in the encoder part (222) of the artificial intelligence segmentation model (220) and calculating an auxiliary loss function.

[0098] A digitization module (230) according to one embodiment of the present invention may include subdivided electrocardiogram information (232), a waveform detection unit (234), a slope correction unit (236), a signal range estimation unit (238), and an electrocardiogram signal conversion unit (240).

[0099] The digitization module (230) can perform processing to generate a digital signal based on the subdivided electrocardiogram information.

[0100] A waveform detection unit (234) according to one embodiment of the present invention can detect a waveform included in subdivided electrocardiogram information (232). The waveform detection unit (234) can convert the pixel thickness in the waveform channel. For example, the pixel thickness can be converted (e.g., thickness 1) to minimize the pixel thickness and enable efficient analysis through an algorithm (e.g., Skeletonize algorithm) for representing the structure by simplifying objects in the waveform channel of the subdivided electrocardiogram information (232). Subsequently, the image data can be scanned from left to right using a kernel having a predetermined size (e.g., height 1) to calculate the sum of pixel values ​​at each location. Based on the information obtained through the above-described algorithm and kernel, the number and location of the waveform can be estimated. For example, the number of waveforms is determined by calculating the mode of the pixel sum list, and location information of pixels having a value greater than 0 is listed in a list based on the index where the mode exists. A location index can be obtained by averaging the listed list.

[0101] In addition, the waveform detection unit (234) according to one embodiment of the present invention can separate and extract waveform information, grid, and text channel areas by setting an intersection range based on a position index obtained based on the number and location of estimated waveforms. Specifically, it can be extracted in the form of slices.

[0102] A slope correction unit (236) according to one embodiment of the present invention can locate an area where an electrocardiogram lead name exists based on text information included in the subdivided electrocardiogram information (232). Subsequently, a specific point (e.g., the top center) for each area where an electrocardiogram lead name exists can be designated as a text index.

[0103] Also, the text index can be multiple text indices. In this case, the slope of the slice can be estimated by calculating the height difference between the first text index and the last text index. Subsequently, correction can be performed on the estimated slope, and correction can be omitted if the slope is lower than a preset value.

[0104] Specifically, once the slope is estimated, the vertex positions of the slice can be corrected by the slope, and the four corrected vertices can be projected onto a new plane through perspective transformation to correct the sloped slice.

[0105] In the case of physical electrocardiogram readings and captured electrocardiogram images, the printed or scanned image of the reading itself may be tilted, and the captured electrocardiogram image may also be tilted; therefore, by correcting the tilt, an accurate digital electrocardiogram signal can be obtained.

[0106] A signal range estimation unit (238) according to one embodiment of the present invention may use an algorithm to accurately estimate the range of a waveform signal from subdivided electrocardiogram information (232). Specifically, the thickness of a pixel (e.g., thickness 1) may be transformed by applying an algorithm (e.g., Skeletonize algorithm, etc.) to simplify and represent an object for a grid line channel. Broken grid lines may be supplemented through an erosion and expansion algorithm, and horizontal lines may be extracted using a horizontal line kernel. The distance between the extracted horizontal lines may be calculated using a kernel having a predetermined height (e.g., height 1), listed in a distance list, and the width of one grid cell may be estimated by calculating the average value of the list.

[0107] In addition, grid channel regions where no waveform exists can be removed based on the estimated grid width. Furthermore, the number of grids can be estimated by dividing the height of the cut grid channel region by the width of one grid cell. For example, since one grid cell on an electrocardiogram generally represents 0.1 mV, the signal range of the waveform can be estimated by multiplying the estimated number of grids by 0.1.

[0108] An electrocardiogram signal conversion unit (240) according to one embodiment of the present invention can convert data obtained based on the processing described above into a digital electrocardiogram signal. For example, a kernel of a predetermined height (e.g., height 1) is used for the extracted waveform region to arrange the height index of the part where pixel values ​​exist in one dimension, thereby converting the waveform signal into a digital form. In addition, through linear interpolation and normalization, loss or missing values ​​can be corrected by linear interpolation, and normalization can be applied to adjust to the estimated signal range value. Subsequently, amplitude conversion and division are performed, the signal range is converted into amplitude by subtracting the average value from the normalized signal, and the waveform is divided based on the center point of the electrocardiogram lead name extracted from the text channel to generate a digital signal per lead.

[0109] FIG. 6 is a flowchart illustrating a method for learning and segmenting an artificial intelligence model that processes electrocardiogram information based on electrocardiogram information according to an embodiment of the present invention.

[0110] A method for training and segmenting an artificial intelligence model based on electrocardiogram information according to an embodiment of the present invention may include the steps of: training an artificial intelligence model based on extracted electrocardiogram information (S210); inputting an electrocardiogram reading image into the artificial intelligence model (S220); extracting features of the electrocardiogram reading image using an encoder of the artificial intelligence model (S230); and performing segmentation processing based on the extracted features using a decoder of the artificial intelligence model (S240).

[0111] The step of training an artificial intelligence model based on extracted electrocardiogram information (S210) may further include a step of training the artificial intelligence model using the electrocardiogram information (e.g., waveform information, text, grid lines) extracted through the preprocessing process as described above as the correct training data.

[0112] Specifically, waveform information, text, and grid lines among the extracted electrocardiogram information are generated into a binary mask having three channels, and the three-channel binary mask is used as the correct answer for training an artificial intelligence model to proceed with training.

[0113] The step (S220) of inputting an electrocardiogram image into an artificial intelligence model may input an electrocardiogram image into an artificial intelligence model trained using a 3-channel binary mask as the correct answer. Here, the electrocardiogram image may include at least one of an electrocardiogram image used for training, an electrocardiogram image taken of an actual electrocardiogram report, and a generated electrocardiogram image, but is not limited thereto.

[0114] The step (S230) of extracting electrocardiogram image features using an encoder of an artificial intelligence model can extract features of the 3 channels (waveform information, text, grid lines) of the input electrocardiogram image through the encoder of an artificial intelligence model trained with a 3-channel binary mask as the correct answer. Additionally, the encoder can be designed with a structure in which the total number of channels increases as the layers pass, and the dimensionality decreases, thereby allowing high-level features to be extracted.

[0115] The step (S240) of subdividing processing based on features extracted by the decoder of the artificial intelligence model can increase the resolution to an image of the same level as the input data and reflect the features based on high-level features obtained from the encoder unit and data processed in each layer of the encoder unit. Additionally, the structure of the decoder unit can be designed such that the total number of channels decreases as the layers pass, and the dimensionality increases, thereby generating an output that reflects the information of the input electrocardiogram image at each layer.

[0116] FIG. 7 is a flowchart illustrating a method for converting electrocardiogram information into a digital electrocardiogram signal based on subdivided electrocardiogram information according to an embodiment of the present invention.

[0117] A method for converting electrocardiogram information into a digital electrocardiogram signal based on subdivided electrocardiogram information according to an embodiment of the present invention may include the steps of receiving subdivided electrocardiogram information (S310), detecting an electrocardiogram waveform region (S320), slicing based on the detected electrocardiogram waveform region (S330), correcting the slope based on the electrocardiogram information (S340), estimating the range of the electrocardiogram signal (S350), and converting the electrocardiogram information into a digital signal.

[0118] The step of receiving electrocardiogram information (S310) may be a step in which a digitization module receives the segmented electrocardiogram information output from the decoder of the artificial intelligence segmentation model and prepares for digital conversion.

[0119] The step of detecting an electrocardiogram waveform region (S320) may be a step of detecting a waveform region based on detailed electrocardiogram information (e.g., waveform information, text, grid lines, etc.). Specifically, the number and location of waveforms may be estimated based on the electrocardiogram information.

[0120] Additionally, the step of slicing based on the detected electrocardiogram waveform region (S330) may further include the step of obtaining a waveform location index based on the estimated number and location of waveforms, and the step of separating and extracting waveform, grid line, and text channel regions based on the waveform location index.

[0121] The step of correcting the slope based on electrocardiogram information (S340) can improve the accuracy of signal analysis by correcting the sloped waveform and text of the subdivided electrocardiogram information and electrocardiogram image based on the electrocardiogram information. Specifically, the slope can be corrected by detecting a text index, calculating the slope, and performing perspective transformation correction.

[0122] The step of estimating the range of the electrocardiogram signal (S350) may be a step of performing processing to accurately estimate the range of the waveform signal from the electrocardiogram information. Specifically, the signal range may be calculated by applying an algorithm to represent the structure by simplifying the object (e.g., a Skeletonize algorithm), supplementing broken grids, and removing grid channel regions where no waveform exists to separate the waveform and grid regions.

[0123] The step of converting electrocardiogram information into a digital signal (S360) can convert the waveform signal into a digital form using the height index described above, perform linear interpolation on the deficit value, and apply normalization to adjust it to match the signal range value. Subsequently, the signal range can be changed to amplitude by subtracting the average value from the normalized signal, and digital data per lead can be generated by dividing the waveform based on text among the electrocardiogram information.

[0124] FIG. 8 is a flowchart illustrating a method for generating a digital electrocardiogram signal based on an electrocardiogram recording according to an embodiment of the present invention.

[0125] Referring to FIG. 8, a method for generating a digital electrocardiogram signal based on an electrocardiogram recording according to an embodiment of the present invention may include a step of preprocessing to extract first electrocardiogram information based on a first electrocardiogram recording (S410), a step of training an artificial intelligence model based on the first electrocardiogram recording and the first electrocardiogram information (S420), a step of subdividing a second electrocardiogram recording into second electrocardiogram information using the second electrocardiogram recording as input to the artificial intelligence model (S430), and a step of digitally converting into an electrocardiogram signal based on the subdivided second electrocardiogram information (S440).

[0126] The step of preprocessing to extract first electrocardiogram information based on the first electrocardiogram recording (S410) may be a step of performing the preprocessing described above to extract first electrocardiogram information.

[0127] Specifically, the first electrocardiogram recording may be an actual electrocardiogram reading record used for training an artificial intelligence model or an electrocardiogram image included in a database. The first electrocardiogram recording may be generated as an electrocardiogram image using an image generation tool, or the color space may be converted, and the first electrocardiogram information may be extracted from the converted color space image.

[0128] Additionally, the first electrocardiogram information may include at least one of waveform information, text, and grid lines, and may be a 3-channel binary mask in which the three pieces of information are combined.

[0129] Next, the step (S420) of training an artificial intelligence model based on the first electrocardiogram recording and the first electrocardiogram information may be a step of training using the first electrocardiogram recording as input and the first electrocardiogram information as the correct answer for training the artificial intelligence segmentation model.

[0130] Additionally, the step (S430) of subdividing the second electrocardiogram record into second electrocardiogram information using the second electrocardiogram record as input to an artificial intelligence model may be a step of outputting a result subdivided into second electrocardiogram information using the second electrocardiogram record as input to an artificial intelligence subdivision model learned as described above. Specifically, when a second electrocardiogram record in image form (RGB or HSV color space) is input to an artificial intelligence subdivision model, it may be subdivided into second electrocardiogram information and output.

[0131] The step (S440) of digitally converting into an electrocardiogram signal based on subdivided second electrocardiogram information can process the subdivided second electrocardiogram information and digitally convert it into a digital electrocardiogram signal.

[0132] Specifically, the step (S440) of digitally converting into an electrocardiogram signal based on the second electrocardiogram information based on the subdivided second electrocardiogram information may include at least one of the steps of detecting an electrocardiogram waveform region of the second electrocardiogram information, slicing the region based on the detected electrocardiogram waveform region, correcting the slope, estimating the range of the signal, and converting into a digital signal.

[0133] FIGS. 9a and 9b are drawings illustrating experimental results obtained through a method for generating a digital electrocardiogram signal based on an electrocardiogram recording according to an embodiment of the present invention.

[0134] FIG. 9a is the result of evaluating signal generation performance through a method for generating a digital electrocardiogram signal based on an electrocardiogram recording according to one embodiment of the present invention.

[0135] As shown in Fig. 9a, when the HSV color space is utilized during the learning process, it can be seen that different result values ​​are obtained in each contaminated situation and clear electrocardiogram image data.

[0136] FIG. 9b is the result of evaluating the performance of heart disease classification through a method of generating a digital electrocardiogram signal based on an electrocardiogram recording according to one embodiment of the present invention.

[0137] As illustrated in FIG. 9b, it can be seen that even when inferring heart disease simultaneously with generating a digital signal, the method of the present invention can secure a relatively constant level of inference capability.

[0138] An embodiment according to the present invention may be implemented in the form of a computer program that can be executed through various components on a computer, and such a computer program may be recorded on a computer-readable medium. In this case, the medium may include a magnetic medium such as a hard disk, a floppy disk, and a magnetic tape, an optical recording medium such as a CD-ROM and a DVD, a magneto-optical medium such as a floptical disk, and a hardware device specifically configured to store and execute program instructions, such as a ROM, RAM, or flash memory.

[0139] Meanwhile, the above-mentioned computer program may be one specifically designed and configured for the present invention, or one known and available to those skilled in the art of computer software. Examples of computer programs may include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc.

[0140] According to one embodiment, the method according to various embodiments of the present disclosure may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or distributed online (e.g., download or upload) through an application store (e.g., Play Store™) or directly between two user devices. In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created in a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0141] Although the detailed description of the invention above has been given with reference to preferred embodiments of the invention, those skilled in the art should understand that various modifications and changes can be made to the invention without departing from the spirit and scope of the invention as described in the claimed patent claims.

Claims

1. A method for generating a digital electrocardiogram signal based on an electrocardiogram recording, A step of preprocessing to extract first electrocardiogram information based on the first electrocardiogram recording; A step of training an artificial intelligence segmentation model based on the above-mentioned first electrocardiogram recording and first electrocardiogram information; A step of subdividing the second electrocardiogram record into second electrocardiogram information using the second electrocardiogram record as input to the learned artificial intelligence subdivision model; and A method comprising the step of converting the subdivided second electrocardiogram information into a digital electrocardiogram signal.

2. In Paragraph 1, The above preprocessing step is, A step of converting the color space of the first electrocardiogram recording; and The method includes the step of extracting the first electrocardiogram information from the first electrocardiogram recording in which the color space has been converted, and The above color space is a method of converting from an RGB color space to an HSV color space.

3. In Paragraph 1, The above preprocessing step is, A method further comprising the step of generating an electrocardiogram image using an image generation tool for the first electrocardiogram recording.

4. In Paragraph 1, A method in which the first electrocardiogram information and the second electrocardiogram information are a 3-channel binary mask.

5. In Paragraph 1, The above artificial intelligence segmentation model is a U-Net-based artificial intelligence model, a method.

6. In Paragraph 1, The above artificial intelligence segmentation model includes an encoding unit and a decoding unit, and The above encoding unit includes a plurality of layers, and each of the plurality of layers of the encoding unit reduces its dimensionality as processing proceeds, and A method in which the decoding unit comprises a plurality of layers, and each of the plurality of layers of the decoding unit increases in dimension as processing proceeds.

7. In Paragraph 1, Step of training the above artificial intelligence segmentation model to classify heart disease; and A method further comprising the step of inputting the second electrocardiogram recording into the artificial intelligence segmentation model to perform heart disease classification.

8. In Paragraph 1, The step of converting the above digital electrocardiogram signal is, A step of estimating the number and location of waveforms based on at least one of waveform information, text, and grid lines included in the second electrocardiogram information; and A method comprising the step of obtaining a position index based on the number and location of the estimated waveforms.

9. In Paragraph 8, The step of converting the above digital electrocardiogram signal is, A method further comprising the step of dividing the second electrocardiogram information based on the above position index.

10. In Paragraph 1, The step of converting the above digital electrocardiogram signal is, A method further comprising the step of correcting the slope based on a text index included in the second electrocardiogram information.

11. In Paragraph 1, The step of converting the above digital electrocardiogram signal is. A step of converting waveform information included in the second electrocardiogram information into a digital signal; A step of obtaining amplitude by normalizing the waveform information of the second electrocardiogram information converted into a digital signal; and A method comprising the step of generating the digital electrocardiogram signal based on text included in the second electrocardiogram information.

12. A device for generating a digital electrocardiogram signal based on an electrocardiogram recording, At least one memory; and Includes at least one processor; and The above processor is, A device for preprocessing to extract first electrocardiogram information based on a first electrocardiogram recording, training an artificial intelligence segmentation model based on the first electrocardiogram recording and the first electrocardiogram information, segmenting a second electrocardiogram recording into second electrocardiogram information using the trained artificial intelligence segmentation model as input, and converting the segmented second electrocardiogram information into a digital electrocardiogram signal.

Images