System and method for electrocardiogram digitization

Abstract

A digitization system and method for electrocardiograms (ECGs) are proposed. The method includes: a camera device capturing an ECG printout; a processor obtaining image frames of the ECG printout from the image stream buffer of the camera device; during the capture, the processor running a first artificial intelligence model analyzes the image frames to generate multiple ECG printout parameters, wherein the first artificial intelligence model is pre-trained on multiple synthetic ECG images, the multiple synthetic ECG images covering various numerical settings, various grid and lead separator styles, and various combinations of distortion parameters for each ECG printout parameter; and during the capture, the processor running at least a second artificial intelligence model generates ECG digital data based on the image frames and the multiple ECG printout parameters.

Claims

1. A method for digitizing an electrocardiogram, comprising: The method comprises: capturing an electrocardiogram (ECG) chart using a camera device; obtaining a frame of the ECG chart from an image streaming buffer of the camera device using a processor; during capture, analyzing the frame using the processor running a first artificial intelligence model to generate multiple ECG chart parameters, wherein the first artificial intelligence model is pre-trained with multiple synthetic ECG images, the synthetic ECG images covering multiple numerical settings, multiple grid and lead separator styles, and multiple distortion parameter arrangements for each of the ECG chart parameters; and during capture, generating ECG digital data based on the frame and the ECG chart parameters using the processor running at least a second artificial intelligence model; wherein the frame is a first frame, the ECG chart parameters are multiple first ECG chart parameters, and the ECG digital data is first ECG digital data, and the method further comprises: after obtaining the first frame, obtaining multiple second frames of the ECG chart from the image streaming buffer of the camera device multiple times using the processor; During the filming process, the processor running the first artificial intelligence model analyzes the second frames multiple times to generate multiple sets of second electrocardiogram (ECG) chart parameters; during the filming process, the processor running at least the second artificial intelligence model generates multiple sets of second ECG digital data multiple times based on the second frames and the multiple sets of second ECG chart parameters; during the filming process, the processor running a classification model generates multiple classification results and multiple classification confidence levels based on the first frame and the second frames, or based on the first frame, the second frames, the first ECG digital data, and the second ECG digital data. When the processor determines that at least one of the following automatic shooting conditions is met, it controls the camera device to automatically shoot to generate an image of the electrocardiogram (ECG) chart: in the first frame and the second frames, the average difference between consecutive frames is less than the image stability threshold; each of the classification confidence values ​​is greater than the confidence threshold; the classification variability between the consecutive frames is less than the variability threshold; and the variability between the first ECG chart data and the second ECG digital data is less than the variability tolerance value.

2. The electrocardiogram digitization method as described in claim 1, wherein the electrocardiogram paper parameters include lead arrangement, paper speed and voltage gain, and the distortion parameters include analog rotation, perspective changes, handwritten marks, shadows, creases and wrinkles.

3. The electrocardiogram digitization method as described in claim 1, further comprising, after the processor obtains the image frame of the electrocardiogram drawing from the image streaming buffer of the camera device, the method further comprising: The processor performs multiple preprocessing steps on the image grid, including contrast enhancement, an electrocardiogram (ECG) grid detection algorithm, rotation correction, and perspective distortion removal. The ECG grid algorithm includes grayscale conversion, Gaussian blurring, adaptive threshold binarization, performing a first contour detection to find the largest contour in the binarization result, executing a convex hull algorithm to generate a mask based on the largest contour, performing a second contour detection to correct errors, extracting multiple corner points of a rectangle containing the mask, and cropping the image grid based on these corner points.

4. The method for digitizing an electrocardiogram as described in claim 1, wherein the processor, which at least runs the second artificial intelligence model, generates the digital electrocardiogram data based on the image and the electrocardiogram text parameters, comprising: The processor, running the second artificial intelligence model, detects multiple bounding boxes containing electrocardiogram (ECG) bands from the image grid; the processor clips the image grid into multiple sub-image grids based on the bounding boxes; and the processor, running a third artificial intelligence model, analyzes the sub-image grids based on the ECG text parameters to output the ECG digital data.

5. The method for digitizing an electrocardiogram as described in claim 1, wherein the processor, which at least runs the second artificial intelligence model, generates the digital electrocardiogram data based on the image and the electrocardiogram text parameters, comprising: The processor, which runs the second artificial intelligence model, marks multiple pixels in the image that represent electrocardiogram (ECG) bands, wherein the second artificial intelligence model is U-Net; and the processor generates the ECG digital data based on the pixels and the ECG text parameters.

6. The method for digitizing an electrocardiogram as described in claim 1 further includes: During the recording, the image frame and the ECG chart parameters are displayed in real time by a preview element of the camera device; during the recording, the processor receives the updated image frame or receives the updated ECG chart parameters through an input device; and the ECG digitization method as described in claim 1 is executed based on the updated image frame or the updated ECG chart parameters to generate updated ECG digital data.

7. A digitization system for electrocardiograms, comprising: A camera device for capturing an electrocardiogram (ECG) chart; and a processor electrically connected to the camera device, the processor being used to obtain a frame of the ECG chart from an image streaming buffer of the camera device; during capture, a first artificial intelligence model is run to analyze the frame and generate multiple ECG chart parameters, and at least a second artificial intelligence model is run to generate ECG digital data based on the frame and the ECG chart parameters; wherein the first artificial intelligence model is pre-trained with multiple synthetic ECG images, the synthetic ECG images covering multiple numerical settings, multiple grid and lead separator styles, and multiple combinations of distortion parameters for each of the ECG chart parameters; Wherein, the image frame is a first image frame, the electrocardiogram (ECG) chart parameters are multiple first ECG chart parameters, and the ECG digital data is first ECG digital data. The processor is further configured to: after acquiring the first image frame, repeatedly acquire multiple second image frames of the ECG chart from the image streaming buffer of the camera device; during the recording period, run the first artificial intelligence model multiple times to analyze the second image frames to generate multiple sets of second ECG chart parameters; during the recording period, at least run the second artificial intelligence model multiple times to generate multiple sets of second ECG digital data based on the second image frames and the multiple sets of second ECG chart parameters; during the recording period, run a classification model based on the first image frame and the... The system generates multiple classification results and multiple classification confidence levels based on the first image frame, the second image frame, the first electrocardiogram (ECG) digital data, and the second ECG digital data. When at least one of the following automatic shooting conditions is met, the system controls the camera to automatically capture images to generate photographs of the ECG chart: the average difference between consecutive images in the first and second image frames is less than an image stability threshold; each of the classification confidence levels is greater than a confidence threshold; the classification variability between consecutive images is less than a variability threshold; and the variability between the first ECG chart data and the second ECG digital data is less than a variability tolerance value.

8. The electrocardiogram digitization system as claimed in claim 7, wherein the electrocardiogram paper parameters include lead arrangement, paper speed and voltage gain, and the distortion parameters include analog rotation, perspective distortion, handwritten marks, shadows, creases and wrinkles.

Images