High-robustness electrocardiogram paper digitization method for low-quality and missing data

Abstract

The invention discloses a high-robustness electrocardiographic paper digitization method for low-quality and missing data, which comprises the following steps: step 1, image preprocessing: carrying out de-inclination, binaryzation and de-noising operations on an electrocardiographic paper image, and obtaining scale information in electrocardiographic paper; step 2, scale extraction: detecting and identifying line segment intervals of background grids on electrocardiographic paper to serve as references for subsequent signal quantitative analysis; step 3, grid removal: removing background grids of the electrocardiogram paper, and retaining effective signals in the electrocardiogram; step 4, breakpoint adaptive connection: automatically detecting and connecting electrocardiogram signal breakpoints to ensure the integrity of the signals; and 5, missing signal compensation: compensating signal overlapping or losing at the jump point on the electrocardiogram paper to generate a complete digital signal data set. According to the invention, high-precision digitalization of various electrocardiographic papers can be realized, and data support is provided for automatic analysis and interpretation of subsequent electrocardiographic signals.

Description

Technical Field

[0001] The invention belongs to the field of electrocardiogram paper digitization, and in particular is a highly robust electrocardiogram paper digitization method for low-quality and missing data, which is used for realizing digital processing and automatic analysis of low-quality electrocardiogram paper. Background Art

[0002] In clinical practice, the electrocardiogram (ECG) is one of the important tools for diagnosing cardiovascular diseases. However, many medical institutions still use traditional paper formats for ECG records. These ECG sheets often have low quality and missing data due to factors such as ink evaporation, blurred handwriting, and paper folding. In particular, ECG paper images taken by doctors' handheld devices usually have problems such as shadows, wrinkles, tilts, ink stains, signal overlap, and signal disconnection, making the digitization process of ECG sheets full of challenges. When dealing with these special situations, existing digitization methods still lack a unified database and evaluation standards, resulting in the need to improve the digitization effect in special situations.

[0003] In order to achieve more efficient and accurate digital processing of ECGs, there is an urgent need for a highly robust digitization method for ECG paper with low quality and missing data, which can provide a reliable data basis for the automated analysis and interpretation of ECG signals. Summary of the invention

[0004] The object of the present invention is to provide a highly robust electrocardiogram paper digitization method for low-quality and missing data, which overcomes the shortcomings of the above-mentioned prior art.

[0005] To achieve the above purpose, the technical solution adopted by the present invention is as follows:

[0006] In a first aspect, the present invention provides a highly robust electrocardiogram paper digitization method for low-quality and missing data, comprising the following steps:

[0007] Step 1: Image preprocessing: De-tilt, binarize, and denoise the ECG paper image, and obtain the scale information in the ECG paper by manually delineating the target area and extracting the scale based on Hough transform;

[0008] Step 2: Scale extraction: Detect and identify the line intervals of the background grid on the ECG paper as a reference for subsequent signal quantitative analysis;

[0009] Step 3: Grid removal: Use the Attention U-Net model to remove the background grid in the ECG paper and retain the effective signal in the ECG. The model is trained with a simulation data set to effectively segment the ECG signal under different grid styles and noise backgrounds.

[0010] Step 4: Breakpoint adaptive connection: Through image processing and tracking algorithm, the breakpoints of the ECG signal are automatically detected and connected to ensure the integrity of the signal;

[0011] Step 5: Missing signal compensation: An unsupervised learning model based on occlusion autoencoder is used to compensate for signal overlap or loss at the transition points on the ECG paper to generate a complete digital signal data set.

[0012] Furthermore, in step 1, the jdeskew Python package is used to implement image tilt correction through MATLAB calling, and the image tilt angle is obtained using Fourier amplitude spectrum and adaptive radial projection method.

[0013] Furthermore, the step 2 comprises:

[0014] (1) Grayscale conversion: If the input image is not a grayscale image, it is converted to a grayscale image;

[0015] (2) Edge detection: Use the Canny algorithm for edge detection;

[0016] (3) Hough transform: Hough transform is applied to the results of edge detection in order to detect straight lines in the image;

[0017] (4) Peak detection: Use the houghpeaks function to find the peak in the Hough transform result;

[0018] (5) Line segment acquisition: The line segments in the image are acquired through the houghlines function according to the results and peak values ​​of the Hough transform;

[0019] (6) Screening and calculating line segment intervals: Traverse the found line segments and classify them as vertical or horizontal according to their angles. Then, for the two categories of vertical and horizontal, calculate the differences in the rho values ​​of these line segments in the Hough space. These differences represent the actual spatial intervals of the line segments in the image.

[0020] (7) Analyze the intervals to determine the ruler: By analyzing the distribution and consistency of these intervals, a common interval is found as the typical interval between line segments in the image, namely the "ruler";

[0021] (8) Determine the scale value: By comparing the consistency of the intervals, the final scale value is determined according to a predefined threshold.

[0022] Furthermore, in step 3, by constructing a simulation data set and combining the attention mechanism of the Attention U-Net model, low-level edge information and high-level semantic information are adaptively fused to enhance the robustness of de-gridding; constructing simulation training data, the specific steps of creation are:

[0023] (1) Preparation phase and data loading

[0024] Load the data file, which contains data variables and double limb leads;

[0025] (2) Image preprocessing and selection

[0026] Traverse the data in the data variable and process them in groups;

[0027] (3) Background grid selection

[0028] Randomly select a background grid image, randomly adjust its contrast, and change the size of the adjusted grid image to a set pixel;

[0029] (4) Generate label image

[0030] Plot the data points of each array to create a curve image and binarize them as sample labels;

[0031] (5) Add noise and transparency processing

[0032] Add random white noise and custom noise model based on the sample label image;

[0033] (6) Generate sample images

[0034] Adjust the transparency of the label image and the grid image, and combine them to generate the final sample image;

[0035] After obtaining the samples, they are divided into training set, validation set and test set according to a certain ratio, and the Attention U-Net model is used for training to obtain the de-gridding model.

[0036] Furthermore, in step 4, according to the detection and gradual expansion of the curve seed point, a dynamic search window is used to identify and reconstruct the missing electrocardiogram curve, which specifically includes:

[0037] (1) Image loading and preprocessing

[0038] Grayscale and binarization: Convert the image to grayscale and then binarize it to simplify the image into black and white pixels;

[0039] Cleaning and filtering: Use morphological operations and median filtering to remove noise;

[0040] Edge detection: Apply edge detection algorithms to identify boundaries and features in images;

[0041] (2) Curve tracing preparation

[0042] Confidence initialization: Initialize a confidence array to evaluate the reliability of curve tracing in the image;

[0043] Seed point identification: identifying potential starting points for curve tracing based on specific criteria;

[0044] Seed selection and search width setting: A seed is randomly selected from the identified seeds, and the search width is calculated based on the average distance between the pixel pairs used as seeds;

[0045] (3) Curve tracing

[0046] Initial Setup: Set the initial parameters for curve tracing, including direction and baseline for color-coding detection;

[0047] Tracking loop: Starting from the initial seed point, it iteratively expands the curve, adjusts its path according to the existence of pixels in the search area, and further expands the search range to continue tracking when there are no pixels in the small search range;

[0048] Direction change: If the end of the image or no pixels are reached, the algorithm changes direction to ensure complete coverage;

[0049] (4) Curve Perfection

[0050] Connect breakpoints: Identify and connect the breakpoints of the curve to ensure continuity;

[0051] Curve Termination: When multiple possible paths are detected, adjust the curve to follow the median path, ensuring a single continuous curve.

[0052] Furthermore, in step 5, based on the MAE model, training is performed using data from the PTB-XL ECG database, and the correlation between multi-lead signals is utilized to compensate for and reconstruct the missing signals.

[0053] Furthermore, the method is implemented through the MATLAB App design tool, integrating image tilt correction, target area division, scale acquisition, grid removal, breakpoint adaptive connection and missing signal compensation functional modules for digital processing and signal reconstruction of electrocardiogram paper.

[0054] In a second aspect, the present invention provides a highly robust electrocardiogram paper digitizing device for low-quality and missing data, comprising:

[0055] An image preprocessing module is used to perform de-tilting, binarization and noise removal operations on the ECG paper image;

[0056] The scale extraction module is used to extract the line intervals of the background grid on the electrocardiogram paper for subsequent signal quantitative analysis;

[0057] A grid removal module, used to remove the background grid in the electrocardiogram paper and retain the effective signal in the electrocardiogram;

[0058] Breakpoint adaptive connection module, used to automatically detect and connect ECG signal breakpoints to ensure signal integrity;

[0059] Missing signal compensation module: used to compensate for signal overlap or loss at the transition points on the electrocardiogram paper to generate a complete digital signal data set.

[0060] In a third aspect, the present invention provides an electronic device comprising a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the electrocardiogram paper digitization method as described in the first aspect.

[0061] In a fourth aspect, the present invention provides a readable storage medium, wherein a program or instruction is stored on the readable storage medium, and when the program or instruction is executed by a processor, the steps of the electrocardiogram paper digitization method as described in the first aspect are implemented.

[0062] The beneficial effects of the present invention are as follows: the present invention can solve various problems encountered in the digitization process of traditional electrocardiogram paper, such as image tilt, grid interference, signal disconnection and data loss, etc. By adopting a series of advanced image processing technologies and deep learning algorithms, the present method can achieve high-precision digitization of various types of electrocardiogram paper, and provide data support for the subsequent automated analysis and interpretation of electrocardiogram signals. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 It is a flow chart of the present invention.

[0064] Figure 2 It is a data flow diagram of the work of the present invention.

[0065] Figure 3 It is a flow chart of the effect of ruler extraction of the present invention.

[0066] Figure 4 It is a flow chart of the effect of breakpoint connection of the present invention. DETAILED DESCRIPTION

[0067] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0068] like Figures 1 to 4 A highly robust electrocardiogram paper digitization method for low-quality and missing data, the method includes the steps of image preprocessing, ruler extraction, grid removal, breakpoint adaptive connection and missing signal compensation, and is used to realize the digitization of low-quality electrocardiogram paper. The specific steps are as follows:

[0069] Step 1: Image preprocessing: De-tilt, binarize, and remove noise from the ECG paper image, and obtain the scale information in the ECG paper;

[0070] Step 2: Scale extraction: Detect and identify the line intervals of the background grid on the ECG paper as a reference for subsequent signal quantitative analysis, including:

[0071] Ⅰ. Grayscale conversion: If the input image is not grayscale, it is converted to grayscale, because edge detection and subsequent steps do not require color information.

[0072] Ⅱ. Edge Detection: Use the Canny algorithm for edge detection. The Canny edge detector is a popular edge detection algorithm that can effectively identify lines and edges in images.

[0073] III. Hough Transform: Hough transform is applied to the result of edge detection to detect straight lines in the image. Hough transform is a feature extraction technique used to identify complex patterns (e.g., straight lines, circles, etc.) from images

[37] . The search angle range is set to -90 to 89.5 degrees, with every 0.5 degree as a search point.

[0074] IV. Peak Detection: Use the houghpeaks function to find the peaks in the result of the Hough transform. These peaks correspond to the most prominent straight lines in the image.

[0075] Ⅴ. Line segment acquisition: The line segments in the image are acquired through the houghlines function according to the results and peak values ​​of the Hough transform.

[0076] Ⅵ. Screening and calculating line segment intervals: The code traverses the found line segments and classifies them into vertical (the absolute value of the angle is between 88 and 92 degrees) or horizontal (the absolute value of the angle is between 0 and 2 degrees) according to the angle of the line segment. Then, for the two categories of vertical and horizontal, the differences in the rho values ​​(the vertical distance from the straight line to the origin, that is, the distance from the nearest point of the straight line to the origin of the image coordinates) of these line segments in the Hough space are calculated. These differences represent the actual spatial intervals of the line segments in the image.

[0077] Ⅶ. Analyze the intervals to determine the ruler: By analyzing the distribution and consistency of these intervals, a common interval is found as the typical interval between line segments in the image, namely the "ruler".

[0078] VIII. Determine the scale value: The code determines the final scale value by comparing the consistency of the intervals according to the predefined threshold. If an interval meets the consistency requirements of most line segment intervals, then this interval is identified as the scale value.

[0079] Step 3: Grid removal: Remove the background grid of the ECG paper and retain the effective signal in the ECG, including:

[0080] Construct simulation training data. The specific steps are as follows:

[0081] I. Preparation phase and data loading

[0082] Load the data file, which contains the data variable, double limb leads, a sampling frequency of 200 Hz, and a length of 2624457 data points.

[0083] II. Image Preprocessing and Selection

[0084] Traverse the data in the data variable and process them in groups of 2200 data points (11s, to avoid signal overlap).

[0085] Ⅲ. Background grid selection

[0086] A background grid image is randomly selected, its contrast is randomly adjusted, and the resized grid image is changed to 512x1536 pixels (a single image can generate 3 512×512 samples for model input).

[0087] IV. Generate Label Images

[0088] Plot the data points of each array to create a curve image and binarize them as sample labels.

[0089] Ⅴ. Add noise and transparency processing

[0090] Add random white noise and customized noise models (cross, vertical line, horizontal line, ink dot, etc.) based on the sample label image.

[0091] VI. Generate sample images

[0092] Adjust the transparency of the label image and the grid image, and combine them to generate the final sample image. The random parts include the selection of the grid, the contrast setting of the grid, the thickness of the curve, the transparency of the curve, the type and size of the white noise, and the custom noise.

[0093] After obtaining the samples, the training set, validation set, and test set are divided into training set, validation set, and test set in a ratio of 6:2:2, and the Attention U-Net model is used for training to obtain the de-gridding model.

[0094] Step 4: Breakpoint adaptive connection: Automatically detect and connect ECG signal breakpoints to ensure signal integrity, including:

[0095] Ⅰ. Image loading and preprocessing

[0096] Grayscale and Binarization: Convert the image to grayscale and then binarize it to simplify the image into black and white pixels.

[0097] Cleaning and filtering: Use morphological operations (opening) and median filtering to remove noise.

[0098] Edge Detection: Apply edge detection algorithms to identify boundaries and features in an image.

[0099] Ⅱ. Curve Tracing Preparation

[0100] Confidence initialization: Initialize a confidence array for evaluating the reliability of curve tracing in the image.

[0101] Seed point identification: Identify potential starting points (seeds) for curve tracing based on certain criteria (e.g., there are exactly two pixels in a column, etc.).

[0102] Seed selection and search width setting: A seed is randomly selected from the identified seeds, and the search width is calculated based on the average distance between pairs of pixels that are seeds.

[0103] III. Curve tracing

[0104] Initial Setup: Set the initial parameters for curve tracing, including direction and baseline for detection using color coding.

[0105] Tracking loop: Starting from an initial seed point, it iteratively expands the curve, adjusting its path based on the presence of pixels in the search area, and further expanding the search range to continue tracking when no pixels are found in a small search range.

[0106] Direction change: If the end of the image or no pixels are reached, the algorithm changes direction to ensure complete coverage.

[0107] IV. Curve Perfection

[0108] Connect Breakpoints: Identify and connect the breakpoints of a curve to ensure continuity.

[0109] Curve Termination: When multiple possible paths are detected, adjust the curve to follow the median path, ensuring a single continuous curve.

[0110] Step 5: Missing signal compensation: Compensate for signal overlap or loss at the transition points on the ECG paper to generate a complete digital signal data set.

[0111] The present invention can realize high-precision digitization of various types of ECG paper, and provide data support for the subsequent automated analysis and interpretation of ECG signals. Based on the idea of ​​MAE, the ECG signals in the public database PTB-XL are pre-processed in the form of image signals and input into the model, and the corresponding image compensation model is trained to enable the model to learn the relationship between the signals of the standard 12 leads, so as to compensate for the signal missing or poor digitization effect at the overlapping points or jump points of the digitized ECG signals.

[0112] like Figure 2The workflow of the present invention is as follows: after the electrocardiogram paper is input, pre-processing operations such as image tilt correction, manual calibration of the target area, and automatic scale extraction are first performed. Subsequently, a model designed for removing the grid is input to remove the background grid of the interference signal, providing a purified image basis for subsequent signal digitization. During the digital conversion process, the breakpoints are automatically detected and connected to ensure the integrity of the signal. If a lead signal that cannot be directly digitized is found, a compensation model is used to estimate and restore the missing signal to form a complete digital signal data set.

[0113] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the above embodiments do not limit the protection scope of the present invention in any form, and all technical solutions obtained by equivalent replacement and the like fall within the protection scope of the present invention. The parts not involved in the present invention are the same as the prior art or can be implemented by the prior art.

Claims

1. A highly robust electrocardiogram paper digitization method for low-quality and missing data, characterized in that: The steps include: Step 1: Image preprocessing: De-tilt, binarize, and denoise the ECG paper image, and obtain the scale information in the ECG paper by manually delineating the target area and extracting the scale based on Hough transform; Step 2: Scale extraction: Detect and identify the line intervals of the background grid on the ECG paper as a reference for subsequent signal quantitative analysis; Step 3: Grid removal: Use the Attention U-Net model to remove the background grid in the ECG paper and retain the effective signal in the ECG. The model is trained with a simulation data set to effectively segment the ECG signal under different grid styles and noise backgrounds. Step 4: Breakpoint adaptive connection: Through image processing and tracking algorithm, the breakpoints of the ECG signal are automatically detected and connected to ensure the integrity of the signal; Step 5: Missing signal compensation: An unsupervised learning model based on occlusion autoencoder is used to compensate for signal overlap or loss at the transition points on the ECG paper to generate a complete digital signal data set.

2. A highly robust electrocardiogram paper digitization method for low-quality and missing data according to claim 1, characterized in that: In step 1, the jdeskew Python package is used to implement image tilt correction through MATLAB calling, and the image tilt angle is obtained using Fourier amplitude spectrum and adaptive radial projection method.

3. A highly robust electrocardiogram paper digitization method for low-quality and missing data according to claim 1, characterized in that: The step 2 comprises: (1) Grayscale conversion: If the input image is not a grayscale image, it is converted to a grayscale image; (2) Edge detection: Use the Canny algorithm for edge detection; (3) Hough transform: Hough transform is applied to the results of edge detection in order to detect straight lines in the image; (4) Peak detection: Use the houghpeaks function to find the peak in the Hough transform result; (5) Line segment acquisition: The line segments in the image are acquired through the houghlines function according to the results and peak values ​​of the Hough transform; (6) Screening and calculating line segment intervals: Traverse the found line segments and classify them as vertical or horizontal according to their angles. Then, for the two categories of vertical and horizontal, calculate the differences in the rho values ​​of these line segments in the Hough space. These differences represent the actual spatial intervals of the line segments in the image. (7) Analyze the intervals to determine the ruler: By analyzing the distribution and consistency of these intervals, a common interval is found as the typical interval between line segments in the image, namely the "ruler"; (8) Determine the scale value: By comparing the consistency of the intervals, the final scale value is determined according to a predefined threshold.

4. A highly robust electrocardiogram paper digitization method for low-quality and missing data according to claim 1, characterized in that: In step 3, by constructing a simulation data set and combining the attention mechanism of the Attention U-Net model, low-level edge information and high-level semantic information are adaptively fused to enhance the robustness of de-gridding; constructing simulation training data, the specific steps are as follows: (1) Preparation phase and data loading Load the data file, which contains data variables and double limb leads; (2) Image preprocessing and selection Traverse the data in the data variable and process them in groups; (3) Background grid selection Randomly select a background grid image, randomly adjust its contrast, and change the size of the adjusted grid image to a set pixel; (4) Generate label image Plot the data points of each array to create a curve image and binarize them as sample labels; (5) Add noise and transparency processing Add random white noise and custom noise model based on the sample label image; (6) Generate sample images Adjust the transparency of the label image and the grid image, and combine them to generate the final sample image; After obtaining the samples, they are divided into training set, validation set and test set according to a certain ratio, and the Attention U-Net model is used for training to obtain the de-gridding model.

5. A highly robust electrocardiogram paper digitization method for low-quality and missing data according to claim 1, characterized in that: In step 4, according to the detection and gradual expansion of the curve seed point, a dynamic search window is used to identify and reconstruct the missing electrocardiogram curve, which specifically includes: (1) Image loading and preprocessing Grayscale and binarization: Convert the image to grayscale and then binarize it to simplify the image into black and white pixels; Cleaning and filtering: Use morphological operations and median filtering to remove noise; Edge detection: Apply edge detection algorithms to identify boundaries and features in images; (2) Curve tracing preparation Confidence initialization: Initialize a confidence array to evaluate the reliability of curve tracing in the image; Seed point identification: identifying potential starting points for curve tracing based on specific criteria; Seed selection and search width setting: A seed is randomly selected from the identified seeds, and the search width is calculated based on the average distance between the pixel pairs used as seeds; (3) Curve tracing Initial Setup: Set the initial parameters for curve tracing, including direction and baseline for color-coding detection; Tracking loop: Starting from the initial seed point, it iteratively expands the curve, adjusts its path according to the existence of pixels in the search area, and further expands the search range to continue tracking when there are no pixels in the small search range; Direction change: If the end of the image or no pixels are reached, the algorithm changes direction to ensure complete coverage; (4) Curve Perfection Connect breakpoints: Identify and connect the breakpoints of the curve to ensure continuity; Curve Termination: When multiple possible paths are detected, adjust the curve to follow the median path, ensuring a single continuous curve.

6. A highly robust electrocardiogram paper digitization method for low-quality and missing data according to claim 1, characterized in that: In step 5, based on the MAE model, training is performed using data from the PTB-XL ECG database, and the correlation between multi-lead signals is utilized to compensate for and reconstruct the missing signals.

7. A highly robust electrocardiogram paper digitization method for low-quality and missing data according to claim 1, characterized in that: The method is implemented through the MATLAB App design tool, which integrates image tilt correction, target area division, scale acquisition, grid removal, breakpoint adaptive connection and missing signal compensation functional modules for digital processing and signal reconstruction of electrocardiogram paper.

8. A highly robust electrocardiogram paper digitization device for low-quality and missing data, characterized in that: include: An image preprocessing module is used to perform de-tilting, binarization and noise removal operations on the ECG paper image; The scale extraction module is used to extract the line intervals of the background grid on the electrocardiogram paper for subsequent signal quantitative analysis; A grid removal module, used to remove the background grid in the electrocardiogram paper and retain the effective signal in the electrocardiogram; Breakpoint adaptive connection module, used to automatically detect and connect ECG signal breakpoints to ensure signal integrity; Missing signal compensation module: used to compensate for signal overlap or loss at the transition points on the electrocardiogram paper to generate a complete digital signal data set.

9. An electronic device, characterized in that: It comprises a processor, a memory and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the electrocardiogram paper digitization method as described in any one of claims 1 to 7.

10. A readable storage medium, characterized in that: The readable storage medium stores a program or instruction, and when the program or instruction is executed by a processor, the steps of the electrocardiogram paper digitization method as described in any one of claims 1 to 7 are implemented.

Images