Image processing method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention discloses an image processing method and device, electronic equipment and a storage medium. The method comprises the steps that at least one single-lead electrocardiogram is obtained from an electrocardiogram image; each single-lead electrocardiogram comprises an electrocardiogram curve corresponding to one lead; for each pixel block in the target single-lead electrocardiogram, obtaining a corresponding color feature and a position feature, and constructing a corresponding feature component by using the corresponding color feature and the position feature; the target single-lead image is any single-lead image, and each pixel block comprises at least one pixel; and carrying out pixel clustering on the target single-lead electrocardiogram by utilizing the feature component corresponding to each pixel block, and extracting an electrocardiogram curve from the target single-lead electrocardiogram according to a clustering result.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of image processing technology, and in particular to an image processing method, device, electronic device and storage medium. Background Art

[0002] In the history of medicine, there are two examination and diagnostic techniques that have been used clinically for more than a hundred years and have remained popular. One is X-ray examination, and the other is electrocardiogram examination. Electrocardiogram is the process of bioelectric changes in the heart during each movement cycle, and is a manifestation of the heart's reciprocating activity. Electrocardiogram examination is the fastest and most accurate non-invasive technology for diagnosing acute myocardial ischemia and arrhythmia.

[0003] The current electrocardiograph has long entered the digital stage from the analog stage. The digitization of electrocardiographs has the following significance: first, it can improve the speed, accuracy and automation of diagnosis, reduce the burden on relevant staff and improve work efficiency; second, it can establish a more general digital electrocardiograph database, which plays an important role in clinical research and scientific research analysis; third, with the help of the universality of the network, online diagnosis can be achieved. Adapting to the current requirements of network transmission, the digital extraction of electrocardiograph information can enable multiple doctors to diagnose at the same time, improve the quality of diagnosis, and solve the problem that patients with serious conditions cannot go to other places for medical treatment.

[0004] However, in the current electrocardiogram digitization process, the accuracy of electrocardiogram curve recognition and extraction is poor. Summary of the invention

[0005] The embodiments of the present application provide an image processing method, device, electronic device and storage medium, which improve the accuracy of electrocardiogram curve recognition and extraction.

[0006] The technical solution of the embodiment of the present application is implemented as follows:

[0007] The present application provides an image processing method, including:

[0008] Acquire at least one single-lead electrocardiogram from the electrocardiogram image; each single-lead electrocardiogram includes an electrocardiogram curve corresponding to one lead;

[0009] For each pixel block in the target single-lead electrocardiogram, a corresponding color feature and a position feature are obtained, and a corresponding feature component is constructed using the corresponding color feature and the position feature; the target single-lead image is any single-lead image, and each pixel block includes at least one pixel;

[0010] The target single-lead electrocardiogram is pixel-clustered using the characteristic components corresponding to each pixel block, and an electrocardiogram curve is extracted from the target single-lead electrocardiogram according to the clustering result.

[0011] In the above method, the electrocardiogram image includes a plurality of electrocardiogram curves corresponding to a plurality of leads, and obtaining at least one single-lead electrocardiogram from the electrocardiogram image includes:

[0012] Calculating edge features of the electrocardiogram image, and determining an electrocardiogram curve region from the electrocardiogram image based on the edge features; the electrocardiogram curve region includes the multiple electrocardiogram curves;

[0013] identifying a layout form of the plurality of electrocardiogram curves from the electrocardiogram curve region;

[0014] In the case where the layout form is multiple rows and multiple columns, the electrocardiogram region is split into columns, and each column of the electrocardiogram obtained is split into rows, and each image block obtained by the split is determined as a single-lead electrocardiogram;

[0015] In the case where the layout form is multiple rows and a single column, a column of ECG curves in the ECG curve area is split into rows, and each of the split image blocks is determined as a single-lead ECG; wherein each image block contains an ECG curve.

[0016] In the above method, it also includes:

[0017] After grayscale conversion of the target column of ECG curves, the target column of ECG curves is projected onto the vertical axis to obtain a grayscale vertical axis distribution diagram; the target column of ECG curves is any column of ECG curves in the ECG curve area;

[0018] The target column electrocardiogram curve is split into rows by using the periodic square wave characteristics of the grayscale vertical axis distribution diagram on the vertical axis.

[0019] In the above method, the electrocardiogram image is a multi-lead electrocardiogram image, including a plurality of electrocardiogram curves corresponding to a plurality of leads, and obtaining at least one single-lead electrocardiogram from the electrocardiogram image includes:

[0020] matching the electrocardiogram image with different electrocardiogram templates stored in a database;

[0021] In the case where the electrocardiogram image matches any electrocardiogram template, a plurality of image blocks are calibrated from the electrocardiogram image according to the layout of the electrocardiogram template that matches the electrocardiogram image; or, in the case where the electrocardiogram image does not match any electrocardiogram template, a plurality of image blocks are calibrated from the electrocardiogram image according to the received lead calibration instruction; wherein each image block includes an electrocardiogram curve corresponding to a lead;

[0022] Each image block is calibrated to be a single-lead ECG.

[0023] In the above method, clustering pixels of the target single-lead electrocardiogram using the characteristic components corresponding to each pixel block, and extracting an electrocardiogram curve from the target single-lead electrocardiogram according to the clustering result, includes:

[0024] Counting the color types included in the target single-lead electrocardiogram to obtain K colors; K is a natural number greater than 1;

[0025] Using a K-means clustering algorithm, the pixels in the target single-lead electrocardiogram are clustered into K sets according to the characteristic components corresponding to each pixel block; each set contains pixels of one color;

[0026] Determine a set of pixels containing a target color in the K sets as a target set; the target color is the color of the electrocardiogram curve of the target single-lead electrocardiogram;

[0027] A line consisting of pixels included in the target set is extracted from the target single-lead electrocardiogram and determined as an electrocardiogram curve in the target single-lead electrocardiogram.

[0028] In the above method, it also includes:

[0029] Acquire a text information region from the electrocardiogram;

[0030] Using different text recognition engines to respectively recognize the characters contained in the text information area to obtain multiple character sequences; each character sequence is recognized by one text recognition engine;

[0031] Aligning the multiple character sequences and comparing them, and determining inconsistent characters as erroneous characters;

[0032] Correct the erroneous characters to determine a correct character sequence corresponding to the text information area.

[0033] In the above method, the step of obtaining a text information area from the electrocardiogram image includes:

[0034] Demarcating the text information area from the electrocardiogram according to the layout of the electrocardiogram template matching the electrocardiogram;

[0035] Alternatively, according to the received character marking instruction, marking the text information area from the electrocardiogram image;

[0036] Alternatively, when the edge feature of the electrocardiogram is calculated, the text information region is acquired from the electrocardiogram based on the edge feature.

[0037] In the above method, the step of correcting the erroneous characters to determine the correct character sequence of the text information area includes:

[0038] Constructing a candidate character set for the erroneous characters based on the principle of similar shape and similar sound;

[0039] Taking any character sequence among the multiple character sequences as an observed state and the correct character sequence as a hidden state, a hidden Markov model is constructed;

[0040] The hidden Markov model is used to select the correct character corresponding to the wrong character from the candidate character set to update the character sequence, thereby obtaining the correct character sequence.

[0041] The present application provides an image processing device, including:

[0042] An acquisition module, used for acquiring at least one single-lead electrocardiogram from the electrocardiogram image; each single-lead electrocardiogram includes an electrocardiogram curve corresponding to one lead;

[0043] A construction module, for acquiring corresponding color features and position features for each pixel block in a target single-lead electrocardiogram, and constructing corresponding feature components using the corresponding color features and position features; the target single-lead image is any single-lead image, and each pixel block includes at least one pixel;

[0044] The extraction module is used to perform pixel clustering on the target single-lead electrocardiogram using the characteristic components corresponding to each pixel block, and extract an electrocardiogram curve from the target single-lead electrocardiogram according to the clustering result.

[0045] An embodiment of the present application provides an electronic device, including: a processor, a memory, and a communication bus;

[0046] The communication bus is used to realize the communication connection between the processor and the memory;

[0047] The processor is used to execute one or more programs stored in the memory to implement the above-mentioned image processing method.

[0048] An embodiment of the present application provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned image processing method is implemented.

[0049] Embodiments of the present application Embodiments of the present application provide an image processing method, device, electronic device and storage medium, the method comprising: obtaining at least one single-lead electrocardiogram from an electrocardiogram image; each single-lead electrocardiogram contains an electrocardiogram curve corresponding to a lead; for each pixel block in the target single-lead electrocardiogram, obtaining the corresponding color feature and position feature, and constructing the corresponding feature component using the corresponding color feature and position feature; the target single-lead image is any single-lead image, and each pixel block includes at least one pixel; pixel clustering is performed on the target single-lead electrocardiogram using the feature component corresponding to each pixel block, and the electrocardiogram curve is extracted from the target single-lead electrocardiogram according to the clustering result. The technical solution provided in the embodiment of the present application adopts the feature component constructed by the color feature and position feature of the pixel block as the basis for pixel clustering, thereby extracting the electrocardiogram curve, and improving the accuracy of electrocardiogram curve recognition and extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 A schematic diagram of an exemplary application scenario provided for an embodiment of the present application;

[0051] Figure 2 A flowchart of an image processing method provided in an embodiment of the present application;

[0052] Figure 3 An exemplary electrocardiogram provided in the present application is shown in FIG. Figure 1 ;

[0053] Figure 4 A schematic diagram of a process of intelligent layout analysis provided in an embodiment of the present application Figure 1 ;

[0054] Figure 5 A schematic diagram of a process of intelligent layout analysis provided in an embodiment of the present application Figure 2 ;

[0055] Figure 6 A schematic diagram of an exemplary interface of lead layout analysis provided in an embodiment of the present application;

[0056] Figure 7 A schematic diagram of a manual layout analysis process provided in an embodiment of the present application;

[0057] FIG8( a ) is a schematic diagram of an exemplary pixel arrangement provided in an embodiment of the present application. Figure 1 ;

[0058] FIG8( b ) is a schematic diagram of an exemplary pixel arrangement provided in an embodiment of the present application. Figure 2 ;

[0059] FIG8(c) is a schematic diagram of an exemplary pixel arrangement provided in an embodiment of the present application. Figure 3;

[0060] FIG8(d) is a schematic diagram of an exemplary pixel arrangement provided in an embodiment of the present application. Figure 4 ;

[0061] FIG9( a) is a schematic diagram of an exemplary pixel arrangement provided in an embodiment of the present application. Figure 5 ;

[0062] FIG9( b ) is a schematic diagram of an exemplary pixel arrangement provided in an embodiment of the present application. Figure 6 ;

[0063] FIG9(c) is a schematic diagram of an exemplary pixel arrangement provided in an embodiment of the present application. Figure 7 ;

[0064] FIG9( d ) is an exemplary pixel arrangement schematic diagram 8 provided in an embodiment of the present application;

[0065] FIG9( e) is a ninth exemplary pixel arrangement diagram provided in an embodiment of the present application;

[0066] FIG9( f) is a schematic diagram 10 of an exemplary pixel arrangement provided in an embodiment of the present application;

[0067] FIG9( g) is a schematic diagram of an exemplary pixel arrangement provided in an embodiment of the present application. Figure 11 ;

[0068] FIG9(h) is a schematic diagram of an exemplary pixel arrangement provided in an embodiment of the present application. Figure 12 ;

[0069] FIG. 10( a ) is a schematic diagram of an exemplary pixel arrangement provided in an embodiment of the present application. Figure 13 ;

[0070] FIG. 10( b ) is a schematic diagram of an exemplary pixel arrangement provided in an embodiment of the present application. Figure 14 ;

[0071] FIG. 10( c ) is a schematic diagram of an exemplary pixel arrangement provided in an embodiment of the present application. Figure 15 ;

[0072] FIG. 10( d ) is a schematic diagram of an exemplary pixel arrangement provided in an embodiment of the present application. Figure 16 ;

[0073] Fig.11 An exemplary binary encoding diagram after conversion provided in an embodiment of the present application;

[0074] Fig.12 An exemplary electrocardiogram provided in the present application is shown in FIG. Figure 2 ;

[0075] Fig.13 A flowchart of a text recognition method provided in an embodiment of the present application;

[0076] Fig.14 A schematic diagram of the structure of an image processing device provided in an embodiment of the present application Figure 1 ;

[0077] Fig.15 A schematic diagram of the structure of an image processing device provided in an embodiment of the present application Figure 2 ;

[0078] Fig.16 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0079] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0080] The following will specifically describe the technical solution of the present application and how the technical solution of the present application solves the above technical problems through embodiments and in conjunction with the accompanying drawings. The following embodiments may be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

[0081] In addition, the technical solutions described in the embodiments of the present application can be combined arbitrarily without conflict.

[0082] The embodiment of the present application provides an image processing method, which is applicable to hospital in-hospital scenes / pre-hospital emergency scenes. Due to differences in the years of purchase and differences in the ecological layout of manufacturers, some low-end, old, and closed ecosystem manufacturers' ECG machines either do not provide a digital protocol for quick access, or the business threshold for software docking is high, and they cannot be effectively integrated into the smart hospital system; the technical solution provided by this application will provide manufacturers with a formed video output interface, or use a camera to collect ECG images, structure medical image data, facilitate the unified management of ECG data from heterogeneous manufacturers, improve the use cycle of old equipment / equipment from closed ecosystem manufacturers, reduce the time cycle for medical equipment docking, and facilitate the flexible replacement and upgrade of pre-hospital emergency vehicle equipment. In addition, for pre-hospital emergency scenes, this proposal greatly reduces the minimum transmission bandwidth requirement (from more than 100Kbps to less than 5Kbps) by digitizing the curves and medical reports in the image at the same time, which is particularly easy to transmit in a weak network environment during the ambulance transfer process; it is worth noting that the input source is a video signal such as an ECG machine, and the intermediate image and text extraction are mainly completed locally. After the vector extraction is completed, the captured device data can be reproduced locally or through the network. Related scenarios such as Figure 1 shown.

[0083] Figure 2 A schematic diagram of a flow chart of an image processing method provided in an embodiment of the present application. Figure 2 As shown, in the embodiment of the present application, the image processing method mainly includes the following steps:

[0084] S101. Acquire at least one single-lead electrocardiogram from an electrocardiogram image; each single-lead electrocardiogram includes an electrocardiogram curve corresponding to one lead.

[0085] In an embodiment of the present application, the image processing device can obtain at least one single-lead electrocardiogram from the electrocardiogram image, and each single-lead electrocardiogram includes an electrocardiogram curve corresponding to one lead.

[0086] It should be noted that, in the embodiments of the present application, the electrocardiogram may be an electrocardiogram including only one electrocardiogram curve corresponding to one lead, or may be an electrocardiogram including multiple electrocardiogram curves, each of which corresponds to one lead, and the image processing device may obtain a single-lead electrocardiogram including a single electrocardiogram curve from the electrocardiogram. If the electrocardiogram includes multiple electrocardiogram curves corresponding to multiple leads, the number of single-lead electrocardiograms extracted therefrom may be multiple, which is not limited in the embodiments of the present application.

[0087] In the embodiments of the present application, the extraction of a single-lead electrocardiogram may be achieved by using an intelligent layout analysis method or a manual layout analysis method, which are described in detail below.

[0088] In an embodiment of the present application, an electrocardiogram image includes multiple electrocardiogram curves corresponding to multiple leads, and at least one single-lead electrocardiogram is obtained from the electrocardiogram image by intelligent layout analysis, including: calculating edge features of the electrocardiogram image, and determining an electrocardiogram curve region from the electrocardiogram image based on the edge features; the electrocardiogram curve region includes multiple electrocardiogram curves; identifying the layout form of the multiple electrocardiogram curves from the electrocardiogram curve region; when the layout form is multiple rows and multiple columns, the electrocardiogram curve region is split into columns, and each column of the obtained electrocardiogram curves is split into rows, and each image block obtained by the split is determined as a single-lead electrocardiogram; when the layout form is multiple rows and a single column, a column of electrocardiogram curves in the electrocardiogram curve region is split into rows, and each image block obtained by the split is determined as a single-lead electrocardiogram; wherein each image block includes an electrocardiogram curve corresponding to a lead.

[0089] Figure 3 An exemplary electrocardiogram provided in the present application is shown in FIG. Figure 1 . For example Figure 3 The electrocardiogram shown is shown in Figure 4In the process shown, the calibration of the electrostatic curve area in the intelligent layout analysis mainly involves the following steps:

[0090] S1, load ECG image;

[0091] S2, successively calculating the edge features of the upper, lower and right edges of the ECG image;

[0092] S3, returning to identify the corresponding boundary line on the interface;

[0093] S4, user adjusts the boundary line (optional);

[0094] S5, the user confirms the boundary line;

[0095] S6, start analyzing the upper, lower and right information columns and mark the ECG curve area;

[0096] S7, return ECG equipment information, including user calibration voltage X (unit: x mv / 1mm), layout of multiple ECG curves;

[0097] S8. User adjusts ECG device information (optional)

[0098] S9, the user confirms the ECG device information;

[0099] S10, storing features of the electrocardiogram, including overall frequency domain distribution information (histogram, etc.), frequency domain distribution information of each boundary status bar, etc.;

[0100] S11, indicating that calibration is complete.

[0101] In an embodiment of the present application, in the intelligent layout analysis, after completing the above-mentioned calibration, the calibrated ECG curve area (covering the four boundary positions) can be further split into rows and columns to obtain an image block containing the ECG curve of a single lead as a single-lead ECG for subsequent processing. The related processing objects include curves, backgrounds, grids, lead text labels (curve labels), limb lead color labels (optional), and lead start labels (optional). In addition, for unknown layouts, the number of row blocks will be obtained from the number of grayscale ordinate projection areas of the lead start mark, and the number of column blocks will be obtained from the number of grayscale abscissa projection areas of the lead start mark. See . Figure 5 , mainly including the following steps:

[0102] S1, read the area information of the ECG curve area, if the ECG curve area has only one row of ECG curves, jump to S2, otherwise jump to S18;

[0103] S2, grayscale conversion, projection to the vertical coordinate, to obtain the grayscale vertical axis distribution diagram;

[0104] S3, using the periodic square wave characteristics on the vertical axis, split a series of ECG curves into M image blocks; M is a natural number greater than 1;

[0105] S4, counting the color features that appear;

[0106] S5, the most marked color is the background color;

[0107] S6. Count the grayscale projection of the vertical coordinate. If a certain color has the largest number of periodic square waves, then this color is the color of the grid.

[0108] S7, marking the lead start mark, and determining the lead start mark harness color; specifically, if the coordinate distribution (xy space distribution) of a certain color has only one square wave (including one without tails on both sides and a completely solid middle), and the corresponding horizontal coordinate projection has only one protruding area and the vertical coordinate has M protruding areas, then it is marked as the lead start mark, and the color area is the lead start mark harness color;

[0109] S8. Mark the limb lead color mark and determine the limb lead area color; specifically, if a certain color coordinate distribution (xy space distribution) is approximately a solid rectangular distribution in space (closed on all sides, not a square wave, with other colors inside, the color comes from the lead identifier), it is marked as the limb lead color mark, and the color area is the limb lead area color;

[0110] S9, among the remaining candidate colors, the one with the most colors and immediately after the lead start mark (if any) is marked as the lead text mark;

[0111] S10, displaying the analysis results;

[0112] S11. Manually check, calibrate, and confirm the color classification results;

[0113] S12, analyzing the lead curve from top to bottom, and reading the area where the single-lead electrocardiogram (an image block obtained by splitting) is located;

[0114] S13, reading the color of the electrocardiogram curve in the single-lead electrocardiogram (optional);

[0115] S14, extracting the mark of the electrocardiogram curve;

[0116] S15, ECG curve extraction and refinement;

[0117] S16, record the curve coordinate information to the database, if there is a gap in the curve coordinate, call interpolation and mark it in the background;

[0118] S17, confirm that all single-lead ECGs have been completed, if not, jump to S12;

[0119] S18, column splitting, splitting the ECG curve area into N columns, and executing the logic of each column starting from S2; N is a natural number greater than 1.

[0120] It should be noted that, in the embodiments of the present application, Figure 5 The steps of extracting the ECG curve after step S12 are related steps in the subsequent steps S102 and S103, which will be described in detail later.

[0121] It should be noted that in an embodiment of the present application, in the intelligent layout analysis, as described above, the image processing device can perform row splitting of each column of ECG curves in the following manner: the target column of ECG curves is gray-scale converted and projected onto the vertical axis to obtain a gray-scale vertical axis distribution diagram; the target column of ECG curves is any column of ECG curves in the ECG curve area; and the target column of ECG curves is split into rows using the periodic square wave characteristics of the gray-scale vertical axis distribution diagram on the vertical axis.

[0122] It is understandable that in the embodiments of the present application, row splitting can be performed based on the periodic square wave characteristics of the image on the longitudinal axis. It should be noted that in the embodiments of the present application, the longitudinal axis projection, that is, taking a straight line in this direction, counting the number of black pixels perpendicular to the straight line, and adding up the sum as the projection value of the pixel row. The distribution of the projection values ​​of all pixel rows is displayed as an image to obtain the longitudinal axis projection. The segmentation based on the longitudinal axis projection is to map the image into this feature, and then determine the splitting position based on this feature for splitting. The splitting steps of the longitudinal axis projection mainly include:

[0123] S1, traverse the pixels row by row;

[0124] S2. For each row, traverse the pixel points in the row, count the number of black pixels in the row, accumulate the number of pixels in the row, and store it in the projection array;

[0125] S3, traverse the projection array, if the array value is 0, the pixel row where the array index is located can be split;

[0126] S4. Obtain the row splitting result.

[0127] In an embodiment of the present application, an ECG image includes multiple ECG curves corresponding to multiple leads, and at least one single-lead ECG is obtained from the ECG image by manual layout analysis, including: matching the ECG image with different ECG templates stored in a database; when the ECG image matches any ECG template, calibrating multiple image blocks from the ECG image according to the layout of the ECG template matching the ECG image; or, when the ECG image does not match any ECG template, calibrating multiple image blocks from the ECG image according to the received lead calibration instruction; determining each calibrated image block as a single-lead ECG; each image block includes an ECG curve corresponding to a lead.

[0128] Figure 6 This is a schematic diagram of an exemplary interface for lead layout analysis provided in an embodiment of the present application. Figure 6 , see Figure 7 As shown in the process, manual layout analysis mainly involves the following steps:

[0129] S1. Start the image preprocessing software and perform layout analysis. The software will assist in completing the template calibration. The interface displayed by the software is Figure 4 The interface shown;

[0130] S2, loading ECG image;

[0131] S3, query the database to see if there is an ECG template with matching image features (spectral distribution, etc.); if so, calibrate multiple image blocks from the ECG image according to the layout of the ECG template matching the ECG image, otherwise proceed to step S4;

[0132] S4, the user calibrates the corresponding image block for each lead in the order from top to bottom and from left to right;

[0133] S5. After determining the image blocks corresponding to each lead, the user names the lead according to the lead identifier on the left side of the curve;

[0134] S6. Repeat S4 and S5 until all the calibration and naming are completed.

[0135] It should be noted that in the embodiments of the present application, for the above-mentioned manual layout analysis and intelligent layout analysis, the user can choose the method to be used according to actual needs and application scenarios, and the embodiments of the present application are not limited.

[0136] S102. For each pixel block in the target single-lead electrocardiogram, obtain corresponding color features and position features, and construct corresponding feature components using the corresponding color features and position features; the target single-lead image is any single-lead image, and each pixel block includes at least one pixel.

[0137] In an embodiment of the present application, after obtaining at least one single-lead electrocardiogram, the image processing device can obtain corresponding color features and position features for each pixel block in the target single-lead electrocardiogram, and construct corresponding feature components using the corresponding color features and position features.

[0138] It should be noted that in an embodiment of the present application, the image processing device may divide the target single-lead electrocardiogram into pixel blocks, each pixel block includes at least one pixel. For example, each pixel block includes four pixels adjacent to each other. Each pixel block may also include one pixel, which is not limited in the embodiment of the present application.

[0139] It can be understood that in the embodiments of the present application, the image processing device can obtain the color features and position features of each pixel block in the target single-lead electrocardiogram, and construct its feature components. Specifically, each pixel block includes a pixel. For example, the feature component of pixel i is yi, yi = (ri, gi, bi, pi), wherein the color feature includes a red component ri, a green component gi and a blue component bi, and the position feature is pi.

[0140] S103, clustering pixels of the target single-lead electrocardiogram using the characteristic components corresponding to each pixel block, and extracting an electrocardiogram curve from the target single-lead electrocardiogram according to the clustering result.

[0141] In an embodiment of the present application, after constructing the feature components corresponding to each pixel block, the image processing device can use the feature components corresponding to each pixel block to perform pixel clustering on the target single-lead electrocardiogram, and extract the ECG curve from the target single-lead electrocardiogram according to the clustering results.

[0142] In an embodiment of the present application, an image processing device performs pixel clustering on a target single-lead electrocardiogram using characteristic components corresponding to each pixel block, and extracts an electrocardiogram curve from the target single-lead electrocardiogram according to the clustering result, including: counting the color types contained in the target single-lead electrocardiogram to obtain K colors; K is a natural number greater than 1; using a K-means clustering algorithm to cluster the pixels in the target single-lead electrocardiogram into K sets according to the characteristic components corresponding to each pixel block; each set contains pixels of one color; a set of pixels containing a target color in the K sets is determined as a target set; the target color is the color of the electrocardiogram curve contained in the target single-lead electrocardiogram; a line composed of pixels contained in the target set is extracted from the target single-lead electrocardiogram, and determined as the electrocardiogram curve in the target single-lead electrocardiogram.

[0143] It can be understood that the clustering algorithm is a typical unsupervised learning algorithm used to divide samples with similarity into the same class. The K-means clustering algorithm is one of the representatives of clustering algorithms. It mainly selects K cluster centers by giving the number of cluster categories K to meet the minimum average error criterion function. The K-means clustering algorithm first randomly selects K samples from the data set as the initial cluster centers. Next, for each sample in the data set, the error criterion function between the sample and the cluster center is calculated to divide it into the class where the sample is located when the error criterion function is minimum. After completing the division of each sample, for each category, the cluster center of the class is recalculated, that is, the centroid of all samples belonging to the class. Then, the sample division and cluster center calculation process are repeated until the cluster center no longer changes. Since the K-means clustering algorithm classifies samples based on the similarity of features, and the image is composed of pixels, the segmentation of the image can be regarded as the identification and division of pixels with similar features. Therefore, the K-means clustering algorithm can be used to perform image segmentation operations. For color electrocardiogram, the electrocardiogram curve and the background grid have obvious color differences, and the electrocardiogram curve can be extracted based on the K-means clustering algorithm. The K-means clustering algorithm needs to determine the following three points, including the selection of K value, the selection of initial clustering center, and the selection of clustering features.

[0144] In the embodiment of the present application, regarding the selection of the K value, since a color ECG image is usually mainly composed of an ECG waveform curve, a grid and a background, wherein the ECG waveform curve is mainly black, the background is white, and the grid is red, based on this, the K value can be the number of colors, that is, the K value can be determined to be 3. Of course, the above are only exemplary colors, and the proposed algorithm is also applicable as long as the ECG curve, background color, and grid color are different.

[0145] In the embodiment of the present application, regarding the selection of the initial cluster center, since the K-means clustering algorithm adopts a method of randomly selecting the initial cluster center, the clustering effect will be affected to a certain extent. The present application makes improvements on the method of selecting the initial cluster center. Considering that the clustering object is a pixel, the color feature is an important feature to distinguish the ECG curve from the background and the grid. In addition, the position feature can also be selected as a feature for clustering. Therefore, clustering is performed with the feature component constructed in the above step S102.

[0146] In the embodiment of the present application, in order to improve the extraction effect of the mean algorithm, an enhancement strategy based on Laplace edge detection and erosion is proposed based on edge detection and morphological operations. The image contrast is enhanced by performing convolution and erosion operations on the image, so that the clustering effect is optimized. Taking each pixel block as an example, that is, each pixel is represented by a corresponding feature vector, the K-means clustering algorithm based on Laplace edge detection and erosion enhancement strategy mainly includes the following steps:

[0147] S1, use the 8-neighborhood Laplacian operator with a center of 5 and image convolution to sharpen and enhance the target single-lead ECG;

[0148] S2, using the structural element to perform an erosion operation on the target single-lead electrocardiogram, and the eroded pixels constitute a cluster sample point set A; wherein each pixel in A is represented by its corresponding feature component;

[0149] S3, randomly select c1 from A as the initial cluster center;

[0150] S4, select the next cluster center ci, where c i =x'∈A, its probability is Where D(x) represents the shortest distance from pixel x to the selected cluster center;

[0151] S5, repeat step S2 until K initial cluster centers are determined;

[0152] S6. For any pixel x, calculate the distance to the K initial cluster centers and divide it into the set corresponding to the initial cluster center with the closest distance;

[0153] S7. For any set, recalculate the cluster center of the set;

[0154] S8. Repeat steps S4 and S5 until the cluster center of each set remains unchanged.

[0155] In an embodiment of the present application, after extracting the ECG curve, the image processing device may also reconstruct a background grid for the extracted ECG curve.

[0156] In the embodiment of the present application, the background grid reconstruction is to calculate the time corresponding to each pixel of the horizontal coordinate and the voltage corresponding to each pixel of the vertical coordinate. Since the time and voltage corresponding to each large grid (composed of a certain number of grids) in the electrocardiogram are fixed values, for example, 0.2s and 0.5mV respectively, it is only necessary to calculate the number of pixels contained in each large grid to find the corresponding relationship between the pixel point and the time and voltage. If the screen resolution and horizontal refresh rate are known, the number of pixels Pn contained in each large grid can be directly calculated, and the calculation formula is as follows:

[0157]

[0158] Wherein, DPI is the screen resolution; v is the horizontal refresh rate of the ECG image; and t is the time represented by a large grid.

[0159] In the embodiment of the present application, after extracting the ECG curve, the image processing device may further refine the ECG curve.

[0160] In the embodiment of the present application, for ECG curve extraction, it is first ensured that the extracted ECG curve is composed of single pixels, that is, the horizontal pixels correspond to the vertical pixels one by one, thus ensuring the accuracy of converting pixel data into ECG signal data.

[0161] It is understandable that in the embodiments of the present application, the image skeletonization algorithm deletes the contour pixels of the ordinary image, retains only the skeleton pixels, and forms a binary image connected by a single pixel. The basic principle of the skeleton algorithm is to find the central axis of the target object on the basis of maintaining the geometric shape of the object in the original image, thereby deleting redundant information and retaining only valid information, so that the information of the image is simplified. Finding the central axis of the image is to iterate the pixels of the target area, continuously delete the boundary points and obtain the skeleton of the target. When deleting the boundary points, certain constraints need to be met, including that the endpoints of the straight line cannot be deleted, the region cannot be over-eroded, and the connectivity of the region cannot be destroyed.

[0162] In the embodiments of the present application, the Zhang-Suen curve skeleton refinement algorithm ensures the connectivity of the refined curve and runs at a relatively fast speed. However, the result after refinement by the algorithm cannot ensure that the curve is a single pixel and cannot meet the requirement that a single pixel on the horizontal axis corresponds to a single pixel on the vertical axis when extracting ECG curve data. Therefore, an improvement is made based on the Zhang-Suen skeleton refinement algorithm so that the refined curve is composed of a single pixel.

[0163] In the embodiment of the present application, the improved curve skeleton refinement algorithm also constructs a 3×3 window centered on the pixel P1 on the ECG curve. The pixel with a pixel value of 1 is called a target point, and the pixel with a pixel value of 0 is called a background point. After analyzing the Zhang-Suen algorithm, it was found that the main reason why the refined curve does not satisfy a single pixel is that some pixels that meet other conditions but do not meet A(P1)=1 are omitted. These pixels can be divided into three categories. In the first category of pixels, there are 2 target pixels in the 8 neighborhood pixels of P1, and there are 4 types in total, such as Figure 8(a) to Figure 8(d) As shown in the figure, in the second type of pixel points, there are 3 target pixels among the 8 neighboring pixels of P1, and there are 8 types in total, such as Figure 9(a) to Figure 9(h) As shown, among the third type of pixels, the 8 neighboring pixels of P1 contain 4 target pixels, and there are 4 types in total, such as Figure 10(a) to Figure 10(d) To represent these target pixels, the 8-neighborhood points are binary coded, and the pixel values ​​of each pixel from P2 to P9 are represented by binary numbers. If the pixel value of the point is 1, the corresponding binary bit is also 1, and if the pixel value is 0, the corresponding binary bit is also 0. The 8-neighborhood of Figure 10(a) is converted into Fig.11Binary encoding of the target pixels belonging to the above three categories is performed on the binary encoding and converted into decimal, and the first category target point encoding set is {65,5,20,80}, the second category encoding set is {13,97,22,208,67,88,52,133}, and the third category target point encoding set is {141,99,54,216}. In order to ensure that there are no breakpoints in the curve, the three categories of target point sets should be properly screened so that the refined curve can satisfy a single pixel and ensure the connectivity of the curve. After screening, the first category target point encoding set is {65,5,20,80}, the second category encoding set is {13,22,52,133}, and the third category target point encoding set is {141,54}. The total set of deleted points is {65,5,20,80,13,22,52,133,141,54}.

[0164] In the embodiment of the present application, the steps of the improved Zhang-Suen curve skeleton thinning algorithm are as follows:

[0165] S1. Determine the eight neighboring pixels of P1 and mark the pixels that meet the following conditions:

[0166] (a)2≤B(P 1 )≤6;

[0167] (b)A(P 1 )=1aorB(P 1 )∈{65,5,20,80,13,22,52,133,141,54};

[0168] (c)P 2 ×P 4 ×P 6 =0;

[0169] (d)P 4 ×P 6 ×P 8 =0;

[0170] Among them, B(P 1 ) is P 1 The 8-neighborhood pixel binary code is

[0171] S2, same as S1 above, except that condition (c) is changed to P 2 ×P 4 ×P 8 =0, condition (d) is changed to P 2 ×P 6 ×P 8 =0;

[0172] S2: Repeat the above steps S1 and S2 until there are no pixels that meet the conditions.

[0173] In an embodiment of the present application, after extracting the ECG curve, the image processing device can also perform ECG data extraction. ECG data extraction is the process of converting pixel information stored in the image into ECG signals. The specific operation is to traverse the ECG curve column by column, and record the row coordinates of the first pixel value of 0 on each column. Since the image coordinate origin is the upper left corner, and the extracted ECG signal coordinate origin is the lower left corner, the recorded row coordinates need to be converted, that is, the row coordinate value is subtracted from the number of pixels in the image row. Then, according to the corresponding relationship between pixels and time-voltage, the recorded curve pixel value is converted, and the ECG curve information is converted into ECG signal data for storage.

[0174] In the embodiment of the present application, for the electrocardiogram image, the image processing device can not only perform electrocardiogram curve extraction according to the above S101 to S103, but also perform character recognition, which is described in detail below. Fig.12 The electrocardiogram image shown is used to identify the characters in the selected part.

[0175] In an embodiment of the present application, the character recognition method mainly includes the following steps: obtaining a text information area from an electrocardiogram image; using different text recognition engines to respectively identify the characters contained in the text information area to obtain multiple character sequences; each character sequence is recognized by a text recognition engine; aligning the multiple character sequences and comparing them, and determining inconsistent characters as erroneous characters; correcting the erroneous characters to determine the correct character sequence corresponding to the text information area.

[0176] It should be noted that, in the embodiments of the present application, obtaining a text information area from an electrocardiogram includes: calibrating a text information area from the electrocardiogram according to a layout of an electrocardiogram template matching the electrocardiogram; or calibrating a text information area from the electrocardiogram according to a received character calibration instruction; or, when edge features of the electrocardiogram are calculated, obtaining a text information area from the electrocardiogram based on the edge features. The specific method of obtaining the text information area is not limited in the embodiments of the present application.

[0177] It is understandable that after deep learning was applied to the field of text recognition, the character recognition rate has increased year by year, but in recent years it has clearly reached a bottleneck. The recognition rate for document images with good image quality is as high as 98%. It is a very challenging task to optimize a single text recognition engine on this basis to obtain a higher recognition rate. A text recognition model and two commercial text recognition engines were used to form multiple text recognition engines. The recognition results of each text recognition engine were first aligned through multi-sequence alignment. Then, the inconsistencies were used as possible erroneous characters, that is, wrong characters. Then, for each character, a candidate character set was constructed based on the principles of similar shape and sound. Finally, the best character was selected through the hidden Markov model to complete the error correction. In the hidden Markov model, this paper takes the recognition result of the text recognition engine as the observation state and the corresponding correct character as the hidden state. The error correction problem is converted into the problem of solving the most likely hidden state given the observation state. The entire error correction process is as follows: Fig.13 shown.

[0178] In the embodiments of the present application, text error correction generally includes two stages: error detection and error correction. Traditional error detection often uses statistical language models such as N-Gram, which regards adjacent words with too low co-occurrence probability as erroneous characters. However, since this scheme only uses statistical information in the corpus and does not consider the semantics and syntax of the sentence itself, its detection effect often does not meet people's expectations. Based on this situation, the embodiments of the present application propose a recognition result alignment error detection scheme based on multiple text recognition engines, which uses the similarity of biological base pairs to determine the best alignment of multiple sequences, and uses inconsistent characters after alignment as erroneous characters.

[0179] In the embodiment of the present application, due to the different model structures of multiple text recognition engines, it is difficult for a certain character to be recognized as another identical wrong character at the same time, because the recognition rate of each text recognition engine itself is very high, and even if there is a recognition error, it may be a variety of errors. Assume that pA(i), pB(i) and pC(i) are the probabilities that text recognition engines A, B, and C recognize a certain character in the character set D as character i. Then the probability that the three text recognition engines recognize a certain character u as the same wrong character at the same time is pA, B, C (iwrong) as shown in the following formula:

[0180] p A,B,C (i wrong )=∑ i∈D,i≠u p A(i) p B(i) p C(i) (2)

[0181] In the embodiment of the present application, the recognition rate of the selected text recognition engines exceeds 95%. Assuming that they are completely independent, then in theory, the probability of three text recognitions causing simultaneous recognition errors is less than 0.0125%, and the probability of simultaneously recognizing the same wrong character is much lower than this value. In fact, when testing on the data set mentioned above, it was found that the three text recognition engines simultaneously recognized a character as another wrong character, accounting for only 1.5% of all the wrong recognition cases, and most of them were words that were not entered in the dictionary. For this reason, the embodiment of the present application uses the inconsistent places after the recognition results of multiple different text recognition engines are aligned as the places where the recognition errors are made, and uses this as the basis for the next step of error correction.

[0182] In the embodiments of the present application, the recognition output alignment algorithm adopted draws on the Needleman-Wunsch algorithm in the field of biology, and is expanded to multiple sequence alignment on this basis. The algorithm can be divided into three steps: (1) pairwise alignment. The recognition results of the text recognition engine are aligned pairwise and a distance matrix is ​​constructed; (2) a guide tree is generated. The guide tree of all recognition results is generated according to the distance matrix using the adjacency method; (3) progressive alignment. According to the degree of distance between each result in the guide tree, the recognition results with high similarity are compared first, and then the result sequences with similar distances are gradually introduced and the alignment is continuously updated until all text recognition engine results are aligned.

[0183] In the embodiments of the present application, the Needleman-Wunsch algorithm is used to perform pairwise comparison of the recognition results of the text recognition engine. The algorithm can be divided into two major steps: (1) First, the similarity between the two character sequences is calculated, and the distance and similarity between each character and another character are calculated in turn, and all the results are saved in a two-dimensional array; (2) The maximum score of the two-dimensional array from the lower right corner to the upper left corner is calculated according to the backtracking method. It can be seen that Needleman-Wunsch is an algorithm that considers the comparison from a global perspective, that is, it can align two text sequences at the minimum cost. The specific process of the algorithm is introduced below.

[0184] Assume that the recognition results of two text recognition engines, that is, the recognized character sequences are SeqA and SeqB, and SeqA = {s1, s2, ..., sn}, SeqB = {t1, t2, ..., tm}. It should be noted that the lengths of SeqA and SeqB are n and m, respectively, and n and m are not necessarily equal. First, formulate a scoring rule, that is, different scores are obtained for equal characters, unequal characters, and when placeholders are inserted, as shown in the following formula:

[0185]

[0186] Among them, k1, k2 and k3 are the scores of equal characters, unequal characters and inserted placeholders, respectively, and "_" represents the placeholder. Next, the entire two-dimensional array M needs to be filled in sequence according to the boundary scores, as shown in the following formula:

[0187]

[0188] Among them, M i,j Represents the score of the two-dimensional array M at position (i, j), which can be reached by three paths, namely M i-1,j-1 、M i-1,j and M i,j-1 The meanings of the three paths are: (1) From M i-1,j-1 The line arrives, that is, neither side adds a placeholder at this time, and the score is C (s i ,t j ); (2) from M i-1,j When a vertical line arrives, SeqB adds a placeholder; (3) From M i,j-1 When a horizontal line is reached, a placeholder is added to SeqA.

[0189] Finally, the optimal path from the lower right corner to the upper left corner of the two-dimensional matrix is ​​calculated, that is, the path with the highest score. This process is a backtracking process, and there are two basic principles: if the characters of the current two sequences are equal, that is, s i =t j , then backtrack to (i-1,j-1); if the characters are not equal, backtrack in the direction with the largest value among (i-1,j-1), (i-1,j) and (i,j-1) until reaching the upper left corner (0,0). The final path is the optimal path.

[0190] After obtaining the pairwise comparison results and the distance matrix, the next step is to build a guide tree to determine the order of subsequent comparisons. It is generally believed that character sequences with close distances have higher homology and higher similarity, so the two closest text sequences can be selected as the starting root sequences for comparison based on the distance matrix, and then character sequences with longer distances can be added in turn until all character sequences are compared.

[0191] In the error detection model of the embodiment of the present application, there are only three recognition result sequences, so the guide tree must first compare the two character sequences with closer sequences, and then compare the third character sequence, and finally obtain the result comparison of the three text recognition engines.

[0192] In an embodiment of the present application, the image processing device corrects incorrect characters and determines the correct character sequence of the text information area, including: constructing a candidate character set for the incorrect characters based on the principles of similar shapes and similar sounds; constructing a hidden Markov model with any character sequence among multiple character sequences as the observed state and the correct character sequence as the hidden state; using the hidden Markov model to select the correct characters corresponding to the incorrect characters from the candidate character set for character sequence update to obtain the correct character sequence.

[0193] In an embodiment of the present application, by aligning the character sequences output by the recognition of multiple text engines, inconsistent positions can be obtained and used as the places that need to be corrected. Here, a candidate character set needs to be constructed for the incorrect characters first, and it is ensured that the corresponding correct characters are as likely as possible in the candidate character set. Then, the error correction only needs to be selected from the candidate character set. The candidate character set should theoretically contain all Chinese character dictionaries, numbers, letters, and various symbols. However, in practice, for the sake of improving efficiency and accuracy, generally only several of the most likely characters are selected as its candidate character set.

[0194] The construction of the candidate character set mainly involves two features: pronunciation and glyph. The pronunciation of each Chinese character includes an initial consonant and a final vowel. For a certain Chinese character A, assuming its initial consonant and final vowel are iA and vA respectively, and correspondingly, the initial consonant and final vowel of character B are iB and vB respectively, then the pronunciation similarity PA,B of A and B can be calculated according to the following formula:

[0195]

[0196] In an embodiment of the present application, the glyph structure, four-corner code, and number of strokes are selected as the judgment features for the glyph similarity degree. The glyph structure is mainly classified according to the font structure presented by the Chinese character itself. For example, "司" can be regarded as a right-upper enclosed structure, and "房" can be regarded as a left-upper enclosed structure.

[0197] The four-corner code uses the numbers 0 to 9 to represent ten pen shapes at the four corners of a Chinese character. According to the strokes of the upper left, upper right, lower left, and lower right of the character in turn, four-digit numbers can be obtained, so it is called the four-corner code.

[0198] Its coding rule is:

[0199] Horizontal is 1, vertical is 2, dot and press are 3, cross is 4, insertion is 5, square is 6; 7 is for corners, 8 and 9 are for small, and when there is a horizontal under the dot, it becomes 0

[0200] Then the glyph similarity can be calculated according to the following formula:

[0201] F A，B =W s S A，B +W C C A，B +WE E A，B

[0202] Among them, FA,B is the similarity of the glyphs, SA,B, CA,B and EA,B are the structural similarity, four-corner similarity and stroke similarity respectively, and the corresponding WS, WC and WE are the structural similarity weight, four-corner similarity weight and stroke similarity weight respectively, satisfying 0≤WS,WC,WE≤1,WS+WC+WE=1.

[0203] Finally, the candidate character set is constructed based on pronunciation and glyph. The construction process is: for each character, the entire character library is traversed, and then a similarity score SimA, B is obtained by weighted summation based on the pronunciation and glyph similarity, that is:

[0204] Sim A，B =W P P A，B +W F F A，B (6)

[0205] Among them, 0≤WP, WF≤1, WP+WF=1. When SimA, B exceed the set threshold, it is determined that the two characters are candidate characters for each other.

[0206] After all the character sequences recognized by the text recognition engines are completed through recognition output alignment error detection and candidate character set construction, it is necessary to select the best result from the candidate characters corresponding to the error position through the error correction model. In the embodiment of the present application, the recognized character sequence is used as the observation state, and the corresponding correct character sequence is used as the hidden state, and the text error correction problem is modeled as the problem of finding the most likely hidden state sequence under a given observation state sequence. It should be noted that in order to simplify the model here, only the case where the length of the hidden state and the observation state are equal is considered, that is, the hidden state of an empty character or multiple characters corresponding to a certain observation state is not considered, that is, the errors of multiple words and fewer words occurring in text recognition errors are not considered.

[0207] Assume that a recognition result is a character sequence O of length T, and its corresponding correct sequence is I, where I = (i1, i2, ..., iT), O = (o1, o2, ..., oT). V is a set containing all possible recognition result character sequences, and Q is a set containing all corresponding correct characters. That is, V = {v1, v2, ..., vM}, Q = {q1, q2, ..., qN}.

[0208] Let A represent the probability matrix of the correct character transferring to the next correct character: A = [a ij ] N×N ,in:

[0209] a ij=P(i t+1 =q j |i t =q i ) (7)

[0210] Here 1≤i≤N, 1≤j≤N. aij represents the probability that if the correct character at time t is qi, then at time t+1 its correct character will be converted to qj. It should be noted that since there may be reduplication in the character sequence: for example, "binbinyouli" and "youxinchongchong", that is, there is a situation where qi is transferred to qi itself, so i and j may be equal. Let B represent the emission probability matrix: that is, B=[b j (k)] N×M ,in:

[0211] b j (k) = P(o t =v k |i t =q j ) (8)

[0212] Here 1≤k≤M, 1≤j≤N. j (k) represents the probability that if the correct character at time t is qj, then it is recognized as vk at the corresponding time. π represents the initial correct character probability vector, that is, π = π u ),in:

[0213] π u =P(i 1 =q u ) (9)

[0214] Here 1≤u≤N. u Indicates that at the initial time t = 1 the system is in the correct character state q u Finally, the hidden Markov model can be expressed as follows:

[0215] λ=(A,B,π) (10)

[0216] In the embodiment of the present application, the text error correction problem is transformed into the following problem: direct error correction: directly searching for the most likely corresponding correct character sequence given a recognition result sequence.

[0217] First, two N×T 2D matrices T1 and T2 are defined, where T1[k, t] stores the most likely path X=(x 1 , x 2 ,......,x t ), and the correct character state x at time t t =i kThe probability of. And T2[k, t] represents the most likely path from time 1 to time t, X = (x 1 , x 2 ,......,x t-1 , x t =i k ) at time t-1, the correct character state x t-1 ,Right now:

[0218]

[0219]

[0220] The Viterbi algorithm is essentially a dynamic programming algorithm, and its calculation process is as follows:

[0221] Input:

[0222] observationspace: B={v1, v2,..., vM}

[0223] statespace: Q={q1, q2,..., qN}

[0224] initialstateprobabilities: Π=(π1, π2,..., πN)

[0225] observationssquence:O=(o1,o2,...,oT)

[0226] transitionmatrix: A = [a ij ] N×N

[0227] emissionmatrix: B = [b j (k)] N×M

[0228] Output:

[0229] Mostlikelyhiddenstatesequence:X=(x 1 , x 2 , ..., x t )

[0230] STEP1: Initial:

[0231] foreachobservationi=1,2,...,Ndo

[0232]

[0233] T2[k,1]←0

[0234] endfor

[0235] STEP2: Traverse:

[0236] foreachobservationi=2,3,...,Ndoforeachstate

[0237] k=1,2,...,Ndo

[0238]

[0239]

[0240] STEP3: Output:

[0241] zT←argmax u (T 1 [u,T])

[0242] xT←sz

[0243] fort=T,T-1,...,2do

[0244] zt-1←T2[zt,t]

[0245] xt-1←SZ

[0246] endfor

[0247] In summary, the most likely correct character can be calculated given a recognition result sequence and suspected error positions, or the most likely correct sequence corresponding to a given recognition text sequence can be directly calculated by the Viterbi algorithm without the need for prior knowledge of the error detection position.

[0248] The image processing method provided in the embodiment of the present application, first, supports the recognition of text information in the ECG image, extracts the calibration voltage information and layout information, and then, based on the layout information, first splits the columns and then splits the rows, obtains the curve information of each ECG lead one by one, and matches and associates them with the corresponding lead identifiers. For the scenario where the multi-lead layout cannot be directly obtained from the ECG display, a strategy for extracting the number of block rows and columns from the grayscale projection features of the lead mark is provided to solve the layout analysis; secondly, a layout analysis method based on prior knowledge for image and text recognition is provided, which guides users to mark and store the display layout of the ECG instruments mainly used by medical institutions. The existing layout template can be used to explicitly distinguish the ECG curve area, the lead character area, and the calibration voltage area, where the stored content can be indexed and distinguished according to the brand-model-interface hierarchy; thirdly, an ECG curve extraction method is provided, which utilizes the Laplace edge detection and corrosion enhancement strategy, and finally applies the mean curve to extract the curve. When using k-means clustering, the ECG pixels are divided into three categories according to color and position features, and the features selected during clustering include color features and position features; fourthly, using the strategy of collective wisdom, multiple OCR recognition engines are integrated. When the character sequences recognized by multiple text recognition engines are inconsistent, multiple sequence comparison results are given, and then the hidden Markov model is used to perform text error correction.

[0249] The image processing method provided in the embodiment of the present application has the following advantages: a general solution is proposed for the problem of different standards for displaying multi-lead ECG images by different manufacturers; in the manual layout analysis solution, the problem of different standards for displaying multi-lead ECG images by different manufacturers is not only applicable to the scene of ECG layout text extraction, but also to the scene of pure ECG curve display, and can also adapt to the scene of ECG curve intersection; a text error correction model based on hidden Markov is established. Through multiple sequence alignment to detect possible errors in the recognition results, a candidate character set is constructed for each erroneous character based on the principle of similar shape and similar sound, and finally the best character is selected through the hidden Markov model to complete the error correction. The recognition result sequences of multiple text recognition engines are used as observation states, and the corresponding correct character sequences are used as hidden states. They are modeled into a hidden Markov model, and the error correction problem is converted into the problem of finding the most likely corresponding correct character sequence given the recognition result sequence. The problem is then solved through the backtracking algorithm, and finally a more accurate recognition result is obtained than that of a single text recognition engine. There is no need to upgrade the existing equipment in the hospital, and the access to various medical equipment is maximized to promote the implementation of smart hospitals. The coupling between systems is reduced, and visible image acquisition is used to facilitate the access of general ECG equipment data on ambulances. Medical ECG equipment that is directly accessed through video signals / indirectly collected by cameras can digitize information and transmit it to the server with low bandwidth, which is especially convenient for pre-hospital emergency scenarios, such as weak network transmission and other network environments.

[0250] The present application provides an image processing device. Fig.14 Schematic diagram of the structure of an image processing device provided in an embodiment of the present application. Fig.14 As shown, the image processing device includes:

[0251] The acquisition module 201 is used to acquire at least one single-lead electrocardiogram from the electrocardiogram image; each single-lead electrocardiogram includes an electrocardiogram curve corresponding to one lead;

[0252] A construction module 202 is used to obtain corresponding color features and position features for each pixel block in the target single-lead electrocardiogram, and construct corresponding feature components using the corresponding color features and position features; the target single-lead image is any single-lead image, and each pixel block includes at least one pixel;

[0253] The extraction module 203 is used to perform pixel clustering on the target single-lead electrocardiogram using the characteristic components corresponding to each pixel block, and extract an electrocardiogram curve from the target single-lead electrocardiogram according to the clustering result.

[0254] In one embodiment of the present application, the ECG image includes multiple ECG curves corresponding to multiple leads, and the acquisition module 201 is used to calculate the edge features of the ECG image, and based on the edge features, determine the ECG curve area from the ECG image; the ECG curve area includes the multiple ECG curves; identify the layout form of the multiple ECG curves from the ECG curve area; when the layout form is multiple rows and columns, the ECG curve area is split into columns, and each column of ECG curves is split into rows, and each image block obtained by the split is determined as a single-lead ECG; when the layout form is multiple rows and a single column, a column of ECG curves in the ECG curve area is split into rows, and each image block obtained by the split is determined as a single-lead ECG; wherein each image block includes an ECG curve corresponding to a lead.

[0255] In one embodiment of the present application, the acquisition module 201 is used to convert the grayscale of the target column ECG curve and project it onto the vertical axis to obtain a grayscale vertical axis distribution diagram; the target column ECG curve is any column of ECG curves in the ECG curve area; and the target column ECG curve is split into rows using the periodic square wave characteristics of the grayscale vertical axis distribution diagram on the vertical axis.

[0256] In one embodiment of the present application, the ECG image includes multiple ECG curves corresponding to multiple leads, and the acquisition module 201 is used to match the ECG image with different ECG templates stored in a database; when the ECG image matches any ECG template, multiple image blocks are calibrated from the ECG image according to the layout of the ECG template matching the ECG image; or, when the ECG image does not match any ECG template, multiple image blocks are calibrated from the ECG image according to the received lead calibration instruction; each image block includes an ECG curve corresponding to a lead; and each calibrated image block is determined as a single-lead ECG.

[0257] In one embodiment of the present application, the extraction module 203 is used to count the color types contained in the target single-lead electrocardiogram to obtain K colors; K is a natural number greater than 1; using the K-means clustering algorithm, the pixels in the target single-lead electrocardiogram are clustered into K sets according to the feature components corresponding to each pixel block; each set contains pixels of one color; the set of pixels containing the target color in the K sets is determined as the target set; the target color is the color of the electrocardiogram curve in the target single-lead electrocardiogram; a line composed of pixels contained in the target set is extracted from the target single-lead electrocardiogram and determined as the electrocardiogram curve in the target single-lead electrocardiogram.

[0258] In one embodiment of the present application, the extraction module 203 is also used to obtain a text information area from the electrocardiogram image; use different text recognition engines to respectively identify the characters contained in the text information area to obtain multiple character sequences; each character sequence is recognized by a text recognition engine; align the multiple character sequences and compare them, and determine inconsistent characters as erroneous characters; correct the erroneous characters to determine the correct character sequence corresponding to the text information area.

[0259] In one embodiment of the present application, the extraction module 203 is used to mark the text information area from the ECG image according to the layout of the ECG template matching the ECG image; or, to mark the text information area from the ECG image according to a received character marking instruction; or, when the edge features of the ECG image are calculated, to obtain the text information area from the ECG image based on the edge features.

[0260] In one embodiment of the present application, the extraction module 203 is used to construct a candidate character set for the erroneous character based on the principles of similar shape and similar sound; to construct a hidden Markov model with any character sequence among the multiple character sequences as the observed state and the correct character sequence as the hidden state; and to use the hidden Markov model to select the correct character corresponding to the erroneous character from the candidate character set to update the character sequence and obtain the correct character sequence.

[0261] It should be noted that, in the embodiments of the present application, the image processing device may also divide its structure in other forms. Fig.15 A schematic diagram of the structure of an image processing device provided in an embodiment of the present application Figure 2 .like Fig.15 As shown, the structure of the image processing device can also be composed of a data management module 301, an image processing module 302, and an information display module 303. The data management module 301 implements the management of the ECG image data and the extracted ECG signal data. The image processing module 302 is a core module, which mainly implements the function of extracting ECG data from the ECG image, and the information display module 303 implements the graphic display of the ECG signal, the text information display, etc. The relevant steps are not repeated here.

[0262] The present application embodiment provides an electronic device, Fig.16 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown in FIG. Fig.16 As shown, the electronic device includes: a processor 401, a memory 402 and a communication bus 403;

[0263] The communication bus 403 is used to realize the communication connection between the processor 401 and the memory 402;

[0264] The processor 401 is used to execute one or more programs stored in the memory 402 to implement the above-mentioned image processing method.

[0265] The embodiment of the present application provides a computer-readable storage medium on which a computer program is stored, and the computer program implements the above-mentioned image processing method when executed by a processor. The computer-readable storage medium can be a volatile memory (volatile memory), such as a random access memory (Random-Access Memory, RAM); or a non-volatile memory (non-volatile memory), such as a read-only memory (Read-Only Memory, ROM), a flash memory (flash memory), a hard disk (Hard Disk Drive, HDD) or a solid-state drive (Solid-State Drive, SSD); or it can be a respective device including one or any combination of the above memories, such as a mobile phone, a computer, a tablet device, a personal digital assistant, etc.

[0266] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of hardware embodiments, software embodiments, or embodiments in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) that contain computer-usable program code.

[0267] The present application is described with reference to implementation flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the process in the flowchart. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0268] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which is implemented in the implementation flow diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0269] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process described in the flowchart. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0270] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by any technician familiar with the technical field within the technical scope disclosed in the utility application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. An image processing method, characterized in that: include: Acquire at least one single-lead electrocardiogram from the electrocardiogram image; each single-lead electrocardiogram includes an electrocardiogram curve corresponding to one lead; For each pixel block in the target single-lead electrocardiogram, a corresponding color feature and a position feature are obtained, and a corresponding feature component is constructed using the corresponding color feature and the position feature; the target single-lead image is any single-lead image, and each pixel block includes at least one pixel; The target single-lead electrocardiogram is pixel-clustered using the characteristic components corresponding to each pixel block, and an electrocardiogram curve is extracted from the target single-lead electrocardiogram according to the clustering result.

2. The method according to claim 1, characterized in that The electrocardiogram image includes a plurality of electrocardiogram curves corresponding to a plurality of leads, and obtaining at least one single-lead electrocardiogram from the electrocardiogram image includes: Calculating edge features of the electrocardiogram image, and determining an electrocardiogram curve region from the electrocardiogram image based on the edge features; the electrocardiogram curve region includes the multiple electrocardiogram curves; identifying a layout form of the plurality of electrocardiogram curves from the electrocardiogram curve region; In the case where the layout form is multiple rows and multiple columns, the electrocardiogram region is split into columns, and each column of the electrocardiogram obtained is split into rows, and each image block obtained by the split is determined as a single-lead electrocardiogram; When the layout is a multi-row single column, a column of ECG curves in the ECG curve area is split into rows, and each image block obtained by the split is determined as a single-lead ECG; wherein each image block contains an ECG curve corresponding to a lead.

3. The method according to claim 2, characterized in that The method further comprises: After grayscale conversion of the target column of ECG curves, the target column of ECG curves is projected onto the vertical axis to obtain a grayscale vertical axis distribution diagram; the target column of ECG curves is any column of ECG curves in the ECG curve area; The target column electrocardiogram curve is split into rows by utilizing the periodic square wave characteristics of the grayscale vertical axis distribution diagram on the vertical axis.

4. The method according to claim 1, characterized in that: The electrocardiogram image includes a plurality of electrocardiogram curves corresponding to a plurality of leads, and obtaining at least one single-lead electrocardiogram from the electrocardiogram image includes: matching the electrocardiogram image with different electrocardiogram templates stored in a database; In the case where the electrocardiogram image matches any electrocardiogram template, a plurality of image blocks are calibrated from the electrocardiogram image according to the layout of the electrocardiogram template that matches the electrocardiogram image; or, in the case where the electrocardiogram image does not match any electrocardiogram template, a plurality of image blocks are calibrated from the electrocardiogram image according to the received lead calibration instruction; wherein each image block includes an electrocardiogram curve corresponding to a lead; Each calibrated image block is determined as a single-lead electrocardiogram.

5. The method according to claim 1, characterized in that: The target single-lead electrocardiogram is pixel-clustered using the characteristic components corresponding to each pixel block, and an electrocardiogram curve is extracted from the target single-lead electrocardiogram according to the clustering result, including: Counting the color types included in the target single-lead electrocardiogram to obtain K colors; K is a natural number greater than 1; Using a K-means clustering algorithm, the pixels in the target single-lead electrocardiogram are clustered into K sets according to the characteristic components corresponding to each pixel block; each set contains pixels of one color; Determine a set of pixels containing a target color in the K sets as a target set; the target color is the color of the electrocardiogram curve of the target single-lead electrocardiogram; A line consisting of pixels included in the target set is extracted from the target single-lead electrocardiogram and determined as an electrocardiogram curve in the target single-lead electrocardiogram.

6. The method according to claim 1, characterized in that The method further comprises: Acquire a text information region from the electrocardiogram; Using different text recognition engines to respectively recognize the characters contained in the text information area to obtain multiple character sequences; each character sequence is recognized by one text recognition engine; Aligning the multiple character sequences and comparing them, and determining inconsistent characters as erroneous characters; Correct the erroneous characters to determine a correct character sequence corresponding to the text information area.

7. The method according to claim 6, characterized in that The step of obtaining a text information area from the electrocardiogram image includes: Demarcating the text information area from the electrocardiogram according to the layout of the electrocardiogram template matching the electrocardiogram; Alternatively, according to the received character marking instruction, marking the text information area from the electrocardiogram image; Alternatively, when the edge feature of the electrocardiogram is calculated, the text information region is acquired from the electrocardiogram based on the edge feature.

8. The method according to claim 6, characterized in that The step of correcting the erroneous characters to determine a correct character sequence in the text information area includes: Constructing a candidate character set for the erroneous characters based on the principle of similar shape and similar sound; Taking any character sequence among the multiple character sequences as an observed state and the correct character sequence as a hidden state, a hidden Markov model is constructed; The hidden Markov model is used to select the correct character corresponding to the wrong character from the candidate character set to update the character sequence, thereby obtaining the correct character sequence.

9. An image processing device, characterized in that: include: An acquisition module, used for acquiring at least one single-lead electrocardiogram from the electrocardiogram image; each single-lead electrocardiogram includes an electrocardiogram curve corresponding to one lead; A construction module, for acquiring corresponding color features and position features for each pixel block in a target single-lead electrocardiogram, and constructing corresponding feature components using the corresponding color features and position features; the target single-lead image is any single-lead image, and each pixel block includes at least one pixel; The extraction module is used to perform pixel clustering on the target single-lead electrocardiogram using the characteristic components corresponding to each pixel block, and extract an electrocardiogram curve from the target single-lead electrocardiogram according to the clustering result.

10. An electronic device, characterized in that: include: processor, memory, and communication bus; The communication bus is used to realize the communication connection between the processor and the memory; The processor is used to execute one or more programs stored in the memory to implement the image processing method according to any one of claims 1 to 8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the image processing method according to any one of claims 1 to 8 is implemented.

Images