A table image recognition method
By combining image preprocessing, YOLO, and CRNN with the Shi-Tomasi algorithm to identify tabular images in nuclear power plant design drawings, the problem of low efficiency in manual modeling was solved, and efficient automatic modeling conversion was achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHINA NUCLEAR POWER OPERATION TECH CORP
- Filing Date
- 2022-11-28
- Publication Date
- 2026-06-09
AI Technical Summary
In the development of a full-range nuclear power plant simulator, manual modeling is inefficient and has a high error rate, making it difficult to automatically convert design drawings into simulation models, especially for the automatic recognition of tables and images.
By combining image preprocessing, YOLO algorithm, and CRNN text recognition model with Shi-Tomasi detection algorithm, text, lines, inflection points, and frames in table images are identified. Through data augmentation training, the model is trained to achieve automatic recognition and reconstruction of table images.
It improves the efficiency and quality of automatic modeling from design drawings to simulation models, and realizes the automatic recognition and conversion of table images.
Smart Images
Figure CN115797953B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of image processing technology, and specifically relates to a method for recognizing table images. Background Technology
[0002] The development of a full-scale nuclear power plant simulator requires simulation modeling of hundreds of power plant process systems, hundreds of power distribution systems, thousands of DCS control logic diagrams, and various styles of process flow diagrams, teaching control diagrams, and control panel diagrams. Manual modeling is inefficient and prone to errors, a key factor affecting simulator development efficiency. Therefore, there is an urgent need to implement automated modeling methods to automatically convert design drawings into simulation models, thereby improving modeling efficiency and quality.
[0003] Design drawings typically also contain tables to store information such as component icon numbers / notes in the flowchart, or parameter information required for modeling. To achieve automated modeling, these tables need to be accurately identified. Summary of the Invention
[0004] The purpose of this invention is to provide a table image recognition method to solve the problem of automatic recognition of tables in design drawings.
[0005] The technical solution of the present invention is as follows: a table image recognition method, the method specifically includes:
[0006] S1. Acquire the target table image;
[0007] S2. Identify and obtain the text and text location information in the target table image;
[0008] S3. Identify and obtain the position information of the connecting lines in the target table image;
[0009] S4. Based on the position information of the connecting line, obtain the position information of the inflection point in the target table image;
[0010] S4.1 Based on the connection position information, identify the position information of corner points in the target table image;
[0011] S4.2 Based on the corner position information, identify the inflection points of the connecting lines in the target table image;
[0012] If at least two lines are connected to the corner point and both are straight lines, then the corner point is determined to be an inflection point.
[0013] S5. Based on the position information of the connecting lines and inflection points obtained by identification, the table frame in the target image is identified.
[0014] The specific steps for identifying and obtaining the table frame in the target image in step S5 are as follows:
[0015] S5.1 Identify and obtain the connection and inflection point position information, identify and obtain the connection relationship in the target table image, and determine whether there is a connection relationship between two adjacent inflection points based on whether there is a connection between them that meets the predetermined conditions.
[0016] S5.2 By recognizing the inflection points and the connections between them in the target table image, the table frame in the target table image can be obtained.
[0017] In step 4.2, identifying the inflection points of the connecting lines in the target table image can be done by detecting the right-angled sides when the connecting lines are horizontal or vertical. The specific steps are as follows:
[0018] Based on the identified connection and corner position information, it is determined whether the connection lines connecting the corner points in the target table image form right angles. If the connection lines connecting the corner points form right angles, the corner points are identified as inflection points.
[0019] The specific steps for detecting right angles are as follows:
[0020] For any corner point, a rectangular target area is determined with the corner point as the center. The central area of the rectangle is determined at the center of the target area. The areas above and below the central area within the target area are determined as the vertical detection area, and the areas to the left and right of the central area within the target area are determined as the horizontal detection area. When there is a vertical line in the vertical detection area and the length of the vertical line is approximately equal to the height of the vertical detection area, and there is a horizontal line in the horizontal detection area and the length of the horizontal line is approximately equal to the width of the horizontal detection area, it is determined that the line connecting to the corresponding corner point forms a right angle.
[0021] The specific steps for identifying and obtaining text and text location information in the target table image in step S2 are as follows:
[0022] S2.1 Perform image preprocessing on the acquired table images;
[0023] S2.1.1 Perform image grayscale processing on the acquired image to be identified;
[0024] S2.1.2 Filter the grayscale image to reduce image noise;
[0025] S2.1.3 Binarize the filtered image;
[0026] S2.2 Perform target detection on the text region in the table image;
[0027] S2.3 Collects and recognizes the text in the table image.
[0028] In step S2.2, target detection in the text region of the table image is performed by using the YOLO algorithm to obtain boundary coordinates and class probabilities from image pixels, and to confirm and identify the positions of all targets in the table image.
[0029] In step 2.3, the text in the table image is collected and recognized using the CRNN text recognition model. The CRNN text recognition model is trained using a pre-trained model and a constructed data sample set to obtain the corresponding text recognition model.
[0030] The sample data used by the CRNN text recognition model is obtained using data augmentation methods, which can be obtained using any one or a combination of the following methods:
[0031] The original sample data is augmented by inverting, flipping, rotating, and scaling the image.
[0032] Salt-and-pepper noise, Gaussian noise, image blur noise, and image erosion noise are randomly added to the sample images to form new sample data.
[0033] For images containing numbers, letters, and standard symbols that are originally image data samples, a template set of standard characters is established, and a large number of sample images are randomly generated using sample synthesis.
[0034] When recognizing text in the table image, it is necessary to detect the text direction corresponding to each text region, and any one of the following methods can be selected:
[0035] The detection is performed using a binary classification model, which divides horizontal and vertical text into two categories of samples. The binary classification model is trained to determine the direction of the text, and the text direction is then corrected.
[0036] The system performs rule-based detection, determining the direction of the text based on the length and width of the text region.
[0037] After identifying the table frame in the target table image in step S5, the process further includes reconstructing the table based on the identified table frame and filling in the identified text at the corresponding positions in the table, thereby converting the target table image into an editable table format file.
[0038] The specific steps for step S3, which involves identifying and obtaining the position information of the connecting lines in the target table image, are as follows:
[0039] The lines in the target table image are identified as a separate type of element, and the position information of the lines is finally output. After binarizing the target table image, the number of consecutive pixels is determined to be continuous by detecting whether the average pixel value of a consecutive number of pixels is less than a predetermined pixel threshold.
[0040] The significant advantage of this invention is that the table image recognition method described in this invention can automatically recognize the table frame and text in the target table image, realize the automatic recognition of table images in design drawings, improve the automatic conversion from design drawings to simulation models, and improve modeling efficiency and quality. Attached Figure Description
[0041] Figure 1 This is a flowchart of a table image recognition method according to the present invention. Detailed Implementation
[0042] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.
[0043] like Figure 1 As shown, a table image recognition method includes the following steps:
[0044] S1. Obtain the target table image.
[0045] The target table image to be automatically recognized is acquired, wherein the target table image includes a table frame consisting of several cells, and the text to be recognized located within the table frame;
[0046] S2. Identify and obtain the text and text location information in the target table image;
[0047] S2.1 Perform image preprocessing on the acquired table images;
[0048] S2.1.1 Perform image grayscale processing on the acquired image to be identified;
[0049] S2.1.2 Filter the grayscale image to reduce image noise;
[0050] Use Gaussian filtering, median filtering, or mean filtering to process grayscale images;
[0051] S2.1.3 Binarize the filtered image;
[0052] The filtered image is binarized by setting the value of each pixel in the image to 0 or 255, that is, setting the image pixels to black or white.
[0053] S2.2 Perform target detection on the text region in the table image;
[0054] The YOLO algorithm is used to obtain boundary coordinates and class probabilities from image pixels, and the positions and information of all targets in the table image are confirmed and identified.
[0055] S2.3 Collect and recognize the text in the table image;
[0056] The CRNN text recognition model is used to recognize text in a table image. This model is trained using a pre-trained model and a constructed dataset. The sample data used for both the text region detection model and the text recognition model is obtained through data augmentation. The specific steps are as follows:
[0057] S2.3.1 The original sample data is augmented by image inversion, flipping, rotation, and scaling;
[0058] S2.3.2 Randomly add salt-and-pepper noise, Gaussian noise, image blur noise, and image erosion noise to the sample images to form new sample data;
[0059] S2.3.3 For numbers, letters, and standard symbols that are originally image data samples, a large number of sample images are randomly generated by establishing a template set of standard characters and using the samples for synthesis.
[0060] When recognizing text in a table image, it is necessary to detect the direction of the text in each text region. On the one hand, a binary classification model can be used to detect the text, dividing horizontal and vertical text into two categories of samples. The binary classification model can be trained to determine the direction of the text, and then the direction of the text can be corrected. On the other hand, rule-based detection can be used. Since text is composed of multiple characters, the direction of the text can be determined based on the length and width of the text region.
[0061] S3. Identify and obtain the position information of the connecting lines in the target table image;
[0062] The lines in the target table image are identified as a separate type of element, and the position information of the lines is finally output. After binarizing the target table image, the number of consecutive pixels is determined to be continuous by detecting whether there is a number of consecutive pixels whose average pixel value is less than a predetermined pixel threshold. If the number of consecutive pixels exceeds a predetermined threshold and whose average pixel value is close to 0, the number of consecutive pixels is determined to be continuous.
[0063] S4. Based on the detected position information of the connecting lines, identify the position information of the inflection points in the target table image;
[0064] S4.1 Based on the detected position information of the connecting lines, identify the position information of the corner points in the target table image;
[0065] Corner points include the intersections of lines in an image, and the positional information of corner points can be identified using the Shi-Tomasi detection algorithm.
[0066] S4.2 Based on the position information of the corner points in the identified target table image, identify the inflection points of the connecting lines in the target table image;
[0067] Based on the obtained connection positions and corner point positions, determine whether there are at least two lines connecting each corner point in the target table image and both of them are straight lines; for any corner point, if there are at least two lines connecting it and both of them are straight lines, then the corner point is identified as an inflection point.
[0068] For cases where the connecting lines are horizontal or vertical, the inflection point can be determined by detecting the right-angled sides. The specific steps are as follows:
[0069] Based on the identified connection and corner position information, determine whether the connecting lines to each corner in the target table image form a right angle. If the connecting lines to the corner form a right angle, the corner is identified as an inflection point.
[0070] The specific steps for the right angle detection are as follows:
[0071] For any corner point, a rectangular target area is defined centered on that corner point. A central rectangular area is then defined at the center of the target area. The areas above and below the central area within the target area are defined as the vertical detection area, and the areas to the left and right of the central area within the target area are defined as the horizontal detection area. When there is a vertical line within the vertical detection area and the length of the vertical line is approximately equal to the height of the vertical detection area, and there is a horizontal line within the horizontal detection area and the length of the horizontal line is approximately equal to the width of the horizontal detection area, it is determined that the line connecting to the corresponding corner point forms a right angle.
[0072] S5. Based on the location information of the connecting lines and inflection points obtained by recognition, identify the table frame in the target table image;
[0073] S5.1 Based on the location information of the connecting lines and inflection points obtained from the identification, identify the connection relationships in the target table image;
[0074] Detect whether there is a connection between two adjacent inflection points that meets a predetermined condition. If so, determine that there is a connection between the two inflection points. The predetermined condition includes, for example, that the connection is a straight line and the length of the connection reaches a preset multiple threshold of the distance between the two inflection points. The preset multiple threshold can be set to, for example, 0.95 times.
[0075] S5.2 By recognizing the inflection points in the target table image and the connection relationships between the inflection points, the table frame in the target table image can be determined.
[0076] S6. Reconstruct the table based on the identified table frame, and fill in the identified text in the corresponding positions in the table to complete the conversion of the target table image into an editable table format file.
Claims
1. A method for recognizing table images, characterized in that: The method specifically includes: S1. Acquire the target table image; S2. Identify and obtain the text and text location information in the target table image; S3. Identify and obtain the position information of the connecting lines in the target table image; S4. Based on the position information of the connecting line, obtain the position information of the inflection point in the target table image; S4.1 Based on the connection position information, identify the position information of corner points in the target table image; S4.2 Based on the corner position information, identify the inflection points of the connecting lines in the target table image; When at least two lines are connected to the corner point and both are straight lines, the right-angle side is detected to determine the inflection point. The right-angle detection step includes: for any corner point, a rectangular target area is determined with the corner point as the center, and a central area of the rectangle is determined at the center of the rectangular target area. The areas above and below the central area of the target area are determined as the vertical detection area, and the areas to the left and right of the central area of the target area are determined as the horizontal detection area. When there is a vertical line in the vertical detection area and the length of the vertical line is approximately equal to the height of the vertical detection area, and there is a horizontal line in the horizontal detection area and the length of the horizontal line is approximately equal to the width of the horizontal detection area, it is determined that the line connected to the corresponding corner point forms a right angle. S5. Based on the position information of the connecting lines and inflection points obtained by identification, the table frame in the target image is identified.
2. The table image recognition method according to claim 1, characterized in that, The specific steps for identifying and obtaining the table frame in the target image in step S5 are as follows: S5.1 Identify and obtain the connection and inflection point position information, identify and obtain the connection relationship in the target table image, and determine whether there is a connection relationship between two adjacent inflection points based on whether there is a connection between them that meets a predetermined condition. S5.2 By recognizing the inflection points and the connections between them in the target table image, the table frame in the target table image can be obtained.
3. The table image recognition method according to claim 1, characterized in that, In step 4.2, identifying the inflection points of the connecting lines in the target table image can be done by detecting the right-angled sides when the connecting lines are horizontal or vertical. The specific steps are as follows: Based on the identified connection and corner position information, it is determined whether the connection lines connecting the corner points in the target table image form right angles. If the connection lines connecting the corner points form right angles, the corner points are identified as inflection points.
4. The table image recognition method according to claim 1, characterized in that, The specific steps for identifying and obtaining text and text location information in the target table image in step S2 are as follows: S2.1 Perform image preprocessing on the acquired table images; S2.1.1 Perform image grayscale processing on the acquired image to be identified; S2.1.2 Filter the grayscale image to reduce image noise; S2.1.3 Binarize the filtered image; S2.2 Perform target detection on the text regions in the table image; S2.3 Collect and recognize the text in the table image.
5. The table image recognition method according to claim 4, characterized in that, In step S2.2, target detection in the text region of the table image is performed by using the YOLO algorithm to obtain boundary coordinates and class probabilities from image pixels, and to confirm and identify the positions of all targets in the table image. In step 2.3, the text in the table image is collected and recognized using the CRNN text recognition model. The CRNN text recognition model is trained using a pre-trained model and a constructed data sample set to obtain the corresponding text recognition model.
6. The table image recognition method according to claim 5, characterized in that, The sample data used by the CRNN text recognition model is obtained using data augmentation methods, which can be obtained using any one or a combination of the following methods: The original sample data is augmented by inverting, flipping, rotating, and scaling the image. Salt-and-pepper noise, Gaussian noise, image blur noise, and image erosion noise are randomly added to the sample images to form new sample data. For images containing numbers, letters, and standard symbols that are originally image data samples, a template set of standard characters is established, and a large number of sample images are randomly generated using sample synthesis.
7. The table image recognition method according to claim 4, characterized in that, When recognizing text in the table image, it is necessary to detect the text direction corresponding to each text region, and any one of the following methods can be selected: The detection is performed using a binary classification model, which divides horizontal and vertical text into two categories of samples. The binary classification model is trained to determine the direction of the text, and the text direction is then corrected. The system performs rule-based detection, determining the direction of the text based on the length and width of the text region.
8. The table image recognition method according to claim 1, characterized in that, After identifying the table frame in the target table image in step S5, the process further includes reconstructing the table based on the identified table frame and filling in the identified text at the corresponding positions in the table, thereby converting the target table image into an editable table format file.
9. The table image recognition method according to claim 1, characterized in that, The specific steps for step S3, which involves identifying and obtaining the position information of the connecting lines in the target table image, are as follows: The lines in the target table image are identified as a separate type of element, and the position information of the lines is finally output. After binarizing the target table image, the number of consecutive pixels is determined to be continuous by detecting whether the average pixel value of a consecutive number of pixels is less than a predetermined pixel threshold.