Industrial drawing table data extraction method and system and terminal equipment

Abstract

The invention belongs to the technical field of image recognition and processing, and discloses an industrial drawing table data extraction method and system and terminal equipment, and the method comprises the steps: carrying out the preprocessing of an initial industrial drawing image; reprocessing the first industrial drawing image by using an edge detection algorithm and morphological operation, and then detecting to obtain a table edge image; identifying table cells in the table edge image to obtain a table structure model; identifying contents in all cells in the table edge image to obtain target identification contents; according to the structure information of the cells and the business logic rule, whether the target identification content is wrong or not is verified, the wrong target identification content is repaired, the target table data is obtained, and efficient and accurate extraction of the industrial drawing table data is achieved.

Description

Technical Field

[0001] The present application relates to the field of image recognition and processing technology, and in particular to a method, system and terminal device for extracting industrial drawing table data. Background Art

[0002] In industrial production and manufacturing, a large number of industrial drawings are generated. These drawings contain a wealth of information, including tabular data that records key details such as component specifications, material parameters, and process steps. In existing technologies, the acquisition of tabular data from industrial drawings primarily relies on manual data entry. However, manual data entry is extremely inefficient and consumes a significant amount of manpower and time when faced with a massive amount of industrial drawings. Furthermore, manual operations are prone to human error, making it difficult to ensure data accuracy. Furthermore, with the acceleration of industrial digitalization, the need to quickly and accurately extract and convert tabular data from industrial drawings into digital information that can be used for data analysis, management decision-making, and other purposes is becoming increasingly urgent.

[0003] Although some existing image recognition technologies have certain applications in document image table recognition, industrial drawings have unique characteristics, such as complex graphic annotations, diverse table styles and different drawing scales, making it difficult for these general technologies to be directly applied to the accurate extraction of industrial drawing table data.

[0004] Therefore existing technology still needs to be improved and improved. Summary of the Invention

[0005] The present application provides a method, system and terminal device for extracting industrial drawing table data, aiming to solve the problems of low extraction efficiency and poor accuracy in the prior art when extracting industrial drawing table data.

[0006] In a first aspect, an embodiment of the present application provides a method for extracting industrial drawing table data, comprising:

[0007] Preprocessing the initial industrial drawing image to obtain a first industrial drawing image;

[0008] After reprocessing the first industrial drawing image using an edge detection algorithm and morphological operations, the target shape in the second industrial drawing image is detected to obtain a table edge image;

[0009] Identifying cells in the table edge image to obtain a table structure model;

[0010] Recognize the contents of all cells in the table edge image to obtain target recognition content;

[0011] According to the structural information of the cell and the business logic rules, the target recognition content is checked to see if there is any error, and the erroneous target recognition content is repaired to obtain the target table data.

[0012] In a second aspect, an embodiment of the present application provides a system for extracting industrial drawing table data, comprising:

[0013] A preprocessing module, configured to preprocess the initial industrial drawing image to obtain a first industrial drawing image;

[0014] a reprocessing module, configured to detect the target shape in the second industrial drawing image after reprocessing the first industrial drawing image using an edge detection algorithm and morphological operations to obtain a table edge image;

[0015] A cell recognition module, configured to recognize cells in the table edge image to obtain a table structure model;

[0016] A content recognition module is used to recognize the content of all cells in the table edge image to obtain target recognition content;

[0017] The repair module is used to check whether the target recognition content is incorrect based on the structural information of the cell and the business logic rules, and to repair the incorrect target recognition content to obtain the target table data.

[0018] In a third aspect, an embodiment of the present application provides a terminal device comprising: a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the method for extracting industrial drawing table data as described above.

[0019] Compared with the prior art, the present application provides a method, system and terminal device for extracting industrial drawing table data. The method preprocesses the initial industrial drawing image, then performs reprocessing operations using edge detection algorithms and morphological operations, detects the target shape in the second industrial drawing image, obtains a table edge image, and then identifies the cells in the table edge image. The target text in all cells in the identified table structure model is separated and identified, or non-target text is identified using preset symbol recognition rules and template matching methods to obtain target recognition content, thereby realizing automated extraction of table data, accurately identifying the features and structure of the table, and having a complete data verification and repair mechanism. It not only effectively reduces human errors and ensures the accuracy, quality and integrity of data extraction, but also greatly improves the efficiency of data extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 A flowchart of a method for extracting industrial drawing table data provided in this application;

[0022] Figure 2 A schematic diagram of a phase flow chart of the method for extracting industrial drawing table data provided in this application;

[0023] Figure 3 A schematic diagram of obtaining a first industrial drawing image in the method for extracting industrial drawing table data provided in this application;

[0024] Figure 4 A schematic diagram of obtaining a table edge image in the method for extracting industrial drawing table data provided in this application;

[0025] Figure 5 A flowchart of projection analysis of table edge images in the method for extracting industrial drawing table data provided in this application;

[0026] Figure 6 A schematic diagram of identifying non-target text in the method for extracting industrial drawing table data provided by this application;

[0027] Figure 7 A flow chart for obtaining target table data in the method for extracting table data of industrial drawings provided in this application;

[0028] Figure 8 A structural diagram of the industrial drawing table data extraction system provided in this application.

[0029] Figure numerals: 10 - pre-processing module; 20 - re-processing module; 30 - cell identification module; 40 - content identification module; 50 - repair module. DETAILED DESCRIPTION

[0030] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments.

[0031] The components of the embodiments of the present application generally described and illustrated in the drawings herein may be arranged and designed in a variety of different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed application, but rather merely represents selected embodiments of the present application. All other embodiments obtained by those skilled in the art based on the embodiments of the present application without creative effort are within the scope of protection of the present application.

[0032] Hereinafter, the terms "including", "having" and their cognates used in various embodiments of the present application are intended only to indicate specific features, numbers, steps, operations, elements, components or combinations of the aforementioned items, and should not be understood as excluding the existence of one or more other features, numbers, steps, operations, elements, components or combinations of the aforementioned items or adding the possibility of one or more features, numbers, steps, operations, elements, components or combinations of the aforementioned items. In addition, the terms "first", "second", "third" and the like are only used to distinguish descriptions and should not be understood as indicating or implying relative importance.

[0033] Unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by those skilled in the art to which the various embodiments of the present application belong. The terms (such as those defined in generally used dictionaries) will be interpreted as having the same meaning as in the context of the relevant technical field and will not be interpreted as having an idealized meaning or an overly formal meaning unless clearly defined in the various embodiments of the present application.

[0034] 1. Brief description of the existing technology:

[0035] In the early days of extracting tabular data from industrial drawings, the technology was relatively simple and basic. The main steps involved image preprocessing (such as noise reduction), table location, and table structure analysis for character recognition.

[0036] Image preprocessing primarily utilizes a simple grayscale conversion method, converting color industrial drawings directly according to a fixed grayscale calculation formula. This method simply converts color information into grayscale information, failing to optimize for the image's actual characteristics and poorly preserving image detail. Noise reduction also relies solely on a simple neighborhood averaging method. While this method can reduce noise to a certain extent, it can also blur the image and lose key information.

[0037] A relatively crude method is used to locate table borders. This method relies on a simple line-finding algorithm to find the borders of the table. This algorithm relies on the visibility of the table's lines, making it difficult to accurately locate borders of tables with lighter lines or lines that are not easily distinguished from surrounding graphics. Furthermore, it cannot handle tables with partial damage or discontinuous lines; if these problems are encountered, table positioning will be inaccurate.

[0038] Character recognition relies on basic template matching technology. Templates of common characters are pre-collected, and during recognition, the characters in the image are individually compared against these templates. However, this approach has significant limitations. Recognition is poor for industrial drawing characters, which come in a variety of font styles and sizes. Even the slightest change in character shape or size makes accurate matching difficult, leading to numerous character recognition errors. Furthermore, there is a lack of targeted recognition methods for the complex technical terms, special symbols, and formulas found in industrial drawings, making accurate recognition nearly impossible.

[0039] 2. Objective shortcomings of existing technologies:

[0040] The existing process of extracting tabular data from industrial drawings presents numerous challenges. First, image preprocessing is ineffective. Simple grayscale conversion and neighborhood averaging noise reduction methods not only fail to highlight key information in the table, but also blur the image, affecting subsequent table location and character recognition, significantly compromising overall data extraction accuracy.

[0041] Secondly, the table location process relies on a simple line search algorithm, making it extremely inadequate for complex industrial drawings. In actual industrial production, tables on drawings can exhibit various complexities, such as overlapping with other graphics and incomplete lines. Existing technologies simply cannot accurately locate tables, severely impacting the efficiency and accuracy of data extraction.

[0042] Furthermore, basic template matching technology is lagging behind in character recognition. The characters in industrial drawings are diverse and complex, making template matching difficult to handle with varying fonts, sizes, and special characters. This leads to numerous character recognition errors and severely compromises the integrity and accuracy of extracted data. It is also unable to capture specialized terminology, special symbols, and formulas, failing to capture this critical information and failing to meet the comprehensive data requirements of industrial production.

[0043] Finally, existing technologies lack any data verification and repair mechanisms. Errors or omissions in the extracted data cannot be detected and corrected in a timely manner, making the quality of the final data unreliable, making it difficult to effectively apply it in industrial production and providing a reliable basis for production decisions.

[0044] The following describes the design of a method for extracting industrial drawing table data through some specific embodiments.

[0045] See also Figure 1 , an embodiment of the present application provides a method for extracting industrial drawing table data, comprising steps S100-S500:

[0046] S100: Preprocess an initial industrial drawing image to obtain a first industrial drawing image.

[0047] For example, please refer to the flowchart of the method for extracting industrial drawing table data in this application. Figure 2 ,The method for extracting table data from industrial ,drawings includes the following stages: image preprocessing, table ,location, table structure analysis, data extraction, and data verification and ,repair.

[0048] The first stage, image preprocessing: grayscale processing, contrast enhancement, Gaussian filtering and noise reduction are performed on the input initial industrial drawing image. That is, in this application, the input initial industrial drawing image is grayscaled, and the color image is converted into a grayscale image to simplify the subsequent processing process. An adaptive histogram equalization algorithm is used to enhance the contrast of the grayscale image, highlighting the lines and text information in the drawing, especially the boundaries and internal data of the table. Gaussian filtering is used to remove noise interference in the image, avoid the impact of noise on subsequent table feature extraction, improve image quality, and ultimately obtain the first industrial drawing image.

[0049] It can be understood that in this application, a series of customized image preprocessing operations, such as grayscale processing, can effectively simplify image data; the adaptive histogram equalization algorithm can adaptively enhance image contrast according to the characteristics of the image itself, highlight the boundaries and internal data of the table, and make subsequent table positioning and character recognition more accurate; the rational use of Gaussian filtering can accurately remove image noise, while ensuring that key image information is not lost, improving image quality and reducing the interference of noise on table feature extraction. Then, this customized preprocessing process is greatly optimized for the characteristics of industrial drawings, effectively improving the overall processing effect.

[0050] In one implementation, see Figure 3 Step S100: pre-processing the initial industrial drawing image to obtain a first industrial drawing image, including:

[0051] S110, grayscale processing is performed on the initial industrial drawing image to obtain a grayscale image;

[0052] S120 , using an adaptive histogram equalization algorithm to perform contrast enhancement processing on the grayscale image, and then performing filtering and denoising processing to obtain a first industrial drawing image.

[0053] In the field of computer graphics, color images are composed of three color channels: R (red), G (green), and B (blue). The value range of each channel is usually 0-255, and the color combination of color images can reach 256×256×256 cases. Grayscale images satisfy R=G=B, and their values ​​are only 256 possible. Grayscale color images can simplify complex multi-channel operations into single-channel operations, greatly reducing the amount of calculation. Common grayscale processing methods include component method, maximum value method, average value method, and weighted average method. The weighted average method takes into account the difference in sensitivity of the human eye to different colors (highest sensitivity to green, lowest sensitivity to blue), and its calculation formula is: Gray=0.299×R+0.587×G+0.114×B (1).

[0054] Adaptive histogram equalization divides an image into several local regions, calculates a histogram for each region, and redistributes the brightness to enhance contrast. The steps include image segmentation, local histogram equalization, and image region merging. Image segmentation involves dividing the original industrial drawing image into several overlapping regions or blocks. The region size needs to be adjusted based on image details, such as selecting small regions for densely labeled areas in industrial drawings and large regions for large areas with a background color. Local histogram equalization independently calculates the grayscale histogram for each segmented region and performs equalization to improve image contrast within the region. Image region merging involves reassembling the processed small regions according to their original positional relationships to produce an image with enhanced contrast.

[0055] Exemplarily, the process of pre-processing the initial industrial drawing image is as follows:

[0056] In industrial production, industrial drawings have diverse color modes. Grayscale processing can unify the image color space, reduce the amount of data, and speed up subsequent processing. First, grayscale the initial industrial drawing image, converting the color image into a grayscale image, can simplify subsequent processing.

[0057] Taking a common RGB color image as an example, the cvtColor function in the OpenCV library is used to convert the RGB to grayscale formula: Gray = 0.299R + 0.587G + 0.114B (1). This formula is used to perform a weighted average of the three RGB components, converting the RGB value of each pixel to the corresponding grayscale value, thus achieving the transformation of the image from color to grayscale and obtaining a more reasonable grayscale image. In formula (1), R represents the red component; G represents the green component; and B represents the blue component; the values ​​before the three components correspond to their respective weights.

[0058] Among them, the cvtColor function in the OpenCV library is the core function for image color space conversion. Its functions include color space conversion and channel number adjustment, which can realize single-channel Multi-channel conversion (such as grayscale to color image).

[0059] Then, because traditional histogram equalization performs global processing on the entire image, it is easy to cause over-enhancement and lose some details. Therefore, in order to further highlight the table information, this application uses an adaptive histogram equalization (CLAHE) algorithm to perform contrast enhancement on the grayscale image, which prevents over-enhancement by limiting the maximum value in the histogram, and achieves this by shearing the histogram and evenly distributing the sheared part. The interpolation method is then used to smooth the transition of the window boundary pixels to avoid artifacts at the sub-region boundary. In specific operations, the CLAHE algorithm divides the image into multiple small blocks (such as 8x8 sub-blocks) and performs histogram equalization on each sub-block separately. For example, when processing a certain mechanical parts processing drawing, the parameters of the CLAHE algorithm can be set to: contrast limit of 4.0, tile grid size of 8x8. This can not only enhance the contrast within each sub-block, but also avoid excessive differences between blocks, making the lines and text of the table clearer and more legible.

[0060] Secondly, since industrial drawings are prone to noise during scanning and transmission, such as Gaussian noise and salt and pepper noise, the grayscale image is further subjected to filtering and noise reduction processing after further contrast enhancement processing.

[0061] Among them, Gaussian filtering is a linear smoothing filter, which is used for image denoising in this application, and is particularly effective in suppressing Gaussian noise. Its principle is to obtain the value of each pixel in the image by taking the weighted average of its own value and the values ​​of other pixels in its neighborhood. Among them, the two-dimensional Gaussian function expression is:

[0062] G(x,y)=2πσ21e-2σ 2 x 2 +y 2 (2);

[0063] In formula (2), σ is the standard deviation, which controls the shape of the Gaussian function.

[0064] During the specific operation, a user-specified template (convolution kernel) scans each pixel in the image. The weighted average grayscale value of the pixels within the neighborhood determined by the template replaces the value of the template's center pixel. For example, when processing the mechanical parts processing drawings mentioned above, the filter kernel size can be set to 5x5 and the standard deviation to 1.5. The filter kernel size determines the range of the neighborhood involved in the weighted average, and the standard deviation controls the weight distribution. Larger filter kernels and standard deviations can more effectively smooth the image, but may blur details; smaller values ​​preserve more details but have a relatively weaker noise reduction effect.

[0065] In industrial drawing processing, Gaussian filtering can effectively remove noise while ensuring that key image information is not lost, reducing the interference of noise on table feature extraction. Gaussian filtering effectively removes Gaussian noise by performing weighted averaging on each pixel in the image and its neighboring pixels. Compared with the ideal filter, the system function of the Gaussian filter is smooth, avoiding ringing. In this application, a discretized window sliding window convolution method is used to implement Gaussian filtering, taking into account both computational efficiency and denoising effect. Then, after Gaussian filtering, the noise in the image is effectively suppressed, providing a clear image foundation for subsequent table feature extraction.

[0066] S200 , after reprocessing the first industrial drawing image using an edge detection algorithm and morphological operations, detect the target shape in the second industrial drawing image to obtain a table edge image.

[0067] As an example, the second stage, table location, involves edge detection on the first industrial drawing image using an edge detection algorithm. For example, the Canny edge detection algorithm is used to extract edge information from the initial industrial drawing image, initially outlining the table's outline. Morphological operations, such as dilation and erosion, are then used to process the edge image, connecting disconnected table edge lines and removing irrelevant small edges. Next, the target shape in the second industrial drawing image is detected to further clarify the table's outline. For example, a Hough transform is used to detect the table's straight line features and determine the position of the table's row and column lines, thereby accurately locating the table's area within the drawing.

[0068] The Canny edge detection algorithm involves Gaussian filtering, calculating gradient magnitude and direction, non-maximum suppression, and dual-threshold detection and edge connection. Gaussian filtering removes image noise; calculating gradient magnitude and direction uses the Sobel operator and other methods to calculate the image's gradient in the x and y directions, yielding the gradient magnitude M(x, y) and direction θ(x, y). Non-maximum suppression retains pixels that may be edges. Dual-threshold detection and edge connection identify edges within the image.

[0069] It can be understood that in this application, by performing edge detection and morphological operations on the first industrial drawing image, and then detecting the specific shape of the table, the exact position of the table in the drawing can be accurately found, laying the foundation for subsequent table structure analysis and data extraction.

[0070] In one implementation, see Figure 4 Step S200: After reprocessing the first industrial drawing image using an edge detection algorithm and morphological operations, the target shape in the second industrial drawing image is detected to obtain a table edge image, including:

[0071] S210, extracting edge information from the first industrial drawing image using an edge detection algorithm to obtain an initial edge image;

[0072] S220, performing dilation and erosion operations on the initial edge image to obtain a second industrial drawing image;

[0073] S230 , using a Hough transform algorithm to detect row lines and column lines of the table in the second industrial drawing image to obtain a table edge image.

[0074] Among them, the Hough transform algorithm is an algorithm used to detect specific shapes (such as straight lines, circles, etc.) in images. Its basic principle is to map the straight lines in the image space to the parameter space through a certain mathematical transformation, and determine which parameters correspond to the straight lines in the image through statistical voting in the parameter space.

[0075] For example, the specific process of obtaining the table edge image is as follows:

[0076] Edge information is extracted using edge detection algorithms such as the Canny edge detection algorithm. The Canny edge detection algorithm is a classic edge detection method that can accurately detect edges in images through steps such as Gaussian filtering for noise reduction, calculation of gradient amplitude and direction, non-maximum suppression, dual threshold detection, and edge connection.

[0077] Among them, Gaussian filtering denoising is to use a Gaussian filter to perform a convolution operation on the input image to smooth the image, thereby reducing the impact of noise in the image on edge detection. Calculating the gradient magnitude and direction is to use the Sobel operator (or other gradient operators) to calculate the gradient components of the image in the horizontal direction (X direction) and the vertical direction (Y direction), respectively, so as to detect the gradient information of each pixel in the image and determine the possible edge position. Non-maximum suppression is to compare the gradient magnitude of each pixel with its adjacent pixels according to its gradient direction; and if the gradient magnitude of the current pixel is not the local maximum, it is set to 0 (that is, the pixel is suppressed), thereby achieving the refinement of the image edge, retaining the pixel with the local maximum gradient value, and removing redundant non-edge pixels. Dual threshold detection is to set two thresholds: a high threshold S T_H S and low threshold S T_L S (Usually S T_LS \approx0.4S T_H S ), compare the gradient magnitude with these two thresholds: if the gradient magnitude is greater than S T_H S , it is marked as a "strong edge" if the gradient magnitude is less than S T_L S, then exclude directly; if the gradient amplitude is between S T_L S and S T_H S If the pixel is between 0 and 1, it is temporarily marked as a "weak edge" and requires further judgment to distinguish strong and weak edges to reduce false detections and missed detections. Edge connection checks whether each "weak edge" pixel is connected to a "strong edge" pixel. If so, the weak edge is retained; otherwise, it is removed. This achieves the goal of supplementing continuous edges by connecting weak edges.

[0078] Then, in industrial drawing table positioning, we first apply the Canny algorithm to the preprocessed grayscale image (i.e., the first industrial drawing image) to extract edge information to obtain an initial edge image. The low threshold can be set to 50 and the high threshold to 150. The low threshold is used to mark weak edges, and the high threshold is used to mark strong edges. Through dual-threshold detection and edge connection, the edge information of the table is effectively extracted, and the general outline of the table is preliminarily outlined.

[0079] Since edge detection results alone may be discontinuous or affected by noise, the initial edge image is optimized using a combination of dilation and erosion operations from morphological operations. Dilation expands the target area in the image, connecting broken edges and filling small holes and breaks. Erosion, on the other hand, shrinks the target area to remove extraneous fine edges, isolated noise points, and small interference areas. By alternating between dilation and erosion (opening and closing operations), the complete outline of the table is accurately delineated.

[0080] In my actual operation, I first perform a dilation operation on the image obtained by Canny edge detection, using a 3x3 rectangular kernel as the structuring element, to make the table edges more continuous. Then, I perform an erosion operation on the second industrial drawing image, also using a 3x3 rectangular kernel to remove the excess noise and fine edges introduced by the dilation, further clarifying the table outline.

[0081] Finally, for tables in industrial drawings, rows and columns appear as straight line features in image space. The Hough transform algorithm is used to detect the rows and columns of the table in the second industrial drawing image. This involves mapping the parameters of these lines (such as slope and intercept, or angle and distance in polar coordinates) into parameter space. Within this parameter space, a vote is performed for each possible line parameter. The line corresponding to the parameter with the most votes becomes the row or column of the table.

[0082] In practice, for the edge image after morphological processing (i.e., the second industrial drawing image), the Hough transform parameters can be set: accumulator resolution of 1 pixel, angular resolution of 1 degree, line length threshold of 50 pixels, and line gap threshold of 10 pixels. The Hough transform can accurately detect the position of the row and column lines of the table in the image, thereby determining the precise area of ​​the table in the drawing, laying the foundation for subsequent table structure analysis and data extraction.

[0083] It can be understood that the edge detection-based algorithm combined with morphological operations in this application can accurately outline the outline of the table to achieve table positioning, and then apply the Hough transform to the outline image to accurately detect the horizontal row and vertical column line parameters in the table, thereby determining their precise position in the image. It can effectively deal with complex situations such as tilt and deformation that may exist in tables in industrial drawings, and achieve precise positioning.

[0084] S300: Identify cells in the table edge image to obtain a table structure model.

[0085] For example, after obtaining the edge image of the table, we enter the third stage, table structure analysis:

[0086] Regular cells can be calculated through projection analysis, or the spacing changes between adjacent row and column lines can be calculated to identify irregular cells in the table edge image, and finally a table structure module can be constructed.

[0087] It can be understood that in this application, different cells are processed accordingly in different ways, thereby improving the accuracy and effect of obtaining multiple cells and providing further guarantee for the accurate extraction of subsequent data.

[0088] In one implementation method, identifying cells in a table edge image to obtain a table structure model includes:

[0089] Projection analysis is performed on the table edge image, or the spacing changes between adjacent row lines and column lines in the table edge image are calculated to identify all cells in the table edge image and construct a table structure model.

[0090] The table includes multiple cells, which are divided into regular cells and irregular cells. A regular cell refers to a cell of a default standard unit in the table, and an irregular cell refers to a cell that occupies multiple standard unit spaces or less than one standard unit space, such as a merged or split cell.

[0091] Exemplarily, since cells include regular and irregular ones, the edge image needs to be vertically projected and horizontally projected for regular cells, and the positions of rows and columns of all cells in the regular cells are confirmed based on the projection results. For irregular cells, the position and range of the merged cells can be determined by comparing the difference between the spacing between adjacent row lines or column lines and the average spacing, thereby identifying all cells in the table edge image so that the table structure model can be constructed.

[0092] It can be understood that in this application, by using corresponding methods to identify different cells in the table edge image, it is possible to quickly and accurately identify various types of cells in the table edge image, thereby effectively improving the accuracy of data extraction.

[0093] In one implementation, see Figure 5 , perform projection analysis on the edge image of the table, including:

[0094] S301, analyzing the size and distribution of cells in the table edge image to obtain regular cells and irregular cells;

[0095] S302 , performing vertical projection analysis and horizontal projection analysis on the regular cells to obtain projection data, and confirming the row and column positions of all cells in the regular cells based on the peaks and troughs of the projection curve in the projection data.

[0096] Vertical projection is used to detect the position of column lines in tables. Vertical projection projects the image vertically and calculates the sum of the grayscale values ​​or other characteristic values ​​of each column of pixels to form a vertical projection curve. Horizontal projection is used to detect the position of row lines in tables. Horizontal projection projects horizontally and calculates the corresponding characteristic values ​​of each row of pixels to form a horizontal projection curve. In industrial drawing tables, the blank spaces between rows and columns appear as troughs on the projection curve, while the table content area appears as peaks.

[0097] For example, the specific process of projection analysis is as follows:

[0098] After obtaining the table edge image, first locate the table area and convert it into a binary image to make the distinction between the table lines and the background more obvious.

[0099] Then, the size and distribution pattern of the cells in the table edge image are analyzed. For example, the overall distribution pattern of the aspect ratio of each cell is calculated, and the cells are grouped by size using a clustering algorithm (such as K-Means) to identify regular patterns. This can distinguish regular cells from irregular cells in the table edge image.

[0100] Secondly, for regular cells, with the help of image processing libraries such as OpenCV in Python, it is easy to implement vertical and horizontal projections of the table area. By analyzing the peaks and troughs of the projection curve, the number of rows in the table can be determined to be [specific number of rows] and the number of columns to be [specific number of columns].

[0101] For example, for a 100x200 pixel table area, we traverse each column of pixels and count the number of white pixels (representing the table lines) in that column to obtain the vertical projection data. Similarly, we traverse each row of pixels and count the number of white pixels to obtain the horizontal projection data.

[0102] After obtaining the projection data, the peaks and troughs of the projection curve are analyzed. In the vertical projection curve, the peaks usually correspond to the positions of the table column lines, while the troughs represent the gaps between columns. By setting an appropriate threshold (e.g., peak height greater than 10 pixels), the number of columns in the table can be accurately determined. In the horizontal projection curve, the peaks correspond to the positions of the row lines, and the troughs correspond to the gaps between rows. This is used to determine the number of rows in the table, thereby confirming the row and column positions of all cells in the regular cell.

[0103] It can be understood that in this application, for regular tables (which contain regular cells), vertical and horizontal projection analysis is performed on the table edge image, and by analyzing the peaks and troughs of the projection curve, the number of rows and columns of the table can be quickly and accurately determined. For irregular tables (which contain irregular cells), the changes in the spacing between adjacent row and column lines are calculated to identify the merging and splitting of cells, and a correct table structure model is constructed, which can effectively cope with the complex and diverse table styles in industrial drawings.

[0104] In one implementation method, calculating the spacing variation between adjacent row lines and column lines in a table edge image includes:

[0105] Calculate the actual difference between the spacing between adjacent row lines or adjacent column lines and the corresponding average spacing, compare the actual difference with the target difference range, and determine the position and range of the merged cells in the irregular cells based on the comparison results.

[0106] Exemplarily, for a table area with merged cells (i.e., irregular cells), by calculating the changes in the spacing between adjacent row and column lines, for example, in a table with merged cells, the spacing between adjacent row or column lines in the row or column where the merged cells are located will be different from the spacing between other normal cells. Then, by comparing the actual difference between the spacing between adjacent row lines or the spacing between adjacent column lines and the average spacing, if the actual difference exceeds the target difference range (such as more than 20%), it can be determined that there is a cell merger in the area, and the position and range of the merged cells can be determined, thereby identifying the merging and splitting of cells, and then constructing a correct table structure model to ensure the accurate extraction of subsequent data.

[0107] S400: Recognize the contents of all cells in the table edge image to obtain target recognition content.

[0108] For example, after constructing the table structure model, we enter the fourth stage, data extraction:

[0109] Different methods are used to recognize target text and non-target text (such as formulas or special symbols) in the table structure model to obtain target recognition content.

[0110] In one implementation method, step S400, identifying the contents of all cells in the table edge image to obtain target identification content, includes:

[0111] S410: Separate and recognize the target text in all cells in the table structure model, or recognize non-target text using preset symbol recognition rules and template matching methods to obtain target recognition content.

[0112] Exemplarily, during the data extraction phase:

[0113] First, character segmentation technology is used to separate the target text in all cells in the table structure model, and then a convolutional neural network is used to perform character recognition to obtain the target recognition content. In this application, the target text includes not only text but also numerical values.

[0114] However, for non-target text, such as formulas or special symbols, specific symbol recognition rules (i.e., preset symbol recognition rules) and template matching methods are used for processing to obtain the overall target recognition content (including the recognized target text and non-target text).

[0115] It can be understood that in this application, different recognition technologies are used to perform corresponding recognition processing on the target text and non-target text in the cell, so as to accurately and efficiently identify various cell data and ensure that cell data containing special content can also be accurately and quickly extracted.

[0116] In one implementation method, non-target characters include: target mathematical formula symbols and other symbols other than the target mathematical formula symbols; separating and recognizing target characters in all cells in the table structure model, including:

[0117] S411, using a threshold segmentation algorithm to separate the target text from the background in all cells, and using a convolutional neural network model to perform character recognition on the separated target text;

[0118] See also Figure 6 , using preset symbol recognition rules and template matching methods to identify non-target text, including:

[0119] S412, predetermining a recognition rule for the target mathematical formula symbol based on the shape characteristics of the target mathematical formula symbol, and recognizing the target mathematical formula symbol according to the recognition rule to obtain a non-target text;

[0120] S413. After establishing a template library of other symbols, similarity calculation is performed between the template symbols in the template library and the non-target text, and the template symbol with the highest similarity is retained as the non-target text.

[0121] For example, the specific process of identification is as follows:

[0122] For each cell, character segmentation and recognition technology is used. First, a threshold segmentation algorithm is used to separate the target text within the cell from the background. For the segmented character image (i.e., the separated target text), a convolutional neural network (CNN) model is used for character recognition, and the recognition results are used as the cell data content.

[0123] In cell data extraction, threshold segmentation is the key first step. Commonly used threshold segmentation algorithms include global threshold segmentation (such as the Otsu algorithm) and local threshold segmentation (such as adaptive threshold segmentation). Among them, for cell images with uniform illumination, the Otsu algorithm automatically determines a global threshold by calculating the grayscale histogram of the image, and divides the image into two parts: foreground (text) and background. For example, when processing a certain cell image, the threshold calculated by the Otsu algorithm is 127, and the image is binarized with this threshold, successfully separating the text from the background. However, for cell images with uneven illumination, an adaptive threshold segmentation algorithm is used to calculate the local threshold based on the grayscale characteristics of the neighborhood of each pixel, thereby achieving more accurate segmentation.

[0124] Then, after the character segmentation is completed, the convolutional neural network model is used to perform character recognition on the separated target text. CNN has powerful feature extraction capabilities and can automatically learn the characteristics of characters by constructing multiple layers of convolutional layers, pooling layers, and fully connected layers. Taking the LeNet-5 model as an example, the segmented character image (size 28x28 pixels) is input, and it is processed by two layers of convolutional layers, two layers of pooling layers, and three layers of fully connected layers. Finally, the recognition result is output through the Softmax classifier (Softmax classifier is a multi-classification model widely used in machine learning). Among them, in the stage of training the convolutional neural network model, a large amount of sample data containing common characters in industrial drawings can be used to train the model to adjust the parameters of the model so that it can accurately recognize various characters.

[0125] However, for non-target text (including target mathematical formula symbols and other symbols except target mathematical formula symbols), it is difficult to accurately identify them by relying solely on the CNN model. Therefore, specific symbol recognition rules and template matching methods are needed. Among them, for the target mathematical formula symbols, it is necessary to predefine the recognition rules of the target mathematical formula symbols based on their shape characteristics. For example, for common mathematical formula symbols, such as the "+" symbol, they can be identified by judging the length and position relationship of their horizontal and vertical lines.

[0126] For symbols other than the target mathematical formula symbols, such as complex special symbols, a template library of other symbols can be established in advance, and the template symbols in the template library can be matched with non-target text, and the similarity between the two can be calculated. The symbol corresponding to the template with the highest similarity is retained as the recognized non-target text.

[0127] As you can understand, the data extraction process utilizes character segmentation and recognition technology: for standard cells, a threshold segmentation algorithm is used to separate text from the background, and character recognition is then performed using a convolutional neural network model, ensuring accurate recognition. For cells containing specialized terminology, complex formulas, and special symbols, specific symbol recognition rules and template matching methods are used. This addresses the shortcomings of traditional OCR technology and deep learning models in handling specialized content, ensuring the integrity of the extracted data.

[0128] S500: Verify whether the target recognition content is incorrect based on the cell structure information and business logic rules, and repair the incorrect target recognition content to obtain target table data.

[0129] For example, after the target identification content is identified, the fifth stage, data verification and repair, is entered:

[0130] First, the target recognition content in the cell is verified. This can be done based on the cell's structural information and business logic rules. For example, verification can be performed based on table structure information, using structural information such as the number of rows and columns in the table, and whether cells are merged or split, to check whether the data conforms to the expected format. Alternatively, verification can be performed based on business logic rules, establishing a series of rules based on the actual use of industrial drawings and business needs to verify the rationality of the data. Then, any incorrect target recognition content is repaired to obtain the target table data. However, if the target recognition content is correct, the target recognition content is directly saved to obtain the target table data.

[0131] It can be understood that a data verification and repair link is set up in this application. By verifying whether the target identification content is incorrect based on the cell structure information and business logic rules, and repairing the erroneous content, data verification and repair are achieved, which effectively improves the quality and availability of the data and can ensure that the extracted data can meet the actual needs of industrial production.

[0132] In one implementation, see Figure 7 , based on the cell structure information and business logic rules, check whether the target recognition content is incorrect and repair the incorrect target recognition content, including:

[0133] S501, verifying whether the target identification content is incorrect based on whether the value in the target identification content is within the target range or whether the type of the target identification content meets the expected type;

[0134] S502: Verify whether the target identification content is missing based on whether the key field in the cell is missing.

[0135] For example, the process of verifying and repairing the target identification content is as follows:

[0136] To verify the target recognition content based on the cell structure information, you can check the correspondence between rows and columns: Based on the table structure determined by projection analysis (for example, 5 rows and 4 columns), check each row and column for missing or redundant data. For merged cell areas, you can verify that their contents are correctly allocated to the corresponding positions. For example, if a row has merged cells (such as spanning two columns), ensure that the merged content is correctly reflected in the data.

[0137] However, according to the business logic rules of the cell (for example, value range verification, required field check, data type verification, and relevance verification), the target identification content is verified to see if there are any errors. Among them, the value range verification is to verify whether the value in the target identification content is within the target range. Taking a table that records part dimensions as an example, if a column of data in a cell represents the length of the part, its unit is millimeters. According to actual production conditions, the length value should be within a certain reasonable range (that is, the target range, such as 0-1000 mm). Then, by traversing the data extracted from the target identification content, check whether each value is within the specified range. If it exceeds the range, it is determined to be erroneous data.

[0138] Required fields are checked to see if any key fields in the table are missing data. For example, if a part number column contains a null value, it is considered missing data. Data type verification verifies that the extracted data conforms to the expected type (such as string, integer, floating point number, etc.). Relationship verification checks whether the logical relationship between different fields is correct. For example, if a row contains both "Part Number" and "Processing Operation", it is necessary to confirm whether the two match.

[0139] For errors or missing data (i.e., incorrect target identification content) found during verification, a variety of repair strategies can be used. For example, for missing numerical data, the statistical characteristics of other data in the same column can be used for estimation and filling. If the data in the same column is [10, 12, 15, 18, None, 22], the mean of the data in this column is calculated to be (10+12+15+18+22) / 5=15, and the median is 15. You can choose to fill the missing values ​​with the mean or median. For missing text data, if the contents of adjacent cells have a certain pattern, such as "Part A-01" and "Part A-02", the missing third cell may contain "Part A-03", then fill it in according to this pattern.

[0140] However, if the error type and correction rules can be determined for erroneous data, appropriate modifications can be made, such as decimal point correction or unit conversion. Decimal point correction involves identifying numerical data that significantly deviates from the normal range due to an incorrect decimal point position. For example, if a column normally ranges from 10-100 but contains 1000, it can be assumed to be 100.0. Unit conversion involves converting inconsistent numerical units to standard units.

[0141] It can be understood that the data verification and repair technology of the present invention performs data verification based on the structural information and business logic rules of the table, which can promptly detect errors or missing data. Repair is performed by utilizing the data relationships between adjacent cells and the statistical characteristics of the entire table. For example, if missing numerical data is estimated and filled using the mean or median of other data in the same column, this mechanism can reduce the error rate of data and improve its integrity, greatly improving the quality and usability of extracted data, thereby making it more in line with the actual needs of industrial production.

[0142] See also Figure 8 , an embodiment of the present application provides a system for extracting industrial drawing table data, comprising:

[0143] The preprocessing module 10 is used to preprocess the initial industrial drawing image to obtain a first industrial drawing image. The reprocessing module 20 is used to detect the target shape in the second industrial drawing image after reprocessing the first industrial drawing image using an edge detection algorithm and morphological operations to obtain a table edge image. The cell recognition module 30 is used to identify the cells in the table edge image to obtain a table structure model. The content recognition module 40 is used to identify the content in all cells in the table edge image to obtain target recognition content. The repair module 50 is used to verify whether the target recognition content is incorrect based on the cell structure information and business logic rules, and to repair the erroneous target recognition content to obtain the target table data.

[0144] Exemplarily, the system of this embodiment corresponds to the method for extracting industrial drawing table data of the above embodiment. The options in the above embodiment are also applicable to this embodiment, so they will not be described again here.

[0145] Among them, this application has the following beneficial effects:

[0146] 1. Efficient and Accurate: Compared to the traditional method of manually entering data into industrial drawing tables, this application significantly improves data extraction efficiency through a series of automated steps, including image preprocessing, table positioning, table structure analysis, data extraction, and data verification and repair. Furthermore, by employing advanced algorithms and models, such as adaptive histogram equalization and convolutional neural networks (CNNs), human error can be effectively reduced, ensuring the accuracy of data extraction and meeting the high-precision data requirements of industrial production.

[0147] 2. Strong adaptability: This application employs a variety of targeted processing methods to address the complex graphic annotations, diverse table styles, and varying drawing scales found in industrial drawings. For example, in the image preprocessing stage, adaptive histogram equalization is used to highlight table boundaries and internal data, while Gaussian filtering is used to remove noise. In the table location and structure analysis stage, edge detection, morphological operations, and projection analysis are combined to effectively identify table features and structures in the case of irregular tables and varying drawing scales.

[0148] Furthermore, in another embodiment of the present application, the extraction of industrial drawing table data can also be achieved by the following method:

[0149] First, there's the template matching method: Templates for various industrial drawing tables are pre-created and then matched against the drawing to be processed. The similarity between the image and the template is calculated to determine the table's position and structure. This method can quickly and accurately extract data from industrial drawings with fixed formats and styles. However, its limitations lie in the need for a large inventory of templates and limited adaptability to irregular and diverse table styles. Whenever a new table style is encountered, the template must be recreated, which is a significant workload.

[0150] Second, a rule-based approach involves developing a series of rules for extracting data from industrial drawing tables. For example, data is extracted based on line characteristics and text layout patterns. This approach offers the advantage of high interpretability and rapid processing of tables with simple rules. However, the complexity of industrial drawings makes it difficult to cover all cases with rules. When faced with complex graphic annotations and special symbols, rule development and maintenance are challenging, making it prone to missed and misjudgment.

[0151] Third, a method combining semantic analysis can be used: natural language processing (NLP) can be used to perform semantic analysis on textual information in industrial drawings to understand the meaning and relationships of table data. This approach can better understand table content at a semantic level, improving the accuracy and completeness of data extraction. However, the application of NLP in the industrial sector currently faces many challenges, such as understanding professional terminology and acquiring domain knowledge. The technology still needs to be matured, and the computational cost is relatively high.

[0152] The present application also provides a terminal device. Exemplarily, the terminal device includes a processor and a memory, wherein the memory stores a computer program, and the processor runs the computer program to enable the terminal device to execute the above-mentioned industrial drawing table data extraction method or the functions of each module in the above-mentioned industrial drawing table data extraction system.

[0153] Among them, the processor can be an integrated circuit chip with signal processing capabilities. The processor can be a general-purpose processor, including a central processing unit (CPU), a graphics processing unit (GPU) and a network processor (NP), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or at least one of other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc., which can implement or execute the disclosed methods, steps and logic block diagrams in the embodiments of the present application.

[0154] The memory may be, but is not limited to, a random access memory (RAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), etc. The memory is used to store a computer program, and the processor may execute the computer program accordingly after receiving an execution instruction.

[0155] The above is only a specific implementation method of the present application, but the scope of protection of the present application is not limited thereto. Any technician familiar with this technical field can easily think of changes or replacements within the technical scope disclosed in this application, which should be covered by the scope of protection of the present application.

Claims

1. A method for extracting industrial drawing table data, characterized in that: include: Preprocessing the initial industrial drawing image to obtain a first industrial drawing image; After reprocessing the first industrial drawing image using an edge detection algorithm and morphological operations, the target shape in the second industrial drawing image is detected to obtain a table edge image; Identifying cells in the table edge image to obtain a table structure model; Recognize the contents of all cells in the table edge image to obtain target recognition content; According to the structural information of the cell and the business logic rules, the target recognition content is checked to see if there is any error, and the erroneous target recognition content is repaired to obtain the target table data.

2. The method for extracting industrial drawing table data according to claim 1, characterized in that: After reprocessing the first industrial drawing image using an edge detection algorithm and a morphological operation, detecting the target shape in the second industrial drawing image to obtain a table edge image includes: Extracting edge information from the first industrial drawing image using the edge detection algorithm to obtain an initial edge image; Performing dilation and erosion operations on the initial edge image to obtain the second industrial drawing image; The row lines and column lines of the table in the second industrial drawing image are detected using a Hough transform algorithm to obtain the table edge image.

3. The method for extracting industrial drawing table data according to claim 1, characterized in that: The identifying of cells in the table edge image to obtain a table structure model includes: Projection analysis is performed on the table edge image, or a change in spacing between adjacent row lines and column lines in the table edge image is calculated to identify all cells in the table edge image and construct the table structure model.

4. The method for extracting industrial drawing table data according to claim 1, characterized in that: The identifying of the contents in all cells in the table edge image to obtain target identification content includes: The target text in all cells in the table structure model is separated and recognized, or the non-target text is recognized using a preset symbol recognition rule and a template matching method to obtain target recognition content.

5. The method for extracting industrial drawing table data according to claim 1, characterized in that: The preprocessing of the initial industrial drawing image to obtain the first industrial drawing image includes: grayscale the initial industrial drawing image to obtain a grayscale image; The grayscale image is subjected to contrast enhancement processing by using an adaptive histogram equalization algorithm, and then subjected to filtering and denoising processing to obtain the first industrial drawing image.

6. The method for extracting industrial drawing table data according to claim 4, characterized in that: The non-target characters include: target mathematical formula symbols and other symbols except the target mathematical formula symbols; the separating and identifying the target characters in all cells in the table structure model includes: Using a threshold segmentation algorithm to separate the target text from the background in all cells, and using a convolutional neural network model to perform character recognition on the separated target text; The method of recognizing non-target text using a preset symbol recognition rule and a template matching method includes: predetermining a recognition rule for the target mathematical formula symbol according to shape characteristics of the target mathematical formula symbol, and recognizing the target mathematical formula symbol according to the recognition rule to obtain the non-target text; After the template library of the other symbols is established, similarity calculation is performed between the template symbols in the template library and the non-target text, and the template symbol with the highest similarity is retained as the non-target text.

7. The method for extracting industrial drawing table data according to claim 1, characterized in that: The checking whether the target identification content is incorrect based on the structural information of the cell and the business logic rules, and repairing the incorrect target identification content, includes: Verifying whether the target identification content is incorrect based on whether a value in the target identification content is within a target range or whether a type of the target identification content conforms to an expected type; Whether the target identification content is missing is checked based on whether the key field in the cell is missing.

8. The method for extracting industrial drawing table data according to claim 3, characterized in that: The performing projection analysis on the table edge image includes: Analyzing the size and distribution of cells in the table edge image to obtain regular cells and irregular cells; Performing vertical projection analysis and horizontal projection analysis on the regular cells to obtain projection data, and confirming the row and column positions of all cells in the regular cells based on the peaks and troughs of the projection curve in the projection data; The calculating the change in spacing between adjacent row lines and column lines in the table edge image includes: The actual difference between the spacing between adjacent row lines or adjacent column lines and the corresponding average spacing is calculated, and the actual difference is compared with the target difference range, and the position and range of the merged cells in the irregular cells are determined according to the comparison result.

9. A system for extracting industrial drawing table data, characterized in that: include: A preprocessing module, configured to preprocess the initial industrial drawing image to obtain a first industrial drawing image; a reprocessing module, configured to reprocess the first industrial drawing image using an edge detection algorithm and morphological operations, and then detect the target shape in the second industrial drawing image to obtain a table edge image; A cell recognition module, configured to recognize cells in the table edge image to obtain a table structure model; A content recognition module is used to recognize the content of all cells in the table edge image to obtain target recognition content; The repair module is used to check whether the target recognition content is incorrect based on the structural information of the cell and the business logic rules, and to repair the incorrect target recognition content to obtain the target table data.

10. A terminal device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, the steps of the method for extracting industrial drawing table data according to any one of claims 1 to 8 are implemented.

Images