A method for recognizing text information in engineering drawings

Abstract

The application discloses a kind of identification methods of text information in engineering drawing, belong to the field of character recognition.First, the annotation dataset of text area in engineering drawing is made using DSA algorithm, the problem that existing engineering drawing text recognition annotation dataset is difficult to obtain is solved;Further, the superiority that deep learning can better extract image features is used, and the CenterNet model of deep learning is trained to identify the area image G containing table and text;Then, using the advantage that OpenCV technology is more sensitive to the texture characteristics of image and easier to detect and exclude the interference of table lines, the line detection and contour detection technology in OpenCV technology are used to crop the area image G;Finally, the OCR technology is used to recognize the text of the cropped cell image, and the recognized text is judged to be a key attribute or a value attribute according to the engineering field dictionary, solving the problem of difficult word recognition with semantic meaning, which can effectively, automatically and quickly identify the key text information in engineering drawing.

Description

Technical Field

[0001] This invention relates to the field of image and text recognition, specifically to a method for recognizing text information in engineering drawings. Background Technology

[0002] Text information in engineering drawings often appears in a corner of the drawing, frequently accompanied by tables, recording important information such as the project name, construction company, and drafter. Current main technologies for detecting and recognizing text from images include deep learning and OCR. Deep learning-based image text recognition technology requires a large amount of labeled data for training. However, engineering drawings are unique to construction companies and contain sensitive data, making it impossible to find relevant labeled data online. Therefore, deep learning alone cannot accurately extract text from engineering drawings. Traditional OCR technology uses an indiscriminate recognition strategy when detecting and recognizing text in images, ignoring the context and semantics of the text, only detecting and recognizing the text in the image. Text in engineering drawings recognized solely by OCR technology is meaningless text without punctuation, and therefore cannot automatically extract meaningful information such as the project name and construction company. Summary of the Invention

[0003] To address the aforementioned shortcomings in existing technologies, this invention provides a method for recognizing text information in engineering drawings, thereby solving the problems of difficulty in obtaining existing engineering drawing text recognition annotation datasets and difficulty in recognizing semantically meaningful words, and improving the efficiency and effectiveness of text information recognition in engineering drawings.

[0004] To achieve the above objectives, the technical solution adopted by the present invention is as follows.

[0005] A method for recognizing text information in engineering drawings, comprising: Add text annotations to engineering drawings and extract the text areas within the drawings; A labeled dataset was created from the extracted text regions using the DSA algorithm; The CenterNet model is trained based on the labeled dataset. The trained CenterNet model is then used to identify table and text regions in engineering drawings, resulting in the region image G containing tables and text. For the region image G, use OpenCV's cv2.getStructuringElement module to extract horizontal and vertical lines, and obtain a table border image G1 containing text; Use OpenCV's cv2.erode module to erase meaningless lines in G1 to obtain image G2; Use OpenCV's cv2.findContours module to detect all cells in the table in G1 and obtain the set of coordinates of all cells; Based on the set of cell coordinates, G2 is segmented into an image set G3 = {g i |i=1,2,3...n}, where g i Let n be the image of the i-th cell; n is the number of cell images. Perform OCR text recognition on each cell image in G3; Construct an engineering domain dictionary. Based on the engineering domain dictionary, determine whether the identified text is a key attribute or a value attribute: if it is a key attribute, fill it into the key column of the engineering information table; if it is a value attribute, fill it into the corresponding value column of the engineering information table, until all the identified text is identified.

[0006] Optionally, when annotating text areas on engineering drawings, the Sprite Annotation Assistant tool can be used.

[0007] Optionally, the step of creating a labeled dataset for the extracted text regions using the DSA algorithm specifically includes: Input image set X , set of labeled information S and the scale of the problem N ; Initialize the training and validation sets, let i =0; Determine whether i < N ; when i < N At that time, further judgment is needed to determine whether i N ; like i N ,Will S [ i ]+ X [ i Add to the training set; like i N ,Will S [ i ]+ X [ i Add to the validation set; make i ++, returns the result of the judgment. i < N The steps, until i ≥ NOutput the training set and validation set.

[0008] Optionally, when training the CenterNet model, the number of classes in the CenterNet model can be set to 2 based on the detection target of the text region.

[0009] Optionally, when training the CenterNet model, the resolution parameter of the CenterNet model is set to (512, 512) based on the pixels of the scanned part of the engineering drawing.

[0010] Optionally, the mean and variance of the labeled dataset are calculated before training the CenterNet model.

[0011] Optionally, the mean of the dataset is calculated as follows: ;in The mean, The pixel values ​​of all images in the dataset. This represents the total number of pixels in all images in the dataset.

[0012] Optionally, the variance is calculated as follows: .

[0013] Optionally, the engineering field dictionary includes keywords corresponding to the project name, construction unit, and drafter.

[0014] Optionally, when filling the recognized text information into the project information table, automatic matching and filling can be performed based on the correspondence between key attributes and value attributes.

[0015] The present invention has the following beneficial effects: This invention discloses a method for recognizing text information in engineering drawings. First, it utilizes the DSA algorithm to create a labeled dataset of text regions within the engineering drawings, solving the problem of difficulty in obtaining existing labeled datasets for text recognition in engineering drawings. Then, leveraging the superior image feature extraction capabilities of deep learning, a CenterNet deep learning model is trained to recognize region images G containing tables and text. Next, taking advantage of OpenCV technology's greater sensitivity to image texture features and its ability to more easily detect and eliminate table line interference, line detection and contour detection techniques in OpenCV are used to crop the region image G. Finally, OCR technology is used to recognize text in the cropped region (cell image), determining whether the recognized text is a key attribute or a value attribute based on an engineering domain dictionary. This solves the problem of difficulty in recognizing semantically meaningful words in engineering drawings due to nested text and tables and chaotic text information layout. It can effectively, automatically, and quickly recognize key text information such as project names, construction units, and drafters in engineering drawings. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating a method for recognizing text information in engineering drawings according to the present invention. Figure 2 A flowchart of the DSA method for constructing a dataset in an embodiment of the present invention. Detailed Implementation

[0017] The specific embodiments of the present invention are described below to enable those skilled in the art to understand the present invention. However, it should be understood that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the present invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.

[0018] A method for recognizing text information in engineering drawings, such as Figure 1 As shown, the process includes the following steps S1 to S4.

[0019] S1. Annotate the text area of ​​the engineering drawing and extract the text area within the drawing.

[0020] Specifically, this invention utilizes a sprite annotation assistant tool to annotate text areas on engineering drawings. Then, the annotated drawings are processed using the DSA algorithm to create an annotation dataset.

[0021] In practical applications, engineering drawings are prone to rotation (0°, 90°, 180°, or 270°) during scanning and format conversion due to differences in paper size and improper conversion parameter settings. During recognition, angle correction processing is necessary for these rotated drawings to ensure accurate detection of text and table areas. Furthermore, text and tables in engineering drawings are typically located at the edges of the drawing, not in the center. Therefore, a central region filtering step is required during recognition to eliminate interference from non-target tables in the center. Specifically, a central region range needs to be constructed first, and then intersection is used to determine if the current table is within the central region. If it is, the table is ignored; otherwise, it is retained as the target table.

[0022] S2. Use the DSA algorithm to create a dataset of extracted text regions, and use the trained CenterNet model to recognize the table and text region images of engineering drawings.

[0023] like Figure 2 As shown, the process of creating a labeled dataset from the extracted text regions using the DSA algorithm includes: input image set X , set of labeled information S and the scale of the problem N Initialize the training and validation sets, leti =0; Check if i < N ;when i < N At that time, further judgment is needed to determine whether i N ;like i N ,Will S [ i ]+ X [ i Add to the training set; if i N ,Will S [ i ]+ X [ i Add to the validation set; let i ++, returns the result of the judgment. i < N The steps, until i ≥ N Output the training set and validation set.

[0024] The CenterNet deep learning model is trained using a dataset to obtain model parameters capable of automatically recognizing text regions. CenterNet uses a fully convolutional network to directly obtain a 4x downsampled heatmap without needing to pre-set anchors, thus significantly reducing the number of network parameters and computational cost. In CenterNet, deformable convolutions are used before upsampling, which makes the network's receptive field more precise. At the same time, the resolution of the 4x downsampled feature map is also much higher than that of general networks, thus enabling it to detect image targets very well.

[0025] The parameter settings for training the CenterNet model are as follows.

[0026] (1) Based on the detection target of the text region, set the number of categories of CenterNet to 2.

[0027] (2) Engineering drawings are mainly scanned documents, most of which have high pixel counts. Therefore, the resolution parameter of CenterNet is set to (512, 512).

[0028] (3) Calculate the mean and variance of the dataset. Although Centernet has a built-in module for calculating the mean and variance, because the text areas of the engineering drawings have been annotated after inspection, it is necessary to calculate the mean and variance of the text area images manually. The formula is: Text area mean: ; Text region variance: ; in, The pixel values ​​of all images in the dataset. This represents the total number of pixels in all images in the dataset. The mean, Let Variance be the variance.

[0029] Using the trained CenterNet model, we can obtain images of the tables and text areas of engineering drawings, denoted as G.

[0030] S3. Use the OpenCV algorithm to recognize the table and text region images in step S2 and extract the text; the specific method includes steps S31 to S35.

[0031] S31. Using OpenCV's cv2.getStructuringElement module, perform line detection on the table and text region image G. After extracting horizontal and vertical lines, obtain the table border image G1 containing the text.

[0032] S32. Using the OpenCV image erasure module cv2.erode, erase the meaningless lines in G1 to obtain G2.

[0033] S33. Using OpenCV's contour detection module cv2.findContours, detect all cells in the table in G1 and obtain the set of coordinates of all cells.

[0034] S34. Use the set of cell coordinates to segment the image of G2, resulting in a set of image cells G3 = {g i |i=1,2,3....n};where g i Let i be the image of the i-th cell, where i is the index of the cell image and n is the number of cell images.

[0035] S35. For each cell image g in G3 i Perform OCR text recognition.

[0036] S4. Construct an engineering domain dictionary, use the constructed dictionary to determine the text identified in step S3, and fill the identified information into the engineering information table until all the identified text has been identified.

[0037] Specifically, judging g i The text in the table is either a key attribute or a value attribute: if it is a key attribute, it will be entered into the key column of the project information table; if it is a value attribute, it will be entered into the corresponding value column of the project information table.

[0038] This invention organically combines a sprite annotation assistant with a coordinate information extraction algorithm, providing an automatic annotation data method for constructing a deep learning dataset for text region detection, and solving the practical problem of difficulty in obtaining datasets for text recognition in engineering drawings.

[0039] When the dataset is limited, this invention integrates deep learning and OpenCV algorithms, which can effectively solve the problems of insufficient training data and poor training results in traditional deep learning methods.

[0040] This invention combines an engineering information domain dictionary with OCR to automatically identify the attribute information of text, thus solving the problem that OCR only recognizes text but does not understand its semantics.

[0041] The method of this invention has a high level of automation, which can greatly reduce the workload of operators, automatically identify text information in engineering drawings, and lay the foundation for automatically filling in engineering information reports.

[0042] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0043] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0044] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0045] Specific embodiments have been used to illustrate the principles and implementation methods of this invention. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of this invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this invention. Therefore, the content of this specification should not be construed as a limitation of this invention.

[0046] Those skilled in the art will recognize that the embodiments described herein are intended to help the reader understand the principles of the invention, and should be understood that the scope of protection of the invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific modifications and combinations based on the technical teachings disclosed in this invention without departing from the spirit of the invention, and these modifications and combinations are still within the scope of protection of this invention.

Claims

1. A method for recognizing text information in engineering drawings, characterized in that, include: Add text annotations to engineering drawings and extract the text areas within the drawings; A labeled dataset was created from the extracted text regions using the DSA algorithm; The CenterNet model is trained based on the labeled dataset. The trained CenterNet model is then used to identify table and text regions in engineering drawings, resulting in the region image G containing tables and text. For the region image G, use OpenCV's cv2.getStructuringElement module to extract horizontal and vertical lines, and obtain a table border image G1 containing text; Use OpenCV's cv2.erode module to erase meaningless lines in G1 to obtain image G2; Use OpenCV's cv2.findContours module to detect all cells in the table in G1 and obtain the set of coordinates of all cells; Based on the set of cell coordinates, G2 is segmented into an image set G3 = {g i |i=1,2,3...n}, where g i Let n be the image of the i-th cell; n is the number of cell images. Perform OCR text recognition on each cell image in G3; Construct an engineering domain dictionary. Based on the engineering domain dictionary, determine whether the identified text is a key attribute or a value attribute: if it is a key attribute, fill it into the key column of the engineering information table; if it is a value attribute, fill it into the corresponding value column of the engineering information table, until all the identified text is identified.

2. The method for recognizing text information in engineering drawings according to claim 1, characterized in that, When annotating text areas on engineering drawings, use the Sprite Annotation Assistant tool.

3. The method for recognizing text information in engineering drawings according to claim 1, characterized in that, The process of creating an annotated dataset for the extracted text regions using the DSA algorithm specifically includes: Input image set X , set of labeled information S and the scale of the problem N ; Initialize the training and validation sets, let i =0; Determine whether i < N ; when i < N At that time, further judgment is needed to determine whether i N ; like i N ,Will S [ i ]+ X [ i Add to the training set; like i N ,Will S [ i ]+ X [ i Add to the validation set; make i ++, returns the result of the judgment. i < N The steps, until i ≥ N Output the training set and validation set.

4. The method for recognizing text information in engineering drawings according to claim 1, characterized in that, When training the CenterNet model, the number of classes in the CenterNet model is set to 2 based on the detection target of the text region.

5. The method for recognizing text information in engineering drawings according to claim 1 or 4, characterized in that, When training the CenterNet model, the resolution parameter of the CenterNet model is set to (512, 512) based on the pixels of the scanned part of the engineering drawing.

6. The method for recognizing text information in engineering drawings according to claim 1, characterized in that, Before training the CenterNet model, calculate the mean and variance of the labeled dataset.

7. The method for recognizing text information in engineering drawings according to claim 6, characterized in that, The mean of the dataset is calculated as follows: ;in The mean, The pixel values ​​of all images in the dataset. This represents the total number of pixels in all images in the dataset.

8. The method for recognizing text information in engineering drawings according to claim 7, characterized in that, The variance is calculated as follows: .

9. The method for recognizing text information in engineering drawings according to claim 1, characterized in that, The engineering field dictionary contains keywords corresponding to project names, construction units, and drafters.

10. The method for recognizing text information in engineering drawings according to claim 1, characterized in that, When filling the recognized text information into the project information table, the system automatically matches and fills the information based on the correspondence between key attributes and value attributes.

Images