An Automatic Identification Method and System for Drawing Titles of Engineering Drawings Based on Template Matching

Abstract

The present invention discloses a method and system for automatically identifying the title block of engineering drawings based on template matching. This method uses existing drawings to create a title block template library, and trains a title block position detection model, a title block classification model, and a title block text recognition model respectively; for the target drawing, the title block position detection model is used to locate the coordinate position of the title block, and the title block to be recognized is cropped and obtained; the title block classification model is used to determine the title block template style to which the title block to be recognized belongs, and at the same time, the title block text recognition model is used to obtain the text content and position coordinates of the text box in the title block to be recognized; by using the position information and structural relationship between the template title block attributes and the attribute value text boxes, and the landing position of the center point of the attribute value text box in the title block to be recognized, the attribute values of the corresponding text boxes in the title block to be recognized are obtained, realizing the correspondence between attributes and attribute values. The present invention realizes the accurate recognition of the title block of the drawing in a more intelligent and efficient manner, reducing the labor and time costs while ensuring the accuracy.

Description

Technical Field

[0001] The present invention belongs to the technical field of image recognition, and particularly relates to a method and system for automatically recognizing the title block of engineering drawings based on template matching. This technical method is applicable to the automatic recognition of the title block columns of engineering drawings in the engineering field. Background Art

[0002] With the continuous expansion of the scale of engineering projects and the wide application of Computer-Aided Design (CAD) technology, as an indispensable part of engineering design, the output quantity of engineering drawings is increasing day by day, and the complexity is also rising continuously. Engineering drawings are the core output of the engineering design process and important documents in engineering projects. The title block area of the drawings usually contains basic information such as project name, drawing number, version, drawing unit, date, etc. These information have great reference value for drawing classification, storage and management. Currently, generally relying on manual recognition methods, the title block information is extracted, verified and entered from a large number of drawings, and then classified according to the title block information, and the drawings are stored in the archives in the form of folders. This traditional manual entry method is very cumbersome, inefficient and error-prone, resulting in low-quality drawing filing and unable to achieve efficient retrieval and management. Therefore, there is an urgent need to develop an automatic recognition method for drawing title blocks.

[0003] The automatic recognition of the drawing title block mainly consists of two steps: title block position detection and title block table recognition. The difficulty in title block position detection lies in the fact that there is no strictly unified standard for the title block layout, and its position on the drawing is not fixed, making it difficult to accurately locate and recognize the title block frame. The difficulty in title block table recognition lies in the diverse types and inconsistent formats of title blocks, the relatively complex table structure, and the significant differences in the names of attribute fields in the table. For detecting the position of the title block, currently, most methods use complex image processing techniques combined with rule judgment to locate the title block in the drawing. For example, Chinese patents CN117711010A, CN118410196A, and CN118887696A all involve this method. However, this method has high requirements for the clarity of the drawing image, the processing process is relatively complex, and it is also interfered by other rectangular frames in the drawing, resulting in low accuracy of title block detection. For title block table recognition, the main difficulty lies in the correspondence between attributes and attribute values in the title block column. Currently, most methods use rule matching to extract attribute keyword fields from the text content obtained after OCR (Optical Character Recognition) processing of the title block image. Although OCR technology has become mature and high-precision recognition of image text in complex scenarios can basically be achieved, due to the diverse formats of title blocks, the inconsistent attribute names, and the relatively complex table structure of a single title block, only the content and coordinate position information of the text area in the image can be obtained after OCR processing, and the logical structure relationship between these text areas cannot be obtained, resulting in poor correspondence between the attributes and attribute values in the title block directory. Summary of the Invention

[0004] Aiming at the defects existing in the prior art, the purpose of the present invention is to provide an automatic recognition method and system for engineering drawing title blocks based on template matching. The present invention first uses the existing drawings to create a title block style template library, and trains a title block position detection model, a title block classification model, and an OCR title block text recognition model respectively. For the target drawing, first use the title block position detection model to locate the coordinate position of the title block in the target drawing, and crop the drawing to obtain the title block image to be recognized. For the obtained title block image, use the title block classification algorithm to determine the title block template to which the title block to be recognized belongs, and at the same time use the title block text recognition model to obtain the content and coordinate position of the text box in the title block image. According to the position information and structural relationship of the text areas in the template title block, use the landing position of the center point of the text box corresponding to the attribute value in the template title block to determine the text box of the attribute value in the title block image to be recognized, so as to achieve the correspondence between attributes and attribute values and achieve the purpose of accurately recognizing the title block.

[0005] In order to achieve the above purpose, the present invention adopts the following technical solutions:

[0006] On the one hand, the present invention proposes an automatic recognition method for engineering drawing titles based on template matching, including the following steps:

[0007] S101, establish a title template library based on the existing title formats; the template library at least includes template title images and corresponding label files, and the label files at least include the label names and position coordinates of template title attributes and corresponding attribute value text boxes, as well as the size information of the template title;

[0008] S102, make a drawing title detection data set, and use an object detection algorithm to train and obtain a title position detection model;

[0009] S103, make a title classification data set, and use an image classification algorithm to train and obtain a title classification model;

[0010] S104, make a title text detection data set, use an object detection algorithm to train and obtain a title text detection model, and combine it with an optical character recognition model to obtain a title text recognition model;

[0011] S105, use the title position detection model to detect the target drawing, and obtain the position coordinates of the title of the target drawing; crop the target drawing through the position coordinates to obtain the title to be recognized;

[0012] S106, use the title classification model to classify the style of the title to be recognized, and determine the classification of the title to be recognized in the title template library;

[0013] S107, use the title text recognition model to perform OCR recognition on the title to be recognized, and obtain the position coordinates and text content of the detected text boxes in the title to be recognized;

[0014] S108, scale the size of the template title so that the image size of the template title is the same as that of the title to be recognized, and at the same time scale the text box coordinates on the template title accordingly;

[0015] S109, match the attribute value text boxes corresponding to each attribute in the scaled template title with the text boxes on the title to be recognized, and obtain the values in the text boxes matched on the title to be recognized as the attribute values corresponding to the attributes, so as to achieve a one-to-one correspondence between the attributes and the attribute values.

[0016] Further, in step S101, the attributes include multiple or all of project name, stage, specialty, drawing name, drawing number, formation date, designer, approver, verifier, reviewer, checker, etc., and the size information includes the width and height of the template title image; the label file also includes synonymous attribute reference information.

[0017] Further, in step S102, the object detection algorithm uses the PP-YOLO model algorithm, and the PaddlePaddle training framework and its end-to-end object detection suite PaddleDetection are used to train the model to obtain the label position detection model.

[0018] Further, in step S103, the image classification algorithm uses the ResNet50 model algorithm, and the PaddlePaddle training framework and its image classification suite PaddleClas are used to train the model to obtain the label classification model.

[0019] Further, in step S104, the object detection algorithm uses the DB model algorithm, and the PaddlePaddle training framework and its text recognition suite PaddleOCR are used to train the model to obtain the label text detection model; the text recognition model uses the CRNN model provided in PaddleOCR.

[0020] Further, step S106 further includes:

[0021] If the to-be-recognized label image is deflected, compare the aspect ratios of the to-be-recognized label image and the template label image, and adjust it to be upright. Among them, the deflection includes 90°, 180°, or 270°, and the aspect ratio is the ratio of the length of the image in the X-axis direction to the width in the Y-axis direction, including the following steps:

[0022] (1) If the aspect ratios of the to-be-recognized label image and the template label image are both greater than 1 or both less than 1, judge whether the deflection angle is 180° or 0° through Chinese font direction detection. If it is 180°, flip the to-be-recognized label image by 180°. If it is 0°, there is no need to correct the angle.

[0023] (2) If the aspect ratio of the to-be-recognized label image and the template label image is one greater than 1 and the other less than 1, flip the to-be-recognized label image by 90° to make its deflection angle change to 180° or 0°. Judge whether the deflection angle is 180° or 0° through Chinese font direction detection. If it is 180°, flip the to-be-recognized label image by 180°. If it is 0°, there is no need to correct the angle.

[0024] Further, in step S108, the resizing of the template label includes using the resize function of OpenCV to adjust the image size of the template label to be the same as that of the to-be-recognized label.

[0025] Further, in step S109, the method for matching the text boxes corresponding to the attributes in the scaled template drawing title with the text boxes on the drawing title to be recognized and obtaining the values in the text boxes matched on the drawing title to be recognized is as follows:

[0026] Based on the position coordinates and text content of the text boxes detected by the drawing title text recognition model, determine one by one the landing positions of the center points of the text boxes corresponding to the attributes in the scaled template drawing title on the drawing title to be recognized, and obtain the text content in the text box where the landing position is located.

[0027] Further, the method for determining the landing positions of the center points of the text boxes of the attribute values in the scaled template drawing title on the drawing title to be recognized includes the following steps:

[0028] (1) Obtain the center point of an attribute value text box M in the scaled template drawing title, denoted as ;

[0029] (2) Traverse all the text boxes in the drawing title to be recognized. If there exists a text box N with coordinates and that satisfies and , then determine that the center point of the text box M falls into the text box N of the drawing title to be recognized.

[0030] On the other hand, the present invention proposes an automatic recognition system for engineering drawing titles based on template matching, which is used to implement the above-mentioned automatic recognition method for engineering drawing titles based on template matching, including:

[0031] A drawing title template library module, which is used to establish a drawing title template library based on the existing drawing title formats;

[0032] A drawing title position detection and cropping module, which is used to detect the target drawing using the drawing title position detection model, obtain the position coordinates of the drawing title of the target drawing; crop the target drawing through the position coordinates to obtain the drawing title to be recognized;

[0033] A drawing title classification module, which is used to classify the drawing title to be recognized using the drawing title classification model to determine the classification to which the drawing title to be recognized belongs in the drawing title template library;

[0034] A drawing title text recognition module, which is used to perform OCR recognition on the drawing title to be recognized using the drawing title text recognition model to obtain the position coordinates and text content of the text boxes detected in the drawing title to be recognized;

[0035] A template drawing title scaling module, which is used to scale the size of the template drawing title so that the image size of the template drawing title is the same as the image size of the drawing title to be recognized, and at the same time perform corresponding scaling on the text box coordinates on the template drawing title;

[0036] The drawing label attribute and attribute value matching module is used to match the text boxes corresponding to the attributes in the scaled template drawing label with the text boxes on the drawing label to be recognized, and obtain the values in the text boxes matched on the drawing label to be recognized as the attribute values corresponding to the attributes, so as to achieve a one-to-one correspondence between the attributes and the attribute values.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] (1) The automatic recognition method of engineering drawing labels based on template matching provided by the present invention innovatively uses a deep learning algorithm to locate the position of the drawing label in the drawing, which is simpler than traditional image processing methods, has higher applicability and intelligence, and is not interfered by non-drawing label areas in the drawing image.

[0039] (2) The automatic recognition method of engineering drawing labels based on template matching provided by the present invention uses an image classification algorithm to classify the styles of the drawing labels to be recognized, with high classification accuracy and greater intelligence.

[0040] (3) The automatic recognition method of engineering drawing labels based on template matching provided by the present invention utilizes the position information and structural relationship between the template drawing label attributes and the attribute value text boxes, as well as the position matching information between the template drawing label attribute value text boxes and the text boxes on the drawing label to be recognized, to obtain the attribute values of the corresponding text boxes in the drawing label to be recognized, and achieve a one-to-one correspondence between the attributes and the attribute values, which is more flexible and efficient than complex attribute field rule matching or parsing the drawing label table.

[0041] (4) The automatic recognition method of engineering drawing labels based on template matching provided by the present invention is applicable to the recognition of various styles of drawing labels, and can accurately recognize irregular drawing label styles with complex structures.

[0042] (5) The automatic recognition method of engineering drawing labels based on template matching provided by the present invention has relatively low requirements for the clarity of the drawing image, and it is only necessary to accurately recognize the fonts in the drawing label table to be recognized.

[0043] (6) In the automatic recognition method of engineering drawing labels based on template matching provided by the present invention, the drawing label recognition process can be automatically executed, which reduces the labor and time costs while ensuring the accuracy, and greatly improves the intelligence level. Description of the Drawings

[0044] Figure 1 It is a flowchart of an automatic recognition method of engineering drawing labels based on template matching provided by an embodiment of the present invention;

[0045] Figure 2 It is an example diagram of a template drawing label in an embodiment of the present invention;

[0046] Figure 3 This is a sample example diagram in the drawing label template library constructed in an embodiment of the present invention. A is a template drawing label image included in this sample, and B is the label file corresponding to the template drawing label image;

[0047] Figure 4 This is a schematic diagram of the attribute value mapping between the template drawing label and the to-be-recognized drawing label after OCR in an embodiment of the present invention;

[0048] Figure 5 This is the structure diagram of the automatic recognition system for engineering drawing labels based on template matching provided by an embodiment of the present invention.

[0049] In the figure: 01, drawing label template library module; 02, drawing label position detection and cropping module; 03, drawing label classification module; 04, drawing label text recognition module; 05, template drawing label scaling module; 06, drawing label attribute and attribute value matching module. Detailed implementation manners

[0050] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. The described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0051] Term explanation:

[0052] Labelme: An open-source image annotation tool mainly used for data preparation in the field of computer vision;

[0053] PP-YOLO model: An object detection model deeply optimized and improved by the Baidu PaddlePaddle team based on YOLOv3;

[0054] PaddlePaddle (Baidu PaddlePaddle): An open-source deep learning platform developed by Baidu, providing rich APIs and tools, including but not limited to model construction, training, optimization, deployment, etc., enabling developers and researchers to more easily construct, train, and deploy deep learning models;

[0055] PaddleDetection: An open-source object detection framework based on the Baidu PaddlePaddle deep learning platform;

[0056] ResNet50: A deep residual network (Residual Network), which is a variant in the ResNet family;

[0057] PaddleClas: An open-source image classification suite under the Baidu PaddlePaddle deep learning platform;

[0058] OCR (Optical Character Recognition): Used to convert text in various types of documents (usually scanned paper documents, PDF files, or images taken by digital cameras) into machine-encoded text;

[0059] PPOCRLabel: A semi-automated graphic annotation tool designed specifically for the OCR field;

[0060] DB model (Differentiable Binarization): An algorithm for scene text detection. The core idea is to integrate the binarization process into the segmentation network, enabling the threshold to be predicted adaptively at various positions in the image, thus better distinguishing foreground and background pixels;

[0061] PaddleOCR: An open-source OCR tool library developed based on the Baidu PaddlePaddle deep learning framework;

[0062] CRNN (Convolutional Recurrent Neural Network): A deep learning model mainly used for sequence text recognition tasks in images;

[0063] OpenCV (Open Source Computer Vision Library): A cross-platform computer vision and machine learning software library initiated and open-sourced by Intel Corporation;

[0064] cv2.findContours method: A method in the OpenCV library used to find contours in a binary image.

[0065] It should be noted that in this embodiment, CAD engineering drawings collected by a certain engineering design institute are used as an example. The original drawings are drawn by AutoCAD software, and more than 1000 copies are collected in total, all preprocessed into drawing images in PNG or TIF format.

[0066] Refer to Figure 1 , this embodiment provides an automatic recognition method for engineering drawing titles based on template matching, specifically including the following steps:

[0067] Step S101: Based on the existing drawing label format, establish a drawing label template library. The template library shall at least include template drawing label images and corresponding label files. The label files shall at least include the label names and position coordinates of the template drawing label attributes and the corresponding attribute value text boxes, as well as the size information of the template drawing label.

[0068] Specifically, in this embodiment, all the drawings are from a certain engineering design institute. The existing drawing label formats of this engineering design institute are sorted out and summarized, and a total of 12 template drawing labels are collected. The image of each template drawing label is processed in the following way:

[0069] (1) Process the corresponding drawings containing each template drawing label into drawing images in PNG or TIF format;

[0070] (2) Use the Labelme annotation tool to mark the position coordinates of the template drawing label on the drawing image;

[0071] (3) According to the position coordinate information, use OpenCV to crop the drawing image to obtain the template drawing label image.

[0072] In this embodiment, each template drawing label image is saved in PNG format, and it can also be saved in TIF format, with high clarity and complete detailed information, and the fonts in the image can be accurately recognized. Figure 2 The figure shows an example of one of the template drawing label images.

[0073] Furthermore, on the 12 collected template drawing label images, use the Labelme annotation tool to mark the attribute text boxes and the corresponding attribute value text boxes in each template drawing label image, and obtain the four coordinate points of the rectangle text boxes of the attributes and the corresponding attribute values, denoted as where represents the coordinates of the point at the lower left corner of the rectangle text box, represents the coordinates of the point at the upper right corner, and so on. It should be noted that in this embodiment, the X-axis direction refers to the horizontal direction of the image, the positive X-axis direction refers to the direction from the left to the right of the image, and the negative X-axis direction refers to the direction from the right to the left of the image; the Y-axis direction refers to the vertical direction of the image, the positive Y-axis direction refers to the direction from the bottom edge to the top edge of the image, and the negative Y-axis direction refers to the direction from the top edge to the bottom edge of the image.

[0074] Furthermore, the marked attributes include: project name, stage, specialty, drawing name, drawing number, formation date, designer, approver, verifier, reviewer, checker. At the same time, it is necessary to obtain the size information of the template drawing label image, that is, the width and height of the image. As Figure 3 shown, this is a sample in the drawing label template library created in this embodiment, which includes a template drawing label image ( Figure 3 A in), and the label file corresponding to this template drawing label image (Figure 3 In B) of [reference], the label file contains the label names of each attribute and the corresponding attribute values, the position coordinate information of their rectangular text boxes, and the width and height information of the template drawing title image. It should be understood that taking the attribute of "project name" as an example, in the label file, "project name_k" represents the label name of the attribute text box, and "project name_v" represents the label name of the corresponding attribute value text box.

[0075] It should be noted that there are some slight differences in the attribute names of the drawing titles of some drawings. For example, the attribute name of "designer" is called "design" on the drawing titles of some drawings, and "checker" is also called "check". For these different names referring to the same attribute, marks need to be made on the drawing title. As Figure 3 shown in B) of [reference], add a column of "synonymous attribute reference" in the label file to mark different names referring to the same attribute. For example, in this embodiment, "formation date" and "date" both refer to the attribute of "formation date", which facilitates better correspondence between the text boxes of the to-be-recognized drawing title and the template drawing title in the subsequent process.

[0076] Step S102, make a drawing title detection data set, and use the object detection algorithm to train and obtain a drawing title position detection model. Specifically, it includes the following steps:

[0077] (1) Make a drawing title detection data set;

[0078] Select 1000 drawing images from the above-mentioned collected drawings preprocessed into PNG format, use the Labelme annotation tool to perform rectangular frame annotation on the positions of the drawing titles in the drawing images, and obtain the position coordinates of the drawing title rectangular frames. After the annotation is completed, make a drawing title detection data set. The data set contains 1000 drawing images and the label files corresponding to each drawing image, which contain the drawing title position information, and are organized in the format required by the model training framework. Finally, randomly divide the data set into 600 training sets, 200 validation sets, and 200 test sets.

[0079] (2) Use the object detection algorithm to train and obtain a drawing title position detection model.

[0080] The object detection algorithm used in this example is the PP-YOLO model algorithm, the training framework used is PaddlePaddle (Baidu PaddlePaddle), and at the same time, the end-to-end object detection suite PaddleDetection of the PaddlePaddle framework is used.

[0081] Specifically, before the training starts, configure the training parameters of the PP-YOLO model, including the learning rate, the number of iterations, etc. In this embodiment, the backbone network of the model is ResNet101, and the optimizer selected is the momentum gradient descent algorithm. Based on the Stochastic Gradient Descent (SGD), the first-order momentum (Momentum) is introduced to increase inertia. The initial learning rate is set to 0.005, the momentum is 0.9, the decay weight is 0.0001, and the number of iterations is set to 1000. During the model training process, at the end of each iteration, the model is evaluated on the validation set to monitor its performance. After the training is completed, the test set is used to conduct the final evaluation of the model. The results show that the detection accuracy of this model reaches 98%, which can be used for the subsequent position detection of the to-be-identified labels.

[0082] Step S103: Make a label classification dataset and use an image classification algorithm to train and obtain a label classification model.

[0083] (1) Make a label classification dataset;

[0084] Utilize the position coordinate information of the label rectangular frames obtained in step S102, and use OpenCV to crop the drawing images to obtain the label images corresponding to each drawing. A total of 1000 label images can be collected. The collected label images are augmented with data in the form of rotation transformation. More specifically: randomly select 50% of the image samples and randomly rotate these samples by 90°, 180°, or 270° to generate new sample images. After processing, the label samples can be expanded to 1500. According to the template format in the label template library constructed in step S101, manually classify and label the template formats to which the 1500 augmented label images belong to make a label classification dataset. The said label classification dataset includes 1500 label images and a label file, and the label file records the file name of each label image and the corresponding category label. Finally, randomly divide the dataset into 1000 training sets, 200 validation sets, and 300 test sets.

[0085] It should be noted that data augmentation is an effective method to expand the data scale. Without substantially increasing the data, more features can be learned from limited data. It can reduce the imbalance between different samples, enhance the learnable features, and make the trained model have stronger generalization ability. In this embodiment, the purpose of rotating the labels is to ensure that the labels can be correctly classified under the deflection conditions (90°, 180°, or 270°).

[0086] (2) Use an image classification algorithm to train and obtain a label classification model.

[0087] In this example, the image classification algorithm used is the ResNet50 model algorithm, the training framework used is PaddlePaddle, and the image classification suite PaddleClas of the PaddlePaddle framework is also used. Before the training starts, configure the training parameters of the ResNet50 model, including the learning rate, the number of iterations, etc. In this embodiment, the initial learning rate is set to 0.001, the number of iterations is set to 500, the optimizer selected is the momentum gradient descent algorithm, the momentum is 0.9, and the decay weight is 0.0001. The model training process is similar to that in step S102 and will not be elaborated here. Finally, the obtained label classification model has a classification accuracy rate of 99% and can be used for the classification of the subsequent labels to be recognized.

[0088] Step S104: Make a label text detection dataset, use an object detection algorithm to train a label text detection model, and combine the label text detection model with an optical character recognition (OCR) model to obtain a two-stage OCR label text recognition model. The specific steps are as follows:

[0089] (1) Make a label text detection dataset, use an object detection algorithm to train a label text detection model, including:

[0090] 1) Make a label text detection dataset;

[0091] For the 1000 label images obtained in step S103, use the PPOCRLabel annotation tool to perform rectangular box annotation on the text in the label images. First, perform rectangular box annotation on the text through the automatic annotation mode, and then supplement the un-detected parts through manual annotation. Finally, generate two label files for text detection and optical character recognition, which are used to make the label text detection dataset. Specifically, the label text detection dataset contains 1000 folders, each folder contains a label image, as well as the corresponding text detection label file and optical character recognition label file. The text detection label file contains the position information of each text rectangular box in the label image, that is, the coordinates of the text rectangular box, and the optical character recognition label file contains the text content corresponding to the detected text rectangular box. Finally, randomly divide the dataset into 600 training sets, 200 validation sets, and 200 test sets.

[0092] 2) Use an object detection algorithm to train a label text detection model.

[0093] In this example, the object detection algorithm used is the DB model algorithm, the training framework used is PaddlePaddle, and the text recognition suite PaddleOCR of the Paddle framework is also used. Before the training starts, configure the training parameters of the DB model, including the learning rate, the number of iterations, etc. In this embodiment, the backbone network of the DB model algorithm uses ResNet50 combined with FPN, the optimizer uses the Adam algorithm, the initial learning rate is set to 0.001, and the number of iterations is set to 1200. The model training process is similar to that in step S102 and will not be elaborated here. Finally, the obtained label text detection model has a text detection accuracy rate of over 99%.

[0094] (2) Combine the label text detection model with the text recognition model to obtain a two-stage OCR label text recognition model.

[0095] In this embodiment, the text recognition model uses the CRNN model provided in PaddleOCR. Combine the trained label text detection model with the text recognition model to finally obtain a two-stage OCR label text recognition model.

[0096] It should be noted that generally, OCR adopts a two-stage algorithm. The first stage is text detection, and the second stage is text recognition. First, detect the position of the text box in the image, and then recognize the text inside the text box. Currently, there are also end-to-end OCR algorithms, but due to their low accuracy and the more mature two-stage OCR algorithm, the two-stage OCR algorithm is adopted. For the two-stage OCR algorithm, the recognition accuracy mainly depends on the detection of the text box. Therefore, in this embodiment, the text detection model in the two-stage OCR algorithm is trained separately.

[0097] So far, this embodiment has completed the relevant work of constructing the initial label template library and model training. Next, execute the label recognition process.

[0098] Step S105, use the label position detection model to detect the target drawing, obtain the position coordinate information of the label of the target drawing; then crop the drawing through the position coordinates of the label to obtain the label image to be recognized.

[0099] Specifically, it includes the following steps:

[0100] (1) Preprocess the target drawing into a drawing image in PNG or TIF format;

[0101] (2) Use the label position detection model trained in step S102 to detect the position of the label in the target drawing, and the model outputs the position coordinate information of the label detected from the drawing;

[0102] (3) According to the position coordinate information, use OpenCV to crop the drawing image to obtain the title block image to be recognized, and use the cv2.findContours method to detect the outermost rectangle in the title block image, that is, the title block table, and then further crop and remove the blank area around the title block table.

[0103] Step S106, use the title block classification model to classify the template styles of the title block to be recognized, and determine the classification of the title block to be recognized in the title block template library.

[0104] Specifically, use the title block classification model trained in step S103 to classify the title block image to be recognized obtained in step S105, and obtain the category of the title block image to be recognized and the template title block image in the corresponding title block template library. In this embodiment, the template title block style in the title block template library corresponding to the title block to be recognized is as Figure 3 shown as A in.

[0105] Further, if the title block image to be recognized is deflected, the title block image to be recognized must be adjusted to the positive direction. It should be noted that the title block image to be recognized obtained in S105 is likely to have three deflection situations, such as deflection by 90°, 180°, or 270°. Image deflection will affect the subsequent text recognition effect, so the title block image needs to be adjusted to the positive direction. It should be noted that generally, the uploaded drawing file will be uploaded in the correct orientation, so the situation of title block deflection is not common. PaddleOCR comes with a text direction classifier, but it only supports the determination of deflection angles of 0° and 180°. It should be understood that in the vast majority of cases, the style of the title block table is rectangular, and no title block image with a square shape has been found in this example. Specifically, in this example, first compare the aspect ratios of the length and width of the title block image to be recognized and the template title block image. Assume that the length of an image in the X-axis direction is h, the width in the Y-axis direction is w, and the aspect ratio is r. If h > w, then r > 1; if h < w, then r < 1. If the aspect ratio ratios of the title block image to be recognized and the template title block image are both greater than 1 or both less than 1, it means that the title block image to be recognized may have a deflection angle of 0° or 180° (deflection of 0° means it is in the positive direction and no angle correction is required). By using Chinese font direction detection, it can be determined whether the deflection angle of the image is 180° or 0°. If it is 180°, flipping the image by 180° can obtain the positive title block image to be recognized. If the aspect ratio ratios of the title block image to be recognized and the template title block image are one greater than 1 and the other less than 1, it means that the title block image to be recognized may have a deflection angle of 90° or 270°. At this time, flip the title block image to be recognized by 90° to change the deflection angle to 0° or 180°, and the subsequent processing method is the same as above.

[0106] Step S107, use the label text recognition model to perform OCR recognition on the label image to be recognized, and obtain the position coordinates and text content of the text boxes detected in the label to be recognized.

[0107] Specifically, use the label text recognition model obtained in step S104 to perform OCR on the label image to be recognized obtained in step S106. The model outputs the text content and position coordinates of the text boxes detected in the label image.

[0108] Step S108, scale the size of the template label so that the image size of the template label is the same as the image size of the label to be recognized, and at the same time scale the text box coordinates on the template label accordingly.

[0109] Specifically, in this example, use the resize function of OpenCV to adjust the size of the template label image to be the same as the size of the label image to be recognized. If the size of the label image to be recognized is , and the size of the template label image is , where H and H m respectively represent the heights of the corresponding images, and W and W m respectively represent the widths of the corresponding images, then the width scaling ratio of the template label is , and the height scaling ratio is . For each text box coordinate point (x, y) in the template label image, use the scaling ratio for corresponding adjustment, and the adjusted coordinate point is (x , y ).

[0110] It should be noted that in this embodiment, it is preferred to scale the template label image because the template label image has high clarity and complete detailed information. Even after scaling, the fonts in the image can still be accurately recognized.

[0111] Step S109, match the text boxes corresponding to the attribute values of each attribute in the scaled template label with the text boxes on the label to be recognized, and obtain the values in the text boxes matched on the label to be recognized as the attribute values corresponding to the attributes, so as to achieve a one-to-one correspondence between the attributes and the attribute values.

[0112] It should be understood that the correspondence between attributes and attribute values in the to-be-recognized label is unknown, while the correspondence between attributes and attribute values in the template label is known. Therefore, as long as the text box corresponding to an attribute in the template label matches a text box on the to-be-recognized label, the value in the text box of the to-be-recognized label is taken as the attribute value corresponding to the attribute. Specifically, based on the position coordinates and text content of the text boxes of the to-be-recognized label detected by the label text recognition model, it is determined one by one whether the center point of each attribute value text box in the scaled template label image falls within a text box of the to-be-recognized label image. If the center point falls within a text box, the text content in the text box is determined as the attribute value corresponding to the corresponding attribute, that is, the attribute value corresponding to the corresponding attribute in the to-be-recognized label, thereby realizing the one-to-one correspondence between attributes and attribute values in the to-be-recognized label. Figure 4 This is a schematic diagram of the attribute value mapping between the template label and the to-be-recognized label after OCR in this embodiment.

[0113] More specifically, the method for determining whether the center point of the attribute value text box in the scaled template label image falls within a text box of the to-be-recognized label is as follows: Assume that the coordinates of a text box M in the scaled template label image are and the coordinates of its center point are , where , . For all text boxes in the to-be-recognized label, if there exists a text box N with coordinates and that satisfies and , then the center point of text box M falls within text box N, text box M matches text box N, and the value in text box N is the attribute value corresponding to text box M. Similarly, the text box of the to-be-recognized label corresponding to the attribute value text box of other template labels is determined using the center point position to obtain all attribute values.

[0114] It should be noted that in this embodiment, the method of mapping the center point of the template drawing title text box to the to-be-recognized drawing title is preferably selected, which is hereinafter referred to as "forward mapping". The reason is that in the template drawing title, the positions and structural information of the attributes and the corresponding attribute value text boxes are known. As long as the landing positions of the center points of the attribute value text boxes corresponding to all attributes on the to-be-recognized drawing title are determined, the one-to-one correspondence between the attributes and the attribute values can be completed, thereby saving computing resources and time and improving efficiency. If "reverse mapping", that is, mapping the center point of the to-be-recognized drawing title text box to the template drawing title, it is necessary to traverse and determine the landing positions of the center points of all text boxes on the to-be-recognized drawing title on the template drawing title, and the efficiency is relatively low. It should be understood that reverse mapping can also be performed when there is sufficient time resource; in addition, when performing forward mapping, in addition to determining the landing position of the center point of the attribute value text box on the template drawing title on the to-be-recognized drawing title, it can also be further verified by determining the landing position of the center point of the attribute text box on the to-be-recognized drawing title.

[0115] Furthermore, this embodiment also provides an automatic recognition system for engineering drawing titles based on template matching, as Figure 5 shown, including:

[0116] A drawing title template library module 01, which is used to establish a drawing title template library based on the existing drawing title formats; the template library at least includes a template drawing title image and a corresponding label file, and the label file at least includes the label names and position coordinates of the template drawing title attributes and the corresponding attribute value text boxes, as well as the size information of the template drawing title;

[0117] A drawing title position detection and cropping module 02, which is used to detect a target drawing using a drawing title position detection model, obtain the position coordinates of the drawing title of the target drawing; crop the target drawing through the position coordinates to obtain a to-be-recognized drawing title;

[0118] A drawing title classification module 03, which is used to classify the to-be-recognized drawing title into the template styles of the drawing title template library using a drawing title classification model, and determine the classification to which the to-be-recognized drawing title belongs in the drawing title template library;

[0119] A drawing title text recognition module 04, which is used to perform OCR recognition on the to-be-recognized drawing title using a drawing title text recognition model, and obtain the position coordinates and text content of the detected text boxes in the to-be-recognized drawing title;

[0120] A template drawing title scaling module 05, which is used to scale the size of the template drawing title so that the image size of the template drawing title is the same as the image size of the to-be-recognized drawing title, and at the same time scale the text box coordinates on the template drawing title accordingly;

[0121] The module 06 for corresponding the drawing label attributes and attribute values is used to match the text boxes corresponding to the attributes in the scaled drawing label with the text boxes on the drawing label to be recognized, obtain the values in the text boxes matched on the drawing label to be recognized as the attribute values corresponding to the attributes, and realize the one-to-one correspondence between the attributes and the attribute values.

[0122] For the detailed descriptions of the modules in the system, please refer to the relevant descriptions corresponding to the above method embodiments, which will not be elaborated here.

Claims

1. An automatic recognition method for the title block of engineering drawings based on template matching, characterized in that, It includes the following steps: S101, based on the existing drawing label format, establish a drawing label template library; the template library at least includes template drawing label images and corresponding label files, and the label files at least include the label names and position coordinates of template drawing label attributes and corresponding attribute value text boxes, as well as the size information of the template drawing label; S102, make a drawing label detection data set, and use an object detection algorithm to train and obtain a drawing label position detection model; S103, make a drawing label classification data set, and use an image classification algorithm to train and obtain a drawing label classification model; S104, make a drawing label text detection data set, use an object detection algorithm to train and obtain a drawing label text detection model, and combine it with an optical character recognition model to obtain a drawing label text recognition model; S105, use the drawing label position detection model to detect the target drawing, and obtain the position coordinates of the drawing label of the target drawing; crop the target drawing through the position coordinates to obtain the drawing label to be recognized; S106, use the drawing label classification model to classify the drawing label template styles of the drawing label to be recognized, and determine the classification of the drawing label to be recognized in the drawing label template library; S107, use the drawing label text recognition model to perform OCR recognition on the drawing label to be recognized, and obtain the position coordinates and text content of the text boxes detected in the drawing label to be recognized; S108, scale the size of the template drawing label so that the image size of the template drawing label is the same as that of the drawing label to be recognized, and at the same time scale the coordinates of the text boxes on the template drawing label accordingly; S109, match the attribute value text boxes corresponding to each attribute in the scaled template drawing label with the text boxes on the drawing label to be recognized, and obtain the values in the text boxes matched on the drawing label to be recognized as the attribute values corresponding to the attributes, so as to achieve a one-to-one correspondence between the attributes and the attribute values, specifically including: Based on the position coordinates and text content of the text boxes detected by the drawing label text recognition model, determine one by one the landing positions of the centers of the attribute value text boxes corresponding to each attribute in the scaled template drawing label on the drawing label to be recognized, and obtain the text content in the text boxes where the landing positions are located.

2. The automatic recognition method for engineering drawing titles based on template matching according to claim 1, characterized in that In step S101, the attributes include multiple or all of project name, stage, specialty, drawing name, drawing number, formation date, designer, approver, checker, reviewer, checker, etc., and the size information includes the width and height of the template drawing label image; the label file also includes synonymous attribute reference information.

3. The automatic identification method for the title block of engineering drawings based on template matching according to claim 1, characterized in that In step S102, the object detection algorithm uses the PP-YOLO model algorithm, and uses the PaddlePaddle training framework and its end-to-end object detection suite PaddleDetection to train the model to obtain the drawing label position detection model.

4. The automatic recognition method for the title block of engineering drawings based on template matching according to claim 1, characterized in that In step S103, the image classification algorithm uses the ResNet50 model algorithm, and uses the PaddlePaddle training framework and its image classification suite PaddleClas to train the model to obtain the drawing label classification model.

5. The automatic recognition method for the title block of engineering drawings based on template matching according to claim 1, wherein In step S104, the target detection algorithm adopts the DB model algorithm, and uses the PaddlePaddle training framework and its text recognition suite PaddleOCR to train the model to obtain the label text detection model; the text recognition model uses the CRNN model provided in PaddleOCR.

6. The automatic recognition method for the title block of engineering drawings based on template matching according to claim 1, characterized in that, Step S106 further includes: If the to-be-recognized label image is deflected, compare the aspect ratios of the to-be-recognized label image and the template label image, and adjust it to be upright. Among them, the deflection includes 90°, 180°, or 270°, and the aspect ratio is the ratio of the length of the image in the X-axis direction to the width in the Y-axis direction, including the following steps: (1) If the aspect ratios of the to-be-recognized label image and the template label image are both greater than 1 or both less than 1, judge whether the deflection angle is 180° or 0° through Chinese font direction detection. If it is 180°, flip the to-be-recognized label image by 180°. If it is 0°, there is no need to correct the angle; (2) If the aspect ratio of the to-be-recognized label image and the template label image is one greater than 1 and the other less than 1, flip the to-be-recognized label image by 90° to make its deflection angle become 180° or 0°. Judge whether the deflection angle is 180° or 0° through Chinese font direction detection. If it is 180°, flip the to-be-recognized label image by 180°. If it is 0°, there is no need to correct the angle.

7. The automatic recognition method for engineering drawing titles based on template matching according to claim 1, characterized in that In step S108, the scaling of the size of the template label includes using the resize function of OpenCV to adjust the image size of the template label to be the same as that of the to-be-recognized label.

8. The automatic recognition method for the title block of engineering drawings based on template matching according to claim 1, characterized in that In step S109, the method for determining the landing position of the center point of the attribute value text box in the scaled template label on the to-be-recognized label includes the following steps: (1) Obtain the center point of an attribute value text box M in the scaled template drawing label, denoted as ; (2)Traverse all the text boxes in the to-be-recognized label. If there exists a text box N with coordinates and that satisfies and , then it is determined that the center point of the text box M falls into the text box N of the to-be-recognized label.

9. An automatic recognition system for engineering drawing titles based on template matching, which is used to implement the automatic recognition method for engineering drawing titles based on template matching as described in any one of claims 1-8, characterized in that, Including: The label template library module is used to establish a label template library based on the existing label format; The label position detection and cropping module is used to detect the target drawing using the label position detection model, obtain the position coordinates of the label of the target drawing; crop the target drawing through the position coordinates to obtain the to-be-recognized label; The label classification module is used to classify the to-be-recognized label into the label template style using the label classification model, and determine the classification of the to-be-recognized label in the label template library; The label text recognition module is used to perform OCR recognition on the to-be-recognized label using the label text recognition model, and obtain the position coordinates and text content of the detected text box in the to-be-recognized label; The template label scaling module is used to scale the size of the template label so that the image size of the template label is the same as that of the to-be-recognized label, and at the same time scale the text box coordinates on the template label accordingly; The label attribute and attribute value matching module is used to match the attribute value text boxes corresponding to each attribute in the scaled template label with the text boxes on the to-be-recognized label, and obtain the values in the text boxes matched on the to-be-recognized label as the attribute values corresponding to the attributes, so as to achieve a one-to-one correspondence between attributes and attribute values.

Images