Engineering drawing label automatic identification method and system based on template matching

Abstract

The invention discloses an engineering drawing label automatic identification method and system based on template matching. According to the method, an existing drawing is utilized to make a drawing label template library, and a drawing label position detection model, a drawing label classification model and a drawing label text recognition model are trained respectively; for the target drawing, using the drawing label position detection model to position the coordinate position of the drawing label, and cutting to obtain a to-be-identified drawing label; determining a picture label template style to which a to-be-recognized picture label belongs by using the picture label classification model, and acquiring text content and position coordinates of a textbox in the to-be-recognized picture label by using the picture label text recognition model; and acquiring an attribute value of the corresponding textbox in the to-be-identified signature by utilizing the position information and the structural relationship between the template signature attribute and the attribute value textbox and the drop point position of the central point of the attribute value textbox in the to-be-identified signature, and realizing the correspondence between the attribute and the attribute value. According to the invention, accurate identification of the drawing label is realized in a more intelligent and efficient manner, and manpower and time costs are reduced while the precision is ensured.

Description

Technical Field

[0001] The present invention belongs to the technical field of image recognition, and specifically relates to a method and system for automatic recognition of engineering drawing labels based on template matching. The technical method is suitable for automatic recognition of drawing label columns in the engineering field. Background Art

[0002] With the continuous expansion of the scale of engineering projects and the widespread application of computer-aided design (CAD) technology, as an indispensable part of engineering design, the output of engineering drawings is increasing in quantity and complexity. Engineering drawings are the core output of the engineering design process and are important documents in engineering projects. The label area of ​​the drawing usually contains basic information such as project name, drawing number, version, drawing unit, date, etc., which has great reference value for drawing classification, storage and management. At present, manual recognition methods are generally used to extract, check and enter label information from a large number of drawings, and then classify the label information and store the drawings in the form of folders. This traditional manual entry method is very cumbersome, inefficient and error-prone, resulting in low quality of drawing filing and inability to achieve efficient retrieval and management. Therefore, it is urgent to develop an automated recognition method for drawing labels.

[0003] The automatic recognition of drawing labels is mainly divided into two steps: label position detection and label table recognition. The difficulty of label position detection lies in the fact that there is no strict and unified specification for label layout, and the position on the drawing is not fixed, so it is difficult to accurately locate and recognize the label frame; the difficulty of label table recognition lies in the fact that the types of labels are diverse, the formats are different, the table structure is relatively complex, and there are large differences in the names of attribute fields in the table. For detecting the position of label, most of the current methods use complex image processing technology combined with rule judgment to locate the label in the drawing, such as Chinese patents CN117711010A, CN118410196A, and CN118887696A. However, this method has high requirements on the clarity of the drawing image, the processing process is relatively complicated, and it will also be interfered by other rectangular frames in the drawing, resulting in low accuracy of label detection. The difficulty of label table recognition lies in the correspondence between attributes and attribute values ​​in the label column. Currently, most methods use rule matching to extract attribute key fields from the text content obtained after OCR (Optical Character Recognition) processing of label images. Although OCR technology has become mature and image text can basically achieve high-precision recognition in complex scenes, due to the variety of label formats, the lack of uniform attribute names, and the complex table structure of a single label, OCR processing can only obtain the content and position coordinate information of the text area in the image, and cannot obtain the logical structural relationship between these text areas, resulting in the inability to well correspond the attributes in the label catalog with the attribute values. Summary of the invention

[0004] In view of the defects existing in the prior art, the purpose of the present invention is to provide a method and system for automatic identification of engineering drawing labels based on template matching. The present invention first uses existing drawings to create a label style template library, and trains a label position detection model, a label classification model, and an OCR label text recognition model respectively; for the target drawing, the label position detection model is first used to locate the coordinate position of the label in the target drawing, and the drawing is cropped to obtain the label image to be identified; for the acquired label image, the label classification algorithm is used to determine the label template to which the label to be identified belongs, and the label text recognition model is used to obtain the content and coordinate position of the text box in the label image; according to the position information and structural relationship of the text area in the template label, the attribute value text box corresponding to the attribute in the template label is determined by the landing point position of the center point of the attribute value text box in the label image to be identified, thereby realizing the correspondence between the attribute and the attribute value, and achieving the purpose of accurately identifying the label.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] In one aspect, the present invention provides a method for automatically identifying engineering drawing labels based on template matching, comprising the following steps:

[0007] S101, based on the existing label format, establish a label template library; the template library at least includes a template label image and a corresponding label file, the label file at least includes the label name and position coordinates of the template label attribute and the corresponding attribute value text box, and the size information of the template label;

[0008] S102, preparing a drawing label detection dataset, using a target detection algorithm, and training a label position detection model;

[0009] S103, preparing a label classification data set, using an image classification algorithm, and training a label classification model;

[0010] S104, preparing a label text detection data set, using a target detection algorithm, training a label text detection model, and combining it with a text recognition model to obtain a label text recognition model;

[0011] S105, using the label position detection model to detect the target drawing, and obtaining the position coordinates of the label on the target drawing; cutting the target drawing according to the position coordinates to obtain the label to be identified;

[0012] S106, using the image tag classification model to classify the image tag template style to determine the category to which the image tag to be identified belongs in the image tag template library;

[0013] S107, using the image tag text recognition model to perform OCR recognition on the image tag to be recognized, and obtain the position coordinates and text content of the text box detected in the image tag to be recognized;

[0014] S108, scaling the size of the template image label so that the image size of the template image label is consistent with the image size of the image label to be identified, and scaling the coordinates of the text box on the template image label accordingly;

[0015] S109, matching the attribute value text box corresponding to each attribute in the scaled template icon with the text box on the icon to be identified, obtaining the value in the matched text box on the icon to be identified as the attribute value corresponding to the attribute, and realizing a one-to-one correspondence between the attribute and the attribute value.

[0016] Furthermore, in step S101, the attributes include multiple or all of the project name, phase, specialty, drawing name, drawing number, creation date, designer, approver, verifier, reviewer, and checker, and the size information includes the width and height of the template drawing label image; the label file also includes synonymous attribute reference information.

[0017] Furthermore, in step S102, the target detection algorithm adopts the PP-YOLO model algorithm, and uses the PaddlePaddle training framework and its end-to-end target detection suite PaddleDetection to train the model to obtain the image label position detection model.

[0018] Furthermore, in step S103, the image classification algorithm adopts the ResNet50 model algorithm, and the model is trained using the PaddlePaddle training framework and its image classification suite PaddleClas to obtain the image label classification model.

[0019] Furthermore, in step S104, the target detection algorithm adopts the DB model algorithm, and uses the PaddlePaddle training framework and its text recognition suite PaddleOCR to train the model to obtain the image label text detection model; the text recognition model uses the CRNN model provided in PaddleOCR.

[0020] Furthermore, the step S106 further includes:

[0021] If the image of the image label to be identified is deflected, the aspect ratios of the image label to be identified and the template image label are compared and adjusted to a positive direction, wherein the deflection includes a deflection of 90°, 180° or 270°, and the aspect ratio is a ratio of the length of the image in the X-axis direction to the width of the image in the Y-axis direction, including the following steps:

[0022] (1) If the aspect ratios of the image to be identified and the template image are both greater than 1 or less than 1, the deflection angle is determined to be 180° or 0° by detecting the direction of the Chinese font. If it is 180°, the image to be identified is flipped 180°. If it is 0°, no angle correction is required.

[0023] (2) If the aspect ratio of the image to be identified and the template image is greater than 1 and the other is less than 1, the image to be identified is flipped 90° so that its deflection angle is changed to 180° or 0°. The deflection angle is determined to be 180° or 0° through Chinese font direction detection. If it is 180°, the image to be identified is flipped 180°. If it is 0°, there is no need to correct the angle.

[0024] Furthermore, in step S108, scaling the size of the template image signature includes using the resize function of OpenCV to adjust the image size of the template image signature to be consistent with the image size of the image signature to be identified.

[0025] Furthermore, in step S109, the attribute value text box corresponding to each attribute in the scaled template image tag is matched with the text box on the image tag to be identified, and the method for obtaining the value in the matched text box on the image tag to be identified is:

[0026] Based on the position coordinates and text content of the text box detected by the label text recognition model, the center point of the attribute value text box corresponding to each attribute in the scaled template label is determined one by one on the label to be identified, and the text content in the text box where the landing point is located is obtained.

[0027] Furthermore, the method for determining the location of the center point of the attribute value text box in the scaled template icon on the icon to be identified comprises the following steps:

[0028] (1) Get the center point of an attribute value text box M in the scaled template image label, denoted as ;

[0029] (2) Traverse all text boxes in the image label to be identified. If there is a text box with coordinates and Text box N, satisfying and , then it is determined that the center point of the text box M falls into the text box N of the image label to be identified.

[0030] Another aspect of the present invention provides a system for automatically identifying engineering drawing labels based on template matching, which is used to implement the above-mentioned method for automatically identifying engineering drawing labels based on template matching, comprising:

[0031] A label template library module is used to establish a label template library based on the existing label format;

[0032] The label position detection and cutting module is used to detect the target drawing using the label position detection model to obtain the position coordinates of the label on the target drawing; cut the target drawing according to the position coordinates to obtain the label to be identified;

[0033] A label classification module is used to classify the label template style of the label to be identified using the label classification model, and determine the category to which the label to be identified belongs in the label template library;

[0034] The image label text recognition module is used to perform OCR recognition on the image label to be recognized using the image label text recognition model to obtain the position coordinates and text content of the text box detected in the image label to be recognized;

[0035] The template label scaling module is used to scale the size of the template label so that the image size of the template label is consistent with the image size of the label to be identified, and to scale the coordinates of the text box on the template label accordingly;

[0036] The module for matching attribute and attribute value of image tag is used to match the attribute value text box corresponding to each attribute in the scaled template image tag with the text box on the image tag to be identified, and obtain the value in the matched text box on the image tag to be identified as the attribute value corresponding to the attribute, so as to realize a one-to-one correspondence between the attribute and the attribute value.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] (1) The method for automatic identification of engineering drawing labels based on template matching provided by the present invention innovatively uses a deep learning algorithm to locate the position of the label in the drawing. Compared with traditional image processing methods, it is simpler, more applicable and intelligent, and is not affected by non-label areas in the drawing image.

[0039] (2) The method for automatic identification of engineering drawing labels based on template matching provided by the present invention uses an image classification algorithm to classify the styles of the labels to be identified, has a high classification accuracy, and is more intelligent.

[0040] (3) The method for automatic identification of engineering drawing labels based on template matching provided by the present invention utilizes the position information and structural relationship between the template label attributes and the attribute value text box, as well as the position matching information between the template label attribute value text box and the label text box to be identified, to obtain the attribute value of the corresponding text box in the label to be identified, thereby realizing a one-to-one correspondence between the attributes and the attribute values. This is more flexible and efficient than matching complex attribute field rules or parsing label tables.

[0041] (4) The automatic identification method of engineering drawing labels based on template matching provided by the present invention can be applied to the identification of drawing labels of various styles, and can also accurately identify irregular label styles with complex structures.

[0042] (5) The method for automatically identifying engineering drawing labels based on template matching provided by the present invention has relatively low requirements on the clarity of the drawing image, as long as the fonts in the table of the label to be identified can be accurately identified.

[0043] (6) In the method for automatic identification of engineering drawing labels based on template matching provided by the present invention, the drawing label identification process can be automatically executed, which ensures accuracy while reducing manpower and time costs, greatly improving the degree of intelligence. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 A flowchart of a method for automatically identifying engineering drawing labels based on template matching provided by one embodiment of the present invention;

[0045] Figure 2 This is an example diagram of a template signature in one embodiment of the present invention;

[0046] Figure 3 is a sample example image in the image signature template library constructed in an embodiment of the present invention, A is a template image signature image contained in the sample, and B is a label file corresponding to the template image signature image;

[0047] Figure 4 A schematic diagram of attribute value mapping between a template image signature and an image signature to be recognized after OCR in one embodiment of the present invention;

[0048] Figure 5 This is a structural diagram of an automatic recognition system for engineering drawing labels based on template matching provided in one embodiment of the present invention.

[0049] In the figure: 01, label template library module; 02, label position detection and cropping module; 03, label classification module; 04, label text recognition module; 05, template label scaling module; 06, label attribute and attribute value matching module. DETAILED DESCRIPTION

[0050] The technical scheme in the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiment of the present invention. The described embodiment is only a part of the embodiment of the present invention, not all of the embodiments. Based on the embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0051] Terminology explanation:

[0052] Labelme: is an open source image annotation tool mainly used for data preparation in the field of computer vision.

[0053] PP-YOLO model: It is a target detection model that is deeply optimized and improved by the Baidu PaddlePaddle team based on YOLOv3.

[0054] PaddlePaddle: is an open source deep learning platform developed by Baidu. It provides a rich set of APIs and tools, including but not limited to model building, training, optimization, deployment, etc., making it easier for developers and researchers to build, train and deploy deep learning models.

[0055] PaddleDetection: an open source object detection framework based on Baidu's PaddlePaddle deep learning platform;

[0056] ResNet50: is a deep residual network (Residual Network), a variant of the residual network (ResNet) family;

[0057] PaddleClas: is an open source image classification suite under Baidu's PaddlePaddle deep learning platform;

[0058] OCR (Optical Character Recognition): used to convert text in various types of documents (usually scanned paper documents, PDF files, or images taken by digital cameras) into machine-encoded text;

[0059] PPOCRLabel: is a semi-automatic graphic annotation tool designed for the OCR field;

[0060] DB model (Differentiable Binarization): is an algorithm for scene text detection. The core idea is to integrate the binarization process into the segmentation network so that the threshold can be adaptively predicted at various locations in the image, thereby better distinguishing foreground and background pixels.

[0061] PaddleOCR: is an open source OCR tool library developed based on Baidu's PaddlePaddle deep learning framework;

[0062] CRNN (Convolutional Recurrent Neural Network): is a deep learning model mainly used for sequential text recognition tasks in images;

[0063] OpenCV (Open Source Computer Vision Library): is a cross-platform computer vision and machine learning software library initiated and open sourced by Intel;

[0064] cv2.findContours method: A method in the OpenCV library that is used to find contours in a binary image.

[0065] It should be noted that this embodiment takes the CAD engineering drawings collected by a certain engineering design institute as an example. The original drawings are drawn by AutoCAD software. A total of more than 1,000 copies are collected and all are pre-processed into drawing images in PNG or TIF format.

[0066] See also Figure 1 This embodiment provides a method for automatically identifying engineering drawing labels based on template matching, which specifically includes the following steps:

[0067] Step S101, based on the existing label format, establish a label template library, the template library at least contains the template label image and the corresponding label file, the label file at least contains the label name and position coordinates of the template label attribute and the corresponding attribute value text box, and the size information of the template label.

[0068] Specifically, in this embodiment, all drawings come from a certain engineering design institute, and the existing drawing label formats of the engineering design institute are sorted and summarized, and a total of 12 template labels are collected. The image of each template label is processed in the following way:

[0069] (1) Process the corresponding drawings containing the template labels into drawing images in PNG or TIF format;

[0070] (2) Use the Labelme annotation tool to mark the position coordinates of the template label on the drawing image;

[0071] (3) Based on the position coordinate information, use OpenCV to crop the drawing image to obtain the template label image.

[0072] In this embodiment, each template image is saved in PNG format, which can also be saved in TIF format, with high definition and complete detail information, and the fonts in the image can be accurately identified. Figure 2 Shown is an example of one of the template label images.

[0073] Furthermore, on the 12 template image labels collected, the attribute text box and the corresponding attribute value text box in each template image label were marked using the Labelme annotation tool to obtain the four coordinate points of the attribute and the corresponding attribute value rectangular text box, which are expressed as ,in Represents the coordinates of the point at the lower left corner of the rectangular text box. It should be noted that in this embodiment, the X-axis direction refers to the horizontal direction of the image, the positive direction of the X-axis refers to the direction from the left side of the image to the right side, and the negative direction of the X-axis refers to the direction from the right side of the image to the left side; the Y-axis direction refers to the vertical direction of the image, the positive direction of the Y-axis refers to the direction from the bottom side of the image to the top side, and the negative direction of the Y-axis refers to the direction from the top side of the image to the bottom side.

[0074] Furthermore, the marked attributes include: project name, stage, specialty, drawing name, drawing number, date of creation, designer, approver, verifier, reviewer, and checker. At the same time, it is necessary to obtain the size information of the template drawing label image, that is, the width and height of the image. Figure 3 As shown in FIG. 1 , a sample of the image signature template library created in this embodiment includes a template image signature ( Figure 3 A in the template), and the label file corresponding to the template image ( Figure 3 B in the figure), the label file contains the label names of each attribute and the corresponding attribute value, the position coordinate information of their rectangular text boxes, and the width and height information of the template label image. It should be understood that, taking the attribute "project name" as an example, in the label file, "project name_k" represents the label name of the attribute text box, and "project name_v" represents the label name of the corresponding attribute value text box.

[0075] It should be noted that some drawing labels have slightly different attribute names. For example, the attribute name "Designer" may be called "Design" on some drawing labels, and "Verifier" may also be called "Verify". For these different names referring to the same attribute, it is necessary to mark them on the drawing label, such as Figure 3 As shown in B, a column "Synonymous Attribute Reference" is added to the label file to mark different names referring to the same attribute. For example, "Formation Date" and "Date" in this embodiment both refer to the attribute "Formation Date", so that the text boxes of the subsequent to-be-identified graphic label and the template graphic label can be better matched.

[0076] Step S102, creating a drawing label detection dataset, using a target detection algorithm, and training a label position detection model. Specifically, the following steps are included:

[0077] (1) Create a dataset for drawing label detection;

[0078] Select 1000 drawing images from the drawings collected and preprocessed into PNG format, use the Labelme annotation tool to annotate the position of the label in the drawing image with a rectangular box, and obtain the position coordinates of the rectangular box of the label. After the annotation is completed, a label detection dataset is produced, which contains 1000 drawing images and a label file corresponding to each drawing image containing the label position information, and is organized in the format required by the model training framework. Finally, the dataset is randomly divided into 600 training sets, 200 validation sets, and 200 test sets.

[0079] (2) Use the target detection algorithm to train a label location detection model.

[0080] The target detection algorithm used in this example is the PP-YOLO model algorithm, the training framework used is PaddlePaddle (Baidu Paddle), and the end-to-end target detection suite PaddleDetection of the PaddlePaddle framework is used.

[0081] Specifically, before the training starts, the training parameters of the PP-YOLO model are configured, including the learning rate, number of iterations, etc. In this embodiment, the backbone network of the model adopts ResNet101, and the optimizer selects the momentum gradient descent algorithm. On the basis of the stochastic gradient descent algorithm (SGD), the first-order momentum is introduced to increase the inertia. The initial learning rate is set to 0.005, the momentum is 0.9, the attenuation weight is 0.0001, and the number of iterations is set to 1000. During the model training process, an evaluation is performed on the validation set at the end of each iteration to monitor the performance of the model. After the training is completed, the model is finally evaluated using the test set. The results show that the detection accuracy of the model reaches 98%, which can be used for the subsequent position detection of the image labels to be identified.

[0082] Step S103, creating a label classification data set, using an image classification algorithm, and training to obtain a label classification model.

[0083] (1) Create a dataset for image label classification;

[0084] Using the coordinate information of the rectangular frame position of the image label obtained in step S102, OpenCV is used to cut the drawing image to obtain the image label image corresponding to each drawing, and a total of 1000 image label images can be collected. The collected image label images are augmented by rotation transformation. More specifically, 50% of the image samples are randomly selected, and these samples are randomly rotated by 90°, 180° or 270° to generate new sample images. After processing, the image label samples can be expanded to 1500. According to the template format in the image label template library constructed in step S101, the template format to which the expanded 1500 image label images belong is manually classified and marked to produce an image label classification data set. The image label classification data set includes 1500 image label images and a label file, and the file name and corresponding category label of each image label image are recorded in the label file. Finally, the data set is randomly divided into 1000 training sets, 200 validation sets, and 300 test sets.

[0085] It should be noted that data augmentation is an effective method to expand the data scale, which can learn more features from limited data without substantially increasing the data. It can reduce the imbalance between different samples, enhance the learnable features, and make the trained model have stronger generalization ability. The purpose of rotating the label in this embodiment is to ensure that the label can be correctly classified even when it is deflected (90°, 180° or 270°).

[0086] (2) Use the image classification algorithm to train the image label classification model.

[0087] The image classification algorithm used in this example is the ResNet50 model algorithm, and the training framework used is PaddlePaddle, while the image classification suite PaddleClas of the PaddlePaddle framework is used. Before the start of training, configure the training parameters of the ResNet50 model, including the learning rate, number of iterations, etc. In this embodiment, the initial learning rate is set to 0.001, the number of iterations is set to 500, and the optimizer selects the momentum gradient descent algorithm with a momentum of 0.9 and a decay weight of 0.0001. The model training process is similar to step S102 and will not be repeated here. The image label classification model finally obtained through training has a classification accuracy of 99%, which can be used for the classification of subsequent image labels to be identified.

[0088] Step S104, prepare a label text detection data set, use the target detection algorithm to train a label text detection model, combine the label text detection model with the text recognition model, and obtain a two-stage OCR label text recognition model. Specifically, the following steps are included:

[0089] (1) Create a dataset for label text detection and use the target detection algorithm to train a label text detection model, including:

[0090] 1) Create a dataset for image label text detection;

[0091] For the 1000 label images obtained in step S103, use the PPOCRLabel annotation tool to annotate the text in the label images with rectangular frames. First, annotate the text with rectangular frames using the automatic annotation mode, and then annotate the undetected parts using the manual annotation method. Finally, two label files, text detection and text recognition, are generated to make a label text detection data set. Specifically, the label text detection data set contains 1000 folders, each of which contains a label image, and corresponding text detection label files and text recognition label files, wherein the text detection label file contains the position information of each text rectangular frame in the label image, i.e., the coordinates of the text rectangular frame, and the text recognition label file contains the text content corresponding to the detected text rectangular frame. Finally, the data set is randomly divided into 600 training sets, 200 validation sets, and 200 test sets.

[0092] 2) Use the target detection algorithm to train the image label text detection model.

[0093] The target detection algorithm used in this example is the DB model algorithm, and the training framework used is PaddlePaddle, and the text recognition suite PaddleOCR of the PaddlePaddle framework is used. Before the start of training, configure the training parameters of the DB model, including the learning rate, number of iterations, etc. In this embodiment, the backbone network of the DB model algorithm uses ResNet50 combined with FPN, and the optimizer uses the Adam algorithm. The initial learning rate is set to 0.001, and the number of iterations is set to 1200. The model training process is similar to step S102 and will not be repeated here. Finally, the trained image label text detection model has a text detection accuracy of over 99%.

[0094] (2) Combine the image label text detection model with the text recognition model to obtain a two-stage OCR image label text recognition model.

[0095] In this embodiment, the text recognition model uses the CRNN model provided in PaddleOCR, and the trained image signature text detection model is combined with the text recognition model to finally obtain a two-stage OCR image signature text recognition model.

[0096] It should be noted that OCR generally adopts a two-stage algorithm, the first stage is text detection, and the second stage is text recognition. First, the position of the text box in the image is detected, and then the text in the text box is recognized. There is also an end-to-end OCR algorithm, but because its accuracy is not high and the two-stage OCR algorithm is more mature, the two-stage OCR algorithm is adopted. For the two-stage OCR algorithm, the recognition accuracy mainly depends on the detection of the text box, so this embodiment trains the text detection model in the two-stage OCR algorithm separately.

[0097] So far, this embodiment has completed the initial image signature template library construction and model training related work, and then executes the image signature recognition process.

[0098] Step S105, using the label position detection model to detect the target drawing, obtain the position coordinate information of the label of the target drawing; then cut the drawing according to the label position coordinates to obtain the label image to be identified.

[0099] The specific steps include:

[0100] (1) Preprocess the target drawing into a drawing image in PNG or TIF format;

[0101] (2) Using the label position detection model trained in step S102 to detect the position of the label in the target drawing, the model outputs the position coordinate information of the label detected from the drawing;

[0102] (3)According to the position coordinate information, use OpenCV to crop the drawing image to obtain the title block image to be recognized, and use the cv2.findContours method to detect the outermost rectangle in the title block image, that is, the title block table, and then further crop to remove the blank area around the title block table.

[0103] Step S106, use the title block classification model to classify the template styles of the title block to be recognized, and determine the classification of the title block to be recognized in the title block template library.

[0104] Specifically, use the title block classification model trained in step S103 to classify the title block image to be recognized obtained in step S105, and obtain the category of the title block image to be recognized and the template title block image in the corresponding title block template library. In this embodiment, the template title block styles in the title block template library corresponding to the title block to be recognized are as Figure 3 shown as A in

[0105] Furthermore, if the title block image to be recognized is deflected, the title block image to be recognized must be adjusted to be upright. It should be noted that the title block image to be recognized obtained in S105 may very likely have three deflection situations, such as being deflected by 90°, 180°, or 270°. Image deflection will affect the subsequent text recognition effect, so the title block image needs to be adjusted to be upright. It should be noted that generally, when uploading drawing files, they will be uploaded upright, so the situation of title block deflection is not common. PaddleOCR comes with a text direction classifier, but it only supports the determination of deflection angles of 0° and 180°. It should be understood that in the vast majority of cases, the style of the title block table is rectangular, and no title block image with a square shape has been found in this example. Specifically, in this example, first compare the aspect ratios of the length and width of the title block image to be recognized and the template title block image. Assume that the length of an image in the X-axis direction is h, the width in the Y-axis direction is w, and the aspect ratio is r. If h>w, then r>1; if h<w, then r<1. If the aspect ratio ratios of the title block image to be recognized and the template title block image are both greater than 1 or both less than 1, it means that the title block image to be recognized may be deflected by 0° or 180° (deflection of 0° means it is upright and no angle correction is required). By using Chinese font direction detection, it can be determined whether the deflection angle of the image is 180° or 0°. If it is 180°, flipping the image by 180° can obtain the upright title block image to be recognized. If the aspect ratio ratios of the title block image to be recognized and the template title block image are one greater than 1 and the other less than 1, it means that the title block image to be recognized may be deflected by 90° or 270°. At this time, flip the title block image to be recognized by 90° to change the deflection angle to 0° or 180°, and the subsequent processing method is the same as above.

[0106] Step S107 , using the image label text recognition model to perform OCR recognition on the image label to be recognized, and obtaining the position coordinates and text content of the text box detected in the image label to be recognized.

[0107] Specifically, the label text recognition model obtained in step S104 is used to perform OCR on the label image to be recognized obtained in step S106, and the model outputs the text content and position coordinates of the text box detected in the label image.

[0108] Step S108, scaling the size of the template image label so that the image size of the template image label is consistent with the image size of the image label to be identified, and scaling the coordinates of the text box on the template image label accordingly.

[0109] Specifically, in this example, the resize function of OpenCV is used to adjust the size of the template image to be consistent with the size of the image to be identified. , the template image size is , where H and H m Represent the height of the corresponding image, W and W m Represent the width of the corresponding image respectively, then the width scaling ratio of the template label is , the height scaling ratio is For each text box coordinate point (x, y) in the template label image, the scaling ratio is used to make corresponding adjustments. The adjusted coordinate point is (x ,y ).

[0110] It should be noted that, in this embodiment, it is preferred to scale the template label image because the template label image has high definition and complete detail information, and the fonts in the image can still be accurately identified even after scaling.

[0111] Step S109, matching the attribute value text box corresponding to each attribute in the scaled template icon with the text box on the icon to be identified, obtaining the value in the matched text box on the icon to be identified as the attribute value corresponding to the attribute, and realizing a one-to-one correspondence between the attribute and the attribute value.

[0112] It should be understood that the correspondence between the attribute and the attribute value in the to-be-recognized image tag is unknown, while the correspondence between the attribute and the attribute value in the template image tag is known. Therefore, as long as the attribute value text box corresponding to a certain attribute in the template image tag matches a certain text box on the to-be-recognized image tag, the value in the text box of the to-be-recognized image tag is used as the attribute value corresponding to the attribute. Specifically, based on the position coordinates and text content of the to-be-recognized image tag text box detected by the image tag text recognition model, it is determined one by one whether the center point of each attribute value text box in the scaled template image tag falls within a certain text box of the to-be-recognized image tag. If the center point falls within a certain text box, the text content in the text box is determined to be the attribute value of the corresponding attribute, that is, the attribute value corresponding to the corresponding attribute in the to-be-recognized image tag, thereby achieving a one-to-one correspondence between the attribute and the attribute value in the to-be-recognized image tag. Figure 4 Schematic diagram of attribute value mapping between the template image signature and the image signature to be recognized after OCR in this embodiment.

[0113] More specifically, the method for determining whether the center point of the attribute value text box in the scaled template image falls within a text box in the image to be identified is as follows: Assuming that the coordinates of a text box M in the scaled template image are , the coordinates of its center point are ,in, , For all text boxes in the image label to be identified, if there is a text box with coordinates and The text box N satisfies and , then the center point of text box M falls into text box N, and text box M matches text box N. The value in text box N is the attribute value of the attribute corresponding to text box M. Similarly, the center point position is used to determine the text box of the to-be-recognized image tag corresponding to the attribute value text box of other template image tags, and all attribute values ​​are obtained.

[0114] It should be noted that, in the present embodiment, the method of mapping the center point of the template graphic label text box to the graphic label to be identified is preferred, which is referred to as "forward mapping" here. The reason is that in the template graphic label, the position and structural information of the attribute and the corresponding attribute value text box are known. As long as the center point of the attribute value text box corresponding to all attributes is determined on the graphic label to be identified, the one-to-one correspondence between the attribute and the attribute value can be completed, thereby saving computing resources and time and improving efficiency. If "reverse mapping", that is, mapping the center point of the graphic label text box to be identified to the template graphic label, it is necessary to traverse and determine the center point of all text boxes on the graphic label to be identified on the template graphic label, which is less efficient. It should be understood that reverse mapping can also be performed when time resources are sufficient; in addition, when forward mapping, in addition to determining the center point of the attribute value text box on the template graphic label on the graphic label to be identified, it can also be further verified by determining the center point of the attribute text box on the graphic label to be identified.

[0115] Furthermore, this embodiment also provides an automatic recognition system for engineering drawing labels based on template matching, such as Figure 5 As shown, including:

[0116] The label template library module 01 is used to establish a label template library based on the existing label format; the template library at least includes a template label image and a corresponding label file, and the label file at least includes the label name and position coordinates of the template label attribute and the corresponding attribute value text box, as well as the size information of the template label;

[0117] The label position detection and cutting module 02 is used to detect the target drawing using the label position detection model to obtain the position coordinates of the label on the target drawing; and to cut the target drawing according to the position coordinates to obtain the label to be identified;

[0118] The image signature classification module 03 is used to classify the image signature template style of the image signature to be identified using the image signature classification model, and determine the category to which the image signature to be identified belongs in the image signature template library;

[0119] The image label text recognition module 04 is used to perform OCR recognition on the image label to be recognized using the image label text recognition model to obtain the position coordinates and text content of the text box detected in the image label to be recognized;

[0120] The template label scaling module 05 is used to scale the size of the template label so that the image size of the template label is consistent with the image size of the label to be identified, and to scale the coordinates of the text box on the template label accordingly;

[0121] The module 06 for corresponding label attributes to attribute values ​​is used to match the attribute value text boxes corresponding to the attributes in the scaled template label with the text boxes on the label to be identified, and obtain the value in the matched text box on the label to be identified as the attribute value corresponding to the attribute, thereby realizing a one-to-one correspondence between the attribute and the attribute value.

[0122] For a detailed description of each module in the system, please refer to the relevant description corresponding to the above method embodiment, which will not be repeated here.

Claims

1. A method for automatic recognition of engineering drawing labels based on template matching, characterized in that: The steps include: S101, based on the existing label format, establish a label template library; the template library at least includes a template label image and a corresponding label file, the label file at least includes the label name and position coordinates of the template label attribute and the corresponding attribute value text box, and the size information of the template label; S102, preparing a drawing label detection dataset, using a target detection algorithm, and training a label position detection model; S103, preparing a label classification data set, using an image classification algorithm, and training a label classification model; S104, preparing a label text detection data set, using a target detection algorithm, training a label text detection model, and combining it with a text recognition model to obtain a label text recognition model; S105, using the label position detection model to detect the target drawing, and obtaining the position coordinates of the label on the target drawing; cutting the target drawing according to the position coordinates to obtain the label to be identified; S106, using the image tag classification model to classify the image tag template style to determine the category to which the image tag to be identified belongs in the image tag template library; S107, using the image tag text recognition model to perform OCR recognition on the image tag to be recognized, and obtain the position coordinates and text content of the text box detected in the image tag to be recognized; S108, scaling the size of the template image label so that the image size of the template image label is consistent with the image size of the image label to be identified, and scaling the coordinates of the text box on the template image label accordingly; S109, matching the attribute value text box corresponding to each attribute in the scaled template icon with the text box on the icon to be identified, obtaining the value in the matched text box on the icon to be identified as the attribute value corresponding to the attribute, and realizing a one-to-one correspondence between the attribute and the attribute value.

2. The method for automatic identification of engineering drawing labels based on template matching according to claim 1 is characterized in that: In step S101, the attributes include multiple or all of the project name, phase, specialty, drawing name, drawing number, creation date, designer, approver, verifier, reviewer, and checker; the size information includes the width and height of the template drawing label image; the label file also includes synonymous attribute reference information.

3. The method for automatic identification of engineering drawing labels based on template matching according to claim 1 is characterized in that: In step S102, the target detection algorithm adopts the PP-YOLO model algorithm, and the model is trained using the PaddlePaddle training framework and its end-to-end target detection suite PaddleDetection to obtain the image label position detection model.

4. The method for automatic identification of engineering drawing labels based on template matching according to claim 1 is characterized in that: In step S103, the image classification algorithm adopts the ResNet50 model algorithm, and the model is trained using the PaddlePaddle training framework and its image classification suite PaddleClas to obtain the image label classification model.

5. The method for automatic identification of engineering drawing labels based on template matching according to claim 1 is characterized in that: In step S104, the target detection algorithm adopts the DB model algorithm, and uses the PaddlePaddle training framework and its text recognition suite PaddleOCR to train the model to obtain the image label text detection model; the text recognition model uses the CRNN model provided in PaddleOCR.

6. The method for automatic identification of engineering drawing labels based on template matching according to claim 1 is characterized in that: The step S106 further includes: If the image of the image label to be identified is deflected, the aspect ratios of the image label to be identified and the template image label are compared and adjusted to a positive direction, wherein the deflection includes a deflection of 90°, 180° or 270°, and the aspect ratio is a ratio of the length of the image in the X-axis direction to the width of the image in the Y-axis direction, including the following steps: (1) If the aspect ratios of the image to be identified and the template image are both greater than 1 or less than 1, the deflection angle is determined to be 180° or 0° by detecting the direction of the Chinese font. If it is 180°, the image to be identified is flipped 180°. If it is 0°, no angle correction is required. (2) If the aspect ratio of the image to be identified and the template image is greater than 1 and the other is less than 1, the image to be identified is flipped 90° so that its deflection angle is changed to 180° or 0°. The deflection angle is determined to be 180° or 0° through Chinese font direction detection. If it is 180°, the image to be identified is flipped 180°. If it is 0°, there is no need to correct the angle.

7. The method for automatic identification of engineering drawing labels based on template matching according to claim 1 is characterized in that: In step S108, scaling the size of the template image signature includes using the resize function of OpenCV to adjust the image size of the template image signature to be consistent with the image size of the image signature to be identified.

8. The method for automatic identification of engineering drawing labels based on template matching according to claim 1 is characterized in that: In step S109, the attribute value text box corresponding to each attribute in the scaled template image label is matched with the text box on the image label to be identified, and the method for obtaining the value in the matched text box on the image label to be identified is: Based on the position coordinates and text content of the text box detected by the label text recognition model, the center point of the attribute value text box corresponding to each attribute in the scaled template label is determined one by one on the label to be identified, and the text content in the text box where the landing point is located is obtained.

9. The method for automatic identification of engineering drawing labels based on template matching according to claim 8 is characterized in that: The method for determining the location of the center point of the attribute value text box in the scaled template icon on the icon to be identified comprises the following steps: (1) Get the center point of an attribute value text box M in the scaled template image label, denoted as ; (2) Traverse all text boxes in the image label to be identified. If there is a text box with coordinates and The text box N satisfies and , then it is determined that the center point of the text box M falls into the text box N of the image label to be identified.

10. A system for automatically identifying engineering drawing labels based on template matching, used to implement the method for automatically identifying engineering drawing labels based on template matching as claimed in any one of claims 1 to 9, characterized in that: include: A label template library module is used to establish a label template library based on the existing label format; The label position detection and cutting module is used to detect the target drawing using the label position detection model to obtain the position coordinates of the label on the target drawing; cut the target drawing according to the position coordinates to obtain the label to be identified; A label classification module is used to classify the label template style of the label to be identified using the label classification model, and determine the category to which the label to be identified belongs in the label template library; The image label text recognition module is used to perform OCR recognition on the image label to be recognized using the image label text recognition model to obtain the position coordinates and text content of the text box detected in the image label to be recognized; The template label scaling module is used to scale the size of the template label so that the image size of the template label is consistent with the image size of the label to be identified, and to scale the coordinates of the text box on the template label accordingly; The module for matching attribute and attribute value of image tag is used to match the attribute value text box corresponding to each attribute in the scaled template image tag with the text box on the image tag to be identified, and obtain the value in the matched text box on the image tag to be identified as the attribute value corresponding to the attribute, so as to realize a one-to-one correspondence between the attribute and the attribute value.

Images