An engineering drawing steel bar large sample annotation area target detection segmentation method, system, device and storage medium
By constructing a target detection network for the annotation area of rebar details based on key points, the problems of low efficiency and poor generalization in existing technologies are solved, and accurate annotation area segmentation and recognition are achieved in complex environments.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- POWERCHINA HUADONG ENG CORP LTD
- Filing Date
- 2025-06-12
- Publication Date
- 2026-07-21
AI Technical Summary
Existing technologies are inefficient in inspecting reinforcement detail annotations, making it difficult to accurately locate annotation information in complex environments. Furthermore, deep learning methods lack the ability to utilize component-specific symbol information, which limits the generalization of the inspection.
A target detection network for rebar detail annotation areas based on key points is constructed. The loss function of key points and bounding boxes is defined. Key point localization and correction are performed through dataset annotation and morphological image processing techniques to achieve region segmentation.
It improves the recognition accuracy and robustness of the reinforcement detail annotation area, effectively handles annotation area segmentation in complex environments, and enhances the detection generalization ability.
Smart Images
Figure CN120894381B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of artificial intelligence and engineering structural design, and specifically relates to a target detection and segmentation method, system, device and storage medium for the annotation area of steel reinforcement details in engineering drawings. Background Technology
[0002] Target detection in rebar detail annotations is the primary task for automatically extracting rebar detail annotation content. It involves the graphical positioning and structural understanding of the rebar detail annotations, and is the foundation and prerequisite for extracting effective textual information. Due to the large number of rebar detail annotations, their diverse styles, and the complexity of the surrounding environment, accurately locating and detecting this annotation information presents a considerable challenge.
[0003] Currently, the industry typically relies on technicians manually browsing drawings and recording data to locate and record information, but this method is inefficient. Furthermore, machine learning methods such as template matching and image detection struggle to achieve ideal results in engineering drawings with varied shapes and complex environments. For example, Chinese patent CN113658171A discloses an automatic extraction method for rebar processing information based on digital image processing. This method integrates multiple graphics technologies to detect and identify processing information in rebar details, but it heavily relies on customized feature design and struggles to generalize well to unused examples, limiting its application scope. The emergence of deep learning has addressed the shortcomings of manually setting graphic features. For instance, Chinese patent CN109002841A discloses a method for extracting building components based on the Faster-RCNN model. This method first preprocesses the architectural engineering drawing image through binarization and segmentation; then, it trains the Faster-RCNN network using labeled data to obtain a building component recognition model, and stores the extracted building component information in a structured form. While this method expands the generalization of detection methods, it merely uses proprietary data to perform transfer learning on a general target detection neural network, without fully considering the geometric structural features of the target and lacking the utilization of the unique symbolic information of components in the drawings. Summary of the Invention
[0004] The first objective of this invention is to provide a method for target detection and segmentation in the annotation area of reinforcement details in engineering drawings, addressing the problems mentioned above.
[0005] To achieve the above objectives, the present invention adopts the following technical solution:
[0006] A method for target detection and segmentation in the annotation area of reinforcement details in engineering drawings includes the following steps:
[0007] S1. Construct a target detection network for the rebar detail annotation area based on key point guidance;
[0008] S2. Define the loss function of the large-scale annotation key point target detection network, and obtain a comprehensive total loss by balancing the key point loss and bounding box loss through hyperparameters.
[0009] S3. Collect engineering drawings containing reinforcement detail annotations and annotate the dataset;
[0010] S4. Select evaluation metrics for the target detection network model in the large sample annotation area and train the model.
[0011] S5. Use morphological image processing techniques to locate and correct key points.
[0012] S6. Perform region segmentation based on the detailed annotation information of key point locations.
[0013] While adopting the above technical solutions, the present invention may also adopt or combine the following technical solutions:
[0014] As a preferred technical solution of the present invention: in step S1, the target detection network of the rebar detail annotation area includes an input end, a backbone network, a neck network and a head network.
[0015] As a preferred embodiment of the present invention: In step S2, the key point loss and the bounding box loss are calculated in the following manner:
[0016] The vector of overall prediction can be defined as:
[0017] , , ,
[0018] The true annotation keypoint vector can be defined as:
[0019] , ,
[0020] In the formula, and These are the x and y coordinates of the center point of the prediction box; W and H are the width and height of the prediction box. , These are the location confidence and class confidence of the predicted bounding box; It predicts the x and y coordinates and confidence score of the i-th annotation keypoint; It is the actual width and height; These are the actual coordinates of the i-th annotation keypoint;
[0021] Key points It's the location where the reinforcement detail leader line intersects with the annotation line; another key point. This refers to the location where the reinforcement detail annotation number intersects with the annotation line, indicating a loss of alignment at key points. As shown in the formula below:
[0022]
[0023] In the formula, Indicate key points y-axis coordinate, Indicate key points The y-axis coordinate.
[0024] Large-scale annotation keypoint boundary distance loss As shown in the formula below:
[0025]
[0026] In the formula, W is the width of the prediction box. It is the distance constant of the reinforcement detail annotation number. Indicate key points x-axis coordinates Indicate key points x-axis coordinate;
[0027] The Kronecker function is used to represent the visibility of annotation keypoints. The Kronecker function explicitly indicates whether each keypoint is visible by returning a binary value: the function returns 1 when the keypoint is not covered and is visible in the view; otherwise, the function returns 0. The Kronecker function is shown in the following formula:
[0028]
[0029] In the formula, This represents the actual visible state of the i-th point. =0 indicates that the key point of the comment is not visible and its location cannot be determined by inference; =1 indicates that the key points of this comment are visible. =2 indicates that the key point of the annotation is not visible, but the location of the key point can be determined by inference;
[0030] Euclidean distance is used to represent the distance between the actual and predicted coordinates of the nth annotated keypoint. To avoid the influence of large-sample annotation scale on the metrics, standardization is used to eliminate scale differences. The distance calculation formula is as follows:
[0031]
[0032] In the formula, These are the predicted x, y coordinates of the i-th annotation keypoint. These are the actual x and y coordinates of the i-th annotation keypoint.
[0033] To measure the similarity of annotation keypoints, the coordinate distance is standardized to avoid differences in the area of different large-scale annotations, and an exponential function combined with the standard deviation of annotation keypoints is used to limit the range of values, thereby calculating the large-scale annotation keypoint similarity AKS, as shown in the following formula:
[0034]
[0035]
[0036] In the formula, , is the width and height of the true bounding box, and d is the distance between the actual coordinates and the predicted coordinates of the keypoint.
[0037] Large sample annotation key point loss As shown in the formula below:
[0038]
[0039] In the formula, Indicates the similarity of key points in the sample annotation. This represents the loss due to the distance between keypoint boundaries. This indicates the alignment loss.
[0040] CIoU is used to measure the similarity between the predicted bounding box and the target bounding box, and the bounding box loss is... As shown in the formula below:
[0041]
[0042] In the formula, It is the true bounding box corresponding to the j-th anchor box at position (a,b). It predicts the bounding box.
[0043] As a preferred technical solution of the present invention: by combining the loss of key points in large-scale annotation. and bounding box loss These data, collected at different scales and anchor points, are then summarized to arrive at a comprehensive total loss. Total loss function As shown in the formula below:
[0044]
[0045] In the formula, hyperparameters =0.05, =0.5, two hyperparameters used to balance the keypoint loss and the bounding box loss.
[0046] As a preferred technical solution of the present invention: In step S3, the engineering drawings not only need to include steel reinforcement annotations of different shapes, but also need to have annotations of similar shapes or different categories. The engineering drawings exhibit diversity in terms of drafting style, language font and image quality.
[0047] As a preferred technical solution of the present invention: In step S4, the task of extracting information from detail annotations not only requires the model to accurately and comprehensively detect all detail annotations in the drawing, but also requires the model to accurately locate two key points in each detail annotation. The Intersection over Union (IoU) and Key Point Similarity (OKS) are used to measure the accuracy of the detail annotation detection box and the key points, respectively. The IoU is calculated using the following formula:
[0048]
[0049] In the formula, This represents the area of the intersection of the predicted bounding box and the truth bounding box. This represents the area of the union of the predicted bounding box and the truth bounding box;
[0050] Keypoint similarity (OKS) is calculated using the following formula:
[0051]
[0052] In the formula, N represents the number of key points annotated in the large-scale drawing. Let be the Euclidean distance between the predicted value and the ground truth value of the nth keypoint, and s be the area of the bounding box in the large-scale annotation. The normalization factor for the nth key point. The existence identifier of the nth key point The Kronecker function value.
[0053] As a preferred technical solution of the present invention: in step S6, the detailed annotation is divided into three areas, namely the rebar quantity information area, the diameter information area, and the rebar number information area.
[0054] The second objective of this invention is to provide a target detection and segmentation system for the annotation area of reinforcement details in engineering drawings, comprising the following modules:
[0055] - Target detection network construction module, used to build a target detection network for the rebar detail annotation area based on key points;
[0056] - The target detection network loss function definition module is used to define the loss function of the target detection network for large-scale annotation keypoints, and obtain a comprehensive total loss by balancing the keypoint loss and bounding box loss through hyperparameters;
[0057] - Engineering drawing collection and annotation module, used to collect engineering drawings containing reinforcement detail annotations and annotate the dataset;
[0058] - The metric evaluation and model training module is used to select evaluation metrics for the target detection network model in the large sample annotation area and to train the model.
[0059] - Key point localization and correction module, which uses morphological image processing technology to perform key point localization and correction;
[0060] - The region segmentation module performs region segmentation based on the large-scale annotation information of key point locations.
[0061] A third objective of this invention is to provide an electronic device comprising a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus.
[0062] The memory is used to store computer programs;
[0063] A processor is configured to execute a computer program stored in a memory to implement the target detection and segmentation method steps of the reinforcement detail annotation area of engineering drawings as described above.
[0064] Another objective of this invention is to provide a non-volatile storage medium storing an executable program, which, when executed by a processor, implements the target detection and segmentation method steps for the reinforcement detail annotation area of engineering drawings as described above.
[0065] This invention provides a method, system, device, and storage medium for target detection and segmentation of the reinforcement detail annotation area in engineering drawings, which has the following beneficial effects: This invention achieves rule-based segmentation of the annotation area by identifying and correcting the positional information of key points in the reinforcement detail annotation, ensuring the accuracy of different attribute texts in the annotation area; this invention has good robustness against detail annotations with densely interfering graphic elements and characters, vertically placed detail annotations and their key points, partially occluded detail annotations, and detail annotations with missing polylines or numbered circles; this invention achieves joint detection of bounding boxes and their key points, improving the accuracy, robustness, and generalization of reinforcement detail annotation area recognition based on neural networks. Attached Figure Description
[0066] Figure 1 The flowchart illustrates the target detection and segmentation method for the annotation area of steel reinforcement details in engineering drawings provided by this invention.
[0067] Figure 2 A network structure for detecting key points in the reinforcement detail annotation.
[0068] Figure 3 This is a schematic diagram showing the location of key points in the reinforcement detail annotation.
[0069] Figure 4 Examples of reinforcement detail annotations and non-reinforcement detail annotations.
[0070] Figure 5 This is an example of the impact of keypoint offset on the division of different large-scale annotation regions.
[0071] Figure 6 This is the offset neighborhood of the key point.
[0072] Figure 7 Mark the intersection points of the skeleton lines for extra-large samples.
[0073] Figure 8 The area is divided to annotate different textual attribute information for steel reinforcement details. Detailed Implementation
[0074] The present invention will be described in further detail with reference to the accompanying drawings and specific embodiments.
[0075] like Figure 1 As shown, a target detection and segmentation method for the annotation area of reinforcement details in engineering drawings includes the following steps:
[0076] S1. Construct a target detection network for the rebar detail annotation area based on key point guidance;
[0077] Inspired by the YOLO series algorithms, YOLOv7 was used as the backbone network for feature extraction in large-scale annotation keypoint detection. The target detection network for the large-scale rebar annotation area mainly consists of four parts: the input end, the backbone network, the neck network, and the head network. Figure 2 As shown;
[0078] The input end refers to the data processing layer used to read the large-scale annotation dataset of engineering drawings and perform a series of preprocessing operations such as image denoising and data augmentation on the input engineering drawings.
[0079] The backbone network, as the core of the entire network architecture, is constructed using the ELAN (EfcientLayer Aging-gation Network) module and selects the hard-swish activation function to enhance the nonlinear expression ability between neuron nodes. At the same time, it draws on the design concept of YOLOv7 to effectively extract the features of images at different resolutions.
[0080] Neck network refers to a neural network layer that uses an SPPCSPC structure for feature fusion, accelerating the cross-scale feature fusion process through hierarchical path connections.
[0081] The head network refers to a parallel detection structure for keypoint detection and target detection, specifically designed for large-scale annotation structures, with the aim of collaboratively predicting keypoint and bounding box coordinates.
[0082] S2. Define the loss function of the target detection network for key points in the large-scale annotation based on the spatial characteristics of the constructed target detection network and the key points in the large-scale annotation, and obtain a comprehensive total loss by balancing the key point loss and the bounding box loss through hyperparameters.
[0083] Ocean annotation detection can be considered a single-class object detection task. The location information of ocean annotations is usually expressed using the annotation center (…). , The height H and width W of the location boundary are represented. Additionally, each sample contains two associated annotation keypoints, each requiring a defined position and confidence level: x, y, and conf. Each anchor box contains a total of six elements related to the annotation keypoint. Therefore, for each anchor box, the annotation keypoint detection head will predict six elements, and the bounding box detection head will predict six elements.
[0084] Keypoint loss and bounding box loss are calculated as follows:
[0085] The vector of overall prediction can be defined as:
[0086] , , ,
[0087] The true annotation keypoint vector can be defined as:
[0088] , ,
[0089] In the formula, and These are the x and y coordinates of the center point of the prediction box; W and H are the width and height of the prediction box. , These are the location confidence and class confidence of the predicted bounding box; It predicts the x and y coordinates and confidence score of the i-th annotation keypoint; It is the actual width and height; These are the actual coordinates of the i-th annotation keypoint;
[0090] In the two key points of the detailed annotation, the two key points play different roles. Key points It is the location where the leader line of the rebar detail intersects with the annotation line, such as... Figure 3 (in) Another key point This refers to the location where the reinforcement detail annotation number intersects with the annotation line, such as... Figure 3 (in) , Since these two key points are at opposite ends of the detail annotation line, when the detail annotation is placed horizontally, these two points are horizontally aligned. From this, we can obtain the keypoint alignment loss. As shown in the formula below:
[0091]
[0092] In the formula, Indicate key points y-axis coordinate, Indicate key points The y-axis coordinate.
[0093] observe Figure 3 It can be seen that there is a fixed distance between the intersection of the numbered area and the annotation line and the boundary of the large-scale annotation, which is the diameter of the large-scale annotation number. Extending this distance condition, we can obtain the boundary distance loss of the key points of the large-scale annotation. As shown in the formula below:
[0094]
[0095] In the formula, W is the width of the prediction box. It is the distance constant of the reinforcement detail annotation number. Indicate key points x-axis coordinates Indicate key points x-axis coordinate;
[0096] When a large-scale annotation is covered by other graphic elements, making key points difficult to locate or invisible, the Kronecker function is used to indicate the visibility of the annotation key points. The Kronecker function explicitly indicates whether each key point is visible by returning a binary value (0 or 1): the function returns 1 when the key point is not covered and is visible in the view; otherwise, the function returns 0. The Kronecker function is shown in the following formula:
[0097]
[0098] In the formula, This represents the actual visible state of the i-th point. =0 indicates that the key point of the comment is not visible and its location cannot be determined by inference; =1 indicates that the key points of this comment are visible. =2 indicates that the key point of the annotation is not visible, but the location of the key point can be determined by inference;
[0099] Euclidean distance is used to represent the distance between the actual and predicted coordinates of the nth annotated keypoint. To avoid the influence of large-sample annotation scale on the metrics, standardization is used to eliminate scale differences. The distance calculation formula is as follows:
[0100]
[0101] In the formula, These are the predicted x, y coordinates of the i-th annotation keypoint. These are the actual x and y coordinates of the i-th annotation keypoint.
[0102] To measure the similarity of annotation keypoints, the coordinate distance is standardized to avoid differences in the area of different large-sample annotations, and an exponential function combined with the standard deviation of annotation keypoints is used to limit the range of values. The resulting large-sample annotation keypoint similarity (AKS) is shown in the following formula:
[0103]
[0104]
[0105] In the formula, , is the width and height of the true bounding box, and d is the distance between the actual coordinates and the predicted coordinates of the keypoint.
[0106] Assuming the ground truth bounding box of a large-scale annotation is clearly located at (a, b) and has dimensions (H, W), then when predicting the bounding box, the prediction is not limited to the localization of the box, but also includes a detailed prediction of the position of each keypoint relative to the anchor center. This invention calculates the similarity index for each keypoint independently, then sums these similarity indices and sums them with the keypoint boundary distance loss and alignment loss to obtain the final keypoint loss function. Therefore, the large-scale annotation keypoint loss between the j-th anchor box and the ground truth bounding box... As shown in the formula below:
[0107]
[0108] In the formula, Indicates the similarity of key points in the sample annotation. This represents the loss due to the distance between keypoint boundaries. This indicates the alignment loss.
[0109] Using CIoU (Complete Intersection over Union) to measure the similarity between the predicted bounding box and the target bounding box, the bounding box loss of the j-th anchor box is... As shown in the formula below:
[0110]
[0111] In the formula, It is the true bounding box corresponding to the j-th anchor box at position (a,b). It predicts the bounding box.
[0112] By combining the loss of key points in large-sample annotation and bounding box loss These data, collected at different scales and anchor points, are then summarized to arrive at a comprehensive total loss. Total loss function As shown in the formula below:
[0113]
[0114] In the formula, hyperparameters =0.05, =0.5, two hyperparameters used to balance the keypoint loss and the bounding box loss.
[0115] S3. Collect engineering drawings containing reinforcement detail annotations and annotate the dataset;
[0116] The collected engineering drawing dataset follows two main criteria: first, the engineering drawings must include detailed reinforcement elements; second, the drawings should exhibit a certain degree of diversity in drafting style, language fonts, and image quality.
[0117] The collected drawings need to include not only reinforcement annotations of various shapes, but also annotations of similar shapes or different categories. For example... Figure 4 As shown, the dataset includes complete and incomplete rebar annotations, rebar annotations with and without parameters, and rebar annotations with and without circles. The diversity of the drawings themselves must also be considered. High-resolution drawings have clear boundaries and features, thus increasing the image area. Conversely, drawings with low edge ratios, while having a slightly smaller area, will have more blurred graphic and textual features. This invention collects engineering drawings from different design institutes in multiple regions, such as bridge, roadbed, and tunnel engineering drawings from various design institutes. Detailed annotations and key point annotations are then performed using the Labelme annotation software.
[0118] S4. Select evaluation metrics for the target detection network model in the large sample annotation area and train the model.
[0119] The task of extracting information from detail annotations requires the model to not only accurately and comprehensively detect all detail annotations in the drawing, but also to precisely locate two key points in each detail annotation. Accurate location of these key points is crucial for dividing the detail annotation into three information regions. The accuracy of the detail annotation detection bounding box and key points is measured using the Intersection over Union (IoU) and Object Keypoint Similarity (OKS). The IoU is calculated using the following formula:
[0120]
[0121] In the formula, This represents the area of the intersection of the predicted bounding box and the truth bounding box. This represents the area of the union of the predicted bounding box and the truth bounding box;
[0122] Keypoint similarity (OKS) is calculated using the following formula:
[0123]
[0124] In the formula, N represents the number of key points annotated in the large-scale drawing. Let be the Euclidean distance between the predicted value and the ground truth value of the nth keypoint, and s be the area of the bounding box in the large-scale annotation. The normalization factor for the nth key point. The existence identifier of the nth key point The Kronecker function value.
[0125] Accuracy (AP) and recall (AR) are used as the basic indicators for the target detection and recognition accuracy of large sample annotations. Accuracy refers to the proportion of correctly detected targets in the detection results. The higher the indicator, the lower the probability of large sample annotations being falsely detected. Recall refers to the proportion of real targets being detected. The higher the indicator, the lower the probability of large sample annotations being missed.
[0126] S5. Use morphological image processing technology to process the predicted key points of the steel reinforcement detail annotation and the screenshot of the detail annotation area, and make accurate corrections to the location of the predicted key points.
[0127] The accuracy of key point localization will determine the final effect of dividing different text attribute regions. For example... Figure 5As shown, when characters and polylines are close together, even slight shifts in key points can cause text characters to be truncated, thus affecting accurate text recognition. Morphological image processing techniques are used to precisely correct the location of key points. Figure 6 For example, based on a detailed analysis of the rebar detail annotations in the dataset, a square region with a side length of 20 pixels is selected, centered on the predicted key points. This region should contain the exact location of the key points, including the inflection points of the polyline and the intersections of the polyline and the numbered circles.
[0128] Then, as Figure 7 As shown, the Zhang-Suen thinning algorithm is used to extract the skeleton from the image within the square region. Then, the intersection points of the skeleton lines are obtained through morphological hit-and-run transformation. The coordinates of the intersection points are the coordinates of the precise keypoints relative to the square region. Assuming... Figure 7 The coordinates of the key points in the large-scale annotation are ( Let d be the side length of the square region centered on the keypoint, and let d be the precise coordinates of the keypoint relative to the square region obtained through morphological methods. The precise coordinates (x, y) of the key point on the annotated image are calculated as follows:
[0129]
[0130]
[0131] S6. Based on the detailed annotation information of the modified key point location, effectively extract the annotation lines and numbering circles in the reinforcement detailed annotation, and divide the detailed annotation information into three regions: reinforcement quantity information region, diameter information region, and reinforcement numbering information region.
[0132] After the above steps, we obtained the reinforcement detail annotation area map and accurate key points. This step, by applying mathematical morphology techniques and ignoring the influence of factors such as length, effectively extracts the key lines in the image, namely the annotation lines and numbering circles in the reinforcement detail annotation, to achieve more detailed area division.
[0133] like Figure 8 As shown, the detailed annotation is divided into three regions by key points. The upper dark gray region, located above the line segment connecting the two key points, contains the quantity and diameter information of the reinforcing bars; the gray region, located below the line segment, records the length information of the reinforcing bars; and the light gray region, located on the side adjacent to the gray key points, contains the numbering information of the reinforcing bars.
[0134] This invention also provides a target detection and segmentation system for the annotation area of reinforcement details in engineering drawings, comprising the following modules:
[0135] - Target detection network construction module, used to build a target detection network for the rebar detail annotation area based on key points;
[0136] - The target detection network loss function definition module is used to define the loss function of the target detection network for large-scale annotation keypoints, and obtain a comprehensive total loss by balancing the keypoint loss and bounding box loss through hyperparameters;
[0137] - Engineering drawing collection and annotation module, used to collect engineering drawings containing reinforcement detail annotations and annotate the dataset;
[0138] - The metric evaluation and model training module is used to select evaluation metrics for the target detection network model in the large sample annotation area and to train the model.
[0139] - Key point localization and correction module, which uses morphological image processing technology to perform key point localization and correction;
[0140] - The region segmentation module performs region segmentation based on the large-scale annotation information of key point locations.
[0141] The present invention also provides an electronic device, which includes a processor, a communication interface, a memory, and a communication bus. The processor, the communication interface, and the memory communicate with each other via the communication bus.
[0142] Memory, used to store computer programs;
[0143] The processor executes the computer program stored in the memory to implement the target detection and segmentation method steps of the reinforcement detail annotation area of the engineering drawing as described above.
[0144] The present invention also provides a non-volatile storage medium storing an executable program, which, when executed by a processor, implements the target detection and segmentation method steps of the reinforcement detail annotation area of engineering drawings as described above.
[0145] The above specific embodiments are used to explain and illustrate the present invention, and are only preferred embodiments of the present invention, not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made to the present invention within the spirit and scope of the claims shall fall within the protection scope of the present invention.
Claims
1. A method for target detection and segmentation in the annotation area of reinforcement details on engineering drawings, characterized in that: Includes the following steps: S1. Construct a target detection network for the rebar detail annotation area based on key point guidance; S2. Define the loss function of the large-scale annotation key point target detection network, and obtain a comprehensive total loss by balancing the key point loss and bounding box loss through hyperparameters. S3. Collect engineering drawings containing reinforcement detail annotations and annotate the dataset; S4. Select evaluation metrics for the target detection network model in the large sample annotation area and train the model. S5. Use morphological image processing techniques to locate and correct key points. S6. Perform region segmentation based on the detailed annotation information of key point locations; In step S2, the keypoint loss and bounding box loss are calculated as follows: The vector definition of the overall prediction is: , , , The true annotation keypoint vector is defined as: , , In the formula, and These are the x and y coordinates of the center point of the prediction box; W and H are the width and height of the predicted bounding box; , These are the location confidence and class confidence of the predicted bounding box; It predicts the x and y coordinates and confidence score of the i-th annotation keypoint; It is the actual width and height; These are the actual coordinates of the i-th annotation key point; Key points It's the location where the reinforcement detail leader line intersects with the annotation line; another key point. This refers to the location where the reinforcement detail annotation number intersects with the annotation line, indicating a loss of alignment at key points. As shown in the formula below: In the formula, Indicate key points y-axis coordinate, Indicate key points y-axis coordinate; Large-scale annotation keypoint boundary distance loss As shown in the formula below: In the formula, W is the width of the prediction box. It is the distance constant of the reinforcement detail annotation number. Indicate key points x-axis coordinates Indicate key points x-axis coordinate; The Kronecker function is used to represent the visibility of annotation keypoints. The Kronecker function explicitly indicates whether each keypoint is visible by returning a binary value: the function returns 1 when the keypoint is not covered and is visible in the view; otherwise, the function returns 0. The Kronecker function is shown in the following formula: In the formula, This represents the actual visible state of the i-th point. =0 indicates that the key point of the comment is not visible and its location cannot be determined by inference; =1 indicates that the key points of this comment are visible. =2 indicates that the key point of the annotation is not visible, but the location of the key point can be determined by inference; Euclidean distance is used to represent the distance between the actual and predicted coordinates of the nth annotated keypoint. To avoid the influence of large-sample annotation scale on the metrics, standardization is used to eliminate scale differences. The distance calculation formula is as follows: In the formula, These are the predicted x, y coordinates of the i-th annotation keypoint. These are the actual x, y coordinates of the i-th annotation keypoint; To measure the similarity of annotation keypoints, coordinate distances are standardized to avoid differences in annotation area across different sizes, and an exponential function is used in conjunction with the standard deviation of annotation keypoints. To limit the range of values, the similarity AKS of key points in the sample annotation is calculated, as shown in the following formula: In the formula, , is the width and height of the true bounding box, and d is the distance between the actual coordinates and the predicted coordinates of the key points; Assuming the true bounding box of a large-scale annotation is already clearly located at (a, b) and has dimensions (H, W), then when predicting the bounding box, the prediction is not limited to the localization of the box, but also includes a detailed prediction of the position of each keypoint relative to the anchor center; in this case, the large-scale annotation keypoint loss... As shown in the formula below: In the formula, Indicates the similarity of key points in the sample annotation. This represents the loss due to the distance between keypoint boundaries. Indicates alignment loss; CIoU is used to measure the similarity between the predicted bounding box and the target bounding box, and the bounding box loss is... As shown in the formula below: In the formula, It is the true bounding box corresponding to the j-th anchor box at position (a,b). It predicts the bounding box.
2. The target detection and segmentation method for the reinforcement detail annotation area of engineering drawings according to claim 1, characterized in that: In step S1, the target detection network for the rebar detail annotation area includes an input end, a backbone network, a neck network, and a head network.
3. The target detection and segmentation method for the annotation area of reinforcement details in engineering drawings according to claim 1, characterized in that: By combining the loss of key points in large-sample annotation and bounding box loss These data, collected at different scales and anchor points, are then summarized to arrive at a comprehensive total loss. Total loss function As shown in the formula below: In the formula, hyperparameters =0.05, =0.5, two hyperparameters used to balance the keypoint loss and the bounding box loss.
4. The target detection and segmentation method for the annotation area of reinforcement details in engineering drawings according to claim 1, characterized in that: In step S3, engineering drawings not only need to include reinforcement annotations of various shapes, but also annotations of similar shapes or different categories. Engineering drawings exhibit diversity in drafting style, language fonts and image quality.
5. The target detection and segmentation method for the annotation area of reinforcement details in engineering drawings according to claim 1, characterized in that: In step S4, the task of extracting information from detail annotations requires the model not only to accurately and comprehensively detect all detail annotations in the drawing, but also to precisely locate two key points in each detail annotation. The Intersection over Union (IoU) and Keypoint Similarity (OKS) are used to measure the accuracy of the detail annotation detection box and the key points, respectively. The IoU is calculated using the following formula: In the formula, This represents the area of the intersection of the predicted bounding box and the truth bounding box. This represents the area of the union of the predicted bounding box and the truth bounding box; Keypoint similarity (OKS) is calculated using the following formula: In the formula, N represents the number of key points annotated in the large-scale drawing. Let be the Euclidean distance between the predicted value and the ground truth value of the nth keypoint, and s be the area of the bounding box in the large-scale annotation. The normalization factor for the nth key point. The existence identifier of the nth key point The Kronecker function value.
6. The target detection and segmentation method for the annotation area of reinforcement details in engineering drawings according to claim 1, characterized in that: In step S6, the detailed annotation is divided into three areas: the rebar quantity information area, the diameter information area, and the rebar number information area.
7. A target detection and segmentation system for the annotation area of reinforcement details in engineering drawings, characterized in that: The system is based on the target detection and segmentation method for the annotation area of reinforcement details in engineering drawings as described in claim 1, and includes the following modules: - Target detection network construction module, used to build a target detection network for the rebar detail annotation area based on key points; - The target detection network loss function definition module is used to define the loss function of the target detection network for large-scale annotation keypoints, and obtain a comprehensive total loss by balancing the keypoint loss and bounding box loss through hyperparameters; - Engineering drawing collection and annotation module, used to collect engineering drawings containing reinforcement detail annotations and annotate the dataset; - The metric evaluation and model training module is used to select evaluation metrics for the target detection network model in the large sample annotation area and to train the model. - Key point localization and correction module, which uses morphological image processing technology to perform key point localization and correction; - The region segmentation module performs region segmentation based on the large-scale annotation information of key point locations.
8. An electronic device, comprising a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus, characterized in that: The memory is used to store computer programs; A processor, the processor being configured to execute a computer program stored in a memory to implement the steps of the target detection and segmentation method for the annotation area of reinforcement details in engineering drawings as described in any one of claims 1-6.
9. A non-volatile storage medium, characterized in that: The non-volatile storage medium stores an executable program, which, when executed by a processor, implements the target detection and segmentation method steps of the reinforcement detail annotation area of engineering drawings as described in any one of claims 1-6.