Two-stage-based robot reinforcing steel bar material list identification method and two-stage-based robot reinforcing steel bar material list identification system

Abstract

The invention discloses a two-stage-based robot reinforcing steel bar material list identification method and system, and the method comprises the following steps: S1, reinforcing steel bar material list data processing, S2, reinforcing steel bar shape parameter acquisition, S3, reinforcing steel bar process category acquisition, and S4, machining process acquisition. The two-stage-based robot reinforcing steel bar material list identification system comprises the following steps of S1, a reinforcing steel bar material list data processing module, S2, a reinforcing steel bar shape parameter extraction module, S3, a reinforcing steel bar shape parameter extraction module and S4, a reinforcing steel bar machining procedure acquisition module. The invention provides a portable reinforcing steel bar sketch shape parameter extraction method which does not need to label data, and can automatically complete acquisition and identification of a reinforcing steel bar material list and automatically acquire a reinforcing steel bar processing procedure.

Description

Technical Field

[0001] This invention belongs to the field of robot control, specifically relating to a two-stage robot rebar material list recognition method and system. Background Technology

[0002] With the advent of the artificial intelligence era, more and more tasks are becoming intelligent and automated. The construction industry, as one of the largest industries, is moving towards robots replacing workers in complex and harsh environments. In construction, steel reinforcement cutting is a crucial step in concrete structure construction, involving core tasks such as steel reinforcement type identification, quantity counting, and location positioning. Its efficiency and accuracy directly affect construction progress and project quality. Currently, the rebar loading process mainly relies on manual identification of drawings. This method depends on experience, and architectural construction drawings often contain dense rebar annotations, making manual processing inefficient. Furthermore, existing automated identification technologies face several bottlenecks in this field: firstly, rebar drawings contain numerous small, densely packed targets with significant scale differences and complex semantic relationships, making single-stage detection models prone to missed or false detections; secondly, complex large-scale models are computationally intensive and cannot meet the demands of real-time robotic operations. Additionally, the construction industry offers diverse drawing formats (such as CAD, PDF scans, and image formats), and existing systems exhibit poor compatibility with these formats.

[0003] Therefore, for the steel bar feeding scenario of construction robots, there is an urgent need for a material list recognition method that takes into account recognition accuracy, efficiency and compatibility, so as to solve the pain points of manual recognition and the limitations of existing technologies, and promote the deep implementation of construction automation. Summary of the Invention

[0004] The technical problem to be solved by this invention is that existing automatic material list image recognition technology cannot simultaneously achieve both recognition accuracy and recognition speed in the steel bar feeding process. The invention aims to resolve the contradiction between recognition efficiency and error rate by combining traditional image recognition with a custom deep network.

[0005] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is to propose a two-stage robot steel bar material list recognition method and system.

[0006] The technical solution adopted in this invention is: a two-stage robot rebar list recognition method, comprising the following steps: Step S1: Rebar material list data processing; Export the material list data file according to the type of software used, and statistically analyze and organize all data formats. Process all types of rebar material list data into a unified format, which can be, but is not limited to, Excel format. Based on the unified data format, extract the rebar sketches from the material list data in batches. Step S2: Obtaining the shape parameters of the reinforcing bars; The simplified diagram of the reinforcing bars is preprocessed. Numerical regions are extracted from the diagram using traditional image processing methods. Then, based on feature matching algorithms and predefined digit templates, the shape parameters of the reinforcing bars are identified and located individually. Finally, the digits are combined according to their positional relationships to obtain parameters representing the shape of the reinforcing bars. Step S3: Obtain the rebar process category; Considering that reinforcement drawings are generally small in size, and often non-square with diverse dimensions, directly processing them using existing depth networks presents size issues. Therefore, an improvement to the transformer-based vit framework is made to adapt to small targets and non-square inputs. The improved vit framework is then used to classify the reinforcement drawings, resulting in corresponding process categories. Step S4: Obtain the processing steps; Based on a predefined database of relationships between simplified diagram types and parameters, and other information from the material list obtained through the software, the processing steps for this type of steel reinforcement are determined. The numerical recognition method shows that the numbers on both sides of the steel bar diagram are 150. The diagram type recognition method shows that the steel bar type is 3 (from the fourth category in the predefined categories), therefore there are two protruding steel bar ends and the length is 11.9. In addition, the diameter of the steel bar is C8 from the file line index after the unified data format (obtained in step S1). Combining this information, the processing steps are obtained, including the total amount of steel bar used L = (150)*2+(150)*2+(0*7.8*8)+(11.9*2*8) and the number of steel bars.

[0007] Preferably, the preprocessing in step S2 includes binarizing the simplified steel reinforcement diagram, scaling and filtering the binarized image for noise reduction, and using traditional image processing methods to extract digital regions from the simplified diagram, including but not limited to first extracting all contours from the binarized image and then filtering the digital contours based on features such as the area of ​​the contours. Preferably, the predefined digital template in step S2 can be a set of digital regions pre-selected from a large amount of data, and the feature matching algorithm can be based on matching the shape features of the numbers in the grayscale image, or any image processing algorithm that can perform feature matching between the digital regions extracted from the simplified image and the predefined digital template. The template-based feature similarity matching algorithm can be expressed as: ; in, For predefined digital template diagrams, The image shows the digital region to be identified, and )represent and The feature similarity between them, where index is the index of the maximum similarity. The numbers to be identified.

[0008] Specifically, after digital identification and positioning, the combined numbers can be filtered based on prior knowledge of the shape parameters of the reinforcing bars and the positional relationships between the numbers.

[0009] Preferably, the improved transformer-based vit framework described in step S3 includes: firstly, analyzing the training data, including the dimensions of the simplified diagrams and their aspect ratios. After analysis, representative values ​​such as the median of the simplified diagram dimensions can be selected as the input dimensions of the vit network. Simultaneously, to accommodate the non-square nature of the reinforcement diagrams, non-square convolution kernels are used for patch operations in vit. The improved transformer-based vit model can employ data augmentation during the training phase to compensate for insufficient software data collection. Considering that the process type of the reinforcement diagram is independent of the numerical values ​​on the diagram and only related to the line layout, a series of parameter values ​​can be randomly generated, and the original parameters can be replaced using the parameter positions obtained in step S2, thereby enriching the data.

[0010] A two-stage robotic rebar billet recognition system includes the following steps: Step S1: Rebar Material List Data Processing Module; The software retrieves the rebar material list file, processes the file into a predefined format, extracts the rebar sketches from all rows of the material list file, converts them into a unified format file, and includes all the rebar sketches as the output of this module. Step S2: Rebar shape parameter extraction module; This module batch-reads simplified rebar diagrams, extracts all contours from the diagrams using a contour extraction algorithm, filters out numerical contours based on the area features of the contours, extracts numbers from the rebar diagrams using a template-based feature matching algorithm, and obtains the shape parameters of the rebars based on the positional relationships between the numbers, which are then used as the output of this module. Step S3: Rebar shape parameter extraction module; The system reads batch of simplified rebar diagrams and preprocesses them to the dimensions input to the network, including but not limited to scaling and cropping. These simplified rebar diagrams are then input into a pre-trained type classification network, and the output of this module is the types of all simplified rebar diagrams. Step S4: Rebar processing procedure acquisition module; The system reads other parameters of each line of steel bars from the output file of the steel bar material list data processing module, the steel bar process type output by the steel bar process category extraction module, and the steel bar shape parameters output by the steel bar shape parameter extraction module, and calculates the processing procedures based on certain rules.

[0011] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are: 1. This invention proposes a convenient method for extracting shape parameters from simplified steel bar diagrams that does not require labeled data.

[0012] 2. The two-stage robotic steel bar material list recognition system proposed in this invention can automatically complete the acquisition and recognition of steel bar material lists, as well as the automatic acquisition of steel bar processing procedures.

[0013] 3. In view of the small size and diverse dimensions of the steel reinforcement diagrams in the steel reinforcement bill of materials, this invention proposes an improved vit frame based on transformer.

[0014] 4. This invention proposes a two-stage steel bar material list identification method, which can resolve the contradiction between efficiency and error rate. Attached Figure Description

[0015] To facilitate understanding by those skilled in the art, the present invention will be further described below with reference to the accompanying drawings.

[0016] Figure 1 This is a block diagram of a two-stage robotic steel rebar material single-image recognition system.

[0017] Figure 2 This is a digital contour map extracted based on traditional image processing methods.

[0018] Figure 3 This is a graph showing the parameter results identified based on traditional image processing methods.

[0019] Figure 4 A network diagram for an improved transformer-based vit framework.

[0020] Figure 5 This is a simplified diagram of the reinforcing steel bars. Detailed Implementation

[0021] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] Example 1: The following describes a two-stage robot rebar billet identification method provided by the present invention, including the following steps: Step S1: Rebar Material List Data Processing We will compile and organize all data formats, including but not limited to PDF and Excel, based on the types of rebar detailing software available on the market (such as E-rebar quantity calculation software). We will process all types of rebar material list data into a unified format, which can be, but is not limited to, Excel. Based on this unified data format, we will batch extract simplified rebar diagrams from the material list files.

[0023] Step S2: Obtaining Rebar Shape Parameters The simplified diagram of the reinforcing bars is preprocessed. Digital regions on the diagram are extracted based on traditional image processing methods. The shape parameters of the reinforcing bars are digitally identified and located based on feature matching algorithms and predefined digital templates. The parameters representing the shape of the reinforcing bars are obtained by combining the numbers according to their positional relationships.

[0024] Specifically, preprocessing includes selecting an appropriate threshold to binarize the simplified rebar diagram, scaling the binarized image, and filtering for noise reduction. The specific scaling value depends on the size of the template digits. The preprocessing steps can be represented as follows:

[0025] Specifically, traditional image processing methods for extracting numerical regions from a simplified diagram include, but are not limited to, first using OpenCV functions to extract all contours from the binarized image, and then filtering the numerical contours based on features such as contour area. The simplified diagram contains numbers representing rebar parameters and lines indicating the direction of the construction process. The area of ​​the rectangles corresponding to the line contours is significantly larger than the rectangles corresponding to other numerical contours, and the numbers are of consistent font size, close in position, and occupy roughly the same proportion of the image. Therefore, we can first remove large-area contours based on their rectangle area, then remove contours that are far from all other numerical positions, and finally remove contours whose rectangle area deviates from the average among those that are close in position.

[0026] Specifically, a predefined digital template can be a set of digital regions pre-selected from a large amount of data.

[0027] Specifically, feature matching algorithms can employ matching based on the shape features of numbers in grayscale images, as well as any image processing algorithm capable of matching the extracted digit regions from a simplified image with predefined digit templates. A template-based feature similarity matching algorithm can be expressed as: ; in, For predefined digital template diagrams, The image shows the digital region to be identified. )represent and The feature similarity between them, where index is the index of the maximum similarity. The numbers to be identified.

[0028] Specifically, after digital identification and positioning, the combined numbers can be filtered based on prior knowledge of the shape parameters of the reinforcing bars and the positional relationships between the numbers.

[0029] Step S3: Obtaining the Reinforcing Steel Process Category Considering that the dimensions of reinforcement drawings are generally small, and they are often non-square and vary in size, directly processing these drawings using existing depthwise networks presents size issues. Therefore, an improvement to the transformer-based vit framework is implemented by modifying the network's input dimensions and the convolution kernels in the patch blocks to adapt the network to small targets and non-square inputs. The improved vit framework is then used to classify the reinforcement drawings and obtain the corresponding process categories.

[0030] Specifically, the improved Transformer-based vit framework includes: first, analyzing the training data, including the dimensions and aspect ratios of the simplified diagrams. After analysis, representative values ​​such as the median of the simplified diagram dimensions can be selected as the input dimensions for the vit network. Simultaneously, to accommodate the non-square nature of the reinforcement diagrams, the patch operation in vit uses a 3x5 non-square convolution kernel. The overall network frame is as follows: Figure 4 As shown.

[0031] Specifically, the improved transformer-based vit model can employ data augmentation during the training phase to compensate for the inadequacy of software data collection. Considering that the process type of the reinforcement diagram is independent of the numerical values ​​on the diagram and only related to the line layout, a series of parameter values ​​can be randomly generated and the original parameters can be replaced using the parameter positions obtained in step S2, thereby enriching the data.

[0032] Step S4: Obtaining processing steps Based on a predefined database of relationships between simplified diagram types and parameters, and other information from the material list obtained through the software, the processing steps for this type of steel reinforcement are determined. For example... Figure 5As shown, the numbers on both sides of the steel bar diagram are 150, obtained through digital recognition. The steel bar type is identified as 3 (from the fourth category in the predefined categories), indicating two protruding steel bar ends with a length of 11.9. Furthermore, the diameter of the steel bar is C8, obtained from the file line index after unifying the data format. Combining this information, the processing steps are determined, including the total amount of steel bar used (L = (150)*2 + (150)*2 + (0*7.8*8) + (11.9*2*8) and the number of steel bars.

[0033] A two-stage robotic rebar billet recognition system includes the following steps: Step S1: Rebar Material List Data Processing Module The software retrieves the rebar material list file, processes the file into a predefined format, extracts the rebar sketches from all rows of the material list file, converts them into a unified format file, and uses all the rebar sketches as the output of this module.

[0034] Step S2: Rebar Shape Parameter Extraction Module The module reads batch reinforcement drawings, obtains all contours on the drawings based on contour extraction algorithms, filters out digital contours based on the area features of the contours, extracts numbers on the reinforcement drawings based on template feature matching algorithms, and obtains the shape parameters of the reinforcement based on the positional relationship between the numbers, which are then used as the output of this module.

[0035] Step S3: Rebar Process Category Extraction Module Batch read rebar sketches and preprocess them to the dimensions of the network input, including but not limited to scaling and cropping. Input the rebar sketches into a pre-trained type classification network to obtain the types of all rebar sketches as the output of this module.

[0036] Step S4: Rebar Processing Procedure Acquisition Module The system reads other parameters of each line of steel bars from the output file of the steel bar material list data processing module, the steel bar process type output by the steel bar process category extraction module, and the steel bar shape parameters output by the steel bar shape parameter extraction module, and calculates the processing procedures based on certain rules.

[0037] The preferred embodiments of the present invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to any specific implementation. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention. The invention is limited only by the claims and their full scope and equivalents.

Claims

1. A two-stage-based robot steel material single identification method, characterized in that, Comprising the following steps: Step S1: Steel reinforcement material order data processing; According to the type of software used, export the material order data file, and count and organize all data formats. Process all types of steel reinforcement material order data into a unified format, which can be but is not limited to excel format. According to the unified format of the data, batch extract the steel reinforcement sketch in the material order data, Step S2: Steel reinforcement shape parameter acquisition; Pretreat the steel reinforcement sketch, extract the digital area on the sketch based on the traditional image processing method, identify and locate the shape parameters of the steel reinforcement based on the feature matching algorithm and the predefined digital template, and combine the numbers according to the positional relationship between the numbers to obtain the parameters representing the shape of the steel reinforcement, Step S3: Steel reinforcement process category acquisition; Considering that the size of the steel reinforcement sketch is generally small and non-square and the size is diverse, directly using the existing deep network to process the sketch has size problems, therefore, the transformer-based vit framework is improved to adapt to small targets and non-square inputs. The improved vit is used to classify the steel reinforcement sketch to obtain the corresponding process category, wherein, Step S4: Processing process acquisition; Based on the relationship library between the predefined sketch type and the sketch parameters, and the other information of the material order obtained by the software, the processing process of the steel reinforcement of this type is obtained; Through the digital recognition method, it can be obtained that the numbers of the two edges on the steel reinforcement sketch are both 150, and through the sketch type recognition, it can be obtained that the type of the steel reinforcement is 3 (from the fourth type in the predefined category), therefore, there are two prominent steel reinforcement heads, and the length is 11.

9. In addition, the diameter of the steel reinforcement can be obtained from the file line index after the unified data format (obtained in step S1), which is C8, combined with these information, the processing process is obtained, including the total amount of one steel reinforcement L = (150)*2+(150)*2+(0*7.8*8)+(11.9*2*8) and the number of steel reinforcements.

2. The two-stage based method for single reinforcement material recognition of a robot according to claim 1, wherein, The pretreatment in step S2 includes binarization processing of the steel reinforcement sketch, size scaling and filtering of the binarized image, traditional image processing method to extract the digital area on the sketch, including but not limited to first extracting all the contours on the binarized image, and then screening the digital contours based on the area and other features of the contours.

3. The two-stage based method for single rebar material recognition of a robot according to claim 1, wherein, The predefined digital template in step S2 can be a set of digital region pre-selected from a large amount of data, and the feature matching algorithm can adopt a matching based on the digital shape features in the gray scale image, and all image processing algorithms that can realize the feature matching between the digital region extracted on the sketch and the predefined digital template. The feature similarity matching algorithm based on the template can be expressed as: ; wherein, is a predefined digital template image, is a digital region image to be recognized, and ) represents and a feature similarity between and, index is an index taking the maximum similarity, is a recognized number; Specifically, after digital recognition and positioning, the combined numbers can be screened according to the prior knowledge of the shape parameters of the steel reinforcement and the positional relationship between the numbers.

4. The two-stage based method for single reinforcement material recognition of a robot according to claim 1, wherein, The improved transformer-based vit framework in step S3 includes analyzing the training data, including the size of the sketch, the aspect ratio of the sketch, etc. After analysis, the median of the sketch size and other representative values can be selected as the input size of the vit network. At the same time, in order to adapt to the non-square characteristics of the steel bar sketch, the patch division operation in vit selects a non-square convolution kernel for implementation. The improved transformer-based vit model can adopt data enhancement in the training stage to make up for the lack of software data collection. Considering that the process type of the steel bar sketch is irrelevant to the values on the sketch, but only related to the layout of the lines on the sketch, a series of parameter values can be randomly generated, and the original parameters can be replaced through the parameter positions obtained in step S2, so as to expand the richness of the data.

5. A two-stage based robotic steel material single identification system, characterized in that, Comprising the following steps: Step S1: Steel material list data processing module; Obtain the steel material list file from the software, process the file into a predefined format, extract all the steel bar sketches on the material list file, and convert the file and all the steel bar sketches into a unified format as the output of this module, Step S2: Steel shape parameter extraction module; Batch read the steel bar sketch, obtain all the contours on the sketch based on the contour extraction algorithm, select the digital contours based on the area characteristics of the contours, extract the numbers on the steel bar sketch based on the template feature matching algorithm, and obtain the shape parameters of the steel bar based on the positional relationship between the numbers as the output of this module, Step S3: Steel shape parameter extraction module; Batch read the steel bar sketch, preprocess the steel bar sketch into the size of the network input, including but not limited to scaling, cropping. Input the steel bar sketch into the pre-trained type classification network, and obtain the type of all the steel bar sketches as the output of this module, Step S4: Steel processing procedure acquisition module; Read the other parameters of each row of steel in the file output by the steel material list data processing module, the steel process type output by the steel process type extraction module, and the shape parameters of the steel output by the steel shape parameter extraction module, and calculate the processing procedure based on certain rules.

Images