A steel bar binding quality automatic identification and defect repairing construction method based on machine vision
By constructing a global benchmark adaptive update mechanism and dynamic correction compensation technology, the problem of repair inaccuracy caused by benchmark offset in rebar binding quality inspection was solved, and efficient and accurate repair operations were achieved in a dynamic environment.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- HEBEI CONSTR GRP
- Filing Date
- 2026-03-17
- Publication Date
- 2026-06-02
AI Technical Summary
Existing technologies are unable to adapt to dynamic disturbance environments in the quality inspection and defect repair of rebar tying at construction sites, leading to benchmark shifts and inaccurate repair operations. Furthermore, the lack of a dynamic compensation mechanism throughout the entire process affects the stability and accuracy of the operation.
A global baseline adaptive update mechanism is constructed. The intersection of steel bars is tracked in real time by machine vision as the global baseline anchor point. The overall pose transformation matrix of the global baseline system is calculated, the coordinates of the defect target point are corrected in real time, and the dynamic correction and pose compensation are performed by the repair execution mechanism to ensure the accuracy and stability of the repair operation.
It enables automated identification and high-precision repair of rebar binding quality in complex dynamic environments, solves the problem of repair inaccuracy caused by benchmark offset, and improves the operational stability and repair accuracy at the construction site.
Smart Images
Figure CN122134705A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of automated construction and machine vision inspection technology, specifically to a construction method for automatic identification and defect repair of rebar binding quality based on machine vision. Background Technology
[0002] Reinforcing bar tying is a core process in the construction of the main structure of a building project, and its quality directly determines the load-bearing capacity, durability, and overall safety performance of the reinforced concrete structure. Currently, on construction sites, the inspection and repair of reinforcing bar tying quality largely rely on manual labor, which suffers from many problems such as low work efficiency, high rate of missed inspections, poor consistency in repair quality, high labor costs, and significant safety risks associated with working at heights. This makes it difficult to meet the high-efficiency, high-quality, and standardized construction requirements of modern building construction.
[0003] With the development of machine vision and construction robot technology, machine vision-based methods for identifying rebar tying quality and automated tying robot solutions have emerged, achieving to some extent the automated detection of rebar tying quality and the automated execution of some tasks. However, in actual construction site applications, existing technologies have the following technical shortcomings:
[0004] Construction sites are highly dynamic disturbance environments, with various dynamic disturbance factors such as the slight displacement of steel mesh caused by people stepping on it and equipment collisions, high-frequency vibration of the work platform, and dynamic obstruction and interference caused by the movement of people and equipment. Most existing technologies use a static benchmark system calibrated before construction. Once the benchmark system is disturbed and shifts, it will directly lead to inaccurate positioning benchmarks for subsequent defect identification and deviation of target point coordinates for repair execution, ultimately causing the repair operation to fail.
[0005] Existing automated repair solutions lack a dynamic compensation mechanism throughout the entire process, making it impossible to correct deviations caused by dynamic disturbances on site in real time. They can only achieve fixed-point operations in static environments. In construction sites with dynamic disturbances, the operation stability is extremely poor, the repair accuracy drops significantly, and problems such as end effectors colliding with steel bars and causing secondary damage to the structure may even occur.
[0006] Therefore, there is an urgent need for a method that can adapt to dynamic disturbances at the construction site and achieve real-time updates of benchmarks and real-time correction of operations. Summary of the Invention
[0007] The purpose of this invention is to overcome the shortcomings of existing technologies and provide a machine vision-based method for automatic identification and defect repair of rebar tying quality. This method addresses the pain points of identification benchmark shifts caused by minor displacements of rebar due to external forces, vibrations of work platforms, and dynamic interference from personnel and equipment at construction sites. It constructs a global benchmark adaptive update mechanism that spans the entire operation cycle of identification, repair, and re-inspection. By using rebar intersections as global benchmark anchor points, the spatial position changes of these anchor points are continuously tracked at high-frequency monitoring. When a displacement deviation is detected, the overall pose transformation matrix of the global benchmark system is calculated in real time using singular value decomposition (SVD) based on the coordinate pairs of all valid benchmark anchor points. This transformation matrix accurately represents the three-dimensional rotation and translation variables generated by construction dynamic disturbances, achieving a quantitative description of the spatial pose shift of the global benchmark system. Furthermore, homogeneous coordinate transformation synchronously transmits the pose shift of the global benchmark system to all defect target points, ensuring that the three-dimensional coordinates of the defect target points are strictly synchronized with the global benchmark anchor points. This solves the problem of subsequent repair operations being inaccurate due to benchmark shifts, achieving continuity and consistency of the spatial benchmark in complex dynamic environments.
[0008] To solve the above-mentioned technical problems, this invention provides the following technical solution: a construction method for automatic identification and defect repair of rebar tying quality based on machine vision, the specific steps of which are as follows:
[0009] S100. The target rebar area is scanned by the machine vision acquisition unit, the fixed feature points of the rebar intersections in the area to be constructed are extracted, an initial global reference system is constructed, the three-dimensional spatial coordinates of the initial global reference anchor point in the world coordinate system of the construction scene are determined, the rebar intersections with binding defects are identified as defect target points, and the initial three-dimensional positioning coordinates of the defect target points relative to the global reference anchor point are calculated to generate an initial repair operation task sequence.
[0010] S200. During the entire operation cycle, the global reference anchor points are visually tracked and monitored in real time. The spatial displacement deviation of the global reference anchor points caused by construction dynamic disturbances is calculated. The global reference system is updated in real time based on the spatial displacement deviation data, and the dynamic correction of the three-dimensional spatial coordinates of all global reference anchor points is completed.
[0011] S300: Based on the updated global benchmark system and the spatial displacement deviation data of the global benchmark anchor point, the initial three-dimensional positioning coordinates of all defect target points are synchronously corrected and updated, and the dynamic corrected accurate three-dimensional coordinates of the defect target points are output.
[0012] S400: Based on the precise three-dimensional coordinates of the defect target point after dynamic correction, the initial working path of the repair actuator and the target pose of the end effector are generated. At the same time, based on the real-time spatial displacement deviation data of the global reference anchor point and the coordinate update data of the defect target point, the working path, motion trajectory and pose of the repair actuator are dynamically corrected and compensated, and the repair actuator is controlled to complete the precise repair operation of the defect target point.
[0013] After completing the repair work on single and batch defect targets, the S500 collects the image data of the repaired steel mesh through the machine vision acquisition unit, and completes the re-inspection and verification of the repair quality based on the real-time updated global benchmark system.
[0014] Furthermore, in S100, the process of constructing the initial global reference system and the initial global reference anchor point is as follows:
[0015] The global RGB image and depth point cloud data of the steel mesh in the construction area are acquired simultaneously by the machine vision acquisition unit. The global RGB image and depth point cloud data are preprocessed to obtain high-quality image data and standardized point cloud data.
[0016] Based on the high-quality image data and point cloud data, the pixel coordinates and three-dimensional spatial coordinates of all steel bar intersections in the construction area are identified and extracted through a feature extraction network to generate a steel bar intersection feature set. At the same time, a digital twin model of the steel bar mesh topology is constructed based on the horizontal and vertical distribution pattern of the steel bar mesh.
[0017] Based on the feature set of rebar intersections and the digital twin model, rebar intersections that meet the preset screening rules of uniform distribution, feature stability, unobstructedness, and anti-disturbance are selected as candidate reference anchor points.
[0018] Stability verification is performed on all candidate reference anchor points. By solving the coordinates of multiple consecutive frames of images, unstable candidate reference anchor points whose coordinate fluctuations exceed the preset stability threshold are eliminated, and multiple non-coplanar steel bar intersection points are obtained as initial global reference anchor points.
[0019] Calculate the initial three-dimensional spatial coordinates of all initial global reference anchor points in the world coordinate system of the construction scene, record the image feature descriptors of each initial global reference anchor point, and construct an initial global reference system based on the initial global reference anchor points.
[0020] Furthermore, in S200, the construction dynamic disturbances include the displacement of the reinforcing bars caused by external forces, the vibration of the work platform, the dynamic interference caused by personnel movement, and the dynamic interference caused by equipment operation. Throughout the entire operation cycle, the machine vision acquisition unit captures the dynamic disturbances of the global reference anchor points in real time. When the original global reference anchor points are obscured, features are lost, or tracking fails, a global reference anchor point dynamic replacement mechanism is executed: based on the image data and digital twin model of the current reinforcing mesh, new candidate reference anchor points that meet the screening rules are re-selected.
[0021] Based on the current global benchmark system constructed from the remaining valid global benchmark anchor points, the three-dimensional spatial coordinates of the new candidate benchmark anchor points are solved and their stability is verified.
[0022] After verification, the invalid global benchmark anchor point is replaced with a new candidate benchmark anchor point, and the global benchmark anchor point is updated to ensure the continuity and stability of the global benchmark system.
[0023] Furthermore, in S200, the calculation steps for the spatial displacement deviation of the global reference anchor point are as follows:
[0024] The machine vision acquisition unit collects the current image features and depth information of the global reference anchor points in real time, and calculates the real-time three-dimensional spatial coordinates of each global reference anchor point in the world coordinate system of the construction scene at the current moment.
[0025] Set the global reference anchor point at the current moment. The real-time three-dimensional spatial coordinates are Compared to the previous moment Updated global reference anchor point 3D spatial coordinates By comparing and calculating the spatial pose transformation, the overall spatial pose offset of the global reference system is obtained. , , The first Each global reference anchor point was in the construction scene's world coordinate system at the previous moment. axis, axis, The coordinate values along the axis. , , The first Each global reference anchor point is located in the world coordinate system of the construction scenario at the current moment. axis, axis, The coordinate values along the axis. Indicates the transpose operation;
[0026] Based on the coordinates of all valid global reference anchor points, solve for the real-time pose transformation matrix in the world coordinate system of the construction scene. ,in, Let be the rotation matrix, representing the three-dimensional rotation variables of the global reference system caused by construction dynamic disturbances. Let be the translation vector, representing the three-dimensional translation variable of the global reference system caused by construction dynamic disturbances. It is the transpose matrix;
[0027] Consistency verification is performed on the spatial displacement deviations of all valid global reference anchor points, outlier data is removed, and the real-time pose transformation matrix is optimized based on the verified valid data. It outputs the final overall spatial pose offset of the global reference system.
[0028] Furthermore, in S300, the process of synchronously correcting and updating the initial three-dimensional positioning coordinates of the defect target is as follows:
[0029] Real-time pose transformation matrix obtained from S300 calculation For all defect targets to be repaired, the initial three-dimensional positioning coordinates of the defect targets at the previous time step are extracted and converted into homogeneous coordinate form, that is, for the th... The defective target at the previous moment The corrected 3D positioning coordinates are Convert to homogeneous coordinate form The homogeneous coordinate form is used for multiplication with a 4×4 homogeneous transformation matrix, and the expression is: ,in, , , The first The defective target at the previous moment The coordinate values of the X, Y, and Z axes in the world coordinate system of the construction scene;
[0030] Based on the real-time pose transformation matrix The solution yields the defect target point at the current time. Globally corrected 3D homogeneous coordinates ,in, Given a 4×1 homogeneous coordinate matrix, extracting the first three elements yields the defect target point at the current time step. Globally corrected 3D spatial coordinates in the world coordinate system of the construction scene;
[0031] After completing the global coordinate synchronization correction of all defect target points, the dynamically corrected precise three-dimensional coordinates of the defect target points are output and synchronized to the motion controller of the repair actuator in real time.
[0032] Furthermore, in S400, based on the dynamically corrected precise three-dimensional coordinates of the defect target point, combined with the operational safety constraints of the repair actuator and the obstacle distribution data at the construction site, the initial operational path of the repair actuator from its current pose to the target defect target point is planned, the initial motion trajectory of each joint of the repair actuator and the target pose of the end effector are generated, and the real-time spatial displacement deviation data of the global reference anchor point and the real-time coordinate update data of the defect target point are transmitted to the motion controller of the repair actuator in real time. The operational path of the repair actuator is replanned in real time, the motion trajectory of the repair actuator is interpolated and corrected online, and the pose of the end effector is compensated and adjusted in real time.
[0033] Furthermore, in S500, after completing the repair work of single and batch defect targets, the machine vision acquisition unit collects image data and depth point cloud data of the corresponding area of the repaired rebar mesh, and simultaneously triggers the real-time update process of the global benchmark system to obtain the latest global benchmark system data. Based on the latest global benchmark system, the characteristic parameters of the repaired rebar binding nodes are calculated. The characteristic parameters include the binding node position, the number of wire wrapping turns, the binding tightness, the rebar spacing, and the rebar intersection offset. The calculated repaired characteristic parameters are compared with the preset qualified rebar binding quality standard parameters to determine whether the repair work of the defect target is qualified.
[0034] Furthermore, in S500, when the re-inspection is deemed qualified, the defect information of the defect target, the repair operation data, and the re-inspection results are archived and recorded, and the next repair task cycle begins.
[0035] If the re-inspection is deemed unqualified or new construction dynamic disturbances are detected during the re-inspection process, S200-S400 will be immediately re-triggered to control the repair execution mechanism to complete the secondary repair work. After the secondary repair is completed, the re-inspection verification process will be executed again until the re-inspection is qualified.
[0036] If new undetected defects in rebar binding quality are identified during the re-inspection process, the defect target points corresponding to the new defects will be added to the initial repair task sequence and included in subsequent repair operations.
[0037] Furthermore, the method is implemented based on a supporting hardware system, which includes a machine vision acquisition unit, a repair execution mechanism, an edge computing unit, a motion controller, and a mobile work platform.
[0038] The machine vision acquisition unit includes multiple synchronously triggered industrial RGB cameras and multiple laser depth cameras. All cameras are fixedly deployed on a mobile work platform, with their relative poses to the repair execution mechanism fixed and hand-eye calibration completed. The output end of the machine vision acquisition unit is communicatively connected to the edge computing unit for real-time acquisition and transmission of image data and depth data of the steel mesh.
[0039] The repair actuator is a six-degree-of-freedom robotic arm. The end of the six-degree-of-freedom robotic arm is equipped with a fully automatic rebar binding actuator. The control end of the six-degree-of-freedom robotic arm is connected to the motion controller to complete the automated repair operation of the defect target point.
[0040] The edge computing unit is used to perform functions such as image data processing, rebar binding quality defect identification, global reference anchor point tracking, spatial displacement deviation calculation, defect target point coordinate correction, and job task scheduling.
[0041] The motion controller is used to perform control functions such as motion trajectory planning, real-time dynamic correction, and pose compensation of the repair actuator;
[0042] The mobile work platform is used to carry and fix the machine vision acquisition unit, repair execution mechanism, edge computing unit, and motion controller to achieve full coverage of the area to be constructed.
[0043] Compared with existing technologies, this machine vision-based automatic identification and defect repair method for rebar tying quality has the following advantages:
[0044] I. This invention addresses the pain points of identification benchmark shifts caused by minor displacements of reinforcing bars due to external forces, vibrations of work platforms, and dynamic interference from personnel and equipment at construction sites. It constructs a global benchmark adaptive update mechanism that spans the entire operation cycle of identification, repair, and re-inspection. By using the intersections of reinforcing bars as global benchmark anchor points, the spatial position changes of these anchor points are continuously tracked at high-frequency monitoring. When a displacement deviation is detected, the overall pose transformation matrix of the global benchmark system is calculated in real time using singular value decomposition (SVD) based on the coordinate pairs of all valid benchmark anchor points. This transformation matrix accurately characterizes the three-dimensional rotation and translation variables generated by construction dynamic disturbances, achieving a quantitative description of the spatial pose shift of the global benchmark system. Furthermore, homogeneous coordinate transformation synchronously transmits the pose shift of the global benchmark system to all defect target points, ensuring that the three-dimensional coordinates of the defect target points are strictly synchronized with the global benchmark anchor points. This solves the problem of subsequent repair operations being completely inaccurate due to benchmark shifts, achieving continuity and consistency of the spatial benchmark in complex dynamic environments.
[0045] Second, this invention addresses the shortcomings of target inaccuracy and poor operational stability caused by continuous disturbance during the repair process. It establishes a real-time control link, continuously monitoring the real-time displacement of the global reference anchor point as the repair actuator moves toward the target defect point. The deviation data is transmitted to the motion controller, and the working path and end pose of the actuator are dynamically corrected. This ensures that the end effector can accurately hit the target point even under multiple disturbances such as platform vibration and rebar micro-movement, thus guaranteeing the stability and accuracy of repair operations in complex construction environments.
[0046] Other advantages, objectives and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination or study, or may be learned from the practice of the invention. Attached Figure Description
[0047] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.
[0048] Figure 1 A flowchart illustrating the steps of a machine vision-based construction method for automatic identification and defect repair of rebar tying quality;
[0049] Figure 2 This is a flowchart illustrating the steps involved in establishing the initial global reference system and the initial global reference anchor point in an embodiment of the present invention.
[0050] Figure 3 This is a flowchart illustrating the steps of S500 in this embodiment of the invention for quality re-inspection and verification after repair. Detailed Implementation
[0051] To better understand the above technical solutions, a detailed description of the solutions will be provided below in conjunction with the accompanying drawings and specific embodiments. Obviously, the described embodiments are merely some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.
[0052] To address the shortcomings of existing technologies in rebar tying quality identification and repair, which suffer from issues such as positioning benchmark failure, defect target point coordinate deviation, reduced repair accuracy, and even failure due to dynamic disturbances at the construction site, this invention provides a machine vision-based automatic rebar tying quality identification and defect repair construction method. This method aims to establish a global benchmark dynamic update mechanism throughout the entire work cycle, a defect target point coordinate linkage correction mechanism, and a real-time dynamic correction and pose compensation mechanism for the repair execution mechanism. This forms a comprehensive control system from benchmark construction, dynamic tracking, coordinate correction, precise execution to quality re-inspection, enabling automated identification and high-precision repair of rebar tying quality defects, thereby improving the intelligence level and work quality of building construction.
[0053] This invention is primarily applied to the quality inspection and automated repair of rebar tying at construction sites, particularly for identifying and repairing defects in the tying quality of rebar mesh during main structure construction. In these scenarios, rebar mesh often experiences slight displacement due to construction disturbances. Traditional static calibration methods are inadequate for dynamic environments, leading to the failure of automated equipment. This invention uses rebar intersections as global reference anchor points, relying on a machine vision acquisition unit to track the spatial displacement of these anchor points in real time. It calculates the overall pose transformation matrix of the global reference system and simultaneously corrects the three-dimensional coordinates of all defect target points. Simultaneously, the deviation data is transmitted in real time to the motion controller of the repair actuator, dynamically compensating for the work path, motion trajectory, and end-effector pose. This ensures that the repair actuator can accurately hit the target point even under dynamic disturbances, solving the core problem of inaccurate repair caused by reference offset in existing technologies.
[0054] This embodiment provides a construction method for automatic identification and defect repair of rebar binding quality based on machine vision, which is implemented with a supporting hardware system. The hardware system includes a machine vision acquisition unit, a repair execution mechanism, an edge computing unit, a motion controller, and a mobile work platform.
[0055] The machine vision acquisition unit includes multiple synchronously triggered industrial RGB cameras and multiple laser depth cameras. All cameras are fixedly deployed on a mobile work platform, with their relative poses to the repair actuators fixed, and hand-eye calibration is completed to ensure accurate conversion between image data and the robot coordinate system.
[0056] The repair actuator is a six-degree-of-freedom robotic arm, with a fully automatic rebar binding actuator at its end for completing automated repair work on defect targets;
[0057] The edge computing unit is used to perform functions such as image data processing, rebar binding quality defect identification, global reference anchor point tracking, spatial displacement deviation calculation, defect target point coordinate correction, and job task scheduling.
[0058] The motion controller is used to perform control functions such as motion trajectory planning, real-time dynamic correction, and pose compensation of the repair actuator.
[0059] The mobile work platform is used to support and fix the above-mentioned units, so as to achieve full coverage of the area to be constructed.
[0060] The following combination Figure 1 The flowchart shown illustrates a construction method for automatic identification and defect repair of rebar tying quality based on machine vision, and provides a detailed explanation of the specific implementation steps of the control method of this invention:
[0061] S100 Initial Global Baseline System Construction: The target rebar area is scanned by the machine vision acquisition unit, the fixed feature points of the rebar intersections in the area to be constructed are extracted, the initial global baseline system is constructed, the three-dimensional spatial coordinates of the initial global baseline anchor point in the world coordinate system of the construction scene are determined, the rebar intersections with binding defects are identified as defect target points, and the initial three-dimensional positioning coordinates of the defect target points relative to the global baseline anchor points are calculated to generate the initial repair operation task sequence.
[0062] S200, Global Baseline Dynamic Update: Throughout the entire operation cycle, the global baseline anchor points are visually tracked and monitored in real time. The spatial displacement deviation of the global baseline anchor points caused by construction dynamic disturbances is calculated. Based on the spatial displacement deviation data, the global baseline system is updated in real time to complete the dynamic correction of the three-dimensional spatial coordinates of all global baseline anchor points.
[0063] S300, Synchronous Correction of Three-Dimensional Coordinates of Defect Targets: Based on the updated global benchmark system and the spatial displacement deviation data of the global benchmark anchor point, the initial three-dimensional positioning coordinates of all defect targets are synchronously corrected and updated, and the accurate three-dimensional coordinates of the defect targets after dynamic correction are output.
[0064] S400, Real-time Dynamic Correction and Pose Compensation for Repair Operations: Based on the precise three-dimensional coordinates of the defect target point after dynamic correction, the initial operation path of the repair actuator and the target pose of the end effector are generated. At the same time, based on the real-time spatial displacement deviation data of the global reference anchor point and the coordinate update data of the defect target point, the operation path, motion trajectory and pose of the repair actuator are dynamically corrected and compensated, so as to control the repair actuator to complete the precise repair operation of the defect target point.
[0065] S500, Post-Repair Quality Re-inspection and Verification: After completing the repair work on single and batch defect targets, the machine vision acquisition unit collects image data of the repaired steel mesh, and the repair quality is re-inspected and verified based on the real-time updated global benchmark system.
[0066] In the specific implementation process, the machine vision acquisition unit performs an initial full-coverage scan of the target rebar area, extracts fixed feature points of the rebar intersections within the construction area, and constructs an initial global benchmark system. This establishes a unified benchmark foundation for subsequent dynamic tracking and coordinate correction throughout the entire operation cycle, which is completed independently by the edge computing unit. The repair execution mechanism is in standby mode. Figure 2 As shown, the specific steps for constructing the initial global benchmark system and the initial global benchmark anchor points include:
[0067] The machine vision acquisition unit simultaneously triggers multiple industrial RGB cameras and laser depth cameras to acquire global RGB images and depth point cloud data of the steel mesh in the construction area. After receiving the raw images and point cloud data, the edge computing unit performs preprocessing operations: denoising, contrast enhancement, and distortion correction are performed on the RGB images to obtain high-quality image data; filtering, downsampling, and coordinate normalization are performed on the depth point cloud data to obtain standardized point cloud data.
[0068] Based on high-quality image data and standardized point cloud data, the edge computing unit uses a pre-trained deep learning feature extraction network to identify and extract the pixel coordinates and three-dimensional spatial coordinates of all rebar intersections in the construction area, generating a rebar intersection feature set. Based on the horizontal and vertical distribution pattern of the rebar mesh, combined with the extracted intersection coordinate data, a digital twin model of the rebar mesh topology is constructed. This model contains the topological connection relationship and spatial location information of all rebar intersections, serving as the basic data support for subsequent benchmark selection and coordinate calculation.
[0069] Based on the feature set of rebar intersections and the digital twin model, rebar intersections that meet the preset screening rules of uniform distribution, feature stability, unobstructedness, and disturbance resistance are selected as candidate reference anchor points. The specific screening rules include: candidate anchor points should have obvious corner features in the image to facilitate long-term tracking; candidate anchor points should be evenly distributed throughout the entire rebar mesh area to avoid local concentration; there should be no significant obstructions around the candidate anchor points to ensure long-term visibility; and the rebars at the location of the candidate anchor points should have high structural stability and be less prone to independent displacement due to construction disturbances.
[0070] Stability verification is performed on all candidate reference anchor points. The edge computing unit continuously collects multiple frames of image data, calculates the three-dimensional spatial coordinates of each candidate anchor point in different frames, calculates its coordinate fluctuation range, and removes unstable candidate anchor points whose coordinate fluctuation exceeds the preset stability threshold. Multiple non-coplanar steel bar intersections are obtained as initial global reference anchor points.
[0071] The initial three-dimensional spatial coordinates of all initial global reference anchor points in the world coordinate system of the construction scene are calculated, and the image feature descriptors of each initial global reference anchor point are recorded. Based on these initial global reference anchor points, an initial global reference system is constructed. The edge computing unit identifies the rebar intersections with binding defects as defect targets based on the rebar intersection feature set and the preset binding quality judgment rules. The unit calculates the initial three-dimensional positioning coordinates of each defect target point relative to the global reference anchor points, generates an initial repair operation task sequence, and stores it in the task queue of the edge computing unit, waiting for subsequent repair execution.
[0072] Throughout the entire operation cycle, the machine vision acquisition unit continuously performs real-time visual tracking and monitoring of the global reference anchor points at a high-frequency acquisition frequency. It calculates the spatial displacement deviation of the global reference anchor points caused by construction dynamic disturbances, including but not limited to: displacement of reinforcing bars due to external forces, vibration of the work platform, dynamic interference from personnel movement, and dynamic interference from equipment operation. The machine vision acquisition unit captures the image features and depth information of the global reference anchor points in real time. The edge computing unit calculates the real-time three-dimensional spatial coordinates of each global reference anchor point in the world coordinate system of the construction scene at the current moment based on each frame of image data. The calculation steps for the spatial displacement deviation of the global reference anchor points are as follows:
[0073] For the There are several global reference anchor points, whose three-dimensional spatial coordinates after the update at the previous time k are: At the current moment, the real-time three-dimensional spatial coordinates of k+1 are: ,in, , , The first Each global reference anchor point was in the construction scene's world coordinate system at the previous moment. axis, axis, The coordinate values along the axis. , , The first Each global reference anchor point is located in the world coordinate system of the construction scenario at the current moment. axis, axis, The coordinate values along the axis. Indicates the transpose operation;
[0074] The edge computing unit compares the current coordinates of all valid global reference anchor points with their previous coordinates, and obtains the overall spatial pose offset of the global reference system through spatial pose transformation calculation. The specific solution method is as follows: based on the coordinate pairs of all valid global reference anchor points, the real-time pose transformation matrix in the world coordinate system of the construction scene is solved through singular value decomposition. , where R is a 3×3 rotation matrix, representing the three-dimensional rotation variable of the global reference system caused by construction dynamic disturbance, and t is a 3×1 translation vector, representing the three-dimensional translation variable;
[0075] During the solution process, the edge computing unit performs consistency verification on the spatial displacement deviation of all valid global reference anchor points, removes abnormal data points whose displacement is abnormal due to local disturbances, and optimizes the solution of the real-time pose transformation matrix based on the verified valid data. It outputs the final overall spatial pose offset of the global reference system.
[0076] When existing global reference anchor points are occluded, features are lost, or tracking fails, the edge computing unit automatically executes a dynamic replacement mechanism for global reference anchor points: based on the image data and digital twin model of the current steel mesh, new candidate reference anchor points are re-selected according to the screening rules in S100; based on the current global reference system constructed from the remaining valid global reference anchor points, the three-dimensional spatial coordinates of the new candidate reference anchor points are calculated, and stability is verified; after the verification is passed, the failed global reference anchor points are replaced with the new candidate reference anchor points, and the global reference anchor point set is updated to ensure that the continuity and stability of the global reference system are not affected by the failure of local anchor points.
[0077] Based on the updated global reference system of S200 and the calculated real-time pose transformation matrix The edge computing unit synchronously corrects and updates the initial three-dimensional positioning coordinates of all defect target points. The specific correction process is as follows:
[0078] For the The defective target point has its three-dimensional positioning coordinates after k-correction at the previous time step as follows: ;
[0079] The edge computing unit converts it into homogeneous coordinate form, that is ,in, , , The first The defective target at the previous moment The coordinate values in the X, Y, and Z axes of the construction scene world coordinate system are then used, and the real-time pose transformation matrix is calculated based on S200. Calculate the globally corrected three-dimensional homogeneous coordinates of the defect target at the current time k+1: By extracting the first three elements, we can obtain the global corrected three-dimensional spatial coordinates of the defect target point in the world coordinate system of the construction scene at the current time k+1.
[0080] The edge computing unit traverses all defect targets in the initial repair task sequence, performs the above coordinate correction calculations in sequence, generates a dynamically corrected accurate three-dimensional coordinate dataset of defect targets, and synchronizes it to the motion controller in real time through a high-speed communication interface as the reference data for subsequent repair execution.
[0081] After receiving the precisely 3D coordinates of the defect target point after dynamic correction, the motion controller, combined with the operational safety constraints of the repair actuator and the obstacle distribution data of the construction site, plans the initial operation path of the repair actuator from its current pose to the target defect target point, and generates the initial motion trajectory of each joint of the repair actuator and the target pose of the end effector. During the movement of the repair actuator towards the target defect target point, the motion controller, based on the received real-time deviation data, performs real-time replanning of the operation path of the repair actuator, online interpolation correction of the motion trajectory, and real-time compensation adjustment of the pose of the end effector. Specifically, within each control cycle, the motion controller updates the data based on the global reference anchor point displacement deviation and defect target point coordinates received at the current moment; it recalculates the expected pose of the end effector relative to the target target point, and generates the expected angles of each joint through inverse kinematics calculation. This drives the servo motors of each joint of the six-degree-of-freedom robotic arm to adjust the motion trajectory and end effector pose in real time, ensuring that the fully automatic rebar tying actuator mounted at the end can still accurately reach the target target position and complete repair operations such as wire tying and fastening under multiple dynamic disturbances such as platform vibration and rebar micro-motion. This method establishes a real-time control link of visual tracking, coordinate correction, and motion control, enabling dynamic response and pose compensation to continuous disturbances during the repair process, thereby improving the stability and accuracy of repair operations in complex dynamic environments.
[0082] After completing the repair of a single defect target point, the machine vision acquisition unit immediately acquires image data and depth point cloud data of the corresponding area of the repaired rebar mesh, simultaneously triggering a real-time update process for the global benchmark system to obtain the latest global benchmark system data. Based on the latest global benchmark system, the edge computing unit calculates the feature parameters of the repaired rebar binding nodes. These feature parameters include, but are not limited to: binding node position, number of wire wraps, binding tightness, rebar spacing, and rebar intersection offset. Figure 3 As shown, the specific steps for quality re-inspection and verification of S500 after repair are as follows:
[0083] The edge computing unit compares the calculated repair feature parameters with the preset qualified rebar binding quality standard parameters to determine whether the repair work of the defect target point is qualified. When the re-inspection determines that it is qualified, the defect information, repair work data and re-inspection results of the defect target point are archived and recorded, and the defect target point is removed from the task queue to enter the next repair task cycle.
[0084] When the re-inspection is deemed unqualified, or when new construction dynamic disturbances are detected during the re-inspection process, causing the benchmark offset to exceed the allowable range, the edge computing unit immediately re-triggers the S200-S400 process, controls the repair execution mechanism to complete the secondary repair work, and executes the re-inspection verification process again after the secondary repair is completed, until the re-inspection is qualified. If the re-inspection fails three times in a row, the system will automatically alarm to prompt manual intervention.
[0085] When new undetected rebar binding quality defects are identified during the re-inspection process, the edge computing unit adds the defect target point corresponding to the new defect to the initial repair task sequence and incorporates it into subsequent repair operations to ensure that no area is missed in the repair.
[0086] Through the above steps, this embodiment relies on the collaborative work of the machine vision acquisition unit, edge computing unit, and repair execution mechanism to construct a full-process control system from initial benchmark construction, dynamic update, coordinate correction, real-time correction to quality re-inspection. It solves the core pain points of benchmark offset and repair inaccuracy caused by dynamic disturbances at the construction site, realizes automated identification and high-precision repair of rebar binding quality defects, improves work efficiency and intelligence level while ensuring construction quality, and adapts to the application needs of various complex building construction sites.
[0087] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.
[0088] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.
Claims
1. A construction method for automatic identification and defect repair of rebar tying quality based on machine vision, characterized in that, The steps of this method are as follows: S100. The target rebar area is scanned by the machine vision acquisition unit, the fixed feature points of the rebar intersections in the area to be constructed are extracted, an initial global reference system is constructed, the three-dimensional spatial coordinates of the initial global reference anchor point in the world coordinate system of the construction scene are determined, the rebar intersections with binding defects are identified as defect target points, and the initial three-dimensional positioning coordinates of the defect target points relative to the global reference anchor point are calculated to generate an initial repair operation task sequence. S200. During the entire operation cycle, the global reference anchor points are visually tracked and monitored in real time. The spatial displacement deviation of the global reference anchor points caused by construction dynamic disturbances is calculated. The global reference system is updated in real time based on the spatial displacement deviation data, and the dynamic correction of the three-dimensional spatial coordinates of all global reference anchor points is completed. S300: Based on the updated global benchmark system and the spatial displacement deviation data of the global benchmark anchor point, the initial three-dimensional positioning coordinates of all defect target points are synchronously corrected and updated, and the dynamic corrected accurate three-dimensional coordinates of the defect target points are output. S400: Based on the precise three-dimensional coordinates of the defect target point after dynamic correction, the initial working path of the repair actuator and the target pose of the end effector are generated. At the same time, based on the real-time spatial displacement deviation data of the global reference anchor point and the coordinate update data of the defect target point, the working path, motion trajectory and pose of the repair actuator are dynamically corrected and compensated, and the repair actuator is controlled to complete the precise repair operation of the defect target point. After completing the repair work on single and batch defect targets, the S500 collects the image data of the repaired steel mesh through the machine vision acquisition unit, and completes the re-inspection and verification of the repair quality based on the real-time updated global benchmark system.
2. The construction method for automatic identification and defect repair of rebar tying quality based on machine vision according to claim 1, characterized in that, In S100, the process of constructing the initial global reference system and the initial global reference anchor point is as follows: The global RGB image and depth point cloud data of the steel mesh in the construction area are acquired simultaneously by the machine vision acquisition unit. The global RGB image and depth point cloud data are preprocessed to obtain high-quality image data and standardized point cloud data. Based on the high-quality image data and point cloud data, the pixel coordinates and three-dimensional spatial coordinates of all steel bar intersections in the construction area are identified and extracted through a feature extraction network to generate a steel bar intersection feature set. At the same time, a digital twin model of the steel bar mesh topology is constructed based on the horizontal and vertical distribution pattern of the steel bar mesh. Based on the feature set of rebar intersections and the digital twin model, rebar intersections that meet the preset screening rules are selected as candidate reference anchor points. Stability verification is performed on all candidate reference anchor points. By solving the coordinates of multiple consecutive frames of images, unstable candidate reference anchor points whose coordinate fluctuations exceed the preset stability threshold are eliminated, and multiple non-coplanar steel bar intersection points are obtained as initial global reference anchor points. Calculate the initial three-dimensional spatial coordinates of all initial global reference anchor points in the world coordinate system of the construction scene, record the image feature descriptors of each initial global reference anchor point, and construct an initial global reference system based on the initial global reference anchor points.
3. The construction method for automatic identification and defect repair of rebar tying quality based on machine vision according to claim 1, characterized in that, In S200, the construction dynamic disturbance includes the displacement of the steel bars caused by external forces, the vibration of the work platform, the dynamic interference caused by personnel movement, and the dynamic interference caused by equipment operation. Throughout the entire operation cycle, the machine vision acquisition unit captures the dynamic disturbance of the global reference anchor point in real time. When the original global reference anchor point is occluded, its features are lost, or the tracking fails, the global reference anchor point dynamic replacement mechanism is executed: based on the image data and digital twin model of the current steel mesh, new candidate reference anchor points that meet the screening rules are re-selected. Based on the current global benchmark system constructed from the remaining valid global benchmark anchor points, the three-dimensional spatial coordinates of the new candidate benchmark anchor points are solved and their stability is verified. After the verification is successful, the invalid global reference anchor point is replaced with the new candidate reference anchor point, and the global reference anchor point is updated.
4. The construction method for automatic identification and defect repair of rebar tying quality based on machine vision according to claim 1, characterized in that, In S200, the calculation steps for the spatial displacement deviation of the global reference anchor point are as follows: The machine vision acquisition unit collects the current image features and depth information of the global reference anchor points in real time, and calculates the real-time three-dimensional spatial coordinates of each global reference anchor point in the world coordinate system of the construction scene at the current moment. Set the global reference anchor point at the current moment. The real-time three-dimensional spatial coordinates are Compared to the previous moment Updated global reference anchor point 3D spatial coordinates By comparing and calculating the spatial pose transformation, the overall spatial pose offset of the global reference system is obtained. , , The first Each global reference anchor point was in the construction scene's world coordinate system at the previous moment. axis, axis, The coordinate values along the axis. , , The first Each global reference anchor point is located in the world coordinate system of the construction scenario at the current moment. axis, axis, The coordinate values along the axis. Indicates the transpose operation; Based on the coordinates of all valid global reference anchor points, solve for the real-time pose transformation matrix in the world coordinate system of the construction scene. ,in, Let be a rotation matrix. It is a translation vector. It is the transpose matrix; Consistency verification is performed on the spatial displacement deviations of all valid global reference anchor points, outlier data is removed, and the real-time pose transformation matrix is optimized based on the verified valid data. It outputs the final overall spatial pose offset of the global reference system.
5. The construction method for automatic identification and defect repair of rebar tying quality based on machine vision according to claim 1, characterized in that, In step S300, the process of synchronously correcting and updating the initial three-dimensional positioning coordinates of the defect target is as follows: Real-time pose transformation matrix obtained from S300 calculation For all defect targets to be repaired, the initial three-dimensional positioning coordinates of the defect targets at the previous time step are extracted and converted into homogeneous coordinate form, that is, for the th... The defective target at the previous moment The corrected 3D positioning coordinates are Convert to homogeneous coordinate form ,in, , , The first The defective target at the previous moment The coordinate values of the X, Y, and Z axes in the world coordinate system of the construction scene; Based on the real-time pose transformation matrix The solution yields the defect target point at the current time. Globally corrected 3D homogeneous coordinates ,in, Given a 4×1 homogeneous coordinate matrix, extracting the first three elements yields the defect target point at the current time step. Globally corrected 3D spatial coordinates in the world coordinate system of the construction scene; After completing the global coordinate synchronization correction of all defect target points, the dynamically corrected precise three-dimensional coordinates of the defect target points are output and synchronized to the motion controller of the repair actuator in real time.
6. The construction method for automatic identification and defect repair of rebar tying quality based on machine vision according to claim 1, characterized in that, In S400, based on the dynamically corrected precise three-dimensional coordinates of the defect target, combined with the operational safety constraints of the repair actuator and the obstacle distribution data at the construction site, the initial operation path of the repair actuator from its current pose to the target defect target is planned, the initial motion trajectory of each joint of the repair actuator and the target pose of the end effector are generated, and the real-time spatial displacement deviation data of the global reference anchor point and the real-time coordinate update data of the defect target are transmitted to the motion controller of the repair actuator in real time. The operation path of the repair actuator is replanned in real time, the motion trajectory of the repair actuator is interpolated and corrected online, and the pose of the end effector is compensated and adjusted in real time.
7. The construction method for automatic identification and defect repair of rebar tying quality based on machine vision according to claim 1, characterized in that, In S500, after completing the repair work of single and batch defect target points, the machine vision acquisition unit collects image data and depth point cloud data of the corresponding area of the repaired rebar mesh, and simultaneously triggers the real-time update process of the global benchmark system to obtain the latest global benchmark system data. Based on the latest global benchmark system, the characteristic parameters of the repaired rebar binding nodes are calculated. The characteristic parameters include the binding node position, the number of wire wrapping turns, the binding tightness, the rebar spacing, and the rebar intersection offset. The calculated repaired characteristic parameters are compared with the preset qualified rebar binding quality standard parameters to determine whether the repair work of the defect target point is qualified.
8. The construction method for automatic identification and defect repair of rebar tying quality based on machine vision according to claim 7, characterized in that, In S500, when the re-inspection is deemed qualified, the defect information of the defect target, the repair operation data, and the re-inspection results are archived and recorded, and the next repair task cycle begins. If the re-inspection is deemed unqualified or new construction dynamic disturbances are detected during the re-inspection process, S200-S400 will be immediately re-triggered to control the repair execution mechanism to complete the secondary repair work. After the secondary repair is completed, the re-inspection verification process will be executed again until the re-inspection is qualified. If new undetected defects in rebar binding quality are identified during the re-inspection process, the defect target points corresponding to the new defects will be added to the initial repair task sequence and included in subsequent repair operations.
9. A construction method for automatic identification and defect repair of rebar tying quality based on machine vision according to claim 1, characterized in that, The method is implemented based on a supporting hardware system, which includes a machine vision acquisition unit, a repair execution mechanism, an edge computing unit, a motion controller, and a mobile work platform. The machine vision acquisition unit includes multiple synchronously triggered industrial RGB cameras and multiple laser depth cameras. All cameras are fixedly deployed on a mobile work platform, with their relative poses to the repair execution mechanism fixed and hand-eye calibration completed. The output end of the machine vision acquisition unit is communicatively connected to the edge computing unit for real-time acquisition and transmission of image data and depth data of the steel mesh. The repair actuator is a six-degree-of-freedom robotic arm. The end of the six-degree-of-freedom robotic arm is equipped with a fully automatic rebar binding actuator. The control end of the six-degree-of-freedom robotic arm is connected to the motion controller to complete the automated repair operation of the defect target point. The edge computing unit is used to perform functions such as image data processing, rebar binding quality defect identification, global reference anchor point tracking, spatial displacement deviation calculation, defect target point coordinate correction, and job task scheduling. The motion controller is used to perform control functions such as motion trajectory planning, real-time dynamic correction, and pose compensation of the repair actuator; The mobile work platform is used to carry and fix the machine vision acquisition unit, repair execution mechanism, edge computing unit, and motion controller to achieve full coverage of the area to be constructed.