Multi-scene adaptive steel bar connecting robot visual intelligent operation method and system

Abstract

The present application relates to the field of building construction automation, and provides a multi-scene adaptive steel bar connecting robot visual intelligent operation method and system, wherein through a low-confidence node multi-view review mechanism, a steel bar topological structure diagram, a graph neural network path traversal and a kinematics constraint trajectory optimization, the present application solves the problem of unreliable recognition caused by complex illumination and dense occlusion in multiple scenes, and realizes high-precision, collision-free and high-efficiency steel bar connecting automation operation.

Description

Technical Field

[0001] This invention relates to the field of construction automation, specifically to a multi-scenario adaptive visual intelligent operation method and system for rebar connection robots. Background Technology

[0002] Currently, rebar tying mainly employs manual tying, handheld electric / pneumatic tying guns, fixed laser welding equipment, and a small number of track-mounted vision-guided tying robots. Manual tying is labor-intensive and inefficient; handheld tools reduce some of the operational burden, but still rely on manual movement and alignment; laser welding offers high connection strength and requires no consumables, but the equipment is fixed and has high environmental requirements; fixed vision robots are only suitable for orderly production lines in precast component plants and cannot be used for mobile operations.

[0003] Existing technologies generally suffer from the following shortcomings: First, they have poor adaptability to different scenarios, lacking mobile automated equipment that can adapt to both the orderly environment of a factory and the muddy and uneven terrain of a construction site; second, they use a single binding method, unable to flexibly choose binding or welding based on the diameter of the rebar, stress requirements, and the working environment (indoor / outdoor, humid / dry); third, they rely on manual visual identification of nodes, which can easily lead to missed or incorrect binding in situations with dense rebar, dim lighting, or reflective conditions; and fourth, mobile platforms lack robustness against complex terrain, varying lighting, and vibration, making it difficult to achieve continuous operation around the clock.

[0004] Especially in multi-scenario operations, visual recognition results lack the ability to express uncertainty. Different lighting conditions (such as strong outdoor light, alternating shadows, low nighttime illumination, indoor reflections, or local overexposure) can lead to problems such as underexposure, overexposure, increased noise, or local feature obscuring in the acquired images, resulting in some rebar intersections not being effectively identified. Existing systems typically output the exact node positions directly, lacking a mechanism for quantitatively evaluating and conveying the recognition confidence level. Low-confidence nodes may still be directly included in the work path planning, causing the robotic arm to perform binding / welding actions based on incorrect or missing nodes, leading to misbinding, missed binding, or collisions between the robotic arm and the rebar cage, seriously affecting work safety and quality.

[0005] In summary, existing solutions have not yet achieved the organic integration of multi-scenario adaptive mobility, dual-mode connection execution, intelligent visual recognition and path planning, and also lack a robust decision-making mechanism for visual uncertainty. Summary of the Invention

[0006] To address the problems of poor scene adaptability, low operation efficiency, and insufficient intelligence in existing technologies, this invention provides a multi-scene adaptive visual intelligent operation method and system for rebar connection robots. This system enables adaptive image acquisition in complex construction environments, accurate identification and positioning of rebar nodes, and intelligent path planning that conforms to structural logic, significantly improving the automation level and operation quality of rebar connection operations.

[0007] This invention is achieved through the following technical solution:

[0008] The first invention provides a multi-scenario adaptive visual intelligent operation method for rebar connection robots, comprising the following steps:

[0009] Step 1: Adapt the rebar connection robot to the work scenario and move the adapted rebar connection robot to the work area.

[0010] Step 2: Perform adaptive image acquisition on the work area, input the acquired images into a 3D node detection network for rebar node identification and localization, and output a list of rebar intersection nodes; each node in the rebar intersection node list contains a rebar intersection node and its semantic label;

[0011] Step 3: Construct a rebar topology diagram based on semantic tags for rebar intersection nodes. According to the determined rebar connection method, traverse the rebar topology diagram to generate a work path that conforms to the rebar structure operation logic.

[0012] Step 4: The rebar connection robot connects each rebar node according to the work path, and after completing the connection of a single rebar node, it performs quality verification on the rebar node.

[0013] In the above method, the adaptive adaptation mechanism ensures that the robot can reach the work area stably and reliably, which solves the problem that existing fixed or track-based equipment has limited mobility and cannot adapt to complex terrain.

[0014] By detecting ambient light intensity and color temperature, the system automatically adjusts the brightness and color temperature of the adaptive lighting module and simultaneously adjusts the exposure parameters and gain of the industrial camera to suppress high-light reflection and shadow interference, ensuring clear and high-contrast images. Subsequently, the acquired images are input into a 3D node detection network. This network can infer 3D spatial information from 2D images, accurately regress the spatial bounding box center coordinates of each rebar intersection node, and output corresponding semantic labels. This step enables intelligent identification and localization of dense rebar nodes, effectively reducing the false negative and false positive rates.

[0015] When constructing the rebar topology diagram, each rebar intersection node is treated as a graph node, and edges between nodes are established based on the actual physical connections of the rebar mesh. This graph structure not only contains the geometric location information of the nodes but also embodies the topological logic of the rebar mesh. When generating the work path, the topology diagram is traversed according to the determined connection method.

[0016] Following a pre-planned work path, the robot controls its robotic arm to move sequentially to each rebar node and invokes the corresponding actuators to complete the connection work. Simultaneously, after completing the connection of each individual node, a quality check is immediately performed on that node. This closed-loop feedback mechanism of "work-check" ensures the work quality of each node, avoids batch errors, and significantly improves the overall construction quality of the project.

[0017] In one optional embodiment, the 3D node detection network includes:

[0018] The RGB feature extraction branch is used to extract RGB features from the acquired image and generate an RGB feature map; wherein, the RGB features include: steel bar surface texture features, edge features, and reflective area features;

[0019] The geometric feature extraction branch is used to extract geometric features from the acquired image and generate a depth map; wherein, the geometric features include rebar contour features, intersection angle features, and local spatial structure features;

[0020] The pseudo-point cloud generation module is used to back-project two-dimensional pixels into three-dimensional points based on the depth estimation results of the camera intrinsic parameters and depth map;

[0021] The voxel token encoding module is used to divide 3D points into a voxel grid and combine the voxel center position encoding in the voxel grid with the point feature aggregation result to generate voxel tokens.

[0022] A sparse Transformer encoder for context enhancement of voxel tokens around rebar intersection nodes;

[0023] The 3D inspection head is used to output the 3D coordinates, semantic labels, confidence level, and coordinate uncertainty of the rebar intersection.

[0024] In one optional embodiment, the acquired images are input into a 3D node detection network for rebar node identification and localization, including the following steps:

[0025] Step 21: Input the acquired image into the RGB feature extraction branch and the geometric feature extraction branch for feature extraction, and generate an RGB feature map containing RGB features and a depth map containing geometric features;

[0026] Step 22: Perform discrete depth prediction on the depth map pixel by pixel, and use the predicted discrete depth to scatter each pixel to discrete points to obtain the three-dimensional coordinates of the pseudo point cloud.

[0027] Step 23: Extract the corresponding RGB features from the RGB feature map according to the three-dimensional coordinates, and concatenate them with the geometric features to generate the original feature vector of the pseudo point cloud;

[0028] Step 24: Divide all pseudo point clouds into uniform voxels, encode the center coordinates of each voxel to generate a position encoding vector, and add the position encoding vector to the original feature vector to generate a voxel token.

[0029] Step 25: Input the voxel token into the sparse Transformer encoder for context enhancement to obtain the enhanced voxel feature sequence;

[0030] Step 26: Use a 3D detection head to detect the enhanced voxel feature sequence, and output the 3D coordinates, semantic labels, confidence level, and coordinate uncertainty of the rebar intersection node based on the detection results.

[0031] In one optional embodiment, a three-dimensional detection head is used to detect the enhanced voxel feature sequence, including the following steps:

[0032] Step 261: Process the enhanced voxel feature sequence through four parallel sub-branches, and output the center regression parameters, confidence parameters, semantic classification parameters and uncertainty parameters.

[0033] Step 262: Calculate the center coordinates of the rebar nodes based on the central regression parameters, and then filter candidate nodes based on the center coordinates of the rebar nodes;

[0034] Step 263: For each candidate node, determine the category corresponding to the highest probability based on its semantic classification parameters, and calculate the confidence level of the corresponding category to generate a candidate node set; the candidate node set includes: the center coordinates of the rebar node, the confidence level parameter, the confidence level of the corresponding category, and the uncertainty parameter;

[0035] Step 264: Perform three-dimensional nonmaximum suppression on the candidate node set to generate the final node set, and use the uncertain parameters in the final node set as the standard deviation of the node coordinates in each direction. If the uncertain parameters in the final node set are greater than the uncertain threshold, then trigger uncertainty gating.

[0036] In an optional embodiment, the detection of the enhanced voxel feature sequence using a three-dimensional detection head further includes:

[0037] Each candidate node is given a confidence flag. If the confidence parameter of the candidate node is lower than the confidence threshold, the confidence flag is set to low confidence, and the candidate node is set as a low confidence candidate node.

[0038] Traverse the final node set and add all low-confidence candidate nodes to the re-examination queue;

[0039] The queue of nodes to be re-inspected is traversed, the uncertainty direction of the current view is calculated for each node to be re-inspected, the optimal new view is re-planned based on the uncertainty direction, and the rebar connection robot is controlled to move the camera to the optimal new view and acquire images.

[0040] In one alternative embodiment, constructing a rebar topology diagram based on semantic tags for rebar intersections includes:

[0041] Treat each rebar intersection node as a graph node. Based on the rebar mesh topology prior, if two graph nodes are located on the same rebar and are adjacent, then an undirected edge is established.

[0042] Each graph node is assigned 3D location features, semantic labels, connection methods, confidence levels, and uncertainties as node features; and each undirected edge is assigned Euclidean distance, direction, and reinforcement properties as edge features.

[0043] In this embodiment, the rebar attribute is used to indicate whether they belong to the same rebar.

[0044] In one optional embodiment, the topology diagram of the reinforcing bars is traversed according to the determined connection method, including the following steps:

[0045] Step 31: Model the traversal of the steel reinforcement topology graph as a sequence representation learning problem with graph structure constraints; where the set of all unvisited nodes is used as the state space, and the selection of the next visited node is used as the action space.

[0046] Step 32: Select unvisited nodes from the state space, concatenate the rebar connection method with the node features and edge features of the unvisited nodes to form input features, input the input features into the graph neural network based on the attention mechanism, and output the currently visited nodes and the next target visited node;

[0047] Step 33: Use the improved A* algorithm to smoothly filter the collision detection of the currently visited nodes and output the trajectory point sequence;

[0048] Step 34: Update the state space based on the trajectory point sequence, and repeat steps 31 to 34 until there are no unvisited nodes in the state space.

[0049] In one alternative embodiment, the input features are fed into an attention-based graph neural network, comprising the following steps:

[0050] Step 331: Input the input features into the graph encoder, which includes 3 layers of GATv2. In each layer of GATv2, attention is calculated on the input features, and the node features are updated according to the calculation results.

[0051] Step 332: Use an attention-based sequential generation network to perform attention calculation on the updated node features and their context, and select the next node based on the calculation results;

[0052] Step 333: Calculate the probability distribution of the selected next node using the softmax function, and select the point with the highest probability as the next target node to visit.

[0053] In one alternative embodiment, an improved A* algorithm is used to smoothly filter the collision detection of currently visited nodes, including the following steps:

[0054] Step 341: Using the robotic arm base as a coordinate system, the robotic arm workspace is discretized into a voxel grid to form a three-dimensional semantic map. Based on the semantic information, a cost coefficient is assigned to the three-dimensional semantic map to form a three-dimensional semantic cost map.

[0055] Step 342: Take the current position of the robotic arm end effector as the starting point, take the next target access node as the target point, traverse the graph nodes between the starting point and the target point, and generate the initial path;

[0056] Step 343: Use B-spline smoothing and collision detection technology to backtrack the initial path, and use kinematic constraints to smooth and filter the backtracked path to generate a joint space trajectory; wherein, the kinematic constraints include joint accessibility and judgment and direction constraints.

[0057] A second aspect of the present invention provides a multi-scenario adaptive rebar connection robot vision intelligent operation system, comprising:

[0058] The scene adaptation and autonomous navigation module is used to adapt the rebar connection robot to the work scene and move the adapted rebar connection robot to the work area.

[0059] The adaptive image acquisition and rebar node recognition and localization module is used to adaptively acquire images of the work area, input the acquired images into a 3D node detection network for rebar node recognition and localization, and output the rebar intersection nodes and their semantic labels.

[0060] The connection method selection and semantic path planning module is used to construct a rebar topology diagram based on semantic tags for rebar intersection nodes, and to traverse the rebar topology diagram according to the determined rebar connection method to generate a work path that conforms to the rebar structure operation logic.

[0061] The connection and quality verification module is used by the rebar connection robot to connect rebar nodes one by one according to the work path, and to perform quality verification on the rebar node after completing the connection of a single rebar node.

[0062] This invention solves the problem of unreliable recognition caused by complex lighting and dense occlusion in multiple scenarios by using a low-confidence node multi-view verification mechanism, rebar topology diagram and graph neural network path traversal and kinematic constraint trajectory optimization, and realizes high-precision, collision-free and high-efficiency automated operation of rebar connection. Attached Figure Description

[0063] To more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of the present invention and should not be considered as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort. In the drawings:

[0064] Figure 1 This is a flowchart illustrating the multi-scenario adaptive rebar connection robot visual intelligent operation method provided in an embodiment of the present invention. Detailed Implementation

[0065] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the embodiments and accompanying drawings. The illustrative embodiments and descriptions of the present invention are only used to explain the present invention and are not intended to limit the present invention.

[0066] Example 1

[0067] like Figure 1 As shown, the multi-scenario adaptive visual intelligent operation method for rebar connection robots includes the following steps:

[0068] Step 1: Adapt the rebar connection robot to the work scenario and move the adapted rebar connection robot to the work area.

[0069] Step 2: Perform adaptive image acquisition on the work area, input the acquired images into a 3D node detection network for rebar node identification and localization, and output a list of rebar intersection nodes; each node in the rebar intersection node list contains a rebar intersection node and its semantic label;

[0070] Step 3: Construct a rebar topology diagram based on semantic tags for rebar intersection nodes. According to the determined rebar connection method, traverse the rebar topology diagram to generate a work path that conforms to the rebar structure operation logic.

[0071] Step 4: The rebar connection robot connects each rebar node according to the work path, and after completing the connection of a single rebar node, it performs quality verification on the rebar node.

[0072] It should be noted that in actual construction scenarios, ground conditions are complex and varied. The adaptive adaptation mechanism ensures that the robot can stably and reliably reach the work area, solving the problem that existing fixed or track-based equipment has limited mobility and cannot adapt to complex terrain.

[0073] Due to the complex lighting conditions at construction sites, including direct sunlight, shadows, darkness at night, and reflections from rebar surfaces, traditional fixed-parameter image acquisition methods struggle to obtain high-quality image data. This method detects ambient light intensity and color temperature, automatically adjusting the brightness and color temperature of the adaptive lighting module while simultaneously adjusting the exposure parameters and gain of the industrial camera. This suppresses high-light reflections and shadow interference, ensuring clear and high-contrast images. The acquired images are then input into a 3D node detection network. This network infers 3D spatial information from 2D images, accurately regresses the spatial bounding box center coordinates of each rebar intersection node, and outputs corresponding semantic labels. This step enables intelligent identification and localization of dense rebar nodes, effectively reducing both false negative and false positive rates.

[0074] When constructing the rebar topology diagram, each rebar intersection node is treated as a graph node, and edges between nodes are established based on the actual physical connections of the rebar mesh. This graph structure not only contains the geometric position information of the nodes but also embodies the topological logic of the rebar mesh. When generating the work path, the topology diagram is traversed according to the determined connection method (such as binding, welding, etc.). Unlike traditional path planning based solely on geometric distance, the work path generated by this method fully considers the operational logic of the rebar structure. For example, the robotic arm tends to move sequentially along the same rebar, avoiding disorderly jumps within the rebar mesh, thereby reducing large-amplitude reciprocating movements of the robotic arm and improving work efficiency and safety.

[0075] Following a pre-planned work path, the robot controls its robotic arm to move sequentially to each rebar node and invokes the corresponding actuators (such as binding guns or welding guns) to complete the connection operation. Simultaneously, after completing the connection of each individual node, a quality check is immediately performed. For example, an image of the node is re-captured using a vision system to check if the binding wires are tightened or the weld is full. If the check fails, a redo or alarm is triggered immediately. This closed-loop feedback mechanism of "operation-check" ensures the quality of work at each node, avoids batch errors, and significantly improves the overall construction quality of the project.

[0076] In one optional embodiment, the 3D node detection network includes:

[0077] The RGB feature extraction branch is used to extract RGB features from the acquired image and generate an RGB feature map; wherein, the RGB features include: steel bar surface texture features, edge features, and reflective area features;

[0078] The geometric feature extraction branch is used to extract geometric features from the acquired image and generate a depth map; wherein, the geometric features include rebar contour features, intersection angle features, and local spatial structure features;

[0079] The pseudo-point cloud generation module is used to back-project two-dimensional pixels into three-dimensional points based on the depth estimation results of the camera intrinsic parameters and depth map;

[0080] The voxel token encoding module is used to divide 3D points into a voxel grid and combine the voxel center position encoding in the voxel grid with the point feature aggregation result to generate voxel tokens.

[0081] A sparse Transformer encoder for context enhancement of voxel tokens around rebar intersection nodes;

[0082] The 3D inspection head is used to output the 3D coordinates, semantic labels, confidence level, and coordinate uncertainty of the rebar intersection.

[0083] It should be noted that the 3D node detection network includes an RGB feature extraction branch, a geometric feature extraction branch, a pseudo-point cloud generation module, a voxel token encoding module, a sparse Transformer encoder, and a 3D detection head. The RGB feature extraction branch extracts RGB features from the acquired image and generates an RGB feature map; these RGB features include: rebar surface texture features, edge features, and reflective area features. The geometric feature extraction branch extracts geometric features from the acquired image and generates a depth map; these geometric features include rebar contour features, intersection angle features, and local spatial structure features. The pseudo-point cloud generation module backprojects 2D pixels into 3D points based on camera intrinsics and depth estimation results from the depth map. The voxel token encoding module divides the 3D points into a voxel mesh and combines the voxel center position encoding in the voxel mesh with the point feature aggregation results to generate voxel tokens. The sparse Transformer encoder performs context enhancement on the voxel tokens around the rebar intersection nodes.

[0084] In one optional embodiment, the acquired images are input into a 3D node detection network for rebar node identification and localization, including the following steps:

[0085] Step 21: Input the acquired image into the RGB feature extraction branch and the geometric feature extraction branch for feature extraction, and generate an RGB feature map containing RGB features and a depth map containing geometric features;

[0086] Step 22: Perform discrete depth prediction on the depth map pixel by pixel, and use the predicted discrete depth to scatter each pixel to discrete points to obtain the three-dimensional coordinates of the pseudo point cloud.

[0087] Step 23: Extract the corresponding RGB features from the RGB feature map according to the three-dimensional coordinates, and concatenate them with the geometric features to generate the original feature vector of the pseudo point cloud;

[0088] Step 24: Divide all pseudo point clouds into uniform voxels, encode the center coordinates of each voxel to generate a position encoding vector, and add the position encoding vector to the original feature vector to generate a voxel token.

[0089] Step 25: Input the voxel token into the sparse Transformer encoder for context enhancement to obtain the enhanced voxel feature sequence;

[0090] Step 26: Use a 3D detection head to detect the enhanced voxel feature sequence, and output the rebar intersection nodes and their semantic labels based on the detection results.

[0091] It should be noted that the dual-branch feature extraction module comprises two parallel backbone networks. One branch uses a pre-trained convolutional neural network to extract RGB feature maps, which are rich in texture, color, and edge information of the rebar surface, helping to distinguish the rebar from the background. The other branch uses a specially designed geometric feature extraction network to extract geometric features from the image, focusing on expressing the outline, shape, and spatial layout of the rebar. Through parallel extraction by the two branches, the network can simultaneously capture appearance features and geometric structural features, providing a rich information foundation for subsequent 3D localization.

[0092] The feature attachment module receives the depth map and performs discrete depth prediction for each pixel, that is, predicts the probability distribution of the pixel within a preset depth range. Then, using the predicted depth values, the two-dimensional pixel coordinates are back-projected into three-dimensional space, "scattering" each pixel onto its corresponding three-dimensional discrete point, thereby constructing a pseudo-point cloud of the scene. This process maps two-dimensional image features to three-dimensional space, overcoming the limitation of two-dimensional detection not being able to directly obtain spatial depth.

[0093] While constructing the pseudo-point cloud, the module indexes back the two-dimensional RGB feature map based on the three-dimensional coordinates, extracts the RGB feature vectors at the corresponding positions, and concatenates them with the geometric feature vectors extracted in step 21. This feature fusion mechanism enables each point in the pseudo-point cloud to not only have spatial coordinates but also rich semantic features, achieving a dual expression of "geometric + semantic".

[0094] To adapt to network architectures like Transformer that process sequential data, the voxel encoding module divides the unstructured pseudo-point cloud into a regular 3D voxel mesh. For each non-empty voxel, its center coordinates are calculated, and the center coordinates are encoded using sine / cosine functions or a learnable position encoding matrix to generate a position encoding vector. Subsequently, the position encoding vector is aggregated (e.g., by averaging or maximizing) with the original feature vectors of all points within the voxel and added to generate voxel tokens containing spatial location information. This step transforms the discrete point cloud data into an ordered sequence of tokens, laying the foundation for subsequent global context modeling.

[0095] The Sparse Transformer encoder utilizes a self-attention mechanism to process voxel token sequences. Compared to traditional convolutional neural networks, the Transformer has a global receptive field, enabling it to capture long-range dependencies. In reinforced concrete mesh scenarios, nodes often exhibit strong topological relationships (such as parallel or perpendicular relationships). The Sparse Transformer, through contextual enhancement, leverages the contextual information of surrounding nodes to eliminate the uncertainty in identifying individual nodes. For example, when a node's features are blurred due to occlusion, the encoder can infer the node's existence and attributes based on the spatial distribution characteristics of its neighboring nodes, thereby significantly improving detection robustness.

[0096] In one optional embodiment, a three-dimensional detection head is used to detect the enhanced voxel feature sequence, including the following steps:

[0097] Step 261: Process the enhanced voxel feature sequence through four parallel sub-branches, and output the center regression parameters, confidence parameters, semantic classification parameters and uncertainty parameters.

[0098] Step 262: Calculate the center coordinates of the rebar nodes based on the central regression parameters, and then filter candidate nodes based on the center coordinates of the rebar nodes;

[0099] Step 263: For each candidate node, determine the category corresponding to the highest probability based on its semantic classification parameters, and calculate the confidence level of the corresponding category to generate a candidate node set; the candidate node set includes: the center coordinates of the rebar node, the confidence level parameter, the confidence level of the corresponding category, and the uncertainty parameter;

[0100] Step 264: Perform three-dimensional nonmaximum suppression on the candidate node set to generate the final node set, and use the uncertain parameters in the final node set as the standard deviation of the node coordinates in each direction. If the uncertain parameters in the final node set are greater than the uncertain threshold, then trigger uncertainty gating.

[0101] In this embodiment, the trigger uncertainty gate is the robotic arm actively changing its perspective to perform a secondary observation of the node, and updating the coordinates after multi-frame fusion.

[0102] It should be noted that the 3D detection head is designed with a multi-task learning architecture, which includes four parallel fully connected layers or small convolutional network branches.

[0103] The center regression parameters are used to predict the offset of the node center relative to the voxel center;

[0104] The confidence parameter is used to predict the probability that the voxel contains a rebar node;

[0105] Semantic classification parameters are used to predict the probability that a node belongs to a specific category such as a main reinforcement node or a distributed reinforcement node;

[0106] Uncertainty parameters are used to assess the reliability of prediction results and are usually output in the form of variance or standard deviation.

[0107] Three-dimensional non-maximum suppression (NMS) is used to remove overlapping candidate boxes and retain the best detection results. In this process, the uncertainty parameter plays a crucial role in screening and quality control. The uncertainty parameter is considered as the standard deviation of node coordinates in each direction. If this standard deviation exceeds a preset uncertainty threshold, it indicates that the network lacks confidence in the localization of that node. At this point, an uncertainty gating mechanism will be triggered.

[0108] As a specific implementation method, triggering uncertainty gating can involve sending a signal to the control system, instructing the robotic arm to pause its current operation, actively changing its perspective to conduct a secondary observation of the node, and correcting the coordinates through multi-frame data fusion, thereby ensuring operational accuracy. This uncertainty-based closed-loop feedback design effectively avoids the risk of connection failures or collisions caused by visual errors.

[0109] In an optional embodiment, step 264 further includes: for each node subjected to three-dimensional nonmaximum suppression, collecting the category predictions of all original candidates within the radius surrounding the node, and using weighted voting to determine the final semantic category of the node.

[0110] It should be noted that, to further improve the accuracy of semantic classification, this embodiment introduces a weighted voting mechanism. After NMS processing, instead of directly using a single prediction result, all original candidate predictions within a certain radius around the node are collected. Weighted voting is performed based on the confidence level of each candidate or its distance from the central node, and a weighted score for each category is calculated. The category with the highest score is determined as the final semantic category of the node. This mechanism effectively mitigates classification errors caused by local occlusion or noise, and enhances the stability of the algorithm under complex conditions.

[0111] In one alternative embodiment, constructing a rebar topology diagram based on semantic tags for rebar intersections includes:

[0112] Treat each rebar intersection node as a graph node. Based on the rebar mesh topology prior, if two graph nodes are located on the same rebar and are adjacent, then an undirected edge is established.

[0113] Each graph node is assigned 3D location features, semantic labels, connection methods, confidence levels, and uncertainties as node features; and each undirected edge is assigned Euclidean distance, direction, and reinforcement properties as edge features.

[0114] In this embodiment, the rebar attribute is used to indicate whether they belong to the same rebar.

[0115] Each rebar intersection node is treated as a graph node. Based on this, and using the rebar mesh topology prior, if two graph nodes are located on the same rebar and are adjacent, an undirected edge is created. It should be understood that "adjacent" here means that there are no other intersection nodes in the direction of the rebar extension. This edge construction method accurately reproduces the physical connection structure of the rebar mesh, providing a topological foundation for subsequent path planning.

[0116] Each graph node is assigned 3D location features, semantic labels, connection methods, confidence levels, and uncertainties as node features. The 3D location features describe the node's spatial coordinates; the semantic labels distinguish between nodes with main reinforcing bars and nodes with distributed reinforcing bars; the connection methods indicate whether the node should be tied or welded; and the confidence and uncertainty parameters directly inherit the output of the 3D node detection network in step 2, characterizing the reliability of the node's location. Simultaneously, each undirected edge is assigned Euclidean distance, direction, and reinforcing bar attributes as edge features. The Euclidean distance reflects the physical span between nodes, the direction feature describes the direction of the reinforcing bars (horizontal or vertical), and the reinforcing bar attributes identify whether the edge belongs to the same reinforcing bar. This multi-dimensional feature assignment ensures that the topology graph not only contains geometric information but also rich semantic and structural information.

[0117] In one optional embodiment, the topology diagram of the reinforcing bars is traversed according to the determined connection method, including the following steps:

[0118] Step 31: Model the traversal of the steel reinforcement topology graph as a sequence representation learning problem with graph structure constraints; where the set of all unvisited nodes is used as the state space, and the selection of the next visited node is used as the action space.

[0119] Step 32: Select unvisited nodes from the state space, concatenate the rebar connection method with the node features and edge features of the unvisited nodes to form input features, input the input features into the graph neural network based on the attention mechanism, and output the currently visited nodes and the next target visited node;

[0120] Step 33: Use the improved A* algorithm to smoothly filter the collision detection of the currently visited nodes and output the trajectory point sequence;

[0121] Step 34: Update the state space based on the trajectory point sequence, and repeat steps 31 to 34 until there are no unvisited nodes in the state space.

[0122] It should be noted that modeling the traversal of the steel reinforcement topology graph as a sequence representation learning problem with graph structure constraints transforms the complex graph traversal problem into a sequence generation problem similar to that in natural language processing, enabling the network to learn the implicit operational logic.

[0123] By introducing connection patterns as input features, the network can dynamically adjust its traversal strategy based on different operational requirements (e.g., tying typically requires continuous work along the direction of the rebar, while welding may allow for more jumps). The improved A* algorithm transforms the discrete node sequence output by the graph neural network into a continuous spatial trajectory. Steps 31 to 34 are repeated until no unvisited nodes are found in the state space, thus completing the traversal planning for the entire rebar mesh.

[0124] Furthermore, the original feature vector of each node vi is specifically constructed as: f i (0)=[x i ,y i ,z i Position-coded components, onehot(l) i ), onehot(c i ),d i ,conf i ], where x i y i z i The value is the 3D coordinate of the node after normalization to the [0,1] interval; the position encoding component performs multi-scale sine and cosine encoding on x, y, and z respectively, with a frequency of 2k (k=0,...,5) to help the network capture the spatial relative relationships between nodes; onehot(l i () is the one-hot encoding of semantic tags, distinguishing between main reinforcement nodes and distributed reinforcement nodes; onehot(c i ) represents the unique thermal code for the connection method, indicating whether the node will be bound or welded; d i The diameter of the reinforcing bar at this node; conf iThe confidence score for node detection is calculated. The original feature vectors are linearly mapped to a latent dimension D=128 and then used as input to the graph encoder.

[0125] In one alternative embodiment, the input features are fed into an attention-based graph neural network, comprising the following steps:

[0126] Step 331: Input the input features into the graph encoder, which includes 3 layers of GATv2. In each layer of GATv2, attention is calculated on the input features, and the node features are updated according to the calculation results.

[0127] Step 332: Use an attention-based sequential generation network to perform attention calculation on the updated node features and their context, and select the next node based on the calculation results;

[0128] Step 333: Calculate the probability distribution of the selected next node using the softmax function, and select the point with the highest probability as the next target node to visit.

[0129] It's worth noting that GATv2 adaptively aggregates information from neighboring nodes by calculating attention coefficients between nodes. For example, for a given node to be visited, the network might pay more attention to neighboring nodes belonging to the same rebar, while reducing its focus on nodes that are close but not directly physically connected. This attention mechanism allows the network to capture the inherent structural dependencies within the rebar mesh, thereby predicting the next-hop node that conforms to construction logic.

[0130] Taking the first layer GATv2 as an example, it consists of four stages. The first stage is a linear transformation, which is as follows:

[0131] ;

[0132] In the above formula, This is the intermediate vector after the linear transformation. For learnable weight matrix, This represents the hidden state of node i in the previous layer.

[0133] The second stage is the calculation of the attention coefficient, which for each edge (i,j)∈E is:

[0134] ;

[0135] In the above formula, is a learnable attention weight vector, and LeakyReLU is an activation function with a negative slope.

[0136] The third stage is attention coefficient normalization, which includes:

[0137] ;

[0138] In the above formula, Let i be the set of neighbors of node i, such that the sum of the attention coefficients of all neighbors of each node is 1.

[0139] The fourth stage is node feature updating:

[0140] ;

[0141] In the above formula, ELU is the exponential linear unit activation function, which updates node features after weighted aggregation of neighbor information.

[0142] After three layers of GATv2 processing, the hidden state of each node is finally output. It should be understood that... It also includes edge features, thereby incorporating edge information into the node feature update process.

[0143] Furthermore, the attention-based sequential generation network employs a Pointer Network-style decoding mechanism. At time t, the context vector c t Update via GRU: c t =GRU(hv t-1 , c t-1 ), where v t-1 Let v0 be the index of the last visited node. Initially, v0 represents the virtual node corresponding to the current position of the robotic arm or the nearest node. For each unvisited node j∈U, calculate the compatibility score:

[0144] ;

[0145] In the above formula, , , For a learnable parameter matrix, Let be the hidden state of node j output by the graph encoder. This is the context vector.

[0146] Then, the probability distribution is obtained using the softmax function:

[0147] ;

[0148] This yields the probability distribution of the next node, where U represents the node with the highest probability selected from the set of unvisited nodes as the next target node to be visited.

[0149] Furthermore, regarding the training process of the graph neural network, a hybrid training strategy of supervised learning and reinforcement learning is adopted. In the supervised pre-training stage, a large number of steel mesh scenes (2~50 nodes) are randomly generated through a simulation environment (such as Isaac Sim). For small-scale scenes (≤15 nodes), an integer programming solver is used to obtain the precise optimal traversal order that takes into account the movement distance and operation difficulty as pseudo-labels. For large-scale scenes, an LKH solver is used to obtain an approximate optimal solution.

[0150] Regarding the training process of the graph neural network, a hybrid training strategy of supervised learning and reinforcement learning is adopted. In the supervised pre-training stage, a large number of steel mesh scenes (2~50 nodes) are randomly generated through a simulation environment (such as Isaac Sim). For small-scale scenes (≤15 nodes), an integer programming solver is used to obtain the exact optimal traversal order that takes into account the movement distance and operation difficulty as pseudo-labels. For large-scale scenes, an LKH solver is used to obtain an approximate optimal solution.

[0151] During the reinforcement learning fine-tuning phase, the REINFORCE policy gradient algorithm is employed, using the negative value of the total task time as the reward function to further optimize the traversal strategy in simulation or real-world environments. For data augmentation, the simulation scene is randomly rotated and translated to simulate different installation positions, and some nodes are randomly deleted to simulate occlusion and missed detection scenarios, enhancing the network's generalization ability. In the inference phase, a single-step greedy prediction mode is used, keeping the entire network's forward inference time within 50ms.

[0152] In one alternative embodiment, an improved A* algorithm is used to smoothly filter the collision detection of currently visited nodes, including the following steps:

[0153] Step 341: Using the robotic arm base as a coordinate system, the robotic arm workspace is discretized into a voxel grid to form a three-dimensional semantic map. Based on the semantic information, a cost coefficient is assigned to the three-dimensional semantic map to form a three-dimensional semantic cost map.

[0154] Step 342: Take the current position of the robotic arm end effector as the starting point, take the next target access node as the target point, traverse the graph nodes between the starting point and the target point, and generate the initial path;

[0155] Step 343: Use B-spline smoothing and collision detection technology to backtrack the initial path, and use kinematic constraints to smooth and filter the backtracked path to generate a joint space trajectory; wherein, the kinematic constraints include joint accessibility and judgment and direction constraints.

[0156] It should be noted that, to ensure the planned path is feasible and smooth in the actual physical space, this embodiment uses an improved A* algorithm to smoothly filter the collision detection of currently visited nodes. Areas with reinforced concrete and obstacles are assigned extremely high cost coefficients (considered as hard obstacles), open areas are assigned low cost coefficients, and areas approaching the operational limits are assigned medium cost coefficients. The introduction of this semantic cost map allows the path search process to proactively avoid dangerous areas, improving operational safety.

[0157] Using the current position of the robotic arm's end effector as the starting point and the next target node as the target point, an initial path is generated by traversing the graph nodes between the starting and target points. During this process, a dynamic heuristic weight adjustment strategy is employed: larger weights are used when the target point is far away to accelerate the search; smaller weights are used when the target point is approached to improve path accuracy. This dynamic adjustment strategy effectively balances search efficiency and path quality.

[0158] Kinematic constraints include joint reachability and judgment, as well as orientation constraints. Since the A* algorithm typically generates polyline paths, directly using them for robotic arm control can lead to stiff movements or even exceed joint limitations. B-spline smoothing techniques can fit polylines into smooth curves, while the introduction of kinematic constraints ensures that every point on the path is within the robotic arm's reachable workspace, and that the end effector's posture meets operational requirements (e.g., perpendicular to the rebar plane). The resulting joint space trajectory can directly drive the robotic arm to move smoothly and accurately to the target node, realizing the transformation from logical path to physical trajectory.

[0159] Specifically, the robotic arm's workspace is discretized into a voxel mesh. The semantic cost map assigns a cost coefficient to each voxel, with the cost coefficient for areas with reinforced concrete and obstacles set to ∞ (considered hard obstacles, impassable during path search), the cost coefficient for open, accessible areas set to 1.0 (basic cost), and the cost coefficient for areas nearing the robotic arm's operational limits set to 5.0 to 10.0 (medium cost, path is achievable but requires caution). This differentiated cost coefficient setting allows path search to proactively avoid high-risk areas, prioritizing safe and efficient routes.

[0160] The initial heuristic weights are set to w. start =2.0, the termination weight is set to wend=1.0, and the distance switching threshold is set to d. switch =0.05m. During the search process, when the Euclidean distance from the current node n to the target point G is greater than d... switch At that time, keep the weight w=w start =2.0 to accelerate search convergence; when the distance is less than d switch And the current w is greater than w end When the weights are switched to w=w end =1.0 to improve path accuracy.

[0161] The core calculation process of path search is as follows: Starting from the starting point S, set g(S)=0, and calculate the initial evaluation value f(S)=w·h(S,G), where the heuristic function h(S,G) is the Euclidean distance from the starting point to the target.

[0162] For each extended node n, its 26-neighborhood with neighbor m is first filtered for validity—if m is already in the closed_set, the cost coefficient C(m) = ∞, or it exceeds the workspace boundary, it is skipped. At the same time, kinematic reachability is pre-judged; if the displacement of the robotic arm end effector from its current position to m exceeds the maximum extension distance L of the robotic arm, it is skipped. max Then skip it.

[0163] Then the A* algorithm performs candidate cumulative cost calculation, where the formula is as follows:

[0164] ;

[0165] In the above formula, The actual cumulative cost from the starting point to n. Let n be the Euclidean distance from m. Let be the cost coefficient for voxel m. In this embodiment, the Euclidean distance from n to m is set as follows: δ for the direct neighbor, δ√2 for the face diagonal neighbor, and δ√3 for the volume diagonal neighbor.

[0166] Then, the heuristic value of the Euclidean distance is calculated using the heuristic function and the evaluation function:

[0167] ;

[0168] ;

[0169] In the above formula, Let m be the heuristic value of the Euclidean distance from node m to target G. To evaluate the function value, To accumulate costs, This represents the current heuristic weights.

[0170] like Smaller than the current If m is not in the point set, then update. = , And Add the node to the point set. When the distance to target G is less than 0.01m, the target is considered reached and the passage is completed. The pointer is traced back to obtain the original polyline path.

[0171] Smoothing and kinematic constraint refinement specifically include:

[0172] First, the point sequence of the original polyline path is used as control points to generate a cubic uniform B-spline curve C(t), t∈[0,1]. The curve is sampled at equal intervals, with the sampling interval not exceeding 0.5×δ (i.e. 0.75cm). Each sampling point is checked to see if it falls within the hard obstacle object with a cost coefficient C=∞. If there is a collision, a new control point is inserted near the collision location along the normal direction and refitted. This process is repeated until there is no collision or the maximum number of iterations is reached. If a collision still cannot be avoided, the process is to backtrack to the original A* polyline path.

[0173] In terms of kinematic constraint refinement, the curvature κ at various points on the smooth curve is calculated and compared with the maximum permissible curvature κ at the end of the robotic arm. max (Calculated from joint velocity and acceleration limits) is compared, and if the local curvature exceeds the limit, the curve segment is subjected to Savitzky-Golay low-pass filtering.

[0174] The smooth trajectory is then parameterized in time: based on the maximum velocity v at the end of the robotic arm. max =0.5m / s² and maximum acceleration a max =1.0m / s², and the arrival time of each path point is calculated using a trapezoidal velocity curve. Finally, the end-effector position sequence is converted into a joint angle sequence through inverse kinematics, and the joint spatial trajectory is output for execution by the robotic arm controller.

[0175] Existing rebar connection equipment typically employs greedy nearest neighbor or random traversal strategies for path planning, relying solely on geometric distance as the basis for selecting the next node and ignoring the inherent topological connection logic of the rebar mesh. This approach has several limitations: the robotic arm moves in a hopping manner within the rebar mesh, frequently exhibiting large reciprocating movements and resulting in a high proportion of idle travel; tying operations require continuous advancement along the same rebar, and welding operations also have specific constraints on the node visit order, which random traversal ignores, potentially leading to work sequences that violate construction specifications; traditional A* algorithms only consider geometric obstacle avoidance, failing to distinguish between accessible but cautious areas and absolutely inaccessible hard obstacle zones, easily resulting in paths that the robotic arm cannot actually execute in densely rebar-lined areas.

[0176] To address the aforementioned limitations, this embodiment achieves complete optimization from discrete nodes to structured operation paths through a three-level transformation involving topology graph construction, intelligent traversal of graph neural networks, and improved A* physical trajectory generation.

[0177] The first-level transformation converts the list of discrete rebar intersection nodes output by visual inspection into a topological graph G=(V,E) with physical connection semantics. Each node in the graph carries multi-dimensional features such as three-dimensional coordinates, semantic labels, connection methods, confidence and uncertainty. Each edge is established based on the topological prior of the rebar mesh and is given equilateral features such as Euclidean distance, direction and rebar attributes. This makes the topological graph not only encode geometric distance, but also restore the physical connection structure and semantic attributes of the rebar mesh.

[0178] In the second-level transformation, the graph neural network adaptively aggregates neighbor information through a 3-layer GATv2 attention mechanism. For a node to be visited, the network pays more attention to neighboring nodes that belong to the same steel bar as it and reduces the weight of nodes that are close but have no direct physical connection, so that the updated node features contain global topological dependencies. The sequential decoder maintains the context vector with GRU and combines it with the historical information of visited nodes. It generates a compatibility score for each unvisited node through attention calculation. The node with the highest probability is selected as the next target by the softmax output probability distribution, ensuring that the path has continuity and global optimality. The inference stage adopts a single-step greedy prediction mode, and the forward inference time is controlled within 50ms.

[0179] In the third-level transformation, the improved A* algorithm assigns differentiated cost coefficients through a three-dimensional semantic cost map, enabling path search to actively avoid dangerous areas. It balances search efficiency and path accuracy through dynamic heuristic weight adjustment, and fits the polyline path into a smooth curve and converts it into a joint space trajectory through kinematic constraint embedding and B-spline smoothing techniques.

[0180] The aforementioned three-level transformation does not operate independently, but rather forms a tightly coordinated closed-loop mechanism: the graph neural network determines the optimal access order of nodes at the topological graph level, outputting a node access sequence that conforms to the structural logic and connection constraints of the steel reinforcement structure; the improved A* algorithm transforms the movement between every two adjacent nodes into a collision-free, smooth joint space trajectory that satisfies kinematic constraints at the physical space level; after completing the trajectory planning for each node, the state space is updated, and the graph neural network is called again to predict the next node, until all nodes have been traversed. This two-layer architecture of logical decision-making and physical execution ensures that path planning neither violates structural logic nor is it practically feasible in physical space.

[0181] Example 2

[0182] This embodiment uses an outdoor bridge pier construction area with gravel, puddles, and mud as an example to illustrate the method of Embodiment 1. In this construction area, there is a steel mesh with more than 200 intersections, and the surface of the steel bars produces strong reflections and long shadows; some areas are blocked by the scaffolding above, resulting in dim lighting; now it is necessary to tie the intersections of the steel mesh.

[0183] Step 1: The chassis of the rebar connection robot integrates a switchable module. When sensors detect muddy ground and elevation differences, it automatically switches from wheeled to tracked mode to increase ground pressure; it also actively adjusts the chassis suspension height to stably move to the designated position below the pier.

[0184] Step 2: Adaptive image acquisition is performed using a camera mounted on the rebar connection robot. The camera automatically adjusts exposure and gain according to ambient light and enables HDR mode to effectively suppress overexposure and enhance shadow contrast. The 3D node detection network outputs the 3D coordinates, semantic label, confidence score, and coordinate uncertainty in each direction for each node. For 23 low-confidence nodes with a confidence score below 0.6 or an uncertainty greater than 3 mm, they are marked and added to the re-inspection queue. The robot arm is actively driven to move the camera to the next optimal viewpoint to re-acquire images. The coordinates and confidence scores are updated through multi-view Kalman filtering fusion, which improves the overexposed nodes with a confidence score of only 0.35 and an uncertainty of 8 mm to 0.89 and 1.5 mm, respectively, and the shadow nodes from 0.28 to 0.94. Finally, the confidence score of all nodes is higher than 0.85 and the uncertainty is less than 3 mm, which completely solves the problem of missed detection and false detection under complex lighting conditions.

[0185] In methods that do not employ adaptive image acquisition, overexposed areas on the rebar surface exhibit large white spots, and details in shadow areas are completely lost. Furthermore, existing detection networks only output node coordinates without providing confidence or uncertainty scores, resulting in the complete missed detection of 10 nodes in overexposed areas, the incorrect classification of 15 nodes in shadow areas as background, and coordinate deviations of up to 20 millimeters for 8 nodes in reflective areas. Moreover, due to the lack of a low-confidence judgment mechanism, all low-quality coordinates are directly fed into subsequent path planning.

[0186] Step 3: First, each rebar intersection node is treated as a graph node. Undirected edges are established based on the adjacent relationships on the same rebar. Each node is assigned a 3D position, semantic label, connection method, confidence level, and uncertainty. Each edge is assigned Euclidean distance, direction, and rebar attributes, thereby constructing a complete rebar topology graph. Then, the traversal problem is modeled as a sequence representation learning with graph structure constraints. A three-layer GATv2 graph encoder and attention-based sequential generation network are used to traverse the topology graph. The output is a binding sequence that spirals outward from the center of the pier, with the main bars first and then the stirrups. The total movement path is shorter than that of traditional methods, while fully conforming to the process logic and avoiding deformation of the rebar mesh.

[0187] Ordinary path planning selects the binding sequence based on the Euclidean principle of shortest distance, starting from the edge corner nodes and then jumping to the diagonal nodes, which causes the robotic arm to make multiple ineffective round trips. Moreover, it completely violates the construction process of "from the middle to the perimeter, main reinforcement first and then stirrups", causing local deformation of the steel mesh.

[0188] Step 4: When each node is tied, the end torque sensor immediately detects the pre-tightening force of the tie wire. If it is less than 15 N·m, it will automatically perform a supplementary tightening. If it is still not qualified, it will be marked and reported.

[0189] Example 3

[0190] This embodiment provides a multi-scenario adaptive rebar connection robot vision intelligent operation system, including:

[0191] The scene adaptation and autonomous navigation module is used to adapt the rebar connection robot to the work scene and move the adapted rebar connection robot to the work area.

[0192] The adaptive image acquisition and rebar node recognition and localization module is used to adaptively acquire images of the work area, input the acquired images into a 3D node detection network for rebar node recognition and localization, and output the rebar intersection nodes and their semantic labels.

[0193] The connection method selection and semantic path planning module is used to construct a rebar topology diagram based on semantic tags for rebar intersection nodes, and to traverse the rebar topology diagram according to the determined rebar connection method to generate a work path that conforms to the rebar structure operation logic.

[0194] The connection and quality verification module is used by the rebar connection robot to connect rebar nodes one by one according to the work path, and to perform quality verification on the rebar node after completing the connection of a single rebar node.

[0195] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A multi-scenario adaptive visual intelligent operation method for rebar connection robots, characterized in that, Includes the following steps: Step 1: Adapt the rebar connection robot to the work scenario and move the adapted rebar connection robot to the work area. Step 2: Adaptive image acquisition is performed on the work area. The acquired images are input into a 3D node detection network for rebar node identification and localization, and the rebar intersection nodes and their semantic labels are output. Step 3: Construct a rebar topology diagram based on semantic tags for rebar intersection nodes. According to the determined rebar connection method, traverse the rebar topology diagram to generate a work path that conforms to the rebar structure operation logic. Step 4: The rebar connection robot connects each rebar node according to the work path, and after completing the connection of a single rebar node, it performs quality verification on the rebar node.

2. The multi-scenario adaptive rebar connection robot vision intelligent operation method according to claim 1, characterized in that, The three-dimensional node detection network includes: The RGB feature extraction branch is used to extract RGB features from the acquired image and generate an RGB feature map; wherein, the RGB features include: steel bar surface texture features, edge features, and reflective area features; The geometric feature extraction branch is used to extract geometric features from the acquired image and generate a depth map; wherein, the geometric features include rebar contour features, intersection angle features, and local spatial structure features; The pseudo-point cloud generation module is used to back-project two-dimensional pixels into three-dimensional points based on the depth estimation results of the camera intrinsic parameters and depth map; The voxel token encoding module is used to divide 3D points into a voxel grid and combine the voxel center position encoding in the voxel grid with the point feature aggregation result to generate voxel tokens. A sparse Transformer encoder for context enhancement of voxel tokens around rebar intersection nodes; The 3D inspection head is used to output the 3D coordinates, semantic labels, confidence level, and coordinate uncertainty of the rebar intersection.

3. The multi-scenario adaptive rebar connection robot visual intelligent operation method according to claim 2, characterized in that, The acquired images are input into a 3D node detection network for rebar node identification and localization, including the following steps: Step 21: Input the acquired image into the RGB feature extraction branch and the geometric feature extraction branch for feature extraction, and generate an RGB feature map containing RGB features and a depth map containing geometric features; Step 22: Perform discrete depth prediction on the depth map pixel by pixel, and use the predicted discrete depth to scatter each pixel to discrete points to obtain the three-dimensional coordinates of the pseudo point cloud. Step 23: Extract the corresponding RGB features from the RGB feature map according to the three-dimensional coordinates, and concatenate them with the geometric features to generate the original feature vector of the pseudo point cloud; Step 24: Divide all pseudo point clouds into uniform voxels, encode the center coordinates of each voxel to generate a position encoding vector, and add the position encoding vector to the original feature vector to generate a voxel token. Step 25: Input the voxel token into the sparse Transformer encoder for context enhancement to obtain the enhanced voxel feature sequence; Step 26: Use a 3D detection head to detect the enhanced voxel feature sequence, and output the 3D coordinates, semantic labels, confidence level, and coordinate uncertainty of the rebar intersection node based on the detection results.

4. The multi-scenario adaptive rebar connection robot visual intelligent operation method according to claim 3, characterized in that, The enhanced voxel feature sequence is detected using a three-dimensional detection head, including the following steps: Step 261: Process the enhanced voxel feature sequence through four parallel sub-branches, and output the center regression parameters, confidence parameters, semantic classification parameters and uncertainty parameters. Step 262: Calculate the center coordinates of the rebar nodes based on the central regression parameters, and then filter candidate nodes based on the center coordinates of the rebar nodes; Step 263: For each candidate node, determine the category corresponding to the highest probability based on its semantic classification parameters, calculate the confidence level of the corresponding category, and generate a candidate node set; The candidate node set includes: center coordinates of the rebar nodes, confidence parameters, confidence levels for the corresponding categories, and uncertainty parameters; Step 264: Perform three-dimensional nonmaximum suppression on the candidate node set to generate the final node set, and use the uncertain parameters in the final node set as the standard deviation of the node coordinates in each direction. If the uncertain parameters in the final node set are greater than the uncertain threshold, then trigger uncertainty gating.

5. The multi-scenario adaptive rebar connection robot visual intelligent operation method according to claim 4, characterized in that, The detection of enhanced voxel feature sequences using a three-dimensional detection head also includes: Each candidate node is given a confidence flag. If the confidence parameter of the candidate node is lower than the confidence threshold, the confidence flag is set to low confidence, and the candidate node is set as a low confidence candidate node. Traverse the final node set and add all low-confidence candidate nodes to the re-examination queue; The queue of nodes to be re-inspected is traversed, the uncertainty direction of the current view is calculated for each node to be re-inspected, the optimal new view is re-planned based on the uncertainty direction, and the rebar connection robot is controlled to move the camera to the optimal new view and acquire images.

6. The multi-scenario adaptive rebar connection robot vision intelligent operation method according to claim 1, characterized in that, Based on semantic tags, the rebar intersection nodes are constructed into a rebar topology diagram, including: Treat each rebar intersection node as a graph node. Based on the rebar mesh topology prior, if two graph nodes are located on the same rebar and are adjacent, then an undirected edge is established. Each graph node is assigned 3D location features, semantic labels, connection methods, confidence levels, and uncertainties as node features; and each undirected edge is assigned Euclidean distance, direction, and reinforcement properties as edge features.

7. The multi-scenario adaptive rebar connection robot visual intelligent operation method according to claim 6, characterized in that, Based on the determined rebar connection method, the rebar topology diagram is traversed, including the following steps: Step 31: Model the traversal of the steel reinforcement topology graph as a sequence representation learning problem with graph structure constraints; where the set of all unvisited nodes is used as the state space, and the selection of the next visited node is used as the action space. Step 32: Select unvisited nodes from the state space, concatenate the rebar connection method with the node features and edge features of the unvisited nodes to form input features, input the input features into the graph neural network based on the attention mechanism, and output the currently visited nodes and the next target visited node; Step 33: Use the improved A* algorithm to smoothly filter the collision detection of the currently visited nodes and output the trajectory point sequence; Step 34: Update the state space based on the trajectory point sequence, and repeat steps 31 to 34 until there are no unvisited nodes in the state space.

8. The multi-scenario adaptive rebar connection robot vision intelligent operation method according to claim 7, characterized in that, The input features are fed into an attention-based graph neural network, which includes the following steps: Step 331: Input the input features into the graph encoder, which includes 3 layers of GATv2. In each layer of GATv2, attention is calculated on the input features, and the node features are updated according to the calculation results. Step 332: Use an attention-based sequential generation network to perform attention calculation on the updated node features and their context, and select the next node based on the calculation results; Step 333: Calculate the probability distribution of the selected next node using the softmax function, and select the point with the highest probability as the next target node to visit.

9. The multi-scenario adaptive rebar connection robot vision intelligent operation method according to claim 8, characterized in that, An improved A* algorithm is used to smoothly filter the collision detection of currently visited nodes, including the following steps: Step 341: Using the robotic arm base as a coordinate system, the robotic arm workspace is discretized into a voxel grid to form a three-dimensional semantic map. Based on the semantic information, a cost coefficient is assigned to the three-dimensional semantic map to form a three-dimensional semantic cost map. Step 342: Take the current position of the robotic arm end effector as the starting point, take the next target access node as the target point, traverse the graph nodes between the starting point and the target point, and generate the initial path; Step 343: Use B-spline smoothing and collision detection technology to backtrack the initial path, and use kinematic constraints to smooth and filter the backtracked path to generate a joint space trajectory; wherein, the kinematic constraints include joint accessibility and judgment and direction constraints.

10. A multi-scenario adaptive rebar connection robot visual intelligent operation system, used to implement the multi-scenario adaptive rebar connection robot visual intelligent operation method as described in any one of claims 1 to 9, characterized in that, include: The scene adaptation and autonomous navigation module is used to adapt the rebar connection robot to the work scene and move the adapted rebar connection robot to the work area. The adaptive image acquisition and rebar node recognition and localization module is used to adaptively acquire images of the work area, input the acquired images into a 3D node detection network for rebar node recognition and localization, and output the rebar intersection nodes and their semantic labels. The connection method selection and semantic path planning module is used to construct a rebar topology diagram based on semantic tags for rebar intersection nodes, and to traverse the rebar topology diagram according to the determined rebar connection method to generate a work path that conforms to the rebar structure operation logic. The connection and quality verification module is used by the rebar connection robot to connect rebar nodes one by one according to the work path, and to perform quality verification on the rebar node after completing the connection of a single rebar node.

Images