A control method, system and device of a steel bar binding robot and a storage medium

Abstract

This application discloses a control method, system, device, and storage medium for a rebar tying robot, belonging to the technical field of automation and robotics. The control method for the rebar tying robot includes: extracting a foreground 3D point cloud from the original 3D point cloud of the rebar cage; setting points in the foreground 3D point cloud located within the rebar intersection area as candidate points, constructing a candidate set containing all the candidate points; iteratively performing a conformity calculation operation based on the candidate set, and setting the candidate plane with the highest conformity as a reference plane; and controlling the movement of the rebar tying robot according to the pose relationship between the rebar tying robot and the reference plane. This application can accurately identify rebar mesh, improving the control accuracy and work quality of the rebar tying robot.

Description

Technical Field

[0001] This application relates to the fields of building automation and robotics, and in particular to a control method, system, equipment and storage medium for a rebar tying robot. Background Technology

[0002] In the automated production and binding of rebar cages, a crucial and essential prerequisite is that the rebar binding robot must be able to autonomously travel to the target rebar cage and achieve precise alignment with it in the final meter. The accuracy and stability of this docking process directly determine the success or failure of all subsequent precision operations of the robotic arm (such as automatic binding).

[0003] Because the rebar cage contains a large number of identical or highly similar units (hundreds or thousands of rebar intersections) arranged repeatedly in space, and because the area around the rebar cage is often filled with debris or other equipment, it is difficult to directly locate the rebar cage using visual positioning algorithms. This results in low control accuracy for the rebar tying robot, affecting the quality of the work.

[0004] Therefore, how to accurately identify rebar mesh and improve the control accuracy and work quality of rebar tying robots is a technical problem that needs to be solved by those skilled in the art. Summary of the Invention

[0005] The purpose of this application is to provide a control method, system, equipment, and storage medium for a rebar tying robot, which can accurately identify rebar mesh and improve the control accuracy and work quality of the rebar tying robot.

[0006] To address the aforementioned technical problems, this application provides a control method for a rebar tying robot, comprising:

[0007] Extract the foreground 3D point cloud from the original 3D point cloud of the rebar cage; wherein the rebar cage includes at least one layer of rebar mesh, and the foreground 3D point cloud is the 3D point cloud of the rebar mesh layer closest to the rebar binding robot;

[0008] Points in the foreground 3D point cloud that are located within the area where the steel bars intersect are set as candidate points, and a candidate set containing all the candidate points is constructed.

[0009] Based on the candidate set, an iterative compliance calculation operation is performed, and the candidate plane with the highest compliance is set as the reference plane; wherein, the compliance calculation operation includes: selecting 3 non-collinear candidate points from the candidate set as current reference points, setting the plane where the 3 current reference points are located as candidate planes, setting candidate points in the candidate set whose distance from the candidate plane is less than a preset value as interior points, and determining the compliance of the candidate plane based on the number and spatial distribution of interior points;

[0010] The movement of the rebar tying robot is controlled according to the pose relationship between the rebar tying robot and the reference plane.

[0011] Optionally, the conformity of the candidate plane can be determined based on the number and spatial distribution of interior points, including:

[0012] The ratio of the number of interior points to the number of candidate points in the candidate set is set as the coplanarity value;

[0013] Project all interior points onto the candidate plane, and perform line detection based on Hough transform on the projected interior points to obtain multiple line segments;

[0014] Determine the length and angle of each line segment, and perform weighted statistics on the angle of each line segment with the length as the weight to obtain the significance value of each angle;

[0015] The two angles with the highest significance values ​​are set as the first principal direction and the second principal direction, and the sine value of the angle between the first principal direction and the second principal direction is set as the orthogonality value.

[0016] Projection points are obtained by projecting all interior points in a preset direction, and the coverage rate is determined based on the two-dimensional convex hull area corresponding to all projection points; wherein, the original three-dimensional point cloud is generated based on the image acquired by the camera, and the preset direction is the principal optical axis direction of the camera;

[0017] The conformity of the candidate plane is calculated based on the coplanarity value, the orthogonality value, and the coverage.

[0018] Optionally, before extracting the foreground 3D point cloud from the original 3D point cloud of the steel cage, the following steps are also included:

[0019] Acquire RGB (red, green, blue) and depth images of the steel cage, and generate the original 3D point cloud based on the RGB and depth images;

[0020] Accordingly, determining the coverage rate based on the two-dimensional convex hull area corresponding to all projection points includes:

[0021] The ratio of the area of ​​the two-dimensional convex hull corresponding to all projection points to the total pixel area of ​​the RGB image is set as the coverage rate.

[0022] Optionally, controlling the movement of the rebar tying robot based on its pose relationship with the reference plane includes:

[0023] Obtain the three-dimensional coordinates of the body reference point and the body orientation vector of the rebar tying robot;

[0024] The distance between the fuselage reference point and the reference plane is set as the distance deviation value;

[0025] The projection of the normal vector of the reference plane onto the horizontal plane is set as the first vector;

[0026] The projection of the fuselage orientation vector onto the horizontal plane is set as the second vector;

[0027] Set the angle between the first vector and the second vector as the angle deviation value;

[0028] The movement of the rebar tying robot is controlled based on the distance deviation value and the angle deviation value.

[0029] Optionally, controlling the movement of the rebar tying robot based on the distance deviation value and the angle deviation value includes:

[0030] The target linear velocity is determined based on the distance deviation value, and the target angular velocity is determined based on the angle deviation value.

[0031] Motion commands are generated based on the target linear velocity and the target angular velocity, and then sent to the motion controller of the rebar tying robot to control the movement of the rebar tying robot.

[0032] Optionally, before setting points in the foreground 3D point cloud that are located within the rebar intersection area as candidate points, the method further includes:

[0033] The foreground image is obtained by back-projecting the foreground 3D point cloud;

[0034] Identify the rebar intersections in the foreground image, and set a corresponding rebar intersection region for each rebar intersection; wherein the center point of the rebar intersection region is the rebar intersection.

[0035] Optionally, points in the foreground 3D point cloud located within the area where the reinforcing bars intersect are set as candidate points, including:

[0036] Set the median pixel depth within the neighborhood of the intersection of the reinforcing bars as the depth reference value;

[0037] Points in the foreground 3D point cloud that are located within the area where the steel bars intersect and whose pixel depth differs from the depth reference value by less than a critical value are set as candidate points.

[0038] This application also provides a control system for a rebar tying robot, including:

[0039] A foreground point cloud extraction module is used to extract the foreground 3D point cloud from the original 3D point cloud of the rebar cage; wherein the rebar cage includes at least one layer of rebar mesh, and the foreground 3D point cloud is the 3D point cloud of the rebar mesh layer closest to the rebar binding robot;

[0040] The intersection point cloud determination module is used to set points in the foreground 3D point cloud that are located within the steel bar intersection area as candidate points, and to construct a candidate set containing all the candidate points;

[0041] The reference plane determination module is used to iteratively perform a conformity calculation operation based on the candidate set, and set the candidate plane with the highest conformity as the reference plane; wherein, the conformity calculation operation includes: selecting 3 non-collinear candidate points from the candidate set as current reference points, setting the plane where the 3 current reference points are located as candidate planes, setting candidate points in the candidate set whose distance from the candidate plane is less than a preset value as interior points, and determining the conformity of the candidate plane based on the number and spatial distribution of interior points;

[0042] The motion control module is used to control the movement of the rebar tying robot according to the pose relationship between the rebar tying robot and the reference plane.

[0043] This application also provides a storage medium on which a computer program is stored, wherein the computer program, when executed, implements the steps of the control method for the rebar tying robot described above.

[0044] This application also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor, when calling the computer program in the memory, implements the steps of the control method for the rebar tying robot described above.

[0045] This application discloses a control method for a rebar tying robot. The method extracts a foreground 3D point cloud from the original 3D point cloud of the rebar cage and determines candidate points within the rebar intersection area from the foreground 3D point cloud, thus obtaining a candidate set. Based on the candidate points in the candidate set, this application iteratively performs a conformity calculation operation, selecting the candidate plane with the highest conformity from multiple candidate planes generated during the iteration process as the reference plane. The conformity of the aforementioned candidate plane is determined based on the number and spatial distribution of interior points. The reference plane selected based on the conformity can accurately characterize the geometric structure of the rebar mesh and effectively suppress local noise or occlusion interference, providing a stable and reliable pose reference for the rebar tying robot. Therefore, this application can accurately identify the rebar mesh, improving the control accuracy and work quality of the rebar tying robot. This application also provides a control system for the rebar tying robot, a storage medium, and an electronic device, all possessing the above-mentioned beneficial effects, which will not be elaborated further here. Attached Figure Description

[0046] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 A flowchart illustrating a control method for a rebar tying robot provided in this application embodiment;

[0048] Figure 2 A filtered point cloud provided in the embodiments of this application A schematic diagram;

[0049] Figure 3 This is a schematic diagram of the control system of a rebar tying robot provided in an embodiment of this application. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0051] Please see below. Figure 1 , Figure 1 A flowchart illustrating a control method for a rebar tying robot provided in an embodiment of this application.

[0052] Specific steps may include:

[0053] S101: Extract the foreground 3D point cloud from the original 3D point cloud of the steel cage.

[0054] This embodiment can be applied to the control device of a rebar tying robot, which is a robot used to perform rebar tying operations on rebar cages.

[0055] The aforementioned rebar cage includes at least one layer of rebar mesh. The original 3D point cloud can be obtained by scanning the rebar cage with a 3D sensing device (such as a depth camera or LiDAR). The original 3D point cloud includes the point cloud of each layer of rebar mesh and the background. This step can perform planar segmentation and background culling operations on the original 3D point cloud to obtain the foreground 3D point cloud. The aforementioned foreground 3D point cloud is the 3D point cloud of the rebar mesh layer closest to the rebar tying robot.

[0056] Depth cameras use specific sensor technologies (such as structured light or Time of Flight, or ToF) to capture 3D scene information. They can provide depth information of objects, enabling robots to better understand their spatial environment.

[0057] S102: Set the points in the foreground 3D point cloud that are located within the area where the steel bars intersect as candidate points, and construct a candidate set containing all the candidate points.

[0058] In this process, there are multiple intersecting steel bars in the steel mesh. The area where the horizontal and vertical steel bars intersect is called the steel bar intersection area. This step can determine the location information of the steel bar intersection area in the layer of steel mesh closest to the steel bar binding robot based on the foreground 3D point cloud, and then set the points in the foreground 3D point cloud that are located in the steel bar intersection area as candidate points.

[0059] The nearest layer of rebar mesh can have multiple rebar intersection areas, and each rebar intersection area can have multiple candidate points. After determining the candidate points, a candidate set containing all candidate points can be constructed.

[0060] S103: Perform compliance calculation operation iteratively based on the candidate set, and set the candidate plane with the highest compliance as the reference plane.

[0061] After obtaining the candidate set, a conformity calculation operation can be performed iteratively based on the candidate set. Each conformity calculation operation determines a candidate plane and its conformity (also known as structural conformity). In this embodiment, the number of iterations of the above conformity calculation operation can be counted; if the number of iterations is greater than the maximum number of iterations, the operation of setting the candidate plane with the highest conformity as the reference plane is performed.

[0062] Specifically, the process of each compliance calculation operation is as follows: Select three non-collinear candidate points from the candidate set as the current reference points, set the plane where the three current reference points are located as the candidate plane, set the candidate points in the candidate set whose distance from the candidate plane is less than a preset value as interior points, and determine the compliance of the candidate plane based on the number and spatial distribution of interior points.

[0063] In the above process, three non-collinear candidate points can be randomly selected from the candidate set as the current reference points so that a new candidate plane can be determined in each iteration.

[0064] S104: Control the movement of the rebar tying robot according to the pose relationship between the rebar tying robot and the reference plane.

[0065] After determining the reference plane, the pose relationship between the rebar tying robot and the reference plane can be determined based on the current pose of the rebar tying robot. Then, the movement of the rebar tying robot can be controlled based on the pose relationship so that the rebar tying robot can complete the docking operation of the rebar cage.

[0066] This embodiment extracts the foreground 3D point cloud from the original 3D point cloud of the rebar cage, and determines candidate points within the rebar intersection area from the foreground 3D point cloud, thus obtaining a candidate set. Based on the candidate points in the candidate set, this embodiment iteratively performs a conformity calculation operation, selecting the candidate plane with the highest conformity from multiple candidate planes generated during the iteration process as the reference plane. The conformity of the aforementioned candidate plane is determined based on the number and spatial distribution of interior points. The reference plane selected based on the conformity can accurately characterize the geometric structure of the rebar mesh and effectively suppress local noise or occlusion interference, providing a stable and reliable pose reference for the rebar tying robot. Therefore, this embodiment can accurately identify the rebar mesh, improving the control accuracy and work quality of the rebar tying robot.

[0067] As for Figure 1 As further described in the corresponding embodiment, during the iterative execution of the conformity calculation operation, this embodiment can calculate the conformity from three dimensions: coplanarity value, orthogonality value, and coverage. Specifically, the conformity of the candidate plane can be calculated based on the coplanarity value, the orthogonality value, and the coverage. For example, the coplanarity value, the orthogonality value, and the coverage can be weighted to obtain the conformity of the candidate plane.

[0068] The process for determining the coplanarity value is as follows: the ratio of the number of interior points to the number of candidate points in the candidate set is set as the coplanarity value. The number of interior points refers to the number of interior points of the candidate plane determined in this iteration; the number of candidate points in the candidate set refers to the total number of candidate points contained in the candidate set.

[0069] The process for determining the orthogonality value is as follows: Project all interior points onto the candidate plane (i.e., the candidate plane determined in this iteration), perform line detection based on Hough transform on the projected interior points to obtain multiple line segments; determine the length and angle of each line segment, and perform weighted statistical analysis on the angle of each line segment with length as the weight to obtain the significance value of each angle; set the two angles with the highest significance values ​​as the first principal direction and the second principal direction, and set the sine of the angle between the first principal direction and the second principal direction as the orthogonality value. In the above process, for each detected line segment, its length is used as the weight to weight and the direction corresponding to its angle is weighted and accumulated, thereby forming a significance distribution in the angle domain; the longer the line, the greater its contribution to the angle, and finally the significance value of each direction is obtained, reflecting the dominance of the structure in that direction.

[0070] The process for determining the coverage rate is as follows: Projecting all interior points in a preset direction to obtain projection points, and determining the coverage rate based on the two-dimensional convex hull area corresponding to all projection points; wherein, the original three-dimensional point cloud is generated based on the image acquired by the camera, and the preset direction is the principal optical axis direction of the camera.

[0071] As a feasible implementation, before extracting the foreground 3D point cloud from the original 3D point cloud of the steel cage, RGB and depth images of the steel cage can be acquired, and the original 3D point cloud can be generated based on the RGB and depth images. Correspondingly, the process of determining the coverage rate based on the two-dimensional convex hull area corresponding to all projection points includes: setting the ratio of the two-dimensional convex hull area corresponding to all projection points to the total pixel area of ​​the RGB image as the coverage rate.

[0072] As for Figure 1 A further description of the corresponding embodiment: the process of controlling the movement of the rebar tying robot based on the pose relationship between the rebar tying robot and the reference plane includes: acquiring the three-dimensional coordinates of the robot's body reference point and the robot's body orientation vector; setting the distance between the robot's body reference point and the reference plane as a distance deviation value; setting the projection of the reference plane's normal vector onto the horizontal plane as a first vector; setting the projection of the robot's body orientation vector onto the horizontal plane as a second vector; setting the angle between the first vector and the second vector as an angle deviation value; and controlling the movement of the rebar tying robot based on the distance deviation value and the angle deviation value.

[0073] Specifically, the process of controlling the movement of the rebar tying robot based on the distance deviation value and the angle deviation value includes: determining the target linear velocity based on the distance deviation value, determining the target angular velocity based on the angle deviation value; generating a motion command based on the target linear velocity and the target angular velocity, and sending the motion command to the motion controller of the rebar tying robot to control the movement of the rebar tying robot.

[0074] As for Figure 1 In a further description of the corresponding embodiment, before setting the points in the foreground 3D point cloud that are located within the rebar intersection area as candidate points, the rebar intersection area can also be determined by the following method: back-projecting the foreground 3D point cloud to obtain a foreground image; identifying the rebar intersection points in the foreground image, and setting a corresponding rebar intersection area for each rebar intersection point; wherein, the center point of the rebar intersection area is the rebar intersection point.

[0075] Accordingly, the process of setting points in the foreground 3D point cloud that are located within the rebar intersection area as candidate points includes: setting the median pixel depth within the neighborhood of the rebar intersection (i.e., the neighborhood of the rebar intersection) as a depth reference value to suppress noise and outlier effects; and setting points in the foreground 3D point cloud that are located within the rebar intersection area and whose pixel depth differs from the depth reference value by less than a critical value as candidate points. This operation effectively filters out mismatched points caused by occlusion, reflection, or background interference, improving the reliability of candidate points.

[0076] The process described in the above embodiments is illustrated below through an example of a rebar tying robot in a practical application.

[0077] To address the challenge of high-precision docking between rebar tying robots and rebar cages, especially in industrial environments, the industry typically employs the following technical solutions.

[0078] (1) Manually Marked Solution: Visual markers with unique IDs (identifiers), such as AprilTags (visual reference markers) or QR codes, are manually affixed or placed in or near the work target. The rebar tying robot identifies these markers through a camera and uses a visual library to directly and accurately calculate the six-degree-of-freedom pose (position and orientation) of the camera relative to the markers, thereby achieving positioning and docking. This method heavily relies on the artificial modification of the environment. In large-scale or dynamically changing factory environments, it is impractical to pre-deploy and maintain thousands of markers. It increases additional operational procedures and costs, violating the original intention of full-process automation. The markers may be obscured by stains or dust, or become difficult to identify due to wear or light reflection, leading to the failure of the entire system. In addition, the placement and angle of the markers directly affect the final accuracy. The robot must be able to clearly see the markers within the camera's field of view to work. This method is ineffective in scenarios with obstructions or where docking tasks need to be initiated from a distance.

[0079] (2) Navigation scheme based on general SLAM (Simultaneous Localization and Mapping): Using general simultaneous localization and mapping techniques, such as LiDAR-based or vision-based localization and mapping algorithms, the robot builds a global map in the environment. Then, the coordinates of the target steel cage's work point are marked on the map. The robot performs self-localization on the map and plans a path to travel to that coordinate point. SLAM technology solves the navigation problem at the room scale, and its positioning accuracy is usually at the centimeter level. However, the final meter docking task requires millimeter-level relative pose accuracy. SLAM can only bring the robot to the vicinity of the target, but it cannot achieve the final precise alignment.

[0080] (3) Scheme based on a single depth camera or 3D LiDAR: Spatial scanning and object recognition are performed using a single depth camera or 3D LiDAR. Although these technologies can provide information in three-dimensional space, they are insufficient in terms of data processing speed, occlusion handling, and cost-effectiveness. In particular, a single technology is difficult to meet the requirements of high-precision positioning and environmental adaptability at the same time.

[0081] (4) Standard Model Fitting Approach: Acquire the overall 3D point cloud data of the target using a depth camera, and then directly use the standard Random Sample Consensus (RANSAC) algorithm to fit an ideal geometric model (usually a plane) into this point cloud, using this fitted plane as the docking reference. The only criterion for the standard RANSAC algorithm is the number of inliers, i.e., finding a model that fits the most data points. In real-world scenes full of background interference and noise, this can lead to disastrous errors. For example, the algorithm might find a plane that partially overlaps with the target's steel cage and belongs to the background wall, simply because it contains more inliers, even though this plane is physically incorrect. This method treats all point cloud data equally and cannot distinguish which points form a structurally reasonable plane. A tilted plane composed of a few local points that happens to fit a lot of noise points may be as good as, or even better than, a correct plane composed of globally distributed, structurally regular points in standard RANSAC. It lacks utilization of prior knowledge of the target structure (such as the orthogonality of the steel mesh).

[0082] A 3D point cloud refers to a collection of data points representing the surface of an object in three-dimensional space. Similar to a 3D skeleton composed of countless tiny coordinate points, it can accurately reproduce the object's geometry and spatial position. 3D point clouds are typically acquired by depth cameras or 3D LiDAR by measuring the distances between the sensor and various points on the object's surface.

[0083] Random Sample Consensus (RANSAC) is an iterative algorithm used to fit a correct mathematical model from a set of observations containing a large number of outliers (i.e., noisy or irrelevant data). In short, RANSAC does not use all the data at once. Instead, it randomly selects a small subset of data points (the minimum number required to fit the model) to build a candidate model (e.g., randomly selecting 3 points from a set to form a plane). It then checks how many other points in the dataset fit this candidate model (these fitting points are called inliers). This process is repeated until the model with the most inliers is selected as the final result.

[0084] It is evident that when faced with the structured and repetitive characteristics of the reinforcing cages addressed in this embodiment, all the aforementioned traditional solutions suffer from fundamental deficiencies. Achieving such high-precision docking in a real reinforcing cage operation environment faces three core technical challenges, which existing technical solutions struggle to overcome effectively:

[0085] The challenge of perceptual ambiguity: The target object (reinforcing cage) is typically composed of numerous identical or highly similar units (hundreds or thousands of rebar intersections) arranged repeatedly in space. Traditional robot vision localization algorithms heavily rely on finding unique visual feature points (such as corner points or textures in an image), but this fails completely in this scenario because each unit looks the same, making it impossible for the robot to establish a stable and unique reference point. Furthermore, the metallic material of the target surface often exhibits high reflectivity and weak texture, further exacerbating the difficulty of feature extraction.

[0086] Regarding the challenges of data imperfection and model mismatch: 3D point cloud data acquired by robotic sensors (such as depth cameras and LiDAR) inevitably contains a large amount of noise and outliers. More importantly, real-world targets (manually tied steel cages) are not perfect geometric models conforming to mathematical definitions due to construction tolerances and deformations (e.g., they are not perfectly flat planes). When an algorithm attempts to match an ideal mathematical model with noisy and imperfect real-world data, it is highly susceptible to matching failures or getting trapped in an incorrect local optimum.

[0087] The challenges of background interference and information purification: In real factory or construction site environments, the target object does not exist in a clean background. It is often surrounded by clutter, other equipment, or people moving around, all of which contribute to background interference and enter the robot's sensors. Accurately extracting key information belonging solely to the target object from massive amounts of raw data mixed with significant background noise, and then using this information for localization, presents a significant challenge.

[0088] Based on the above analysis, the core technical challenge currently facing rebar tying robots is that existing navigation and positioning technologies either rely too heavily on unreliable surface appearance information or employ simple geometric fitting methods that cannot cope with the imperfections of real-world data. Therefore, a novel technical solution is needed that can intelligently understand the internal structure of the rebar cage without the need for pre-set manual markings (such as QR codes), ignoring the repetitiveness of appearance and data noise, thereby achieving robust and high-precision autonomous docking in complex environments.

[0089] To address the technical problems existing in the aforementioned related technologies, this embodiment provides a solution for docking a rebar tying robot based on the structural deconstruction of a rebar cage. This solution enables the rebar tying robot to perform high-precision and high-reliability autonomous positioning and docking with the rebar cage at the rebar prefabrication or construction site. By deeply understanding and reasoning about the target's internal geometric structure, this solution eliminates reliance on unreliable surface appearance information, fundamentally solving the problems of perceptual ambiguity and insufficient positioning accuracy in existing technologies when dealing with industrial targets with repetitive and weakly textured features.

[0090] This solution abandons the reliance on unreliable pixel appearances and uses an intelligent process of layered filtering, multiple assumptions, and optimal interpretation to directly infer the target's internal geometric structure from a noisy and ambiguous scene, and uses this as a benchmark to achieve high-precision docking.

[0091] The rebar tying robot used in this solution is equipped with a depth camera and an embedded artificial intelligence computer.

[0092] Depth cameras are used to capture RGB-D (color depth) images of the environment, obtaining depth information such as the color, shape, size, and relative position of objects.

[0093] Embedded artificial intelligence computers include a computing unit (also known as a data computing and processing unit) and a motion and control unit. The computing unit is used to process the data collected by the depth camera, perform data fusion, and realize environmental perception and target detection. The motion and control unit is used to perform path planning, dynamic obstacle avoidance, and control the movement of the robot based on the processed data.

[0094] In the task of docking the robot with the steel cage, this embodiment determines the docking reference by understanding the target's internal structure.

[0095] This embodiment does not use the traditional principle of maximizing the number of interior points, but instead uses a comprehensive score that includes an assessment of structural rationality to select the optimal docking reference plane from numerous candidate models.

[0096] The structural conformity comprehensive scoring method provided in this embodiment combines three specific technical indicators—data fidelity (coplanarity), structural regularity (orthogonality), and global distribution (coverage)—in a weighted manner to serve as a criterion for evaluating the quality of a geometric model (such as a plane).

[0097] The dimension reduction method in docking control provided in this embodiment provides a bridge connecting perception and control. After determining the optimal docking reference plane, the mathematical parameters of the three-dimensional plane are directly calculated and converted into two one-dimensional control quantities (normal distance error and angle deviation) that the robot chassis can directly execute.

[0098] The docking process of the rebar tying robot includes the following steps 1 to 6:

[0099] Step 1: 3D data plane segmentation and background removal.

[0100] The computing unit receives one frame of RGB image and one frame of depth image acquired in real time by an RGB-D depth camera. Using the camera's intrinsic parameters, the computing unit combines the received images into a single raw 3D point cloud. The computing unit is in The RANSAC plane segmentation algorithm is executed once. The algorithm's configuration parameters are set to find and identify the plane that is closest to the camera and has the largest area. The computational unit extracts all points belonging to this identified plane, forming a filtered point cloud. (i.e., the foreground 3D point cloud), and discard all points that do not belong to the plane.

[0101] Please see Figure 2 , Figure 2 A filtered point cloud provided in the embodiments of this application A schematic diagram.

[0102] Step 2: Determine the intersection points of the reinforcing bars through two-dimensional image detection.

[0103] This step uses the filtered point cloud output from step 1. For input.

[0104] The computing unit will Perform back projection to generate a two-dimensional image with the background removed. This image shows the surface layer of the steel reinforcement, without any background or interference from the many other layers of reinforcement behind it.

[0105] The computing unit will The input is fed into a trained YOLOv8 (an object detection model) deep learning detection model, which is used to identify all steel bar intersections in the image.

[0106] This step outputs a list of inspection results. Each element in the list (i.e., the rebar intersection) contains a rectangular bounding box. (i.e., the area where the reinforcing bars intersect) and a pixel coordinate representing the center of that intersection point. .

[0107] The rectangular bounding box mentioned above can be used for YOLOv8 models in The image shows the automatically generated output after identifying the rebar intersection as the target. When the robot is far from the rebar cage, the bounding box size output by the YOLO model is correspondingly smaller. When the robot is very close to the rebar cage, the bounding box size output by the YOLO model is correspondingly larger. This approach demonstrates strong adaptability and robustness to changes in the distance between the robot and the target.

[0108] Step 3: Generate a 3D point cloud cluster of rebar intersections (i.e., a cross point cloud cluster of rebar intersections, also known as a candidate set).

[0109] This step is the list of detection results output from the previous step. And the filtered point cloud output from the previous step. For input.

[0110] For each element in the detection result list, the calculation unit performs the following operations in sequence:

[0111] Rectangular bounding box Projected to The process begins by segmenting the point cloud within the bounding box and filtering out all point clouds outside the box. Since the bounding box contains intersection points, the filtered result is a set of point cloud clusters representing the intersection points of the reinforcing bars. This step outputs a list of 3D point cloud clusters. . and The coordinates of the candidate point.

[0112] Step 4: Solve the docking reference plane.

[0113] This step takes the point cloud cluster list output from the previous step as input.

[0114] The computational unit executes a structure conformity iterative algorithm on the 3D point cloud cluster. The execution process of this algorithm is as follows: steps 41-43:

[0115] Step 41, Initialization: Set an iteration count N (e.g., N=1000). Initialize a variable. The highest score is set to 0, and an empty variable is initialized. (i.e., the optimal plane variable) is used to store the optimal plane.

[0116] Step 42: Iterate through the following loop steps 42a-42e N times:

[0117] Step 42a: Randomly select 3 non-collinear coordinate points (i.e., the current reference points) from the 3D point cloud cluster.

[0118] Step 42b: Based on the three coordinate points selected in step 42a, calculate a unique candidate plane. The mathematical parameters (including normal vector and distance).

[0119] Step 42c: For candidate planes Scoring will be conducted.

[0120] This step determines the set of interior points: traversing all points in the 3D point cloud cluster, and comparing them with... The vertical distance is less than the preset distance threshold All points are classified as interior points of the candidate plane.

[0121] Based on the set of interior points, three sub-scores are calculated for the candidate plane: coplanarity score (i.e., the coplanarity value mentioned above), orthogonality score (i.e., the orthogonality value mentioned above), and coverage score (i.e., the coverage mentioned above).

[0122] Coplanarity score The calculation formula is: (number of interior points) / (total number of points in the 3D point cloud cluster).

[0123] Orthogonality scoring The calculation formula is: Project all points in the interior point set onto the candidate plane. The algorithm then performs line detection and angle scoring based on the Hough Transform on these projected point sets. It searches and fits several salient line segments in the point cloud. The output is a list of lines, each containing its angle and length. The angles (ranging from 0 to 180 degrees) of all detected lines are weighted (weighted by the line length) to construct an angle distribution histogram. The two highest energy peaks in the angle histogram are then identified and designated as the first principal directions. Second main direction These two represent the two main directions of the reinforcing steel lines in the current plane. The sine of the angle between the two main directions is calculated as the orthogonality score. This yields an orthogonality score between 0 and 1.

[0124] Coverage score The calculation formula is as follows: Extract the original two-dimensional pixel coordinates corresponding to all points in the set of interior points. Calculate the area of ​​the convex hull formed by these two-dimensional coordinate points on the image plane, and then divide it by the entire image (i.e., the two-dimensional image with background removed generated in step 2). The total pixel area is used to obtain a ratio between 0 and 1.

[0125] Traditional methods often misidentify the background as the docking target when dealing with scenes containing walls, floors, or other equipment due to the larger number of background point clouds. This embodiment introduces orthogonality and coverage scoring, giving the algorithm the ability to distinguish the rationality of the structure. Even if the point cloud of the target steel cage does not dominate the field of view, this method can still accurately identify it based on its unique structural characteristics (square grid, widely distributed), fundamentally solving the docking failure problem caused by background interference.

[0126] Step 42d: Combine the above three sub-scores into a single comprehensive score. :

[0127] .

[0128] This comprehensive score is the compliance rate mentioned above.

[0129] Step 42e: Compare the current... and .if If it's higher, then update. For the current and update For the current .

[0130] Step 43: Determine the optimal plane.

[0131] After the loop in steps 42a-42e ends, The data stored in the database is the final docking reference plane (i.e., the reference plane mentioned above).

[0132] This step outputs the mathematical parameters of the docking reference plane. Specifically, the above mathematical parameters are the four coefficients A, B, C, and D in the plane equation Ax+By+Cz+D=0.

[0133] The three values ​​A, B, and C together constitute the normal vector n = (A, B, C) of the plane. This vector is perpendicular to the surface of the plane and defines its orientation. In practical calculations, this normal vector is usually normalized, i.e., its length is... This is to facilitate subsequent distance and angle calculations; sqrt represents the square root calculation.

[0134] D is a scalar constant that defines the position of the plane relative to the origin (0,0,0) of the coordinate system. When the normal vector n is normalized... The value is the vertical distance from the origin of the coordinate system to the plane.

[0135] Therefore, the mathematical parameters of the docking reference plane are specifically a set of four floating-point numbers: {A, B, C, D}. This set of parameters describes the precise attitude and position of the plane in three-dimensional space.

[0136] Step 3 directly generates coordinates for each detected intersection point from the raw depth data, ensuring the initial accuracy of the input data. Step 4, the structural conformity iterative algorithm, utilizes the global distribution information of dozens or even hundreds of such high-precision points, further eliminating random errors at individual points through optimal fitting. This embodiment does not rely on unreliable positioning of individual feature points, but determines the docking reference through global optimal calculation of the overall target structure. The final docking accuracy can be stabilized at the millimeter level, fully meeting the stringent requirements of subsequent precise robotic arm operations such as automated binding. This embodiment achieves high-precision docking accuracy, meeting the requirements of subsequent precision operations.

[0137] Step 5: Robot docking with servo control.

[0138] This step uses the docking reference plane parameters output in the previous step, as well as the robot's real-time status information (including the three-dimensional coordinates of a predefined vehicle reference point). and the direction the car is facing. ) is the input. The vehicle reference point is the fuselage reference point mentioned earlier, and the vehicle nose orientation vector is the fuselage orientation vector mentioned earlier.

[0139] The computing unit executes a continuous closed-loop control process that repeats the following operations at a fixed frequency (e.g., 20Hz):

[0140] Distance error calculation: Based on Based on the coordinates and parameters of the docking reference plane, calculate... The perpendicular distance to the plane. This distance value is defined as the distance error (i.e., the distance deviation value). .

[0141] Angular deviation calculation: Project the normal vector of the docking reference plane onto the horizontal ground to obtain the first vector. Orient the robot's front end toward the vector. It is also projected onto the horizontal ground to obtain the second vector. .calculate and The angle between them is defined as the angle deviation (i.e., the angle deviation value). .

[0142] Control command generation: Distance error The input is given to the first PID (Proportional-Integral-Derivative) controller, which outputs a target linear velocity. At the same time, the angle deviation The input is given to a second PID controller, which outputs a target angular velocity. .

[0143] Command issued: Set the target linear velocity and target angular velocity The motion commands are combined into a single motion command and sent to the motion controller of the robot chassis, which then drives the chassis to physically execute the motion.

[0144] This step outputs continuous speed commands to drive the robot chassis.

[0145] Traditional control processes are complex and computationally intensive, easily leading to robot response delays and motion jitter. This invention bypasses the complex six-degree-of-freedom pose calculation and path planning, directly reducing the control target to two one-dimensional physical quantities: distance and angle. This embodiment makes the PID controller computation extremely efficient, enabling the robot to respond quickly and smoothly to minute changes in the target position. The entire docking process is seamless, stable, and reliable. This method allows for a fast and smooth docking process with a smooth robot motion trajectory.

[0146] Step 6: The assignment is completed and the task is finished.

[0147] The chassis will repeat steps 1-5 above until the distance error is reached. The absolute value is less than the preset distance tolerance And angular deviation The absolute value is less than the preset angle tolerance At this point, the process ends and the connection is complete.

[0148] Existing technologies (such as standard RANSAC) often incorrectly identify the background as the target when faced with 3D point clouds containing both background (such as walls and ground) and noise. This is because they can only identify the plane with the most points, but cannot determine which plane is the physically correct rebar cage plane. To address the above technical problems, this embodiment provides an optimal plane solution method based on structural conformity iteration, which can extract the correct docking reference in a disturbed environment.

[0149] This embodiment proposes a novel scoring process for comprehensively evaluating candidate planes during the RANSAC iterative process. After finding a candidate plane and its interior points in each iteration, it does not only count them, but also performs orthogonality quantization and coverage quantization calculations, and uses the results as the basis for decision-making.

[0150] In the orthogonality quantification calculation process, this embodiment projects the three-dimensional coordinates of all interior points of the plane onto the two-dimensional plane. Principal component analysis is performed on these projected two-dimensional point sets to calculate two principal direction vectors. Then, the alignment of these two vectors with the theoretical X and Y axes of the reinforcing cage mesh is calculated. This calculation step transforms the physical fact that the reinforcing cage mesh is square into a concrete index that can be executed and quantified by a computer. In practice, a plane composed of points on a wall will not obtain a high orthogonality score.

[0151] In the coverage quantization calculation, this embodiment identifies the pixel positions of all interior points on the original 2D image. The area of ​​the convex hull polygon formed by these pixels is calculated and then divided by the total area of ​​the image. This calculation step addresses the practical problem of local optima, where the algorithm might be misled by a small cluster of locally aligned points. By requiring coverage, we force the algorithm to find a plane that can account for most of the intersections within the global field of view.

[0152] By combining the results of the two calculation steps above with the traditional weighted sum of interior points, this invention enables the planar solution algorithm to possess structural cognitive capabilities for the first time. This embodiment no longer blindly fits the geometry, but actively seeks a plane that geometrically and physically resembles the main working face of a steel cage. This effectively immunizes against background and interference in practice, fundamentally improving the accuracy and reliability of the positioning.

[0153] Traditional robot docking control processes are lengthy and cumbersome, requiring sequential execution of sensor perception, calculation of the robot's complete six-degree-of-freedom pose (X, Y, Z, roll, pitch, and heading) relative to the target, path planner planning a complex trajectory, and motion controller tracking that trajectory. These processes are computationally intensive, slow to respond, and errors at any step can accumulate and be amplified, leading to jitter or instability in the final docking motion.

[0154] This embodiment provides a task-driven, dimension-reduced servo control scheme for achieving fast and stable robot docking. In engineering practice, this embodiment completely bypasses the complex control process described above. After calculating the globally optimal docking reference plane, the following two direct dimension-reduced calculations regarding distance error and angle deviation can be performed:

[0155] The dimensionality reduction calculation process for distance error includes: calculating the shortest vertical distance from a preset robot body reference point to the docking reference plane. This single value directly corresponds to the amount by which the robot needs to move forward or backward.

[0156] The dimensionality reduction calculation process for angular deviation includes: calculating the angle between the robot's current heading vector and the normal vector of the docking reference plane, projected onto the horizontal ground. This single value directly corresponds to the amount by which the robot needs to turn left or right.

[0157] This embodiment reduces a complex three-dimensional alignment problem into two independent, decoupled one-dimensional control problems. The PID controller no longer needs to process complex pose matrices; instead, it directly receives these two physically meaningful error values ​​and outputs direct linear and angular velocities. This embodiment has extremely low computational requirements, allowing it to run at high speeds on inexpensive embedded hardware. Its response is extremely fast; the robot can respond almost instantaneously to minute changes in planar position. The motion in this embodiment is extremely smooth and stable because it consistently minimizes the two most direct and critical error quantities, avoiding overshoot and oscillations inherent in complex control processes.

[0158] This embodiment directly utilizes the inherent geometric features of the rebar cage for identification and positioning, completely eliminating the need for pre-placement of any QR codes, April Tags, or other manual markers on the rebar cage or work site. This greatly simplifies the on-site deployment process, reduces material and maintenance costs, and allows this solution to be used plug-and-play on any standard rebar cage, truly achieving fully automated integration. Therefore, this embodiment requires no pre-set manual markings, has low deployment costs, and offers high flexibility and automation.

[0159] Traditional visual methods are easily affected by factors such as rebar surface corrosion, moisture reflection, and changes in lighting. This invention focuses only on the three-dimensional spatial geometric relationships formed by the intersections of rebars, which remain highly stable under the aforementioned interference conditions. Therefore, this embodiment is insensitive to the surface conditions of the rebars (such as corrosion and reflection) and has strong environmental adaptability; in real environments with varying lighting conditions and complex working conditions, such as construction sites and factories, this embodiment can still maintain robust performance.

[0160] Please see Figure 3 , Figure 3 This is a schematic diagram of the control system for a rebar tying robot provided in an embodiment of this application. The system may include:

[0161] The foreground point cloud extraction module 301 is used to extract the foreground 3D point cloud from the original 3D point cloud of the rebar cage; wherein the rebar cage includes at least one layer of rebar mesh, and the foreground 3D point cloud is the 3D point cloud of the rebar mesh layer closest to the rebar binding robot.

[0162] The intersection point cloud determination module 302 is used to set points in the foreground three-dimensional point cloud that are located within the steel bar intersection area as candidate points, and to construct a candidate set containing all the candidate points;

[0163] The reference plane determination module 303 is used to iteratively perform a conformity calculation operation based on the candidate set, and set the candidate plane with the highest conformity as the reference plane; wherein, the conformity calculation operation includes: selecting 3 non-collinear candidate points from the candidate set as current reference points, setting the plane where the 3 current reference points are located as the candidate plane, setting the candidate points in the candidate set whose distance from the candidate plane is less than a preset value as interior points, and determining the conformity of the candidate plane according to the number and spatial distribution of interior points;

[0164] The motion control module 304 is used to control the movement of the rebar tying robot according to the pose relationship between the rebar tying robot and the reference plane.

[0165] This embodiment extracts the foreground 3D point cloud from the original 3D point cloud of the rebar cage, and determines candidate points within the rebar intersection area from the foreground 3D point cloud, thus obtaining a candidate set. Based on the candidate points in the candidate set, this embodiment iteratively performs a conformity calculation operation, selecting the candidate plane with the highest conformity from multiple candidate planes generated during the iteration process as the reference plane. The conformity of the aforementioned candidate plane is determined based on the number and spatial distribution of interior points. The reference plane selected based on the conformity can accurately characterize the geometric structure of the rebar mesh and effectively suppress local noise or occlusion interference, providing a stable and reliable pose reference for the rebar tying robot. Therefore, this embodiment can accurately identify the rebar mesh, improving the control accuracy and work quality of the rebar tying robot.

[0166] Optionally, the process by which the reference plane determination module 303 determines the conformity of the candidate plane based on the number and spatial distribution of interior points includes:

[0167] The ratio of the number of interior points to the number of candidate points in the candidate set is set as the coplanarity value;

[0168] Project all interior points onto the candidate plane, and perform line detection based on Hough transform on the projected interior points to obtain multiple line segments;

[0169] Determine the length and angle of each line segment, and perform weighted statistics on the angle of each line segment with the length as the weight to obtain the significance value of each angle;

[0170] The two angles with the highest significance values ​​are set as the first principal direction and the second principal direction, and the sine value of the angle between the first principal direction and the second principal direction is set as the orthogonality value.

[0171] Projection points are obtained by projecting all interior points in a preset direction, and the coverage rate is determined based on the two-dimensional convex hull area corresponding to all projection points; wherein, the original three-dimensional point cloud is generated based on the image acquired by the camera, and the preset direction is the principal optical axis direction of the camera;

[0172] The conformity of the candidate plane is calculated based on the coplanarity value, the orthogonality value, and the coverage.

[0173] Furthermore, it also includes:

[0174] An image acquisition module is used to acquire RGB images and depth images of the steel cage before extracting the foreground 3D point cloud from the original 3D point cloud of the steel cage, and to generate the original 3D point cloud based on the RGB images and the depth images.

[0175] Accordingly, the process of the reference plane determination module determining the coverage rate based on the two-dimensional convex hull area corresponding to all projection points includes: setting the ratio of the two-dimensional convex hull area corresponding to all projection points to the total pixel area of ​​the RGB image as the coverage rate.

[0176] Furthermore, the process by which the motion control module 304 controls the movement of the rebar tying robot based on the pose relationship between the rebar tying robot and the reference plane includes:

[0177] Obtain the three-dimensional coordinates of the body reference point and the body orientation vector of the rebar tying robot;

[0178] The distance between the fuselage reference point and the reference plane is set as the distance deviation value;

[0179] The projection of the normal vector of the reference plane onto the horizontal plane is set as the first vector;

[0180] The projection of the fuselage orientation vector onto the horizontal plane is set as the second vector;

[0181] Set the angle between the first vector and the second vector as the angle deviation value;

[0182] The movement of the rebar tying robot is controlled based on the distance deviation value and the angle deviation value.

[0183] Furthermore, the process by which the motion control module 304 controls the movement of the rebar tying robot based on the distance deviation value and the angle deviation value includes: determining the target linear velocity based on the distance deviation value, determining the target angular velocity based on the angle deviation value; generating a motion command based on the target linear velocity and the target angular velocity, and sending the motion command to the motion controller of the rebar tying robot to control the movement of the rebar tying robot.

[0184] Furthermore, it also includes:

[0185] The intersection region determination module is used to back-project the foreground 3D point cloud to obtain a foreground image before setting the points in the foreground 3D point cloud that are located within the rebar intersection region as candidate points; identify the rebar intersection points in the foreground image, and set a corresponding rebar intersection region for each rebar intersection point; wherein the center point of the rebar intersection region is the rebar intersection point.

[0186] Furthermore, the process by which the intersection point cloud determination module 302 sets points in the foreground 3D point cloud that are located within the rebar intersection area as candidate points includes: setting the median of the pixel depth in the neighborhood of the rebar intersection point as a depth reference value; and setting points in the foreground 3D point cloud that are located within the rebar intersection area and whose pixel depth differs from the depth reference value by less than a critical value as candidate points.

[0187] Since the embodiments of the system part correspond to the embodiments of the method part, please refer to the description of the embodiments of the method part for the embodiments of the system part, and they will not be repeated here.

[0188] This application also provides a storage medium on which a computer program is stored, which, when executed, can perform the steps provided in the above embodiments. The storage medium may include various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0189] This application also provides an electronic device that may include a memory and a processor. The memory stores a computer program, and when the processor calls the computer program in the memory, it can implement the steps provided in the above embodiments. Of course, the electronic device may also include various network interfaces, power supplies, and other components.

[0190] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from the principles of this application, and these improvements and modifications also fall within the protection scope of this application.

[0191] It should also be noted that, in this specification, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

Claims

1. A control method for a rebar tying robot, characterized in that, include: Extract the foreground 3D point cloud from the original 3D point cloud of the rebar cage; wherein the rebar cage includes at least one layer of rebar mesh, and the foreground 3D point cloud is the 3D point cloud of the rebar mesh layer closest to the rebar binding robot; Points in the foreground 3D point cloud that are located within the area where the steel bars intersect are set as candidate points, and a candidate set containing all the candidate points is constructed. Based on the candidate set, an iterative compliance calculation operation is performed, and the candidate plane with the highest compliance is set as the reference plane; wherein, the compliance calculation operation includes: selecting 3 non-collinear candidate points from the candidate set as current reference points, setting the plane where the 3 current reference points are located as candidate planes, setting candidate points in the candidate set whose distance from the candidate plane is less than a preset value as interior points, and determining the compliance of the candidate plane based on the number and spatial distribution of interior points; The movement of the rebar tying robot is controlled according to the pose relationship between the rebar tying robot and the reference plane; The conformity of candidate planes is determined based on the number and spatial distribution of interior points, including: The ratio of the number of interior points to the number of candidate points in the candidate set is set as the coplanarity value; Project all interior points onto the candidate plane, and perform line detection based on Hough transform on the projected interior points to obtain multiple line segments; Determine the length and angle of each line segment, and perform weighted statistics on the angle of each line segment with the length as the weight to obtain the significance value of each angle; The two angles with the highest significance values ​​are set as the first principal direction and the second principal direction, and the sine value of the angle between the first principal direction and the second principal direction is set as the orthogonality value. Projection points are obtained by projecting all interior points in a preset direction, and the coverage is determined based on the two-dimensional convex hull area corresponding to all projection points; wherein, the original three-dimensional point cloud is generated based on the image acquired by the camera, and the preset direction is the principal optical axis direction of the camera; The conformity of the candidate plane is calculated based on the coplanarity value, the orthogonality value, and the coverage rate; This process includes, before extracting the foreground 3D point cloud from the original 3D point cloud of the steel cage, the following steps: Acquire RGB and depth images of the steel cage, and generate the original 3D point cloud based on the RGB and depth images; Accordingly, determining the coverage rate based on the two-dimensional convex hull area corresponding to all projection points includes: The ratio of the area of ​​the two-dimensional convex hull corresponding to all projection points to the total pixel area of ​​the RGB image is set as the coverage rate; Before setting points within the rebar intersection area in the foreground 3D point cloud as candidate points, the method further includes: The foreground image is obtained by back-projecting the foreground 3D point cloud; Identify the rebar intersections in the foreground image, and set a corresponding rebar intersection region for each rebar intersection; wherein the center point of the rebar intersection region is the rebar intersection.

2. The control method for the rebar tying robot according to claim 1, characterized in that, Controlling the movement of the rebar tying robot based on its pose relationship with the reference plane includes: Obtain the three-dimensional coordinates of the body reference point and the body orientation vector of the rebar tying robot; The distance between the fuselage reference point and the reference plane is set as the distance deviation value; The projection of the normal vector of the reference plane onto the horizontal plane is set as the first vector; The projection of the fuselage orientation vector onto the horizontal plane is set as the second vector; Set the angle between the first vector and the second vector as the angle deviation value; The movement of the rebar tying robot is controlled based on the distance deviation value and the angle deviation value.

3. The control method for the rebar tying robot according to claim 2, characterized in that, Controlling the movement of the rebar tying robot based on the distance deviation value and the angle deviation value includes: The target linear velocity is determined based on the distance deviation value, and the target angular velocity is determined based on the angle deviation value. Motion commands are generated based on the target linear velocity and the target angular velocity, and then sent to the motion controller of the rebar tying robot to control the movement of the rebar tying robot.

4. The control method for the rebar tying robot according to claim 1, characterized in that, Points within the rebar intersection area of ​​the foreground 3D point cloud are set as candidate points, including: Set the median pixel depth within the neighborhood of the intersection of the reinforcing bars as the depth reference value; Points in the foreground 3D point cloud that are located within the area where the steel bars intersect and whose pixel depth differs from the depth reference value by less than a critical value are set as candidate points.

5. A control system for a rebar tying robot, characterized in that, include: A foreground point cloud extraction module is used to extract the foreground 3D point cloud from the original 3D point cloud of the rebar cage; wherein the rebar cage includes at least one layer of rebar mesh, and the foreground 3D point cloud is the 3D point cloud of the rebar mesh layer closest to the rebar binding robot; The intersection point cloud determination module is used to set points in the foreground 3D point cloud that are located within the steel bar intersection area as candidate points, and to construct a candidate set containing all the candidate points; The reference plane determination module is used to iteratively perform a conformity calculation operation based on the candidate set, and set the candidate plane with the highest conformity as the reference plane; wherein, the conformity calculation operation includes: selecting 3 non-collinear candidate points from the candidate set as current reference points, setting the plane where the 3 current reference points are located as candidate planes, setting candidate points in the candidate set whose distance from the candidate plane is less than a preset value as interior points, and determining the conformity of the candidate plane based on the number and spatial distribution of interior points; A motion control module is used to control the movement of the rebar tying robot according to the pose relationship between the rebar tying robot and the reference plane; The process by which the reference plane determination module determines the conformity of candidate planes based on the number and spatial distribution of interior points includes: The ratio of the number of interior points to the number of candidate points in the candidate set is set as the coplanarity value; Project all interior points onto the candidate plane, and perform line detection based on Hough transform on the projected interior points to obtain multiple line segments; Determine the length and angle of each line segment, and perform weighted statistics on the angle of each line segment with the length as the weight to obtain the significance value of each angle; The two angles with the highest significance values ​​are set as the first principal direction and the second principal direction, and the sine value of the angle between the first principal direction and the second principal direction is set as the orthogonality value. Projection points are obtained by projecting all interior points in a preset direction, and the coverage is determined based on the two-dimensional convex hull area corresponding to all projection points; wherein, the original three-dimensional point cloud is generated based on the image acquired by the camera, and the preset direction is the principal optical axis direction of the camera; The conformity of the candidate plane is calculated based on the coplanarity value, the orthogonality value, and the coverage rate; The control system of the rebar tying robot also includes: An image acquisition module is used to acquire RGB images and depth images of the steel cage before extracting the foreground 3D point cloud from the original 3D point cloud of the steel cage, and to generate the original 3D point cloud based on the RGB images and the depth images. Accordingly, the process of the reference plane determination module determining the coverage rate based on the two-dimensional convex hull area corresponding to all projection points includes: setting the ratio of the two-dimensional convex hull area corresponding to all projection points to the total pixel area of ​​the RGB image as the coverage rate; The intersection region determination module is used to back-project the foreground 3D point cloud to obtain a foreground image before setting the points in the foreground 3D point cloud that are located within the rebar intersection region as candidate points; identify the rebar intersection points in the foreground image, and set a corresponding rebar intersection region for each rebar intersection point; wherein the center point of the rebar intersection region is the rebar intersection point.

6. An electronic device, characterized in that, It includes a memory and a processor, wherein the memory stores a computer program, and the processor, when calling the computer program in the memory, implements the steps of the control method for the rebar tying robot as described in any one of claims 1 to 4.

7. A storage medium, characterized in that, The storage medium stores computer-executable instructions, which, when loaded and executed by a processor, implement the steps of the control method for the rebar tying robot as described in any one of claims 1 to 4.

Images