Autonomous navigation method and system for a steel bar tying robot
Through the combination of depth camera and lidar, static map and point cloud alignment technology are built, which solves the problem of low navigation accuracy of steel bar binding robots in complex environments, and accurately recognizes and navigation of steel bar cages, improving automation level and safety sex.
Patent Information
- Application Number
- CN202411254184.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-09
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2044-09-09
AI Technical Summary
The existing reinforced bar tying robots have low navigation accuracy in complex three-dimensional spaces, making it difficult to accurately identify the location of the reinforced bar cage. They are greatly affected by shading and dynamic changes in the environment, resulting in inaccurate navigation.
Using an autonomous navigation method combined with depth camera and lidar, by constructing a static map, extracting point cloud data, iterating the nearest point algorithm alignment, the coordinate information of the steel cage is calculated, and path planning is carried out to control the robot motion.
It improves the navigation accuracy and operation safety of the steel bar binding robot, can accurately identify the location of the steel bar cage in complex factory environments, and enhances the automation level and operation efficiency.
Smart Images

Figure CN118752498B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of building automation technology, and in particular to an autonomous navigation method and system for a steel bar tying robot. Background Art
[0002] In a steel cage prefabrication plant, steel bar tying is a critical and time-consuming task. Currently, the cutting and arrangement of steel bars can be automated, but the tying of steel bars still needs to be done manually, which is not only inefficient but also poses safety risks. With the development of automation technology, the application of steel bar tying robots has become an important means to improve factory operation efficiency and safety.
[0003] At present, the research and development of rebar tying robots is focusing on overcoming the limitations of existing two-dimensional navigation systems. Although two-dimensional systems have shown preliminary navigation capabilities in some applications, their application in complex three-dimensional space is limited, especially because it is difficult to effectively capture all the features of an object and is easily affected by occlusion and dynamic changes in the environment, making it difficult to achieve the required navigation accuracy. The development of a rebar tying robot that can operate accurately in three-dimensional space is of great significance to improving the level of automation and operational safety in the construction industry.
[0004] Therefore, how to accurately identify the location of the steel cage and improve the navigation accuracy of the steel bar tying robot is a technical problem that technical personnel in this field currently need to solve. Summary of the invention
[0005] The purpose of this application is to provide an autonomous navigation method and system for a steel bar tying robot, which can accurately identify the location of the steel cage and improve the navigation accuracy of the steel bar tying robot.
[0006] In order to solve the above technical problems, the present application provides an autonomous navigation method of a steel bar tying robot, wherein the steel bar tying robot is provided with a depth camera and a laser radar, and the autonomous navigation method of the steel bar tying robot comprises:
[0007] Constructing a static map using the three-dimensional depth data collected by the depth camera and the first point cloud data collected by the laser radar;
[0008] Extracting first sub-point cloud data of the steel cage from the first point cloud data;
[0009] Restore the depth image and the color image taken by the depth camera to obtain second point cloud data, and extract second sub-point cloud data of the steel cage from the second point cloud data;
[0010] Aligning the first sub-point cloud data and the second sub-point cloud data by an iterative closest point algorithm to obtain aligned target point cloud data;
[0011] Calculating the coordinate information of the steel cage according to the static map and the target point cloud data;
[0012] A navigation path is obtained by performing path planning according to the coordinate information of the steel cage, and the movement of the steel bar tying robot is controlled according to the navigation path.
[0013] Optionally, extracting first sub-point cloud data of the steel cage from the first point cloud data includes:
[0014] Filtering the first point cloud data using a voxel filter;
[0015] Removing outliers from the filtered first point cloud data;
[0016] Using a plane segmentation algorithm to segment the first point cloud data after removing outliers, a plurality of point cloud regions are obtained;
[0017] The point cloud area is clustered, and the clustering result is identified using a first deep learning model to obtain first sub-point cloud data of the steel cage.
[0018] Optionally, restoring the second point cloud data according to the depth image and the color image taken by the depth camera, and extracting the second sub-point cloud data of the steel cage from the second point cloud data, comprises:
[0019] Preprocessing the depth image captured by the depth camera using the cross-platform computer vision library OpenCV;
[0020] Restoring the preprocessed depth image and the color image based on camera parameters to obtain the second point cloud data;
[0021] A second deep learning model is used to extract second sub-point cloud data of the steel cage from the second point cloud data.
[0022] Optionally, the depth image captured by the depth camera is preprocessed using the cross-platform computer vision library OpenCV, including:
[0023] The cross-platform computer vision library OpenCV is used to perform noise processing and edge enhancement operations on the depth image taken by the depth camera.
[0024] Optionally, performing path planning according to the coordinate information of the steel cage to obtain a navigation path includes:
[0025] Determining the binding position according to the coordinate information of the steel cage;
[0026] Path planning is performed according to the current position of the steel bar tying robot and the tying position to obtain the navigation path.
[0027] Optionally, controlling the movement of the steel bar tying robot according to the navigation path includes:
[0028] Using the depth camera and the laser radar to detect whether there are obstacles on the navigation path;
[0029] If so, updating the navigation path according to the position of the obstacle so as to control the movement of the steel bar tying robot according to the updated navigation path;
[0030] If not, the steel bar tying robot is controlled to move according to the navigation path.
[0031] Optionally, constructing a static map using the three-dimensional depth data collected by the depth camera and the first point cloud data collected by the laser radar includes:
[0032] Using the depth camera to collect three-dimensional depth data of the working area;
[0033] Using the laser radar to collect first point cloud data of the working area;
[0034] The three-dimensional depth data and the first point cloud data are processed using a preset algorithm to obtain the two-dimensional static map; wherein the preset algorithm is a simultaneous positioning and map construction algorithm based on laser scanning data.
[0035] The present application also provides an autonomous navigation system for a steel bar tying robot, wherein the steel bar tying robot is provided with a depth camera and a laser radar, and the autonomous navigation system for the steel bar tying robot comprises:
[0036] A map construction module, used to construct a static map using the three-dimensional depth data collected by the depth camera and the first point cloud data collected by the laser radar;
[0037] A point cloud extraction module, used to extract first sub-point cloud data of the steel cage from the first point cloud data; and also used to restore the second point cloud data according to the depth image and the color image taken by the depth camera, and extract the second sub-point cloud data of the steel cage from the second point cloud data;
[0038] A point cloud alignment module, used for aligning the first sub-point cloud data and the second sub-point cloud data by an iterative closest point algorithm to obtain aligned target point cloud data;
[0039] A coordinate determination module, used to calculate the coordinate information of the steel cage according to the static map and the target point cloud data;
[0040] The navigation module is used to perform path planning according to the coordinate information of the steel cage to obtain a navigation path, and control the movement of the steel bar tying robot according to the navigation path.
[0041] Optionally, the point cloud extraction module includes:
[0042] A filtering unit, configured to filter the first point cloud data using a voxel filter;
[0043] An outlier removal unit, used to remove outliers in the filtered first point cloud data;
[0044] A point cloud segmentation unit, used to segment the first point cloud data after removing outliers using a plane segmentation algorithm to obtain a plurality of point cloud regions;
[0045] A clustering unit is used to cluster the point cloud area and use a first deep learning model to identify the clustering results to obtain the first sub-point cloud data of the steel cage.
[0046] Optionally, the point cloud extraction module includes:
[0047] A preprocessing unit, used to preprocess the depth image taken by the depth camera using the cross-platform computer vision library OpenCV;
[0048] A restoration unit, configured to restore the preprocessed depth image and the color image based on camera parameters to obtain the second point cloud data;
[0049] A data extraction unit is used to extract second sub-point cloud data of the steel cage from the second point cloud data using a second deep learning model.
[0050] The present application provides an autonomous navigation method for a steel bar tying robot, wherein the steel bar tying robot is provided with a depth camera and a laser radar, and the autonomous navigation method for the steel bar tying robot comprises: constructing a static map using three-dimensional depth data collected by the depth camera and first point cloud data collected by the laser radar; extracting first sub-point cloud data of a steel cage from the first point cloud data; restoring second point cloud data according to a depth image and a color image taken by the depth camera, and extracting second sub-point cloud data of the steel cage from the second point cloud data; aligning the first sub-point cloud data and the second sub-point cloud data by an iterative closest point algorithm to obtain aligned target point cloud data; calculating coordinate information of the steel cage according to the static map and the target point cloud data; performing path planning according to the coordinate information of the steel cage to obtain a navigation path, and controlling the movement of the steel bar tying robot according to the navigation path.
[0051] The solution provided by the present application is implemented based on a steel bar binding robot equipped with a depth camera and a laser radar, and then a static map is constructed based on the three-dimensional depth data collected by the depth camera and the first point cloud data collected by the laser radar, so as to provide basic environmental information for subsequent navigation. The present application extracts the first sub-point cloud data of the steel cage from the first point cloud data, and also restores the depth image and color image taken by the depth camera to obtain the second point cloud data, and extracts the second sub-point cloud data of the steel cage from the second point cloud data. The point cloud information corresponding to the steel cage extracted based on the depth camera and the laser radar can be obtained by the above method. The present application aligns the first sub-point cloud data and the second sub-point cloud data, thereby ensuring the consistency between different data sources, thereby improving the accuracy of the steel cage position identification. The present application can calculate the coordinate information of the steel cage according to the static map and the aligned target point cloud data, so as to navigate according to the coordinate information of the steel cage. The depth camera can provide rich texture and color information, and the laser radar can provide accurate distance and angle measurement. The present application integrates the information collected by the depth camera and the laser radar through point cloud alignment, which can accurately identify the location of the steel cage and improve the navigation accuracy of the steel bar binding robot. The present application also provides an autonomous navigation system for a steel bar tying robot, which has the above-mentioned beneficial effects and will not be described in detail here. BRIEF DESCRIPTION OF THE DRAWINGS
[0052] In order to more clearly illustrate the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0053] Figure 1 A flowchart of an autonomous navigation method for a steel bar tying robot provided in an embodiment of the present application;
[0054] Figure 2 A schematic diagram of the appearance of an autonomous navigation system for a steel bar tying robot provided in an embodiment of the present application;
[0055] Figure 3 A flowchart of a rebar tying robot navigation method based on a three-dimensional laser radar and a depth camera provided in an embodiment of the present application;
[0056] Figure 4 A schematic diagram of a static map provided in an embodiment of the present application;
[0057] Figure 5 A schematic diagram of target feature recognition provided by an embodiment of the present application;
[0058] Figure 6A schematic diagram of point cloud alignment provided in an embodiment of the present application;
[0059] Figure 7 A schematic diagram of obtaining the coordinate information of a reinforcement cage provided in an embodiment of the present application;
[0060] Figure 8 A schematic diagram of automatic navigation to a lashing position provided in an embodiment of the present application;
[0061] Fig. 9 A schematic diagram of the structure of an autonomous navigation system for a steel bar tying robot provided in an embodiment of the present application. DETAILED DESCRIPTION
[0062] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0063] See below Figure 1 , Figure 1 A flow chart of an autonomous navigation method for a steel bar tying robot provided in an embodiment of the present application.
[0064] Specific steps may include:
[0065] S101: constructing a static map using the three-dimensional depth data collected by the depth camera and the first point cloud data collected by the laser radar;
[0066] Among them, this embodiment can be applied to a control device connected to a steel bar tying robot, and the above steel bar tying robot is provided with a depth camera and a laser radar. This embodiment can also be applied to an embedded artificial intelligence computer carried by the steel bar tying robot, see Figure 2 , Figure 2 A schematic diagram of the appearance of an autonomous navigation system of a rebar tying robot provided in an embodiment of the present application. The rebar tying robot includes a laser radar, a depth camera and an embedded artificial intelligence computer. The laser radar and the depth camera are respectively connected to the embedded artificial intelligence computer.
[0067] The above-mentioned depth camera is used to collect the color data and three-dimensional depth data of the steel cage. A color image (RGB color image) can be obtained according to the color data, and a depth image can be obtained according to the three-dimensional depth data. The above-mentioned laser radar can emit a laser beam and measure the time it takes for it to reflect back. The point cloud data obtained is the first point cloud data.
[0068] Before this step, a depth camera can be used to capture the depth information of the surrounding environment to form three-dimensional depth data; a laser radar can also be used to emit a laser beam and receive its reflected signal to measure the distance and angle to the surrounding objects to obtain the first point cloud data. Both the three-dimensional depth data and the first point cloud data contain the location information of each point in the scene. The three-dimensional depth data and the first point cloud data can be fused to build a static map based on the fused data.
[0069] As a feasible implementation, the depth camera can be used to collect three-dimensional depth data of the working area; the laser radar can also be used to collect the first point cloud data of the working area; the three-dimensional depth data and the first point cloud data are processed using a preset algorithm Gmapping (a simultaneous positioning and mapping algorithm based on laser scanning data) to obtain the two-dimensional static map. Simultaneous Localization and Mapping (SLAM) allows the robot to perform positioning and mapping simultaneously in an unknown environment.
[0070] S102: extracting first sub-point cloud data of the steel cage from the first point cloud data;
[0071] Among them, this step can perform filtering, outlier removal, point cloud segmentation, clustering and other operations on the first point cloud data in sequence, and input the clustering results into the first deep learning model to obtain the first sub-point cloud data of the steel cage.
[0072] Specifically, in this embodiment, the first sub-point cloud data of the steel cage can be extracted in the following manner:
[0073] Step A1: filtering the first point cloud data using a voxel grid filter;
[0074] Step A2, removing outliers in the filtered first point cloud data;
[0075] The first point cloud data after filtering may contain some outliers due to scanning errors, environmental noise or other reasons. In order to avoid these outliers from interfering with the subsequent segmentation and recognition steps, the present application can detect the outliers in the first point cloud data after filtering and remove the outliers.
[0076] Step A3: using a plane segmentation algorithm to segment the first point cloud data after removing outliers to obtain a plurality of point cloud regions;
[0077] In the LiDAR scanning environment, there are usually many planar structures (such as the ground, walls, etc.), and the plane segmentation algorithm is used to divide the point cloud data into multiple point cloud regions.
[0078] Step A4: clustering the point cloud area, and using the first deep learning model to identify the clustering results to obtain the first sub-point cloud data of the steel cage.
[0079] This embodiment can cluster each point cloud area so as to cluster similar point cloud areas into a larger category. After clustering is completed, this embodiment can use a deep learning model to identify the clustering results to determine which categories belong to the steel cage. The first deep learning model can extract point cloud features from the clustering results, output the category of each clustering result according to the point cloud features, and then set the clustering result of the category of the steel cage as the first sub-point cloud data of the steel cage.
[0080] S103: restoring the second point cloud data according to the depth image and the color image taken by the depth camera, and extracting the second sub-point cloud data of the steel cage from the second point cloud data;
[0081] Among them, the depth camera can simultaneously obtain the depth image and color image of the scene. The depth image contains the distance information of each point in the scene to the camera, while the color image provides the color and texture information of the scene. By combining the data of the depth image and the color image, three-dimensional point cloud data (i.e., the second sub-point cloud data) can be generated.
[0082] In the second point cloud data, the steel cage is usually represented by a series of discrete point sets with specific shapes and sizes. According to the point cloud features in the second point cloud data, second sub-point cloud data of the steel cage can be extracted from the second point cloud data.
[0083] In this embodiment, a second deep learning model can be pre-trained, and the second deep learning model can learn the point cloud features of the steel cage and perform classification. Specifically, in this embodiment, the second deep learning model can be used to extract point cloud features from the second point cloud data, and the second sub-point cloud data of the steel cage can be extracted from the second point cloud data according to the point cloud features.
[0084] S104: aligning the first sub-point cloud data and the second sub-point cloud data by an iterative closest point algorithm to obtain aligned target point cloud data;
[0085] Among them, the first sub-point cloud data and the second sub-point cloud data may be in different spatial positions and orientations. In this step, the first sub-point cloud data and the second sub-point cloud data can be aligned by an iterative closest point (ICP) algorithm, so as to control the first sub-point cloud data to be aligned with the second sub-point cloud data, or control the second sub-point cloud data to be aligned with the first sub-point cloud data.
[0086] Specifically, this embodiment can find the closest point pair between the first sub-point cloud data and the second sub-point cloud data, use the found closest point pair to calculate an optimal transformation matrix, so that the two sub-point cloud data are aligned as much as possible after the transformation, and apply the calculated transformation matrix to the second sub-point cloud data to align it with the first sub-point cloud data. The aligned target point cloud data can be obtained by iteratively performing the above operations. The aligned target point cloud data includes information collected by both the depth camera and the lidar.
[0087] S105: Calculating the coordinate information of the steel cage according to the static map and the target point cloud data;
[0088] Among them, this embodiment can align the target point cloud data with the static map, identify the point cloud cluster of the steel cage from the target point cloud data through point cloud segmentation and feature extraction technology, calculate the center of mass or main feature points of the point cloud cluster as the preliminary coordinates of the steel cage, and verify and adjust the preliminary coordinates in combination with the known information in the static map (such as the expected position of the steel cage or environmental characteristics) to obtain the coordinate information of the steel cage.
[0089] As a feasible implementation mode, the above-mentioned target point cloud data may be data obtained after aligning the second sub-point cloud data with the first sub-point cloud data. This embodiment may calculate the position of the target point cloud data in the static map as the coordinate information of the steel cage detected by the laser radar. This embodiment may also determine the coordinate information of the steel cage detected by the depth camera and the inertial measurement unit IMU (Inertial Measurement Unit). The actual coordinate information of the steel cage is obtained by performing weighted calculation on the coordinate information of the steel cage detected by the laser radar, the depth camera and the inertial measurement unit IMU respectively.
[0090] S106: Perform path planning according to the coordinate information of the steel cage to obtain a navigation path, and control the movement of the steel bar tying robot according to the navigation path.
[0091] Among them, this embodiment can perform path planning based on the current position of the steel bar tying robot and the coordinate information of the steel bar cage to obtain a navigation path, so as to control the movement of the steel bar tying robot based on the navigation path.
[0092] The solution provided in this embodiment is implemented based on a steel bar binding robot equipped with a depth camera and a laser radar, and then a static map is constructed based on the three-dimensional depth data collected by the depth camera and the first point cloud data collected by the laser radar, so as to provide basic environmental information for subsequent navigation. In this embodiment, the first sub-point cloud data of the steel cage is extracted from the first point cloud data, and the depth image and color image taken by the depth camera are restored to obtain the second point cloud data, and the second sub-point cloud data of the steel cage is extracted from the second point cloud data. Through the above method, the point cloud information corresponding to the steel cage extracted based on the depth camera and the laser radar can be obtained. In this embodiment, the first sub-point cloud data and the second sub-point cloud data are aligned, thereby ensuring the consistency between different data sources, thereby improving the accuracy of the steel cage position identification. In this embodiment, the coordinate information of the steel cage can be calculated according to the static map and the aligned target point cloud data, so as to navigate according to the coordinate information of the steel cage. The depth camera can provide rich texture and color information, and the laser radar can provide accurate distance and angle measurement. In this embodiment, the information collected by the depth camera and the laser radar is integrated through point cloud alignment, so as to accurately identify the location of the steel cage and improve the navigation accuracy of the steel bar binding robot.
[0093] As for Figure 1 Further introduction to the corresponding embodiment, the second point cloud data can be obtained and the second sub-point cloud data of the steel cage can be determined in the following manner: using the cross-platform computer vision library OpenCV (also known as the cross-platform computer vision and machine learning software library) to preprocess the depth image taken by the depth camera; based on the camera parameters, the preprocessed depth image and the color image are restored to obtain the second point cloud data; using the second deep learning model to extract the second sub-point cloud data of the steel cage from the second point cloud data.
[0094] Specifically, in the above process, the preprocessing of the depth image captured by the depth camera using the cross-platform computer vision library OpenCV may include noise processing and edge enhancement operations.
[0095] This embodiment can align the preprocessed depth image with the color image based on the intrinsic and extrinsic parameters of the camera to restore the point cloud data in the three-dimensional space, that is, the second point cloud data. This embodiment can also use the trained second deep learning model to extract the feature point cloud of the steel cage from the second point cloud data, that is, the second sub-point cloud data. In this way, accurate information can be provided for the position recognition and path planning of the steel cage.
[0096] As for Figure 1For further introduction to the corresponding embodiment, the navigation path can be obtained by planning in the following manner: determining the binding position according to the coordinate information of the steel cage; performing path planning according to the current position of the steel binding robot and the binding position to obtain the navigation path.
[0097] This embodiment can determine the specific binding position according to the coordinate information of the steel cage. Based on the current position of the steel tying robot and the required binding position, path planning is performed to obtain a navigation path. In the process of controlling the movement of the steel tying robot according to the navigation path, the depth camera and the laser radar can be used to detect whether there are obstacles in the navigation path; if so, the navigation path is updated according to the position of the obstacle, so as to control the movement of the steel tying robot according to the updated navigation path; if not, the movement of the steel tying robot is controlled according to the navigation path.
[0098] The process described in the above embodiment is explained below through an embodiment in actual application.
[0099] In the related art, there are several conventional navigation methods for steel bar tying robots:
[0100] Method 1: Use a 2D vision system combined with a 2D laser radar for navigation. This method uses a 2D vision system to identify the sign or shape of the steel cage, and uses the laser radar for ranging and obstacle avoidance. However, in the above method, the 2D vision system has difficulty processing complex 3D information, especially when the steel cage is stacked or partially blocked, its recognition and positioning capabilities are greatly reduced.
[0101] Method 2: Use RFID (Radio Frequency Identification) technology for navigation. This method installs RFID tags on the steel cage and uses RFID readers for positioning. However, the above method depends on the installation position and angle of the RFID tag and the coverage range of the RFID signal, which is easily affected by environmental factors and has limited positioning accuracy.
[0102] Method 3: Navigation based on a single depth camera or 3D LiDAR. This method uses a single depth camera or 3D LiDAR for spatial scanning and object recognition. However, the above methods have shortcomings in data processing speed, occlusion processing, and cost-effectiveness. In particular, a single technology is difficult to simultaneously meet the requirements of high-precision positioning and environmental adaptability.
[0103] In a changing factory environment, traditional navigation systems cannot effectively capture the features of objects in a complex three-dimensional space, especially in the presence of occlusion and dynamic changes in the environment. It is difficult to accurately identify and locate the position of the steel cage to be tied. In addition, the movement of workers and the placement of equipment in the factory may change frequently, which requires the tying robot to have a high degree of adaptability and real-time response capabilities. It can be seen that in order to achieve accurate automatic tying, it is necessary to achieve accurate positioning and navigation of the steel tying robot in the factory environment.
[0104] In order to solve the technical problems existing in the above-mentioned related technologies, this embodiment provides a self-propelled rebar tying robot navigation system and solution based on multi-source data fusion of three-dimensional laser radar and depth camera. The solution uses a depth camera to perform high-precision three-dimensional scanning of the surrounding environment and uses image processing technology to identify the characteristics of the rebar cage to be tied. At the same time, the laser radar provides point cloud data of the environment, and the two data are combined through advanced data fusion algorithms to achieve accurate three-dimensional positioning of the rebar cage. In the complex and ever-changing factory environment, the rebar tying robot can also accurately move to the designated rebar cage position to operate.
[0105] This embodiment discloses a navigation system and method for automated rebar tying in a rebar cage prefabrication factory. The system achieves precise positioning and navigation of the rebar tying robot in a complex factory environment through data fusion of a depth camera and a lidar, thereby improving the efficiency of rebar tying and the safety of operations. The present invention includes key technologies such as target feature recognition, point cloud data processing, automatic navigation, and real-time environment modeling. Data fusion refers to the process of merging data from multiple sources into a unified representation. In a navigation system, data fusion typically involves combining data from sensors such as lidar, depth cameras, and IMUs to gain a comprehensive understanding of the environment.
[0106] The navigation system of the steel bar tying robot includes a depth camera, a laser radar and an embedded artificial intelligence computer. The depth camera is used to capture the three-dimensional image of the environment and obtain the depth information such as the shape, size and relative position of the object; the laser radar obtains the point cloud data of the environment by emitting a laser beam and measuring the time it takes for it to reflect back, which is used to build a three-dimensional model of the environment. The embedded artificial intelligence computer includes a data processing unit and a navigation and control unit. The data processing unit is used to process the data collected by the depth camera and the laser radar, perform data fusion, and realize environmental perception and target positioning. The navigation and control unit is used to perform path planning, dynamic obstacle avoidance, and control the movement of the robot based on the processed data.
[0107] The rebar tying robot navigation system of this embodiment is based on the fusion technology of three-dimensional laser radar and depth camera, which overcomes the defects of the existing methods by combining the advantages of the two sensors. Through the high-precision three-dimensional visual information provided by the depth camera and the spatial point cloud data of the laser radar, accurate three-dimensional positioning and navigation are achieved using the data fusion algorithm, which can theoretically effectively improve the operation accuracy and efficiency in the complex environment of the rebar prefabrication plant. The main advantage of the fusion technology used in this embodiment is that it can fully utilize the advantages of both technologies, namely the high-resolution three-dimensional image of the depth camera and the large-range spatial scanning capability of the laser radar, so as to achieve more accurate environmental perception and positioning. Compared with traditional methods, it has obvious improvements in handling occlusion, adaptability to dynamic environments, and positioning accuracy.
[0108] This embodiment provides a method for a steel bar tying robot to determine the coordinate position of a steel bar cage through a three-dimensional point cloud alignment method. Because current SLAM technologies all achieve navigation through two-dimensional maps of laser point clouds, this embodiment uses a depth camera to obtain the color, texture, and richer geometric features of the steel bar cage (which laser point clouds cannot do), and can more accurately identify the target steel bar cage. Then the point cloud obtained by the depth camera is segmented and aligned to the laser point cloud, thereby obtaining the coordinate point information on the map, and performing subsequent navigation and obstacle avoidance operations. Current steel bar tying robots almost all walk and tie on the surface of the steel bar cage laid on the ground. The steel bar tying robot provided in this embodiment can identify the steel bar cage to be tied, and walk to the target to perform subsequent tying operations, and can achieve obstacle avoidance and other actions on the way.
[0109] See also Figure 3 , Figure 3 A flowchart of a rebar tying robot navigation method based on a three-dimensional laser radar and a depth camera provided in an embodiment of the present application includes the following steps:
[0110] S301: Build a static map.
[0111] See also Figure 4 , Figure 4 A schematic diagram of a static map provided in an embodiment of the present application can be used to construct a static map through laser radar scanning, point cloud acquisition and map generation operations. The principle of constructing a static map is to use a laser radar (LIDAR, Light Laser Detection and Ranging) to perform high-frequency rotation scanning to capture point cloud data of the surrounding environment. The point cloud data represents the three-dimensional coordinates of the surface of an object.
[0112] When the steel bar binding robot is started, the depth camera and laser radar simultaneously perform a comprehensive scan of the working environment. The depth camera is responsible for capturing the three-dimensional image of the environment and providing the depth information of the object. The depth camera measures the distance of the object by emitting infrared light and capturing the time difference of its reflection (Time of Flight, ToF principle), thereby obtaining three-dimensional depth information; the laser radar generates point cloud data of the environment and records the spatial position of the object. It obtains point cloud data, that is, three-dimensional coordinate points on the surface of the object, for environmental modeling by emitting laser beams and measuring their reflection time. Finally, the Gmapping algorithm is used to construct a two-dimensional static map of the environment. The depth camera uses a model with a resolution of 640x480 pixels and a frame rate of 30 frames per second, a depth range of 0.5-5 meters, a field of view of 70 degrees horizontally and 50 degrees vertically to ensure sufficient scanning coverage and accuracy. The laser radar configuration includes a scanning frequency of 10Hz, a detection range of 30m, a resolution of more than 0.5 degrees, and a distance measurement accuracy of ±2 cm, and a response time of less than 50 milliseconds, to ensure fast and accurate capture of environmental data. In this embodiment, the three-dimensional point cloud data of the environment can be collected in real time through the laser radar driver. This embodiment can also use LOAM (Lidar Odometry and Mapping in Real-time) to process point cloud data, generate and update three-dimensional maps in real time, and estimate the robot's own position.
[0113] After the static map is constructed, a local area cruise search may be performed and the process proceeds to step S302 .
[0114] S302: Target feature recognition.
[0115] See also Figure 5 , Figure 5 A schematic diagram of target feature recognition provided in an embodiment of the present application, wherein target feature recognition includes steel cage recognition based on a deep image data source and steel cage recognition based on a laser point cloud data source. The steel cage recognition process based on a deep image data source can be implemented based on a depth camera, deep image processing, and an image segmentation model (i.e., a second deep learning model). The steel cage recognition process based on a laser point cloud data source can be implemented based on a laser radar, laser point cloud processing, and a point cloud segmentation model (i.e., a first deep learning model).
[0116] Specifically, the steel cage recognition process based on the depth image data source is as follows: The image obtained by the depth camera contains depth information such as the size, shape and position of the object. The depth image data is preprocessed using the OpenCV library for denoising and edge enhancement. The depth map and color map are restored into point cloud data, and finally the deep learning model MASK R-CNN (i.e., the second deep learning model) is used to identify and segment the point cloud of the steel cage. This embodiment can also identify the steel cage through the YOLO algorithm. The YOLO (You Only Look Once) algorithm is a target detection algorithm based on deep learning.
[0117] Specifically, the process of steel cage recognition based on laser point cloud data source is as follows: laser point cloud information of objects in the environment is obtained by laser radar, and voxel grid filter is used to reduce the data volume while retaining structural information. Statistical analysis denoising method is used to remove outliers, and plane segmentation algorithm (such as random sample consensus algorithm RANSAC, Random Sample Consensus) is used to separate the ground and other large plane objects from the point cloud, so as to more clearly identify steel cages and other targets. Clustering algorithm (DBSCAN, Density-Based Spatial Clustering of Applications with Noise) is used to cluster the point cloud to identify and separate independent objects such as steel cages, and finally a convolutional neural network model CNN or a point transformer model Point Transformer model (i.e., the first deep learning model) is used to identify and instance segment the point cloud of the steel cage.
[0118] S303: Aligning the laser point cloud with the point cloud of the depth map.
[0119] See also Figure 6 , Figure 6 A schematic diagram of point cloud alignment provided in an embodiment of the present application. The point cloud alignment process of the laser point cloud and the depth map includes operations such as point cloud conversion, point cloud registration and spatial alignment. The principle of point cloud alignment is: segment the target steel cage from the depth map, convert each pixel point in the segmented steel cage image area into a point in the point cloud, and thus obtain a sub-point cloud representing the steel cage. Point cloud alignment matches these targets with the three-dimensional point cloud data.
[0120] Since the sub-point cloud representing the steel cage is obtained in S302, the present embodiment can apply the Iterative Closest Point (ICP) algorithm to perform point cloud registration on the depth map point cloud and the laser point cloud, thereby achieving spatial alignment in the point cloud map. The specific implementation of the algorithm is: through an iterative process, find the best correspondence between the two sets of point cloud data, and determine a set of rigid body transformations (including rotation and translation) so that one set of point clouds (source point clouds) can be optimally aligned to another set of point clouds (target point clouds).
[0121] The Iterative Closest Point algorithm is an algorithm widely used in the field of 3D graphics and robotics for geometric alignment between two sets of point clouds. The ICP algorithm uses an iterative process to find the best rigid body transformation (including rotation and translation) so that one set of point clouds can be optimally aligned to another set of point clouds. The basic idea of the ICP algorithm is to minimize the distance between the point pairs between the two sets of point clouds. Specifically, for each point in the source point cloud, the algorithm finds the nearest point in the target point cloud, and then calculates an optimal rigid body transformation (rotation and translation) so that the distance (i.e., error) between the source point cloud and the target point cloud is minimized under this transformation. Algorithm steps: A) Initialization: Select or estimate a preliminary rigid body transformation as a starting point. If there is no prior information, you can start with a unit transformation (i.e., no rotation and translation). B) Closest point matching: For each point in the source point cloud, find the closest point in the target point cloud. This step is done by calculating the Euclidean distance. C) Calculate the best rigid body transformation: Based on the current point pair matching, use the least squares method to calculate the best rotation matrix and translation vector. This usually involves solving a linear or nonlinear system of equations. D) Apply transformation and iterate: Apply the calculated optimal rigid body transformation to the source point cloud, and then repeat steps b and c until a termination condition is met (such as the transformation update amount is less than a threshold, or the preset maximum number of iterations is reached).
[0122] S304: Obtain the coordinate information of the reinforcement cage.
[0123] See also Figure 7 , Figure 7 A schematic diagram of obtaining the coordinate information of a steel cage provided in an embodiment of the present application, wherein the process of obtaining the coordinate information of the steel cage includes data fusion and coordinate transformation. Data fusion refers to: combining point cloud data and segmentation results, and using a fusion algorithm (such as a Kalman filter or a particle filter) to estimate the precise position of the steel cage. Coordinate transformation refers to: converting the local coordinates of the steel cage into coordinates in the global map coordinate system.
[0124] This step can calculate the precise position and direction of the steel cage in the construction site map based on the aligned point cloud data and static map data, and can also calculate the precise position and direction of the steel cage in the construction site map based on the aligned point cloud data and map.
[0125] Specifically, this embodiment can use the Kalman filter to integrate the position information from different sensors to estimate the most accurate position of the steel cage. Assume that the steel cage position updated by the Kalman filter is , the sensor's attitude transformation matrix is T, then the position of the steel cage in the global coordinates is It can be expressed as: .
[0126] The above formula represents the coordinate conversion method. In this embodiment, a Kalman filter can be used to integrate data from different sensors (depth camera, lidar, IMU) to estimate the precise position of the steel cage. These sensors all provide relevant data about the position of the steel cage. In order to process the data comprehensively and accurately, this embodiment can consider the attitude transformation matrix of each sensor when performing fine spatial position estimation. The attitude data of all relevant sensors can be integrated in the above manner. After processing by the Kalman filter, the contribution of each sensor is calculated using their respective 𝑇, and finally a comprehensive global position estimate is obtained. The above sensors may include depth cameras, lidars, and inertial measurement units (IMUs). represents the position of the sensor in global coordinates. T is calculated based on the orientation of the sensor relative to the global coordinate system. It is the estimated position of the steel cage in the local coordinate system (sensor coordinate system) obtained by Kalman filter processing.
[0127] Specifically, this embodiment can use a Kalman filter to process and fuse sensor data, which specifically includes the following operations:
[0128] The data from each sensor first undergoes necessary preprocessing, such as filtering, noise removal, and data format conversion. The Kalman filter receives these processed data at each measurement time point and fuses and updates them based on the previous state estimate. Based on the last position estimate, the Kalman filter predicts the new position of the steel cage. When new data arrives from the sensor, the Kalman filter compares this new data with the predicted position and updates the position estimate, reducing the prediction error and improving the accuracy of the position estimate.
[0129] The position of the rebar cage in local coordinates means that the position of the rebar cage is estimated relative to the position of the rebar tying robot. This means that the coordinates of the rebar cage are based on the coordinate system of the rebar tying robot, where the current position and orientation of the rebar tying robot define the origin and direction of the coordinate system.
[0130] S305: Automatically navigate to the lashing position.
[0131] See also Figure 8 , Figure 8A schematic diagram of an automatic navigation to a binding position provided in an embodiment of the present application, wherein the process of automatic navigation to a binding position includes path planning and dynamic obstacle avoidance. The path planning process is: based on the coordinate information of the steel cage, the target point information to which the AGV (Automated Guided Vehicle) is transported is obtained. Dynamic obstacle avoidance refers to: using a local path planning algorithm for dynamic obstacle avoidance to cope with a dynamically changing environment.
[0132] Based on the coordinate information of the steel cage, this embodiment can use the target point The path planning algorithm plans the shortest path in the environment model based on the current position and target position of the robot. During this process, the algorithm takes into account the geometric constraints of the environment (such as walls, pillars, etc.) and dynamic constraints (such as workers in motion). During the movement of the robot, the depth camera and lidar continue to be used to scan the surrounding environment in order to detect new obstacles or environmental changes in a timely manner. The dynamic window approach (DWA) is used in local path planning to dynamically adjust the path according to the results of real-time environmental perception to avoid impending collisions. During the entire movement process, the control system needs to continuously adjust the speed and steering of the robot to adapt to changes in path planning and obstacle avoidance needs.
[0133] By executing the above S301~S305, the robot will safely reach the position of the steel cage according to the predetermined path and prepare for the binding operation. Through the above steps, the self-propelled steel binding robot can realize autonomous navigation from the current position to the steel cage, and effectively handle various obstacles in the path to ensure the continuity and safety of the operation. This embodiment can improve the automation level of steel binding in the steel cage prefabrication plant, especially the accuracy in positioning and navigation. This embodiment is mainly aimed at allowing the steel binding robot to accurately identify and locate the steel cage to be tied in a changeable factory environment, especially in a complex three-dimensional space and in the presence of occlusion. It can be seen that this embodiment effectively improves the operating accuracy and efficiency of the steel binding robot in the factory, reduces labor costs and safety risks, and is of great significance to improving the automation level of the construction industry.
[0134] This embodiment combines the depth camera and LiDAR technology, and realizes the precise alignment of the three-dimensional point cloud data of the steel cage through an efficient point cloud alignment algorithm (ICP algorithm), so as to accurately locate the steel cage in a complex factory environment. This method makes full use of the high-resolution advantage of the depth camera in capturing the shape and size of objects, and the high-precision advantage of LiDAR in spatial position measurement. Through data fusion and point cloud alignment technology, the local coordinates of the steel cage are converted into coordinates in the global map coordinate system. This not only greatly improves the positioning accuracy of the navigation system, but also provides reliable spatial information support for subsequent automatic navigation and binding operations.
[0135] This embodiment realizes the efficient integration of depth and geometric information; specifically, this embodiment effectively integrates depth information and geometric information through the data fusion of depth camera and laser radar, and improves the comprehensiveness and accuracy of environmental perception. This embodiment realizes real-time point cloud data processing; specifically, this embodiment adopts advanced point cloud processing technologies such as voxel grid filtering, statistical analysis denoising, plane segmentation and clustering algorithms, and deep learning models (MASK R-CNN and PointTransformer) to realize efficient recognition and instance segmentation of steel cages. This embodiment realizes accurate point cloud registration technology; specifically, this embodiment applies ICP and other point cloud registration technologies to realize accurate spatial alignment between depth map point cloud and laser point cloud, providing key technical support for accurate positioning of steel cages. This embodiment realizes position estimation based on Kalman filtering; specifically, this embodiment uses Kalman filter to process data from different sensors, optimizes the estimation process of steel cage position, and ensures the accuracy and stability of positioning information.
[0136] This embodiment achieves high-precision positioning of the steel cage in a complex environment through the data fusion and point cloud alignment technology of the depth camera and laser radar, providing reliable technical support for the autonomous navigation and precise operation of the self-propelled steel bar tying robot, showing significant innovation and application value. The implementation of this method does not rely on preset markings or specific visual identification of the environment, enabling it to operate stably in different working environments, enhancing the versatility and adaptability of the system.
[0137] See also Fig. 9 , Fig. 9 This is a schematic diagram of the structure of an autonomous navigation system of a steel bar tying robot provided in an embodiment of the present application. The steel bar tying robot is provided with a depth camera and a laser radar. The autonomous navigation system of the steel bar tying robot includes:
[0138] A map construction module 901 is used to construct a static map using the three-dimensional depth data collected by the depth camera and the first point cloud data collected by the laser radar;
[0139] The point cloud extraction module 902 is used to extract first sub-point cloud data of the steel cage from the first point cloud data; it is also used to restore the second point cloud data according to the depth image and the color image taken by the depth camera, and extract the second sub-point cloud data of the steel cage from the second point cloud data;
[0140] A point cloud alignment module 903 is used to align the first sub-point cloud data and the second sub-point cloud data by using an iterative closest point algorithm to obtain aligned target point cloud data;
[0141] A coordinate determination module 904 is used to calculate the coordinate information of the steel cage according to the static map and the target point cloud data;
[0142] The navigation module 905 is used to perform path planning according to the coordinate information of the steel cage to obtain a navigation path, and control the movement of the steel bar tying robot according to the navigation path.
[0143] The solution provided in this embodiment is implemented based on a steel bar binding robot equipped with a depth camera and a laser radar, and then a static map is constructed based on the three-dimensional depth data collected by the depth camera and the first point cloud data collected by the laser radar, so as to provide basic environmental information for subsequent navigation. In this embodiment, the first sub-point cloud data of the steel cage is extracted from the first point cloud data, and the depth image and color image taken by the depth camera are restored to obtain the second point cloud data, and the second sub-point cloud data of the steel cage is extracted from the second point cloud data. Through the above method, the point cloud information corresponding to the steel cage extracted based on the depth camera and the laser radar can be obtained. In this embodiment, the first sub-point cloud data and the second sub-point cloud data are aligned, thereby ensuring the consistency between different data sources, thereby improving the accuracy of the steel cage position identification. In this embodiment, the coordinate information of the steel cage can be calculated according to the static map and the aligned target point cloud data, so as to navigate according to the coordinate information of the steel cage. The depth camera can provide rich texture and color information, and the laser radar can provide accurate distance and angle measurement. In this embodiment, the information collected by the depth camera and the laser radar is integrated through point cloud alignment, so as to accurately identify the location of the steel cage and improve the navigation accuracy of the steel bar binding robot.
[0144] Furthermore, the point cloud extraction module includes:
[0145] A filtering unit, configured to filter the first point cloud data using a voxel filter;
[0146] An outlier removal unit, used to remove outliers in the filtered first point cloud data;
[0147] A point cloud segmentation unit, used to segment the first point cloud data after removing outliers using a plane segmentation algorithm to obtain a plurality of point cloud regions;
[0148] A clustering unit is used to cluster the point cloud area and use a first deep learning model to identify the clustering results to obtain the first sub-point cloud data of the steel cage.
[0149] Furthermore, the point cloud extraction module includes:
[0150] A preprocessing unit, used to preprocess the depth image taken by the depth camera using OpenCV;
[0151] A restoration unit, configured to restore the preprocessed depth image and the color image based on camera parameters to obtain the second point cloud data;
[0152] A data extraction unit is used to extract second sub-point cloud data of the steel cage from the second point cloud data using a second deep learning model.
[0153] Furthermore, the process of preprocessing the depth image taken by the depth camera by the point cloud extraction module 902 using OpenCV includes: performing noise processing and edge enhancement operations on the depth image taken by the depth camera using OpenCV.
[0154] Furthermore, the process in which the navigation module 905 performs path planning according to the coordinate information of the steel cage to obtain a navigation path includes: determining the binding position according to the coordinate information of the steel cage; performing path planning according to the current position of the steel binding robot and the binding position to obtain the navigation path.
[0155] Furthermore, the process of the navigation module 905 controlling the movement of the rebar tying robot according to the navigation path includes: using the depth camera and the lidar to detect whether there are obstacles in the navigation path; if so, updating the navigation path according to the position of the obstacle so as to control the movement of the rebar tying robot according to the updated navigation path; if not, controlling the movement of the rebar tying robot according to the navigation path.
[0156] Furthermore, the process of the map construction module 901 constructing a static map using the three-dimensional depth data collected by the depth camera and the first point cloud data collected by the lidar includes: using the depth camera to collect three-dimensional depth data of the working area; using the lidar to collect first point cloud data of the working area; using a preset algorithm to process the three-dimensional depth data and the first point cloud data to obtain the two-dimensional static map; wherein the preset algorithm is a simultaneous positioning and map construction algorithm based on laser scanning data.
[0157] Since the embodiments of the system part correspond to the embodiments of the method part, please refer to the description of the embodiments of the method part for the embodiments of the system part, which will not be repeated here.
[0158] The present application also provides a storage medium on which a computer program is stored, and when the computer program is executed, the steps provided in the above embodiment can be implemented. The storage medium may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.
[0159] The present application also provides an electronic device, which may include a memory and a processor, wherein a computer program is stored in the memory, and when the processor calls the computer program in the memory, the steps provided in the above embodiment may be implemented. Of course, the electronic device may also include various network interfaces, power supplies and other components.
[0160] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the various embodiments can be referred to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of this application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of this application.
[0161] It should also be noted that, in this specification, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises", "comprising" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.
Claims
1. An autonomous navigation method for a steel bar tying robot, characterized in that: The steel bar tying robot is provided with a depth camera and a laser radar, and the autonomous navigation method of the steel bar tying robot includes: Constructing a static map using the three-dimensional depth data collected by the depth camera and the first point cloud data collected by the laser radar; Extracting first sub-point cloud data of the steel cage from the first point cloud data; Restore the depth image and the color image taken by the depth camera to obtain second point cloud data, and extract second sub-point cloud data of the steel cage from the second point cloud data; Aligning the first sub-point cloud data and the second sub-point cloud data by an iterative closest point algorithm to obtain aligned target point cloud data; Calculating the coordinate information of the steel cage according to the static map and the target point cloud data; A navigation path is obtained by performing path planning according to the coordinate information of the steel cage, and the movement of the steel bar tying robot is controlled according to the navigation path.
2. The autonomous navigation method of the steel bar tying robot according to claim 1, characterized in that: Extracting first sub-point cloud data of the steel cage from the first point cloud data includes: Filtering the first point cloud data using a voxel filter; Removing outliers from the filtered first point cloud data; Using a plane segmentation algorithm to segment the first point cloud data after removing outliers, a plurality of point cloud regions are obtained; The point cloud area is clustered, and the clustering result is identified using a first deep learning model to obtain first sub-point cloud data of the steel cage.
3. The autonomous navigation method of the steel bar tying robot according to claim 1, characterized in that: The method further comprises: restoring the second point cloud data according to the depth image and the color image taken by the depth camera, and extracting the second sub-point cloud data of the steel cage from the second point cloud data, including: Preprocessing the depth image captured by the depth camera using a cross-platform computer vision library; Restoring the preprocessed depth image and the color image based on camera parameters to obtain the second point cloud data; A second deep learning model is used to extract second sub-point cloud data of the steel cage from the second point cloud data.
4. The autonomous navigation method of the steel bar tying robot according to claim 3 is characterized in that: The depth image captured by the depth camera is preprocessed using a cross-platform computer vision library, including: A cross-platform computer vision library is used to perform noise processing and edge enhancement operations on the depth image taken by the depth camera.
5. The autonomous navigation method of the steel bar tying robot according to claim 1, characterized in that: Performing path planning according to the coordinate information of the steel cage to obtain a navigation path includes: Determining the binding position according to the coordinate information of the steel cage; Path planning is performed according to the current position of the steel bar tying robot and the tying position to obtain the navigation path.
6. The autonomous navigation method of the steel bar tying robot according to claim 5, characterized in that: Controlling the movement of the steel bar tying robot according to the navigation path includes: Using the depth camera and the laser radar to detect whether there are obstacles on the navigation path; If so, updating the navigation path according to the position of the obstacle so as to control the movement of the steel bar tying robot according to the updated navigation path; If not, the steel bar tying robot is controlled to move according to the navigation path.
7. The autonomous navigation method of the steel bar tying robot according to claim 1, characterized in that: Constructing a static map using the three-dimensional depth data collected by the depth camera and the first point cloud data collected by the laser radar, including: Using the depth camera to collect three-dimensional depth data of the working area; Using the laser radar to collect first point cloud data of the working area; The three-dimensional depth data and the first point cloud data are processed using a preset algorithm to obtain the two-dimensional static map; wherein the preset algorithm is a simultaneous positioning and map construction algorithm based on laser scanning data.
8. An autonomous navigation system for a steel bar tying robot, characterized in that: The steel bar tying robot is provided with a depth camera and a laser radar, and the autonomous navigation system of the steel bar tying robot includes: A map construction module, used to construct a static map using the three-dimensional depth data collected by the depth camera and the first point cloud data collected by the laser radar; A point cloud extraction module, used to extract first sub-point cloud data of the steel cage from the first point cloud data; and also used to restore the second point cloud data according to the depth image and the color image taken by the depth camera, and extract the second sub-point cloud data of the steel cage from the second point cloud data; A point cloud alignment module, used for aligning the first sub-point cloud data and the second sub-point cloud data by an iterative closest point algorithm to obtain aligned target point cloud data; A coordinate determination module, used to calculate the coordinate information of the steel cage according to the static map and the target point cloud data; The navigation module is used to perform path planning according to the coordinate information of the steel cage to obtain a navigation path, and control the movement of the steel bar tying robot according to the navigation path.
9. The autonomous navigation system of the steel bar tying robot according to claim 8, characterized in that: The point cloud extraction module comprises: A filtering unit, configured to filter the first point cloud data using a voxel filter; An outlier removal unit, used to remove outliers in the filtered first point cloud data; A point cloud segmentation unit, used to segment the first point cloud data after removing outliers using a plane segmentation algorithm to obtain a plurality of point cloud regions; A clustering unit is used to cluster the point cloud area and use a first deep learning model to identify the clustering results to obtain the first sub-point cloud data of the steel cage.
10. The autonomous navigation system of the steel bar tying robot according to claim 8, characterized in that: The point cloud extraction module comprises: A preprocessing unit, configured to preprocess the depth image captured by the depth camera using a cross-platform computer vision library; A restoration unit, configured to restore the preprocessed depth image and the color image based on camera parameters to obtain the second point cloud data; A data extraction unit is used to extract second sub-point cloud data of the steel cage from the second point cloud data using a second deep learning model.
