Double-arm cooperation method of humanoid vibrating robot for unstructured environment operation

Abstract

The invention relates to the technical field of humanoid vibration robots, solves the technical problem that the prior art is difficult to adapt to concrete vibration construction operation in a complex unstructured scene, and particularly relates to a double-arm cooperation method of a humanoid vibration robot for unstructured environment operation. The method comprises the following steps: fusing collected multi-source sensor data, constructing an operation environment map, and identifying a target operation area and a target vibration operation position; a generalized comprehensive risk index is constructed to dynamically grade a target operation area, a cooperative motion track of the double mechanical arms is planned, a control instruction is generated, and the crawler-type mobile vehicle body and the double six-degree-of-freedom cooperative mechanical arms are controlled to complete concrete vibrating operation; when obstacle interference exists in an operation area, the double six-degree-of-freedom cooperative mechanical arms complete vibration operation in a cooperative mode. The concrete vibrating device can adapt to complex and changeable unstructured construction environments, and the flexibility, the collaboration and the safety of concrete vibrating operation are improved.

Description

Technical Field

[0001] This invention relates to the field of humanoid vibratory robot technology, and more particularly to a dual-arm collaborative method for humanoid vibratory robots operating in unstructured environments. Background Technology

[0002] In the construction of large-scale water conservancy and hydropower projects such as arch dams and roller-compacted concrete dams, concrete vibration is a crucial step in ensuring the strength and seepage prevention performance of the dam body. Currently, concrete vibration operations mainly rely on manual hand-held vibrators or mechanical trolley equipment, which presents problems of high labor intensity and harsh working environments. Although single-arm and tandem vibration robots mounted on hydraulic excavators or gantry cranes have emerged in recent years, achieving some degree of mechanization, their level of intelligence and environmental adaptability are still insufficient when facing unstructured and complex steel reinforcement construction scenarios.

[0003] Existing automated vibration compaction equipment mostly operates based on preset fixed trajectories, and its adaptability to complex working environments needs to be improved. In actual construction, to meet the stress requirements of the dam structure and the support requirements of the formwork, a large number of inclined reinforcing bars or reinforcing mesh structures are usually laid out within the compaction surface. Existing single-arm robots are limited by their mechanical structure and cannot flexibly handle inclined or grid-like obstacles. In addition, when facing flexible obstacles, a single robotic arm cannot perform vibration actions while avoiding flexible obstacles.

[0004] While existing large-scale, multi-row vibratory compaction equipment offers high operational efficiency, it is bulky, lacks flexibility, and has significant blind spots. It often fails to reach narrow areas such as the edges of the compaction surface, the vibratory plates, and areas inaccessible to the multi-row vibratory rods, requiring manual compaction. Although some research has attempted to introduce dual-arm robots to improve flexibility, current control systems mostly involve simply stacking two single arms, lacking a collaborative working mechanism between the arms.

[0005] At the level of environmental perception and motion planning algorithms, existing technologies have certain limitations. On the one hand, perception methods are limited, relying on single images or LiDAR, making it difficult to construct high-precision environmental maps. On the other hand, existing motion planning algorithms lack hierarchical and zoning strategies for complex working environments, i.e., they lack top-level planning and decision-making mechanisms. This results in inefficient operation in obstacle-free areas, while facing problems of low recognition accuracy and obstacle avoidance failure in complex reinforced concrete areas. This lack of environmental adaptability makes it difficult for existing vibratory compaction robots to achieve a balance between construction efficiency and safety. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention provides a collaborative method for humanoid vibratory robots operating in unstructured environments. This method solves the technical problem that existing technologies are difficult to adapt to concrete vibration construction operations in complex unstructured scenarios, thereby improving the flexibility, coordination, and safety of concrete vibration operations.

[0007] To solve the above-mentioned technical problems, the present invention provides the following technical solution: a collaborative method for a humanoid vibratory robot with two arms for operation in unstructured environments, the method comprising the following steps: S1. Collect multi-source sensor data, including unstructured environmental information and vibration operation status information, in the concrete vibration operation environment, and perform fusion processing on the collected multi-source sensor data to generate an operation environment map containing spatial distribution features and semantic features. S2. Based on the work environment map, analyze the target work area and identify the steel reinforcement structure and workable area within the target work area to obtain the target vibration operation location; S3. Based on the spatial constraints, environmental complexity, and operability range of the target work area, divide the target work area into work zones to generate work area division results for dual robotic arm collaborative operation. S4. Based on the target vibration operation location and the division of the operation area, plan the coordinated motion trajectory of the two robotic arms, and perform simulation verification and adjustment on the coordinated motion trajectory; S5. The adjusted cooperative motion trajectory is converted into control commands to control the humanoid vibrating robot to move to the workable area, and the dual six-degree-of-freedom cooperative robotic arm carrying the vibration execution module completes the concrete vibration operation according to the cooperative motion trajectory.

[0008] Furthermore, the collection of the multi-source sensor data includes: LiDAR for measuring the distance between a humanoid vibratory robot and other objects; RGB industrial camera for acquiring information on the state of concrete surfaces and images of the work area; distance sensor for measuring the distance between the vibratory rod and the concrete surface; positioning device for acquiring the spatial position of the humanoid vibratory robot; and attitude sensor for measuring the attitude information of the humanoid vibratory robot.

[0009] Furthermore, in step S2, the specific process includes the following steps: S21. Based on the spatial distribution characteristics of point cloud data in the work environment map and the semantic characteristics of image data, the target work area containing steel structure, formwork and concrete area is parsed out. S22. Using the RANSAC algorithm or a weighted RANSAC algorithm based on normal vector consistency constraints, identify single steel bars or densely connected steel bars in the target work area and determine the workable area. S23. Based on the spatial distribution characteristics of point cloud data, construct a point cloud distribution probability density function for extracting the transverse and longitudinal reinforcement direction features of the steel mesh within the workable area, where: For transverse and longitudinal reinforcement k A point cloud in x One-dimensional array formed by the axial directions , x Point cloud probability density function along the axis for: z Point cloud probability density function along the axis for: In the above formula, These are the parameters of the Gaussian mixture model; M, N The number of Gaussian distributions represents the physical quantity of longitudinal and transverse reinforcing bars. and The first m The mixing coefficients and probability density function of a Gaussian distribution; satisfy ; and They are the first m The mean vector and covariance matrix of a Gaussian distribution; S24. Determine the intersection nodes of the transverse and longitudinal reinforcement bars based on their directional characteristics. The coordinate set of the intersection nodes is as follows: ,in, , ; S25. Determine the geometric center position of the corresponding steel reinforcement mesh based on the spatial relationship of adjacent intersection nodes, and use the geometric center position as the target vibration operation position within the corresponding steel reinforcement mesh, wherein: For the vertical first i root, i+ 1st and horizontal j root, j A steel mesh consisting of +1 steel bar has its geometric center located at... Then, the final vibration point is determined based on the vibration spacing, which is the target vibration operation position.

[0010] Furthermore, in step S22, the specific process includes the following steps: S221. The RANSAC algorithm is used to fit the cylindrical steel bars in order to identify the single steel bar in the target working area. S222. Based on the spatial positional relationship between sampling points and the preset geometric model in point cloud data and the consistency of surface normals, construct an interior point determination cost model for judging candidate rebar points. The expression is: In the above formula, This is the current sampling point; Model The parameters of the cylindrical model are iteratively fitted to the current rebar point cloud; D This represents the Euclidean distance from the point to the surface of the preset geometric model; This is the local normal vector of the sampling point; This is the normal vector at the corresponding position on the model surface; and These are weight parameters; S223. Based on the aforementioned interior point determination cost model The point cloud data is filtered and the main direction of the steel bars and the corresponding geometric model are fitted. S224. For areas where reinforcing bars intersect or adhere, the point cloud data is segmented based on the consistency of local geometric features of the point cloud, and the intersecting or adhered areas are split into multiple independent reinforcing bar objects to complete the fine identification of reinforcing bars within the target work area and determine the workable area.

[0011] Furthermore, in step S3, the specific process includes the following steps: S31. Construct a generalized comprehensive risk index for partitioning the target work area according to the complexity of the work area. The expression is: In the above formula, x represents the position of the robotic arm's end effector; x obs Location of the nearest obstacle; ε To prevent extremely small positive numbers with a denominator of zero; k 1 represents the distance weighting coefficient, used to adjust the degree of influence of obstacle distance on the overall risk assessment; Represents the visual entropy factor; u rgb This represents the RGB image captured by the camera; k 2 represents the texture entropy weighting coefficient; J represents the current Jacobian matrix of the robotic arm; J T `det` represents the transpose of the Jacobian matrix; `det` represents the matrix determinant operation. k 3 represents the kinematic weighting coefficient; S32. Pre-set a high-risk area threshold for quantitative classification of the complexity of the target work area. Low-risk area threshold ,in: High-risk area threshold Defined as the dynamic limit risk value for a robotic arm to perform an emergency stop under full load without overshooting or collision; low-risk region threshold. Defined as the baseline risk value when a robotic arm moves at a rated speed in an unobstructed space; S33, Generalized Comprehensive Risk Index Based on Target Work Area The robotic arm's current workspace is dynamically divided into the following three risk level zones: like This area is a region of strong visual-tactile coupling, i.e., a danger zone. like This is the dynamic buffer zone for potential energy dissipation, i.e., the transition zone; like This is the free configuration energy efficiency optimization zone, i.e., the safe zone.

[0012] Furthermore, the three types of work zones employ different robotic arm motion control methods, wherein: In hazardous areas, enhance the compliance characteristics of the robotic arm's end effector; In the transition zone, the movement trajectory of the robotic arm is smoothly transitioned; Within the safe zone, trajectory optimization is performed with the goal of improving operational efficiency.

[0013] Furthermore, it also includes triggering corresponding control modes based on the identified rebar shape, namely: When a flexible obstacle is detected in the working area of ​​one of the robotic arms during the vibration operation, and the flexible obstacle interferes with the predetermined vibration position, the dual robotic arm cooperative operation mode is triggered. The flexible obstacle includes, but is not limited to, cables, hoses or other construction components that can undergo elastic deformation. Based on the current working area results and the position of the robotic arm, the robotic arm that is far from the target vibration working position is set as the assisting arm, and the robotic arm that is close to the target vibration working position is set as the working arm. Under the control command, the assisting arm uses the end effector to perform elastic pushing or pulling operations on the flexible obstacle located on the concrete surface to form a temporary working space near the target vibration position and maintain the space state within a preset time window. If, during the vibration operation, it is detected that the assisting arm is maintaining the position of the flexible obstacle while the working arm enters the target vibration operation position according to the planned trajectory, the vibration execution module is controlled to complete the corresponding vibration operation. After the vibration is completed, the working arm first exits the operation area, and then the assisting arm releases its effect on the flexible obstacle and exits in reverse order, thereby completing the vibration operation without damaging the structure of the flexible obstacle.

[0014] Furthermore, the humanoid vibratory robot includes: The tracked mobile vehicle body serves as the supporting foundation for the entire humanoid vibratory robot. Its bottom is equipped with a drive track mechanism to enable rapid movement on unstructured concrete surfaces, adapting to uneven and obstacle-prone unstructured working environments. The dual six-degree-of-freedom collaborative robotic arms are fixed to the top sides of the tracked mobile vehicle in a laterally symmetrical installation manner. The installation center distance between the dual six-degree-of-freedom collaborative robotic arms is adapted to the working width, and they can move independently or cooperate to complete the vibration operation. The vibration execution module includes a vibrator and a flexible connection assembly, used to complete the concrete vibration operation according to the said coordinated motion trajectory; The energy module provides stable power for the drive of the tracked mobile vehicle, the movement of the dual six-degree-of-freedom collaborative robotic arm, the vibration operation of the vibration execution module, the data acquisition of the multi-source sensor module, and the calculation and control of the central decision control module. A multi-source sensor module includes at least a lidar, an RGB industrial camera, a ranging sensor, a positioning device, and an attitude sensor; The central decision-making and control module establishes signal connections with the tracked mobile vehicle, the dual six-degree-of-freedom collaborative robotic arm, the vibration execution module, the energy module, and the multi-source sensor module via shielded wired cables.

[0015] Furthermore, the flexible connection assembly includes an upper connecting flange, a lower connecting flange, a buffer spring, and a guide post for limiting the radial displacement of the buffer spring; The upper connecting flange is fixed to the end effector of the dual six-degree-of-freedom collaborative robotic arm by bolts, the lower connecting flange is connected to the tail of the vibrating rod by threads, and the buffer spring is sleeved on the guide post between the upper connecting flange and the lower connecting flange, and the two ends of the buffer spring are welded and fixed to the upper connecting flange and the lower connecting flange respectively.

[0016] Furthermore, the central decision control module is used to fuse and process the data collected by the multi-source sensor module, construct a working environment map, and generate control commands for the collaborative operation of the dual six-degree-of-freedom robotic arm based on the working environment map.

[0017] By employing the above technical solution, the present invention provides a collaborative method for the dual arms of a humanoid vibratory robot for unstructured environments, which has at least the following beneficial effects: First, it significantly improves the environmental adaptability of concrete vibration operations in unstructured environments. This invention constructs an operational environment map containing spatial distribution features and semantic features through multi-source sensor fusion, enabling accurate perception and identification of complex construction environments such as reinforcing bars, concrete, and obstacles. This allows the vibration robot to operate stably in unstructured environments with dense reinforcing bars and complex structures.

[0018] Secondly, it enables quantitative assessment and adaptive control of vibration tamping operation risks, improving operational safety and reliability. By constructing a generalized comprehensive risk index to dynamically classify the target operation area, and adaptively switching the robotic arm control strategy according to different risk levels, it effectively reduces the risks of robotic arm collisions, unusual configurations, and operational instability, ensuring both operational safety and efficiency.

[0019] Finally, this invention enhances the collaborative operation capability of dual robotic arms, improving construction continuity and efficiency under complex conditions. It is applicable to collaborative operation of dual robotic arms in complex conditions such as those with flexible obstacles. By dividing the roles and coordinating the control of the assisting arm and the working arm, it expands the workspace, enables continuous concrete vibration operations, reduces the need for manual intervention, and improves overall construction efficiency. Attached Figure Description

[0020] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a flowchart of the dual-arm collaborative method of the humanoid vibrating robot in this invention; Figure 2 This is a schematic diagram of the humanoid vibrating robot in this invention.

[0021] In the diagram: 1. Tracked mobile vehicle body; 2. Dual six-degree-of-freedom collaborative robotic arm; 3. Vibration execution module; 4. Energy module; 5. Multi-source sensor module; 6. Central decision control module. Detailed Implementation

[0022] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. This will allow for a full understanding of how the present application uses technical means to solve technical problems and achieve technical effects, and to facilitate its implementation.

[0023] Example 1: This embodiment takes the concrete compaction surface in the construction of large-scale water conservancy and hydropower projects such as arch dams and roller-compacted concrete dams as an example, and proposes a collaborative method for a humanoid vibratory robot with two arms for operation in unstructured environments. This method aims to solve the technical problem that existing technologies are difficult to adapt to in complex unstructured scenarios for concrete vibration compaction, thereby improving the flexibility, coordination, and safety of concrete vibration operations. Figure 1 As shown, the method includes the following steps: S1. Collect multi-source sensor data, including unstructured environmental information and vibration operation status information, from the concrete compaction operation environment. Then, fuse the collected multi-source sensor data to generate an operation environment map containing spatial distribution features and semantic features. In this embodiment, the multi-source sensor data collection is accomplished by a multi-source sensor module installed on a humanoid compaction robot. This module includes at least a LiDAR for measuring the distance between the humanoid compaction robot and other objects; an RGB industrial camera for collecting concrete surface status information and operation area images; a distance sensor for measuring the distance between the vibrator and the concrete surface; a positioning device for obtaining the spatial position of the tracked mobile vehicle; and an attitude sensor for measuring the attitude information of the tracked mobile vehicle. An FPGA-based hardware synchronization trigger is electrically connected to the LiDAR, RGB industrial camera, distance sensor, positioning device, and attitude sensor to trigger the synchronous data collection of each sensor, achieving data time synchronization.

[0024] As a further preferred embodiment, in the fusion processing, the point cloud data acquired by the lidar and the image data acquired by the RGB industrial camera are matched based on the time synchronization information of each sensor, the point cloud data and the image data are spatially aligned, and the working environment map is generated by combining spatial geometric features and image texture features.

[0025] Preferably, this embodiment can also remove noise points in point cloud data based on edge features and texture information in image data to improve the accuracy of the work environment map.

[0026] More specifically, this embodiment achieves color-based point cloud data rendering by mapping RGB information in image data to point clouds; it enhances the semantic information of the point cloud by mapping semantic labels detected by the camera to corresponding pixels in the depth image using the PointPainting algorithm. A 3D convolutional neural network (such as the BEVFusion architecture) is used to simultaneously process the enhanced depth image and the original RGB image, extracting a fused feature map containing spatial and semantic information. Combining algorithms such as LOAM and ORB-SLAM2, real-time localization and mapping are achieved through point cloud registration (ICP algorithm) and image feature matching. Kalman filtering is used to fuse IMU data, filtering high-frequency vibration noise; a global optimization algorithm is used to adjust the parameters of the point cloud and image to ensure spatial, color, and temporal consistency. Finally, panoramic stitching is achieved by connecting the edge of the point cloud and the overlapping area of ​​the image to generate a detailed 3D scene map of the work environment.

[0027] S2. Based on the work environment map, analyze the target work area and identify the reinforcing steel structure and workable areas within the target work area to obtain the target vibration operation location. Step S2 specifically includes the following steps: S21. Based on the spatial distribution characteristics of point cloud data and the semantic characteristics of image data in the work environment map, the target work area containing steel structure, formwork and concrete area is parsed out, and the work environment map is stored in the storage module to provide a data foundation for subsequent work zoning and dual robotic arm collaborative work planning.

[0028] S22. Using the RANSAC algorithm or a weighted RANSAC algorithm based on normal vector consistency constraints, identify single rebars or densely connected rebars within the target working area and determine the workable area. To further refine the identification of rebar obstacles within the target working area, for general single rebars, identification can be achieved by fitting the rebar cylinder using the RANSAC algorithm. However, for densely connected rebars, the traditional RANSAC algorithm only considers Euclidean distance constraints, making it difficult to distinguish between adjacent rebars in densely connected rows. Based on this, this embodiment proposes a weighted RANSAC algorithm based on normal vector consistency constraints, constructing an interior point decision cost model including angle constraints for identification. Therefore, in step S22, the specific process includes the following steps: S221. Use the RANSAC algorithm to fit the cylindrical steel bar to identify the single steel bar in the target working area. For example, use the RANSAC algorithm to extract the cylindrical model from the point cloud data and match the diameter, length and spatial orientation of the steel bar.

[0029] S222. Based on the spatial positional relationship between sampling points and the preset geometric model in point cloud data and the consistency of surface normals, construct an interior point determination cost model for judging candidate rebar points. The expression is: In the above formula, This is the current sampling point; Model The parameters of the cylindrical model are iteratively fitted to the current rebar point cloud; D This represents the Euclidean distance from the point to the surface of the preset geometric model; This is the local normal vector of the sampling point; This is the normal vector at the corresponding position on the model surface; and These are weight parameters; In this embodiment, the preset geometric model refers to a standardized geometric shape (such as a cylinder, cuboid, or sphere) predefined before 3D reconstruction, used to match the actual collected point cloud data. In a concrete vibration scenario, reinforcing bars are typically represented as cylindrical models, while the edges of the formwork are represented as cuboid models. By fitting the preset geometric model to the actual point cloud, the position, size, and orientation of structures such as reinforcing bars and formwork can be quickly identified, providing a benchmark for subsequent vibration path planning.

[0030] Spatial positional relationships refer to the relative positions (such as the distance between reinforcing bars and formwork, and the coordinates of reinforcing bar intersections) and directions (such as the principal axis direction of the reinforcing bars) of objects in three-dimensional space. By aligning the spatial positions of the reinforcing bars and formwork using a LiDAR and an RGB industrial camera, the relative positions of the reinforcing bars and formwork can be accurately calculated, avoiding collisions between the vibrator and formwork or missed areas of reinforcing bars. Surface normal consistency refers to the uniformity of the normal directions on the object's surface. In point cloud registration, areas with high normal consistency (such as smooth formwork surfaces) can be quickly registered using ICP (Iterative Closest Point); areas with inconsistent normals (such as reinforcing bar intersections) require optimization through multi-view point cloud fusion or normal filtering. For example, using normal filtering algorithms (such as the Sobel operator) to smooth the surface normals of the reinforcing bars improves registration accuracy.

[0031] S223. Based on the aforementioned interior point determination cost model The point cloud data is filtered and fitted with the main direction of the reinforcing bars and the corresponding geometric model. This embodiment filters the point cloud data to obtain high-confidence point clouds by defining thresholds for in-points (i.e., points conforming to the preset geometric model) and out-points (noise or anomalies), thereby eliminating noise points (such as concrete splash points). For example, for a cylindrical model, the cost function is the distance from a point to the cylinder's axis; points with a distance less than the threshold are considered in-points. Finally, the least squares method is used to optimize the fitting parameters, generating an accurate geometric model of the reinforcing bars for subsequent vibration path planning.

[0032] S224. For areas where reinforcing bars intersect or adhere, the point cloud data is segmented based on the consistency of local geometric features of the point cloud, and the intersecting or adhered areas are split into multiple independent reinforcing bar objects, thereby completing the fine identification of reinforcing bars within the target work area and determining the workable area.

[0033] This embodiment utilizes the abrupt changes in point cloud density at rebar intersections to segment the intersection regions using a region growing algorithm. For example, a density threshold is set, and regions with abrupt density changes are marked as intersection points. Alternatively, combining point cloud geometric features with image texture features, the PointPainting algorithm maps image semantic labels to the point cloud, and a 3D convolutional neural network (such as BEVFusion) is used to achieve accurate segmentation of intersections or adhered regions. For instance, in intersection regions, point cloud density and image texture information are fused, and an attention mechanism is used to dynamically adjust the segmentation weights.

[0034] S23. Based on the spatial distribution characteristics of point cloud data, construct a point cloud distribution probability density function to extract the transverse and longitudinal reinforcement direction features of the steel mesh within the workable area. For reinforcement in complex areas of the dam surface, such as the upstream and downstream steel mesh of the dam body, using... N transverse reinforcing bars and M Taking a steel mesh composed of longitudinal reinforcing bars as an example. For transverse and longitudinal reinforcing bars...k A point cloud in x One-dimensional array formed by the axial directions , x Point cloud probability density function along the axis It can be described as: z Point cloud probability density function along the axis It can be described as: In the above formula, These are the parameters of the Gaussian mixture model; M, N The number of Gaussian distributions represents the physical quantity of longitudinal and transverse reinforcing bars. and The first m The mixing coefficients and probability density function of a Gaussian distribution; satisfy ; and They are the first m The mean vector and covariance matrix of a Gaussian distribution; probability density function. It can be described as: In the above formula, the parameters of the Gaussian mixture model The expectation-maximization algorithm can be used for estimation.

[0035] Therefore, it can be obtained x Axial direction m Peak point set The physical meaning is longitudinal. M The location of the reinforcing bars. Similarly, we can obtain... z Axial direction Peak point set The physical meaning is horizontal. N Location of the reinforcing steel bars.

[0036] S24. Determine the intersection points of transverse and longitudinal reinforcement bars based on their directional characteristics. The set of coordinates for these intersection points is obtained from the cross-sectional and longitudinal cross-sectional intersections. ,in, , .

[0037] S25. Determine the geometric center position of the corresponding steel mesh based on the spatial relationship of adjacent intersection nodes, and use the geometric center position as the target vibration operation position within the corresponding steel mesh. For the vertical first... i root, i+1st and horizontal j root, j For a reinforcing mesh consisting of +1 steel bar, the potential vibration point can be determined as the geometric center of the reinforcing mesh. Then, the final vibration point is determined based on the vibration spacing, which is the target vibration operation position.

[0038] S3. Based on the spatial constraints, environmental complexity, and the operable range of the robotic arm within the target work area, the target work area is divided into work zones, generating a work area division result for dual-robotic arm collaborative operation. Since the complexity of actual vibration areas varies, this embodiment constructs a generalized comprehensive risk index to improve vibration efficiency and intelligence. Based on this, the complexity of the work area is divided. Step S3 specifically includes the following steps: S31. Construct a generalized comprehensive risk index for partitioning the target work area according to the complexity of the work area. The expression is: In the above formula, the first term characterizes the spatial risk between the robotic arm's end effector and environmental obstacles. x represents the position of the robotic arm's end effector; x obs Location of the nearest obstacle; ε To prevent extremely small positive numbers with a denominator of zero; k 1 represents the distance weighting coefficient, used to adjust the degree of influence of obstacle distance on the overall risk assessment; the closer the robotic arm's end effector is to the obstacle, the greater the corresponding risk value. In the second item, The visual entropy factor is obtained by calculating the texture entropy of the reinforcing bars within the field of view of an RGB industrial camera; u rgb This represents the RGB image captured by the camera; k 2 represents the texture entropy weighting coefficient. The higher the value, the greater the degree of disorder and confusion of the steel bars in the current area, the stronger the environmental uncertainty, and the higher the corresponding operational risk assessment. The third term is the kinematic risk factor. J represents the current Jacobian matrix of the robotic arm; J T `det` represents the transpose of the Jacobian matrix; `det` represents the matrix determinant operation. k 3 represents the kinematic weighting coefficient. This term indicates the maneuverability of the robotic arm. When the robotic arm approaches an unusual configuration, its maneuverability decreases significantly, the denominator approaches zero, causing this term to rise sharply. This indicates that the robotic arm itself has extremely poor flexibility and the operational risk is extremely high.

[0039] S32. Pre-set a high-risk area threshold for quantitative classification of the complexity of the target work area. Low-risk area threshold Among them, the threshold for high-risk areas. Defined as the dynamic limit risk value for a robotic arm performing an emergency stop under full load without overshooting or collision, its value is determined through offline calibration experiments, and is set to 0.8 in this embodiment. Low-risk area threshold. Defined as the baseline risk value when the robotic arm moves at a rated speed in an unobstructed space, it is set to 0.3 in this embodiment.

[0040] S33, Generalized Comprehensive Risk Index Based on Target Work Area The robotic arm's current workspace is dynamically divided into the following three risk level zones: Category I is the area of ​​strong visual-tactile coupling constraint, i.e., the danger zone. Within this area, if the system determines that the environment is extremely complex or the robotic arm's posture is restricted, the control strategy automatically switches to Cartesian impedance control, giving the robotic arm's end effector compliance characteristics to tolerate physical contact during operation.

[0041] Type II is the dynamic buffer zone for potential energy dissipation, i.e., the transition zone. Within this area, the system applies an improved artificial potential field method, superimposing a tangential velocity damping field on the original gravitational and repulsive fields, enabling the robotic arm to smoothly decelerate and adjust its posture when approaching obstacles, thus eliminating jitter; Category III is the free configuration energy efficiency optimization zone, i.e., the safe zone. Within this area, the environment is open and the robotic arm is highly flexible. The system uses cubic spline interpolation and time-optimal programming algorithms to drive the robotic arm to perform large-scale spatial movements at maximum speed and acceleration, thereby improving work efficiency.

[0042] Different robotic arm motion control methods are adopted for different work zones. In work zones with high constraints (i.e., danger zones), the compliance characteristics of the robotic arm end effector are enhanced; in work zones with medium constraints (i.e., transition zones), the robotic arm motion trajectory is smoothly transitioned; and in work zones with low constraints (i.e., safety zones), the trajectory is optimized with the goal of improving work efficiency.

[0043] This embodiment implements an adaptive collaborative strategy for specific scenarios. The central decision control unit triggers a corresponding control mode based on the identified rebar shape, namely: When the central decision-making and control module detects a flexible obstacle in the working area of ​​one of the robotic arms during the vibration operation, and the flexible obstacle interferes with the predetermined vibration position, the dual-robotic arm collaborative operation mode is triggered. The flexible obstacle includes, but is not limited to, cables, hoses, or other construction components that can undergo elastic deformation.

[0044] When the central decision-making and control module detects that a collaborative operation mode is in place during the vibration operation, it designates the robotic arm furthest from the target vibration position as the assisting arm and the robotic arm closest to the target vibration position as the working arm, based on the current distribution of the work area and the position of the robotic arm. Under control commands, the assisting arm uses its end effector to elastically push or pull on the flexible obstacle on the concrete surface to create a temporary workable space near the target vibration position and maintains this space state within a preset time window.

[0045] When the central decision control module detects that the assisting arm is maintaining the position of the flexible obstacle during the vibration operation, the working arm cuts into the target vibration operation position according to the planned trajectory and controls the vibration execution module to complete the corresponding vibration operation. After the vibration is completed, the working arm first withdraws from the operation area, and then the assisting arm releases its effect on the flexible obstacle and withdraws in reverse order, thereby completing the vibration operation without damaging the structure of the flexible obstacle.

[0046] S4. Based on the target vibration operation location and the division of the operation area, the cooperative motion trajectory of the two robotic arms is planned, and the cooperative motion trajectory is simulated, verified, and adjusted. This embodiment uses a simulation platform for collaborative operation of two robotic arms to simulate, verify, and adjust the cooperative motion trajectory. The simulation platform is used to pre-verify the cooperative motion of the two six-degree-of-freedom collaborative robotic arms during the vibration operation of a concrete dam surface. The central decision control module, based on the aforementioned determined vibration target location and operation area division results, calls the adaptive collaborative operation model to plan the motion trajectory of the two robotic arms, and performs collision checks and trajectory smoothing on the motion trajectory in the simulation environment. When potential interference or motion discontinuity is detected during the simulation, the motion trajectory parameters and related control parameters of the two robotic arms are adjusted to obtain an optimized trajectory that meets the requirements of collaborative operation.

[0047] S5. The adjusted cooperative motion trajectory is converted into control commands to control the humanoid vibrating robot to move to the workable area, and the dual six-degree-of-freedom cooperative robotic arm carrying the vibration execution module completes the concrete vibration operation according to the cooperative motion trajectory.

[0048] After completing the simulation verification and optimization of the collaborative motion trajectory of the two robotic arms, the central decision control module converts the optimized collaborative motion trajectory into corresponding control commands and sends them to the motion control module. According to the control commands, the motion control module first controls the tracked mobile vehicle to move along the planned path to the preset working position within the target vibration work area, and obtains the vehicle's current position through a positioning device. Subsequently, it controls the two six-degree-of-freedom collaborative robotic arms, each carrying a vibration execution module, to sequentially reach each target vibration work position according to the adjusted collaborative motion trajectory, inserting them into the concrete as required, and completing the vibration operation according to preset vibration depth, vibration spacing, and vibration time parameters. During the vibration operation, multi-source sensors collect real-time status information such as vibration depth, robotic arm posture, surface image, and distance to obstacles, and feed it back to the central decision control module. The central decision control module dynamically adjusts the control commands based on the feedback information to ensure that the vibration parameters meet the requirements of concrete construction specifications, while avoiding obstacles such as steel bars and embedded parts, so as to realize adaptive and collaborative vibration operation of the dual six-degree-of-freedom robotic arms in the same working area.

[0049] Example 2: This embodiment provides a humanoid vibratory robot to implement the aforementioned humanoid vibratory robot dual-arm collaborative method, and applies it to the actual concrete area vibration operation in Embodiment 1. For example... Figure 2 As shown, the humanoid vibratory robot consists of a tracked mobile vehicle 1, a dual six-degree-of-freedom collaborative robotic arm 2, a vibratory execution module 3, an energy module 4, a multi-source sensor module 5, and a central decision-making and control module 6.

[0050] The tracked mobile vehicle 1 serves as the supporting foundation for the entire humanoid vibratory robot system. Its bottom is equipped with a drive track mechanism to carry the various modules of the system and enable rapid movement on unstructured concrete surfaces, adapting to uneven and obstacle-prone unstructured working environments.

[0051] The dual six-degree-of-freedom collaborative robotic arms 2 are fixed to the top two sides of the tracked mobile vehicle body 1 in a laterally symmetrical installation manner. The installation center distance between the dual six-degree-of-freedom collaborative robotic arms 2 is adapted to the working width, and they can move independently or cooperate to complete the vibration operation, providing the vibration execution module with multi-degree-of-freedom operation capability.

[0052] The vibration execution module 3 includes a vibratory rod and a flexible connection assembly. The flexible connection assembly includes an upper connecting flange, a lower connecting flange, and a buffer spring. The upper connecting flange is bolted to the end effector of the dual six-degree-of-freedom collaborative robotic arm 2, and the lower connecting flange is threaded to the tail of the vibratory rod. The buffer spring is fitted onto a guide post between the upper and lower connecting flanges, and both ends of the buffer spring are welded to the upper and lower connecting flanges respectively. The guide post limits the radial displacement of the buffer spring, ensuring the stability of the vibratory rod's operating posture. The flexible connection assembly provides an elastic buffer connection between the vibratory rod and the dual six-degree-of-freedom collaborative robotic arm 2, effectively absorbing the vibration load generated by the vibratory rod during vibration operations and reducing the impact of vibration on the rigidity and accuracy of the dual six-degree-of-freedom collaborative robotic arm 2.

[0053] Energy module 4 uses a lithium iron phosphate battery pack as its core power supply unit. The lithium iron phosphate battery pack establishes a circuit connection with each power-consuming module through a waterproof junction box, providing stable power for the drive of the tracked mobile vehicle 1, the movement of the dual six-degree-of-freedom collaborative robotic arm 2, the vibration operation of the vibration execution module 3, the data acquisition of the multi-source sensor module 5, and the computational control of the central decision-making and control module 6. Simultaneously, energy module 4 incorporates a voltage monitoring and overload protection unit to ensure power supply safety.

[0054] The multi-source sensor module 5 includes at least a lidar, an RGB industrial camera, a ranging sensor, a positioning device, and an attitude sensor. The lidar and RGB industrial camera are respectively mounted on the top and front of the tracked mobile vehicle 1 to acquire point cloud data and image data in front of the work area, with their fields of view overlapping by no less than 80%. Additionally, two more RGB industrial cameras and a ranging sensor are fixedly connected to the end of the dual six-degree-of-freedom collaborative robotic arm 2 via brackets to acquire images and distance information of the concrete vibration surface. The positioning device and attitude sensor are horizontally mounted on the tracked mobile vehicle 1.

[0055] The central decision control module 6 establishes signal connections with the tracked mobile vehicle 1, the dual six-degree-of-freedom collaborative robotic arm 2, the vibration execution module 3, the energy module 4, and the multi-source sensor module 5 respectively through shielded wired cables. The central decision control module 6 is used to fuse and process the data collected by the multi-source sensor module 5, construct the operation environment map, and generate control commands for the dual six-degree-of-freedom collaborative robotic arm 2 to work collaboratively based on the operation environment map.

[0056] Those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0057] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. Since the above embodiments are substantially similar to the method embodiments, their descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0058] The above embodiments provide a detailed description of the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A collaborative method for the dual arms of a humanoid vibratory robot operating in unstructured environments, characterized in that, The method includes the following steps: S1. Collect multi-source sensor data, including unstructured environmental information and vibration operation status information, in the concrete vibration operation environment, and perform fusion processing on the collected multi-source sensor data to generate an operation environment map containing spatial distribution features and semantic features. S2. Based on the work environment map, analyze the target work area and identify the steel structure and workable area within the target work area to obtain the target vibration operation location; S3. Based on the spatial constraints, environmental complexity, and operability range of the target work area, divide the target work area into work zones to generate work area division results for dual robotic arm collaborative operation. S4. Based on the target vibration operation location and the division of the operation area, plan the coordinated motion trajectory of the two robotic arms, and perform simulation verification and adjustment on the coordinated motion trajectory; S5. The adjusted cooperative motion trajectory is converted into control commands to control the humanoid vibrating robot to move to the workable area, and the dual six-degree-of-freedom collaborative robotic arm carrying the vibration execution module completes the concrete vibration operation according to the cooperative motion trajectory.

2. The method for collaborative operation of a humanoid vibrating robot with two arms according to claim 1, characterized in that, The collection of the multi-source sensor data includes: LiDAR for measuring the distance between a humanoid vibratory robot and other objects; RGB industrial camera for acquiring information on the state of concrete surfaces and images of the work area; distance sensor for measuring the distance between the vibratory rod and the concrete surface; positioning device for acquiring the spatial position of the humanoid vibratory robot; and attitude sensor for measuring the attitude information of the humanoid vibratory robot.

3. The method for collaborative operation of a humanoid vibratory robot with two arms according to claim 1, characterized in that, In step S2, the specific process includes the following steps: S21. Based on the spatial distribution characteristics of point cloud data in the work environment map and the semantic characteristics of image data, the target work area containing steel structure, formwork and concrete area is parsed out. S22. Using the RANSAC algorithm or the weighted RANSAC algorithm based on normal vector consistency constraints, identify single steel bars or densely connected steel bars in the target work area and determine the workable area. S23. Based on the spatial distribution characteristics of point cloud data, construct a point cloud distribution probability density function for extracting the transverse and longitudinal reinforcement direction features of the steel mesh within the workable area, where: For transverse and longitudinal reinforcement k A point cloud in x One-dimensional array formed by the axial directions , x Point cloud probability density function along the axis for: z Point cloud probability density function along the axis for: In the above formula, These are the parameters of the Gaussian mixture model; M, N The number of Gaussian distributions represents the physical quantity of longitudinal and transverse reinforcing bars. and The first m The mixing coefficients and probability density function of a Gaussian distribution; satisfy ; and They are the first m The mean vector and covariance matrix of a Gaussian distribution; S24. Determine the intersection nodes of the transverse and longitudinal reinforcement bars based on their directional characteristics. The coordinate set of the intersection nodes is as follows: ,in, , ; S25. Determine the geometric center position of the corresponding steel reinforcement mesh based on the spatial relationship of adjacent intersection nodes, and use the geometric center position as the target vibration operation position within the corresponding steel reinforcement mesh, wherein: For the vertical first i root, i+ 1st and horizontal j root, j A steel mesh consisting of +1 steel bar has its geometric center located at... Then, the final vibration point is determined based on the vibration spacing, which is the target vibration operation position.

4. The method for collaborative operation of a humanoid vibratory robot with two arms according to claim 3, characterized in that, In step S22, the specific process includes the following steps: S221. The RANSAC algorithm is used to fit the cylindrical steel bars in order to identify the single steel bar in the target working area. S222. Based on the spatial positional relationship between sampling points and the preset geometric model in point cloud data and the consistency of surface normals, construct an interior point determination cost model for judging candidate rebar points. The expression is: In the above formula, This is the current sampling point; Model The parameters of the cylindrical model are iteratively fitted to the current rebar point cloud; D This represents the Euclidean distance from the point to the surface of the preset geometric model; This is the local normal vector of the sampling point; This is the normal vector at the corresponding position on the model surface; and These are weight parameters; S223. Based on the aforementioned interior point determination cost model The point cloud data is filtered and the main direction of the steel bars and the corresponding geometric model are fitted. S224. For areas where reinforcing bars intersect or adhere, the point cloud data is segmented based on the consistency of local geometric features of the point cloud, and the intersecting or adhered areas are split into multiple independent reinforcing bar objects to complete the fine identification of reinforcing bars within the target work area and determine the workable area.

5. The method for collaborative operation of a humanoid vibrating robot with two arms according to claim 1, characterized in that, In step S3, the specific process includes the following steps: S31. Construct a generalized comprehensive risk index for partitioning the target work area according to the complexity of the work area. The expression is: In the above formula, x represents the position of the robotic arm's end effector; x obs Location of the nearest obstacle; ε To prevent extremely small positive numbers with a denominator of zero; k 1 represents the distance weighting coefficient, used to adjust the degree of influence of obstacle distance on the overall risk assessment; Represents the visual entropy factor; u rgb This represents the RGB image captured by the camera; k 2 represents the texture entropy weighting coefficient; J represents the current Jacobian matrix of the robotic arm; J T `det` represents the transpose of the Jacobian matrix; `det` represents the matrix determinant operation. k 3 represents the kinematic weighting coefficient; S32. Pre-set a high-risk area threshold for quantitative classification of the complexity of the target work area. Low-risk area threshold ,in: High-risk area threshold Defined as the dynamic limit risk value for a robotic arm to perform an emergency stop under full load without overshooting or collision; low-risk region threshold. Defined as the baseline risk value when a robotic arm moves at a rated speed in an unobstructed space; S33, Generalized Comprehensive Risk Index Based on Target Work Area The robotic arm's current workspace is dynamically divided into the following three risk level zones: like This area is a region of strong visual-tactile coupling constraint, i.e., a danger zone. like This is the dynamic buffer zone for potential energy dissipation, i.e., the transition zone; like This is the free configuration energy efficiency optimization zone, i.e., the safe zone.

6. The method for collaborative operation of a humanoid vibratory robot with two arms according to claim 5, characterized in that, The three types of work zones employ different robotic arm motion control methods, among which: In hazardous areas, enhance the compliance characteristics of the robotic arm's end effector; In the transition zone, the movement trajectory of the robotic arm is smoothly transitioned; Within the safe zone, trajectory optimization is performed with the goal of improving operational efficiency.

7. The method for collaborative operation of a humanoid vibratory robot with two arms according to claim 6, characterized in that, It also includes triggering corresponding control modes based on the identified rebar shape, namely: When a flexible obstacle is detected in the working area of ​​one of the robotic arms during the vibration operation, and the flexible obstacle interferes with the predetermined vibration position, the dual robotic arm cooperative operation mode is triggered. The flexible obstacle includes, but is not limited to, cables, hoses or other construction components that can undergo elastic deformation. Based on the current working area results and the position of the robotic arm, the robotic arm that is far from the target vibration working position is set as the assisting arm, and the robotic arm that is close to the target vibration working position is set as the working arm. Under control commands, the assisting arm uses an end effector to elastically push or pull flexible obstacles on the concrete surface to create a temporary working space near the target vibration position and maintain the space state within a preset time window. If, during the vibration operation, it is detected that the assisting arm is maintaining the position of the flexible obstacle while the working arm cuts into the target vibration operation position according to the planned trajectory, the vibration execution module is controlled to complete the corresponding vibration operation. After vibration is completed, the working arm first withdraws from the working area, and then the assist arm releases its influence on the flexible obstacle and withdraws in reverse order, thus completing the vibration operation without damaging the structure of the flexible obstacle.

8. The method for collaborative operation of a humanoid vibratory robot with two arms according to claim 1, characterized in that, The humanoid vibrating robot includes: Tracked mobile vehicle (1), which serves as the supporting base for the entire humanoid vibratory robot, has a drive track mechanism at its bottom for rapid movement on unstructured concrete surfaces to adapt to uneven and obstacle-prone unstructured working environments. The dual six-degree-of-freedom collaborative robotic arms (2) are fixed on the top two sides of the tracked mobile vehicle body (1) in a laterally symmetrical installation manner. The installation center distance between the dual six-degree-of-freedom collaborative robotic arms (2) is adapted to the working width, and they can move independently or cooperate to complete the vibration operation. The vibration execution module (3) includes a vibrating rod and a flexible connection assembly, which is used to complete the concrete vibration operation according to the said coordinated motion trajectory; The energy module (4) is used to provide stable power for the drive of the tracked mobile vehicle (1), the action of the dual six-degree-of-freedom collaborative robotic arm (2), the vibration work of the vibration execution module (3), the data acquisition of the multi-source sensor module (5), and the operation control of the central decision control module (6). The multi-source sensor module (5) includes at least a lidar, an RGB industrial camera, a ranging sensor, a positioning device, and an attitude sensor; The central decision control module (6) establishes signal connections with the tracked mobile vehicle (1), the dual six-degree-of-freedom collaborative robotic arm (2), the vibration execution module (3), the energy module (4), and the multi-source sensor module (5) respectively via shielded wired cables.

9. The method for collaborative operation of a humanoid vibratory robot with two arms according to claim 8, characterized in that, The flexible connection assembly includes an upper connecting flange, a lower connecting flange, a buffer spring, and a guide post for limiting the radial displacement of the buffer spring. The upper connecting flange is fixed to the end effector of the double six-degree-of-freedom collaborative robotic arm (2) by bolts, the lower connecting flange is connected to the tail of the vibrating rod by threads, the buffer spring is sleeved on the guide post between the upper connecting flange and the lower connecting flange, and the two ends of the buffer spring are welded and fixed to the upper connecting flange and the lower connecting flange respectively.

10. The method for collaborative operation of a humanoid vibrating robot with two arms according to claim 8, characterized in that, The central decision control module (6) is used to fuse the data collected by the multi-source sensor module (5), construct the working environment map, and generate control instructions for the collaborative operation of the dual six-degree-of-freedom collaborative robotic arm (2) based on the working environment map.

Images