A humanoid robot global automatic valve screwing system and control method

Abstract

The present invention belongs to the field of humanoid robot valve control technology, and specifically relates to a humanoid robot global automatic valve tightening system and control method, comprising: a mobile chassis, a lifting module, a waist pitch module, a robot body, an end tool, a sensor assembly, and a control system. The robot body is equipped with humanoid arms, and the end tool is configured at the end of the humanoid arms. The sensor assembly is used for environmental perception, collecting valve height and posture information, and operating torque information. The control system is used to achieve rough positioning of the target valve, automatically plan and control the mobile chassis to approach the valve based on the rough positioning results, and adjust the height based on the collected valve height information. The control system also controls the humanoid arms and end tool to complete the tightening operation based on the recognized valve posture. The present invention can automatically identify and operate different types of valves through an intelligent control system, reducing the time and cost of manual operation.

Description

Technical Field

[0001] The present invention belongs to the technical field of humanoid robot valve control, and in particular relates to a humanoid robot global automatic valve screwing system and a control method. Background Art

[0002] With the continuous advancement of industrial automation and intelligence, there is a growing demand for robotic automation in complex operational scenarios. In petrochemical, water supply, heating, and other industrial piping systems, valves are crucial components for pipeline control, and their opening and closing directly impact the normal operation of the piping system. However, due to the diverse types of valves (such as ball valves, gate valves, and butterfly valves), their operation methods vary, and some valves may rust or experience high operating resistance due to long-term use, manual operation is time-consuming and labor-intensive, and may pose safety risks.

[0003] Especially in hazardous or complex operating environments, such as those involving high temperatures, high pressures, or toxic gas leaks, manually tightening valves is not only inefficient but can also pose a safety threat to operators. Furthermore, the large number and widespread distribution of valves in routine pipeline system maintenance makes manual inspection and operation of each valve high-cost and time-consuming.

[0004] Therefore, there is an urgent need for a humanoid robot control method based on automatic valve screwing, so as to replace manual labor in industrial and life scenarios to complete the automatic operation of valves, reduce manual labor intensity and safety risks, and improve the efficiency and accuracy of valve operation. Summary of the Invention

[0005] Purpose of the invention: The purpose of the present invention is to address the deficiencies of the existing technology and provide a humanoid robot global automatic valve turning system and control method, which can automatically identify and operate different types of valves through an intelligent control system, reducing the time and cost of manual operation.

[0006] Technical solution: The humanoid robot global automatic valve turning system of the present invention comprises:

[0007] A mobile chassis equipped with a multi-wheel drive system for movement and navigation;

[0008] a lifting module, connecting the mobile chassis and the robot body, and configured to automatically adjust the working position of the robot body based on the sensed valve height information;

[0009] A waist pitch module, linked to the lifting module, is configured to support the pitch adjustment and left-right rotation of the robot body to adjust the working posture;

[0010] The robot body is equipped with two humanoid arms, each of which has multiple degrees of freedom joints;

[0011] The end tool is configured at the end of the humanoid arms, is used to grasp and screw the valve, and can automatically adjust the grasping force and clamping angle according to the shape and size of the valve;

[0012] Sensor components, used for environmental perception, valve height and posture information collection, and operating torque information collection;

[0013] The control system includes: a valve recognition module, which is used to perform environmental perception and recognition based on the YOLO deep learning algorithm to achieve rough positioning of the target valve; a path planning module, which is used to automatically plan and control the mobile chassis to approach the valve based on the rough positioning result; a height adjustment module, which is used to control the lifting module to adjust the height in real time based on the collected valve height information; and a screwing operation module, which is used to identify the valve posture and control the humanoid arms and end tool to complete the screwing operation based on the identified valve posture.

[0014] To further improve the above technical solution, the sensor assembly includes:

[0015] Head depth camera, configured on the robot head, for environment scanning and valve coarse positioning;

[0016] The chest depth camera is installed on the robot's chest and is used for close-range identification and precise positioning of target valves.

[0017] LiDAR, configured on the mobile chassis and robot body, is used to detect circumferential obstacles;

[0018] Ultrasonic sensors are located on the front and rear sides of the mobile chassis to detect obstacles in the front and rear directions;

[0019] Torque sensors are installed in the joints of the humanoid arms and the end tool to monitor the torque and force conditions during operation in real time.

[0020] Furthermore, when the robot approaches the valve, it uses sensor components including laser radar, depth camera, and ultrasonic sensor to scan and obtain spatial layout information of the target area in real time. Based on the obtained spatial layout information, it uses laser SLAM and multi-sensor fusion technology, combined with a preset path planning algorithm, to generate the optimal path from the current position to the valve.

[0021] According to the generated path, the robot moves step by step according to the predetermined motion instructions by controlling the joints of the mobile chassis and the robot body; based on the collected valve height information, the real-time height difference between the robot and the valve is calculated based on the fusion data of the depth camera, lidar and ultrasonic sensor, and the lifting module and waist pitch module are adjusted synchronously for dynamic adjustment.

[0022] The control method of the humanoid robot global automatic valve screwing system includes the following steps:

[0023] S101: Valve coarse positioning: Based on the YOLO deep learning algorithm, the sensor component acquires images and performs real-time image recognition and processing to achieve the coarse positioning result of the target valve;

[0024] S102, approaching the valve: Based on the coarse positioning results, automatically plan and control the mobile chassis to approach the valve;

[0025] S103, real-time height adjustment: When approaching the valve, the robot continuously collects valve height information and controls the lifting module to adjust the height to ensure that the robot's humanoid arms are within the appropriate height range;

[0026] S104, posture recognition and screwing execution: When the robot approaches the valve, it accurately recognizes the valve posture and adjusts the posture and strength of the two arms according to the recognition result to complete the screwing operation.

[0027] Furthermore, the step S101 of valve rough positioning includes the following steps:

[0028] S201, obtaining a valve image dataset based on a binocular vision camera;

[0029] S202, labeling the valve data set to generate a labeled data set;

[0030] S203, performing data enhancement processing on the labeled data set;

[0031] S204, using the YOLO algorithm to perform real-time recognition on the enhanced image to generate a two-dimensional bounding box of the valve;

[0032] S205 , generating three-dimensional spatial information of the valve through the depth map, and estimating the position of the valve in the robot space coordinate system.

[0033] Furthermore, the step S102 of approaching the valve includes the following steps:

[0034] S301. Acquire spatial layout data of the target area, construct a three-dimensional map of the environment, and dynamically update obstacle information. The spatial layout data includes the location, size, shape, distance between the valve and the obstacle, and ground conditions of the obstacle.

[0035] S302. Generate a path from the current position to the valve based on the spatial layout data using laser SLAM and multi-sensor fusion technology in combination with a path planning algorithm, wherein the path planning takes into account avoiding obstacles and physical limitations of the robot.

[0036] S303: Control the humanoid robot to move along the planned path, and monitor the movement state and make adjustments through a real-time feedback system.

[0037] Furthermore, the step S103 of real-time height adjustment includes the following steps:

[0038] S401, using a depth camera, laser radar, and ultrasonic sensor to collect real-time vertical distance information between the valve and the robot end tool, and obtain the target valve position and current relative height data;

[0039] S402: Calculate the difference between the current valve height and the robot's preset target grasping height to obtain a height adjustment amount, and simultaneously calculate a waist pitch angle adjustment amount to keep the waist upright;

[0040] S403, controlling the lifting module to adjust the height according to the height adjustment amount, and synchronously controlling the waist pitch module to adjust the pitch angle, and ensuring the stability of the robot's movement process through closed-loop control.

[0041] Furthermore, the identification of the valve posture in S104 includes the following steps:

[0042] S601, acquiring an image dataset of a target valve from different perspectives by moving a depth camera, and recording depth information;

[0043] S602: Record the internal and external camera parameter information of each image, and perform camera calibration and pose estimation;

[0044] S603, segmenting the valve image using the Mask R-CNN algorithm to obtain a labeling mask;

[0045] S604, registering the multi-view image data with the depth information to generate point cloud data of the scene;

[0046] S605, using the BundleSDF algorithm to perform 3D reconstruction on the point cloud data to construct a 3D model of the valve;

[0047] S606: Input the valve 3D model into the FoundationPose algorithm to obtain a 6D pose estimation of the valve from the camera's perspective;

[0048] S607: Combine the hand-eye calibration results to obtain the pose conversion relationship between the camera and the robotic arm. Calculate and record the relative pose of the current robotic arm end tool and the valve as the valve grasping posture.

[0049] Furthermore, the valve gripping in S104 includes the following steps:

[0050] S701. Acquire the current position of the valve in the camera coordinate system in real time from the depth camera, calculate the desired position of the end tool of the manipulator in the manipulator base coordinate system under the current environment, solve the desired angles of each joint of the manipulator through inverse kinematics of the manipulator, and move the manipulator to the desired position;

[0051] S702. When the robotic arm reaches the desired position, it uses the equipped end tool to grab or clamp the valve.

[0052] Furthermore, the valve screwing in step S104 includes the following steps: after the robot arm clamps the valve using the end tool, it adopts an adaptive admittance control algorithm to perform force tracking along the tangent direction of the valve edge to achieve the screwing of the valve;

[0053] The adaptive admittance control algorithm comprises the following steps:

[0054] S801: Obtain force data read by the six-dimensional force sensor, perform gravity compensation and coordinate transformation, and obtain force data in the robotic arm end coordinate system;

[0055] S802: Set a desired force, and calculate a desired trajectory estimate and an environmental stiffness estimate based on an adaptive admittance control algorithm;

[0056] S803: Based on the expected trajectory estimation and the environmental stiffness estimation, force tracking is performed to control the robotic arm to achieve valve tightening.

[0057] Beneficial effects: Compared with the prior art, the advantages of the present invention are:

[0058] This invention utilizes a humanoid robot-based automated valve control method. Through an intelligent control system, it can automatically identify and operate different types of valves, reducing the time and cost of manual operation. Compared to traditional manual operation, this invention significantly improves valve operation efficiency, particularly in pipeline systems that require frequent operation.

[0059] Robots replace manual labor in valve tightening operations, eliminating the safety hazards of manual labor in hazardous environments such as high temperatures, high pressures, and corrosive gas leaks. By reducing operators' direct exposure to hazardous environments, the risk of accidents and labor intensity are significantly reduced, ensuring operator safety.

[0060] By pre-specifying the valve's size and shape, the present invention can automatically identify and locate the desired valve type (e.g., ball valve, gate valve, butterfly valve, etc.). This adaptability allows the robot to be widely used in various piping systems and valves of different specifications, demonstrating its high versatility.

[0061] The robot of this invention can adapt to various complex operating environments, including those with unstable valve positions, limited operating space, and complex mechanical properties. It demonstrates strong environmental adaptability. By automatically adjusting its posture and height, the robot ensures safe and stable operation in complex environments, thus avoiding the challenges of traditional manual operation in these environments.

[0062] Based on the force-controlled admittance algorithm, the robotic arm has good flexibility during the valve screwing process and can be dynamically adjusted according to the actual contact force, avoiding damage to the valve or robotic arm due to excessive operating force, thereby improving system reliability. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 This is a schematic structural diagram of the global automatic valve screwing system of a humanoid robot according to the present invention;

[0064] The reference numerals in the figure are: 1, radar; 2, depth camera; 3, humanoid arms; 4, pitch module; 5, lifting module; 6, radar; 7, mobile chassis;

[0065] Figure 2 is a flow chart of the control method of the present invention;

[0066] Figure 3 This is a flow chart of the valve coarse positioning in the control method of the present invention;

[0067] Figure 4 This is a schematic diagram of the dynamic height adjustment of the lifting module of the present invention;

[0068] Figure 5 This is a flow chart for obtaining the valve gripping posture in the present invention;

[0069] Figure 6 It is a schematic diagram of the valve of the present invention;

[0070] Figure 7 It is a schematic diagram of the valve image after processing by the present invention;

[0071] Figure 8 It is a schematic diagram of a three-dimensional model of the valve of the present invention;

[0072] Figure 9 It is a flow chart of valve gripping in the present invention;

[0073] Figure 10 Schematic diagram of the valve grasping posture in the present invention;

[0074] Figure 11 is a schematic diagram of a grip valve in the present invention;

[0075] Figure 12 Schematic diagram of the valve screwing in the present invention

[0076] Figure 13 It is a schematic diagram of the adaptive admittance control algorithm of the present invention. DETAILED DESCRIPTION

[0077] The technical solution of the present invention is described in detail below with reference to the accompanying drawings, but the protection scope of the present invention is not limited to the embodiments.

[0078] Example 1: Figure 1 The humanoid robot's fully automated valve-tightening system, shown here, utilizes an industrial-grade mobile chassis as its walking mechanism, equipped with a high-precision LiDAR and a multi-wheel drive system, enabling flexible movement and navigation. The chassis supports laser SLAM (Simultaneous Localization and Mapping) technology, creating real-time maps in dynamic environments. The inertial measurement unit (IMU) enables path planning and precise positioning.

[0079] Mobile Chassis: The robot utilizes an industrial-grade mobile chassis as its walking mechanism, equipped with a high-precision LiDAR and multi-wheel drive system, enabling flexible movement and navigation. The chassis supports laser SLAM (Simultaneous Localization and Mapping) technology, which builds real-time maps in dynamic environments. It also uses an inertial measurement unit (IMU) for path planning and precise positioning.

[0080] Lifting Module: The mobile chassis connects to the robot's main body via the lifting module. This module utilizes an electric drive system to adjust the robot's main body's operating position within a range of heights. This device senses valve height in real time and automatically adjusts to ensure the robot's arms are precisely aligned with the target valve.

[0081] Waist Pitch Module: This module works in conjunction with the Lift Module to support torso pitch adjustment, ensuring the robot maintains a stable working posture in operating scenarios at various heights and angles. This module also features left and right rotation, allowing the robot to capture and manipulate surrounding objects in all directions.

[0082] Torso and Humanoid Arms: The torso module is equipped with humanoid arms, each with multiple degrees of freedom joints. Combined with high-precision servo motors and torque sensors, they enable flexible movement along complex paths. The ends of the arms are equipped with specialized gripping and screwing devices (end-of-arm tools), which automatically adjust gripping force and clamping angle based on the shape and size of different valves.

[0083] The sensor components include depth cameras, laser radars, torque sensors, and ultrasonic sensors. Among them, the depth cameras are configured on the head and chest of the robot to capture the shape, size, and posture information of the valve. The head depth camera provides a wide-angle field of view for environmental scanning and coarse positioning; the chest depth camera focuses on close-range identification and precise positioning of the target valve. The laser radar is configured on the mobile chassis and the robot body to detect obstacles in the environment and, combined with the SLAM algorithm, perform autonomous navigation and obstacle avoidance operations. The torque sensor is installed on the humanoid arm joints and end tools to monitor the torque and force conditions during operation in real time to ensure the safe operation of the valve. Ultrasonic sensors are configured in front and behind the mobile chassis for front and rear obstacle identification and distance perception.

[0084] Example 2: The humanoid robot global automatic valve screwing system used in Example 1 is used to control the method. The specific flow chart is as follows: Figure 1 As shown, it is mainly divided into the following 4 steps:

[0085] S101, valve coarse positioning

[0086] Through the visual component, the robot performs real-time image recognition and processing based on the YOLO (You Only Look Once) deep learning algorithm, thereby achieving rough positioning of the target valve.

[0087] S102, close to the valve

[0088] Based on the rough positioning results, the robot automatically plans and executes the motion path, accurately approaching the valve.

[0089] S103, real-time height adjustment

[0090] In the process of approaching the valve, the robot continuously collects real-time height information of the valve, and uses the robot lifting module to make dynamic adjustments to ensure that the robot arms are within the appropriate height range to ensure stable grasp of the valve.

[0091] S104, posture recognition and twisting execution

[0092] When the robot approaches the valve, it begins to accurately identify the valve posture and adjusts the posture and strength of its arms based on the identification results to complete the screwing operation.

[0093] In step S101, the robot uses the visual component and the YOLO (You Only Look Once) deep learning algorithm to perform real-time image recognition and processing, thereby achieving a rough positioning of the target valve. The specific flow chart is as follows: Figure 2 As shown, the following steps are included:

[0094] S201. Obtain a valve image dataset based on a binocular vision camera: Specifically select the source of the dataset, such as valves in industrial environments such as hydropower stations and petrochemical plants. Although these valves have relatively complex structures, they are more suitable for practical application scenarios.

[0095] S202, labeling the collected valve dataset to generate a labeled dataset: manually labeling using the LabelImg tool to mark the valve's bounding box and the position of its center point;

[0096] S203, enhance the labeled data: formulate a specific enhancement strategy and process the image through various methods such as rotation, flipping, scaling, and brightness adjustment to expand the data set;

[0097] S204, using the YOLO algorithm to process the images, identify the target valve in real time, and generate a preliminary two-dimensional bounding box;

[0098] S205 , generating three-dimensional spatial information of the target valve through the depth map, and estimating the position of the valve in the robot's spatial coordinate system by calculating the three-dimensional coordinates.

[0099] The target valve described in this embodiment does not specify a valve of a specific size or shape. By establishing a data set containing a variety of valve types, sizes, and shapes, the YOLO algorithm is trained so that the algorithm can automatically adapt to changes in different valves.

[0100] Step S102: The robot automatically plans and executes a motion path based on the rough positioning results to accurately approach the valve, including the following steps:

[0101] S301: Acquire spatial layout data of the target area

[0102] The robot uses a variety of sensors, including lidar, depth cameras, and ultrasonic sensors, to scan and acquire spatial layout information of the target area in real time. This process uses laser SLAM technology to construct a three-dimensional map of the surrounding environment and dynamically update obstacle information in the target area. This spatial layout data includes the relative position, size, shape, distance between the valve and the obstacle, and ground conditions of the obstacles. Furthermore, combined with an inertial navigation system, the robot can sense changes in its own position in real time and automatically adapt to dynamic changes in its surroundings. This data is represented as a point cloud and, after processing, serves as input for subsequent path planning, ensuring the robot's ability to navigate complex and dynamic environments.

[0103] S302: Determine the robot's planned path in the target area based on the spatial layout data

[0104] Based on the acquired spatial layout data, the robot uses laser SLAM and multi-sensor fusion technology, combined with a preset path planning algorithm, to generate an optimal path from its current position to the valve. During path planning, the robot not only considers the shortest path to the valve but also dynamically avoids obstacles and optimizes the robot's physical limitations (such as size, turning radius, maximum speed, acceleration, etc.). Combined with the high-resolution map generated by SLAM, the robot can adjust its path in real time to avoid new obstacles, and its inertial measurement unit ensures path execution accuracy. Path planning also considers the relative posture between the robot and the valve, allowing the robot to precisely align its grasping angle when approaching the valve.

[0105] S303: Control the humanoid robot to move along the planned path

[0106] Following the generated planned path, the robot moves step by step according to predetermined motion instructions through its chassis control system and joint drive systems. During movement, the robot monitors its motion status using real-time feedback systems such as lidar and IMU sensors to ensure accurate path execution. Using an inertial navigation system and real-time environmental perception system, the robot can adjust its posture, speed, and acceleration in real time and avoid obstacles. During the robot's execution, the feedback system makes real-time corrections to its current posture, speed, acceleration, and distance to obstacles to avoid errors or collisions, ensuring efficient and safe progress along the planned path. The robot also dynamically adjusts to environmental changes, optimizing its trajectory in real time to ensure it remains in the ideal motion state.

[0107] In step S103, while approaching the valve, the robot continuously collects real-time height information of the valve and uses the robot lifting module to make dynamic adjustments to ensure that the robot arms are within the appropriate height range to ensure stable grip of the valve.

[0108] S401, collect real-time height information between the valve and the robot

[0109] As the robot approaches the valve, it first uses depth cameras, lidar, ultrasonic sensors, and other devices to collect real-time vertical distance data between the valve and the robot's end effector. This information is processed using sensor fusion technology to obtain accurate valve position and current relative height data.

[0110] S402: Calculate the required height adjustment amount and simultaneously calculate the waist pitch angle

[0111] After collecting the height information, the robot calculates the difference between the current valve height and the robot's preset target grasping height through the control system to obtain the height difference that needs to be adjusted. At this time, the robot not only needs to adjust the lifting position of the chassis according to the height difference, but also needs to calculate the pitch angle of the waist to ensure that the robot's waist remains upright while the height is adjusted. Through the collaborative control algorithm, the robot synchronously calculates the pitch angle adjustment of the waist based on the adjustment amount of the lifting module and the target posture. Figure 3 shown.

[0112] S403, controlling the lifting module to perform dynamic height adjustment and synchronize waist pitch.

[0113] After calculating the required height adjustment and waist pitch angle, the robot control system will simultaneously control the chassis lifting module and the waist pitch mechanism. The lifting module will adjust the overall height of the robot according to the calculated height difference. At the same time, the robot monitors the real-time pitch angle of the waist through IMU and sensor feedback, and the waist pitch mechanism will be adjusted synchronously according to the target pitch angle to ensure that the robot always maintains a vertical waist during the adjustment process. This process uses a closed-loop control system to correct the adjustment amount through real-time feedback to ensure that the robot's movement process is stable and accurate. Figure 4 shown.

[0114] Step S104, when the robot approaches the valve, it begins to accurately identify the valve posture, and adjusts the posture and strength of the arms according to the identification results to complete the screwing operation, including the following steps: S501, valve grasping posture acquisition; S502, grasping the valve; S503, valve screwing.

[0115] Step S501: The specific steps for obtaining the valve grasping posture are as follows:

[0116] S601, multi-view acquisition of valve dataset

[0117] By moving the depth camera to obtain images of the target operating valve at different viewing angles, ensure that the images cover all parts of the valve and avoid occlusion. At the same time, record the depth information, such as Figure 6 The valve used in this embodiment is just a simple example. In actual use, a valve dataset can be established based on the size and shape of the specific valve to be operated, and then a valve model can be established to obtain the valve posture.

[0118] S602: Recording camera parameter information

[0119] Record the camera's extrinsic and intrinsic parameters for each image, and perform camera calibration and pose estimation.

[0120] S603, valve segmentation marking

[0121] The Mask R-CNN algorithm is used to segment the collected valve image data to obtain the annotation mask. Figure 7 shown.

[0122] S604, point cloud generation

[0123] The multi-view image data is registered with the depth information to generate high-quality point cloud data of the scene.

[0124] S605, Model Reconstruction

[0125] Convert the image, depth information, camera pose and other information into the BundleSDF input format, use the BundleSDF algorithm to perform 3D reconstruction, and build a high-quality 3D valve model. Figure 8 shown.

[0126] S606, valve posture acquisition

[0127] Input the constructed valve 3D model into the FoundationPose algorithm to obtain the valve 6D pose estimation from the camera perspective. .

[0128] S607, valve gripping posture acquisition

[0129] Obtain the pose conversion relationship between the camera and the robotic arm through hand-eye calibration At the same time, combined with the end tool currently equipped with the robot arm, by pre-teaching the robot arm to move to a reasonable valve grasping position, the valve position in the camera coordinate system obtained in S106 is used , calculate and record the relative position between the current end of the robotic arm and the valve , the calculation formula is as follows:

[0130]

[0131] in, 、 They are 、 The inverse matrix of It can be solved by the forward kinematics of the robot arm.

[0132] Step S502, Figure 9 This is a valve gripping flow chart of an embodiment of the present invention. The specific process is as follows:

[0133] S701, obtain the desired joint angle of the robot arm to grasp the valve and move

[0134] Based on the above steps of S60, Figure 10 As shown, the 6D posture of the valve within the current camera view can be identified , use the FoundationPose algorithm to obtain the current pose of the valve in the camera coordinate system in real time , based on the pre-calculated desired position of the end of the manipulator and the valve , calculate the expected position of the end of the manipulator in the manipulator base coordinate system in the current environment , the calculation formula is as follows:

[0135]

[0136] Then use the inverse kinematics of the robotic arm to solve the expected angles of each joint of the robotic arm , and move the robot arm to the desired position, such as Figure 11 shown.

[0137] S702, robotic arm grasps valve

[0138] When the robot arm reaches the desired position, use the equipped end tool to grab or clamp the valve to ensure that the end tool and the valve do not slide relative to each other when the valve is screwed. Figure 12 shown.

[0139] Step S503, the specific steps of valve tightening are as follows: after the robot arm uses the end tool to clamp the valve, it quickly uses the adaptive admittance algorithm to control and track the force along the tangent direction of the valve edge clamped by the robot arm to achieve valve tightening.

[0140] In the embodiment of the present invention, the adaptive admittance control is as follows: Figure 13 As shown, the specific control steps are as follows:

[0141] S801. Obtain force data read by the six-dimensional force sensor , the influence of the six-dimensional force sensor and the end gravity is eliminated through gravity compensation, and the force data based on the six-dimensional force sensor coordinate system is converted into force data based on the robotic arm end coordinate system by using coordinate transformation.

[0142] S802. Setting Expectation , is brought into the adaptive admittance controller of the present invention to calculate the desired trajectory estimate and environmental stiffness estimation , the calculation formula is as follows:

[0143]

[0144]

[0145]

[0146] in, 、 To estimate the initial value, 、 is an adaptive parameter. In this embodiment, 450000、

[0147] S803. Perform force tracking based on the estimated reference trajectory and environmental stiffness to achieve valve tightening. The calculation formula is as follows:

[0148]

[0149]

[0150]

[0151]

[0152]

[0153] Among them, m is the mass coefficient, b is the damping coefficient, and k is the stiffness coefficient, all of which are constant matrices. is the adaptive control coefficient, and its update rate is as follows:

[0154]

[0155]

[0156] in, , to prevent the denominator from being 0, is the update rate.

[0157] The actual motion trajectory of the robotic arm calculated by the adaptive admittance controller This is input into the robot controller, which performs force tracking to achieve valve turning.

[0158] As described above, although the present invention has been shown and described with reference to specific preferred embodiments, it should not be construed as limiting the present invention itself. Various changes may be made to it in form and detail without departing from the spirit and scope of the present invention as defined in the appended claims.

Claims

1. A control method for a global automatic valve screwing system of a humanoid robot, characterized in that: The global automatic screw valve system includes: A mobile chassis equipped with a multi-wheel drive system for movement and navigation; a lifting module, connecting the mobile chassis and the robot body, and configured to automatically adjust the working position of the robot body based on the sensed valve height information; A waist pitch module, linked to the lifting module, is configured to support the pitch adjustment and left-right rotation of the robot body to adjust the working posture; The robot body is provided with humanoid arms, wherein the humanoid arms have joints with multiple degrees of freedom; The end tool is configured at the end of the humanoid arms, is used to grasp and screw the valve, and can automatically adjust the grasping force and clamping angle according to the shape and size of the valve; The sensor assembly is used for environmental perception, valve height and posture information collection, and operating torque information collection. The sensor assembly includes: a head depth camera, configured on the robot head, for environmental scanning and valve coarse positioning; a chest depth camera, configured on the robot chest, for close-range identification and precise positioning of the target valve; LiDAR, deployed on the mobile chassis and robot body, detects obstacles in the circumferential direction. Ultrasonic sensors, deployed on the front and rear sides of the mobile chassis, detect obstacles in the front and rear directions. Torque sensors, deployed on the humanoid arm joints and end-of-line tools, monitor torque and force during operation in real time. The control system includes: a valve recognition module for performing environmental perception and recognition based on the YOLO deep learning algorithm to achieve rough positioning of the target valve; a path planning module for automatically planning and controlling the mobile chassis to approach the valve based on the rough positioning results; a height adjustment module for controlling the lifting module to adjust the height in real time based on the collected valve height information; and a screwing operation module for identifying the valve posture and controlling the humanoid arms and end-of-line tool to complete the screwing operation based on the identified valve posture; The control method comprises the following steps: S101: Valve coarse positioning: Based on the YOLO deep learning algorithm, the sensor component acquires images and performs real-time image recognition and processing to achieve the coarse positioning result of the target valve; S102, approaching the valve: Based on the coarse positioning results, automatically plan and control the mobile chassis to approach the valve; S103, real-time height adjustment: When approaching the valve, the robot continuously collects valve height information and controls the lifting module to adjust the height to ensure that the robot's humanoid arms are within the appropriate height range; S104, posture recognition and screwing execution: When the robot approaches the valve, it accurately recognizes the valve posture and adjusts the posture and strength of the two arms according to the recognition result to complete the screwing operation; The S102 approaching the valve includes the following steps: S301. Acquire spatial layout data of the target area, construct a three-dimensional map of the environment, and dynamically update obstacle information. The spatial layout data includes the location, size, shape, distance between the valve and the obstacle, and ground conditions of the obstacle. S302. Generate a path from the current position to the valve based on the spatial layout data using laser SLAM and multi-sensor fusion technology in combination with a path planning algorithm, wherein the path planning takes into account avoiding obstacles and physical limitations of the robot. S303, controlling the humanoid robot to move along the planned path, and monitoring the motion state and making adjustments through a real-time feedback system; The real-time height adjustment in S103 includes the following steps: S401, using a depth camera, laser radar, and ultrasonic sensor to collect real-time vertical distance information between the valve and the robot end tool, and obtain the target valve position and current relative height data; S402: Calculate the difference between the current valve height and the robot's preset target grasping height to obtain a height adjustment amount, and simultaneously calculate a waist pitch angle adjustment amount to keep the waist upright; S403, controlling the lifting module to adjust the height according to the height adjustment amount, and synchronously controlling the waist pitch module to adjust the pitch angle, ensuring the stability of the robot's movement process through closed-loop control; The identification of the valve posture in S104 includes the following steps: S601, acquiring an image dataset of a target valve from different perspectives by moving a depth camera, and recording depth information; S602: Record the internal and external camera parameter information of each image, and perform camera calibration and pose estimation; S603, segmenting the valve image using the Mask R-CNN algorithm to obtain a labeling mask; S604, registering the multi-view image data with the depth information to generate point cloud data of the scene; S605, using the BundleSDF algorithm to perform 3D reconstruction on the point cloud data to construct a 3D model of the valve; S606: Input the valve 3D model into the FoundationPose algorithm to obtain a 6D pose estimation of the valve from the camera's perspective; S607. Combined with the hand-eye calibration results, the posture conversion relationship between the camera and the robotic arm is obtained, and the relative posture of the current robotic arm end tool and the valve is calculated and recorded as the valve grasping posture.

2. The control method of the humanoid robot global automatic valve screwing system according to claim 1, characterized in that: When the robot approaches the valve, it uses sensor components including laser radar, depth camera, and ultrasonic sensor to scan and obtain spatial layout information of the target area in real time. Based on the obtained spatial layout information, it uses laser SLAM and multi-sensor fusion technology, combined with a preset path planning algorithm, to generate the optimal path from the current position to the valve. According to the generated path, the robot moves step by step according to the predetermined motion instructions by controlling the joints of the mobile chassis and the robot body; based on the collected valve height information, the real-time height difference between the robot and the valve is calculated based on the fusion data of the depth camera, lidar and ultrasonic sensor, and the lifting module and waist pitch module are adjusted synchronously for dynamic adjustment.

3. The control method of the humanoid robot global automatic valve screwing system according to claim 1, characterized in that: The step S101 of valve rough positioning includes the following steps: S201, obtaining a valve image dataset based on a binocular vision camera; S202, labeling the valve data set to generate a labeled data set; S203, performing data enhancement processing on the labeled data set; S204, using the YOLO algorithm to perform real-time recognition on the enhanced image to generate a two-dimensional bounding box of the valve; S205 , generating three-dimensional spatial information of the valve through the depth map, and estimating the position of the valve in the robot space coordinate system.

4. The control method of the humanoid robot global automatic valve screwing system according to claim 1, characterized in that: The valve grasping in S104 includes the following steps: S701. Acquire the current position of the valve in the camera coordinate system in real time from the depth camera, calculate the desired position of the end tool of the manipulator in the manipulator base coordinate system under the current environment, solve the desired angles of each joint of the manipulator through inverse kinematics of the manipulator, and move the manipulator to the desired position; S702. When the robotic arm reaches the desired position, it uses the equipped end tool to grab or clamp the valve.

5. The control method of the humanoid robot global automatic valve screwing system according to claim 1, characterized in that: The valve tightening in step S104 includes the following steps: After the robot arm uses the end tool to clamp the valve, it uses the adaptive admittance control algorithm to track the force along the tangent direction of the valve edge to achieve valve screwing; The adaptive admittance control algorithm comprises the following steps: S801: Obtain force data read by the six-dimensional force sensor, perform gravity compensation and coordinate transformation, and obtain force data in the robotic arm end coordinate system; S802: Set a desired force, and calculate a desired trajectory estimate and an environmental stiffness estimate based on an adaptive admittance control algorithm; S803: Based on the expected trajectory estimation and the environmental stiffness estimation, force tracking is performed to control the robotic arm to achieve valve tightening.

Images