Robotic grasping method, apparatus, computing device, and readable storage medium

Abstract

The application discloses a robot grabbing method and device, a computing device and a computer readable storage medium. The method comprises the following steps: obtaining joint state space information of a robot; inputting the joint state space information into a pre-trained reinforcement learning collaborative decision network; outputting chassis control instructions and mechanical arm grabbing instructions in parallel by the reinforcement learning collaborative decision network; controlling the movement of the mobile chassis according to the chassis control instructions, and controlling the mechanical arm to perform a grabbing action according to the mechanical arm grabbing instructions, so that the mobile chassis and the mechanical arm are controlled in parallel. In this way, the grabbing efficiency of the robot in a dynamic environment can be improved.

Description

Technical Field

[0001] This application relates to the field of robotics, and in particular to a robot grasping method, apparatus, computing device, and readable storage medium. Background Technology

[0002] Currently, wheeled humanoid robots are widely used in industrial sorting, household services, and logistics handling to perform target grasping tasks. However, existing robot grasping control schemes still have significant shortcomings in dealing with complex dynamic environments where both the robot itself and the target object are moving. In practical applications, robot grasping decisions are mostly based on the chassis first locating and stabilizing before the robotic arm plans the grasping action, or on feedback adjustment based on visual servoing. However, these methods cannot well adapt to complex dynamic scenarios, especially when grasping on moving conveyor belts and / or picking up objects while they are moving.

[0003] Therefore, how to ensure the efficient and stable execution of grasping tasks in a dynamic environment where both the robot and the target object are moving has become an urgent technical problem to be solved. Summary of the Invention

[0004] The purpose of this application is to provide a robot grasping method, device, computing device, and storage medium that can improve the robot's grasping efficiency in dynamic environments.

[0005] To achieve the above objectives: In a first aspect, a robot grasping method includes: acquiring joint state space information of the robot, the joint state space information including the robot's own dynamic state information, the target object's motion state information, and environmental constraint state information; inputting the joint state space information into a pre-trained reinforcement learning collaborative decision-making network, and having the reinforcement learning collaborative decision-making network output chassis control commands and robotic arm grasping commands in parallel, wherein the chassis control commands and robotic arm grasping commands are output based on the robot's own dynamic state information, the target object's motion state information, and the environmental constraint state information; The mobile chassis is controlled to move according to chassis control commands, and the robotic arm is controlled to perform grasping actions according to robotic arm grasping commands, so that the mobile chassis and robotic arm can be controlled in parallel and in a coordinated manner.

[0006] In one embodiment, the steps of constructing a reinforcement learning collaborative decision network include: constructing a physical simulation environment with a dynamic training scenario; executing a grasping test task in the physical simulation environment and obtaining sample joint state space information during the execution of the grasping task; calculating a score value for the task completion effect of the grasping test task based on a reward function; and training an Actor-Critic network using a proximal policy optimization algorithm, with the sample joint state space information as input and the score value as the optimization objective, and updating the network parameters to obtain the reinforcement learning collaborative decision network.

[0007] In one implementation, training the Actor-Critic network and updating its parameters includes: performing a grabbing test task in a physical simulation environment using a self-game approach and updating the network parameters thereon.

[0008] In one embodiment, the method further includes: determining a reward function based on a successful grasping reward, a grasping quality reward, a smoothness reward, a collision reward, and an efficiency reward; and calculating a score for the task completion effect of the grasping test task based on the reward function, including: adding a first point to the successful grasping reward when the current grasping test task indicates that the object has been successfully grasped and lifted to a safe height; adding a second point to the grasping quality reward for each step when the current grasping test task indicates that the grasping posture is stable and the object has not slipped; deducting a third point from the smoothness reward for each step when the current grasping test task indicates that the robot arm collides with the environment and / or the robot collides with itself and / or the chassis collides with an obstacle when the current grasping test task indicates that the collision reward deducts a fourth point for each step; and deducting a fifth point from the efficiency reward for each step when the current grasping test task indicates that the robot is completely stationary and has not completed the grasping.

[0009] In one embodiment, acquiring the robot's joint state space information includes: acquiring the robot's own dynamic state information in real time through an encoder and an inertial measurement unit, the robot's own dynamic state information including the real-time speed and real-time acceleration of the mobile chassis and the instantaneous velocity vector of the robotic arm's end effector; and acquiring the motion state information of a target object through an RGB-D depth camera, the target object's motion state information including the target object's three-dimensional position, attitude, and relative velocity vector relative to the robot in the robot's base coordinate system. Environmental constraint information is obtained through preset maps and / or sensor detection and / or kinematic model calculation. The environmental constraint information includes obstacle distribution, workspace boundaries and / or the proximity of robotic arm joint limits.

[0010] In one embodiment, the reinforcement learning collaborative decision-making network outputs chassis control commands and robotic arm grasping commands in parallel, including: the reinforcement learning collaborative decision-making network outputs the desired chassis velocity vector as the chassis control command, the chassis velocity vector including linear velocity components and angular velocity components; the reinforcement learning collaborative decision-making network outputs the desired motion direction and speed of the robotic arm end effector in the task space, and / or the grasping execution trigger signal as the robotic arm grasping command.

[0011] In one embodiment, the mobile chassis is controlled to move according to chassis control commands, and the robotic arm is controlled to perform a grasping action according to robotic arm grasping commands, so that the mobile chassis and the robotic arm are controlled in parallel and in coordination, including at least one of the following: when the mobile chassis moves and the target object moves, the robot is controlled to complete the grasping while maintaining the movement of the mobile chassis; when the mobile chassis is stationary and the target object moves, the robotic arm is controlled to track and grasp according to the movement trajectory of the target object; when the mobile chassis and the robotic arm move simultaneously, the mobile chassis and the robotic arm are controlled to complete the grasping at the intersection of their movements.

[0012] Secondly, embodiments of this application provide a robot grasping device, comprising: an acquisition module for acquiring joint state space information of the robot, the joint state space information including the robot's own dynamic state information, the target object's motion state information, and environmental constraint state information; an output module for inputting the joint state space information into a pre-trained reinforcement learning collaborative decision-making network, wherein the reinforcement learning collaborative decision-making network outputs chassis control commands and robotic arm grasping commands in parallel, wherein the chassis control commands and robotic arm grasping commands are output based on the robot's own dynamic state information, the target object's motion state information, and the environmental constraint state information; and a control module for controlling the movement of the mobile chassis according to the chassis control commands and controlling the robotic arm to perform grasping actions according to the robotic arm grasping commands, so that the mobile chassis and the robotic arm are controlled in parallel and collaboratively.

[0013] Thirdly, embodiments of this application provide a computing device, specifically including: a processor and a memory for storing executable instructions; and the steps of implementing a robot grasping method when the processor runs a computing program.

[0014] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program, wherein when the computer program is processed and executed, it implements the steps of a robot grasping method.

[0015] This application provides a robot grasping method, apparatus, computing device, and computer-readable storage medium, comprising: acquiring joint state space information of the robot, the joint state space information including the robot's own dynamic state information, the target object's motion state information, and environmental constraint state information; inputting the joint state space information into a pre-trained reinforcement learning collaborative decision network, the reinforcement learning collaborative decision network outputting chassis control commands and robotic arm grasping commands in parallel, wherein the chassis control commands and robotic arm grasping commands are output based on the robot's own dynamic state information, the target object's motion state information, and the environmental constraint state information; controlling the movement of the mobile chassis according to the chassis control commands, and controlling the robotic arm to perform grasping actions according to the robotic arm grasping commands, so that the mobile chassis and the robotic arm are controlled in parallel and collaboratively. Thus, this embodiment of the application constructs a joint state space by integrating the robot's own dynamic state, the target object's motion state, and environmental constraint information. It then utilizes a reinforcement learning collaborative decision-making network to achieve parallel output of commands from the chassis and the robotic arm. This effectively solves the problems of disconnected grasping decisions and motion, single-dimensional state perception, and lack of predictive coordination between the chassis and the arm in existing technologies. This improves the robot's grasping efficiency, environmental adaptability, and system robustness in dynamic environments. In other words, this embodiment enables the robot to perform on-the-fly object retrieval in complex scenarios where both the robot and the target object are moving, eliminating the serial waiting time required for stopping and then grasping. It is particularly suitable for scenarios with high real-time and dynamic response requirements, such as mobile conveyor belt sorting and on-the-fly object retrieval, ensuring efficient and stable execution of grasping tasks in complex dynamic environments. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating the robot grasping method provided in an embodiment of this application.

[0017] Figure 2 This is a schematic diagram of the structure of the gripping device for the robot provided in an embodiment of this application.

[0018] Figure 3 A schematic diagram of the structure of a computing device provided in an embodiment of this application.

[0019] Processor 410, memory 411, network interface 412, bus system 413. Detailed Implementation

[0020] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0021] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, components, features, and elements with the same names in different embodiments of this application may have the same meaning or different meanings, the specific meaning of which must be determined by its interpretation in that specific embodiment or further in conjunction with the context of that specific embodiment.

[0022] It should be understood that although the terms first, second, third, etc., may be used herein to describe various information, such information should not be limited to these terms. These terms are used only to distinguish information of the same type from one another. For example, without departing from the scope of this document, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if," as used herein, can be interpreted as "when," "when," or "in response to determination." Furthermore, as used herein, the singular forms "a," "an," and "the" are intended to also include the plural forms unless the context indicates otherwise. It should be further understood that the terms "comprising," "including," indicate the presence of the stated feature, step, operation, element, component, item, kind, and / or group, but do not exclude the presence, occurrence, or addition of one or more other features, steps, operations, elements, components, items, kinds, and / or groups. The terms "or" and "and / or" as used herein are to be interpreted as inclusive, or mean any one or any combination thereof. Therefore, "A, B, or C" or "A, B, and / or C" means "any one of the following: A; B; C; A and B; A and C; B and C; A, B, and C". Exceptions to this definition will only occur if the combination of elements, functions, steps, or operations is inherently mutually exclusive in some way.

[0023] It should be understood that although the steps in the flowcharts of this application's embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least a portion of the sub-steps or stages of other steps.

[0024] It should be noted that step designations such as S101 and S102 are used in this document for the purpose of more clearly and concisely describing the corresponding content, and do not constitute a substantial limitation on the order. In specific implementation, those skilled in the art may execute S102 first and then S101, etc., but these should all be within the protection scope of this application.

[0025] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.

[0026] In the following description, the use of suffixes such as "module," "part," or "unit" to denote elements is solely for the purpose of illustrative purposes and has no specific meaning in itself. Therefore, "module," "part," or "unit" may be used interchangeably.

[0027] It is worth noting that with the popularization of industrial automation and home service robots, wheeled humanoid robots have become the core carrier for performing dynamic grasping tasks (such as mobile conveyor belt sorting, mobile object handling, and dynamic assembly). During the grasping process, the robot needs to integrate multi-source heterogeneous data in real time from chassis encoders, inertial measurement units, and vision sensors, including the robot's own motion state (velocity, acceleration), the real-time pose and motion trend of the target object, and information such as obstacle distribution and workspace boundaries in the environment. Real-time fusion and intelligent decision-making of this data, and the generation of efficient collaborative control commands between the chassis and the robotic arm, have become key to improving the success rate and efficiency of dynamic grasping. Currently, traditional grasping methods heavily rely on a serial logic of "first stationary, then grasping" or a fixed trajectory based on teaching data. They cannot effectively perceive and respond to the dynamic coupling relationship between the robot's own motion, the target object's movement, and changes in environmental constraints during the grasping process. This not only results in low efficiency in scenarios where the target object moves or the robot needs to move continuously, easily leading to grasping failure or task delays, but also lacks accuracy and reliability in handling multi-dimensional decision-making needs such as when to grasp and how to coordinate.

[0028] To address the aforementioned issues, this application proposes a robot grasping method that allows the robot to automatically acquire joint state-space information including its own dynamic state, the target object's motion state, and environmental constraints. Through a pre-trained reinforcement learning collaborative decision-making network, it outputs chassis control commands and robotic arm grasping commands in parallel, achieving parallel and collaborative control of chassis movement and grasping actions. This enables a leap from grasping while stationary to grasping while in motion, and from passive reaction to proactive prediction, improving the grasping efficiency, environmental adaptability, and system robustness of wheeled humanoid robots in dynamic environments. It can be widely applied to key tasks such as mobile conveyor belt sorting, mobile object handling, dynamic assembly, and autonomous grasping in complex scenarios.

[0029] like Figure 1 As shown, this application provides a robot grasping method. This robot grasping method can be implemented using software and / or hardware, for example, it can be executed by a control system installed inside the robot. The robot grasping method provided in this application includes the following steps: Step S101: Obtain the robot's joint state space information, which includes the robot's own dynamic state information, the target object's motion state information, and the environmental constraint state information.

[0030] Among them, the robot's own dynamic state information refers to the data that reflects the real-time kinematic characteristics of the robot body.

[0031] Among them, the motion state information of the target object refers to the real-time pose of the target object relative to the robot and its motion trend.

[0032] Among them, environmental constraint state information refers to the external conditions that restrict the robot's movement and grasping operations.

[0033] Step S102: Input the joint state space information into the pre-trained reinforcement learning collaborative decision network, and the reinforcement learning collaborative decision network outputs chassis control commands and robotic arm grasping commands in parallel. The chassis control commands and robotic arm grasping commands are output based on the robot's own dynamic state information, the target object's motion state information, and the environmental constraint state information.

[0034] The reinforcement learning collaborative decision-making network is a neural network model trained using deep learning algorithms, used to achieve end-to-end mapping from environmental states to robot actions. For example, the reinforcement learning collaborative decision-making network can be an Actor-Critic network.

[0035] The Actor network receives joint state space information and uses it as input to directly execute chassis control commands and robotic arm grasping commands.

[0036] The Critic network also receives joint state space information and outputs a state value function to evaluate the quality of the current state, assisting the Actor network in updating parameters during the training phase.

[0037] Parallel output can be understood as the reinforcement learning collaborative decision-making network simultaneously generating chassis control commands and robotic arm grasping commands during a single forward propagation, rather than outputting one command first and then waiting for execution before outputting the other. For example, the output layer of the reinforcement learning collaborative decision-making network is designed with two branches: one branch outputs chassis control commands, and the other branch outputs robotic arm grasping commands. The outputs of both branches are generated simultaneously at the same time step, i.e., within the same inference frame, and are independent of each other. After receiving the two commands, the robot sends the chassis speed command to the chassis's motor driver and the robotic arm grasping command to the robotic arm's joint controller, both executing in parallel in physical time. This parallel output approach breaks the traditional sequential dependency of first stabilizing the chassis and then planning the arm, allowing the robot to perform robotic arm grasping actions while the chassis is moving, thus achieving collaborative control.

[0038] The chassis control commands and robotic arm grasping commands are output based on the robot's own dynamic state information, the target object's motion state information, and environmental constraint state information. This can be understood as emphasizing the causal relationship between the input state and output action of the reinforcement learning collaborative decision-making network. Each set of control commands is a comprehensive decision generated after simultaneously considering the above three types of information. For example, based on the robot's own dynamic state information, the reinforcement learning collaborative decision-making network must know the robot's current speed and acceleration when deciding whether to accelerate, decelerate, or turn the chassis (e.g., if the chassis is already moving at a high speed, the network may choose to maintain the speed to match the target; if the chassis acceleration is too large and causes instability, the network may output a deceleration command). Simultaneously, the robotic arm grasping command also adjusts the target speed based on the current speed of the end effector to avoid excessive relative speed leading to grasping failure. For example, based on the target object's motion state information, the network needs to know the target object's position, orientation, and relative velocity vector. For example, if the relative velocity indicates that the target is moving away from the robot, the reinforcement learning collaborative decision-making network might output a chassis tracking speed command to reduce the relative velocity; if the target posture indicates that the grasping surface is not aligned, the robotic arm command will include posture adjustments. Based on environmental constraint state information, the reinforcement learning collaborative decision-making network can also consider obstacle distribution, workspace boundaries, and joint limit proximity. For example, if there is an obstacle ahead, the chassis control command will output steering or deceleration to avoid the obstacle; if a robotic arm joint is close to its limit, the grasping command will restrict the joint's movement direction to avoid collision with the limit. In summary, through reinforcement learning training, the reinforcement learning collaborative decision-making network automatically learns how to select the optimal chassis and arm collaborative action under different combinations of dynamic states, target states, and environmental constraints. For example, in the aforementioned conveyor belt grasping scenario, the reinforcement learning collaborative decision-making network, based on the state of "chassis speed synchronized with the conveyor belt, relative speed close to zero, and sufficient workspace," outputs the parallel instruction "maintain chassis speed and synchronized tracking of the robotic arm's end effector." In the aforementioned object-receiving scenario, based on the state of "chassis stationary, person approaching, and obstacle present," the network outputs the parallel instruction "chassis remaining stationary and arm extension matching speed." Thus, this end-to-end decision-making based on a joint state space allows the robot's grasping behavior to move beyond rigid one-to-one execution rules, adaptively generating collaborative instructions based on the rapidly changing dynamic environment, thereby improving the success rate and efficiency of grasping.

[0039] Step S103: Control the movement of the mobile chassis according to the chassis control command, and control the robotic arm to perform the grasping action according to the robotic arm grasping command, so that the mobile chassis and the robotic arm can be controlled in parallel and in coordination.

[0040] In summary, the embodiments of this application construct a joint state space that includes the robot's own dynamic state, the target object's motion state, and the environmental constraint state, and utilize a pre-trained reinforcement learning collaborative decision-making network to output chassis control commands and robotic arm grasping commands in parallel. This effectively solves the problems of decision-making and motion separation, single state perception, and poor dynamic response in existing grasping methods, and achieves efficient, stable, and adaptive robot dynamic grasping.

[0041] In some implementations, step S101, namely obtaining the robot's joint state space information, includes: The robot's dynamic state information is collected in real time through encoders and inertial measurement units. The robot's dynamic state information includes the real-time speed and acceleration of the mobile chassis and the instantaneous velocity vector of the end effector of the robotic arm. The motion state information of the target object is obtained by detecting with an RGB-D depth camera. The motion state information of the target object includes its three-dimensional position, orientation and relative velocity vector with respect to the robot in the robot's base coordinate system. Environmental constraint state information is obtained through preset maps and / or sensor detection and / or kinematic model calculation. The environmental constraint state information includes obstacle distribution, workspace boundaries and / or the proximity of robotic arm joint limits.

[0042] An encoder is a sensor that converts rotary or linear displacement into digital pulse signals. For example, an encoder can be mounted on the wheel hub or motor shaft of a mobile chassis to measure the rotational angle and speed of the wheels, thereby calculating the chassis's real-time speed and acceleration.

[0043] An inertial measurement unit (IMU) is an electronic device that integrates an accelerometer and a gyroscope to measure the angular velocity and linear acceleration of an object in three-dimensional space. Here, the IMU is fixed to the robot chassis to assist encoder data and provide more accurate chassis velocity, acceleration information, and attitude angles.

[0044] The mobile chassis is the lower mobile platform of the wheeled humanoid robot, typically equipped with omnidirectional or differential wheels, responsible for the robot's translational and rotational movements in the environment. The mobile chassis also features encoders and an IMU (Integrated Measurement Unit) to provide real-time feedback on its motion status.

[0045] An end effector is a device installed at the very end of a robotic arm that directly contacts the target object; examples include grippers, suction cups, or dexterous hands. The instantaneous velocity vector of the end effector reflects its direction and speed of motion in space and is crucial to the success rate of grasping.

[0046] Real-time velocity refers to the robot's speed and direction of motion at the current instant. Here, the linear velocity of the chassis in the X and Y directions and the angular velocity around the Z axis are obtained by sampling at a high frequency (e.g., 100Hz) using an encoder and IMU.

[0047] Real-time acceleration refers to the rate of change of velocity over time, reflecting the dynamic response of the robot's motion. For example, it can be directly obtained through a differential encoder or the accelerometer of an IMU, and is used to determine whether the robot is in a state of emergency stop, emergency start, or constant speed.

[0048] The instantaneous velocity vector is a vector with magnitude and direction. For example, the instantaneous velocity vector of the chassis includes linear velocity and angular velocity components, while the instantaneous velocity vector of the robotic arm end effector is its three-dimensional linear velocity in spatial motion.

[0049] An RGB-D depth camera is a visual sensor that can simultaneously acquire color (RGB) images and depth images. For example, an RGB-D camera is used to detect the color texture (aiding recognition) and three-dimensional spatial position (using depth data) of a target object.

[0050] The robot base coordinate system is a three-dimensional reference coordinate system fixed to the robot's mobile chassis. It typically has the chassis center as the origin, with the X-axis pointing forward, the Y-axis pointing left, and the Z-axis pointing upward. All target positions and orientations perceived by the robot body are transformed into this robot base coordinate system for unified decision-making.

[0051] Here, pose refers to the orientation of the target object in space relative to the robot's base coordinate system. For example, the 3D pose of the target object can be estimated using feature matching from an RGB-D image or a deep learning model.

[0052] The relative velocity vector with respect to the robot refers to the velocity of the target object in the robot's base coordinate system, which can be calculated by subtracting the velocity of the robot's chassis in the world coordinate system from the velocity of the target object in the world coordinate system. For example, the motion trend of the target object relative to the robot is obtained through inter-frame target pose difference, thereby predicting the timing of the grasping action.

[0053] The preset map is a pre-built static environment model, such as a two-dimensional grid map or a three-dimensional point cloud map of a hospital floor, including the location information of fixed obstacles, such as walls, pillars, and fixed equipment. For example, the preset map is used to quickly obtain the distribution of obstacles and the boundaries of the workspace.

[0054] Sensor detection can be understood as using real-time sensors on the robot, such as lidar, ultrasonic sensors, and contact sensors, to dynamically perceive the surrounding environment. Complementing the preset map, sensor detection can detect dynamic obstacles, such as pedestrians, mobile carts, and temporarily changing scenes.

[0055] The kinematic model is an equation describing the mathematical relationship between the robot's joint angles and the position of the end effector of the robotic arm. For example, the real-time pose of the end effector is calculated based on the current angle of each joint using the forward kinematic model of the robotic arm; at the same time, the inverse kinematic model is used to determine whether the target point is reachable, thereby obtaining the degree of proximity of the robotic arm's joints.

[0056] Here, obstacle distribution can be understood as the location, shape, and contour information of all obstacles in the robot's surrounding environment, including both static and dynamic obstacles. For example, a local occupancy grid map is generated by fusing a pre-set map with real-time LiDAR point cloud data for obstacle avoidance decision-making.

[0057] The workspace boundary is the boundary of the entire spatial range that the robotic arm's end effector can reach under physical constraints. For example, the boundary information is obtained in advance through the kinematic calibration of the robotic arm and updated in real time during runtime in conjunction with the position of the robot chassis.

[0058] The proximity of a robotic arm joint to its mechanical hard or soft safety limit can be understood as the remaining angular margin between the current angle of each joint and its mechanical hard or soft safety limit. For example, if a joint can rotate from -90° to +90° and its current angle is +85°, then the proximity is 5°. Exemplarily, the proximity of the robotic arm joint to its mechanical hard or soft safety limit can be used as a penalty term input into the reward function to reduce joint collision limits.

[0059] Thus, by fusing multi-source heterogeneous data such as encoders, IMUs, RGB-D cameras, preset maps, sensor detection, and kinematic models in real time to obtain a joint state space of the robot's own dynamics, target motion, and environmental constraints, the success rate of dynamic grasping is improved. Specifically, by introducing the robot's own real-time velocity and / or real-time acceleration as well as the relative velocity vector of the target object, the decision network can predict when to grasp, reducing grasping failures or collisions caused by motion lag. It also enhances environmental adaptability, as the fusion of preset maps and real-time sensor detection allows for rapid planning using prior information and can cope with dynamic obstacles and temporary changes, making it suitable for high-dynamic scenarios such as hospitals and factories. Furthermore, it ensures operational safety by incorporating the proximity of the robotic arm's key limits into the state space, effectively preventing damage to the robot due to joint over-limits. At the same time, obstacle distribution information is used for obstacle avoidance.

[0060] In some implementations, a reinforcement learning collaborative decision-making network outputs chassis control commands and robotic arm grasping commands in parallel, including: The chassis velocity vector output by the reinforcement learning collaborative decision-making network is used as the chassis control command. The chassis velocity vector includes linear velocity components and angular velocity components. The desired motion direction and speed of the robotic arm's end effector in the task space, and / or the grasping execution trigger signal, are output by the reinforcement learning collaborative decision network as the robotic arm's grasping command.

[0061] The desired chassis velocity vector can be understood as the Actor network of the reinforcement learning collaborative decision network outputting a three-dimensional continuous motion vector as a chassis control command, i.e., the desired chassis velocity vector, which is used to control the movement of the moving chassis on the horizontal plane.

[0062] The linear velocity component refers to the translational velocity of the chassis along the X-axis (or forward direction) and the translational velocity along the Y-axis (or lateral direction) in its own coordinate system. For an omnidirectional chassis, the two linear velocity components can be controlled independently to achieve translation in any direction.

[0063] The angular velocity component refers to the angular velocity of the chassis rotating around its own Z-axis or vertical axis, which is used to control the steering of the chassis.

[0064] In this approach, the end effector of the robotic arm, in the task space, directly specifies the desired three-dimensional translational velocity vector of the end effector (such as a gripper or suction cup) within the robot's base coordinate system or world coordinate system. This command does not involve the specific angles of each joint; instead, it is decomposed into joint velocity commands in real time through the inverse kinematics of the robotic arm or a velocity-level controller (such as a pseudo-inverse Jacobian matrix), driving the end effector to move in the desired velocity direction and magnitude. This method is suitable for scenarios involving continuous tracking of moving targets.

[0065] The grasping execution trigger signal is a discrete control signal (such as a Boolean value or a single-step trigger pulse). When the network determines that the grasping conditions are met at the current moment (such as the end effector having reached the vicinity of the target object and the relative velocity being close to zero), it outputs the trigger signal (e.g., setting it to 1), instructing the gripper to close or the suction cup to activate, completing the grasping action. This signal is usually used in conjunction with continuous motion commands: before triggering, the network outputs the tracking speed of the end effector; after triggering, it executes the grasping and may switch arm movements (such as lifting the object).

[0066] In actual operation, the reinforcement learning collaborative decision-making network simultaneously outputs the aforementioned chassis velocity vector and robotic arm grasping instructions (including continuous velocity instructions and / or discrete trigger signals) at each decision time step. After receiving these instructions, the robot control system sends the chassis instructions to the chassis motor driver and the robotic arm instructions to the robotic arm controller. Both are executed in parallel without blocking each other.

[0067] Thus, by concretizing the output of the reinforcement learning collaborative decision-making network into the desired chassis velocity vector (including linear velocity / angular velocity components) and the task space motion velocity and grasping trigger signal of the robotic arm end effector, this embodiment of the present application can maintain the end-to-end decision-making advantages of reinforcement learning while having a reasonable output motion space design, taking into account the simplicity, real-time performance and collaborative capabilities of control, which is conducive to engineering deployment and migration.

[0068] In some implementations, the mobile chassis is controlled to move according to chassis control commands, and the robotic arm is controlled to perform a grasping action according to robotic arm grasping commands, so that the mobile chassis and the robotic arm are controlled in parallel and in a coordinated manner, including at least one of the following: When the mobile chassis moves and the target object moves, the robot is controlled to complete the grasping while maintaining the movement of the mobile chassis. When the mobile chassis is stationary and the target object is moving, the control robot arm tracks and grasps the target object according to its movement trajectory. When the mobile chassis and the robotic arm move simultaneously, control the mobile chassis and the robotic arm to complete the grasping at the intersection of their movements.

[0069] Thus, in scenarios where both the chassis and the target are moving, the robot completes the grasping action while the chassis remains in motion, eliminating the sequential waiting time from stopping to grasping and then starting in traditional methods, thereby improving task execution efficiency. This is particularly suitable for high-paced scenarios such as mobile conveyor belt sorting and dynamic assembly. In scenarios where the chassis is stationary and the target is moving, the robotic arm tracks and grasps the target based on its trajectory, enabling the robot to grasp moving targets using only arm movement when chassis movement is unsuitable, enhancing scenario adaptability and safety. In scenarios where both the chassis and the arm are moving simultaneously, the robot controls the chassis and arm to complete the grasping action at their intersection point, achieving spatiotemporal coordination between the chassis and the arm. This allows the robot to handle more complex dynamic intersection tasks (such as grasping moving objects from a mobile platform), further expanding its operational capabilities. Therefore, by covering three typical dynamic grasping scenarios, this approach allows the robot to flexibly select or switch collaborative modes according to actual working conditions, thereby maximizing task efficiency and adaptability while ensuring a high grasping success rate, truly achieving parallel collaborative control of the chassis and the robotic arm.

[0070] In some implementations, the steps of constructing a reinforcement learning decision-making collaborative network include: Construct a physical simulation environment with dynamic training scenarios; Perform a grasping test task in a physical simulation environment and acquire sample joint state space information during the execution of the task; The score for the completion effect of the crawling test task is calculated based on the reward function; A proximal policy optimization algorithm is adopted, using the joint state space information of the samples as input and the score value as the optimization objective to train the Actor-Critic network and update the network parameters to obtain a reinforcement learning collaborative decision-making network.

[0071] Among them, the physical simulation environment can be a high-fidelity physical simulation environment.

[0072] The dynamic training scenario can be a training scenario that includes mobile transmission and / or a dynamic training scenario that includes randomly moving objects.

[0073] Thus, this embodiment of the application reduces the damage caused by trial and error in real robots by training in a physical simulation environment, and can accelerate training in parallel, thereby improving the safety and efficiency of training. Furthermore, by setting dynamic training scenarios such as moving conveyor belts and randomly moving objects, the network learns to cope with various dynamic situations and adapt to real complex environments, thereby improving the generalization ability of training. At the same time, automatic scoring based on reward functions reduces the standard cost of teaching data and the limitations of generalization, so that training does not require manual teaching. Finally, with the joint state space as input and the scoring value as the optimization objective, the chassis-arm collaborative strategy is learned end-to-end, improving the grasping success rate and dynamic efficiency, thereby making the training strategy fully optimized.

[0074] In some implementations, training the Actor-Critic network and updating its parameters includes: performing a grabbing test task in a physical simulation environment using a self-game approach and updating the network parameters thereon.

[0075] Self-play is a reinforcement learning training strategy in which an agent generates training data by interacting with its own historical version or a copy of its current strategy. For example, in a physical simulation environment, a grasping test task no longer uses a preset fixed motion pattern, such as a fixed conveyor belt speed. Instead, the robot's current strategy interacts adversarially or collaboratively with its previously saved self-strategy or the dynamic target behavior derived from it. For instance, the trajectory of the target object can be controlled by another agent based on its historical strategy, which attempts to evade grasping, thus forcing the robot to continuously adapt to more complex and challenging dynamic scenarios. Through this self-playing evolutionary training, the robot continuously plays against itself, gradually improving the robustness and generalization ability of its strategy.

[0076] Thus, in this implementation method, the self-game approach automatically generates diverse dynamic training scenarios through continuous competition between the agent and its own historical strategies. This effectively reduces network overfitting to fixed motion patterns and significantly improves the robustness and generalization ability of the collaborative grasping strategy. Simultaneously, this approach introduces a natural learning mechanism, prompting the robot to continuously explore better chassis-arm collaborative solutions. This enhances adaptability to complex dynamic environments and the optimality of strategies while reducing manual design costs.

[0077] In some implementations, the method further includes: The reward function is determined based on successful capture reward, capture quality reward, smoothness reward, collision reward, and efficiency reward. The score for the completion of the crawling test task is calculated based on the reward function, including: When the current grab test task indicates that the object has been successfully grabbed and lifted to a safe height, the first point is added as a reward for successful grabbing; When the current grasp test task indicates that the grasping posture is stable and the object is not sliding, set the grasping quality bonus to add a second point for each step; When the current capture test task instruction experiences a sudden acceleration change, the smoothness bonus is reduced by the third point for each step. When the current grasping test task indicates that the robotic arm collides with the environment and / or the robot itself collides with and / or the chassis hits an obstacle, the collision reward is set to deduct the fourth point value for each step. When the current grasping test task instructs the robot to remain completely still and has not completed the grasping, the efficiency bonus is reduced by the fifth point for each step.

[0078] The successful grasping reward can be understood as a one-time positive reward given when the robot successfully grasps the target object and raises it to a preset safe height, such as 10cm above the conveyor belt plane, to encourage the network to learn and complete the full grasping task.

[0079] Among them, the grasping quality reward can be understood as the continuous positive reward given at each time step after a successful grasp, if the end effector of the robotic arm remains stable, for example, if the force sensor feedback that the gripping force is within a reasonable range and the object does not slip, to encourage the robot to maintain a stable grasping state and reduce the chance of the object falling midway.

[0080] Among them, the smoothness reward can be understood as a negative penalty given to sudden acceleration changes (such as sudden stops, sudden starts, and violent turns) that occur during the movement of the chassis or robotic arm. It is used to suppress unstable motion, make the robot's movements smoother, and reduce mechanical shock.

[0081] The collision reward can be understood as a large negative penalty given when the robot experiences any collision event, including collisions between the robotic arm and the environment, self-collisions of the robot, and collisions between the chassis and obstacles. This penalty is used to guide the network to learn obstacle avoidance behavior.

[0082] Efficiency reward can be understood as a negative penalty given at each time step when the robot is completely stationary, the chassis speed and the speed of the robotic arm joints are both zero, and the grasping has not yet been completed. This is used to encourage the robot to maintain dynamic grasping and reduce ineffective waiting.

[0083] It should be noted that each step refers to a control cycle of the reinforcement learning collaborative decision-making network, i.e., a decision time step. In actual operation, the network acquires the state and outputs the action at a fixed frequency, such as 50Hz, i.e., every 0.02 seconds. Each time step corresponds to one decision and one reward calculation.

[0084] Thus, the implementation of this application decomposes the grasping task into five quantifiable reward or penalty dimensions: success, quality, smoothness, collision, and efficiency. By combining real-time feedback at each step with a one-time reward, the reinforcement learning network can obtain fine-grained, multi-dimensional optimization signals, thereby learning a dynamic grasping strategy that balances success rate, stability, and motion smoothness more efficiently, while effectively suppressing collisions and ineffective static behavior.

[0085] Specifically, the system components for training reinforcement learning decision collaboration are as follows: (1) Wheeled mobile platform: Omnidirectional mobile chassis, equipped with high-precision encoder and inertial measurement unit, to provide real-time feedback of the chassis's instantaneous velocity vector (v_x, v_y, ω_z) and acceleration vector (a_x, a_y, α_z) in the world coordinate system. (2) Redundant degree of freedom robotic arm: Equipped with a lightweight robotic arm, providing real-time feedback on the angle, angular velocity and torque of each joint. (3) Visual perception module: RGB-D depth camera, used for target object detection and pose estimation, outputting the target object's three-dimensional position (x, y, z), pose (roll, pitch, yaw) and velocity vector (v_obj_x, v_obj_y, v_obj_z) relative to the robot. (4) Edge computing platform: Deploy a trained reinforcement learning decision network to process multimodal inputs and output control commands in real time.

[0086] Specific control methods include: Step S1 (Dynamic State Embedding and Joint State Space Construction): Construct a multi-dimensional joint state space information S. It can be understood that S contains three key types of information: 1) Robot dynamic state D_robot: instantaneous velocity vector of chassis, instantaneous acceleration vector of chassis, instantaneous velocity vector of robotic arm end effector (innovation: explicitly modeling the robot's own kinematic state as decision input) 2) Target object motion state T_obj: The three-dimensional position, orientation, and relative velocity vector of the target object in the robot's base coordinate system (Innovation: Introducing the target's motion trend, rather than just its current position). 3) Environmental constraint state E_env: obstacle distribution, workspace boundary, and proximity of robotic arm joint limits. Step S2 (Reward Function Design): Design a reward function R to guide the agent in learning a dynamic grasping strategy. Instead of using sub-goal decomposition, directly evaluate the task completion performance. 1) R = R_success + R_quality + R_smoothness + R_collision + R_inefficiency, where R represents the score calculated by the reward function, R_success represents the reward for successful capture, R_quality represents the reward for capture quality, R_smoothness represents the reward for smoothness, R_collision represents the reward for collision, and R_inefficiency represents the reward for efficiency. 2) R_success: Successfully grabbing an object and lifting it to a safe height grants a positive reward (+10.0). 3) R_quality: If the grasping posture is stable (based on force sensor feedback) and the object does not slip, a continuous positive reward is given (+0.1 / step). 4) R_smoothness: Encourages smooth movement of the chassis and arms, and imposes a small penalty (-0.01 / step) on sudden acceleration changes to avoid sudden stops and starts; 5) R_collision: A large penalty (-5.0) is imposed for collisions between the robotic arm and the environment, self-collisions of the robot, and collisions between the chassis and obstacles. 6) R_inefficiency: If the robot is completely stationary and has not completed the grasping, an efficiency penalty (-0.05 / step) is imposed to encourage dynamic grasping.

[0087] It should be noted that, here, the first, second, third, fourth, and fifth scores are 10.0, 0.1, -0.01, -5.0, and -0.05, respectively; in other embodiments, the first, second, third, fourth, and fifth scores may also be other scores, and no limitation is made here.

[0088] Step S3 (Training a Collaborative Decision Network Based on Deep Reinforcement Learning): In a high-fidelity physics simulation environment, a training scenario is constructed that includes dynamic targets (moving conveyor belts, randomly moving objects). An Actor-Critic network is trained using a proximal policy optimization algorithm, with joint state space information S as input. The network output is a composite action command A. 1) Chassis motion command: Desired chassis velocity vector (v_x_cmd, v_y_cmd, ω_z_cmd) 2) Arm grasping command: The desired direction and speed of movement of the end effector in the task space, or the grasping execution trigger signal. 3) Commands for the chassis and the arm are output in parallel through the same network, achieving true collaborative control rather than serial planning.

[0089] Step S4 (Dynamic Grasping Execution and Online Adaptation): In actual operation, the perception module updates the state S in real time, and the trained decision network directly outputs coordinated commands between the chassis and the arm. The robot can perform grasping in the following dynamic scenarios: 1) Scenario A (Robot moves, target moves): The robot chassis maintains the tracking speed, and the arm completes the grasping action during the movement. 2) Scenario B (Robot stationary, target in motion): The chassis remains stationary, and the arm tracks and grasps the target based on its movement trajectory. 3) Scenario C (Dual-motion collaboration): The chassis and the arm move simultaneously to complete the grasping at the intersection point.

[0090] Based on this, taking dynamic grasping on a moving conveyor belt as an example, in a sorting scenario, a wheeled humanoid robot needs to grasp packages from a moving conveyor belt with a speed of 0.5m / s.

[0091] 1) Initialization: The robot receives the grasping command, moves towards the conveyor belt, and the chassis maintains a speed of 0.5m / s in the same direction as the conveyor belt.

[0092] 2) State perception: The RGB-D camera detects the three-dimensional position (0.8m, 0.2m, 0.5m) and relative velocity vector (0m / s, 0.5m / s, 0m / s) wrapped in the robot's base coordinate system (because the robot chassis has been synchronized, the relative velocity is zero); the IMU feeds back the current velocity of the chassis (0.5m / s, 0, 0).

[0093] 3) Collaborative Decision-Making: The reinforcement learning network takes the joint state space as input and determines that it is currently in a "dynamic synchronization" state with sufficient workspace for the robotic arm. The network outputs in parallel: 4) Chassis command: Maintain current speed (0.5m / s, 0, 0) 5) Arm command: The end effector moves synchronously along the conveyor belt at a speed of 0.5 m / s, while simultaneously moving towards the package location. 6) Execution and Results: While maintaining movement, the robotic arm synchronously tracks the package and completes the grasping process in a relatively stationary state. Subsequently, the chassis uses gear shifting to transport the package to the designated location. Compared to the traditional "stop-grab-start" solution, the time for a single task is reduced by 45%.

[0094] Based on this, taking the example of receiving an item from a moving person, the robot needs to take the item from the person walking towards it.

[0095] 1) State perception: Visual tracking identifies a human hand approaching at a speed of 1.2 m / s and predicts its trajectory; the robot chassis remains stationary.

[0096] 2) Collaborative Decision-Making: The network determines that the current situation is not suitable for chassis movement (to avoid collisions with people) and outputs instructions: - Chassis command: Remain stationary -Arm command: Extend the distal end forward at a speed matching the approach speed of a human hand, and perform a grasping motion at the intersection point. 3) Execution: The robotic arm continuously fine-tunes its trajectory during the extension process to match the speed of the human hand, and completes a compliant grasp at the moment of contact (no special flexible controller is required, and the grasping timing is learned by the network).

[0097] Thus, based on the specific embodiments described above, this application constructs a joint state space encompassing the robot's own dynamics, target motion, and environmental constraints. It then utilizes a reinforcement learning collaborative decision-making network to output chassis and robotic arm commands in parallel, achieving parallel collaborative control of chassis motion and grasping actions. In a moving conveyor belt grasping scenario, the robot can complete the grasping action without stopping, reducing single task time by 45%. In a moving object receiving scenario, the chassis remains stationary while the arm dynamically matches the target speed, achieving smooth and safe grasping. This method can adapt to various dynamic scenarios such as "robot moving - target moving," "robot stationary - target moving," and "dual-motion collaboration." It requires no manual teaching; the network autonomously learns the grasping timing and collaborative strategies. Simultaneously, collision penalties and smoothness penalties in the reward function ensure operational safety and motion smoothness.

[0098] This application also provides a robot gripping device, please refer to [link to relevant documentation]. Figure 2 The device includes: The acquisition module 21 is used to acquire the robot's joint state space information, which includes the robot's own dynamic state information, the target object's motion state information, and the environmental constraint state information. Output module 22 is used to input joint state space information into a pre-trained reinforcement learning collaborative decision network, and the reinforcement learning collaborative decision network outputs chassis control commands and robotic arm grasping commands in parallel. The chassis control commands and robotic arm grasping commands are output based on the robot's own dynamic state information, the target object's motion state information, and the environmental constraint state information. The control module 23 is used to control the movement of the mobile chassis according to the chassis control command, and to control the robotic arm to perform the grasping action according to the robotic arm grasping command, so as to enable the mobile chassis and the robotic arm to be controlled in parallel and in a coordinated manner.

[0099] In some embodiments, the device further includes: The building module is used to build a physical simulation environment with dynamic training scenarios. The sample acquisition module is used to perform grasping test tasks in a physical simulation environment and acquire sample joint state information during the execution of the grasping task; The calculation module is used to calculate the score of the task completion effect of the crawling test task based on the reward function; The training module is used to train the Actor-Critic network using a proximal policy optimization algorithm, taking the sample joint state space information as input and the score value as the optimization objective, and updating the network parameters to obtain a reinforcement learning collaborative decision-making network.

[0100] In some implementations, the training module is also used to: perform a grabbing test task in a physical simulation environment and update network parameters using a self-game approach.

[0101] In some embodiments, the device further includes: The determination module is used to determine the reward function based on successful capture reward, capture quality reward, smoothness reward, collision reward, and efficiency reward. The calculation module is also used for: When the current grab test task indicates that the object has been successfully grabbed and lifted to a safe height, the successful grab reward is increased by the first point. When the current grasp test task indicates that the grasping posture is stable and the object is not sliding, set the grasping quality bonus to add a second point for each step; When the current capture test task instruction experiences a sudden acceleration change, the smoothness bonus is reduced by the third point for each step. When the current grasping test task indicates that the robotic arm collides with the environment and / or the robot itself collides with and / or the chassis hits an obstacle, the collision reward is set to deduct the fourth point value for each step. When the current grasping test task instructs the robot to remain completely still and has not completed the grasping, the efficiency bonus is reduced by the fifth point for each step.

[0102] In some implementations, the output module 22 is further configured to: The chassis velocity vector output by the reinforcement learning collaborative decision-making network is used as the chassis control command. The chassis velocity vector includes linear velocity components and angular velocity components. The desired motion direction and speed of the robotic arm's end effector in the task space, and / or the grasping execution trigger signal, are output by the reinforcement learning collaborative decision network as the robotic arm's grasping command.

[0103] In some implementations, the control module 23 is also used for at least one of the following: When the mobile chassis moves and the target object moves, the robot is controlled to complete the grasping while maintaining the movement of the mobile chassis. When the mobile chassis is stationary and the target object is moving, the control robot arm tracks and grasps the target object according to its movement trajectory. When the mobile chassis and the robotic arm move simultaneously, control the mobile chassis and the robotic arm to complete the grasping at the intersection of their movements.

[0104] It should be noted that the description of the robot's grasping device above is similar to the description of the robot's grasping method above, and the beneficial effects of the same method will not be repeated. For technical details not disclosed in the robot grasping device embodiments of the present invention, please refer to the description of the robot grasping method embodiments of the present invention.

[0105] Based on the same inventive concept as the foregoing embodiments, this application provides a computing device, such as... Figure 3 As shown, the computing device includes: a processor 410 and a memory 411 storing computer programs; wherein, Figure 3 The processor 410 shown in the diagram does not indicate that there is only one processor 410, but only indicates the positional relationship of the processor 410 relative to other devices. In practical applications, there can be one or more processors 410; similarly, Figure 3 The memory 411 shown in the diagram has the same meaning, that is, it is only used to indicate the positional relationship of memory 411 relative to other devices. In practical applications, there can be one or more memories 411. When the processor 410 runs the computer program, the above-described robot grasping method is implemented.

[0106] The computing device may also include at least one network interface 412. The various components of the computing device are coupled together via a bus system 413. It is understood that the bus system 413 is used to implement communication between these components. In addition to a data bus, the bus system 413 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 3 The general designated all buses as Bus System 413.

[0107] The memory 411 can be volatile or non-volatile, or both. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), ferromagnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM); magnetic surface memory can be disk storage or magnetic tape storage. Volatile memory can be random access memory (RAM), used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), SyncLink Dynamic Random Access Memory (SLDRAM), and Direct Rambus Random Access Memory (DRRAM).The memory 411 described in the embodiments of this application is intended to include, but is not limited to, these and any other suitable types of memory.

[0108] The memory 411 in this embodiment is used to store various types of data to support the operation of the computing device. Examples of this data include: any computer programs used to operate on the computing device, such as operating systems and applications; contact data; phonebook data; messages; pictures; videos, etc. The operating system includes various system programs, such as the framework layer, core library layer, driver layer, etc., used to implement various basic services and handle hardware-based tasks. Applications can include various applications, such as media players, browsers, etc., used to implement various application services. Here, the program implementing the method of this embodiment can be included in the application.

[0109] Based on the same inventive concept as the foregoing embodiments, this embodiment also provides a computer-readable storage medium storing a computer program. The computer-readable storage medium can be a magnetic random access memory (FRAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM), etc.; it can also be various devices including one or any combination of the above-mentioned memories, such as mobile phones, computers, tablet devices, personal digital assistants, etc. When the computer program stored in the computer-readable storage medium is executed by a processor, it implements the above-described robot grasping method. For the specific steps implemented when the computer program is executed by the processor, please refer to [link to relevant documentation]. Figure 1 The description of the illustrated embodiments will not be repeated here.

[0110] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0111] In this document, the terms “comprising,” “including,” or any other variations thereof are intended to cover non-exclusive inclusion, which includes not only the elements listed but also other elements not expressly listed.

[0112] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A grasping method of a robot, characterized by, include: The robot's joint state space information is obtained, which includes the robot's own dynamic state information, the target object's motion state information, and the environmental constraint state information. The joint state space information is input into a pre-trained reinforcement learning collaborative decision-making network, which then outputs chassis control commands and robotic arm grasping commands in parallel. The chassis control commands and robotic arm grasping commands are output based on the robot's own dynamic state information, the target object's motion state information, and the environmental constraint state information. The mobile chassis is controlled to move according to the chassis control command, and the robotic arm is controlled to perform a grasping action according to the robotic arm grasping command, so that the mobile chassis and the robotic arm are controlled in parallel and in coordination.

2. The method of claim 1, wherein, The steps for constructing the reinforcement learning collaborative decision-making network include: Construct a physical simulation environment with dynamic training scenarios; The grasping test task is executed in the physical simulation environment and the joint state space information of the samples during the execution of the grasping task is obtained; The score for the completion effect of the crawling test task is calculated based on the reward function; The Actor-Critic network is trained and its parameters are updated using a proximal policy optimization algorithm. The input is the joint state space information of the samples, and the optimization objective is the score value. The network is then used to obtain the reinforcement learning collaborative decision-making network.

3. The method of claim 2, wherein, The training of the Actor-Critic network and updating of network parameters include: performing the grabbing test task in a physical simulation environment using a self-game approach and updating the network parameters.

4. The method of claim 2, wherein, The method further includes: The reward function is determined based on successful capture reward, capture quality reward, smoothness reward, collision reward, and efficiency reward. The calculation of the score for the task completion effect of the crawling test task based on the reward function includes: When the current grab test task indicates that the object has been successfully grabbed and lifted to a safe height, the successful grab reward is increased by the first point. When the current grasping test task indicates that the grasping posture is stable and the object is not sliding, the grasping quality reward is set to add a second point value for each step; When the current capture test task indication experiences a sudden acceleration change, the smoothness reward is reduced by the third point for each step. When the current grasping test task indicates that the robotic arm collides with the environment and / or the robot collides with itself and / or the chassis hits an obstacle, the collision reward is reduced by the fourth point for each step. When the current grasping test task indicates that the robot is completely stationary and has not completed the grasping, the efficiency reward is reduced by the fifth point for each step.

5. The method of claim 1, wherein, The acquisition of the robot's joint state space information includes: The robot's dynamic state information is collected in real time through encoders and inertial measurement units. The robot's dynamic state information includes the real-time speed and acceleration of the mobile chassis and the instantaneous velocity vector of the end effector of the robotic arm. The motion state information of the target object is obtained by detecting with an RGB-D depth camera. The motion state information of the target object includes its three-dimensional position, orientation and relative velocity vector with respect to the robot in the robot's base coordinate system. Environmental constraint state information is obtained through preset maps and / or sensor detection and / or kinematic model calculation. The environmental constraint state information includes obstacle distribution, workspace boundaries and / or the proximity of robotic arm joint limits.

6. The method of claim 1, wherein, The parallel output of chassis control commands and robotic arm grasping commands by the reinforcement learning collaborative decision-making network includes: The chassis velocity vector output by the reinforcement learning collaborative decision-making network is used as the chassis control command. The chassis velocity vector includes linear velocity components and angular velocity components. The reinforcement learning collaborative decision-making network outputs the desired motion direction and speed of the robotic arm end effector in the task space, and / or the grasping execution trigger signal as the robotic arm grasping command.

7. The method of claim 1, wherein, The step of controlling the movement of the mobile chassis according to the chassis control command and controlling the robotic arm to perform a grasping action according to the robotic arm grasping command, so as to enable the mobile chassis and the robotic arm to be controlled in parallel and in coordination, includes at least one of the following: When the mobile chassis moves and the target object moves, the robot is controlled to complete the grasping while maintaining the movement of the mobile chassis. When the mobile chassis is stationary and the target object is moving, the control robot arm tracks and grasps the target object according to its movement trajectory. When the mobile chassis and the robotic arm move simultaneously, control the mobile chassis and the robotic arm to complete the grasping at the intersection of their movements.

8. A gripping device of a robot, characterized in that include: The acquisition module is used to acquire the robot's joint state space information, which includes the robot's own dynamic state information, the target object's motion state information, and the environmental constraint state information. The output module is used to input the joint state space information into a pre-trained reinforcement learning collaborative decision network, and the reinforcement learning collaborative decision network outputs chassis control commands and robotic arm grasping commands in parallel. The chassis control commands and robotic arm grasping commands are output based on the robot's own dynamic state information, the target object's motion state information, and the environmental constraint state information. The control module is used to control the movement of the mobile chassis according to the chassis control command, and to control the robotic arm to perform a grasping action according to the robotic arm grasping command, so that the mobile chassis and the robotic arm can be controlled in parallel and in a coordinated manner.

9. A computing device, comprising: include: Processor and memory for storing executable instructions; When the processor runs the computer program, it implements the steps of the grasping method of the robot according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The device contains a computer program that, when processed and executed, implements the steps of the grasping method of the robot according to any one of claims 1-7.

Images