A robot pushing-grasping collaborative operation method based on spatial relationship functionality detection

Abstract

The application discloses a robot pushing-grasping collaborative operation method based on spatial relationship utility detection and relates to the technical field of robots. The method comprises the following steps: a pushing-grasping collaborative operation system is constructed by fusing a perception module, a decision module and an execution module; the perception module inputs multimodal images; utility values of each position in an operation space are generated by means of an utility map generation network AMGN; and pushing-grasping actions are output in combination with a strategy network, so that autonomous operation of the robot in an unstructured environment is realized. The application improves overall efficiency and stability and provides a more reliable solution for complex grasping tasks.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of robot technology, and more particularly to a robot pushing-grasping collaborative operation method based on spatial relationship utility detection. BACKGROUND

[0002] With the development of industrial automation and intelligent technology, mechanical arms are widely used in complex environments such as part assembly, medical operation and human-computer interaction. General robot operation skills are one of the important characteristics that robots must have to enter human society and serve humans in the future and achieve large-scale popularization and application. Robot operation skills refer to the ability of a robot to operate a specific object in the environment within a limited time based on its own sensing, perception, decision-making, planning and control capabilities, so that it can reach the target state from the initial state. At present, robot operation skill learning has become a hot spot in the field of robot research. Pushing and grasping are the two most important basic abilities in robot operation skills, which cover the core needs of most object manipulation tasks. These two skills are not only the main way for robots to interact with the environment physically, but also the basis for achieving complex operation tasks. They can be applied to various industrial scenarios, such as picking and placing, bin picking and conveyor tracking.

[0003] With the increasing complexity of grasping scenarios, the object manipulation ability of general robots in unstructured environments is one of the keys to their widespread application. Although significant progress has been made in single grasping in specific scenarios in recent years, when faced with highly cluttered unstructured environments, there are still many challenges, and relying solely on collision-free grasping often makes it difficult to achieve effective operation. Pushing actions play an important role in this case. Through pushing actions, robots can separate closely spaced objects and rearrange the spatial layout, thereby providing feasible grasping space for the effector. The combination of pushing and grasping not only enables robots to cope with complex unstructured environments, but also significantly improves the success rate of robots in completing tasks in cluttered environments. Achieving efficient collaboration of pushing and grasping skills will become a key direction for pushing robot technology to be more widely applied in practical scenarios.

[0004] In recent years, data-driven methods have been widely applied in robot grasping. Although data-driven methods have shown significant advantages in efficiency, robustness, and scalability, their dependence on large amounts of high-quality data, high computational resource requirements, and the black-box nature of the model remain challenges that need to be addressed. In practical applications, the acquisition of labeled data is often costly, and the generalization ability for grasping unknown objects is poor. In complex unstructured scenarios, object pose estimation requires more efficient and reliable solutions. The size of the experimental data is limited and is affected by semantic bias, which can cause the model to overfit during training, thereby reducing its adaptability to new scenarios. Many existing data-driven grasping methods have weak real-time response capabilities in dynamic environments, making it difficult to quickly adapt to environmental changes, which can affect the stability and execution efficiency of the operation in complex real-world scenarios.

[0005] To overcome the limitations of purely data-driven methods, reinforcement learning is a machine learning method based on trial-and-error mechanisms that learns from the interaction between the agent and the environment to optimize decision-making strategies to achieve goals. In the robot grasping task, reinforcement learning-driven methods allow the agent to perceive the environmental state, choose actions based on the strategy, and obtain reward feedback from the environment by executing the action, thereby gradually learning the optimal strategy to complete the grasping task. This method does not rely on large amounts of labeled data, but rather acquires grasping experience through autonomous exploration and learning, significantly improving its adaptability in unstructured and dynamic environments.

[0006] To effectively address the problems of occlusion, position uncertainty, and environmental dynamics of target objects in unstructured environments, the robot needs to make decisions based on spatial relationship information during task execution. The robot needs to determine the position of the target object and adjust the path of grasping or pushing operations based on the distribution of other objects in the environment. Spatial relationships help the robot understand the interaction between the task goal and the environment, thereby executing optimal operations. Utility map generation networks based on deep learning models can automatically extract spatial features from environmental images and learn the spatial layout and interaction between objects. These information provides effective guidance for the robot to make decisions in unstructured environments.

[0007] Therefore, a robot pushing-grasping collaborative operation method based on spatial relationship utility detection is proposed to solve the difficulties existing in the prior art, which is a problem that needs to be solved by those skilled in the art. SUMMARY

[0008] Therefore, the application provides a robot pushing-grasping collaborative operation method based on spatial relationship utility detection, which specifies a grasping object by applying a mask of a target object, projects data onto a 3D point cloud, and thus establishes a depth perception of the environment, wherein the data needs to be re-projected and calibrated during the projection process to ensure the accurate correspondence of the depth information and the color information; a utility map generation network is designed to obtain target object information and generate potential robot execution actions; the efficient utility map generated by the network is stored in a second experience pool to train a policy network, and the policy network learns to obtain an optimal action policy by sampling the utility map in the experience pool. Test experiments in different scenes show that the proposed model can perform optimally in different test scenes, and compared with the existing most advanced method, the grasping success rate is improved, and the learning performance of the robot in grasping the target in a complex environment is improved. The innovative scheme provides solid technical support for realizing real-time and accurate control in the robot field.

[0009] In order to achieve the above object, the application provides the following technical scheme:

[0010] A robot pushing-grasping collaborative operation method based on spatial relationship utility detection comprises the following steps:

[0011] By constructing a pushing-grasping collaborative operation system integrating a perception module, a decision module and an execution module, the perception module inputs multi-modal images, a utility value of each position in an operation space is generated by a utility map generation network AMGN, and a pushing-grasping action is output in combination with a policy network, so as to realize autonomous operation of the robot in an unstructured environment.

[0012] Optionally, multi-modal images are collected by an RGB-D camera on the robot, and the multi-modal images comprise an RGB image, a depth image and a mask image.

[0013] Optionally, the mask image is generated by a RefineNet structure, and a target segmentation image mask is output, which is used to filter the image background and the target object region and extract a region in combination with position attention information.

[0014] Optionally, the output of the utility map is a two-dimensional mapping graph, and the formula is as follows:

[0015]

[0016] wherein, is a set composed of HxW-dimensional real matrices, and F i,j represents a utility value at a position (i, j), which is used to represent the feasibility probability of executing a pushing or grasping operation at the current position, and the utility map is generated by the following network:

[0017]

[0018] where I rgb , I depth , and I mask are RGB image, depth image and mask image respectively, and PE is position encoding.

[0019] Optionally, the utility map generation network comprises a residual feature extraction module and a direction perception module CACNN, and two-dimensional position encoding is introduced:

[0020]

[0021] where pos x and pos y are the positions of the pixel in the image, i and j are integers in the range of d / 4, and d is the dimension of the model. The encoding of each pixel position of the input image is added to each pixel value of the image:

[0022] where I x is the feature vector of the original input image at position (pos y , pos x ), and I y contains the semantic information extracted at position (pos i , pos i ). Without changing the size of the original image, I i carries the original image data and spatial position information.

[0024] Optionally, the strategy network takes the utility map F and the current image state as input, outputs the action selection value Q, and is trained in combination with the reinforcement learning method. The loss function is as follows:

[0025]

[0026] where s i+1 is the current state, a i+1 is the action, γ is the discount factor, Q' is the target value network, N is the number of sampled state-action pairs, r * is the immediate reward, s a is the next state, and a' is the action under the next state s i,j .

[0027] Optionally, the push-grab action is generated by a push-grab collaborative strategy network. The strategy network adopts a two-stage training mechanism. In the first stage, the spatial attention and action prediction accuracy are optimized. In the second stage, the push-grab collaboration is strengthened. The action output by the strategy network is:

[0028] a rgb = argmax depthQ(s, a)

[0029] wherein Q is an estimated action value function, s is a current state in the environment, and a is a set of executable actions under the current state.

[0030] Optionally, the total loss function of the utility map training is defined as follows:

[0031]

[0032] wherein, is the utility map prediction error, is the policy action error, and λ1 and λ2 are loss weighting coefficients.

[0033] According to the above technical solution, compared with the prior art, the robot pushing-grasping collaborative operation method based on spatial relationship utility detection provided by the present disclosure has the following beneficial effects:

[0034] 1) Extremely high grasping precision and relatively high grasping efficiency: the utility map generation network AMGN generates a utility map by combining the design of the coordinate attention module CACNN, which not only provides accurate positioning of the target, but also achieves higher task completion rate and action efficiency. Through comparative analysis, the grasping success rate (88.30%) and action efficiency (3.590) of the utility map generation network AMGN in the dense scene are superior to those of VPG (72.00%, 3.900) and GTI (86.56%, 3.957); this indicates that the network optimization design of the utility map generation network AMGN can better meet the grasping and pushing tasks in the tight visible scene.

[0035] 2) Excellent generalization ability: not only excellent performance in the standard environment, but also effective completion of the grasping task in complex situations such as occlusion and dense object placement; this superiority is due to the optimized strategy that balances exploration and grasping tasks in dynamic environments, improving overall efficiency and stability, and providing a more reliable solution for complex grasping tasks;

[0036] 3) Excellent model stability: in terms of training stability, the loss curve of the model shows extremely low fluctuation amplitude throughout the training process; the algorithm optimizes the weighted sum of reward expectation and policy entropy to guide the robot to achieve a balance between exploration and utilization; this mechanism enables the model to quickly converge in the early training stage, rapidly reducing the loss to a low level, while avoiding falling into a local optimum. BRIEF DESCRIPTION OF DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description only constitute a part of the embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of the provided drawings.

[0038] Figure 1 A flow chart of a robot pushing-grasping collaborative operation method based on spatial relationship utility detection is provided for the present application.

[0039] Figure 2 A utility graph generation network schematic diagram is provided for the present application.

[0040] Figure 3 A specific implementation schematic diagram of the grasping and pushing network is provided for the present application. DETAILED DESCRIPTION

[0041] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments only constitute a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0042] Referring to Figure 1 The present application discloses a robot pushing-grasping collaborative operation method based on spatial relationship utility detection, comprising the following steps:

[0043] By constructing a pushing-grasping collaborative operation system integrating a perception module, a decision module and an execution module, the perception module inputs multi-modal images, generates utility values of each position in the operation space via a utility graph generation network AMGN, and outputs pushing-grasping actions in combination with a strategy network, so as to realize autonomous operation of the robot in an unstructured environment.

[0044] Further, multi-modal images are collected by an RGB-D camera on the robot, and the multi-modal images include RGB images, depth images and mask images.

[0045] Further, the mask images are generated by a RefineNet structure, and output target segmentation images mask, which are used to filter image backgrounds and target object regions, and extract regions in combination with position attention information.

[0046] Further, the output of the utility graph is a two-dimensional mapping graph, and the formula is as follows:

[0047]

[0048] wherein, is a set of real number matrixes with dimension of HxW, F i,j represents the utility value at position (i, j), which is used to represent the probability of feasibility of performing pushing or grasping operation at the current position, and the utility map is generated by the following network:

[0049]

[0050] wherein, I rgb ,I depth ,M mask are RGB image, depth image and mask image respectively, and PE is position encoding.

[0051] Further, the utility map generation network comprises a residual feature extraction module and a direction perception module CACNN, and two-dimensional position encoding is introduced:

[0052]

[0053] wherein, pos x and pos y are the positions of the pixel in the image, i and j are integers within the range of d / 4, and d is the dimension of the model, and the encoding of each pixel position of the input image is added to each pixel value of the image:

[0054]

[0055] wherein, is the feature vector of the original input image at position (pos x , pos y ), which contains the semantic information extracted at position (pos x , pos y ), and without changing the size of the original image, carries the original image data and spatial position information.

[0056] Further, the strategy network takes the utility map F and the current image state as input, outputs the action selection value Q, and is trained in combination with the reinforcement learning method, and the loss function is as follows:

[0057]

[0058] wherein, s i is the current state, a i is the action, γ is the discount factor, Q' is the target value network, N is the number of sampled state-action pairs, r i is the immediate reward, s i+1 is the next state, and a' is the action under the next state s i+1 .

[0059] Further, the pushing and grabbing action is generated by a pushing and grabbing cooperative strategy network, the strategy network adopts a two-stage training mechanism, in the first stage, the spatial attention and the action prediction accuracy are optimized, in the second stage, the pushing and grabbing cooperation is strengthened, and the action is output by the strategy network as:

[0060] a * = argmax a Q(s, a)

[0061] Wherein, Q is an estimated action value function, s is the current state in the environment, and a is a set of executable actions under the current state.

[0062] Further, the total loss function of the utility map training is defined as follows:

[0063]

[0064] Wherein, is the utility map prediction error, is the strategy action error, and λ1 and λ2 are loss weighting coefficients.

[0065] Specifically, in order to improve the description ability of the target state, three types of height maps are preprocessed to represent each state s t , including color height map c t , depth height map d t and mask height map m t . The state at time t is defined as s t = {c t , d t , m t}, and the state s t is stored in the image experience pool to construct a correlation mapping model that can adapt to environmental changes. The experience pool can be used to break the time synchronization problem of the camera, to break the correlation of the input images in adjacent time, to improve the time sequence consistency of the input images, and to optimize the spatial reasoning ability of the target.

[0066] The RGB image is input into a pre-trained RefineNet model for further feature extraction of the object. The adaptive convolution module uses RCU (Residual Convolutional Unit) to process the input features, extracts global and local features through multi-scale convolution, enables the network to better learn the structural information of the target object, and improves the robustness of recognition. The multi-resolution fusion module fuses features of different channels to enhance the boundary perception ability and detail expression ability of the target region, thereby ensuring clear segmentation of the target region, and maintaining good target recognition effect even under complex background or light change conditions. The chain residual pooling module further uses multi-level pooling operations to strengthen the depth information of the target object, optimize the feature fusion result, and improve the accuracy of the mask graph, so that the network can more accurately capture the position distribution of the target object in the three-dimensional environment. The output convolution module calculates the target mask height map through the final convolution layer to ensure that the output image has clearer segmentation boundaries and high-resolution prediction results, so that the robot can obtain more stable input information when performing tasks such as grabbing and target tracking in complex environments.

[0067] In the grabbing task, the optimal grabbing position is determined by the utility Q map. The utility Q map represents the quality of different action choices, and the larger the Q value, the more effective the action. The action policy of the deep reinforcement learning network is continuously adjusted to maximize the reward at each decision step. The learning process of the Q value is based on environmental feedback and reward mechanism to ensure that the robot can learn an efficient task execution strategy. The Q map not only reflects the current state of the environment, but also optimizes the action selection based on the learned historical experience. The Q value of each action selection not only considers the current environmental state, but also considers the maximization of long-term reward. This long-term learning strategy helps the robot to select the optimal action in a complex environment.

[0068] The utility map generation network aims to generate a utility map from the image obtained by the RGB-D camera, which is used to guide the interaction of the robot with the environment and task decision-making. The feature extraction network is used to extract key visual features that are independent of specific actions from the input image. First, the input state information is processed by two layers of residual blocks (Residual Blocks). By introducing a skip connection, the residual structure effectively alleviates the gradient vanishing problem, ensuring smoother flow of features in the deep network. Then, the extracted features are spliced and input into the DenseNet-121 module. DenseNet-121 uses a dense connection structure to enhance the transmission and sharing ability of feature information, and finally generates an action-independent feature map X F .

[0069] Referring to Figure 3A specific implementation diagram of the grasping and pushing network provided by the application is shown.

[0070] The action network framework is composed of PushNet and GraspNet, both of which share the same ResCAFCN (Residual Coordinate-Attention Fully Convolutional Network) network architecture. ResCAFCN is a multi-task action prediction framework based on a full convolutional network (FCN), and its core structure is deeply integrated with a residual module and a coordinate attention mechanism (CA). It supports input images of any size and outputs feature maps of the same spatial resolution.

[0071] It is composed of three layers of residual structures, each of which uses skip connection and realizes upsampling through bilinear interpolation. The use of these residual blocks makes the network converge faster during training, and at the same time avoids the overfitting phenomenon that may occur in deep networks. The first layer of ResCAFCN embeds a coordinate attention module (CACNN), and its core idea is to model the positional relationship in the horizontal and vertical directions, and dynamically enhance the feature expression of key regions. For a given input feature map The coordinate attention mechanism CA performs global average pooling on the feature map in the horizontal and vertical directions respectively, generating direction-aware global description vectors:

[0072]

[0073] wherein, and encode the global spatial information in the vertical and horizontal directions respectively.

[0074] The direction interaction relationship is modeled through convolution and nonlinear activation function, and a two-dimensional attention map is generated using the Sigmoid function:

[0075]

[0076] wherein, G (h) and G (w) are 1x1 convolution layers, and σ is the Sigmoid function. The attention map and respectively represent the spatial significance in the vertical and horizontal directions.

[0077] The attention map and the main feature map X F are fused by Hadamard product channel by channel:

[0078] Y=X F ⊙G(h) ⊙G (w)

[0079] where ⊙ denotes element-wise Hadamard product. The network focuses on the key areas related to the task, significantly improving the spatial positioning accuracy. In the location attention mechanism, the input feature map is weighted and aggregated, and the output enhanced feature map is obtained. i is aggregated by global attention weights:

[0080]

[0081] where α ij denotes the attention weight of position i to position j is the feature vector of position j in the input feature map.

[0082] Y is the enhanced feature representation dynamically adjusted by the global location relationship. The feature y i of each position of Y not only contains its own local information, but also integrates the context information of other positions, thereby significantly improving the model's ability to model long-distance dependencies and spatial correlations. In the ResCAFCN framework, Y further generates the utility Q map (functionality Q map) through normalization, providing spatial guidance for the robot's pushing and grasping tasks.

[0083] The utility Q map generated by Y represents the confidence of performing a push or grasp operation at a specific location, where the light and dark colors indicate the high and low confidence. The network determines the position of the action by selecting the pixel point with the highest confidence value as the target position.

[0084] PushNet and GraspNet achieve target grasping task execution by learning the relationship between image features and robot actions. In the pushing task, the network maps the features to the pushing direction and optimizes the position and angle of the push; while in the grasping task, the network predicts the precise coordinates of the grasping position and the grasping angle, ensuring that the robot can firmly grasp the target object.

[0085] As shown in Figure 2 , in order to obtain accurate three-dimensional action coordinates P(x, y, z), the network converts the selected two-dimensional pixel position into three-dimensional coordinates through the depth prediction map. To ensure the diversity of action planning, multiple possible rotation angles are simulated for each action. The number of discrete rotation angles is 16 (N = 16), and the angle interval of each action is 22.5°. Finally, the utility map generation network outputs a set of optimal utility Q maps (Q max ), providing key motion information for the robot.

[0086] The design of the policy network model benefits from the action binary classifier. Decisions are made based on feedback in the environment, and the policy is gradually optimized through continuous learning. The entire process is divided into two stages: Exploration and Coordination. When the robot fails to find the target object, it needs to acquire more environmental information through exploration behavior. In this stage, the robot only performs push actions to find the target object. After the target object is found, the robot starts to perform the collaborative grasping task.

[0087] The algorithm is based on Deep Q Network (DQN) and builds a phased training strategy to achieve efficient perception and action decision optimization for RGB-D images. The core idea is to dynamically adjust the policy based on the visibility of the target state, ensuring that the robot can autonomously explore and accurately perform operations in complex environments. The algorithm receives an RGB-D image I as input and initializes network parameters, state s t and action In the case where the target is not found, the algorithm first generates and segments the mask height map m t to determine whether the target is visible. When the target is not visible , the algorithm uses an exploration strategy, i.e., calculates the current state using HeightmapProjection(), and calculates the Q value Q p using the push network (PushNet), and selects the push action with the maximum Q value to guide the target into the visible range through environmental interaction. If the target has been detected , the coordination strategy is executed, i.e., the push network (PushNet) and the grasp network (GraspNet) are used to calculate the push Q value Q p and the grasp Q value Q g , respectively. Then, the comprehensive decision score is calculated using the policy fusion function f(maxQ p , maxQ g ), and the final action is selected based on the maximum Q value. The phased strategy of this algorithm not only improves the stability of target recognition, but also improves the task completion rate and generalization ability of the robot in dynamic environments by reasonably combining push and grasp actions. In addition, using DQN for end-to-end reinforcement learning optimization enables the robot to learn the optimal policy through continuous interaction, effectively enhancing the understanding and control precision of the target area.

[0088] To better train the strategy, only a small number of target objects are placed in the simulation environment during the first 1000 training rounds. The goal of this stage is to have the robot first find the location of the target object and move the surrounding objects or clear obstacles through pushing actions, helping the robot better explore the environment and find the location of the target object. In the latter 1500 training rounds, the robot turns to training the coordination strategy, and there are a large number of target objects in the simulation environment. The goal of this stage is to perform grasping operations, and the robot needs to coordinate grasping and pushing actions to ensure efficient grasping of the target.

[0089] See the following examples:

[0090] Example 1:

[0091] The embodiment of the application discloses a robot pushing-grasping collaborative operation method based on spatial relationship utility detection, wherein the implementation of the method includes implementing each module of the robot grasping system. It includes:

[0092] The perception module is responsible for obtaining real-time object information from the environment. The system uses an RGB-D camera to capture RGB images and depth images of the environment, extracts spatial features of the target object, such as the position, shape, and relative relationship of the object. The extracted object information is processed by a feature extraction network to generate a spatial description graph that can be used by the decision module as a guide for decision-making. It can learn geometric structures and utilities in single-object and multi-object scenarios.

[0093] The decision module uses a reinforcement learning algorithm to optimize the robot's behavior in pushing and grasping tasks. The reinforcement learning model enables the robot to make optimal decisions based on real-time changes in the environment through training, selecting appropriate grasping or pushing actions. The decision module receives object spatial information from the perception module and selects appropriate action strategies based on the current state. Through the learning of value functions and policy gradients, the decision module can continuously improve the strategy, thereby achieving optimal action selection in different task scenarios.

[0094] The execution module is responsible for driving the robot to perform actual grasping and pushing operations according to the strategy output by the decision module. The execution module uses robot control algorithms to convert the action path generated by the decision module into specific motion commands, controlling the robot arm and end effector to perform precise actions. Through the execution of planned grasping and pushing actions, the robot can successfully grasp the target object and avoid interference with other objects.

[0095] Example 2:

[0096] The embodiment of the application discloses a robot pushing-grasping collaborative operation method based on spatial relationship utility detection, wherein the implementation of the method includes a robot pushing-grasping operation physical platform.

[0097] The platform is composed of existing laboratory equipment, including a six-degree-of-freedom UR5e robot arm, a double-finger gripper, a Realsense D435i RGB-D camera, an industrial controller, a workbench, and other components. The platform as a whole adopts a modular design, facilitating the reconstruction of the task environment. The RGB-D camera is fixed above the robot arm to capture image data from a top-down perspective, ensuring the comprehensiveness and stability of the perception information. The workbench area is arranged with a set of replaceable target objects, covering different colors, shapes, and sizes to meet the diverse grasping task requirements.

[0098] Embodiment three:

[0099] The embodiment of the application discloses a robot pushing-grasping collaborative operation method based on spatial relationship utility detection, including the software implementation method of the method, the model deployment scheme and the hardware system.

[0100] The real system supports consistent data interfaces with the simulation environment, combined with Python control scripts and PyTorch inference frameworks, to ensure that the policy model can be directly deployed and verified in the real platform. In addition, the system designs a real-time visualization interface and a log recording module, which facilitates the observation of the grasping process, the evaluation of action decisions, and the analysis of error conditions, thereby improving the learning effect and operation intuitiveness.

[0101] The simulation experiment uses CoppeliaSim as the robot simulation platform, which provides virtual environment support for robot operation through its powerful physics engine and flexible modular design. In this environment, the robot can perform object grasping and operation tasks in the virtual environment by executing the policy. CoppeliaSim not only provides high-quality three-dimensional graphics display, but also supports real-time communication with external applications, providing support for training robot models. The robot and the environment interact through the Remote API. The agent learns and makes decisions through the Python+PyTorch framework, and the policy network is used to process the input RGB-D images and generate corresponding action instructions. The hardware devices used are AMD Ryzen Core R7-7745HX and NVIDIA 4060.

[0102] Embodiment four:

[0103] The embodiment of the application discloses a robot pushing-grasping collaborative operation method based on spatial relationship utility detection, including the implementation details of the visual perception method.

[0104] The RGB height map and the depth height map are taken as inputs, low-level features are extracted from the RGB image and the depth image respectively through a feature extractor composed of convolution modules. The RGB image provides color, shape and texture information of the target, while the depth image reflects the spatial layout and distance information of the object. The feature maps extracted through the feature extraction are spliced and input into a DenseNet121 model for high-level feature extraction, and the point cloud data of the original image is generated through a specific algorithm GPG to generate a series of candidate grasping poses.

[0105] The network utilizes the target space, target shape, position, pose and depth information extracted from the RGB image and the depth image to construct a high-dimensional feature vector containing multiple visual features. The vector is further processed through a fully convolutional network (FCN) to generate a feature map containing Q values corresponding to each action. This feature map provides the reinforcement learning system with value evaluation of each action, enabling the system to select the optimal action based on the Q value in different states.

[0106] Embodiment five:

[0107] The embodiment of the application discloses a robot pushing-grasping collaborative operation method based on spatial relationship utility detection, including end-of-arm motion details.

[0108] In the execution phase, the selected pixel points will be back-projected to the robot operating space, and the actual coordinates will be calculated through the camera-robot external parameter relationship. The rotation angle is used to adjust the pose of the end effector. The pixel coordinates are a representation of the 3D position information in the implicit space. To map the optimal pixel points in the image plane to physical positions executable by the robot, spatial transformation from the image coordinate system to the robot base coordinate system needs to be completed. A geometric transformation method based on hand-eye calibration is adopted, and a spatial transformation matrix is constructed combining the camera intrinsic matrix and the camera-robot external parameter matrix Through the transformation, the image coordinates are back-projected to three-dimensional coordinates (x w ,y w ,z w ) in the workspace.

[0109] Firstly, the image coordinates are restored to normalized spatial coordinates in the camera coordinate system according to the camera intrinsic parameters, and then converted to the robot base coordinate system through the external parameter matrix. The final three-dimensional spatial position and the rotation angle together constitute the pose target of the robot end effector, which will be issued to the controller through the execution module to drive the robot to complete the corresponding pushing or grasping operation. After execution, the system will acquire a new frame of environment state image and re-enter the perception-decision-action closed loop to continuously optimize the action strategy, thereby realizing effective closed loop of policy learning and physical execution.

[0110] The various embodiments described in this specification are intended to be illustrative only and in no way limit the scope of the application. Changes and modifications can be made by those skilled in the art, which employ the principles of the application, without departing from the scope of the application. Accordingly, the application is not limited to the embodiments described herein, but instead has scope to encompass any choice whatsoever that is dependent on, or can be substituted in, the principal, new and inventive features that are described and claimed herein.

[0111] The above description of disclosed embodiments is intended to be illustrative only and not limiting of the application. Numerous modifications to these embodiments can be made by those skilled in the art without departing from the spirit or scope of the application. The scope of the application is not limited to the embodiments described herein, but rather extends to any that are dependent on, or can be substituted in, the principal, new and inventive features that are described and claimed herein.

Claims

1. A robot propulsion-grasping cooperative operation method based on spatial relationship functionality detection, characterized in that, include: By constructing a push-grab collaborative operating system that integrates the perception module, decision-making module, and execution module, the perception module inputs multimodal images, generates utility values ​​for each location in the operation space through the utility graph generation network AMGN, and outputs push-grab actions in combination with the policy network, thereby enabling the robot to operate autonomously in unstructured environments. Multimodal images are acquired using an RGB-D camera on the robot. These multimodal images include RGB images, depth images, and mask images. The mask image is generated by the RefineNet structure and outputs a target segmentation image mask, which is used to filter the image background and target object regions, and combines positional attention information to extract regions; The output of the utility diagram is a two-dimensional mapping diagram, as shown in the following formula: in, for A set of real matrices of dimension 1. Indicates position ( i , j The utility value at position ) represents the probability of performing a push or grab operation at the current position. The utility graph is generated by the following network: in, These are RGB image, depth image, and mask image, respectively, with PE representing position encoding. The functional graph generation network includes a residual feature extraction module and a direction-aware module CACNN, and introduces two-dimensional position encoding: in, and The position of a pixel in the image. and for Integers within the range The dimension of the model is the encoding of each pixel position in the input image. Added to each pixel value of the image: in, For the original input image at the position The feature vector at position The semantic information extracted is used to refine the original image without changing its size. It carries the original image data and spatial location information; The total loss function for training utility graphs is defined as follows: in, For the prediction error of the utility map, For strategy action error, , This is the loss weighting coefficient.

2. The robot push-grasp cooperative operation method based on spatial relationship functionality detection according to claim 1, characterized in that, The policy network takes the utility graph F and the current image state as input and outputs the action selection value Q. It is trained using a reinforcement learning method, and the loss function is as follows: in, This is the current state. For action, As a discount factor, For the target value network, The number of sampled state-action pairs. For instant rewards, For the next state, For the next state The following action.

3. The robot push-grasp cooperative operation method based on spatial relationship functionality detection according to claim 1, characterized in that, The pushing and grasping motion is generated by a pushing and grasping collaborative policy network. The policy network employs a two-stage training mechanism. In the first stage, spatial attention and motion prediction accuracy are optimized. In the second stage, pushing and grasping coordination is enhanced. The motion is output by the policy network as follows: in, To estimate the action value function, The current state of the environment. This is the set of actions that can be executed in the current state.

Images