A stacked scene robot grasping method based on semantic guidance and spatial decoupling

Abstract

The application discloses a kind of based on semantic guidance and space decoupling's stacked scene robot grasping method, scene image is collected by RGB-D vision sensor and is converted into environment height map, and target is determined by receiving user natural language instruction;Utilize visual language model joint analysis to generate semantic mask, and construct reinforcement learning environment state in combination with height map;Characteristic is extracted by full convolutional neural network and generates action value map, and action space is constrained by fusing semantic mask, and optimal push-grab action is selected;Based on multi-weight total reward function, network parameter optimization strategy is updated by back propagation, and iteration is executed until target is successfully grasped.The method can avoid blind exploration, solve reward sparsity problem, improve the success rate of grasping, effectively improve the operation ability of robot in height dense and occluded environment, provide a new solution for multi-target object operation, adapt to multiple complex operation scenes, and have high application value.

Description

Technical Field

[0001] This invention belongs to the field of robotics and artificial intelligence, and particularly relates to a robot grasping method for stacked scenes based on semantic guidance and spatial decoupling. This method is applied to multi-object manipulation tasks and can improve the efficiency and success rate of push-grab tasks in complex and disordered environments by combining deep reinforcement learning and semantic understanding. Background Technology

[0002] With the rapid development of artificial intelligence and robotics, robotic grasping and manipulation technologies are widely used in industries such as manufacturing, logistics, and home services. Especially in high-density, disordered stacking environments, traditional single-grasping strategies face significant challenges. In these complex scenarios, objects often occlude or are in close contact, rendering grasping operations infeasible or ineffective. Traditional grasping algorithms mostly rely on precise geometric models and sensor data to predict the object's grasping position, but these methods often fail when objects overlap or occlude.

[0003] To address this issue, the academic community has proposed a "push-grasp cooperative" strategy, which aims to actively alter the environmental structure and separate objects through a "push" action, thereby creating favorable conditions for the subsequent "grasp" action. However, most existing push-grasp cooperative strategies rely on geometric features, such as surface normals or object curvature, ignoring the semantic information of the task target. This often makes it difficult for robots to distinguish between target objects and background obstacles when performing tasks, resulting in blind operations and wasted computational resources.

[0004] Furthermore, the sparsity of rewards in long-sequence tasks remains a significant challenge in reinforcement learning. In push-grab tasks, rewards are often only obtained upon successful completion, making it difficult for the robot to acquire effective feedback signals during long-term training. Therefore, designing a reward function that provides dense feedback during the pushing action and helps the robot train efficiently has become a key issue in improving the efficiency of push-grab strategies.

[0005] To address these issues, this invention proposes a dual-reward mechanism based on semantic guidance and spatial decoupling. By introducing semantic understanding technology and reinforcement learning methods, a push-grasp cooperative operation strategy is constructed, enabling the robot to actively alter the environmental structure through pushing actions before grasping the target object, thereby creating feasible space for the grasping operation. Before the grasping process, the robot uses a visual language model to perform semantic parsing of natural language instructions and constrains the robot's action space through semantic masks, allowing the robot to operate within the relevant area of ​​the task. The dual-reward mechanism, employing spatial decoupling, provides dense feedback signals for reinforcement learning by quantifying changes in the spatial relationships between objects, thereby improving the robot's policy learning efficiency and task success rate. Summary of the Invention

[0006] The purpose of this invention is to propose a robot grasping method for stacked scenes based on semantic guidance and spatial decoupling, in order to solve the problems of target object recognition difficulties, blind operation strategies, and sparse reinforcement learning rewards in existing robot grasping technologies in complex stacked environments. This method is a simple and easy-to-implement method for acquiring target objects in stacked scenes. By quantitatively evaluating the spatial relationships between objects (such as Mask IoU), this method can provide the robot with dense rewards based on the improvement of the environmental structure even when the grasping task has not been completed, thereby significantly improving training efficiency and enhancing the robot's operational capabilities in complex environments.

[0007] The technical solution of this invention: a robot grasping method for stacked scenes based on semantic guidance and spatial decoupling, characterized by the following steps: (1) Acquire depth and color images of all objects in the scene using an RGB-D (Red Green Blue - Depth) vision sensor. It also converts depth image information into an environment height map. It is used to describe the three-dimensional spatial structure of objects (including target objects, obstacles, and background) in a scene; at the same time, the system receives natural language commands input by the user. As task-oriented information (such as "grab the red box"), this instruction defines the specific target object that the robot needs to manipulate in the current task.

[0008] The RGB-D vision sensor is used to acquire the spatial geometric information of all objects in the environment and generate corresponding environmental height map data, using natural language commands. Describing the target object that the robot needs to acquire provides a task basis for the subsequent semantic understanding module, thereby constructing the initial environmental state input of the system: (1) In the formula, express Initial environmental state input for the time-sensitive robot control system; This represents an environment height map generated from a depth image, reflecting the geometric distribution of all objects within the scene; This represents the raw color image captured by the vision sensor; This represents the natural language text commands input by the user.

[0009] (2) Using the visual language basic model, the natural language instructions input in step (1) are jointly analyzed with the RGB-D color image generated by the RGB-D visual sensor. The object region related to the task is obtained through the object detection model, and the corresponding semantic mask is generated. Step (2) specifically includes the following: (2-1) Using the GroundingDINO model, identify the bounding box of the target region in the RGB-D color image obtained in step (1) based on the text instruction. GroundingDINO is a large-scale model based on visual language. It can automatically predict and find the rectangular region in the image that is most relevant to the text description by aligning the cross-modal features of the input natural language description with the image content and using the language-guided query mechanism in deep learning networks. It then outputs the spatial coordinates of the object as the bounding box. (2-2) Using SAM (Segment Anything Model) to identify the target bounding box in step (2-1) Generate pixel-level semantic masks SAM is an image segmentation model that, given a bounding box as a visual cue (Prompt), performs fine-grained edge reasoning through its built-in mask branch to accurately segment pixel regions of target objects. This model generates sharp target object masks through efficient image processing. and obstacle mask This is used to describe the positional distribution of the target object and its surrounding obstacles in the image, thereby providing semantic constraint information for robot motion planning. At this point, the environmental state constructed in step (1) is... After semantic information augmentation, it can be further represented as the semantically augmented environment state of the robot control system: (2) In the formula, express The system environment state that constantly contains rich semantic information; The environment height map generated in step (1) represents the spatial geometric height information of the scene; This represents the pixel-level semantic mask of the target object generated after processing by the base model. This represents the pixel-level semantic mask of the obstacle object generated after processing by the base model.

[0010] (3) Based on the depth and height map of the current scene obtained in step (1) and the pixel-level semantic mask generated in step (2) Information constructs the environmental state of reinforcement learning environment ; The environmental state of the reinforcement learning environment constructed in step (3) Including environmental height maps and the target object mask and obstacle mask pixel-level semantic mask It is used to describe the spatial location, shape features, and semantic attributes of all objects in the current environment.

[0011] The environmental state of the reinforcement learning environment The construction process and robot motion decisions specifically include the following: (3-1) Transform the two-dimensional environmental height map Pixel-level semantic mask of the corresponding task target object Pixel-level semantic masks for obstacles Alignment and stacking are performed in the spatial coordinate system to form a multi-channel tensor, which serves as the input feature map for the deep reinforcement learning network; (3-2) The robot will update the current environmental state at each moment. Input a pre-trained policy network (e.g., DenseNet-121), and through the forward computation of the policy network, automatically extract deep features that fuse spatial geometric information and object semantic information and map them to the action space; (3-3) For each pixel position, calculate the expected reward value, i.e., the Q value, for the robot to perform the "push" or "grab" action, and then select the action to be performed based on the maximum Q value. ; (3-3) The robot control system uses reinforcement learning algorithms (such as DQN) to optimize the parameters of the policy network end-to-end by using a dual dense reward function based on the change of grasping confidence and the change of mask intersection over union (IoU), enabling the robot to learn autonomously and realize the operation strategy of "push-driven grasping" in complex stacked environments.

[0012] (4) Use fully convolutional neural networks (FCN) to process the environment state constructed in step (3). Feature extraction is performed, and a relationship with the environment state is generated. The corresponding action value map Q represents the expected reward value obtained by the robot when performing pushing or grasping actions at different positions; In stacked scenarios, the success rate of direct grasping is extremely low because target objects are often occluded or compressed. The significance of generating an action value map lies in enabling the robot to quantitatively assess the potential benefits of "pushing" or "grabbing" in each area of ​​the current scene through pixel-level value prediction. This allows the robot to proactively separate objects using high-value "pushing" actions when grasping space is insufficient, creating favorable conditions for accurate final grasping. The function expression for the action value map is: (3) in, Here is the original action value map, where 'a' represents the robot's action: (4) In the formula, 'a' represents the robot's action, which includes pushing and grasping actions. This indicates the position of the execution center of the action in the environment state space coordinate system; This indicates the rotation angle when the robot's end effector performs an action.

[0013] (5) To avoid the robot from making ineffective explorations in the scene, the pixel-level semantic mask generated in step (2) is used. The action value map generated in step (4) The data is then fused to generate a semantically constrained action value map. ; Step (5) specifically refers to: performing the Hadamard product operation on the pixel-level semantic mask generated in step (2). The action value map generated in step (4) is applied. This ensures that only action candidate points in semantically relevant regions are retained in the action value map, while the action values ​​of background and irrelevant regions are set to zero. This effectively limits the robot's action range, ensuring it only operates within areas that might affect the acquisition of the target object, thereby reducing ineffective exploration. The semantically constrained action value map can be represented as: (5) In the formula, This represents the final action value map generated after semantic constraints, used to guide the robot to perform effective actions; This represents the original action value map generated by step (4); This represents the Hadamard product operation, which is the element-wise multiplication of two matrices of the same type. This represents the pixel-level semantic mask of the obstacle generated by step (2). In this matrix, the pixel value of the obstacle area is 1, and the pixel value of the rest area is 0.

[0014] The semantically relevant region refers to the pixel-level semantic mask of the obstacle object. The covered area includes obstacle areas that may hinder grasping; the background area refers to an environmental area such as a desktop that does not contain any objects; the irrelevant area refers to an area that contains objects but has no physical interaction with the current grasping task; the action value refers to the expected reward value after performing a specific action a, and the action value map is a matrix composed of the action values ​​corresponding to all pixel positions in the entire image, and the two correspond one-to-one through spatial coordinates.

[0015] (6) Action value map with semantic constraints generated in step (5) The robot selects the action with the highest value as its current execution strategy. Step (6) refers to the system using a search algorithm to obtain the action value map after semantic constraints. The robot finds the pixel with the largest value in the search algorithm. The spatial coordinates and rotation angle of this pixel represent the optimal action to be executed. By comparing the expected reward values ​​Q of different action candidate points, the robot control system can quantitatively determine which action is most valuable for achieving the grasping task and formulate the current execution strategy. The decision-making process can be represented as: (6) In the formula, This indicates the optimal action selected by the system under the current environmental conditions. This indicates a search and return function. The independent variable when it reaches its maximum value The operator.

[0016] The actions include two types: pushing and grasping. When the target object is obstructed by other objects or there is insufficient grasping space, the robot prioritizes pushing, using the end effector of the robotic arm to push surrounding objects and create spatial separation between them. When the target object is in a graspable state, the robot performs grasping to complete the task of acquiring the target object.

[0017] (7) In order to improve the training efficiency of reinforcement learning, this invention designs a dual reward function that includes physical reward and spatial reward, and defines the total reward function for robot control. Represented as: (7) in, For physical rewards, As a space reward, As a reward for the event, and These are weight parameters; Formula (7) indicates that the total reward function of the system is a dual reward function containing both physical and spatial rewards. The physical reward is used to evaluate the impact of the current action on the feasibility of grasping the target object, and its calculation method is as follows: (8) in, This represents the crawling quality scoring function. When a driving action improves the crawling score, the system provides a positive reward. The space reward This is used to measure the degree of spatial decoupling between objects, assuming the set of all objects in the scene is... The change in the Intersection over Union (IoU) of the masks between objects before and after the action is executed is calculated. To assess the improvement of environmental structure: (9) (10) In the formula, and Representing sets Pixel-level semantic masks for different objects, when a pushing action reduces the overlapping area between objects. It is a positive value, after After function mapping Output positive reward values ​​to quantify the improvement in the degree of decoupling from the feedback environment; (8) After each training iteration, i.e. the process of the robot performing a single action and observing environmental feedback, the robot control system calculates the total reward value according to formula (7). The parameters of the reinforcement learning policy network are updated using the backpropagation algorithm of the fully convolutional neural network described in step 4 to minimize the deviation between the predicted value and the actual reward, thereby optimizing the push-grab collaborative strategy. (9) After the parameter update is completed, the robot control system performs the current action. The resulting change in environmental state leads to the next state. If the event reward is not triggered at the current moment. If the successful capture signal is not completed, the system returns to step (1) to reacquire the environmental state and repeats the operations from step (1) to step (8) to make the next round of decision-making and execution; if the target object has been successfully captured and the success reward is triggered, the task is judged to be over, and the target object recognition and capture in the complex stacked scene is completed.

[0018] The working principle of this invention is as follows: By combining semantic guidance with a spatial decoupling mechanism, a method for robots to acquire target objects in complex stacked scenarios is achieved. In traditional grasping methods, robots often rely on visual information to directly predict the grasping position. When the target object is occluded by other objects, the grasping operation becomes difficult to perform. This invention introduces a pushing action, enabling the robot to actively change the environmental structure, thereby gradually exposing the target object and creating feasible conditions for subsequent grasping operations.

[0019] During system operation, the robot first acquires environmental information through an RGB-D vision sensor and determines the task objective using natural language commands. Subsequently, it uses a visual language model to jointly analyze images and text, identifying target objects and surrounding obstacles, and generating a semantic mask. The semantic mask not only helps the robot identify the location of target objects but also clarifies which objects belong to task-relevant areas, thus providing semantic constraints for the robot's action planning. In the action decision-making phase, the system uses a convolutional neural network to extract features from the current state and generate an action value map. The action value map reflects the expected benefits of the robot performing pushing or grasping actions at different locations. By fusing the semantic mask with the action value map, the system can restrict robot actions to semantically relevant regions, thereby avoiding ineffective exploration in irrelevant areas.

[0020] When a robot performs a pushing action, the spatial relationships between objects in the environment change. This invention quantifies the degree of overlap between objects by constructing a spatial decoupling reward function. When the pushing action reduces the overlapping area between objects, it indicates that the environmental structure has improved, thus giving a positive reward; if the pushing action does not improve the environmental structure, a lower reward or penalty is given. In addition, the system also calculates the change in the grasping score of the target object through a grasping quality assessment model and uses it as part of the physical reward.

[0021] By providing both physical and spatial rewards, the system delivers dense feedback signals to the reinforcement learning algorithm, enabling the robot to obtain effective rewards during the pushing phase and thus improving policy learning efficiency. As training progresses, the robot gradually learns to select appropriate operational strategies under different environmental conditions; for example, prioritizing pushing actions when the target object is occluded, and performing grasping actions when the target object is graspable. Ultimately, the robot can stably and efficiently grasp target objects in complex stacked environments.

[0022] Advantages of this invention: Compared with existing technologies, the proposed method for acquiring target objects in stacked scenarios based on semantic guidance and spatial decoupling has significant advantages: ① By introducing a semantic understanding mechanism, the robot can identify target objects and restrict its action space according to natural language instructions, thereby avoiding the blind exploration problem in traditional push-grab strategies; ② The proposed spatial decoupling reward function can quantify changes in the spatial relationships between objects, providing dense reward signals for reinforcement learning, thus effectively solving the reward sparsity problem and improving policy learning efficiency; ③ By constructing a push-grab collaborative operation framework, the robot can actively adjust the scene structure by pushing objects in complex stacked environments, thereby improving the success rate of target object acquisition; ④ This method is applicable to various complex operation scenarios such as logistics sorting, home service robots, and intelligent manufacturing, and has high application value and promotion prospects. Attached Figure Description

[0023] Figure 1 This is a schematic diagram of the overall process of a robot grasping method for stacked scenes based on semantic guidance and spatial decoupling, which is involved in this invention.

[0024] Figure 2 This is a schematic diagram of target detection and semantic segmentation in the semantic guidance module of a stacked scene robot grasping method based on semantic guidance and spatial decoupling, which is involved in the present invention.

[0025] Figure 3 This is a schematic diagram illustrating the calculation principle of the dual reward function in a stacked scene robot grasping method based on semantic guidance and spatial decoupling, as described in this invention.

[0026] Figure 4 is a schematic diagram of the robot performing the push-grab cooperative operation in the embodiment; (wherein, 4-a is the initial stacking scenario, the target object (red square) is tightly surrounded by obstacle objects (blue square), at this time the robot recognizes the target area but there is insufficient grasping space; 4-b is the process of the robot performing the push operation, pushing the obstacle objects through the end effector of the robotic arm, so that the originally tightly stacked objects are physically separated in space (i.e., spatial decoupling); 4-c is the state after the push, the target object has been exposed from the stack, creating feasible space for subsequent grasping actions.) Detailed Implementation

[0027] Example: Figure 1The diagram shown is a schematic representation of the overall process of the robot grasping method for stacked scenes based on semantic guidance and spatial decoupling described in this invention. The diagram fully presents the closed-loop process of this method from environmental perception to successful grasping: it sequentially includes RGB-D data acquisition and height map generation, natural language instruction reception, visual language model semantic parsing and mask generation, reinforcement learning environment state construction, fully convolutional neural network action value map generation, semantic mask constraint of the action space, optimal push-grab action decision, dual reward function calculation, network parameter optimization iteration, until the target object is successfully grasped. It intuitively demonstrates the technical path of integrating semantic guidance and spatial decoupling. (1) The robot system first synchronously acquires color images of the current operating environment through an RGB-D vision sensor. The RGB-D vision sensor can be mounted on a fixed bracket above the robotic arm to acquire spatial structure information of all objects on the desktop or within the operating area in real time. By projecting the depth image, a corresponding environmental height map can be generated. This is used to describe the three-dimensional geometry and relative spatial relationships of all objects in a scene (including target objects, obstacles, and background areas). Let the current environment height map be... .

[0028] The system receives natural language task instructions input by the user. For example, commands like "grab the red box" or "get the blue bottle" can be generated through a speech recognition module or text input and converted into text information to clearly define the target object of the task. This is further enhanced by combining environmental height maps. Color images With task instruction information It can construct the initial environmental state of the robot control system. : (1) In the formula, This represents the initial environmental state input of the robot control system at the current moment. This represents environmental height map data. This represents the raw color image acquired by the sensor. It represents the semantic information of the task.

[0029] In this way, the robot can simultaneously acquire environmental geometric information, visual color information, and task target information, thereby providing a task basis for the subsequent semantic understanding module, so as to facilitate accurate semantic analysis and strategy decision-making.

[0030] (2) The robot control system uses the visual language basic model to process the natural language instructions input in step (1). Compared with RGB color images generated by RGB-D vision sensors Joint analysis is performed to identify target objects and surrounding obstacles in the scene. In this embodiment, the Grounding DINO model is first used based on the input text instructions. Recognize RGB color images Identify the target regions relevant to the task and generate corresponding target detection bounding boxes. GroundingDINO is a large-scale model based on visual language. It can automatically find the rectangular region in an image that is most relevant to the text description by aligning the input natural language description with the cross-modal features of the image content and using the language-guided query mechanism in a deep learning network. This allows it to output the spatial coordinates of the object as a bounding box.

[0031] Then the target detection bounding box As a visual prompt, the SAM2 model generates a pixel-level semantic mask through an image segmentation algorithm. SAM2 is an image segmentation model that can accurately segment pixel regions of a target object by performing fine-grained edge reasoning through its built-in mask branch, given a bounding box.

[0032] The semantic mask Including target object mask and obstacle mask This is used to describe the spatial distribution of different objects in the scene. Semantic masks can accurately identify the location of target objects in the image and simultaneously determine the surrounding obstacle areas that affect the grasping task, thus providing semantic constraint information for the robot's subsequent action planning. At this point, the environmental state constructed in step (1) is... After being expanded with semantic information, it can be further represented as: (2) In the formula, This represents the pixel-level mask of the target object generated by the semantic segmentation model. This represents a pixel-level mask of the identified obstacle. In this way, the system eliminates the perceptual blindness caused by relying solely on geometric features. For example... Figure 2Figure 2 illustrates the target detection and semantic segmentation of the semantic guidance module of this invention, fully demonstrating the entire process of target localization and mask generation under text instruction guidance. Figure 2 first presents the cross-modal matching results between the natural language instruction and the RGB image. The GroundingDINO model outputs precise bounding boxes of the target object and obstacles, with the bounding boxes completely enclosing the target contour without any over-bounding or under-bounding phenomena. Subsequently, based on the bounding box cues, the SAM model generates pixel-level semantic masks. The boundaries between the target object mask and the obstacle mask are clear, without adhesion or missing parts, accurately distinguishing the target and obstacle regions and providing reliable pixel-level semantic basis for subsequent action space constraints.

[0033] (3) Environment state for constructing reinforcement learning algorithm in robot control system The state space describes the spatial structure and semantic attributes of objects in the current environment and serves as input to the policy network. Within this state space, the robot can utilize not only the environmental height map... It can perform motion planning using the provided environmental geometry information and can also utilize pixel-level semantic masks. (Includes target object mask) With obstacle mask Identifying target objects and their surrounding obstacles improves the accuracy and efficiency of action decisions.

[0034] Specifically, this embodiment uses a two-dimensional environmental height map. With the corresponding pixel-level semantic mask , Pixel-level alignment and channel overlay are performed in a spatial coordinate system to construct a multi-channel tensor feature map from the original geometric data and abstract semantic labels. Through this multi-dimensional feature fusion approach, the system transforms the complex physical environment into an environmental state with high-level semantic features. This provides a standardized and structured data foundation for the subsequent extraction of key obstacle avoidance features and target localization features by the strategy network.

[0035] (4) The robot control system uses a fully convolutional neural network to analyze the environmental state constructed in step (3). Feature extraction is performed in the space, and corresponding action value maps are generated. Action value maps represent the expected rewards a robot receives when performing pushing or grasping actions at different locations. In stacked scenarios, the grasping feasible region is often empty due to occlusion or close proximity of the target object. The significance of generating action value maps lies in enabling the robot to quantitatively assess the potential benefits of performing "push" or "grab" actions in different areas of the current scene through pixel-level value prediction. This allows the robot to proactively reconstruct the environmental structure through high-value pushing actions when grasping confidence is low, creating favorable conditions for accurate subsequent grasping of the target object. Its expression is: (3) In the formula, This is the original action value map. The input environment state, For robot actions. Robot actions It can be represented as: (4) In the formula, This indicates the position of the execution center of the action in the environment state space coordinate system. This indicates the rotational direction (i.e., yaw angle) during a pushing or grasping action. This network can predict the optimal action position and direction at the pixel level of the image, enabling the robot to select the most advantageous task completion strategy based on numerical feedback.

[0036] (5) To avoid the robot blindly exploring irrelevant areas, this embodiment uses the pixel-level semantic mask generated in step (2). The action value map generated in step (4) The motion space of the robot is constrained by fusion. Specifically, the semantic mask is applied to the motion value map through the Hadamard product operation, and its expression is: (5) In the formula, This represents an action-value graph with semantic constraints. This is the original action value map. This represents the obstacle mask generated in step (2).

[0037] This operation sets the action value of background and irrelevant regions to zero, allowing the robot to operate only within semantically relevant regions related to the task. Here, semantically relevant regions refer to those defined by the mask. The covered area refers to the obstacle area that may hinder the execution of the grasping task; the background area refers to the environmental area such as the desktop that does not contain any objects; the irrelevant area refers to the area where objects exist but are determined by semantic recognition to have no physical interaction with the current grasping task.

[0038] Action value refers to the value of performing a specific action in a specific state. The expected reward value is calculated by combining the action value map and the action value map, which is a matrix composed of the action values ​​corresponding to all pixel positions in the entire image. The two are correlated one-to-one through a spatial coordinate system. This fusion method can effectively limit the robot's range of motion, ensuring that each "push" action serves to expose a specific "grasp target," thereby significantly reducing ineffective exploration behavior and improving action decision-making efficiency.

[0039] (6) Action value map with semantic constraints generated in step (5) The robot selects the action with the highest value as its current execution strategy. In this embodiment, the system uses a search algorithm to... The system identifies the pixel with the largest value in the matrix; the spatial coordinates of this pixel and its preset rotation angle constitute the optimal action to execute. By comparing the expected reward values ​​of different action candidate points, the system can quantitatively determine which action is most valuable for achieving the final grasping task. Optimal Execution Strategy The expression is: (6) In the formula, Indicates the current environmental state of the system. The selected optimal action to perform. This indicates a search and return function. When the independent variable reaches its maximum value The operator.

[0040] The robot's actions mainly include two types: pushing and grasping. Their execution priority is not preset but dynamically determined by the Q-value distribution in the action value map, with pushing taking priority: when the target object is obstructed by other objects or there is insufficient grasping space, the expected reward value of grasping is lower, while pushing, which improves the environmental structure, will obtain a higher Q-value. In this case, the robot prioritizes pushing. Figure 4 shows a schematic diagram of the robot performing the push-grasp cooperative operation, comprising three sub-figures: Figure 4-a In the initial stacking scene, the target object (red square) is tightly squeezed and surrounded by obstacle objects (blue square), the target object has no exposed grasping pose, and the grasping space is completely insufficient; Figure 4-b To perform the pushing action for the robot, the end effector of the robotic arm acts on the semantic mask region of the obstacle, pushing the obstacle to move outward, and the objects change from close contact to separation. Figure 4-c To ensure that the target object is fully exposed after the scene is completed, with sufficient grasping space and a reasonable pose, the grasping execution conditions are met, thus achieving the technical effect of promoting grasping through pushing.

[0041] After performing a selected action, the robot causes a physical change in the environmental structure, and the system then updates the environmental state and enters the next round of decision-making. Through this dynamic closed loop of "perception-decision-execution", the robot can autonomously determine when to "push" to create space and when to "grab" to complete the task.

[0042] (7) To improve the training efficiency of reinforcement learning and solve the convergence problem caused by the sparse rewards in long-sequence push-grab tasks, this embodiment constructs a dual reward function that includes both physical and spatial rewards. The system's total reward function... Represented as: (7) In the formula, For physical rewards, As a space reward, As a reward for the event, and These are weight parameters.

[0043] Physical reward The method for evaluating the impact of the current action on the feasibility of grasping the target object is as follows: (8) In the formula, This represents the grab quality scoring function, used to evaluate the highest grab confidence of target objects in the scene in real time. When a push action improves the grab score in the scene (i.e., ...), ... This indicates that the action improved the grabbing conditions, and the system provides a positive reward.

[0044] Space Rewards Used to measure the degree of spatial decoupling between objects, let the set of all objects in the scene be . By calculating the change in the intersection-over-union (IoU) ratio between object masks. To assess the improvement of environmental structure: (9) (10) In the formula, and Representing sets Pixel-level semantic masks for different objects. When a pushing action reduces the overlap area between objects... It is a positive value. After function mapping It outputs a positive reward value, thereby quantifying the improvement in the degree of decoupling from the feedback environment. This mechanism ensures that even if the grasping score does not change qualitatively, the robot can still obtain dense gradient signals as long as the object is successfully pushed away. Figure 3The diagram shown illustrates the calculation principle of the dual reward function of this invention, fully demonstrating the calculation logic and fusion effect of physical reward and spatial reward. Figure 3 The algorithm first presents the calculation basis for physical rewards, namely the change in grasping quality score. Promoting actions that increase grasping confidence results in a positive physical reward. Second, it presents the calculation basis for spatial rewards, namely the change in the Intersection over Union (IoU) of object masks. A decrease in the overlapping area of ​​objects reduces IoU and improves spatial decoupling, resulting in a positive spatial reward. Finally, it presents the fusion of physical and spatial rewards after weighted summation into a total reward. This total reward provides dense feedback to the robot throughout the process, without reward sparsity, and provides continuous gradient signals for updating the parameters of the reinforcement learning network.

[0045] The robot calculates the total reward value. The parameters of the reinforcement learning policy network in step (4) are updated in real time using the backpropagation algorithm to minimize the action value prediction bias. After the parameter update is completed, the system enters the next state based on the environmental physical changes caused by the currently executed action. And continue to execute action decisions. If a successful capture signal is not triggered at the current moment (i.e., the task is not completed), the system returns to step (1) to reacquire the environmental state and repeats the operations from step (1) to step (8); if the target object has been successfully acquired and an event reward has been triggered (e.g. If the condition is met, the task is considered complete.

[0046] Through the above steps, the robot can use semantic information to plan actions in complex stacked environments and gradually improve the environmental structure through push-grab cooperative operations, thereby achieving efficient and stable acquisition of target objects.

Claims

1. A robot grasping method for stacked scenes based on semantic guidance and spatial decoupling, characterized in that... Includes the following steps: (1) Acquire depth and color images of all objects in the scene using an RGB-D vision sensor. It also converts depth image information into an environment height map. It is used to describe the three-dimensional spatial structure of objects in a scene; At the same time, the system receives natural language commands input by the user. As task-oriented information, this instruction defines the specific target object that the robot needs to manipulate in the current task; (2) Using the visual language basic model, the natural language instructions input in step (1) are jointly analyzed with the RGB-D color image generated by the RGB-D visual sensor. The object region related to the task is obtained through the object detection model, and the corresponding semantic mask is generated. (3) Based on the depth and height map of the current scene obtained in step (1) and the pixel-level semantic mask generated in step (2) Information constructs the environmental state of reinforcement learning environment ; (4) Use a fully convolutional neural network to process the environment state constructed in step (3). Feature extraction is performed, and a relationship with the environment state is generated. The corresponding action value graph Q represents the expected reward value obtained by the robot when performing pushing or grasping actions at different positions; (5) To avoid the robot from making ineffective explorations in the scene, the pixel-level semantic mask generated in step (2) is used. The action value map generated in step (4) The data is then fused to generate a semantically constrained action value map. ; (6) Action value map with semantic constraints generated in step (5) The robot selects the action with the highest value as its current execution strategy. (7) Define the total reward function for robot control Represented as: (7) in, For physical rewards, As a space reward, As a reward for the event, and These are weight parameters; (8) After each training iteration, i.e. the process of the robot performing a single action and observing environmental feedback, the robot control system calculates the total reward value according to formula (7). The parameters of the reinforcement learning policy network are updated using the backpropagation algorithm of the fully convolutional neural network described in step 4 to minimize the deviation between the predicted value and the actual reward, thereby optimizing the push-grab collaborative strategy. (9) After the parameter update is completed, the robot control system performs the current action. The resulting change in environmental state leads to the next state. If the event reward is not triggered at the current moment. If the successful capture signal is not completed, the system returns to step (1) to reacquire the environmental state and repeats the operations from step (1) to step (8) to make the next round of decision-making and execution; if the target object has been successfully captured and the success reward is triggered, the task is judged to be over, and the target object recognition and capture in the complex stacked scene is completed.

2. The robot grasping method for stacked scenes based on semantic guidance and spatial decoupling according to claim 1, characterized in that... The RGB-D vision sensor is used to acquire the spatial geometric information of all objects in the environment and generate corresponding environmental height map data, using natural language commands. Describe the target object that the robot needs to acquire, which can be used to construct the initial environmental state input of the system: (1) In the formula, express Initial environmental state input for the time-sensitive robot control system; This represents an environment height map generated from a depth image, reflecting the geometric distribution of all objects within the scene; This represents the raw color image captured by the vision sensor; This represents the natural language text commands input by the user.

3. The robot grasping method for stacked scenes based on semantic guidance and spatial decoupling according to claim 1, characterized in that... Step (2) specifically includes the following: (2-1) Using the GroundingDINO model, identify the bounding box of the target region in the RGB-D color image obtained in step (1) based on the text instruction. ; (2-2) Using SAM to identify the target bounding box in step (2-1) Generate pixel-level semantic masks Generate a clear target object mask. and obstacle mask This is used to describe the location distribution of the target object and its surrounding obstacles in the image, at which point the environmental state constructed in step (1) is... After semantic information augmentation, it can be further represented as the semantically augmented environment state of the robot control system: (2) In the formula, express The system environment state that constantly contains rich semantic information; The environment height map generated in step (1) represents the spatial geometric height information of the scene; This represents the pixel-level semantic mask of the target object generated after processing by the base model. This represents the pixel-level semantic mask of the obstacle object generated after processing by the base model.

4. The robot grasping method for stacked scenes based on semantic guidance and spatial decoupling according to claim 1, characterized in that... The environmental state of the reinforcement learning environment constructed in step (3) Including environmental height maps and the target object mask and obstacle mask pixel-level semantic mask It is used to describe the spatial location, shape features, and semantic attributes of all objects in the current environment.

5. The robot grasping method for stacked scenes based on semantic guidance and spatial decoupling according to claim 1, characterized in that... The environmental state of the reinforcement learning environment The construction process and robot motion decisions specifically include the following: (3-1) Transform the two-dimensional environmental height map Pixel-level semantic mask of the corresponding task target object Pixel-level semantic masks for obstacles Alignment and stacking are performed in the spatial coordinate system to form a multi-channel tensor, which serves as the input feature map for the deep reinforcement learning network; (3-2) The robot will update the current environmental state at each moment. Input a pre-trained policy network, and through the forward computation of the policy network, automatically extract deep features that fuse spatial geometric information and object semantic information and map them to the action space; (3-3) For each pixel position, calculate the expected reward value, i.e., the Q value, for the robot to perform the "push" or "grab" action, and then select the action to be performed based on the maximum Q value. ; (3-3) The robot control system uses a reinforcement learning algorithm to optimize the parameters of the policy network end-to-end by using a dual dense reward function based on the change of grasp confidence and the change of mask intersection-union ratio (IoU). This enables the robot to learn autonomously and realize the "push-to-grasp" operation strategy in complex stacked environments.

6. The robot grasping method for stacked scenes based on semantic guidance and spatial decoupling according to claim 1, characterized in that... The functional expression for the action value map in step (4) is: (3) in, Here is the original action value map, where 'a' represents the robot's action: (4) In the formula, 'a' represents the robot's action, which includes pushing and grasping actions; This indicates the position of the execution center of the action in the environment state space coordinate system; This indicates the rotation angle when the robot's end effector performs an action.

7. The method for robot grasping in a stacked scene based on semantic guidance and spatial decoupling as described in claim 1, characterized in that... Step (5) specifically refers to: using the Adama product operation to transform the pixel-level semantic mask generated in step (2) into a pixel-level semantic mask. The action value map generated in step (4) is applied. This ensures that the action value map retains only action candidate points in semantically relevant regions, while setting the action value of background and irrelevant regions to zero. This ensures that operations are only performed within regions that may affect the acquisition of the target object. The semantically constrained action value map can be represented as: (5) In the formula, This represents the final action value map generated after semantic constraints, used to guide the robot to perform effective actions; This represents the original action value map generated by step (4); This represents the Hadamard product operation, which is the element-wise multiplication of two matrices of the same type. This represents the pixel-level semantic mask of the obstacle generated by step (2). In this matrix, the pixel value of the obstacle area is 1, and the pixel value of the rest area is 0.

8. The robot grasping method for stacked scenes based on semantic guidance and spatial decoupling according to claim 1, characterized in that... Step (6) refers to the system using a search algorithm to obtain the action value map after semantic constraints. The robot finds the pixel with the largest value in the algorithm. The spatial coordinates and rotation angle of this pixel represent the optimal action to be executed. By comparing the expected reward values ​​Q of different action candidate points, the robot control system can quantitatively determine which action is most valuable for achieving the grasping task and formulate the current execution strategy. The decision-making process can be represented as: (6) In the formula, This indicates the optimal action selected by the system under the current environmental conditions. This indicates a search and return function. The independent variable when it reaches its maximum value The operator.

9. A robot grasping method for stacked scenes based on semantic guidance and spatial decoupling according to claim 8, characterized in that... The actions include two types: pushing and grasping. When the target object is obstructed by other objects or there is insufficient grasping space, the robot prioritizes pushing, using the end effector of the robotic arm to push surrounding objects and create spatial separation between them. When the target object is in a graspable state, the robot performs grasping to complete the task of acquiring the target object.

10. The robot grasping method for stacked scenes based on semantic guidance and spatial decoupling according to claim 1, characterized in that... Formula (7) indicates that the total reward function of the system is a dual reward function containing both physical and spatial rewards. The physical reward is used to evaluate the impact of the current action on the feasibility of grasping the target object, and its calculation method is as follows: (8) in, This represents the crawling quality scoring function. When a driving action improves the crawling score, the system provides a positive reward. The space reward Used to measure the degree of spatial decoupling between objects, let the set of all objects in the scene be... The change in the Intersection over Union (IoU) of the masks between objects before and after the action is executed is calculated. To assess the improvement of environmental structure: (9) (10) In the formula, and Representing sets respectively Pixel-level semantic masks for different objects, when the pushing action reduces the overlapping area between objects. It is a positive value, after After function mapping Output positive reward values ​​to quantify the improvement in the degree of decoupling from the feedback environment.

Images