Intelligent active perception and behavior interaction method in unknown shielding scene
By constructing a closed-loop reinforcement learning architecture of perception-decision-behavior, combined with a 2.5D occupancy height map and a dense reward function, the problem of insufficient perception information and inefficient action execution of robots in unknown occluded scenarios is solved, realizing efficient and robust non-grasping push operation and improving autonomous operation capability.
Patent Information
- Application Number
- CN202610165602.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-02-05
- Publication Date
- 2026-05-05
AI Technical Summary
In practical application scenarios such as bookshelves and lockers, the layout of objects is cluttered and there is severe spatial occlusion. Existing technologies suffer from problems such as incomplete perception information, inefficient action execution, disconnect between viewpoint planning and propulsion strategies, unstable reinforcement learning training, and insufficient generalization ability, making it difficult to operate efficiently and autonomously in unknown occluded environments.
A closed-loop reinforcement learning architecture of perception-decision-behavior is constructed. It combines a 2.5D occupancy height map, a push point selection mechanism that integrates geometric features and task heuristic weights, and a dense reward function. Data is collected by the robot equipped with an RGB-D camera, and the viewpoint planning and push strategy are deeply integrated to achieve non-grasping push operation.
It significantly improves the robot's environmental understanding and action execution reliability in unknown occluded scenarios, effectively distinguishes operable areas in complex scenarios, reduces the ambiguity of pushing positions, and demonstrates good adaptability and efficient unlocking of occluded areas in both simulation and real environments.
Smart Images

Figure CN121982679A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of robot autonomous operation and reinforcement learning technology, specifically to an intelligent active perception and behavioral interaction method in unknown occlusion scenarios, which can be applied to autonomous operations in cluttered and occluded scenarios such as bookshelf organization and locker cleaning. Background Technology
[0002] In practical applications such as bookshelves and lockers, the layout of objects is often cluttered and severely obstructed. Autonomous robot operation requires addressing the core issues of incomplete perception information and inefficient action execution. Traditional viewpoint planning methods focus solely on environmental observation optimization without coordinating with operational actions; grasping-based operation strategies cause significant disturbances to the scene and have low success rates in occluded environments; while non-grasping pushing methods offer flexibility, existing technologies suffer from the following drawbacks: The separation of perception and behavior: viewpoint planning and driving strategies are designed independently, lacking closed-loop collaborative optimization, resulting in perception information being unable to effectively support action decisions; Blind selection of push points: Traditional sampling methods rely on only a single geometric feature, making it difficult to distinguish the effective operating area in complex scenes, resulting in ambiguity in the push position; Reinforcement learning training is unstable: reward signals are sparse, which can easily lead to problems such as inefficient exploration and slow policy convergence, and it is difficult to adapt to the dynamic changes of unknown occluded environments; Insufficient generalization ability: Existing methods often assume full observation of the scene and regular structure, resulting in poor adaptability in real-world environments with multiple object occlusions and perceived noise.
[0003] Therefore, there is an urgent need for a combined approach that deeply integrates active perception and propulsion strategies, employs a scientific sampling mechanism, has a reasonable reward design, and is easy to train, in order to improve the robot's autonomous operation capabilities in unknown occluded scenarios. Summary of the Invention
[0004] To address the shortcomings of existing technologies, the purpose of this invention is to provide an intelligent active perception and behavioral interaction method in unknown occlusion scenarios. By constructing a perception-decision-behavior closed-loop reinforcement learning architecture, it solves problems such as insufficient perceptual information, low pushing efficiency, and unstable training, achieving efficient and robust non-grasping pushing operations. To achieve the above objective, the technical solution adopted by this invention is as follows: A method for intelligent proactive perception and behavioral interaction in unknown occlusion scenarios is proposed. Its core lies in constructing a closed-loop reinforcement learning architecture, combining a 2.5D occupancy height map, a push point selection mechanism that integrates geometric features and task-heuristic weights, and a dense reward function. This method efficiently explores unknown occlusion environments with minimal invasiveness, achieving scene unlocking and reasonable structural optimization. The method includes the following steps: Step 1: First, the robot is equipped with an RGB-D camera to reach multiple predetermined viewpoints to collect scene data and obtain the core part of the reinforcement learning observation space—the 2.5D occupancy height map, which serves as the core data for the viewpoint planning strategy of the active perception agent and the driving strategy learning of the behavioral interaction agent. Step 2: Based on the initial 2.5D occupancy height map constructed in Step 1, a viewpoint planning strategy for the active perception agent is trained using the reinforcement learning TQC algorithm. The viewpoint planning strategy generates the next viewpoint position in space, allowing the robot to adjust its observation viewpoint to obtain new scene data, and continuously and incrementally updates the 2.5D occupancy height map until the maximum visible range is obtained; Step 3: The 2.5D occupancy height map constructed by the active sensing agent in Step 2 is further filtered using a push point screening mechanism that integrates geometric features and task heuristic weights to determine the push direction that can effectively act on the object without causing the object to become unstable, and finally a batch of candidate push schemes are obtained. Step 4: The robot learns the driving strategy of the behavioral interaction agent through the reinforcement learning TQC algorithm. This strategy selects the best driving point from all the candidate driving schemes in Step 3, performs a non-grasping driving operation, and pushes the target object to move and unlock the occluded area, exploring as many areas as possible with minimal invasiveness. Step 5: Restart the active sensing agent to obtain changes in the scene before and after the push, and determine whether the current environment meets the task termination condition. If not, return to step 2 to restart the viewpoint planning strategy and enter the next loop; if it meets the condition, end the current task and output the environment exploration results.
[0005] The 2.5D occupancy height map is established as follows: using the RGB-D depth camera mounted on the robot, scene depth images and color images are acquired from the current perspective. The pixel coordinates (u,v) of the depth image are converted into three-dimensional camera coordinates (Xc,Yc,Zc) through the camera intrinsic parameter matrix. Then, the coordinates are transformed into three-dimensional coordinates (Xw,Yw,Zw) in the world coordinate system to generate scene point clouds. The point clouds are then cropped according to the set boundaries. The Xw-Yw plane in the world coordinate system is divided into a two-dimensional pixel grid with a fixed resolution. Each two-dimensional pixel grid corresponds to a plane coordinate (x, y). The number of three-dimensional point clouds in each two-dimensional pixel grid is counted to obtain the height occupancy data. If the number of three-dimensional point clouds is greater than or equal to a preset threshold and the height value h is greater than or equal to the height threshold τh, then the two-dimensional pixel grid is determined to be occupied by an object, and the occupancy probability is assigned a value of p=0.9. If the number of three-dimensional point clouds is 0 or h<τh, then it is determined to be an objectless grid, and the value of p=0.1 is assigned. If the number of point clouds is between 0 and the preset threshold, the value of p=0.5 is assigned. Multi-view fusion update: When the robot adjusts its viewpoint, it acquires the height occupancy data under the new viewpoint, updates the height value h of the same pixel grid with a weighted average, and updates the occupancy probability p with a Bayesian fusion rule, ultimately forming a dynamically updated, multi-view fusion 2.5D occupancy height map.
[0006] τh=0.07m.
[0007] The mechanism for selecting driving points that integrates geometric features and task-inspired weights specifically includes: performing raster ray projection based on a 2.5D occupancy height map to obtain all points with visible contact as initial candidate driving points, and scoring and sorting the driving points according to the geometric feature dimension and the task-inspired weight dimension.
[0008] The geometric feature dimensions are: the height and volume features of the object are extracted based on the 2.5D occupancy height map; the height feature is the height value h corresponding to each candidate push point, and valid targets with h≥τh are screened; the volume feature is the volume V of the object to which the candidate point belongs, calculated by the number of connected pixel grids × grid area × average height. The task-heuristic weighting dimension is: combining the positional features of objects in the 2.5D occupancy height map with the environmental exploration requirements; the positional features are: the distance between the candidate point's corresponding planar coordinates (x, y) and the camera; the environmental exploration requirements are: the distance between the candidate point and the unknown area.
[0009] Using an initial 2.5D occupancy height map as the data basis, the viewpoint planning strategy of the active perception agent and the driving strategy of the behavioral interaction agent are trained respectively through the reinforcement learning TQC algorithm.
[0010] Design the reward function, action space, and observation space in the reinforcement learning TQC algorithm; Design a reward function: The viewpoint planning strategy reward consists of two parts: in The motion cost for two consecutive viewpoint changes. The percentage reduction in entropy in the 2.5D occupied height map between two viewpoint changes; The incentive strategy rewards consist of four parts: When the entropy decreases beyond the threshold, the push is considered to have achieved a significant perceptual gain, the current push action is recorded as successful, and the following reward is given: Where δ=15 is the reward gain obtained from the first significant entropy reduction, which is used to encourage the strategy to actively explore information-dense areas; Introduce the change in entropy between two consecutive intervals: like ≤10% indicates insufficient behavioral improvement, and punishment will be imposed: For information gain If the percentage is less than 5% or the action causes a collision, a penalty will be imposed: Considering the execution cost, a combined operational cost term is introduced, which includes the number of pushes k and the displacement distance d: The complete combination of reward functions is as follows: Design action space: The action space of the viewpoint planning strategy is a five-dimensional continuous space in the 6D Cartesian pose of the camera on the end effector, excluding the roll angle; the action space of the push point planning is a one-dimensional discrete space, which is used to select the best push point in the form of an index among the push points generated after the sampling mechanism of fusing geometric features and task heuristic weights. Design Observation Space: In the viewpoint planning strategy phase, a 43-dimensional continuous vector space is defined: the 2.5D occupancy height map is encoded into a 32-dimensional latent space using a variational autoencoder; the 5-dimensional vector space represents the previous target position; one dimension each represents information gain, motion cost, and collision detection flag; and the last 3 dimensions represent the center points of unknown regions in the 2.5D occupancy height map. In the push strategy phase, 37 of these dimensions are retained: the 32-dimensional latent space, information gain, collision detection flag, and the center points of unknown regions in the 3-dimensional height occupancy map are used as the observation space for push point planning, and candidate push point guidance strategies are added for learning.
[0011] Compared with the prior art, the beneficial effects of the present invention are as follows: For the first time, proactive viewpoint planning and propulsion strategies are deeply integrated, which solves the problem of the separation between perception and action in traditional methods and significantly improves the completeness of environmental understanding and the reliability of action execution. The innovative approach proposes determining optimization directions based on unknown regions and occlusion relationships in a 2.5D height occupancy map, providing sufficient perception data to drive decision-making while significantly reducing the amount of data.
[0012] Meanwhile, this invention integrates geometric features and task-heuristic weights to effectively distinguish operable areas in complex scenes and reduce the ambiguity of the push position; and uses a dense reward function to alleviate the reward sparsity problem. The adaptability of the method under different occlusion scenarios has been verified in simulation and real environment experiments. Finally, non-grabbing push minimizes environmental disturbance while efficiently unlocking obscured areas, creating favorable conditions for downstream tasks such as grabbing and cleaning. Attached Figure Description
[0013] Figure 1 A holistic decision-making diagram for proactive perception and behavioral interaction; Figure 2 Example of a 2.5D occupancy height map for visualization; Figure 3 To proactively perceive decision maps; Figure 4 This is a schematic diagram of the simulation experimental environment layout (including the UR5 robotic arm, Robotiq2f85 fixture, and cluttered objects). Figure 5 This is a schematic diagram of the actual experimental environment (30cm×60cm×30cm push frame and test object). Detailed Implementation
[0014] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments: An intelligent proactive perception and behavioral interaction method for unknown occlusion scenarios is proposed. This method constructs a closed-loop reinforcement learning architecture, combining a 2.5D occupancy height map, a pushpoint selection mechanism that integrates geometric features and task-heuristic weights, and a dense reward function. It efficiently explores unknown occlusion environments with minimal invasiveness, achieving scene unlocking and structural optimization. The method includes the following steps: Step 1: First, the robot is equipped with an RGB-D camera to reach multiple predetermined viewpoints to collect scene data and obtain an initial 2.5D occupancy height map. This map serves as the core data for the viewpoint planning strategy of the active perception agent and the driving strategy learning of the behavioral interaction agent. The 2.5D occupancy height map is an integrated environmental representation model that combines two-dimensional planar positioning, three-dimensional height quantization, and spatial occupancy probability, forming a three-in-one environmental data matrix of "coordinates-height-occupancy probability". Step 2: Based on the initial 2.5D occupancy height map constructed in Step 1, the viewpoint planning strategy of the active perception agent is trained by the reinforcement learning TQC algorithm. The viewpoint planning strategy generates the next viewpoint position in space by analyzing the distribution of unknown areas, occlusion relationships and the integrity of visible areas in the occupancy height map. The robot adjusts the observation viewpoint to obtain new scene data and continuously updates it incrementally to the 2.5D occupancy height map until the maximum visible range is obtained. Step 3: The 2.5D occupancy height map constructed by the active sensing agent in Step 2 is further filtered using a push point screening mechanism that integrates geometric features and task heuristic weights to determine the push direction that can effectively act on the object without causing the object to become unstable, and finally a batch of candidate push schemes are obtained. Step 4: The robot learns the driving strategy of the behavioral interaction agent through the reinforcement learning TQC algorithm. This strategy selects the best driving point from all the candidate driving schemes in Step 3, performs a non-grasping driving operation, and pushes the target object to move and unlock the occluded area, exploring as many areas as possible with minimal invasiveness. Step 5: Restart the active sensing agent to obtain changes in the scene before and after the push, and determine whether the current environment meets the task termination condition. If not, return to step 2 to restart the viewpoint planning strategy and enter the next loop; if it meets the condition, end the current task and output the environment exploration results.
[0015] The 2.5D occupancy height map is established as follows: using the RGB-D depth camera mounted on the robot, scene depth images and color images are acquired from the current perspective. The pixel coordinates (u,v) of the depth image are converted into three-dimensional camera coordinates (Xc,Yc,Zc) through the camera intrinsic parameter matrix. Then, the coordinates are transformed into three-dimensional coordinates (Xw,Yw,Zw) in the world coordinate system to generate scene point clouds. The point clouds are then cropped according to the set boundaries. The Xw-Yw plane in the world coordinate system is divided into a two-dimensional pixel grid with a fixed resolution. Each two-dimensional pixel grid corresponds to a plane coordinate (x, y). The number of three-dimensional point clouds in each two-dimensional pixel grid is counted to obtain the height occupancy data. If the number of three-dimensional point clouds is greater than or equal to a preset threshold and the height value h is greater than or equal to the height threshold τh, then the two-dimensional pixel grid is determined to be occupied by an object, and the occupancy probability is assigned a value of p=0.9. If the number of three-dimensional point clouds is 0 or h<τh, and τh is set to 0.07m, then it is determined to be an empty object, and the value of p=0.1 is assigned. If the number of point clouds is between 0 and the preset threshold, then the value of p=0.5 is assigned. Multi-view fusion update: When the robot adjusts its viewpoint, it acquires the height occupancy data under the new viewpoint, updates the height value h of the same pixel grid with a weighted average, and updates the occupancy probability p with a Bayesian fusion rule, ultimately forming a dynamically updated, multi-view fusion 2.5D occupancy height map.
[0016] The mechanism for selecting driving points that integrates geometric features and task-inspired weights specifically includes: performing raster ray projection based on a 2.5D occupancy height map to obtain all points with visible contact as initial candidate driving points, and scoring and sorting the driving points according to the geometric feature dimension and the task-inspired weight dimension.
[0017] The geometric feature dimensions are: height and volume features of the object extracted based on a 2.5D occupancy height map; the height feature is the height value h corresponding to each candidate push point, and valid targets with h ≥ τh are selected; the larger the object volume, the greater the possibility of information gain from the push. The volume feature is calculated by multiplying the number of connected pixel grids by the grid area by the average height, resulting in the volume V of the object to which the candidate point belongs. The task-heuristic weighting dimension is: combining the positional features of objects in the 2.5D occupancy height map with the environmental exploration requirements; the positional features are: the distance between the candidate point's corresponding planar coordinates (x, y) and the camera; the environmental exploration requirements are: the distance between the candidate point and the unknown area.
[0018] Using an initial 2.5D occupancy height map as the data basis, the viewpoint planning strategy of the active perception agent and the driving strategy of the behavioral interaction agent are trained respectively through the reinforcement learning TQC algorithm.
[0019] Design the reward function, action space, and observation space in the reinforcement learning TQC algorithm; Design a reward function: The viewpoint planning strategy reward consists of two parts: in The motion cost for two consecutive viewpoint changes. The percentage reduction in entropy in the 2.5D occupied height map between two viewpoint changes; The incentive strategy rewards consist of four parts: When the entropy decreases beyond the threshold, the push is considered to have achieved a significant perceptual gain, the current push action is recorded as successful, and the following reward is given: Where δ=15 is the reward gain obtained from the first significant entropy reduction, which is used to encourage the strategy to actively explore information-dense areas; Introduce the change in entropy between two consecutive intervals: like ≤10% indicates insufficient behavioral improvement, and punishment will be imposed: For information gain If the percentage is less than 5% or the action causes a collision, a penalty will be imposed: Considering the execution cost, a combined operational cost term is introduced, which includes the number of pushes k and the displacement distance d: The complete combination of reward functions is as follows: Design action space: The action space of the viewpoint planning strategy is a five-dimensional continuous space in the 6D Cartesian pose of the camera on the end effector, excluding the roll angle; the action space of the push point planning is a one-dimensional discrete space, which is used to select the best push point in the form of an index among the push points generated after the sampling mechanism of fusing geometric features and task heuristic weights. Design Observation Space: In the viewpoint planning strategy phase, a 43-dimensional continuous vector space is defined: the 2.5D occupancy height map is encoded into a 32-dimensional latent space using a variational autoencoder; the 5-dimensional vector space represents the previous target position; one dimension each represents information gain, motion cost, and collision detection flag; and the last 3 dimensions represent the center points of unknown regions in the 2.5D occupancy height map. In the push strategy phase, 37 of these dimensions are retained: the 32-dimensional latent space, information gain, collision detection flag, and the center points of unknown regions in the 3-dimensional height occupancy map are used as the observation space for push point planning, and candidate push point guidance strategies are added for learning.
[0020] Example 1: Simulation Environment Experiment 1. Experimental equipment: Please refer to [link / reference]. Figure 4 It uses the PyBullet simulation platform, is equipped with a six-DOF UR5 robotic arm and Robotiq 2f85 gripper, and runs on a computer equipped with an i7-13650HX CPU and an RTX4060 GPU; 2. Scene Setup: Construct a cluttered occlusion scene containing objects of various shapes (cubes, cylinders), with objects randomly placed to simulate the complexity of a real environment; 3. Comparison methods: stochastic push strategy, VPP method (viewpoint planning and push separation); 4. Evaluation indicators: success rate (entropy reduction ≥ 35%), collapse rate, entropy reduction rate, number of steps, and exercise cost; 5. Experimental results: The success rate of the method of this invention reached 73%, and the entropy reduction rate was 47.44%, which is significantly better than random push (45%, 35.12%) and VPP method (30%, 25.35%), verifying the superiority of the technical solution.
[0021] Example 2: Real-world environment experiment; 1. Experimental equipment: Please refer to [link / reference]. Figure 5 Similar to the robotic arm configuration in the simulation environment, the real environment is a 30cm×60cm×30cm push rack for placing everyday items (boxes, tools, etc.). 2. Scene Setup: Design 50 random occlusion scenarios, taking into account perceived noise and dynamic changes in the physical environment; 3. Experimental results: The method of this invention still maintains a high success rate in real environment (slightly lower than the simulation environment but better than the baseline method), and the entropy reduction efficiency is stable, showing good real-world adaptability and anti-interference ability.
[0022] Example 3: Ablation experiment verification; To verify the function of the core module, an ablation experiment was designed to test the performance of different module configurations: Full Model: Contains all core modules; Viewless planning (w / o view planning): Fixed initial observation position; Height Only: This method pushes point sampling to consider only height features. Volume Only Sampling: Push point sampling only considers volume features; Position Only: Push point sampling only considers positional features; Experimental results show that the complete model significantly outperforms other configurations in terms of success rate (73%) and entropy reduction rate (47.44%), verifying the key role of active viewpoint planning and multi-feature fusion sampling strategy.
[0023] It should be noted that the terms "comprising," "including," or any other variations thereof in this invention are intended to cover non-exclusive inclusion, such that a process or method that comprises a list of elements includes not only said elements but also other elements not expressly listed, or elements inherent to such a process or method. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process or method that includes said element.
[0024] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.
Claims
1. A method for intelligent active perception and behavioral interaction in unknown occlusion scenarios, characterized in that, By constructing a closed-loop reinforcement learning architecture, combining a 2.5D occupancy height map, a push point selection mechanism that integrates geometric features and task-heuristic weights, and a dense reward function, we can efficiently explore unknown occluded environments with minimal invasiveness, thereby unlocking the scene and optimizing its structure. This includes the following steps: Step 1: First, the robot is equipped with an RGB-D camera to reach multiple predetermined viewpoints to collect scene data and obtain an initial 2.5D occupancy height map as the core data for learning the viewpoint planning strategy of the active perception agent and the driving strategy of the behavioral interaction agent. Step 2: Based on the initial 2.5D occupancy height map constructed in Step 1, the viewpoint planning strategy of the active perception agent is trained through the reinforcement learning TQC algorithm. The viewpoint planning strategy generates the next viewpoint position in space. The robot adjusts the observation viewpoint to obtain new scene data and continuously updates it incrementally to the 2.5D occupancy height map until the maximum visible range is obtained. Step 3: The 2.5D occupancy height map constructed by the active sensing agent in Step 2 is further filtered using a push point screening mechanism that integrates geometric features and task heuristic weights to determine the push direction that can effectively act on the object without causing the object to become unstable, and finally a batch of candidate push schemes are obtained. Step 4: The robot learns the driving strategy of the behavioral interaction agent through the reinforcement learning TQC algorithm. This strategy selects the best driving point from all the candidate driving schemes in Step 3, performs a non-grasping driving operation, and pushes the target object to move and unlock the occluded area, exploring as many areas as possible with minimal invasiveness. Step 5: Restart the active sensing agent to obtain changes in the scene before and after the push, and determine whether the current environment meets the task termination condition. If not, return to step 2 to restart the viewpoint planning strategy and enter the next loop; if it meets the condition, end the current task and output the environment exploration results.
2. The intelligent active perception and propulsion joint method in unknown occlusion scenarios according to claim 1, characterized in that, The 2.5D occupancy height map is established as follows: using the RGB-D depth camera mounted on the robot, scene depth images and color images are acquired from the current perspective. The pixel coordinates (u,v) of the depth image are converted into three-dimensional camera coordinates (Xc,Yc,Zc) through the camera intrinsic parameter matrix. Then, the coordinates are transformed into three-dimensional coordinates (Xw,Yw,Zw) in the world coordinate system to generate scene point clouds. The point clouds are then cropped according to the set boundaries. The Xw-Yw plane in the world coordinate system is divided into a two-dimensional pixel grid with a fixed resolution. Each two-dimensional pixel grid corresponds to a plane coordinate (x, y). The number of three-dimensional point clouds in each two-dimensional pixel grid is counted to obtain the height occupancy data. If the number of three-dimensional point clouds is greater than or equal to a preset threshold and the height value h is greater than or equal to the height threshold τh, then the two-dimensional pixel grid is determined to be occupied by an object, and the occupancy probability is assigned a value of p=0.
9. If the number of three-dimensional point clouds is 0 or h<τh, then it is determined to be an objectless grid, and the value of p=0.1 is assigned. If the number of point clouds is between 0 and the preset threshold, the value of p=0.5 is assigned. Multi-view fusion update: When the robot adjusts its viewpoint, it acquires the height occupancy data under the new viewpoint, updates the height value h of the same pixel grid with a weighted average, and updates the occupancy probability p with a Bayesian fusion rule, ultimately forming a dynamically updated, multi-view fusion 2.5D occupancy height map.
3. The intelligent active perception and propulsion joint method in unknown occlusion scenarios according to claim 2, characterized in that, τh=0.07m.
4. The intelligent active perception and behavioral interaction method in unknown occlusion scenarios according to claim 1, characterized in that, The mechanism for selecting driving points that integrates geometric features and task-inspired weights specifically includes: performing raster ray projection based on a 2.5D occupancy height map to obtain all points with visible contact as initial candidate driving points, and scoring and sorting the driving points according to the geometric feature dimension and the task-inspired weight dimension.
5. The intelligent active perception and behavioral interaction method in unknown occlusion scenarios according to claim 4, characterized in that, The geometric feature dimensions are: the height and volume features of the object extracted based on the 2.5D occupancy height map; The height feature is the height value h corresponding to each candidate push point, and valid targets with h ≥ τh are selected; the volume feature is the volume V of the object to which the candidate point belongs, calculated by the number of connected pixel grids × grid area × average height. The task-heuristic weighting dimension is: combining the positional features of objects in the 2.5D occupancy height map with the environmental exploration requirements; the positional features are: the distance between the candidate point's corresponding planar coordinates (x, y) and the camera; the environmental exploration requirements are: the distance between the candidate point and the unknown area.
6. The intelligent active perception and behavioral interaction method in unknown occlusion scenarios according to claim 1, characterized in that, Using an initial 2.5D occupancy height map as the data basis, the viewpoint planning strategy of the active perception agent and the driving strategy of the behavioral interaction agent are trained respectively through the reinforcement learning TQC algorithm.
7. The intelligent active perception and behavioral interaction method in unknown occlusion scenarios according to claim 6, characterized in that, Design the reward function, action space, and observation space in the reinforcement learning TQC algorithm; Design a reward function: The viewpoint planning strategy reward consists of two parts: ; ; in The motion cost for two consecutive viewpoint changes. The percentage reduction in entropy in the 2.5D occupied height map between two viewpoint changes; The incentive strategy rewards consist of four parts: When the entropy decreases beyond the threshold, the push is considered to have achieved a significant perceptual gain, the current push action is recorded as successful, and the following reward is given: ; Where δ=15 is the reward gain obtained from the first significant entropy reduction, which is used to encourage the strategy to actively explore information-dense areas; Introduce the change in entropy between two consecutive intervals: ; like ≤10% indicates insufficient behavioral improvement, and punishment will be imposed: ; For information gain If the percentage is less than 5% or the action causes a collision, a penalty will be imposed: ; Considering the execution cost, a combined operational cost term is introduced, which includes the number of pushes k and the displacement distance d: ; The complete combination of reward functions is as follows: ; Design action space: The action space of the viewpoint planning strategy is a five-dimensional continuous space in the 6D Cartesian pose of the camera on the end effector, excluding the roll angle; the action space of the push point planning is a one-dimensional discrete space, which is used to select the best push point in the form of an index among the push points generated after the sampling mechanism of fusing geometric features and task heuristic weights. Design Observation Space: In the viewpoint planning strategy phase, a 43-dimensional continuous vector space is defined: the 2.5D occupancy height map is encoded into a 32-dimensional latent space using a variational autoencoder; the 5-dimensional vector space represents the previous target position; one dimension each represents information gain, motion cost, and collision detection flag; and the last 3 dimensions represent the center points of unknown regions in the 2.5D occupancy height map. In the push strategy phase, 37 of these dimensions are retained: the 32-dimensional latent space, information gain, collision detection flag, and the center points of unknown regions in the 3-dimensional height occupancy map are used as the observation space for push point planning, and candidate push point guidance strategies are added for learning.
Citation Information
Cited By
A robot pushing and grabbing collaborative work method and device based on active interactive perception
CN122125727A
