Robot target search method and robot simulation platform based on active perception and interactive operation collaboration

Abstract

The present invention discloses a robot target search method and a robot simulation platform based on active perception and interactive operation collaboration, the method comprising: acquiring a scene image through a robot wrist camera; adjusting the position and posture of the robot wrist camera based on an active perspective planning strategy until the perspective planning execution step number reaches a limit or the occlusion ratio of the target object is less than a threshold value before entering the next step; controlling the robot's mechanical arm to perform a push or grab operation based on an interactive operation strategy, returning to acquire a scene image after the mechanical arm performs a push operation; or, ending after the mechanical arm performs a grab operation and successfully grabs the target object; or, returning to acquire a scene image after the mechanical arm performs a grab operation and fails to grab the target object. The present invention collaborates with active perception and interactive operation, has strong environmental adaptability, and a high target grab success rate.

Description

Technical Field

[0001] The present invention belongs to the technical field of robot perception interaction, and specifically relates to a robot target search method and a robot simulation platform based on active perception and interactive operation collaboration. Background Art

[0002] At present, in the research of intelligent robots, the rise of deep learning technology has brought robots' perception and understanding of the environment to an unprecedented new height. Intelligent robots can use high-level computer vision models to perform semantic and pixel-level analysis of scenes. The integration of multimodal data such as vision, touch, and hearing has further improved the robot's perception ability, enabling robots to perform well in complex perception and operation tasks such as audio-visual navigation, dexterous operation, and human-computer interaction. Similarly, based on deep reinforcement learning technology, great progress has been made in the learning of robot operation skills. Robots have achieved good performance improvements in operation tasks such as desktop organization and parts grasping and assembly.

[0003] Although deep learning or deep reinforcement learning has improved the perception and interactive operation capabilities of robots, when robots face complex and unstructured environments, the overlapping and occlusion of different types of objects will seriously affect the perception accuracy of robots. Due to the limitation of a single fixed perspective (multiple targets occlude each other, resulting in the inability to identify targets, multiple similar objects causing ambiguity, or a single perspective cannot observe the entire scene), it is difficult for robots to complete target detection and recognition, 6D posture estimation, and subsequent perception and operation tasks. By using the perspective planning strategy in the robot's active vision, the limitation of a single fixed perspective can be broken. By controlling the movement of the visual sensor or actively adjusting the position of the visual sensor, the visibility of targets in complex environments can be improved, the accuracy of target detection and recognition can be improved, and the subsequent operation of the robot can be guided. However, how to improve the quality and efficiency of the robot's active perspective planning based on the task environment, hardware facilities, sensor settings, and planning strategies is a technical problem that needs to be solved urgently.

[0004] In complex desktop operation scenarios, targets are often stacked and hidden, making them invisible. Therefore, relying solely on perspective planning is not enough to complete the task. In unstructured scenarios, how robots can decouple the relationship between spatial pairs through environmental interaction operations to expose targets is also a technical challenge. Deep reinforcement learning combines the perception ability of deep learning with the decision-making ability of reinforcement learning. It can directly control the behavior of robots based on input information, giving robots a way of thinking close to that of humans. It is an important method for robots to acquire operational skills. There has been a lot of research on robot grasping strategies, but relying solely on grasping as an action primitive has poor adaptability to complex environments. Scenes with stacked objects often do not have enough space for robots to perform grasping operations. Secondly, in complex operating environments, the success rate of grasping targets is not high, and existing methods do not consider how robots can select the optimal grasping point based on dynamic changes in the scene.

[0005] In summary, existing technologies only achieve active perception of the robot combined with a single grasping action primitive or multi-action primitive interactive operations under a fixed perspective, isolating the process of active perception and interactive operation, and ignoring the information interaction and feedback between the perception and interactive operation processes. Summary of the invention

[0006] One of the purposes of the present invention is to provide a robot target search method based on active perception and interactive operation, which coordinates active perception and interactive operation, has strong environmental adaptability and high grasping success rate.

[0007] To achieve the above object, the technical solution adopted by the present invention is:

[0008] A robot target search method based on active perception and interactive operation collaboration, the robot has an operable mechanical arm, and a wrist camera is installed on the mechanical arm. The robot target search method based on active perception and interactive operation collaboration includes:

[0009] Step 1: Obtain scene images through the robot's wrist camera;

[0010] Step 2: Based on the active view planning strategy, adjust the position of the robot wrist camera until the view planning execution step number reaches the limit or the occlusion ratio of the target object is less than the set threshold, and then proceed to the next step;

[0011] Step 3: Based on the interactive operation strategy, control the robot's robotic arm to perform a push or grasping operation, and return to step 1 after the robotic arm performs a push operation; or, return to step 1 after the robotic arm performs a grasping operation and fails to grasp the target object; or, end after the robotic arm performs a grasping operation and successfully grasps the target object.

[0012] Several optional methods are also provided below, but they are not intended to be additional limitations on the above-mentioned overall solution, but are merely further supplements or preferences. Under the premise that there are no technical or logical contradictions, each optional method can be combined with the above-mentioned overall solution separately, and multiple optional methods can also be combined.

[0013] Preferably, the posture of the robot wrist camera is adjusted based on the active view planning strategy until the view planning execution step number reaches the limit or the occlusion ratio of the target object is less than the set threshold and then the next step is entered, including:

[0014] Processing the scene image using a target segmentation network to segment the scene into segmentation masks and corresponding categories of each object in the scene;

[0015] Determine whether the segmented objects contain the target object. If so, proceed to the next step; otherwise, proceed to step 3.

[0016] The segmented masks are input into the partial completion network in pairs to obtain the complete mask of the target object;

[0017] Calculate the occlusion ratio of the target object according to the complete mask of the target object and the segmentation mask obtained by segmentation;

[0018] If the number of view planning execution steps reaches the limit or the occlusion ratio of the target object is less than the set threshold, execute step 3; otherwise, input the intermediate layer features of the target segmentation network and the current position of the robot's wrist camera into the manipulator's active view planning algorithm, and send the output of the manipulator's active view planning algorithm to the robot to adjust the posture of the robot's wrist camera, and return to step 1.

[0019] Preferably, the partial completion network is trained by self-supervised learning, and the specific training process is as follows:

[0020] Generate random masks;

[0021] The occlusion mask is obtained by randomly occluding the target visible mask with a random mask;

[0022] Input the occlusion mask and the random mask into the partial completion network in pairs to obtain the target complete mask predicted and output by the partial completion network;

[0023] Calculate the loss function and update the network parameters based on the target complete mask and the target visible mask;

[0024] Continue training until the training end condition is reached.

[0025] Preferably, the method of controlling the robot's mechanical arm to perform a push or grab operation based on the interactive operation strategy includes:

[0026] Taking a scene image acquired by a wrist camera, wherein the scene image is an RGB-D image;

[0027] Processing the RGB-D image using an object segmentation network to segment the segmentation masks and corresponding categories of each object in the scene;

[0028] Convert RGB-D images into point cloud information in three-dimensional space;

[0029] Based on the point cloud information, orthogonal projection is performed in the opposite direction of gravity to obtain the height map of RGB and Depth. Based on the point cloud information and the segmentation mask, orthogonal projection is performed in the opposite direction of gravity to obtain the height map of the mask.

[0030] The RGB, Depth and mask height maps are spliced ​​and rotated through affine transformation and then input into the interactive operation strategy network. The output of the interactive operation strategy network is sent to the robot, so that the robot can control the action of the robotic arm according to the action primitive, action position and action direction corresponding to the maximum action value index. The action primitive is a push or grab operation.

[0031] Preferably, the interactive operation strategy network includes two branches, the first branch is a grasping operation branch, which includes a DenseNet121 network and a GraspNet network, the DenseNet121 network extracts the image features of the height map after affine transformation and rotation, and inputs the image features into the GraspNet network, and then the GraspNet network outputs an action value map of the grasping action according to the image features;

[0032] The second branch is the push operation branch, which includes the DenseNet121 network and the PushNet network. The DenseNet121 network extracts the image features of the height map after affine transformation and rotation, and inputs the image features into the PushNet network. The PushNet network then outputs the action value map of the push action based on the image features.

[0033] The interactive operation strategy network uses the Argmax function after two branches to obtain the action primitive, action position and action direction corresponding to the maximum action value index according to the action value graph of the grasping action and the action value graph of the pushing action.

[0034] The present invention provides a robot target search method based on active perception and interactive operation collaboration. The robot's active perception part can segment and identify scene objects under the current viewing angle according to the camera on the wrist of the robot arm, infer the stacking occlusion relationship, calculate the occlusion ratio of the target object, and autonomously adjust the position of the wrist camera to optimize the observation viewing angle of the target object. Under the optimized observation viewing angle, the interactive operation strategy selects the optimal combination of the robot arm's push and grasping actions, separates the stacked objects in the scene to reduce the scene complexity and expose the target object, improves the grasping success rate of the robot arm, and finally completes the target search task in complex scenes. Compared with the prior art, the robot target search system and method based on active perception and interactive operation collaboration proposed by the present invention have been experimentally verified to significantly improve the robot's perception accuracy and operation efficiency in complex scenes, have strong environmental adaptability, and have a high grasping success rate.

[0035] A second object of the present invention is to provide a robot simulation platform equipped with a high-quality algorithm to improve the environmental adaptability and grasping success rate of the simulation output robot.

[0036] To achieve the above object, the technical solution adopted by the present invention is:

[0037] A robot simulation platform, wherein the robot is a robot model or an actual robot, the robot has an operable mechanical arm, and a wrist camera is installed on the mechanical arm, the robot simulation platform comprises an algorithm simulation module, and the algorithm simulation module runs a pre-stored algorithm to perform the following operations:

[0038] Step 1: Obtain scene images through the robot's wrist camera;

[0039] Step 2: Based on the active view planning strategy, adjust the position of the robot wrist camera until the view planning execution step number reaches the limit or the occlusion ratio of the target object is less than the set threshold, and then proceed to the next step;

[0040] Step 3: Based on the interactive operation strategy, control the robot's robotic arm to perform a push or grasping operation, and return to step 1 after the robotic arm performs a push operation; or, return to step 1 after the robotic arm performs a grasping operation and fails to grasp the target object; or, end after the robotic arm performs a grasping operation and successfully grasps the target object.

[0041] Preferably, the posture of the robot wrist camera is adjusted based on the active view planning strategy until the view planning execution step number reaches the limit or the occlusion ratio of the target object is less than the set threshold and then the next step is entered, including:

[0042] Processing the scene image using a target segmentation network to segment the scene into segmentation masks and corresponding categories of each object in the scene;

[0043] Determine whether the segmented objects contain the target object. If so, proceed to the next step; otherwise, proceed to step 3.

[0044] The segmented masks are input into the partial completion network in pairs to obtain the complete mask of the target object;

[0045] Calculate the occlusion ratio of the target object according to the complete mask of the target object and the segmentation mask obtained by segmentation;

[0046] If the number of view planning execution steps reaches the limit or the occlusion ratio of the target object is less than the set threshold, execute step 3; otherwise, input the intermediate layer features of the target segmentation network and the current position of the robot's wrist camera into the manipulator's active view planning algorithm, and send the output of the manipulator's active view planning algorithm to the robot to adjust the posture of the robot's wrist camera, and return to step 1.

[0047] Preferably, the partial completion network is trained by self-supervised learning, and the specific training process is as follows:

[0048] Generate random masks;

[0049] The occlusion mask is obtained by randomly occluding the target visible mask with a random mask;

[0050] Input the occlusion mask and the random mask into the partial completion network in pairs to obtain the target complete mask predicted and output by the partial completion network;

[0051] Calculate the loss function and update the network parameters based on the target complete mask and the target visible mask;

[0052] Continue training until the training end condition is reached.

[0053] Preferably, the method of controlling the robot's mechanical arm to perform a push or grab operation based on the interactive operation strategy includes:

[0054] Taking a scene image acquired by a wrist camera, wherein the scene image is an RGB-D image;

[0055] Processing the RGB-D image using an object segmentation network to segment the segmentation masks and corresponding categories of each object in the scene;

[0056] Convert RGB-D images into point cloud information in three-dimensional space;

[0057] Based on the point cloud information, orthogonal projection is performed in the opposite direction of gravity to obtain the height map of RGB and Depth. Based on the point cloud information and the segmentation mask, orthogonal projection is performed in the opposite direction of gravity to obtain the height map of the mask.

[0058] The RGB, Depth and mask height maps are spliced ​​and input into the interactive operation strategy network after affine transformation and rotation. The output of the interactive operation strategy network is sent to the robot for the robot to control the action of the robotic arm according to the action primitive, action position and action direction corresponding to the maximum action value index. The action primitive is a push or grab operation.

[0059] Preferably, the interactive operation strategy network includes two branches, the first branch is a grasping operation branch, which includes a DenseNet121 network and a GraspNet network, the DenseNet121 network extracts the image features of the height map after affine transformation and rotation, and inputs the image features into the GraspNet network, and then the GraspNet network outputs an action value map of the grasping action according to the image features;

[0060] The second branch is the push operation branch, which includes the DenseNet121 network and the PushNet network. The DenseNet121 network extracts the image features of the height map after affine transformation and rotation, and inputs the image features into the PushNet network. The PushNet network then outputs the action value map of the push action based on the image features.

[0061] The interactive operation strategy network uses the Argmax function after two branches to obtain the action primitive, action position and action direction corresponding to the maximum action value index according to the action value graph of the grasping action and the action value graph of the pushing action. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 It is a flow chart of the robot target search method based on active perception and interactive operation collaboration of the present invention;

[0063] Figure 2 A flow chart of the active perspective planning strategy of the present invention;

[0064] Figure 3 The training flow chart of the partial completion network of the present invention;

[0065] Figure 4 The application flow chart of the network that partially completes the present invention;

[0066] Figure 5 Flow chart of the interactive operation strategy of the present invention. DETAILED DESCRIPTION

[0067] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0068] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0069] At present, the active perception process of intelligent robots in target search tasks has technical problems such as low segmentation and recognition accuracy of targets, lack of scene understanding ability in complex scenes, and low efficiency in adjusting the posture of robots during perspective planning. In addition, existing intelligent robots often use a single grasping action primitive in the process of interacting with the environment, which has a low grasping success rate and poor adaptability in complex environments.

[0070] In view of the above problems, this embodiment proposes a robot target search method that cooperates with active perception and interactive operation. The robot has an operable mechanical arm, and a wrist camera is installed on the mechanical arm. It is easy to understand that the robot involved in this embodiment itself is an existing structure, which usually includes basic components such as a base, a mechanical arm, a wrist camera and a power mechanism. This embodiment is implemented based on the mechanical arm and the wrist camera, and does not limit the robot to only include the mechanical arm and the wrist camera.

[0071] like Figure 1 As shown, a robot target search method based on active perception and interactive operation collaboration in this embodiment includes the following steps:

[0072] Step 1: Obtain scene images through the robot's wrist camera.

[0073] According to the target search needs, the wrist camera in this embodiment uses an RGB-D depth camera to obtain RGB-D images. In other embodiments, the camera type can also be adjusted according to actual needs.

[0074] Step 2: Based on the active view planning strategy, adjust the position and posture of the robot wrist camera until the view planning execution step number reaches the limit or the occlusion ratio of the target object is less than the set threshold, and then proceed to the next step.

[0075] like Figure 2As shown in the figure, the robot's active perspective planning strategy is centered on scene de-occlusion to improve the visual visibility of the target object, realizing the robot's active perception. It can be divided into three parts: target segmentation using RGB-D images for multimodal data fusion; target visual de-occlusion based on partial mask completion; and active perspective planning of the robotic arm based on deep reinforcement learning. The corresponding steps of each part are as follows.

[0076] Step 2-1: Use the target segmentation network to process the scene image to segment the segmentation masks and corresponding categories of each object in the scene.

[0077] The target segmentation network mainly adds a feature channel exchange module to the target segmentation network RefineNet. The channel measures the importance of the feature channel by the scaling factor of the batch normalization layer in the neural network, and dynamically adjusts the feature channel according to the scaling factor of the batch normalization layer. The target segmentation network uses the RGB-D images collected by the camera on the wrist of the robotic arm to achieve multimodal data fusion and improve the segmentation and recognition accuracy of target objects in complex environments.

[0078] It should be noted that the target segmentation network used in this embodiment is an existing network structure, such as the content disclosed in the document "Wang Y, Huang W, Sun F, et al. Deep multimodal fusion by channel exchanging [C]. Advances in Neural Information Processing Systems, 2020, 4835-4845". In other embodiments, a similar multimodal data fusion method can also be used to segment the target object, such as Segnet, CRFasRNN, PSPNet and other target segmentation networks.

[0079] Step 2-2, determine whether the segmented objects contain the target object, if so, proceed to the next step, i.e., step 2-3; otherwise, proceed to step 3;

[0080] When the target object is not found, this embodiment directly removes the interfering object through the push or grab action of the robot arm, thereby improving the quality and efficiency of subsequent active view planning.

[0081] Step 2-3: Input the segmented masks into pairs into the partial completion network to obtain the complete mask of the target object.

[0082] Partial mask completion is mainly used in visual deocclusion to give intelligent robots scene understanding capabilities. Visual deocclusion first trains a partial completion network through self-supervised learning to infer the spatial stacking occlusion relationship between objects in the scene, and gradually completes the complete mask of the target object based on the spatial stacking occlusion relationship between objects. After completion, the visual occlusion ratio of the target object can be calculated. The occlusion ratio can not only measure the quality of the camera observation view of the robot wrist, but also can be used as a reward function in the robot's active view planning algorithm.

[0083] like Figure 3 As shown, this embodiment trains the partial completion network by self-supervised learning, and the specific training process is as follows: generate a random mask; use the random mask to randomly occlude the target visible mask to obtain an occlusion mask; input the occlusion mask and the random mask into the partial completion network in pairs to obtain the target complete mask predicted and output by the partial completion network; calculate the loss function according to the target complete mask and the target visible mask and update the network parameters; continue training until the training end condition is reached.

[0084] This embodiment uses a random mask to randomly block the visible mask of the target to generate a variety of occlusion masks. This training method can avoid the problem of difficulty in collecting training data and ensure the diversity of training data, thereby improving the training effect of the partial completion network. This embodiment preferably takes the PCNet-M network as the partial completion network, and in other embodiments, it can be adjusted according to actual needs.

[0085] like Figure 4 As shown, when predicting the complete mask of the target object, the present embodiment combines the segmentation masks of each object output by the target segmentation network into pairs of segmentation masks in a round-robin manner, and the pairs of segmentation masks are sequentially input into the PCNet-M network, and the PCNet-M network predicts the occlusion relationship of the paired masks, thereby obtaining a scene occlusion relationship graph, and the complete mask of the target object is completed according to the scene occlusion relationship graph, and the obtained target complete mask is used to calculate the occlusion ratio of the target object.

[0086] Step 2-4: Calculate the occlusion ratio of the target object based on the complete mask of the target object and the segmented mask obtained by segmentation.

[0087] After obtaining the complete mask of the target object, the occlusion ratio of the target object can be calculated according to the segmented mask of the target object. The calculation method can be to subtract the area corresponding to the segmented mask from the area corresponding to the complete mask to obtain the occlusion mask, and use the ratio of the occlusion mask to the complete mask as the occlusion ratio. In other embodiments, other parameters, such as a proportional coefficient, etc., can be added when calculating the occlusion ratio, which is not limited in this embodiment.

[0088] Step 2-5: If the number of view planning execution steps reaches the limit or the occlusion ratio of the target object is less than the threshold, execute step 3; otherwise, input the intermediate layer features of the target segmentation network and the current position of the robot's wrist camera into the manipulator active view planning algorithm, send the output of the manipulator active view planning algorithm to the robot to adjust the posture of the robot's wrist camera, and return to step 1.

[0089] The active perspective planning algorithm of the robot arm is based on the deep reinforcement learning framework. In this embodiment, the Soft-Actor-Critic (SAC) algorithm is used to adjust the target to expose the target object, that is, to optimize and adjust in the direction of reducing the occlusion ratio of the target object. The active perspective planning algorithm of the robot arm uses the high-dimensional image feature information of the middle layer of the target segmentation network under the current perspective and the wrist camera posture information as state input. When optimizing the action strategy function to obtain a high reward value, it will also maximize the entropy value of the action strategy, ensuring that the action strategy of the robot arm has a stronger exploration ability.

[0090] It should be noted that in order to reduce the computing pressure, in this embodiment, the high-dimensional image feature information of the middle layer of the target segmentation network is used for calculation, that is, the features with smaller dimensions output by a certain middle layer of the target segmentation network are taken. There is no restriction on which layer is taken in this embodiment, and it can be selected according to the actual computing power of the device. And based on the deep reinforcement learning framework, different reinforcement learning algorithms can be used to control the posture of the end of the robot arm in real time according to the scene perception information to adjust the wrist camera angle to find the target.

[0091] The number of steps executed by the perspective planning refers to the number of steps executed by the robot arm in the perspective adjustment. In order to prevent the robot from falling into a perspective adjustment loop when the target object occlusion ratio is always large, this embodiment pre-sets the limit adjustment step number. When the perspective planning execution step number reaches the limit, the perspective adjustment is terminated to avoid wasting computing resources. In this embodiment, the setting threshold of the occlusion ratio of the target object is preferably set to 40%. In other embodiments, it can be adjusted according to actual needs.

[0092] Step 3: Based on the interactive operation strategy, control the robot's robotic arm to perform a push or grasping operation, and return to step 1 after the robotic arm performs a push operation; or, return to step 1 after the robotic arm performs a grasping operation and fails to grasp the target object; or, end after the robotic arm performs a grasping operation and successfully grasps the target object.

[0093] Based on the deep reinforcement learning framework, this embodiment proposes an interactive operation strategy for the robot arm with the core of decoupling the spatial stacking relationship through the interactive operation of multiple action primitives. The advantages of the combination of multiple action primitives are: first, the gripper at the end of the robot arm performs a horizontal push action to rearrange the objects, exposing the target objects that are blocked by large-area stacking, and providing an operational space for subsequent robot arm grasping; second, the robot arm can directly remove other objects in the scene by performing a grasping action, reducing the complexity of the stacking in the scene.

[0094] This embodiment enables the robot to learn the potential mapping relationship representation between the stacking state of the environment and the push-and-grasp action primitives, autonomously interact with the environment to obtain high action rewards, continuously optimize the collaborative strategy between action primitives, and achieve target grasping in complex scenarios.

[0095] It is easy to understand that the multi-action primitive combination in this embodiment is illustrated by taking pushing and grabbing as an example. In other embodiments, it can be replaced according to the needs of the robot operation scenario, for example, replaced with suction and grabbing, pushing and sucking, etc.

[0096] like Figure 5 As shown in the figure, the robot wrist camera collects RGB-D images in the scene as the original input of the interactive operation strategy. Subsequently, the RGB and Depth images are fused to complete the segmentation mask recognition of the scene objects. Secondly, the RGB-D information is converted into point cloud information in three-dimensional space. The point cloud data and the segmentation mask are orthogonally projected in the opposite direction of gravity to obtain the height map of RGB, Depth and mask to represent the state information of the environment. Finally, the three height maps are spliced ​​and affine transformed and rotated 16 times to input the correspondence between the interactive operation strategy network learning state and the robot arm action. The robot arm indexes the corresponding action primitive, action position and action direction action according to the maximum action value output by the interactive operation strategy network. The specific steps are as follows:

[0097] Step 3-1: obtain a scene image acquired by the wrist camera, where the scene image is an RGB-D image.

[0098] Step 3-2: Use the target segmentation network to process the RGB-D image to segment the segmentation mask and corresponding category of each object in the scene. The target segmentation network used here can be the same as the target segmentation network in step 2-1 to achieve the purpose of resource reuse. Of course, it can also be adjusted according to needs, which is not limited in this embodiment.

[0099] Step 3-3: Convert the RGB-D image into point cloud information in three-dimensional space.

[0100] Step 3-4: Based on the point cloud information, orthogonal projection is performed in the opposite direction of gravity to obtain the height map of RGB and Depth. Based on the point cloud information and the segmentation mask, orthogonal projection is performed in the opposite direction of gravity to obtain the height map of the mask.

[0101] Step 3-5: splice the RGB, Depth and mask height maps and rotate them through affine transformation before inputting them into the interactive operation strategy network. Send the output of the interactive operation strategy network to the robot, so that the robot can control the action of the robotic arm according to the action primitive, action position and action direction corresponding to the maximum action value index. The action primitive is a push or grab operation.

[0102] Among them, the interactive operation strategy network includes two branches. The first branch is the grasping operation branch, which includes the DenseNet121 network and the GraspNet network. The DenseNet121 network extracts the image features of the height map after affine transformation and rotation, and inputs the image features into the GraspNet network. The GraspNet network then outputs the action value map of the grasping action based on the image features.

[0103] The second branch is the push operation branch, which includes the DenseNet121 network and the PushNet network. The DenseNet121 network extracts the image features of the height map after affine transformation and rotation, and inputs the image features into the PushNet network. The PushNet network then outputs the action value map of the push action based on the image features.

[0104] And the interactive operation strategy network uses the Argmax function after the two branches to obtain the action primitive, action position and action direction corresponding to the maximum action value index according to the action value graph of the grasping action and the action value graph of the pushing action.

[0105] The back-end structures of the robot's interactive operation strategy network, GraspNet and PushNet, are mainly used to further process the image features extracted by the DenseNet121 network, and output the corresponding predicted action value graphs Q-Grasp and Q-Push. Both GraspNet and PushNet are fully convolutional neural networks with a network structure of BN-Relu-Conv-UpSample. The last layer is an upsampling layer using bilinear interpolation. After upsampling, the output image can be guaranteed to be consistent with the original input image size.

[0106] If the target object is fully or partially visible in a complex stacked scene, the robot arm can use the active view planning strategy to enhance the visual visibility of the target and guide the subsequent interaction between the robot arm and the environment. If the target is hidden by other objects and cannot be seen, the robot arm removes the interfering objects through pushing or grasping actions to improve the quality and efficiency of subsequent active view planning.

[0107] The active view planning strategy and interactive operation strategy of the robot arm in this embodiment can be executed alternately and coordinated with each other. After the environment is initialized, the wrist camera collects the scene RGB-D image, and the robot arm actively plans the view to adjust the wrist camera posture. If the view planning execution step number reaches the limit or the target occlusion ratio is less than the set threshold, the robot arm will perform interactive operations based on the scene information under the current view, and select the optimal combination of push and grab action primitives according to the action strategy, thereby improving the target search success rate and efficiency.

[0108] In another embodiment, a robot simulation platform is provided, wherein the robot is a robot model or an actual robot, the robot has an operable mechanical arm, and a wrist camera is installed on the mechanical arm, and the robot simulation platform includes an algorithm simulation module, and the algorithm simulation module runs a pre-stored algorithm to perform the following operations:

[0109] Step 1: Obtain scene images through the robot's wrist camera;

[0110] Step 2: Based on the active view planning strategy, adjust the position of the robot wrist camera until the view planning execution step number reaches the limit or the occlusion ratio of the target object is less than the set threshold, and then proceed to the next step;

[0111] Step 3: Based on the interactive operation strategy, control the robot's robotic arm to perform a push or grasping operation, and return to step 1 after the robotic arm performs a push operation; or, return to step 1 after the robotic arm performs a grasping operation and fails to grasp the target object; or, end after the robotic arm performs a grasping operation and successfully grasps the target object.

[0112] At present, training deep reinforcement learning algorithm models usually requires the robot arm to continuously interact with the environment to accumulate experience data. In actual situations, if the actual robot arm is directly used for algorithm model training, it will not only be inefficient but may even have safety hazards. This embodiment provides a robot simulation platform that can be used for active perception and interactive operation model training based on the actual robot. The robot simulation platform proposed in this embodiment is based on the CoppeliaSim simulation software. PyRep provides the remote control interface of CoppeliaSim, and RLBench provides a high-level task scheduling framework. The entire platform includes robot kinematics solution, dynamics solution, path planning, collision detection, virtual vision, mechanical sensors, etc. The simulation platform can also be connected to the open source robot operating system ROS to control the actual robot.

[0113] Based on the robot simulation platform, this embodiment establishes a multi-view occlusion data set to train the algorithm model for object occlusion relationship and object complete mask prediction. Based on this platform, the efficiency and reliability of the intelligent robot target search method based on active perception and interactive operation collaboration proposed in the present invention are verified. In the simulation environment, the domain randomization strategy is used to collect image information of stacked objects from multiple perspectives, including RGB-D image data from each perspective, real object segmentation masks, object occlusion stacking relationships, and occlusion ratio matrices between objects.

[0114] In one embodiment, the posture of the robot wrist camera is adjusted based on the active view planning strategy until the view planning execution step number reaches the limit or the occlusion ratio of the target object is less than the set threshold, and then the next step is entered, including:

[0115] Processing the scene image using a target segmentation network to segment the scene into segmentation masks and corresponding categories of each object in the scene;

[0116] Determine whether the segmented objects contain the target object. If so, proceed to the next step; otherwise, proceed to step 3.

[0117] The segmented masks are input into the partial completion network in pairs to obtain the complete mask of the target object;

[0118] Calculate the occlusion ratio of the target object according to the complete mask of the target object and the segmentation mask obtained by segmentation;

[0119] If the number of view planning execution steps reaches the limit or the occlusion ratio of the target object is less than the set threshold, execute step 3; otherwise, input the intermediate layer features of the target segmentation network and the current position of the robot's wrist camera into the manipulator's active view planning algorithm, and send the output of the manipulator's active view planning algorithm to the robot to adjust the posture of the robot's wrist camera, and return to step 1.

[0120] In one embodiment, the partial completion network is trained by self-supervised learning, and the specific training process is as follows:

[0121] Generate random masks;

[0122] The occlusion mask is obtained by randomly occluding the target visible mask with a random mask;

[0123] Input the occlusion mask and the random mask into the partial completion network in pairs to obtain the target complete mask predicted and output by the partial completion network;

[0124] Calculate the loss function and update the network parameters based on the target complete mask and the target visible mask;

[0125] Continue training until the training end condition is reached.

[0126] In one embodiment, the controlling the robot's mechanical arm to perform a push or grab operation based on the interactive operation strategy includes:

[0127] Taking a scene image acquired by a wrist camera, wherein the scene image is an RGB-D image;

[0128] Processing the RGB-D image using an object segmentation network to segment the segmentation masks and corresponding categories of each object in the scene;

[0129] Convert RGB-D images into point cloud information in three-dimensional space;

[0130] Based on the point cloud information, orthogonal projection is performed in the opposite direction of gravity to obtain the height map of RGB and Depth. Based on the point cloud information and the segmentation mask, orthogonal projection is performed in the opposite direction of gravity to obtain the height map of the mask.

[0131] The RGB, Depth and mask height maps are spliced ​​and input into the interactive operation strategy network after affine transformation and rotation. The output of the interactive operation strategy network is sent to the robot for the robot to control the action of the robotic arm according to the action primitive, action position and action direction corresponding to the maximum action value index. The action primitive is a push or grab operation.

[0132] In one embodiment, the interactive operation strategy network includes two branches. The first branch is a grasping operation branch, which includes a DenseNet121 network and a GraspNet network. The DenseNet121 network extracts image features of the height map after affine transformation and rotation, and inputs the image features into the GraspNet network. The GraspNet network then outputs an action value map of the grasping action according to the image features.

[0133] The second branch is the push operation branch, which includes the DenseNet121 network and the PushNet network. The DenseNet121 network extracts the image features of the height map after affine transformation and rotation, and inputs the image features into the PushNet network. The PushNet network then outputs the action value map of the push action based on the image features.

[0134] The interactive operation strategy network uses the Argmax function after two branches to obtain the action primitive, action position and action direction corresponding to the maximum action value index according to the action value graph of the grasping action and the action value graph of the pushing action.

[0135] For the specific limitations of the robot simulation platform, please refer to the above-mentioned limitations on the robot target search method based on active perception and interactive operation collaboration, which will not be repeated in this embodiment.

[0136] First, the advantages of the robot's active perspective planning strategy in this embodiment over the existing technology are: the channel exchange algorithm is used to fuse multimodal data to improve the accuracy of target detection; in view of the problem that the existing technology lacks scene understanding in the robot perception process, the partial mask of the object is used to predict the complete mask of the target, and the visibility ratio of the target is calculated to measure the perspective planning quality; combined with the robot's scene perception ability, the deep reinforcement learning algorithm is used to greatly improve the reliability and efficiency of the robot arm's autonomous adjustment of the perspective.

[0137] Secondly, in terms of interactive operation strategy, the present invention adopts a combination of robot arm push and grab action primitives to solve the problem of poor applicability of single grab action primitives in complex environments in the current technology. In view of the problem of target agnosticism of interactive operation strategies in the prior art, the present invention integrates target detection information into state representation and constructs an interactive operation strategy network and reward function, which improves the robot's efficiency in removing stacked occlusions for hidden targets.

[0138] The robot simulation platform in the present invention is used to test and verify the robot target search method with active perception and interactive operation. First, the experimental scene is divided into three different complexities according to the initial visibility of the target object, that is, the target is fully visible, partially blocked or completely blocked in the stacked scene. Secondly, the comparison algorithm is divided into four types: multi-action primitive operation under fixed perspective, single grasping operation under fixed perspective, single grasping operation under active perspective, and multi-action primitive operation coordinated by active perspective planning. Comparative experiments are set up under different scene complexities to verify the performance of the four algorithms respectively. Finally, it is verified that the intelligent robot target search method based on active perception and interactive operation coordination proposed in the present invention has a higher task completion rate and fewer interactive operation steps in scenes of different complexities, which proves that the present invention has higher efficiency and reliability than the existing methods.

[0139] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0140] The above-mentioned embodiments only express several implementation modes of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the invention. It should be pointed out that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the attached claims.

Claims

1. A robot target search method based on active perception and interactive operation collaboration, the robot has an operable mechanical arm, and the mechanical arm is equipped with a wrist camera, characterized in that: The robot target search method based on active perception and interactive operation collaboration includes: Step 1: Obtain scene images through the robot's wrist camera; Step 2: Based on the active view planning strategy, adjust the position of the robot wrist camera until the view planning execution step number reaches the limit or the occlusion ratio of the target object is less than the set threshold, and then proceed to the next step, including: Step 2-1, using the target segmentation network to process the scene image to segment the segmentation masks and corresponding categories of each object in the scene; Step 2-2, determine whether the segmented objects include the target object, if so, proceed to step 2-3; otherwise, proceed to step 3; Step 2-3, the segmented masks are input into the partial completion network in pairs to obtain the complete mask of the target object; Step 2-4, calculating the occlusion ratio of the target object according to the complete mask of the target object and the segmentation mask obtained by segmentation; Step 2-5: If the number of view planning execution steps reaches the limit or the occlusion ratio of the target object is less than the set threshold, execute step 3; otherwise, input the intermediate layer features of the target segmentation network and the current position of the robot's wrist camera into the active view planning algorithm of the manipulator, send the output of the active view planning algorithm of the manipulator to the robot to adjust the position of the wrist camera of the robot's manipulator, and return to step 1; Step 3: Based on the interactive operation strategy, control the robot's robotic arm to perform a push or grasping operation, and return to step 1 after the robotic arm performs a push operation; or, return to step 1 after the robotic arm performs a grasping operation and fails to grasp the target object; or, end after the robotic arm performs a grasping operation and successfully grasps the target object.

2. The robot target search method based on active perception and interactive operation collaboration as claimed in claim 1, characterized in that: The partial completion network is trained by self-supervised learning. The specific training process is as follows: Generate random masks; The occlusion mask is obtained by randomly occluding the target visible mask with a random mask; Input the occlusion mask and the random mask into the partial completion network in pairs to obtain the target complete mask predicted and output by the partial completion network; Calculate the loss function and update the network parameters based on the target complete mask and the target visible mask; Continue training until the training end condition is reached.

3. The robot target search method based on active perception and interactive operation collaboration as claimed in claim 1, characterized in that: The method of controlling the robot's mechanical arm to perform a push or grab operation based on the interactive operation strategy includes: Taking a scene image acquired by a wrist camera, wherein the scene image is an RGB-D image; Processing the RGB-D image using an object segmentation network to segment the segmentation masks and corresponding categories of each object in the scene; Convert RGB-D images into point cloud information in three-dimensional space; Based on the point cloud information, orthogonal projection is performed in the opposite direction of gravity to obtain the height map of RGB and Depth. Based on the point cloud information and the segmentation mask, orthogonal projection is performed in the opposite direction of gravity to obtain the height map of the mask. The RGB, Depth and mask height maps are spliced ​​and rotated through affine transformation and then input into the interactive operation strategy network. The output of the interactive operation strategy network is sent to the robot, so that the robot can control the action of the robotic arm according to the action primitive, action position and action direction corresponding to the maximum action value index. The action primitive is a push or grab operation.

4. The robot target search method based on active perception and interactive operation collaboration as claimed in claim 3, characterized in that: The interactive operation strategy network includes two branches. The first branch is a grasping operation branch, which includes a DenseNet121 network and a GraspNet network. The DenseNet121 network extracts the image features of the height map after affine transformation and rotation, and inputs the image features into the GraspNet network. The GraspNet network then outputs an action value map of the grasping action according to the image features. The second branch is the push operation branch, which includes the DenseNet121 network and the PushNet network. The DenseNet121 network extracts the image features of the height map after affine transformation and rotation, and inputs the image features into the PushNet network. The PushNet network then outputs the action value map of the push action based on the image features. The interactive operation strategy network uses the Argmax function after two branches to obtain the action primitive, action position and action direction corresponding to the maximum action value index according to the action value graph of the grasping action and the action value graph of the pushing action.

5. A robot simulation platform, wherein the robot is a robot model or an actual robot, the robot has an operable mechanical arm, and a wrist camera is installed on the mechanical arm, characterized in that: The robot simulation platform includes an algorithm simulation module, which runs a pre-stored algorithm to perform the following operations: Step 1: Obtain scene images through the robot's wrist camera; Step 2: Based on the active view planning strategy, adjust the position of the robot wrist camera until the view planning execution step number reaches the limit or the occlusion ratio of the target object is less than the set threshold, and then proceed to the next step, including: Step 2-1, using the target segmentation network to process the scene image to segment the segmentation masks and corresponding categories of each object in the scene; Step 2-2, determine whether the segmented objects include the target object, if so, proceed to step 2-3; otherwise, proceed to step 3; Step 2-3, the segmented masks are input into the partial completion network in pairs to obtain the complete mask of the target object; Step 2-4, calculating the occlusion ratio of the target object according to the complete mask of the target object and the segmentation mask obtained by segmentation; Step 2-5: If the number of view planning execution steps reaches the limit or the occlusion ratio of the target object is less than the set threshold, execute step 3; otherwise, input the intermediate layer features of the target segmentation network and the current position of the robot's wrist camera into the active view planning algorithm of the manipulator, send the output of the active view planning algorithm of the manipulator to the robot to adjust the position of the wrist camera of the robot's manipulator, and return to step 1; Step 3: Based on the interactive operation strategy, control the robot's robotic arm to perform a push or grasping operation, and return to step 1 after the robotic arm performs a push operation; or, return to step 1 after the robotic arm performs a grasping operation and fails to grasp the target object; or, end after the robotic arm performs a grasping operation and successfully grasps the target object.

6. The robot simulation platform according to claim 5, characterized in that: The partial completion network is trained by self-supervised learning. The specific training process is as follows: Generate random masks; The occlusion mask is obtained by randomly occluding the target visible mask with a random mask; Input the occlusion mask and the random mask into the partial completion network in pairs to obtain the target complete mask predicted and output by the partial completion network; Calculate the loss function and update the network parameters based on the target complete mask and the target visible mask; Continue training until the training end condition is reached.

7. The robot simulation platform according to claim 5, characterized in that: The method of controlling the robot's mechanical arm to perform a push or grab operation based on the interactive operation strategy includes: Taking a scene image acquired by a wrist camera, wherein the scene image is an RGB-D image; Processing the RGB-D image using an object segmentation network to segment the segmentation masks and corresponding categories of each object in the scene; Convert RGB-D images into point cloud information in three-dimensional space; Based on the point cloud information, orthogonal projection is performed in the opposite direction of gravity to obtain the height map of RGB and Depth. Based on the point cloud information and the segmentation mask, orthogonal projection is performed in the opposite direction of gravity to obtain the height map of the mask. The RGB, Depth and mask height maps are spliced ​​and input into the interactive operation strategy network after affine transformation and rotation. The output of the interactive operation strategy network is sent to the robot for the robot to control the action of the robotic arm according to the action primitive, action position and action direction corresponding to the maximum action value index. The action primitive is a push or grab operation.

8. The robot simulation platform according to claim 7, characterized in that: The interactive operation strategy network includes two branches. The first branch is a grasping operation branch, which includes a DenseNet121 network and a GraspNet network. The DenseNet121 network extracts the image features of the height map after affine transformation and rotation, and inputs the image features into the GraspNet network. The GraspNet network then outputs an action value map of the grasping action according to the image features. The second branch is the push operation branch, which includes the DenseNet121 network and the PushNet network. The DenseNet121 network extracts the image features of the height map after affine transformation and rotation, and inputs the image features into the PushNet network. The PushNet network then outputs the action value map of the push action based on the image features. The interactive operation strategy network uses the Argmax function after two branches to obtain the action primitive, action position and action direction corresponding to the maximum action value index according to the action value graph of the grasping action and the action value graph of the pushing action.

Images