Robot grasping method and apparatus, electronic device, and storage medium

Abstract

Embodiments of the present application provide a robot grasping method and device, electronic equipment and storage medium, and relate to the field of robot intelligent control. First, a camera installed on the robot is used to capture the target object from multiple perspectives to obtain an image set and camera parameters corresponding to each perspective; then the image set and the camera parameters are input into a pre-trained detection model to predict the three-dimensional coordinates of the key points of the target object, obtain a prediction result, and calculate the pose of the target object according to the spatial geometric relationship between the prediction result and the key points; finally, the pose of the target object and the pre-trained grasping model are used to control the robot to complete the grasping of the target object.

Description

Technical Field

[0001] This invention relates to the field of intelligent robot control, and more specifically, to a robot grasping method, device, electronic device, and storage medium. Background Technology

[0002] With the development of intelligent manufacturing technology and information technology, the level of intelligence of robots is constantly improving, and they are being used more and more widely in various industries. In particular, in the industrial field, the use of industrial robots has greatly improved production efficiency.

[0003] In industrial production, robot grasping is one of the most common and basic operations. Researching ways to improve the accuracy and stability of robot grasping is of great significance for further improving industrial production efficiency. Summary of the Invention

[0004] The present invention aims to provide, for example, a robot grasping method, apparatus, electronic device, and storage medium, which can determine the pose of a target object based on images of the target object captured by a camera from multiple perspectives and a pre-trained detection model, and control a robot to grasp the target object according to a pre-established grasping model.

[0005] The embodiments of the present invention can be implemented as follows:

[0006] In a first aspect, embodiments of the present invention provide a robot grasping method applied to an electronic device, wherein the electronic device is communicatively connected to a robot, and the robot is equipped with a camera; the method includes:

[0007] Obtain the image set of the target object captured by the camera from multiple perspectives and the camera parameters corresponding to each perspective;

[0008] The image set and each camera parameter are input into the detection model, and the detection model is used to predict the image set to obtain the predicted value of the first coordinate of the target object, wherein the first coordinate is the three-dimensional coordinate of the key point of the target object;

[0009] Based on the predicted value of the first coordinate and the spatial geometric relationship between the key points, the pose parameters of the target object are calculated, wherein the pose parameters are used to characterize the spatial position and posture of the target object.

[0010] Based on the pose parameters and the grasping model, the robot is controlled to grasp the target object, wherein the grasping model is trained using reinforcement learning.

[0011] In one possible implementation, the detection model includes a two-dimensional detection network and a three-dimensional detection network, and the image set includes images corresponding to multiple viewpoints;

[0012] The step of inputting the image set and each camera parameter into a detection model, and using the detection model to predict the image set to obtain the predicted value of the first coordinate of the target object, includes:

[0013] The image set is input into the target detection network, and the target detection network is used to crop the images in the image set to obtain the cropped images.

[0014] The cropped image is input into the two-dimensional detection network, and the two-dimensional detection network is used to extract features from each cropped image to obtain a two-dimensional feature map and a confidence score for each image. The two-dimensional feature map is used to characterize the two-dimensional coordinates of the key points of the target object in the image, and the confidence score is used to characterize the realism of the target object under the viewpoint corresponding to each image.

[0015] All the two-dimensional feature maps, the confidence scores, and the camera parameters are input into the three-dimensional detection network. The confidence scores are used as weights to process all the two-dimensional feature maps using the three-dimensional detection network to obtain the predicted value of the first coordinates of the target object.

[0016] In one possible implementation, the 3D detection network includes a 3D mapping network, a 3D convolutional network, and a loss function regression model;

[0017] The step of inputting all the two-dimensional feature maps, the confidence scores, and the camera parameters into the three-dimensional detection network, and using the confidence scores as weights to process all the two-dimensional feature maps using the three-dimensional detection network to obtain the predicted value of the first coordinate of the target object includes:

[0018] All the two-dimensional feature maps, the confidence scores, and the camera parameters are input into the three-dimensional mapping network. The three-dimensional feature map of the target object is obtained by back projection. The three-dimensional feature map is used to characterize the three-dimensional spatial information of the target object. The three-dimensional feature map includes multiple channels, and each channel corresponds one-to-one with the key points of each target object.

[0019] The three-dimensional feature map is input into the three-dimensional convolutional network, and the three-dimensional convolutional network is used to extract features from the three-dimensional feature map to obtain the three-dimensional feature map corresponding to each key point.

[0020] The three-dimensional feature maps corresponding to all the key points are input into the loss function regression model. The loss function regression model is used to normalize the three-dimensional feature maps corresponding to all the key points to obtain the predicted value of the first coordinate of the target object.

[0021] In one possible implementation, the robot includes a gripper, and the step of controlling the robot to grasp the target object based on the pose parameters and the grasping model includes:

[0022] Obtain the state parameters at time t, including the pose parameters of the gripper end, the velocity of the gripper end, and the opening and closing state of the gripper;

[0023] The state parameters, the pose parameters of the target object, and the camera parameters corresponding to the maximum confidence are input into the grasping model to obtain the motion parameters at time t. The motion parameters include the velocity of the end of the gripper, the rotation angle of the gripper, and the opening and closing state of the gripper.

[0024] Control the robot to move according to the action parameters, complete the action at time t, and obtain the state parameters at time t+1;

[0025] The process of inputting the state parameters, the pose parameters of the target object, and the camera parameters corresponding to the maximum confidence level into the grasping model to obtain the motion parameters at time t is repeated until the robot completes the grasping of the target object.

[0026] In one possible implementation, the detection model is trained in the following manner:

[0027] Obtain training samples and corresponding labels for the training samples, wherein the training samples include a first image and camera parameters, and the labels represent the three-dimensional coordinates of key points of the reference object;

[0028] The first image is input into the target detection network, and the target detection network is used to crop the first image to obtain the cropped first image.

[0029] The cropped first image is input into the two-dimensional detection network, and the two-dimensional detection network is used to extract features from the cropped first image to obtain the two-dimensional feature map and the confidence level of the first image.

[0030] The two-dimensional feature map corresponding to the first image, the confidence score corresponding to the first image, and the camera parameters are input into the three-dimensional detection network. The confidence score corresponding to the first image is used as a weight to process all the two-dimensional feature maps corresponding to the first image using the three-dimensional detection network to obtain the prediction result of the second coordinate of the reference object. The second coordinate is the three-dimensional coordinate of the key point of the reference object.

[0031] The detection model is trained by backpropagation based on the predicted second coordinates of the reference object, the label, and the preset loss function to obtain the trained detection model.

[0032] In one possible implementation, the loss function is:

[0033]

[0034] Where α is the weight of the loss term for the two-dimensional coordinates of the key point. The loss function representing the two-dimensional feature map, The loss function represents the three-dimensional feature map;

[0035] The loss function for the two-dimensional feature map is:

[0036]

[0037] Among them, F n,k This represents the predicted value of the two-dimensional feature map. The label value represents the two-dimensional feature map;

[0038] The loss function for the three-dimensional feature map is:

[0039]

[0040] Where, d k The predicted values ​​of the three-dimensional coordinates of the key points. This represents the true value of the three-dimensional coordinates of the key point, where γ represents the weight. This is used to enhance the accuracy of the prediction results output by the 3D detection network.

[0041] In one possible implementation, the robot further includes a base, and the step of acquiring training samples and the labels corresponding to the training samples includes:

[0042] The camera is calibrated to obtain the first transformation matrix between the camera coordinate system and the gripper end coordinate system;

[0043] Obtain reference images of the reference object captured by the camera from multiple viewpoints, and the camera parameters corresponding to each viewpoint;

[0044] The coordinates of key points of the reference object in the reference image are labeled to obtain the first image;

[0045] Based on the first image and the camera parameters, the reference three-dimensional coordinates of the key points of the reference object in the camera coordinate system are calculated using the direct linear transformation method.

[0046] Based on the first transformation matrix and the preset transformation matrix, the reference three-dimensional coordinates are converted into real coordinates in the base coordinate system, wherein the preset transformation matrix represents the transformation relationship between the claw end coordinate system and the base coordinate system;

[0047] The labeled reference image and the real coordinates are respectively used as the training sample and the label corresponding to the training sample.

[0048] In one possible implementation, the crawling model is trained in the following manner:

[0049] Obtain the state parameters at time t, including the pose parameters of the gripper end, the velocity of the gripper end, and the opening and closing state of the gripper;

[0050] The state parameters, the pose parameters of the target object, and the camera parameters corresponding to the maximum confidence are input into the pre-constructed grasping model, which includes an actor network and a critic network.

[0051] Using the actor network of the aforementioned capture model, the motion parameters at time t are obtained;

[0052] Control the robot to move according to the action parameters, complete the action at time t, and obtain the reward parameters;

[0053] The reward parameters are input into the critic network, and the critic network is used to predict the Q value to obtain the predicted Q value, wherein the Q value is used to evaluate the value generated by the robot moving according to the action parameters;

[0054] The crawling model is trained by backpropagation based on the predicted Q value, a preset loss function, and a preset gradient function to obtain the trained crawling model.

[0055] In one possible implementation, the crawling model is constructed in the following way:

[0056] A state function is determined based on the pose parameters of the gripper tip, the velocity of the gripper tip, the opening and closing state of the gripper, the pose parameters of the target object, and the camera parameters corresponding to the maximum confidence level; wherein, the pose parameters of the gripper tip characterize the spatial position and attitude of the gripper tip relative to the base.

[0057] The motion function is determined based on the speed at the end of the gripper, the rotation angle of the gripper, and the opening and closing state of the gripper.

[0058] The reward function is determined based on the robot's grasping result, the distance between the gripper and the target object, and the robot's direction of movement.

[0059] Based on the state function, the action function, and the reward function, the grasping model is constructed using a multi-view reinforcement learning grasping algorithm.

[0060] Secondly, embodiments of the present invention also provide a robot grasping device applied to an electronic device, wherein the electronic device is communicatively connected to a robot, and the robot is equipped with a camera; the device includes:

[0061] The acquisition module is used to acquire the image set obtained by the camera from multiple perspectives of the target object and the camera parameters corresponding to each perspective;

[0062] The detection module is used to input the image set and each camera parameter into the detection model, and use the detection model to predict the image set to obtain the predicted value of the first coordinate of the target object, wherein the first coordinate is the three-dimensional coordinate of the key point of the target object;

[0063] The calculation module is used to calculate the pose parameters of the target object based on the predicted value of the first coordinate and the spatial geometric relationship between the key points, wherein the pose parameters are used to characterize the spatial position and attitude of the target object.

[0064] The control module is used to control the robot to grasp the target object according to the pose parameters and the grasping model, wherein the grasping model is trained using reinforcement learning.

[0065] Thirdly, embodiments of the present invention also provide an electronic device, the electronic device comprising:

[0066] One or more processors;

[0067] A memory for storing one or more programs that, when executed by one or more processors, enable the one or more processors to implement the robot grasping method described above.

[0068] Fourthly, embodiments of the present invention also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described robot grasping method.

[0069] Compared to existing technologies, the present invention provides a robot grasping method, apparatus, electronic device, and storage medium. First, a camera mounted on the robot is used to capture images of a target object from multiple perspectives, obtaining an image set and camera parameters corresponding to each perspective. Then, the image set and camera parameters are input into a pre-trained detection model to predict the three-dimensional coordinates of key points of the target object, obtaining prediction results. Based on the prediction results and the spatial geometric relationship between the key points, the pose of the target object is calculated. Finally, based on the pose of the target object and the pre-trained grasping model, the robot is controlled to complete the grasping of the target object. Attached Figure Description

[0070] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0071] Figure 1 This is an application scenario diagram of the robot grasping method provided in the embodiments of the present invention.

[0072] Figure 2 A block diagram of an electronic device provided in an embodiment of the present invention.

[0073] Figure 3 This is a flowchart illustrating the robot grasping method provided in an embodiment of the present invention.

[0074] Figure 4 An example diagram of the target object coordinate system provided in an embodiment of the present invention.

[0075] Figure 5 for Figure 3 The flowchart shown is a schematic diagram of step S120 in the robot grasping method.

[0076] Figure 6 This is a schematic diagram of the detection model provided in an embodiment of the present invention.

[0077] Figure 7 This is a schematic diagram of the structure of a two-dimensional detection network provided in an embodiment of the present invention.

[0078] Figure 8 This is a schematic diagram of the structure of a three-dimensional convolutional network provided in an embodiment of the present invention.

[0079] Figure 9 for Figure 3 The flowchart shown is a schematic diagram of step S140 in the robot grasping method.

[0080] Figure 10 This is a flowchart illustrating the training method for the detection model provided in an embodiment of the present invention.

[0081] Figure 11 for Figure 10 The flowchart shown is a schematic diagram of step S210 in the training method of the detection model.

[0082] Figure 12 This is a flowchart illustrating the training method for the crawling model provided in an embodiment of the present invention.

[0083] Figure 13 This is a block diagram of a robot grasping device provided in an embodiment of the present invention.

[0084] Icons: 10-Electronic device; 20-Network; 11-Memory; 12-Processor; 13-Bus; 200-Robot grasping device; 201-Acquisition module; 202-Detection module; 203-Computation module; 204-Control module. Detailed Implementation

[0085] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0086] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0087] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0088] Furthermore, the terms "first" and "second" are used only to distinguish descriptions and should not be interpreted as indicating or implying relative importance.

[0089] It should be noted that, where there is no conflict, the features in the embodiments of the present invention can be combined with each other.

[0090] With the development of intelligent manufacturing technology and information technology, the level of intelligence of robots is constantly improving, and they are being used more and more widely in various industries. In particular, in the industrial field, the use of industrial robots has greatly improved production efficiency.

[0091] In industrial production, robot grasping is one of the most common and basic operations. Researching ways to improve the accuracy and stability of robot grasping is of great significance for further improving industrial production efficiency.

[0092] In existing technologies, vision-based robot grasping processes mainly include two steps: target object posture recognition and robot motion planning and grasping.

[0093] Target object pose recognition:

[0094] Object pose recognition primarily uses visual recognition methods to estimate the position and orientation (6DoF, 6 Degree of Freedom, 6DoF pose) of an object relative to a coordinate system. Currently, the most widely researched technique is 6DoF object pose estimation, a method for determining the position and orientation of an object in a grasping scene. Position information is mainly represented by three-dimensional coordinates in a specific spatial coordinate system; pose information is commonly represented in four forms: pose transformation matrix, Euler angles, rotation quaternions, and axis angles.

[0095] 6DoF-based object recognition methods allow robots to grasp objects from different angles in three-dimensional space. However, these methods require the algorithm to provide information in six dimensions, including the object's position (three-dimensional spatial coordinates) and pose (Eulerian angles). This task cannot be accomplished using only two-dimensional image information.

[0096] To obtain the pose of an object, a depth camera is usually used to photograph the object, obtaining a color image of the object and a corresponding depth information image. Then, the point cloud information of the photographed object is reconstructed. The 6DoF pose information of the object is solved by matching the feature information of the point cloud with an existing 3D object model.

[0097] Robot motion planning and grasping:

[0098] Currently, common robot motion planning and grasping technologies are mainly based on the Dynamic Motion Primitives (DMP) method. Dynamic Motion Primitives are one of the more common forms of robot motion representation. They can be used as motion feedback controllers to flexibly adjust motion actions without manually adjusting motion parameters or worrying about the stability of the motion system.

[0099] For object pose recognition, 6DoF pose estimation technology is only suitable for grasping known objects. The grasping method needs to be pre-defined. When encountering unknown objects, even if the object's shape is similar to an existing model in the model library, the algorithm may fail, resulting in poor object pose detection accuracy. Furthermore, 3D keypoint detection technology primarily uses color and depth images as input. In complex grasping scenarios (such as overlapping or occlusion between objects), this may affect the accuracy of keypoint detection.

[0100] For robot motion and planning, robot motion planning and grasping technology based on dynamic motion primitives requires the establishment of a rigorous and accurate mathematical model of the robot and its surrounding environment. Although the robot's motion process is stable and controllable, the system's generalization ability is generally limited and it is usually difficult to apply to new environments.

[0101] To address the aforementioned issues, this embodiment provides a robot grasping method. The method determines the pose of the target object based on color images obtained by a camera from multiple perspectives, and controls the robot to grasp the target object based on a grasping model trained using reinforcement learning. This improves the accuracy of robot grasping and enhances its generalization ability.

[0102] The following is a detailed introduction.

[0103] Please refer to Figure 1 , Figure 1 The diagram illustrates an application scenario of the robot grasping method provided in this embodiment of the invention, including an electronic device 10, a robot, a camera, and a target object. Both the robot and the camera are connected to the electronic device 10 via a network 20; the robot includes a base, a robotic arm, and grippers, with the camera mounted at the end of the robotic arm.

[0104] A block diagram of electronic device 10 is shown below. Figure 2 As shown, the electronic device 10 can be, but is not limited to, a mobile phone, tablet computer, laptop computer, server, or other electronic device with processing capabilities. The electronic device 10 includes a memory 11, a processor 12, and a bus 13. The memory 11 and the processor 12 are connected via the bus 13.

[0105] The memory 11 is used to store programs, such as the robot gripping device 200. The robot gripping device 200 includes at least one software function module that can be stored in the memory 11 in the form of software or firmware. After receiving the execution instruction, the processor 12 executes the program to implement the robot gripping method in this embodiment.

[0106] The memory 11 may be, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc.

[0107] The processor 12 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the decoding control method in this embodiment can be completed by the integrated logic circuitry in the processor 12 or by software instructions.

[0108] The processor 12 mentioned above can be a general-purpose processor, including a central processing unit (CPU), a microcontroller unit (MCU), a complex programmable logic device (CPLD), a field programmable gate array (FPGA), an embedded ARM chip, etc.

[0109] The robot can be a grasping robot, including a base, a robotic arm, and grippers, and the robotic arm can move in all directions in space.

[0110] The camera can be a color camera with shooting function. The camera is fixed to the end of the robotic arm via a bracket. It can take pictures of the target object from multiple perspectives through the movement of the robotic arm and obtain color images from multiple perspectives.

[0111] Network 20 can be a wide area network (WAN), a local area network (LAN), or a combination of both, using a wireless link to achieve data transmission.

[0112] When the robot grasps the target object, it first takes pictures of the target object from multiple perspectives using a camera to obtain color images and camera poses corresponding to multiple perspectives. The camera pose represents the three-dimensional coordinates and rotation angle of the camera 20 relative to the base. The color images and camera poses are then sent to the electronic device 10. After receiving the color images and camera poses, the electronic device 10 controls the robot to complete the grasping of the target object based on the pre-stored detection model and grasping model.

[0113] exist Figure 2 Based on the electronic device 10 shown, the robot grasping method provided in this embodiment will be introduced. Please refer to... Figure 3 , Figure 3 A flowchart illustrating the robot grasping method provided in this embodiment is shown. This method is applied to an electronic device 10, which is communicatively connected to a robot equipped with a camera. The method includes the following steps:

[0114] S110: Acquire the image set obtained by the camera from multiple perspectives of the target object and the camera parameters corresponding to each perspective.

[0115] In this embodiment, an Intel RealSense D435 camera is used as a visual sensor to acquire images. It should be noted that this embodiment only uses the camera to acquire color images and does not use the depth information acquired by the camera.

[0116] Typically, four perspectives are selected to photograph the target object, and the resulting four color images are used as an image set. The selected perspectives can be, for example, a top-down view, a frontal view, a left-side view, and a right-side view, with each perspective corresponding to one color image.

[0117] Camera parameters are used to characterize the camera, referring to the camera's position and attitude information relative to the base. Position information is generally represented by three-dimensional coordinates, and attitude information is generally represented by a rotation matrix.

[0118] For example, the camera parameters corresponding to n viewpoints can be expressed as:

[0119] P n =[R n T n ]

[0120] Among them, P n Let R be a 3×4 matrix representing the camera parameters. n It is a three-dimensional matrix representing the camera's pose information, T n It is a three-dimensional column vector representing the three-dimensional coordinates of the camera relative to the base.

[0121] It should be noted that the camera parameters are actual measurements. Since the shooting angles are generally fixed at four angles, the camera parameters can be regarded as four known quantities stored in the electronic device.

[0122] S120, input the image set and each camera parameter into the detection model, use the detection model to predict the image set, and obtain the predicted value of the first coordinate of the target object, wherein the first coordinate is the three-dimensional coordinate of the key point of the target object.

[0123] In this embodiment, the detection model is used to predict the three-dimensional coordinates of key points of the target object. The input is the image under each viewpoint and the camera parameters corresponding to each viewpoint, and the output is the predicted value of the three-dimensional coordinates of the key points of the target object.

[0124] Keypoints are used to characterize the pose of the target object. For example, if the target object is a cube, the keypoints can be the eight vertices of the cube.

[0125] The predicted value of the first coordinate can be represented by a three-dimensional vector. For example, the predicted value of the three-dimensional coordinate of the k-th keypoint is:

[0126] [x k y k , z k ] T

[0127] Where, x k y k and z k These are the x-axis, y-axis, and z-axis coordinates of the k-th key point in the base coordinate system.

[0128] S130, based on the predicted value of the first coordinate and the spatial geometric relationship between the key points, the pose parameters of the target object are calculated, wherein the pose parameters are used to characterize the spatial position and attitude of the target object.

[0129] In this embodiment, the spatial geometric relationship between key points can be obtained by establishing an object coordinate system, as follows: Figure 4 As shown, taking a mug as the target object, the rim of the mug is taken as the origin of the coordinate system. The vector from the rim to the bottom of the mug is calculated and used as the z-axis. The normal vector of the plane formed by the z-axis and the key point of the mug handle is used as the y-axis. Then, the direction of the x-axis is determined according to the property of the orthogonality of the three axes.

[0130] S140 controls the robot to grasp the target object based on the pose parameters and the grasping model, wherein the grasping model is trained using reinforcement learning.

[0131] In this embodiment, since the grasping model is trained using reinforcement learning, after obtaining the pose parameters, it is necessary to determine the robot's next action based on the environmental state. Therefore, compared with the traditional method based on motion primitives, this method is more efficient.

[0132] The following is a detailed description of step S120. Figure 3 Based on this, please refer to Figure 5 Step S120 may include the following detailed steps:

[0133] S1201, Input the image set into the target detection network, and use the target detection network to crop the images in the image set to obtain the cropped image.

[0134] In this embodiment, to better understand the processing of the image set and camera parameters by the detection model, the following will combine... Figure 6 Please refer to the explanation. Figure 6 , Figure 6 The structure of the detection model and its processing of the image set and camera parameters are shown.

[0135] like Figure 6 As shown, the detection model includes an object detection network, a two-dimensional detection network, and a three-dimensional detection network, and the image set includes images corresponding to multiple viewpoints.

[0136] The object detection network, based on RFB-Net, is used to identify the location of the target object in each image and to crop the original image to improve the accuracy of key point detection.

[0137] Before inputting the image set into the 2D detection network, the images in the image set need to be preprocessed using the object detection network. The image set is then input into the object detection network, which crops each image to obtain cropped images. Cropped images can remove blank parts in the image to the maximum extent, so that the key points of the target object in the image can be displayed more clearly, which is beneficial to subsequent feature extraction.

[0138] RFB-Net introduces the concept of a Receptive Field Block (RFB), which enhances the ability of lightweight convolutional neural network models to learn deep features and builds a fast and accurate object detector on this basis.

[0139] Specifically, the receptive field module utilizes multi-branch pooling layers and convolutional kernels of varying receptive field sizes to extract features from the image using dilated convolutions. By embedding the receptive field module group into the shallow layers of the SSD object detection network, an advanced single-stage detector, RFB-Net, is constructed. RFB-Net achieves high detection accuracy while maintaining the detection speed of the original lightweight detector.

[0140] S1202, the image set is input into the two-dimensional detection network, and the two-dimensional detection network is used to extract features from each image to obtain the two-dimensional feature map and the confidence score corresponding to each image. The two-dimensional feature map is used to represent the two-dimensional coordinates of the key points of the target object in the image, and the confidence score is used to represent the realism of the target object under the viewpoint corresponding to each image.

[0141] In this embodiment, the two-dimensional coordinates of the key point in the image refer to the pixel coordinates in the image.

[0142] Please continue to refer to this. Figure 6 After the object detection network, the cropped image is input into the two-dimensional detection network. The two-dimensional detection network is used to extract features from the cropped image and outputs the two-dimensional feature map and the confidence score of each image.

[0143] The two-dimensional detection network is based on a convolutional neural network and includes a residual network, a first neural network, and a second neural network, as shown in the following structure. Figure 7 As shown.

[0144] The residual network is the backbone network. The cropped image is scaled to a uniform size and input into the residual network. The output is an initial two-dimensional image with two-dimensional coordinate labels for key points.

[0145] Compared to traditional convolutional and fully connected networks, residual networks effectively address the problem of information errors or loss during feature extraction in neural networks. By directly connecting the input residual module's data across layers to the module's output, they ensure the integrity of the information. The network only needs to learn the residual parts of the input and output, greatly reducing the difficulty of learning network parameters. Residual networks with more layers not only avoid the gradient vanishing problem but also significantly improve the accuracy of the detection model.

[0146] Following the residual network, two different neural networks are used to process the initial feature images. The first neural network is used to predict the confidence score ω of keypoints in each image. n,k The average confidence score of all keypoints in each image is taken as the confidence score for that image at the corresponding viewpoint. As shown in the following formula, the confidence score for the nth image, that is, the confidence score at the nth viewpoint, is expressed as:

[0147]

[0148] Where k is the number of keypoints included in the nth image.

[0149] The different confidence levels of different viewpoints can be used as prior knowledge to assist in designing more effective robot grasping operations. That is, the higher the confidence level of a certain viewpoint, the less occlusion the object is under, the higher the key point detection accuracy, and the more favorable the posture of the object is for robot grasping.

[0150] The second neural network consists of four deconvolutional layers. It processes the initial two-dimensional feature map to obtain a two-dimensional feature map, where each feature map includes multiple channels, which correspond to multiple key points in each image.

[0151] Compared to some existing networks that use upsampling and cross-layer connections, the second neural network in this embodiment of the invention improves the resolution of the feature map in a simpler way, such as deconvolution, thereby accelerating the prediction speed of the two-dimensional detection network while improving the detection accuracy.

[0152] S1203 inputs all two-dimensional feature maps, confidence scores, and camera parameters into the three-dimensional detection network. The confidence scores are used as weights to process all two-dimensional feature maps using the three-dimensional detection network to obtain the predicted value of the first coordinate of the target object.

[0153] The following is a detailed description of step S1203, which may include the following detailed steps:

[0154] The first step is to input all the two-dimensional feature maps, the confidence scores, and the camera parameters into a three-dimensional mapping network. Using back projection, a three-dimensional feature map of the target object is obtained. The three-dimensional feature map is used to characterize the three-dimensional spatial information of the target object. The three-dimensional feature map includes multiple channels, and each channel corresponds one-to-one with the key points of each target object.

[0155] In this embodiment, the 3D detection network includes a 3D mapping network, a 3D convolutional network, and a loss function regression model.

[0156] Please continue to refer to this. Figure 6 The two-dimensional feature map and confidence score output by the two-dimensional detection network, along with the camera parameters, are input into the three-dimensional mapping network to output the three-dimensional feature map of the target object. It should be noted that the camera parameters are input into the detection model along with the image set, but the previous target detection network and two-dimensional detection network did not use the camera parameters.

[0157] The specific process of processing the two-dimensional feature map using back projection is as follows;

[0158] Given a positive integer N x N y N z These represent the number of unit squares contained in the x, y, and z directions, respectively. A grid containing N squares is constructed centered on the object being detected. x ×N y ×N z A small three-dimensional voxel F 3D The unit side length of each small cell is denoted as L. size The coordinates of a point within a voxel can be represented by (i x i yi z (i) x i y i z ∈N x ×N y ×N z ×L size )express.

[0159] After backprojection, the output three-dimensional feature map has a size of N. x ×N y ×N z ×M, where M represents the number of channels in the 3D feature map, which is equal to the number of keypoints. The coordinates of the k-th keypoint of the target object in the 3D feature map can be represented as:

[0160]

[0161] in, Indicates at camera C n For a point (i) in space under the pose x i y i z The two-dimensional feature map obtained by projection operation, F n {.} k This indicates that a feature map with k channels is obtained by bilinear interpolation mapping of two-dimensional feature values, and the confidence level (conf) is obtained based on the first neural network. n They are then used as weights to fuse a three-dimensional feature map with k channels.

[0162] The second step is to input the 3D feature map into a 3D convolutional network and use the 3D convolutional network to extract features from the 3D feature map to obtain the 3D feature map corresponding to each key point.

[0163] In this embodiment, after back projection, it is necessary to process the voxel F that incorporates multiple viewpoint features. 3D Feature extraction is performed because F 3D This is a feature representation of three-dimensional spatial information. In this embodiment of the invention, a three-dimensional convolutional network is considered to be used for convolutional operations on it.

[0164] F 3D Since the feature map is a multi-channel 3D feature map, this invention constructs a multi-channel 3D convolutional network as a feature extractor. It uses a 3D convolutional structure similar to V2V-PoseNet, a neural network architecture for 3D gesture recognition. V2V-PoseNet takes a voxelized 3D feature map as input and outputs the spatial location of each keypoint and its corresponding probability estimate. Because the network directly processes the voxelized 3D feature map, the spatial information of the object is not distorted.

[0165] For F3D Based on the characteristics of this invention, the original V2V-PoseNet network structure has been improved to make it more suitable for the keypoint detection task in this embodiment. The network contains four types of modules: the first type is the basic 3D convolutional module, which consists of 3D convolutional layers, batch regularization layers, and ReLU activation functions. This module is located at the head and tail of the network; the second type is the 3D residual module, which is obtained by extending the 2D residual network in the depth dimension; the third type is the downsampling module, which mainly includes 3D convolutional layers and max pooling layers; and the last type is the upsampling module, which consists of 3D deconvolutional layers, batch regularization layers, and ReLU activation functions.

[0166] Introducing batch regularization layers and activation functions after the 3D deconvolution layers helps simplify the learning process of the network model. The basic 3D convolutional module has a kernel size of 7×7×7 and outputs feature maps with 64 channels; the 3D residual module has a kernel size of 3×3×3; the downsampling and upsampling modules have kernel sizes of 2×2×2 and a stride of 2.

[0167] The structure of the 3D convolutional network in this embodiment is similar to the hourglass model. The structure of the 3D convolutional network is as follows: Figure 8 As shown, the input 3D feature map first passes through a basic 3D convolution module and a downsampling module, then uses three 3D residual modules to extract effective local features of the object, and finally passes through an encoder and decoder composed of an upsampling module and a downsampling module.

[0168] In the encoder, the downsampling module reduces the size of the 3D feature map through convolution operations, but increases its feature channel number. The increase in the number of feature map channels corresponds to the increase in extracted features, which helps improve the detection performance of the network. In the decoder, the upsampling module expands the size of the 3D feature map while reducing the number of channels to compress the extracted features. The expansion of the 3D feature map size in the decoder helps the network determine the spatial information of key points.

[0169] The encoder and decoder network layers are interconnected by voxel addition of feature maps at the same scale, allowing the decoder to more stably upsample the feature maps. The input features, after passing through the encoder and decoder, output a 3D feature map of the k-th keypoint, denoted as V. k .

[0170] The third step is to input the 3D feature maps corresponding to all key points into the loss function regression model, and use the loss function regression model to normalize the 3D feature maps corresponding to all key points to obtain the predicted value of the first coordinate of the target object.

[0171] In this embodiment, the loss function regression model is softmax, which is used to normalize the detection probability of all object keypoints, denoted as V.k The specific formula is as follows.

[0172]

[0173] W, H, and D represent the width, height, and channel depth of the 3D feature map, respectively. K Let represent the voxel coordinates of the k-th keypoint, and exp(.) represent the exponential operation. Finally, the 3D coordinates of the k-th keypoint of the target object are:

[0174]

[0175] After obtaining the 3D coordinates of each key point, the pose parameters of the target object are obtained through step S130. Then, based on the pose parameters and the grasping model, the robot is controlled to complete the grasping of the target object. A detailed explanation follows.

[0176] exist Figure 3 Based on this, please refer to Figure 9 Step S140 may include the following detailed steps:

[0177] S1401, obtain the state parameters at time t. The state parameters include the pose parameters of the gripper end, the velocity of the gripper end, and the opening and closing state of the gripper.

[0178] In this embodiment, the state parameters at time t are used to characterize the robot's state at time t, and can be represented by a set:

[0179]

[0180] in, This represents the pose parameters of the gripper's end at time t. Let be the Cartesian coordinates of the gripper tip in the base coordinate system. Let be the rotation quaternion of the gripper end position, i.e. This indicates the speed at the end of the gripper; g = {0, 1} indicates the opening and closing state of the gripper, where 0 indicates the gripper is open and 1 indicates the gripper is closed. By default, the gripper is in the open state.

[0181] S1402, input the state parameters, the pose parameters of the target object and the camera parameters corresponding to the maximum confidence into the grasping model to obtain the motion parameters at time t. The motion parameters include the velocity of the gripper end, the rotation angle of the gripper and the opening and closing state of the gripper.

[0182] In this embodiment, the camera parameter corresponding to the maximum confidence level is the confidence level conf corresponding to all viewpoints. n The maximum value of (n∈{1,2,…,N}) conf max The corresponding camera pose is denoted as P. cmax .

[0183] Motion parameters are used to characterize the actions that the robot needs to perform, and can be represented by a set:

[0184]

[0185] in, θ represents the velocity of the robot gripper tip, which is the relative value between the gripper tip position at the next moment and the gripper tip position at the current moment. θ represents the rotation angle of the gripper tip, and g represents the opening and closing state of the gripper.

[0186] S1403, control the robot to move according to the action parameters, complete the action at time t, and obtain the state parameters at time t+1.

[0187] Repeat step S1402 until the robot completes the grasping of the target object.

[0188] The above describes the entire process of a robot grasping method. This method utilizes two models: a detection model and a grasping model. These are pre-trained, and only after training can the model perform actual predictions. In simpler terms, model training is the continuous updating of the model's parameters. During training, labeled training data is needed to better fit this data, allowing the trained model to make predictions on unlabeled or previously unseen data.

[0189] Based on this, the training methods for the detection model and the crawling model will be described in detail below.

[0190] Please refer to Figure 10 The training method for the detection model may include the following steps:

[0191] S210, Obtain training samples and corresponding labels for the training samples, wherein the training samples include the first image and camera parameters, and the labels represent the three-dimensional coordinates of key points of the reference object.

[0192] In this embodiment, training samples are used to train the detection model. After the training samples are input into the detection model, the model outputs prediction results. Labels are used to evaluate the accuracy of the prediction results. The closer the prediction results are to the labels, the more accurate the prediction results are, meaning the model training effect is better.

[0193] The first image is obtained by the camera taking pictures of the reference object from multiple perspectives. There are multiple camera parameters, and each of the multiple camera parameters corresponds to a different perspective.

[0194] It should be noted that there can be multiple reference objects, resulting in multiple sets of first images. Each set of first images includes first images corresponding to multiple viewpoints. Correspondingly, there are also multiple labels, with each label corresponding to each reference object.

[0195] S220, the first image is input into the object detection network, and the object detection network is used to crop the first image to obtain the cropped first image.

[0196] In this embodiment, corresponding to step S1201, the cropping process of the target detection network on the first image is similar to the cropping process on the image set, and will not be described again here.

[0197] To shorten the training time of the detection model, this embodiment fine-tunes the object detection network using training samples and their corresponding labels, based on the existing network training weights. The training samples and their corresponding labels are divided into training and testing sets in an 8:2 ratio. The network model is trained on an Ubuntu 16.04 system. The first images used in both training and testing are uniformly scaled to a uniform resolution after image enhancement preprocessing based on histogram equalization.

[0198] Meanwhile, this embodiment employs data augmentation techniques such as random pruning and random augmentation to enhance the robustness of the detection model. During model training, the batch size is set to 32 samples per training session, with a total of 300 training epochs. The initial learning rate is 0.004, which is reduced to one-tenth of its original value at the 120th, 180th, and 280th training epochs. The weight decay is set to 0.0001, and the momentum factor is 0.9. To prevent gradient explosion, this embodiment uses a model "warm-up" method, where the learning rate is gradually increased from 0.0001 to 0.004 over the first 10 training epochs.

[0199] S230, the first image is input into the two-dimensional detection network, and the two-dimensional detection network is used to extract features from the first image to obtain the two-dimensional feature map and the confidence level of the first image.

[0200] In this embodiment, the process of using a two-dimensional detection network to extract features from the first image is similar to S1202, and will not be described again here.

[0201] S240, the two-dimensional feature map corresponding to the first image, the confidence level corresponding to the first image, and the camera parameters are input into the three-dimensional detection network. The confidence level corresponding to the first image is used as a weight to process all the two-dimensional feature maps corresponding to the first image using the three-dimensional detection network to obtain the prediction result of the second coordinate of the reference object. The second coordinate is the three-dimensional coordinate of the key point of the reference object.

[0202] In this embodiment, the process of using a three-dimensional detection network to process the two-dimensional feature maps corresponding to all the first images is similar to step S1203, and will not be described again here.

[0203] During the training of the detection model, in the back projection stage, this embodiment fills the two-dimensional feature maps from different viewpoints into a grid of 64×64×64 units (each unit grid has a side length L) according to the camera parameters of each viewpoint. size =0.375cm, i.e., 24cm×24cm×24cm in a three-dimensional voxel. The entire detection model was trained using the Adam optimizer, with an initial learning rate set to 0.001 and a decay strategy of 0.3 every 20 training epochs. The model was trained for 200 epochs, with each batch of training data containing 8 samples.

[0204] For 3D convolutional networks in 3D detection networks, 3D data augmentation techniques are used to improve the robustness of 3D convolutional networks. Specific implementation methods include the following two categories:

[0205] The first type is random volume embedding: similar to random cropping methods for 2D images, it randomly embeds the 3D feature map into the detection space S (S∈N). x ×N y ×N z )middle.

[0206] The second type is random rotation: the algorithm randomly rotates the 3D feature map along the vertical axis by 0°, 90°, 180° or 270°. When using random rotation, the detection model only updates the parameters of the 3D convolutional module during backpropagation.

[0207] S250, the detection model is backpropagated and trained based on the prediction result of the second coordinate of the reference object, the label and the preset loss function to obtain the trained detection model.

[0208] In this embodiment, backpropagation training can be understood as a method to update the parameters of the detection model. After multiple backpropagation training sessions, the training of the model is completed when the error between the prediction result of the detection model and the label is less than a preset value.

[0209] It should be noted that although the detection model includes multiple sub-networks, the entire detection model is trained end-to-end, that is, the input is the training sample and the corresponding label of the training sample, and the output is the prediction result of the second coordinate.

[0210] Therefore, the loss function of the detection model can be expressed as:

[0211]

[0212] Where α is the weight of the loss term for the two-dimensional coordinates of the keypoint. The loss function representing the two-dimensional feature map. The loss function represents the 3D feature map.

[0213] The loss function for the two-dimensional feature map is:

[0214]

[0215] Among them, F n,k This represents the predicted value of the two-dimensional feature map. The label value represents the two-dimensional feature map.

[0216] The loss function for the 3D feature map is:

[0217]

[0218] Where, d k The predicted values ​​representing the three-dimensional coordinates of key points. The true values ​​of the 3D coordinates of the key points are represented by γ, which represents the weight. Used to enhance the accuracy of prediction results output by 3D detection networks.

[0219] In this embodiment, the loss function This ensures that the 2D detection network can extract useful keypoint pixel features from the first image at each viewpoint and accurately predict the 2D coordinates of the keypoints. Loss function. In This represents the loss for each keypoint, used to predict the 3D spatial information of the object's keypoints; the weight γ can be set to 0.01.

[0220] The following is a detailed description of step S210. Figure 10 Based on this, please refer to Figure 11 Step S210 may include the following detailed steps:

[0221] S2101, calibrate the camera to obtain the first transformation matrix between the camera coordinate system and the gripper end coordinate system.

[0222] In this embodiment, since the camera is fixed to the end of the robotic arm, an eye-in-hand calibration method is considered. In this case, the positional relationship between the camera coordinate system and the gripper end coordinate system is fixed, while the positional relationship with the base coordinate system changes constantly. Therefore, it is necessary to solve for the transformation matrix from the camera coordinate system (camera) to the gripper end coordinate system (end). Based on the pose relationships of each coordinate system, the following transformation equation can be written:

[0223]

[0224] Where, p base This represents the pose matrix of a point on a calibration plate fixed at a certain position, relative to the coordinate system of the base. The transformation matrix between the gripper end coordinate system and the base coordinate system is a known quantity, p. end This represents the pose matrix from a point on a chessboard grid fixed at a certain position to the coordinate system of the gripper's end. p represents the transformation matrix between the gripper end coordinate system and the base coordinate system. camera This represents the pose matrix of a point in a chessboard grid fixed at a certain position, relative to the camera coordinate system.

[0225] Keeping the relative positions of the base and calibration plate constant, the transformation matrix between the camera and the gripper end coordinate system is solved through multiple movements of the robotic arm from different perspectives. Testing and verification show that a reprojection error of approximately 0.2–0.3 mm after calibration meets the requirements for robot object grasping.

[0226] S2102, acquire reference images of the reference object taken by the camera from multiple perspectives and the camera parameters corresponding to each perspective.

[0227] In this embodiment, as mentioned above, four viewpoints are generally selected to photograph the reference object, resulting in reference images and camera parameters corresponding to the four viewpoints.

[0228] S2103, the coordinates of the key points of the reference object in the reference image are labeled to obtain the first image.

[0229] In this embodiment, after acquiring reference images from multiple perspectives and the corresponding camera parameters, it is first necessary to manually annotate the two-dimensional pixel coordinates of the key points of the reference object in each reference image.

[0230] This embodiment uses Labelme tool for key point annotation. Labelme is a powerful 2D image annotation software that can annotate not only key points, but also object category labels, object detection bounding boxes, and instance segmentation masks. Simply click and drag on the image to be annotated with the mouse, and the software can automatically generate annotation information files in the corresponding format, which facilitates the reading of ground truth (GT) during neural network model training.

[0231] S2104. Based on the first image and camera parameters, the reference three-dimensional coordinates of the key points of the reference object in the camera coordinate system are calculated using the direct linear transformation method.

[0232] In this embodiment, since the two-dimensional pixel coordinates of key points of the reference object and the corresponding camera parameters are known from n (n∈{1, 2…N}) viewpoints:

[0233] P n =[R n T n ]

[0234] As mentioned above, P n It is a 3×4 matrix.

[0235] To obtain the true 3D coordinates of key points on the reference object, this embodiment uses the Direct Linear Transformation (DLT) method to transform the 2D coordinates of key points from n viewpoints: t = [u n v n ,1] T Restored to three-dimensional coordinates: W = [x, y, z, 1] T .

[0236] Based on the projection relationship, n viewpoints can be represented as:

[0237]

[0238] Where, λ n The depth value of this 3D coordinate point is an unknown quantity; [R] n,1 |T n,1 ]、[R n,2 |T n,2 ]、[R n,3 |T n,3 [] represents rows 1, 2, and 3 of the camera parameter matrix, respectively. After elimination and matrix transformation, the reference 3D coordinates of the key points of the reference object in the camera coordinate system are finally obtained through matrix singular value decomposition (SVD).

[0239] S2105, based on the first transformation matrix and the preset transformation matrix, the reference three-dimensional coordinates are converted into the real coordinates in the base coordinate system, wherein the preset transformation matrix represents the transformation relationship between the claw end coordinate system and the base coordinate system.

[0240] S2106, the labeled reference image and the real coordinates are used as training samples and the corresponding labels of the training samples, respectively.

[0241] Please refer to Figure 12 The training method for the grasping model may include the following steps:

[0242] S310, obtain the state parameters at time t. The state parameters include the pose parameters of the gripper end, the velocity of the gripper end, and the opening and closing state of the gripper.

[0243] S320, input the state parameters, the pose parameters of the target object, and the camera parameters corresponding to the maximum confidence into the pre-built grasping model, which includes an actor network and a critic network.

[0244] In this embodiment, the capture model includes an actor network and two critic networks, as well as their corresponding target networks. The target networks are used for stable training, and a total of 6 neural networks are included.

[0245] S330 uses the actor network of the capture model to obtain the motion parameters at time t.

[0246] S340, control the robot to move according to the action parameters, complete the action at time t, and obtain the reward parameters.

[0247] In this embodiment, when the robot observes the current state S... t When that happens, input it into the actor network and output the corresponding decision action A. t This process completes one motion cycle and obtains the reward parameter R at the current moment. t With the state S at the next moment t+1 The algorithm will set (S) t A t R t S t+1 This is stored as an experience in the experience pool, and during the training of the neural network, a batch of experiences is randomly selected to update the parameters of the grasping model.

[0248] Reward parameter R t It can be represented as R(S) t A t ), indicating that the robot is in its current state S. t Select action A below t The reward value generated afterward.

[0249] S350: The reward parameters are input into the critic network, and the critic network is used to predict the Q value to obtain the predicted Q value. The Q value is used to evaluate the value generated by the robot moving according to the action parameters.

[0250] In this embodiment, the Q value is denoted as Q(S) t A t ), refers to the current state S. t Select action A below t Then, the expected value of the sum of reward parameters is obtained until the end of the task (final state), which is used to evaluate the value of the action.

[0251] S360, the crawling model is backpropagated and trained based on the predicted Q value, the preset loss function, and the preset gradient function to obtain the trained crawling model.

[0252] In this embodiment, the two critic networks and their corresponding target networks are denoted as δ. i and δ′ i For each i ∈ {1,2}, the two critic networks will predict different Q-values ​​due to the different initial values ​​of their network parameters. As shown in the following equation, the algorithm selects the smaller Q-value as the target for the critic network update to prevent bias caused by overestimating the Q-value:

[0253]

[0254] In this case, only one critic target network is updated at a time. The parameter y t The target for updating the two networks.

[0255] Furthermore, the grasping model may introduce small errors with each update. These errors accumulate over multiple updates, leading to poor algorithm performance. Therefore, in addition to using a policy delay update technique similar to that in the DDPG algorithm, the algorithm also performs numerical smoothing on a portion of the action space to reduce errors. This involves adding a certain amount of noise to the action output by the actor network. Thus, the original equation becomes:

[0256]

[0257] Here, noise ζ can be regarded as a regularization term, which makes the update of the value function smoother. The noise used here is different from the noise used in the DDPG algorithm: DDPG only adds noise to the final output of the action of interacting with the environment, and its purpose is to improve the algorithm's ability to explore the action space.

[0258] The existing TD3 algorithm processes the action A output by the actor network before calculating the Q-value of the target critic network. t Noise was added to make the Q-value predictions more accurate and robust. A network of commentators was used to update the parameters. The target network y is approximated using the following loss function. t Value:

[0259]

[0260] Let η be the actor network and η' be its corresponding target network. To update the parameters of the actor network η, the gradient of the policy function is minimized using the following formula to obtain the action with the maximum Q value:

[0261]

[0262] Where, N m This represents the number of experiences randomly drawn from the experience pool.

[0263] Furthermore, due to the high complexity of the robot's mechanical structure and the large grasping space, the search space for the robot's actions during interaction with the environment is highly dimensional, resulting in sparse reward feedback for grasping tasks. To obtain the optimal grasping action strategy, the algorithm requires a significant amount of training time.

[0264] To address this, this embodiment also introduces a Hindsight Experience Replay (HER) mechanism, which allows the robot to quickly learn useful grasping planning strategies even in the early training stages when the grasping success rate is low, thereby accelerating the convergence speed of the algorithm.

[0265] The post-event experience replay mechanism uses the "hindsight is 20 / 20" approach to guide the agent's strategy learning. Even if the agent doesn't receive a positive reward in the current round of interaction, it can still accumulate experience based on the current outcome. Even if the exploration fails, the "failure" is recorded, and the previously expected result is replaced with the achieved result, creating a special experience pool. When an interaction encounters a situation consistent with the goal of the stored experience, this experience is transformed into a "successful" experience, helping the agent learn effectively.

[0266] To enable robots to "learn from failure" in grasping tasks, the concept of a goal (G) needs to be introduced, which represents the state S. t The desired state G can be achieved through a series of decision-making processes. Simultaneously, to ensure that the state S, where the capture result is unsuccessful, can be retrieved... t The corresponding experience is used for experience replay training, state S t It should also be regarded as a destination state G.

[0267] Therefore, correspondingly, in the original set of experience (S) t A t R t S t+1 Based on this, a target state G is introduced. Through a simple join operation, the transformation from the original empirical data to the new empirical representation, i.e., S, is completed. t ||G, A t R t S t+1 ||G, where || denotes a join operation. For each iteration of exploration, k new targets G′ are obtained by sampling future states.k and the corresponding k sets of experience values

[0268] When using the HER mechanism to sample multiple target states, the reward function value changes unrestrainedly, leading to variations in the distribution of learning experience in the neural network model. This bias may increase the instability of model training to some extent. To address this, this invention uses the HER mechanism in the early training phase. After a certain threshold of training rounds, the target state G is replaced with the actual grasping end state, and the actual reward is used to update the neural network parameters.

[0269] The construction of the above crawling model may include the following detailed steps:

[0270] The first step is to determine the state function based on the pose parameters of the gripper tip, the velocity of the gripper tip, the opening and closing state of the gripper, the pose parameters of the target object, and the camera parameters corresponding to the maximum confidence level; among them, the pose parameters of the gripper tip represent the spatial position and attitude of the gripper tip relative to the base.

[0271] In this embodiment, the state function can be expressed as:

[0272]

[0273] in, These are the pose parameters of the target object. The other parameters have been explained previously and will not be repeated here.

[0274] The second step is to determine the motion function based on the speed at the end of the gripper, the rotation angle of the gripper, and the opening and closing state of the gripper.

[0275] In this embodiment, the action function can be expressed as:

[0276]

[0277] The third step is to determine the reward function based on the robot's grasping results, the distance between the gripper and the target object, and the robot's direction of movement.

[0278] In this embodiment, the reward function can be expressed as:

[0279]

[0280] in, It is a binary reward function. 0 indicates that the robot failed to grasp the object, and 1 indicates that the object was successfully grasped; α represents the negative of the distance d between the gripper and the target object, and α and β are the weights of the corresponding reward items; The reward for the robot moving along the line connecting the camera position with the highest view confidence level and the target object position is calculated as follows:

[0281]

[0282] Where dist represents the distance of the current gripper tip position relative to the line connecting the optimal viewpoint and the target object. This indicates a pre-set distance threshold, which is set to 80mm in this embodiment.

[0283] The fourth step is to construct a grasping model based on the state function, action function, and reward function using a multi-view reinforcement learning grasping algorithm.

[0284] The crawling strategies in existing technologies are typically as follows:

[0285] After acquiring the pose information of the gripper end and the object, the robot performs inverse kinematics calculations to obtain the motion angles of each joint. Then, each joint motor controls the robot to move near the target object based on the trend of its angle changes. The robot selects a suitable gripping point based on the object's shape characteristics to complete the gripping operation.

[0286] However, these methods do not consider the possibility of obstacles in the robot's movement space. For example, when the target object is occluded, the robot may collide with the obstruction during the grasping process, posing a safety hazard. To address this, these methods require the introduction of additional collision detection modules to prevent such occurrences, which increases the system's setup cost.

[0287] Compared to existing technologies, this invention uses reinforcement learning to determine the state and action space of the robot grasping problem. Then, based on the detection confidence of different perspectives obtained from the two-dimensional detection network, it is introduced into the reward function, enabling the robot to learn the occlusion information around the target object and have better obstacle avoidance capabilities.

[0288] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0289] First, the robot grasping method provided in this embodiment uses multi-view feature fusion to predict the three-dimensional coordinates of key points of the target object. Compared with the existing technology that uses color images and depth images as input to predict the 6DoF pose of the object, it can effectively solve the problem of low detection accuracy caused by overlapping and occlusion of objects in complex scenes.

[0290] Secondly, using sparse keypoints can provide a simplified description of the target object's surface appearance, enabling the detection model to have better generalization performance.

[0291] Finally, a post-training experience replay mechanism is introduced during the training of the grasping model. Even in the early training stage when the robot has a low grasping success rate, it can quickly learn useful grasping planning strategies, thereby accelerating the convergence speed of the algorithm.

[0292] In order to perform the corresponding steps in the above robot grasping method embodiments, an implementation method applied to a robot grasping device is given below.

[0293] Please refer to Figure 13 , Figure 13 A block diagram of the robot grasping device 200 provided in this embodiment is shown. It is applied to an electronic device 10, which is communicatively connected to a robot equipped with a camera. The robot grasping device 200 includes: an acquisition module 201, a detection module 202, a calculation module 203, and a control module 204.

[0294] The acquisition module 201 is used to acquire the image set obtained by the camera from multiple perspectives of the target object and the camera parameters corresponding to each perspective.

[0295] The detection module 202 is used to input the image set and each camera parameter into the detection model, and use the detection model to predict the image set to obtain the predicted value of the first coordinate of the target object, wherein the first coordinate is the three-dimensional coordinate of the key point of the target object.

[0296] The calculation module 203 is used to calculate the pose parameters of the target object based on the predicted value of the first coordinate and the spatial geometric relationship between the key points. The pose parameters are used to characterize the spatial position and attitude of the target object.

[0297] The control module 204 is used to control the robot to grasp the target object based on the pose parameters and the grasping model. The grasping model is trained using reinforcement learning.

[0298] Optionally, the detection model includes a two-dimensional detection network and a three-dimensional detection network, and the image set includes images corresponding to multiple viewpoints;

[0299] Detection module 202 is used for:

[0300] The image set is input into the target detection network, and the target detection network is used to crop the images in the image set to obtain the cropped images;

[0301] The image set is input into the two-dimensional detection network, and the two-dimensional detection network is used to extract features from each image to obtain a two-dimensional feature map and a confidence score for each image. The two-dimensional feature map is used to represent the two-dimensional coordinates of the key points of the target object in the image, and the confidence score is used to represent the realism of the target object under the viewpoint corresponding to each image.

[0302] All two-dimensional feature maps, the confidence scores, and camera parameters are input into a three-dimensional detection network. The confidence scores are used as weights to process all two-dimensional feature maps using the three-dimensional detection network to obtain the predicted value of the first coordinate of the target object.

[0303] Optionally, the 3D detection network includes a 3D mapping network, a 3D convolutional network, and a loss function regression model;

[0304] Detection module 202 is used for:

[0305] All two-dimensional feature maps, confidence scores, and camera parameters are input into a three-dimensional mapping network. Using back projection, a three-dimensional feature map of the target object is obtained. The three-dimensional feature map is used to characterize the three-dimensional spatial information of the target object. The three-dimensional feature map includes multiple channels, and each channel corresponds one-to-one with each key point of the target object.

[0306] The 3D feature map is input into a 3D convolutional network, and the 3D convolutional network is used to extract features from the 3D feature map to obtain the 3D feature map corresponding to each key point.

[0307] The 3D feature maps corresponding to all key points are input into the loss function regression model. The loss function regression model is used to normalize the 3D feature maps corresponding to all key points to obtain the predicted value of the first coordinate of the target object.

[0308] Optional, the robot includes grippers;

[0309] Control module 204 is used for:

[0310] Obtain the state parameters at time t, including the pose parameters of the gripper end, the velocity of the gripper end, and the opening and closing state of the gripper.

[0311] Input the state parameters, the pose parameters of the target object, and the camera parameters corresponding to the maximum confidence into the grasping model to obtain the motion parameters at time t. The motion parameters include the velocity of the gripper end, the rotation angle of the gripper, and the opening and closing state of the gripper.

[0312] The robot is controlled to move according to the action parameters, complete the action at time t, and obtain the state parameters at time t+1;

[0313] The process of repeatedly inputting the state parameters, the pose parameters of the target object, and the camera parameters corresponding to the maximum confidence into the grasping model to obtain the motion parameters at time t continues until the robot completes the grasping of the target object.

[0314] The detection module 202 is also used for:

[0315] Obtain training samples and their corresponding labels. The training samples include the first image and camera parameters, and the labels represent the three-dimensional coordinates of key points of the reference object.

[0316] The first image is input into the object detection network, and the object detection network is used to crop the first image to obtain the cropped first image.

[0317] The first image is input into a two-dimensional detection network, and the two-dimensional detection network is used to extract features from the first image to obtain the two-dimensional feature map and the confidence level of the first image.

[0318] The two-dimensional feature map corresponding to the first image, the confidence score corresponding to the first image, and the camera parameters are input into the three-dimensional detection network. The confidence score corresponding to the first image is used as the weight to process all the two-dimensional feature maps corresponding to the first image using the three-dimensional detection network to obtain the prediction result of the second coordinate of the reference object. The second coordinate is the three-dimensional coordinate of the key point of the reference object.

[0319] The detection model is trained by backpropagation based on the predicted second coordinates of the reference object, the label, and the preset loss function, resulting in the trained detection model.

[0320] Detection module 202 is used for:

[0321] The camera is calibrated to obtain the first transformation matrix between the camera coordinate system and the gripper end coordinate system;

[0322] Acquire reference images of a reference object taken by a camera from multiple perspectives, along with the camera parameters for each perspective.

[0323] The coordinates of key points of the reference object in the reference image are labeled to obtain the first image;

[0324] Based on the first image and camera parameters, the reference three-dimensional coordinates of the key points of the reference object in the camera coordinate system are calculated using the direct linear transformation method.

[0325] Based on the first transformation matrix and the preset transformation matrix, the reference three-dimensional coordinates are converted into the true coordinates in the base coordinate system. The preset transformation matrix represents the transformation relationship between the claw end coordinate system and the base coordinate system.

[0326] The labeled reference image and the real coordinates are used as training samples and their corresponding labels, respectively.

[0327] Control module 204 is also used for:

[0328] Obtain the state parameters at time t, including the pose parameters of the gripper end, the velocity of the gripper end, and the opening and closing state of the gripper.

[0329] The state parameters, the pose parameters of the target object, and the camera parameters corresponding to the maximum confidence are input into the pre-built grasping model, which includes an actor network and a critic network.

[0330] The motion parameters at time t are obtained by using the actor network of the capture model;

[0331] Control the robot to move according to the action parameters, complete the action at time t, and obtain the reward parameters;

[0332] The reward parameters are input into the critic network, which is then used to predict the Q-value. The predicted Q-value is used to evaluate the value generated by the robot moving according to the action parameters.

[0333] The crawling model is trained by backpropagation based on the predicted Q-value, the preset loss function, and the preset gradient function to obtain the trained crawling model.

[0334] Control module 204 is used for:

[0335] The state function is determined based on the pose parameters of the gripper tip, the velocity of the gripper tip, the opening and closing state of the gripper, the pose parameters of the target object, and the camera parameters corresponding to the maximum confidence level; among them, the pose parameters of the gripper tip represent the spatial position and attitude of the gripper tip relative to the base.

[0336] The motion function is determined based on the speed at the end of the gripper, the rotation angle of the gripper, and the opening and closing state of the gripper.

[0337] The reward function is determined based on the robot's grasping result, the distance between the gripper and the target object, and the robot's direction of movement.

[0338] Based on the state function, action function, and reward function, the grasping model is constructed using a multi-view reinforcement learning grasping algorithm.

[0339] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the robot gripping device 200 described above is as follows: Refer to the corresponding processes in the foregoing method embodiments; further details will not be repeated here.

[0340] This embodiment also provides a computer-readable storage medium storing a computer program thereon, which, when executed by the processor 12, implements the robot grasping method disclosed in the above embodiment.

[0341] In summary, the robot grasping method, apparatus, electronic device, and storage medium provided by the embodiments of the present invention firstly capture images of a target object from multiple perspectives using a camera mounted on the robot, obtaining an image set and camera parameters corresponding to each perspective; then, the image set and camera parameters are input into a pre-trained detection model to predict the three-dimensional coordinates of key points of the target object, obtaining prediction results, and calculating the pose of the target object based on the prediction results and the spatial geometric relationship between the key points; finally, based on the pose of the target object and the pre-trained grasping model, the robot is controlled to complete the grasping of the target object.

[0342] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A robot grasping method, characterized in that, The method is applied to an electronic device that is communicatively connected to a robot, the robot being equipped with a camera and including a base and grippers; the method includes: The system acquires an image set of the target object captured by the camera from multiple perspectives and the camera parameters corresponding to each perspective. The camera parameters are used to characterize the position and attitude information of the camera relative to the base. The position information is represented by three-dimensional coordinates, and the attitude information is represented by a rotation matrix. The image set and each camera parameter are input into the detection model, and the detection model is used to predict the image set to obtain the predicted value of the first coordinate of the target object, wherein the first coordinate is the three-dimensional coordinate of the key point of the target object; Based on the predicted value of the first coordinate and the spatial geometric relationship between the key points, the pose parameters of the target object are calculated, wherein the pose parameters are used to characterize the spatial position and posture of the target object. Based on the pose parameters and the grasping model, the robot is controlled to grasp the target object. The grasping model is trained using reinforcement learning. Controlling the robot to grasp the target object based on the pose parameters and the grasping model includes: Obtain the state parameters at time t, including the pose parameters of the gripper end, the velocity of the gripper end, and the opening and closing state of the gripper; The state parameters, the pose parameters of the target object, and the camera parameters corresponding to the maximum confidence are input into the grasping model to obtain the motion parameters at time t. The motion parameters include the velocity of the end of the gripper, the rotation angle of the gripper, and the opening and closing state of the gripper. Control the robot to move according to the action parameters, complete the action at time t, and obtain the state parameters at time t+1; The process of repeatedly inputting the state parameters, the pose parameters of the target object, and the camera parameters corresponding to the maximum confidence into the grasping model to obtain the motion parameters at time t is repeated until the robot completes the grasping of the target object. Specifically, based on the robot's grasping result, the distance between the gripper and the target object, and the robot's direction of motion, the reward function of the grasping model is determined, and the reward function is expressed as follows: in, It is a binary reward function. 0 indicates that the robot failed to grasp the object, and 1 indicates that the object was successfully grasped; Indicates the distance between the gripper and the target object. The opposite number, and The weight of the corresponding reward item; The reward for the robot moving along the line connecting the camera position with the highest view confidence level and the target object position is calculated as follows: in, This indicates the distance of the current gripper tip position relative to the line connecting the optimal viewpoint and the target object. This indicates a pre-set distance threshold.

2. The method as described in claim 1, characterized in that, The detection model includes a target detection network, a two-dimensional detection network, and a three-dimensional detection network, and the image set includes images corresponding to multiple viewpoints; The step of inputting the image set and each camera parameter into a detection model, and using the detection model to predict the image set to obtain the predicted value of the first coordinate of the target object, includes: The image set is input into the target detection network, and the target detection network is used to crop the images in the image set to obtain the cropped images. The cropped image is input into the two-dimensional detection network, and the two-dimensional detection network is used to extract features from each cropped image to obtain a two-dimensional feature map and a confidence score for each image. The two-dimensional feature map is used to characterize the two-dimensional coordinates of the key points of the target object in the image, and the confidence score is used to characterize the realism of the target object under the viewpoint corresponding to each image. All the two-dimensional feature maps, the confidence scores, and the camera parameters are input into the three-dimensional detection network. The confidence scores are used as weights to process all the two-dimensional feature maps using the three-dimensional detection network to obtain the predicted value of the first coordinates of the target object.

3. The method as described in claim 2, characterized in that, The 3D detection network includes a 3D mapping network, a 3D convolutional network, and a loss function regression model; The step of inputting all the two-dimensional feature maps, the confidence scores, and the camera parameters into the three-dimensional detection network, and using the confidence scores as weights to process all the two-dimensional feature maps using the three-dimensional detection network to obtain the predicted value of the first coordinate of the target object includes: All the two-dimensional feature maps, the confidence scores, and the camera parameters are input into the three-dimensional mapping network. The three-dimensional feature map of the target object is obtained by back projection. The three-dimensional feature map is used to characterize the three-dimensional spatial information of the target object. The three-dimensional feature map includes multiple channels, and each channel corresponds one-to-one with the key points of each target object. The three-dimensional feature map is input into the three-dimensional convolutional network, and the three-dimensional convolutional network is used to extract features from the three-dimensional feature map to obtain the three-dimensional feature map corresponding to each key point. The three-dimensional feature maps corresponding to all the key points are input into the loss function regression model. The loss function regression model is used to normalize the three-dimensional feature maps corresponding to all the key points to obtain the predicted value of the first coordinate of the target object.

4. The method according to claim 2, characterized in that, The detection model was trained in the following way: Obtain training samples and corresponding labels for the training samples, wherein the training samples include a first image and camera parameters, and the labels represent the three-dimensional coordinates of key points of a reference object; The first image is input into the target detection network, and the target detection network is used to crop the first image to obtain the cropped first image. The cropped first image is input into the two-dimensional detection network, and the two-dimensional detection network is used to extract features from the cropped first image to obtain the two-dimensional feature map and the confidence level of the first image. The two-dimensional feature map corresponding to the first image, the confidence score corresponding to the first image, and the camera parameters are input into the three-dimensional detection network. The confidence score corresponding to the first image is used as a weight to process all the two-dimensional feature maps corresponding to the first image using the three-dimensional detection network to obtain the prediction result of the second coordinate of the reference object. The second coordinate is the three-dimensional coordinate of the key point of the reference object. The detection model is trained by backpropagation based on the predicted second coordinates of the reference object, the label, and the preset loss function to obtain the trained detection model.

5. The method as described in claim 4, characterized in that, The loss function is: in, The weights of the loss term for the two-dimensional coordinates of the key points. The loss function representing the two-dimensional feature map, The loss function represents the three-dimensional feature map; The loss function for the two-dimensional feature map is: in, This represents the predicted value of the two-dimensional feature map. The label value represents the two-dimensional feature map; The loss function for the three-dimensional feature map is: in, The predicted values ​​of the three-dimensional coordinates of the key points. This represents the true value of the three-dimensional coordinates of the key point. Indicates weight, This is used to enhance the accuracy of the prediction results output by the 3D detection network. Indicates the first The three-dimensional feature map corresponding to each key point.

6. The method as described in claim 4, characterized in that, The robot also includes a base, and the step of acquiring training samples and the labels corresponding to the training samples includes: The camera is calibrated to obtain the first transformation matrix between the camera coordinate system and the gripper end coordinate system; Obtain reference images of the reference object captured by the camera from multiple viewpoints, and the camera parameters corresponding to each viewpoint; The coordinates of key points of the reference object in the reference image are labeled to obtain the first image; Based on the first image and the camera parameters, the reference three-dimensional coordinates of the key points of the reference object in the camera coordinate system are calculated using the direct linear transformation method. Based on the first transformation matrix and the preset transformation matrix, the reference three-dimensional coordinates are converted into real coordinates in the base coordinate system, wherein the preset transformation matrix represents the transformation relationship between the claw end coordinate system and the base coordinate system; The labeled reference image and the real coordinates are respectively used as the training sample and the label corresponding to the training sample.

7. The method according to claim 1, characterized in that, The crawling model was trained in the following way: Obtain the state parameters at time t, including the pose parameters of the gripper end, the velocity of the gripper end, and the opening and closing state of the gripper; The state parameters, the pose parameters of the target object, and the camera parameters corresponding to the maximum confidence are input into the pre-constructed grasping model, which includes an actor network and a critic network. Using the actor network of the aforementioned capture model, the motion parameters at time t are obtained; Control the robot to move according to the action parameters, complete the action at time t, and obtain the reward parameters; The reward parameters are input into the critic network, and the critic network is used to predict the Q value to obtain the predicted Q value, wherein the Q value is used to evaluate the value generated by the robot moving according to the action parameters; The crawling model is trained by backpropagation based on the predicted Q value, a preset loss function, and a preset gradient function to obtain the trained crawling model.

8. The method as described in claim 7, characterized in that, The crawling model is constructed in the following way: A state function is determined based on the pose parameters of the gripper tip, the velocity of the gripper tip, the opening and closing state of the gripper, the pose parameters of the target object, and the camera parameters corresponding to the maximum confidence level; wherein, the pose parameters of the gripper tip characterize the spatial position and attitude of the gripper tip relative to the base. The motion function is determined based on the speed at the end of the gripper, the rotation angle of the gripper, and the opening and closing state of the gripper. The reward function is determined based on the robot's grasping result, the distance between the gripper and the target object, and the robot's direction of movement. Based on the state function, the action function, and the reward function, the grasping model is constructed using a multi-view reinforcement learning grasping algorithm.

9. A robotic grasping device, characterized in that, An electronic device communicatively connected to a robot, the robot being equipped with a camera, and the robot including a base and grippers; the device includes: The acquisition module is used to acquire the image set obtained by the camera from multiple perspectives of the target object and the camera parameters corresponding to each perspective. The camera parameters are used to characterize the position information and attitude information of the camera relative to the base. The position information is represented by three-dimensional coordinates and the attitude information is represented by a rotation matrix. The detection module is used to input the image set and each camera parameter into the detection model, and use the detection model to predict the image set to obtain the predicted value of the first coordinate of the target object, wherein the first coordinate is the three-dimensional coordinate of the key point of the target object; The calculation module is used to calculate the pose parameters of the target object based on the predicted value of the first coordinate and the spatial geometric relationship between the key points, wherein the pose parameters are used to characterize the spatial position and attitude of the target object. The control module is used to control the robot to grasp the target object based on the pose parameters and the grasping model, wherein the grasping model is trained using reinforcement learning. The control module is also used to: Obtain the state parameters at time t, including the pose parameters of the gripper end, the velocity of the gripper end, and the opening and closing state of the gripper; The state parameters, the pose parameters of the target object, and the camera parameters corresponding to the maximum confidence are input into the grasping model to obtain the motion parameters at time t. The motion parameters include the velocity of the end of the gripper, the rotation angle of the gripper, and the opening and closing state of the gripper. Control the robot to move according to the action parameters, complete the action at time t, and obtain the state parameters at time t+1; The process of repeatedly inputting the state parameters, the pose parameters of the target object, and the camera parameters corresponding to the maximum confidence into the grasping model to obtain the motion parameters at time t is repeated until the robot completes the grasping of the target object. Specifically, based on the robot's grasping result, the distance between the gripper and the target object, and the robot's direction of motion, the reward function of the grasping model is determined, and the reward function is expressed as follows: in, It is a binary reward function. 0 indicates that the robot failed to grasp the object, and 1 indicates that the object was successfully grasped; Indicates the distance between the gripper and the target object. The opposite number, and The weight of the corresponding reward item; The reward for the robot moving along the line connecting the camera position with the highest view confidence level and the target object position is calculated as follows: in, This indicates the distance of the current gripper tip position relative to the line connecting the optimal viewpoint and the target object. This indicates a pre-set distance threshold.

10. An electronic device, characterized in that, The electronic device includes: One or more processors; A memory for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the robot grasping method as described in any one of claims 1-8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the robot grasping method as described in any one of claims 1-8.

Images