Robot grabbing posture determination method, device and equipment and storage medium

Abstract

The invention relates to a robot grabbing posture determination method, device and equipment and a storage medium, and the method comprises the steps: carrying out the detection and coordinate conversion of part image data based on a three-dimensional object posture estimation model, and obtaining part coordinates and a posture deflection angle under a mechanical arm base coordinate system; the grabbing posture of a to-be-grabbed part is obtained through a three-dimensional object posture estimation model, part coordinates and a posture deflection angle under a mechanical arm base coordinate system are obtained through coordinate transformation, inverse kinematics calculation is conducted on the part coordinates and the posture deflection angle, and the rotation angle of each joint on the robot is obtained; according to the method, the mechanical arm is rotated to enable the joints on the mechanical arm to be controlled according to the corresponding rotating angles, then the grabbing posture of the robot on the to-be-grabbed part is determined, automation of the robot is achieved, and through the automatic part rotating angle recognition and grabbing strategy, manual intervention is reduced, and the production efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of robotics technology, and in particular to a method, device, equipment and storage medium for determining a robot's grasping posture. Background Art

[0002] With the continuous development of industrial automation, industrial handling robots are playing an increasingly important role in production and logistics. By automating handling and manipulation, these robots not only improve production efficiency but also reduce labor costs and operational risks. However, existing industrial handling robots often face challenges in identifying part rotation angles and selecting gripping strategies when performing their tasks.

[0003] Currently, industrial handling robots primarily rely on traditional machine vision and robotic arm control technologies for part rotation angle recognition and grasping strategy selection. These technologies often suffer from slow response and poor adaptability when handling complex and dynamically changing environments. Furthermore, handling irregularly shaped parts often requires manual intervention, which not only reduces automation but also increases operating costs.

[0004] Therefore, there is an urgent need to propose a robot grasping posture determination method, device, equipment and storage medium to solve the technical problem in the existing technology that industrial handling robots are unable to perform posture recognition and grasping of irregularly shaped parts in a dynamically changing complex environment, resulting in a low degree of automation. Summary of the Invention

[0005] In view of this, it is necessary to provide a robot grasping posture determination method, device, equipment and storage medium to solve the technical problem in the existing technology that industrial handling robots are unable to perform posture recognition and grasping of irregularly shaped parts in a dynamically changing complex environment, resulting in a low degree of automation.

[0006] In order to solve the above problems, in a first aspect, the present invention provides a method for determining a grasping posture of a robot, comprising: Obtaining the image data and target placement position of the part to be grasped collected by the robot; Detecting and converting the part image data based on a three-dimensional object posture estimation model to obtain the part coordinates and posture deflection angle in the robot arm base coordinate system; An inverse kinematics calculation is performed on the part coordinates and the posture deflection angle to obtain the rotation angle of each joint on the robot, and the grasping posture of the robot for the part to be grasped is determined according to the rotation angle.

[0007] In one possible implementation, the detecting and coordinate conversion of the part image data based on the three-dimensional object posture estimation model to obtain the part coordinates and posture deflection angle in the robot arm base coordinate system includes: Detecting the part image data based on a three-dimensional object posture estimation model to obtain the coordinates of the part center point and the part rotation angle; The coordinate system of the part center point coordinates and the part rotation angle are converted to obtain the part coordinates and posture deflection angle in the robot arm base coordinate system.

[0008] In a possible implementation, performing coordinate system conversion on the part center point coordinates and the part rotation angle to obtain the part coordinates and posture deflection angle in the manipulator base coordinate system includes: The coordinates of the center point of the part and the rotation angle of the part are converted from the pixel coordinate system to the camera coordinate system based on the camera intrinsic parameter matrix to obtain the three-dimensional coordinates and three-dimensional posture; The three-dimensional coordinates and the three-dimensional posture are converted from the camera coordinate system to the robot arm base coordinate system based on the hand-eye matrix to obtain the part coordinates and posture deflection angle.

[0009] In one possible implementation, the feature extraction process of the three-dimensional object pose estimation model includes: Perform feature extraction on the part image data through three convolutional layers to obtain basic features, refined features and local characteristics; The basic features, refined features and local characteristics are learned at different resolution levels and upsampled to the original size through three deconvolution layers to obtain grasping features; the grasping features include the coordinates of the part center point, the part rotation angle, the grasping pixel width and the grasping quality; the part rotation angle is the rotation angle of each pixel point, and the grasping quality is used to evaluate the confidence of the grasping of each pixel point.

[0010] In a possible implementation, after determining the robot's grasping posture for the part to be grasped according to the rotation angle, the method further includes: Performing path planning based on the part coordinates and the target placement position to obtain an optimal path; According to the rotation angle and the optimal path, the robot is controlled to place the part to be grasped at the target placement position.

[0011] In a possible implementation, the method further includes: Obtain a sample set of images of parts; Preprocessing the image sample set to obtain a preprocessed sample set; Annotating the images in the preprocessed sample set to obtain an annotated sample set; Training a preset posture estimation model using the labeled sample set to obtain a training result set and an initial posture estimation model; Drawing a curve graph according to the data in the training result set according to the training rounds to obtain a convergence performance graph; The structure and parameters of the initial pose estimation model are tuned according to the convergence performance graph and the minimization loss function to obtain a three-dimensional object pose estimation model.

[0012] In one possible implementation, the minimization loss function is:

[0013] Where, 、 、 It is the confidence prediction value, posture deflection angle prediction value, and grasped pixel width prediction value of the three-dimensional object posture estimation model. 、 、 is the corresponding true annotation value, , 、 is the weight coefficient of each part of the loss, which is used to balance the contribution of the loss of different parts to the total loss. 、 、 They are the loss functions of grasping quality, grasping angle and grasping width respectively.

[0014] In a second aspect, the present invention further provides a robot grasping posture determination device, comprising: An image acquisition module is used to obtain the image data of the part to be grasped and the target placement position collected by the robot; A coordinate conversion module is used to detect and convert the part image data based on a three-dimensional object posture estimation model to obtain the part coordinates and posture deflection angle in the robot arm base coordinate system; An angle calculation module is used to perform inverse kinematics calculation on the part coordinates and the posture deflection angle to obtain the rotation angle of each joint on the robot, and determine the robot's grasping posture for the part to be grasped based on the rotation angle.

[0015] In a third aspect, an embodiment of the present invention discloses an electronic device comprising: a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program, when executed by the processor, implements the various steps of the above-mentioned robot grasping posture determination method embodiment.

[0016] In a fourth aspect, an embodiment of the present invention discloses a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various steps of the above-mentioned robot grasping posture determination method embodiment are implemented.

[0017] The beneficial effects of the present invention are: obtaining the part image data and target placement position of the part to be grasped collected by the robot; detecting and converting the part image data based on the three-dimensional object posture estimation model to obtain the part coordinates and posture deflection angle in the base coordinate system of the robot arm; thereby the grasping posture of the part to be grasped can be obtained through the three-dimensional object posture estimation model, and the part coordinates and posture deflection angle in the base coordinate system of the robot arm can also be obtained through coordinate conversion, so that it can be not affected by the environment and shape of the part to be grasped, and the part coordinates and posture deflection angle are inverse kinematically calculated to obtain the rotation angle of each joint on the robot; so that the joints on the robot arm can be controlled according to the corresponding rotation angle; and then determining the robot's grasping posture for the part to be grasped, and then realizing the automation of the robot through the process of image detection-model processing-angle determination-path optimization-posture determination, and reducing manual intervention and improving production efficiency through automated part rotation angle recognition and grasping strategy. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 A schematic flow chart of an embodiment of a method for determining a robot grasping posture provided by the present invention; Figure 2 A schematic diagram of the structure of an embodiment of the logical architecture of the robot grasping posture determination system provided by the present invention; Figure 3 A schematic diagram of the structure of a neural network for a grasping posture estimation algorithm according to the present invention; Figure 4 A schematic diagram of an embodiment of the model training process provided by the present invention; Figure 5 A schematic structural diagram of an embodiment of a device for determining a robot grasping posture provided by the present invention; Figure 6 This is a schematic structural diagram of an embodiment of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0019] The preferred embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings, wherein the accompanying drawings constitute a part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, and are not used to limit the scope of the present invention.

[0020] like Figure 1 As shown, a specific embodiment of the present invention discloses a method for determining a robot grasping posture, comprising: S101: Acquire part image data and target placement position of a part to be grasped collected by a robot.

[0021] The robot grasping posture determination method provided in the embodiment of the present application can be applied to a robot grasping posture determination system, wherein the robot grasping posture determination system can be a software system running on a terminal device, and the terminal device can be a server, a tablet computer, an augmented reality (AR) / virtual reality (VR) device, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), a mobile phone and other terminal devices. The embodiment of the present application does not impose any restrictions on the specific type of the terminal device.

[0022] Among them, the robot can be a robotic arm, and the camera is installed on the end effector of the robotic arm. When the robotic arm is working, the camera can be used to collect part image data of the part to be grasped. After collection, the part image data can be preprocessed. The preprocessing can include but is not limited to normalization, denoising and other operations, and can also determine the target placement position. The target placement position can be a fixed position, or it can be matched with a corresponding placement slot according to different problems. The specific setting can be based on actual conditions, and the embodiment of the present invention is not limited here.

[0023] S102: Detect and convert the part image data based on the three-dimensional object posture estimation model to obtain the part coordinates and posture deflection angle in the robot arm base coordinate system.

[0024] Among them, a three-dimensional object pose estimation model can be constructed, which can be GGCNN2. GCNN2 is a model for three-dimensional object pose estimation. Its core improvement lies in the use of void convolution technology, which enhances the network's perception ability in feature extraction. GGCNN2 can accurately predict the grasping posture of an object and evaluate the grasping effect of each posture, making it more effective and accurate when handling complex grasping tasks. Therefore, the part image data can be detected by the three-dimensional object pose estimation model to obtain the detection results, and then the coordinate transformation of the detection structure can be performed to obtain the part coordinates and posture deflection angle in the robotic arm base coordinate system that can be used for the robotic arm.

[0025] S103, performing inverse kinematics calculation on the part coordinates and the posture deflection angle to obtain the rotation angle of each joint on the robot, and determining the robot's grasping posture for the part to be grasped according to the rotation angle.

[0026] After obtaining the part coordinates and posture deflection angles in the robot's base coordinate system, the rotation angles of each joint can be calculated based on inverse kinematics. The purpose of inverse kinematics calculations is to determine the angle by which each joint of the robot should rotate so that the end effector can accurately reach the target posture and grasp the part. Based on the rotation angle of each joint, the robot's grasping posture for the part to be grasped can be determined. The grasping posture includes grasping force, grasping width, etc., so that the robot can grasp the part according to the grasping posture.

[0027] The logical architecture of the robot grasping posture determination system in the embodiment of the present invention can be divided into three layers from top to bottom: training end, control end, and application end. Figure 2 As shown, the training end is mainly used for training the deep learning model of the visual control system. It can also collect and process samples, and then make a sample data set. The constructed three-dimensional object posture estimation model is then trained based on the data set. The training can include data echo, effect evaluation and parameter download, etc. The trained three-dimensional object posture estimation model can be set in the target detection module. The control end can collect the image data of the part, and then pre-process the image data through the three-dimensional object posture estimation model of the target detection module to obtain the posture position, and then obtain the angle of each joint through matrix transformation. The angle is transmitted to the application end through the robot control module. The application end shows the user the intuitive operation scene of the robot, can control the various joints of the robotic arm, and can also perform path planning and force control adjustment. The robotic arm can grasp the object to be grasped through the optimal path and force control, and place it in the part placement slot.

[0028] Compared with the prior art, the present embodiment provides the acquisition of part image data and target placement position of the part to be grasped collected by the robot; the part image data is detected and coordinate-converted based on the three-dimensional object posture estimation model to obtain the part coordinates and posture deflection angle in the base coordinate system of the robot arm; thereby, the grasping posture of the part to be grasped can be obtained through the three-dimensional object posture estimation model, and the part coordinates and posture deflection angle in the base coordinate system of the robot arm can also be obtained through coordinate conversion, so that it can be unaffected by the environment and shape of the part to be grasped, and the part coordinates and posture deflection angle are inversely calculated to obtain the rotation angle of each joint on the robot; so that the joints on the robot arm can be controlled according to the corresponding rotation angle; and then the grasping posture of the robot for the part to be grasped is determined, and then the automation of the robot can be realized through the process of image detection-model processing-angle determination-path optimization-posture determination, and the automated part rotation angle recognition and grasping strategy can be used to reduce manual intervention and improve production efficiency.

[0029] In some embodiments of the present invention, step S102 includes: Detect part image data based on the 3D object posture estimation model to obtain the part center point coordinates and part rotation angle; The coordinate system of the part center point coordinates and the part rotation angle are transformed to obtain the part coordinates and posture deflection angle in the robot arm base coordinate system.

[0030] In a specific embodiment of the present invention, a 3D object pose estimation model can be constructed. After training the 3D object pose estimation model, part image data can be input into the 3D object pose estimation model. The 3D object pose estimation model can perform feature extraction and detection on the part image data, and then output the part center point coordinates and part rotation angle. The 3D object pose estimation model is GGCNN2. Since GGCNN2 processes image data, it is necessary to perform coordinate conversion on the part center point coordinates and part rotation angle to obtain the part coordinates and posture deflection angle in the robot arm base coordinate system.

[0031] In some embodiments of the present invention, the coordinate system conversion of the part center point coordinates and the part rotation angle is performed to obtain the part coordinates and the posture deflection angle in the robot arm base coordinate system, including: Based on the camera intrinsic parameter matrix, the coordinates of the part center point and the part rotation angle are converted from the pixel coordinate system to the camera coordinate system to obtain the three-dimensional coordinates and three-dimensional posture; Based on the hand-eye matrix, the three-dimensional coordinates and three-dimensional posture are converted from the camera coordinate system to the robot arm base coordinate system to obtain the part coordinates and posture deflection angle.

[0032] In a specific embodiment of the present invention, the robot arm needs to obtain the grasping position, grasping posture and grasping width through the camera, and the grasping posture can be uniformly defined using formula (1): (1) Where, p is the three-dimensional coordinate of the object relative to the base of the robot arm, The gripper is along the robotic arm z The deflection angle of the axis, w is the opening and closing degree of the clamp, q is the confidence of the grasping posture.

[0033] Since GGCNN2 processes image data, the grasping posture of formula (1) is mapped to the RealSense image coordinate system, as shown in formula (2): (2) Where, p i is the pixel coordinate of the center point of the grab, is the rotation angle of the point relative to the camera coordinate system, To capture the width of the pixel, The capture quality for each pixel.

[0034] The center point of the part to be detected and its deflection angle output by the three-dimensional object posture estimation model are pixel coordinates, which have not yet been converted to the real three-dimensional space coordinate system. The pixel coordinates in the image need to be converted into three-dimensional coordinates and three-dimensional postures in the camera coordinate system through the camera intrinsic parameter matrix. When the robot performs the task, it needs to perform motion control based on its own base coordinate system, so it is also necessary to convert the three-dimensional coordinates and three-dimensional postures into the manipulator base coordinate system through the hand-eye matrix to obtain the manipulator base coordinate system, where the camera intrinsic parameter matrix It can be set according to the actual situation. The conversion from the camera coordinate system to the robot arm base coordinate system can adopt the "eye on hand" calibration method to obtain the hand-eye calibration matrix , then the conversion relationship between formula (1) and formula (2) is shown in formula (3): (3) In some embodiments of the present invention, the feature extraction process of the 3D object pose estimation model includes: The feature extraction of part image data is performed through three convolutional layers to obtain basic features, refined features and local characteristics; Through three deconvolution layers, the basic features, refined features and local features are learned at different resolution levels and upsampled to the original size to obtain the captured features.

[0035] In a specific embodiment of the present invention, Figure 3 As shown, Figure 3 To grasp the neural network structure diagram of the posture estimation algorithm, GGCNN2 accepts part image data (ie, depth image) from the RealSense camera and crops it to 300x300 pixels for input. The network first extracts features through three convolutional layers: the first layer uses 32 9x9 filters to capture basic features such as edges and corners; the second layer's 16 5x5 filters further refine the feature extraction; the third layer's 8 3x3 filters focus on local features. Subsequently, these features are learned at different resolution levels through three deconvolution layers and upsampled to the original size to obtain grasping features. Through this series of convolution and deconvolution operations, GGCNN2 gradually parses the grasping features in the depth image and ultimately generates four key parameters that guide the robotic arm's grasping action. .

[0036] Grasping features can include the coordinates of the part center point, the part rotation angle, the width of the grasped pixel point, and the grasping quality; the part rotation angle is the rotation angle of each pixel point, and the grasping quality is used to evaluate the confidence level of each pixel point grasp. The GGCNN2 network not only has the ability to process depth images and gradually learn grasping features, but also can estimate the grasping posture of each pixel point. The network output includes the grasping quality of each point. q i , which is a scalar between [0,1], used to evaluate the confidence of each pixel grasp; the grasp success rate is q i The value increases as it approaches 1. Grab angle (i.e. rotation angle) It varies in the range of [-π / 2,π / 2], reflecting the direction of the grasping action. w Indicates the width of the gripper opening, with a value from 0 to 150.

[0037] In some embodiments of the present invention, after step S103, the method further comprises; Performing path planning based on the part coordinates and the target placement position to obtain an optimal path; According to the rotation angle and the optimal path, the robot is controlled to place the part to be grasped at the target placement position.

[0038] In a specific embodiment of the present invention, path planning can be performed using part coordinates and target placement locations to obtain an optimal path. The specific process includes setting up a path planning neural network model, a fast random exploration tree, a probabilistic roadmap, and trajectory optimization methods. The specific settings can be made according to the present invention, and the present invention is not limited here. After determining the optimal path, the robotic arm can be controlled by controlling the joints through the rotation angle of each joint, controlling the force through the grasping width, controlling the robotic arm to grasp the part to be grasped, and placing the part to be grasped at the target placement location according to the optimal path.

[0039] In some embodiments of the present invention, Figure 4 As shown, the method further includes: S401: Obtain a sample set of image of parts.

[0040] It should be understood that the method of obtaining the image sample set of parts can be an image sample set of parts of different shapes obtained according to an image acquisition device (for example, a depth camera), or it can be an image sample set of parts of different shapes that is historically stored and called from a storage medium. The specific method can be set according to actual conditions, and the embodiment of the present invention is not limited here.

[0041] S402, preprocessing the image sample set to obtain a preprocessed sample set; S403: Label the images in the preprocessed sample set to obtain a labeled sample set.

[0042] Among them, after obtaining the image sample set, in order to unify the image data, the image data in the image sample set can be preprocessed, for example, the size and format are unified, etc., to obtain the image sample set. The image sample set can also be accurately labeled to obtain a labeled sample set. The specific labeling process can be set according to actual conditions, and the embodiment of the present invention is not limited here.

[0043] S404: Train the preset pose estimation model using the labeled sample set to obtain a training result set and an initial pose estimation model.

[0044] Among them, a preset posture estimation model can be constructed. The preset posture estimation model is an initially constructed three-dimensional object posture estimation model. After obtaining the labeled sample set, the image data in the labeled sample set can be used for training, and the training results of each image can be obtained. When the training of the image data in the labeled sample set is completed, the training result set and the initial posture estimation model are obtained.

[0045] S405 , plotting a curve graph according to the data in the training result set and the training rounds to obtain a convergence performance graph.

[0046] A curve graph may be drawn according to the data in the training result set and the number of training rounds or iterations, thereby obtaining a convergence performance graph.

[0047] S406 , optimizing the structure and parameters of the initial pose estimation model according to the convergence performance graph and the minimization loss function to obtain a three-dimensional object pose estimation model.

[0048] After obtaining the convergence performance graph, you can analyze it. The following issues can be analyzed using the convergence performance graph: Convergence speed: The faster the curve decreases, the higher the model learning efficiency (which may be related to the optimizer and learning rate settings). Convergence stability: Large fluctuations in the curve may indicate that the learning rate is too high or the batch size is inappropriate. Overfitting / underfitting: Overfitting: The validation metrics are significantly worse than the training metrics. Underfitting: Both training and validation metrics are poor (insufficient model capacity or data issues). Early stopping: When the validation loss no longer decreases, training can be terminated early. Based on the analysis results and the minimized loss function, the structure and parameters of the initial pose estimation model are tuned to ensure optimal model performance and obtain a 3D object pose estimation model.

[0049] Minimizing the loss function can be used to measure the gap between the network's predicted crawling parameters and the actual annotation parameters. The minimizing loss function is shown in formula (4): (4) Where, 、 、 It is the confidence prediction value, posture deflection angle prediction value, and grasped pixel width prediction value of the three-dimensional object posture estimation model. 、 、 is the corresponding true annotation value, , 、 is the weight coefficient of each part of the loss, which is used to balance the contribution of the loss of different parts to the total loss. 、 、 The training process is optimized by approximating the loss function L, aiming to minimize the difference between the predicted value and the true value.

[0050] The embodiments of this invention reduce manual intervention and improve production efficiency through automated part rotation angle recognition and grasping strategies. This reduces production interruptions and equipment damage caused by operational errors, lowering operating costs. This enables the robot to adapt to a wider range of part types and operating environments, improving system flexibility and adaptability. This provides new technical solutions for the industrial automation field and promotes the development and application of related technologies.

[0051] In order to better implement the robot grasping posture determination method in the embodiment of the present invention, based on the robot grasping posture determination method, the embodiment of the present invention also provides a robot grasping posture determination device, such as Figure 5 As shown, the robot grasping posture determination device 500 includes: Image acquisition module 501, used to obtain part image data and target placement position of the part to be grasped collected by the robot; A coordinate conversion module 502 is used to detect and convert the part image data based on the three-dimensional object posture estimation model to obtain the part coordinates and posture deflection angle in the robot arm base coordinate system; The angle calculation module 503 is used to perform inverse kinematics calculation on the part coordinates and the posture deflection angle to obtain the rotation angle of each joint on the robot, and determine the grasping posture of the robot for the part to be grasped according to the rotation angle.

[0052] The robot grasping posture determination device 500 provided in the above embodiment can implement the technical solution described in the above embodiment of the robot grasping posture determination method. The specific implementation principles of the above modules or units can refer to the corresponding contents in the above embodiment of the robot grasping posture determination method, which will not be repeated here.

[0053] like Figure 6 As shown, the present invention also provides an electronic device 600. The electronic device 600 includes a processor 601, a memory 602 and a display 603. Figure 6 Only some of the components of the electronic device 600 are shown, but it should be understood that implementation of all of the shown components is not required, and more or fewer components may be implemented instead.

[0054] In some embodiments, the memory 602 may be an internal storage unit of the electronic device 600, such as a hard disk or memory of the electronic device 600. In other embodiments, the memory 602 may also be an external storage device of the electronic device 600, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 600.

[0055] Furthermore, the memory 602 may include both an internal storage unit of the electronic device 600 and an external storage device. The memory 602 is used to store application software installed in the electronic device 600 and various data.

[0056] In some embodiments, the processor 601 may be a central processing unit (CPU), a microprocessor, or other data processing chip, configured to execute program codes or process data stored in the memory 602, such as the robot grasping posture determination method of the present invention.

[0057] In some embodiments, the display 603 can be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. The display 603 is used to display information about the electronic device 600 and to display a visual user interface. Components 601-603 of the electronic device 600 communicate with each other via a system bus.

[0058] In some embodiments of the present invention, when the processor 601 executes the robot grasping posture determination program in the memory 602, the following steps may be implemented: Obtaining the image data and target placement position of the part to be grasped collected by the robot; Based on the three-dimensional object posture estimation model, the part image data is detected and the coordinates are converted to obtain the part coordinates and posture deflection angle in the robot arm base coordinate system, and the robot's grasping posture for the part to be grasped is determined according to the rotation angle.

[0059] It should be understood that, when the processor 601 executes the robot grasping posture determination program in the memory 602 , in addition to the above functions, it can also implement other functions. For details, please refer to the description of the corresponding method embodiment above.

[0060] Furthermore, the embodiments of the present invention do not specifically limit the type of electronic device 600 mentioned. The electronic device 600 may be a portable electronic device such as a mobile phone, tablet computer, personal digital assistant (PDA), wearable device, or laptop computer. Exemplary embodiments of portable electronic devices include, but are not limited to, portable electronic devices running iOS, Android, Microsoft, or other operating systems. The portable electronic devices mentioned above may also be other portable electronic devices, such as a laptop computer with a touch-sensitive surface (e.g., a touch panel). It should also be understood that in some other embodiments of the present invention, the electronic device 600 may not be a portable electronic device, but rather a desktop computer with a touch-sensitive surface (e.g., a touch panel).

[0061] Accordingly, an embodiment of the present application also provides a computer-readable storage medium, which is used to store computer-readable programs or instructions. When the program or instructions are executed by the processor, the robot grasping posture determination method steps or functions provided in the above-mentioned method embodiments can be implemented.

[0062] Those skilled in the art will appreciate that all or part of the process flow of the above-described method embodiment can be implemented by instructing related hardware (such as a processor, controller, etc.) through a computer program, and the computer program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a magnetic disk, an optical disk, a read-only memory, or a random access memory.

[0063] The above is a detailed introduction to the robot grasping posture determination method, device, equipment and storage medium provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for technical personnel in this field, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A method for determining a robot grasping posture, characterized in that: include: Obtaining the image data and target placement position of the part to be grasped collected by the robot; Detecting and converting the part image data based on a three-dimensional object posture estimation model to obtain the part coordinates and posture deflection angle in the robot arm base coordinate system; An inverse kinematics calculation is performed on the part coordinates and the posture deflection angle to obtain the rotation angle of each joint on the robot, and the grasping posture of the robot for the part to be grasped is determined according to the rotation angle.

2. The robot grasping posture determination method according to claim 1, characterized in that: The detecting and coordinate conversion of the part image data based on the three-dimensional object posture estimation model to obtain the part coordinates and posture deflection angle in the robot arm base coordinate system includes: Detecting the part image data based on a three-dimensional object posture estimation model to obtain the coordinates of the part center point and the part rotation angle; The coordinate system of the part center point coordinates and the part rotation angle are converted to obtain the part coordinates and posture deflection angle in the robot arm base coordinate system.

3. The robot grasping posture determination method according to claim 2, characterized in that: The coordinate system conversion of the part center point coordinates and the part rotation angle to obtain the part coordinates and posture deflection angle in the robot arm base coordinate system includes: The coordinates of the center point of the part and the rotation angle of the part are converted from the pixel coordinate system to the camera coordinate system based on the camera intrinsic parameter matrix to obtain the three-dimensional coordinates and three-dimensional posture; The three-dimensional coordinates and the three-dimensional posture are converted from the camera coordinate system to the robot arm base coordinate system based on the hand-eye matrix to obtain the part coordinates and posture deflection angle.

4. The robot grasping posture determination method according to claim 2, characterized in that: The feature extraction process of the three-dimensional object posture estimation model includes: Perform feature extraction on the part image data through three convolutional layers to obtain basic features, refined features and local characteristics; The basic features, refined features and local characteristics are learned at different resolution levels and upsampled to the original size through three deconvolution layers to obtain grasping features; the grasping features include the coordinates of the part center point, the part rotation angle, the grasping pixel width and the grasping quality; the part rotation angle is the rotation angle of each pixel point, and the grasping quality is used to evaluate the confidence of the grasping of each pixel point.

5. The robot grasping posture determination method according to claim 4, characterized in that: After determining the gripping posture of the robot for the part to be gripped according to the rotation angle, the method further includes: Performing path planning based on the part coordinates and the target placement position to obtain an optimal path; According to the rotation angle and the optimal path, the robot is controlled to place the part to be grasped at the target placement position.

6. The robot grasping posture determination method according to claim 1, characterized in that: The method further comprises: Obtain a sample set of images of parts; Preprocessing the image sample set to obtain a preprocessed sample set; Annotating the images in the preprocessed sample set to obtain an annotated sample set; Training a preset posture estimation model using the labeled sample set to obtain a training result set and an initial posture estimation model; Drawing a curve graph according to the data in the training result set according to the training rounds to obtain a convergence performance graph; The structure and parameters of the initial pose estimation model are tuned according to the convergence performance graph and the minimization loss function to obtain a three-dimensional object pose estimation model.

7. The robot grasping posture determination method according to claim 6, characterized in that: The minimization loss function is: Where, 、 、 It is the confidence prediction value, posture deflection angle prediction value, and grasped pixel width prediction value of the three-dimensional object posture estimation model. 、 、 is the corresponding true annotation value, , 、 is the weight coefficient of each part of the loss, which is used to balance the contribution of the loss of different parts to the total loss. 、 、 They are the loss functions of grasping quality, grasping angle and grasping width respectively.

8. A robot grasping posture determination device, characterized in that: include: An image acquisition module is used to obtain the image data of the part to be grasped and the target placement position collected by the robot; A coordinate conversion module is used to detect and convert the part image data based on a three-dimensional object posture estimation model to obtain the part coordinates and posture deflection angle in the robot arm base coordinate system; An angle calculation module is used to perform inverse kinematics calculation on the part coordinates and the posture deflection angle to obtain the rotation angle of each joint on the robot, and determine the robot's grasping posture for the part to be grasped based on the rotation angle.

9. An electronic device, characterized in that: include: A processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein when the computer program is executed by the processor, the steps of the robot grasping posture determination method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the robot grasping posture determination method according to any one of claims 1 to 7 are implemented.

Images