Grabbing pose detection model retraining method, equipment, robot and program product

Abstract

The invention relates to a grabbing pose detection model retraining method and device, a robot and a program product. The method comprises the steps that first grabbing pose information corresponding to a target object grabbed by a human teaching robot is obtained; determining a plurality of reference points on the surface of the target object; on the basis of each datum point, multiple pieces of second grabbing pose information are determined; based on each datum point, the deviation between each piece of second grabbing pose information and the first grabbing pose information is determined; on the basis of each piece of second grabbing pose information, according to the deviation between the corresponding second grabbing pose information and the first grabbing pose information, determining the confidence degree corresponding to the corresponding second grabbing pose information; and according to the three-dimensional visual data of the target object, the second grabbing pose information corresponding to each datum point and the confidence coefficient corresponding to each second grabbing pose information, retraining the pre-trained grabbing pose detection model.

Description

Technical Field

[0001] This disclosure relates to the field of robot model training technology, and more specifically, to a method, device, robot, and program product for retraining a grasping pose detection model. Background Technology

[0002] Existing grasping pose detection models are typically trained under supervision using large-scale, finely labeled grasping pose information, enabling the trained grasping pose detection model to densely predict candidate grasping poses and their confidence levels on the surface of the object to be grasped.

[0003] However, when the type of object to be grasped, the work scenario, or the type of robot changes, the performance of the trained grasping pose detection model often drops sharply. Current technologies involve re-collecting and labeling massive amounts of data as training samples for retraining, which is time-consuming and costly. Summary of the Invention

[0004] One objective of this invention is to provide a new technical solution for retraining a grasping pose detection model.

[0005] According to a first aspect of the present invention, a method for retraining a grasping pose detection model is provided, comprising: Obtain the first grasping pose information corresponding to the target object grasped by the human-taught robot; Determine multiple reference points located on the surface of the target object; Based on each reference point, multiple second grasping pose information are determined; Based on each reference point, the deviation between each second grasping pose information and the first grasping pose information is determined; Based on each second grasping pose information, the confidence level corresponding to the second grasping pose information is determined according to the deviation between the corresponding second grasping pose information and the first grasping pose information. The pre-trained grasping pose detection model is retrained based on the three-dimensional visual data of the target object, the second grasping pose information corresponding to each reference point, and the confidence level corresponding to each second grasping pose information.

[0006] Optionally, the determination of multiple second grasping pose information based on each reference point includes: Construct the target sphere using each reference point as its center; Determine a first number of direction vectors, wherein each direction vector is determined based on a point on the surface of the target sphere and the center of the sphere. Using each direction vector as the spin axis of the robot end effector, and rotating based on the corresponding spin axis, the rotational attitude of the second number of robot end effectors is determined. Based on each rotational posture, determine the grasping depth of the third number of robot end effectors; For each reference point, the corresponding second grasping pose information is determined based on each direction vector, each rotational posture, and each grasping depth.

[0007] Optionally, both the first grasping pose information and the second grasping pose information include translational and rotational information of the robot's end effector, wherein, The step of determining the deviation between each second grasping pose information and the first grasping pose information based on each reference point includes: Based on each second grasping pose information, the translation deviation of the corresponding second grasping pose information is determined according to the translation information in the corresponding second grasping pose information and the translation information in the first grasping pose information, and the rotation deviation of the corresponding second grasping pose information is determined according to the rotation information in the corresponding second grasping pose information and the rotation information in the first grasping pose information.

[0008] Optionally, the step of determining the confidence level corresponding to each second grasping pose information based on the deviation between the corresponding second grasping pose information and the first grasping pose information includes: Obtain the weighting coefficients corresponding to translational deviation and rotational deviation; Based on each second grasping pose information, the confidence level corresponding to the second grasping pose information is determined according to the corresponding translation deviation, the weight coefficient corresponding to the translation deviation, the corresponding rotation deviation, and the weight coefficient corresponding to the rotation deviation.

[0009] Optionally, before determining the confidence level corresponding to the second grasping pose information based on the corresponding translation deviation, the weighting coefficient corresponding to the translation deviation, the corresponding rotation deviation, and the weighting coefficient corresponding to the rotation deviation, the method further includes: The translational deviation corresponding to each second grasping pose information is filtered according to the first preset threshold, and the rotational deviation corresponding to each second grasping pose information is filtered according to the second preset threshold, so as to obtain the second grasping pose information corresponding to the translational deviation being less than the first preset threshold and the rotational deviation being less than the second preset threshold. The confidence level corresponding to the second grasping pose information where the translation deviation is less than the first preset threshold and the rotation deviation is less than the second preset threshold is determined as the set confidence level.

[0010] Optionally, the step of retraining the pre-trained grasping pose detection model based on the three-dimensional visual data of the target object, the second grasping pose information corresponding to each reference point, and the confidence level corresponding to each second grasping pose information includes: The three-dimensional visual data of the target object, the second grasping pose information corresponding to each reference point, and the confidence level corresponding to each second grasping pose information are used as training samples to retrain the pre-trained grasping pose detection model and obtain the prediction confidence level of the second grasping pose information corresponding to each reference point. The loss function is determined based on the prediction confidence of the second grasping pose information corresponding to each reference point and the confidence of the second grasping pose information corresponding to each reference point in the training sample. If the loss function is less than a third preset threshold, the retraining of the pre-trained grasping pose detection model is stopped, and a retrained grasping pose detection model is obtained.

[0011] Optionally, the three-dimensional visual data is an RGB image and a depth image corresponding to the target object, or the three-dimensional visual data is three-dimensional point cloud data corresponding to the target object.

[0012] According to a second aspect of the present invention, a computer device is provided, including a memory and a processor, the memory storing a computer program for controlling the processor to operate in order to perform the method according to any one of the first aspects.

[0013] According to a third aspect of the invention, a robot is provided, comprising a computer device as provided in the second aspect.

[0014] According to a fourth aspect of the present invention, a computer program product is provided, comprising a computer program that, when executed by a processor of a computer device, enables the computer device to perform the method as described in any one of the first aspects.

[0015] The retraining method for the grasping pose detection model disclosed herein, based on a newly added work scene and / or a newly added target object and / or a newly added robot, determines the first grasping pose information by having a human teach the robot to grasp the target object using data. Based on each reference point on the surface of the target object, multiple second grasping pose information are determined. According to the deviation between each second grasping pose information and the first grasping pose information, a confidence level is determined for each corresponding second grasping pose information. The 3D visual data of the target object, the second grasping pose information corresponding to each reference point, and the confidence levels corresponding to each second grasping pose information are used as training data to retrain the pre-trained grasping pose detection model. Since the first grasping pose information is determined based on a human teach the robot and has high accuracy, the confidence levels of each second grasping pose information determined using the first grasping pose information as a reference have high accuracy, providing accurate data for subsequent retraining. Furthermore, compared to existing technologies, it eliminates the need for extensive collection and annotation of training data, saving costs and shortening the training cycle.

[0016] The features and advantages of the embodiments of this specification will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments of this specification and, together with their description, serve to explain the principles of these embodiments.

[0018] Figure 1 This is a flowchart illustrating a method for retraining a grasping pose detection model according to an embodiment of the present invention.

[0019] Figure 2 This is a schematic diagram of a three-dimensional point cloud of a target object according to an embodiment of the present invention.

[0020] Figure 3 This is a schematic diagram of the rotational posture of a robot end effector according to an embodiment of the present invention.

[0021] Figure 4 This is a schematic diagram of the gripping depth of a robot end effector according to an embodiment of the present invention.

[0022] Figure 5 This is a schematic block diagram of a retraining device for a grasping pose detection model according to an embodiment of the present invention.

[0023] Figure 6 This is a schematic diagram of the structure of a computer device according to an embodiment of the present invention. Detailed Implementation

[0024] Various exemplary embodiments of this specification will now be described in detail with reference to the accompanying drawings.

[0025] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the embodiments of this specification or their application or use.

[0026] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0027] To address the aforementioned technical issues, this disclosure provides a method for retraining a grasping pose detection model. Based on a new work scene, and / or a new target object, and / or a new robot, a first grasping pose is determined by a human-taught robot capturing data corresponding to the target object. Multiple second grasping poses are determined based on each reference point on the target object's surface. The confidence level of each second grasping pose is determined based on its deviation from the first grasping pose. The 3D visual data of the target object, the second grasping poses corresponding to each reference point, and the confidence levels of each second grasping pose are used as training data to retrain the pre-trained grasping pose detection model. Since the first grasping pose is determined by a human-taught robot and has high accuracy, the confidence levels of each second grasping pose determined using the first grasping pose as a reference have high accuracy, providing accurate data for subsequent retraining. Furthermore, compared to existing technologies, this method eliminates the need for extensive collection and annotation of training data, saving costs and shortening the training cycle.

[0028] One embodiment of the present invention provides a method for retraining a grasping pose detection model. According to... Figure 1 As shown, the retraining method for the grasping pose detection model in this embodiment includes the following steps S110 to S130.

[0029] Step S110: Obtain the first grasping pose information corresponding to the target object grasped by the human-taught robot.

[0030] For new work scenarios, and / or new target objects, and / or new robots, acquire the first grasping pose information corresponding to the method by which the human-taught robot grasps the target object (e.g., teleoperation, drag-and-drop teaching, or VR interaction). Only a small number of target objects grasped by the human-taught robot are needed to meet subsequent retraining requirements.

[0031] The first grasping pose information consists of the translation and rotation information of the end effector of the human-taught robot as it moves from its original position to the position corresponding to the grasped target object. Translation information can be represented using translation vectors, which are vectors in a defined three-dimensional coordinate system, which can be the robot's body coordinate system. Rotation information can be represented using rotation matrices.

[0032] Step S120: Determine multiple reference points located on the surface of the target object.

[0033] The number and distribution of benchmark points can be set according to requirements.

[0034] Step S130: Based on each reference point, determine multiple second grasping pose information.

[0035] Referring to the information included in the first grasping pose information described above, the second grasping pose information also includes translation and rotation information. Translation information can be represented using translation vectors, which are vectors in a defined three-dimensional coordinate system, which can be the robot's body coordinate system. Rotation information can be represented using rotation matrices.

[0036] In some embodiments, step S130 specifically includes steps S131 to S135.

[0037] Step S131: Construct the target sphere with each reference point as the center.

[0038] Step S132: Determine a first number of direction vectors, wherein each direction vector is determined based on a point on the surface of the target sphere and the center of the sphere.

[0039] Step S133: Using each direction vector as the spin axis of the robot end effector, rotate based on the corresponding spin axis to determine the rotational attitude of the second number of robot end effectors. Step S134: Based on each rotational posture, determine the grasping depth of the third number of robot end effectors.

[0040] Step S135: For each reference point, determine the corresponding second grasping pose information based on each direction vector, each rotational posture, and each grasping depth.

[0041] The second method for determining the grasping pose information provided in this embodiment provides a large and comprehensive data foundation for subsequent retraining.

[0042] Figure 2 A schematic diagram of a three-dimensional point cloud of a target object according to an embodiment of the present invention is shown. Figure 2 In the 3D point cloud shown, N points are sampled as reference points.

[0043] Taking one of the N reference points as an example, the determination of multiple second grasping pose information is explained.

[0044] according to Figure 2 As shown, taking one of N reference points as an example, a target sphere is constructed with this reference point as its center. Based on this target sphere, V direction vectors are determined. Each direction vector is determined based on the center of the target sphere and a point on the surface of the target sphere. That is, V points are selected from the surface of the target sphere, and a direction vector can be determined based on each point and the center of the sphere.

[0045] according to Figure 3As shown, taking one of the V direction vectors as an example, this direction vector is used as the spin rotation axis of the robot's end effector. Based on the corresponding spin rotation axis, the robot's end effector rotates R times to obtain R rotation attitudes of the robot's end effector. Figure 3 The diagram shows the rotational postures of seven robot end effectors obtained by rotating seven times based on the corresponding spin axis. This is merely an illustration and does not constitute any limitation.

[0046] according to Figure 4 As shown, taking a rotational posture as an example, the grasping depth of D robot end effectors is determined. Figure 4 The diagram illustrates four gripping depths based on a rotational posture; this is merely illustrative and does not constitute any limitation.

[0047] In combination with the above Figure 2 , Figure 3 and Figure 4 Based on each reference point, it is possible to determine The second grasping pose information. Based on a target object, it is possible to determine... Each second grasping pose information can be determined based on the corresponding direction vector, rotational attitude information, and grasping depth.

[0048] Step S140: Based on each reference point, determine the deviation between each second grasping pose information and the first grasping pose information.

[0049] Both the first and second grasping pose information include translation and rotation information of the robot's end effector. Step S140 specifically includes: based on each second grasping pose information, determining the translation deviation of the corresponding second grasping pose information according to the translation information in the corresponding second grasping pose information and the translation information in the first grasping pose information; and determining the rotation deviation of the corresponding second grasping pose information according to the rotation information in the corresponding second grasping pose information and the rotation information in the first grasping pose information.

[0050] Translational deviations can be characterized using Euclidean distance.

[0051] Rotational deviation can be determined using the trace of the matrix, as detailed in the following formula.

[0052] in, rot err For rotational deviation, R To correspond to the rotation matrix in the second grasping pose information, R tgt The rotation matrix is ​​the first captured pose information. trace() Let be the trace of the matrix.

[0053] Step S150: Based on each second grasping pose information, determine the confidence level corresponding to the second grasping pose information according to the deviation between the corresponding second grasping pose information and the first grasping pose information.

[0054] The confidence level determination method for each second grasping pose information provided in this embodiment has high accuracy because the first grasping pose information is determined based on the human-taught robot. Thus, by using the first grasping pose information as a reference, the confidence level of each second grasping pose information determined has high accuracy, providing accurate data for subsequent retraining.

[0055] In some embodiments, step S150 specifically includes steps S151 to S152.

[0056] Step S151: Obtain the weighting coefficients corresponding to the translation deviation and the rotation deviation.

[0057] The weighting coefficients for translational deviation and rotational deviation are both preset values ​​that can be obtained directly.

[0058] Step S152: Based on each second grasping pose information, determine the confidence level corresponding to the corresponding second grasping pose information according to the corresponding translation deviation, the weight coefficient corresponding to the translation deviation, the corresponding rotation deviation, and the weight coefficient corresponding to the rotation deviation.

[0059] The confidence level corresponding to the second grasping pose information is determined based on the following formula.

[0060] in, err To correspond to the confidence level of the second grasp pose information, pos err To correspond to the translation deviation, a The weighting coefficients corresponding to the translation deviation. rot err To correspond to rotational deviation, b The weighting coefficients corresponding to the rotational deviation. a and b The sum of them equals 1.

[0061] After determining the confidence level corresponding to the second grasping pose information based on the above calculation formula, each confidence level is discretized into 10 levels: {0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1}. The higher the level, the lower the confidence level.

[0062] In some embodiments, before step S150, the method further includes: filtering translational deviations corresponding to each second grasping pose information according to a first preset threshold, and filtering rotational deviations corresponding to each second grasping pose information according to a second preset threshold, to obtain second grasping pose information corresponding to translational deviations less than the first preset threshold and rotational deviations less than the second preset threshold; determining the confidence level corresponding to the second grasping pose information corresponding to translational deviations less than the first preset threshold and rotational deviations less than the second preset threshold as a set confidence level. A set confidence level indicates that the probability of successfully grasping an object based on the second grasping pose information is 0. The set confidence level can be set according to requirements, for example, set to -1.

[0063] Both the first and second preset thresholds can be set according to requirements.

[0064] Each second grasping pose information is filtered based on a first preset threshold and a second preset threshold. This filters out second grasping pose information with a lower probability of successfully grasping the object, and further distinguishes them as training data for subsequent retraining. This ensures that the training data for subsequent retraining includes both second grasping pose information with a higher probability of successfully grasping the object and second grasping pose information with a lower probability of successfully grasping the object.

[0065] Step S160: Retrain the pre-trained grasping pose detection model based on the three-dimensional visual data of the target object, the second grasping pose information corresponding to each reference point, and the confidence level corresponding to each second grasping pose information.

[0066] 3D vision data can be either the RGB image and depth image of the target object, or the 3D point cloud data of the target object.

[0067] When the 3D data consists of RGB and depth images corresponding to the target object, the RGB and depth images are converted into corresponding 3D point cloud data.

[0068] When acquiring 3D visual data of the target object, maintain the diversity of the target object's pose, lighting direction, background texture, and camera shooting angle to ensure the scene generalization ability of the subsequent model.

[0069] In some embodiments, step S160 specifically includes steps S161 to S163.

[0070] Step S161: The three-dimensional visual data of the target object, the second grasping pose information corresponding to each reference point, and the confidence level corresponding to each second grasping pose information are used as training samples to retrain the pre-trained grasping pose detection model and obtain the prediction confidence level of the second grasping pose information corresponding to each reference point.

[0071] Step S162: Determine the loss function based on the prediction confidence of the second grasping pose information corresponding to each reference point and the confidence of the second grasping pose information corresponding to each reference point in the training samples.

[0072] The loss function is determined based on the difference between the prediction confidence of the second grasp pose information corresponding to each reference point and the confidence of the second grasp pose information corresponding to the corresponding reference point in the training samples. The larger the difference, the larger the loss function.

[0073] Step S163: If the loss function is less than the third preset threshold, stop retraining the pre-trained grasping pose detection model to obtain the retrained grasping pose detection model.

[0074] If the loss function is greater than or equal to the third preset threshold, adjust the retraining of the pre-trained grasping pose detection model, that is, continue to optimize the parameters of the pre-trained grasping pose detection model until the loss function is less than the third preset threshold.

[0075] The pre-trained grasping pose detection model is trained based on a large number of training samples. Each training sample includes 3D visual data of the object to be grasped and corresponding labeled data. Each labeled data includes grasping pose information and a corresponding confidence score. It should be noted that the confidence score in any of the above embodiments represents the probability of successfully grasping the object based on the corresponding grasping pose information. The higher the confidence score or the higher the level, the lower the probability of successfully grasping the object based on the corresponding grasping pose information.

[0076] One embodiment of the present invention provides a retraining device for a grasping pose detection model.

[0077] according to Figure 5 As shown, the retraining device for the grasping pose detection model includes a first grasping pose information acquisition module 510, a reference point determination module 520, a second grasping pose information determination module 530, a pose information deviation determination module 540, a confidence level determination module 550, and a retraining module 560.

[0078] The first grasping pose information acquisition module 510 is used to acquire the first grasping pose information corresponding to the target object grasped by the human teaching robot.

[0079] The reference point determination module 520 is used to determine multiple reference points located on the surface of the target object.

[0080] The second grasping pose information determination module 530 is used to determine multiple second grasping pose information based on each reference point.

[0081] The pose information deviation determination module 540 is used to determine the deviation between each second grasping pose information and the first grasping pose information based on each reference point.

[0082] The confidence determination module 550 is used to determine the confidence level of each second grasping pose information based on the deviation between the corresponding second grasping pose information and the first grasping pose information.

[0083] The retraining module 560 is used to retrain the pre-trained grasping pose detection model based on the three-dimensional visual data of the target object, the second grasping pose information corresponding to each reference point, and the confidence level corresponding to each second grasping pose information.

[0084] In some embodiments, the second grasping pose information determination module 530 is used to construct a target sphere with each reference point as the center; determine a first number of direction vectors, wherein each direction vector is determined based on the center of the sphere and a point on the surface of the target sphere; use each direction vector as the spin axis of the robot end effector, rotate based on the corresponding spin axis, and determine a second number of robot end effector rotation postures; determine a third number of robot end effector grasping depths based on each rotation posture; and determine corresponding second grasping pose information for each reference point based on each direction vector, each rotation posture, and each grasping depth.

[0085] In some embodiments, both the first grasping pose information and the second grasping pose information include translational and rotational information of the robot's end effector. The pose information deviation determination module 540 is used to determine the translational deviation of the corresponding second grasping pose information based on each second grasping pose information, according to the translational information in the corresponding second grasping pose information and the translational information in the first grasping pose information, and to determine the rotational deviation of the corresponding second grasping pose information based on the rotational information in the corresponding second grasping pose information and the rotational information in the first grasping pose information.

[0086] In some embodiments, the confidence determination module 550 is used to obtain the weight coefficients corresponding to the translation deviation and the rotation deviation; based on each second grasping pose information, the confidence level corresponding to the corresponding second grasping pose information is determined according to the corresponding translation deviation, the weight coefficients corresponding to the translation deviation, the corresponding rotation deviation, and the weight coefficients corresponding to the rotation deviation.

[0087] In some embodiments, the device further includes a filtering module. The filtering module is used to filter the translational deviation corresponding to each second grasping pose information according to a first preset threshold, and to filter the rotational deviation corresponding to each second grasping pose information according to a second preset threshold, to obtain second grasping pose information whose translational deviation is less than the first preset threshold and whose rotational deviation is less than the second preset threshold; the confidence level corresponding to the second grasping pose information whose translational deviation is less than the first preset threshold and whose rotational deviation is less than the second preset threshold is determined as a set confidence level.

[0088] In some embodiments, the retraining module 560 is used to retrain the pre-trained grasping pose detection model using the three-dimensional visual data of the target object, the second grasping pose information corresponding to each reference point, and the confidence level corresponding to each second grasping pose information as training samples, to obtain the predicted confidence level of the second grasping pose information corresponding to each reference point; determine the loss function based on the predicted confidence level of the second grasping pose information corresponding to each reference point and the confidence level of the second grasping pose information corresponding to each reference point in the training samples; and stop retraining the pre-trained grasping pose detection model when the loss function is less than a third preset threshold, to obtain the retrained grasping pose detection model.

[0089] The three-dimensional visual data involved in any of the above embodiments is either an RGB image and a depth image corresponding to the target object, or the three-dimensional visual data is three-dimensional point cloud data corresponding to the target object.

[0090] One embodiment of the present invention provides a computer device. According to... Figure 6 As shown, the computer device includes a memory 620 and a processor 610. The memory 620 stores a computer program that controls the processor 610 to operate and execute the methods provided according to any of the above embodiments.

[0091] The processor 610 is used to execute computer instructions, which can be written using instruction sets of architectures such as x86, Arm, RISC, MIPS, and SSE. The memory 620 includes, for example, ROM (Read-Only Memory), RAM (Random Access Memory), and non-volatile memory such as a hard disk, etc., and is not limited thereto.

[0092] One embodiment of the present invention provides a robot. The robot includes the computer device as provided in the above embodiments.

[0093] One embodiment of the present invention provides a computer program product. The computer program product includes a computer program that, when executed by a processor of a computer device, enables the computer device to perform the methods provided in any of the above embodiments.

[0094] This invention can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of the invention.

[0095] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example, but not limited to, electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0096] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0097] The computer program instructions used to perform the operations of this invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuits, such as programmable logic circuits, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), are personalized by utilizing state information from computer-readable program instructions. These electronic circuits can execute computer-readable program instructions to implement various aspects of the present invention.

[0098] Various aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0099] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0100] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0101] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions. It will be known to those skilled in the art that implementation in hardware, implementation in software, and implementation using a combination of software and hardware are equivalent.

[0102] The various embodiments of the present invention have been described above. These descriptions are exemplary and not exhaustive, and are not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein. The scope of the invention is defined by the appended claims.

Claims

1. A method for retraining a grasping pose detection model, characterized in that, include: Obtain the first grasping pose information corresponding to the target object grasped by the human-taught robot; Determine multiple reference points located on the surface of the target object; Based on each reference point, multiple second grasping pose information are determined; Based on each reference point, the deviation between each second grasping pose information and the first grasping pose information is determined; Based on each second grasping pose information, the confidence level corresponding to the second grasping pose information is determined according to the deviation between the corresponding second grasping pose information and the first grasping pose information. Based on the three-dimensional visual data of the target object, the second grasping pose information corresponding to each reference point, and the confidence level corresponding to each second grasping pose information, the pre-trained grasping pose detection model is retrained.

2. The method according to claim 1, characterized in that, The determination of multiple second grasping pose information based on each reference point includes: Construct the target sphere using each reference point as its center; Determine a first number of direction vectors, wherein each direction vector is determined based on a point on the surface of the target sphere and the center of the sphere. Using each direction vector as the spin axis of the robot end effector, and rotating based on the corresponding spin axis, the rotational attitude of the second number of robot end effectors is determined. Based on each rotational posture, determine the grasping depth of the third number of robot end effectors; For each reference point, the corresponding second grasping pose information is determined based on each direction vector, each rotational posture, and each grasping depth.

3. The method according to claim 1, characterized in that, Both the first grasping pose information and the second grasping pose information include translational and rotational information of the robot's end effector, wherein, The step of determining the deviation between each second grasping pose information and the first grasping pose information based on each reference point includes: Based on each second grasping pose information, the translation deviation of the corresponding second grasping pose information is determined according to the translation information in the corresponding second grasping pose information and the translation information in the first grasping pose information, and the rotation deviation of the corresponding second grasping pose information is determined according to the rotation information in the corresponding second grasping pose information and the rotation information in the first grasping pose information.

4. The method according to claim 3, characterized in that, The step of determining the confidence level corresponding to each second grasping pose information based on the deviation between the corresponding second grasping pose information and the first grasping pose information includes: Obtain the weighting coefficients corresponding to translational deviation and rotational deviation; Based on each second grasping pose information, the confidence level corresponding to the second grasping pose information is determined according to the corresponding translation deviation, the weight coefficient corresponding to the translation deviation, the corresponding rotation deviation, and the weight coefficient corresponding to the rotation deviation.

5. The method according to claim 4, characterized in that, Before determining the confidence level corresponding to the second grasping pose information based on the corresponding translation deviation, the weighting coefficient corresponding to the translation deviation, the corresponding rotation deviation, and the weighting coefficient corresponding to the rotation deviation, the method further includes: The translational deviation corresponding to each second grasping pose information is filtered according to the first preset threshold, and the rotational deviation corresponding to each second grasping pose information is filtered according to the second preset threshold, so as to obtain the second grasping pose information corresponding to the translational deviation being less than the first preset threshold and the rotational deviation being less than the second preset threshold. The confidence level corresponding to the second grasping pose information where the translation deviation is less than the first preset threshold and the rotation deviation is less than the second preset threshold is determined as the set confidence level.

6. The method according to claim 1, characterized in that, The step of retraining the pre-trained grasping pose detection model based on the three-dimensional visual data of the target object, the second grasping pose information corresponding to each reference point, and the confidence level corresponding to each second grasping pose information includes: The three-dimensional visual data of the target object, the second grasping pose information corresponding to each reference point, and the confidence level corresponding to each second grasping pose information are used as training samples to retrain the pre-trained grasping pose detection model and obtain the prediction confidence level of the second grasping pose information corresponding to each reference point. The loss function is determined based on the prediction confidence of the second grasping pose information corresponding to each reference point and the confidence of the second grasping pose information corresponding to each reference point in the training sample. If the loss function is less than a third preset threshold, the retraining of the pre-trained grasping pose detection model is stopped, and a retrained grasping pose detection model is obtained.

7. The method according to any one of claims 1-6, characterized in that, The three-dimensional visual data is either the RGB image and depth image corresponding to the target object, or the three-dimensional visual data is the three-dimensional point cloud data corresponding to the target object.

8. A computer device, characterized in that, It includes a memory and a processor, the memory storing a computer program for controlling the processor to operate in order to perform the method according to any one of claims 1 to 7.

9. A robot, characterized in that, Includes the computer device as described in claim 8.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor of a computer device, enables the computer device to perform the method as described in any one of claims 1 to 7.

Images