A posture labeling method, device, system and storage medium

Abstract

The embodiment of the present invention discloses a posture annotation method, device, system and storage medium. The acquired sequence frame images include a reference object and a target object, and the sequence frame images include a reference frame image and a current frame image. Based on the feature points of the reference object, the projection points in the reference frame image and the projection points in the current frame image, a first posture conversion relationship is obtained; based on the feature points of the target object, the projection points in the reference frame image and the first posture conversion relationship, a second posture conversion relationship is determined. The feature points of the reference object and the feature points of the target object are stable and reliable, and the posture conversion matrix can be accurately calculated; based on the calculated second posture conversion relationship and the projection points of the target object in any frame image, the posture of the feature points of the target object is automatically determined and annotated without manual annotation; when the target object is positioned and analyzed based on the posture of the target object, for example, when a robot is used to grasp the target object, the grasping efficiency and accuracy can be improved.

Description

Technical Field

[0001] Embodiments of the present invention relate to robot intelligent detection technology, and in particular, to a posture labeling method, device, system and storage medium. Background Art

[0002] The posture of the target object is an important technology often used in the field of robot intelligent detection. At present, manual labeling methods are often used to label the posture data of the target object. During the operation of the robot, a large amount of coordinate data of the target object needs to be collected. For example, a large amount of coordinate data of the target object is collected by using visual servo business processes such as robot recognition of target objects and grasping target objects. Manually labeling the posture of the target object based on massive coordinate data consumes a lot of manpower and time costs, and the labeling accuracy of the posture data is poor. Summary of the invention

[0003] Embodiments of the present invention provide a posture annotation method, device, system and storage medium to achieve the effect of improving the annotation efficiency and accuracy of posture.

[0004] In a first aspect, an embodiment of the present invention provides a posture annotation method, the method comprising:

[0005] Receiving a sequence of frame images captured by a shooting device at different angles, and determining a reference frame image and a current frame image in the sequence of frame images, wherein each frame image in the sequence of frame images includes a reference object and a target object;

[0006] Determine a first pose transformation relationship between a current frame image coordinate system and a reference frame image coordinate system based on feature points of the reference object, projection points of the feature points of the reference object in the reference frame image, and projection points of the feature points of the reference object in the current frame image;

[0007] Determine a second posture conversion relationship between a target object coordinate system where the target object is located and a current frame image coordinate system based on feature points of the target object, projection points of the feature points of the target object in the reference frame image, and the first posture conversion relationship;

[0008] Based on the second posture conversion relationship and the projection point of the target object in any frame image taken by the shooting device, the posture of the target object is marked.

[0009] In a second aspect, an embodiment of the present invention further provides a posture annotation device, the device comprising:

[0010] An image determination module is used to receive a sequence of frame images captured by a shooting device at different angles, and determine a reference frame image and a current frame image in the sequence of frame images, wherein each frame image in the sequence of frame images includes a reference object and a target object;

[0011] A first pose conversion relationship determination module is used to determine a first pose conversion relationship between a current frame image coordinate system and a reference frame image coordinate system based on feature points of the reference object, projection points of the feature points of the reference object in the reference frame image, and projection points of the feature points of the reference object in the current frame image;

[0012] A second posture conversion relationship determination module is used to determine a second posture conversion relationship between a target object coordinate system where the target object is located and a current frame image coordinate system based on feature points of the target object, projection points of the feature points of the target object in the reference frame image, and the first posture conversion relationship;

[0013] The posture annotation module is used to annotate the posture of the target object based on the second posture conversion relationship and the projection point of the target object in any frame image taken by the shooting device.

[0014] In a third aspect, an embodiment of the present invention further provides a posture annotation system, comprising a robot, a target object and a reference object, wherein the robot comprises a mechanical arm, a shooting device, a memory and a processor; the mechanical arm drives the shooting device to move so that the shooting device collects sequence frame images at multiple angles; wherein each frame image of the sequence frame image comprises a reference object and a target object, the target object comprises a plurality of feature points, and the reference object comprises a plurality of feature points;

[0015] Among them, a computer program exists in the memory and can be run on the processor, and when the processor executes the computer program, the posture labeling method described in any one of the first aspects is implemented.

[0016] In a fourth aspect, an embodiment of the present invention further provides a storage medium comprising computer executable instructions, wherein the computer executable instructions, when executed by a computer processor, implement the pose labeling method as described in any one of the first aspects.

[0017] The technical solution of the embodiment of the present invention receives a sequence of frame images captured by a shooting device at different angles, each frame of the sequence of frame images includes a reference object and a target object, so that the reference object assists the shooting device in shooting; determines the reference frame image and the current frame image in the sequence of frame images, and determines the first pose transformation relationship between the current frame image coordinate system and the reference frame image coordinate system based on the feature points of the reference object, the projection points of the feature points of the reference object in the reference frame image, and the projection points of the feature points of the reference object in the current frame image; determines the second pose transformation relationship between the target object coordinate system where the target object is located and the current frame image coordinate system based on the feature points of the target object, the projection points of the feature points of the target object in the reference frame image, and the first pose transformation relationship. Since the feature points of the reference object and the feature points of the target object are stable and reliable, the first pose transformation matrix and the second pose transformation matrix can be accurately calculated; when any frame image taken by the shooting device is obtained, the pose of the feature points of the target object is automatically determined based on the calculated second pose transformation relationship and the projection point of the target object in any frame image, and the pose of the feature points of the target object is automatically marked without manual marking, thereby reducing labor costs and improving the reliability of pose calculation; further, when the target object is positioned and analyzed based on the pose of the target object, the positioning accuracy can be improved, especially when it is applied in the field of robot grasping target objects, the grasping efficiency and grasping accuracy of the target object can be greatly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 A schematic diagram of a flow chart of a posture labeling method provided in Embodiment 1 of the present invention;

[0019] Figure 2 A schematic diagram of a flow chart of a posture labeling method provided in Embodiment 2 of the present invention;

[0020] Figure 3 A schematic diagram of the posture conversion relationship between various coordinate systems provided in the second embodiment of the present invention;

[0021] Figure 4 A logical schematic diagram of posture annotation provided in the second embodiment of the present invention;

[0022] Figure 5 A schematic diagram of the structure of a posture annotation device provided in Embodiment 3 of the present invention;

[0023] Figure 6 A structural diagram of a posture annotation system provided in Embodiment 4 of the present invention. DETAILED DESCRIPTION

[0024] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0025] Embodiment 1

[0026] Figure 1 This is a flow chart of a posture annotation method provided in the first embodiment of the present invention. This embodiment is applicable to the case of automatic posture annotation. The method can be executed by a posture annotation device, wherein the system can be implemented by software and / or hardware and is generally integrated in a robot or an electronic device with posture annotation function. This embodiment is explained by taking a robot as an example. Figure 1 As shown, the method may include the following steps:

[0027] S110, receiving a sequence of frame images captured by a shooting device at different angles, and determining a reference frame image and a current frame image in the sequence of frame images.

[0028] The shooting device is installed on the robot arm and is used to collect sequence frame images during the operation of the robot. The shooting device can be a camera, a camera head or a laser, or other equipment with image acquisition function. It should be noted that when the shooting device is used to collect sequence frame images, the robot arm drives the shooting device to move to different shooting points according to the pre-planned path, so that the shooting device can collect sequence frame images at different angles.

[0029] In one possible embodiment, the pre-planned path can be determined based on the robot's operating path, the length of the robot arm, the installation position of the camera, and / or the distance the robot moves between adjacent operating points. In another possible embodiment, the path can be determined based on the coordinate data of multiple feature points in the target object to be marked and / or the coordinate data of multiple feature points in the reference object. It should be noted that the planning method of the robot arm's path is not limited to the above two methods, and can also be generated based on other methods.

[0030] Among them, the sequence frame image can be understood as a frame sequence composed of several images in time. Each frame of the sequence frame image includes a reference object and a target object. The target object can be understood as an object to be grasped or identified, which can be a regular object or an irregular object. For example, in the field of construction, when a robot is used to lay wall tiles or wallpaper, the target object can be a wall tile or wallpaper to be grasped; in the field of logistics and transportation, a robot is used to classify express delivery, and the target object can be an express delivery to be identified. The target object can include one or more stable feature points.

[0031] It should be noted that the reference object is generally located in the world coordinate system and is stable and motionless, and includes one or more stable feature points on the reference object, which are used to assist the shooting device in collecting sequence frame images at different angles, so that each frame of the collected image includes the reference object and the target object. Optionally, the reference object is a QR code or a barcode.

[0032] It should be noted that before acquiring the sequence frame images, at least one feature point of the reference object and the target object is determined respectively, so that each frame of the acquired sequence frame images includes the projection point of the reference object and the projection point of the target object. In an optional embodiment, the feature points of the reference object can be determined based on the path pre-planned by the robot. In another optional embodiment, the feature points of the reference object can be randomly distributed or determined based on other rules. The feature points of the target object can include the vertices of the target object, the center point of each edge, the center point of each face, and can also include other feature points, such as the one-third point of each edge, the non-center point of each face, etc.

[0033] The reference frame and the current frame can be any two frames selected from the sequence of frame images. As can be seen from the above description, both the current frame image and the reference frame image include a reference object and a target object. The current frame image includes the projection points of the reference object and the projection points of the target object, and the reference frame image includes the projection points of the reference object and the projection points of the target object.

[0034] S120, determining a first pose transformation relationship between a current frame image coordinate system and a reference frame image coordinate system based on feature points of the reference object, projection points of the feature points of the reference object in the reference frame image, and projection points of the feature points of the reference object in the current frame image.

[0035] Before calculating the first pose transformation relationship, the coordinate system of the reference object, that is, the world coordinate system, is determined, the coordinate system of the reference frame image is defined as the reference frame image coordinate system, and the coordinate system of the current frame image is defined as the current frame image coordinate system.

[0036] Optionally, the method for determining the first pose transformation relationship includes: determining a third pose transformation relationship between a world coordinate system where the reference object is located and a reference frame image coordinate system based on feature points of the reference object and projection points of the feature points of the reference object in the reference frame image; determining a fourth pose transformation relationship between a world coordinate system and a current frame image coordinate system based on feature points of the reference object and projection points of the feature points of the reference object in the current frame image; and determining the first pose transformation relationship based on the third pose transformation relationship and the fourth pose transformation relationship.

[0037] It should be noted that the projection point of the reference object refers to the projection of the feature point of the reference object in the image captured by the shooting device, that is, the feature point of the reference object corresponds to the projection point of each frame image in the sequence of frame images.

[0038] Specifically, the method for determining the third posture transformation relationship includes: obtaining the posture of the feature point of the reference object in the world coordinate system and the first coordinate data of the projection point of the reference object in the reference frame image coordinate system; determining the third posture transformation relationship between the world coordinate system and the reference frame image coordinate system according to the posture and the first coordinate data of the feature point of the reference object in the world coordinate system.

[0039] Specifically, the method for determining the fourth posture transformation relationship includes: obtaining the posture of the feature point of the reference object in the world coordinate system and the second coordinate data of the projection point of the reference object in the current frame image coordinate system; determining the fourth posture transformation relationship between the world coordinate system and the current frame image coordinate system according to the posture and the second coordinate data of the feature point of the reference object in the world coordinate system.

[0040] It should be noted that the above-mentioned posture refers to the six-degree-of-freedom posture of the feature point of the reference object in the world coordinate system, and the six-degree-of-freedom posture may include the movement freedom of the feature point of the reference object along the three coordinate axes of x, y, and z of the world coordinate system and the rotation freedom of the feature point of the reference object around these three coordinate axes. The first coordinate data refers to the two-dimensional coordinates of the projection point of the reference object in the reference frame image coordinate system, and the second coordinate data refers to the two-dimensional coordinates of the projection point of the reference object in the current frame image coordinate system. The third posture conversion relationship can be understood as the conversion relationship between the world coordinate system and the reference frame image coordinate system, and the fourth posture conversion relationship can be understood as the conversion relationship between the world coordinate system and the current frame image coordinate system. Therefore, the world coordinate system is used as a conversion reference, and the first posture conversion relationship between the reference frame image coordinate system and the current frame image coordinate system is determined according to the third posture conversion relationship and the fourth posture conversion relationship.

[0041] S130, determining a second pose transformation relationship between a target object coordinate system where the target object is located and a current frame image coordinate system based on feature points of the target object, projection points of the feature points of the target object in the reference frame image, and the first pose transformation relationship.

[0042] Before calculating the second posture transformation relationship, the coordinate system where the target object is located is defined as the target object coordinate system. Optionally, the method for determining the second posture transformation relationship includes: determining a fifth posture transformation relationship between the target object coordinate system and the reference frame image coordinate system based on the feature points of the target object and the projection points of the feature points of the target object in the reference frame image; and determining the second posture transformation relationship based on the first posture transformation relationship and the fifth posture transformation relationship.

[0043] Specifically, the method for determining the fifth posture transformation relationship includes: obtaining the posture of the feature points of the target object in the target object coordinate system and the third coordinate data of the projection point of the target object in the reference frame coordinate system, and calculating the fifth posture transformation relationship between the target object coordinate system and the reference frame coordinate system based on the posture and the third coordinate data of the feature points of the target object in the target object coordinate system.

[0044] It should be noted that the above-mentioned posture refers to the six-degree-of-freedom posture of the feature point of the target object in the target object coordinate system, and the six-degree-of-freedom posture may include the movement freedom of the feature point of the target object along the three coordinate axes x, y, and z of the target object coordinate system and the rotation freedom around these three coordinate axes. The third coordinate data refers to the two-dimensional coordinates of the projection point of the target object in the reference frame image coordinate system. The fifth posture transformation relationship can be understood as the transformation relationship between the reference frame coordinate system and the target object coordinate system, and the first posture transformation relationship can be understood as the transformation relationship between the reference frame image coordinate system and the current frame image coordinate system. Therefore, the reference frame image coordinate system is used as a transformation reference, and the second posture transformation relationship between the target object coordinate system and the current frame image coordinate system is determined according to the first posture transformation relationship and the fifth posture transformation relationship.

[0045] S140: Based on the second posture conversion relationship and the projection point of the target object in any frame image captured by the shooting device, perform posture annotation on the target object.

[0046] Among them, any frame image captured by the shooting device is located in the current frame image coordinate system. In the current frame image coordinate system, the coordinate data of the projection point of the target object in the image captured by the shooting device is determined, and the position and posture of the feature point of the target object corresponding to the projection point of the target object in the target object coordinate system is determined according to the second position and posture conversion matrix and the coordinate data of the projection point of the target object in any frame image captured.

[0047] It should be noted that the coordinate data of the projection point of the target object in the captured image refers to the two-dimensional coordinates of the projection point of the target object in the current frame image coordinate system, and the posture refers to the six-degree-of-freedom posture of the feature point of the target object in the target object coordinate system. Therefore, based on the coordinate data of the projection point of the target object in the captured image and the second posture conversion matrix, the six-degree-of-freedom posture of the feature point of the target object in the target object coordinate system can be determined, and the posture of the feature point of the target object can be automatically marked.

[0048] The technical solution provided by this embodiment receives a sequence of frame images captured by a shooting device at different angles, each frame of the sequence of frame images includes a reference object and a target object, so that the reference object assists the shooting device in shooting; determines the reference frame image and the current frame image in the sequence of frame images, and determines the first pose transformation relationship between the current frame image coordinate system and the reference frame image coordinate system based on the feature points of the reference object, the projection points of the feature points of the reference object in the reference frame image, and the projection points of the feature points of the reference object in the current frame image; determines the second pose transformation relationship between the target object coordinate system where the target object is located and the current frame image coordinate system based on the feature points of the target object, the projection points of the feature points of the target object in the reference frame image, and the first pose transformation relationship. Since the feature points of the reference object and the feature points of the target object are stable and reliable, the first pose transformation matrix and the second pose transformation matrix can be accurately calculated; when any frame image taken by the shooting device is obtained, the pose of the feature points of the target object is automatically determined based on the calculated second pose transformation relationship and the projection point of the target object in any frame image, and the pose of the feature points of the target object is automatically marked without manual marking, thereby reducing labor costs and improving the reliability of pose calculation; further, when the target object is positioned and analyzed based on the pose of the target object, the positioning accuracy can be improved, especially when it is applied in the field of robot grasping target objects, the grasping efficiency and grasping accuracy of the target object can be greatly improved.

[0049] Embodiment 2

[0050] Figure 2 A flow chart of a posture annotation method provided in the second embodiment of the present invention. The technical solution of this embodiment is refined on the basis of the above embodiment. The method for determining the transformation relationship of each posture and the posture annotation method are specifically refined. For the parts not fully described in this method embodiment, please refer to the above embodiment. For details, please refer to Figure 2 As shown, the method may include the following steps:

[0051] S210: Receive a sequence of frame images captured by a shooting device at different angles, and determine a reference frame image and a current frame image in the sequence of frame images.

[0052] S220, determining a third posture conversion relationship between a world coordinate system where the reference object is located and a reference frame image coordinate system based on feature points of the reference object and projection points of feature points of the reference object in the reference frame image.

[0053] Optionally, a method for determining a third pose transformation relationship includes: obtaining an intrinsic parameter matrix pre-calibrated by a shooting device; calculating a first relationship equation based on the intrinsic parameter matrix, three-dimensional feature points of a reference object, two-dimensional projection points of the feature points of the reference object in a reference frame image, and a current pose transformation relationship between a world coordinate system and a reference frame image coordinate system; iteratively solving the first relationship equation, and if the current pose transformation relationship at the current number of iterations converges, using the current pose transformation relationship at the current number of iterations as the third pose transformation relationship.

[0054] It should be noted that the intrinsic parameter matrix can be determined according to the parameters in the manual of the shooting device, or according to the accuracy of the pose annotation, or by other methods. Specifically, extract N two-dimensional projection points of the reference object on the reference frame image, and the i-th two-dimensional projection point is recorded as The three-dimensional feature point of the reference object in the world coordinate system corresponding to the i-th two-dimensional projection point is recorded as The current pose transformation relationship between the world coordinate system and the reference frame image coordinate system is recorded as T_w2r, and the expression of the first relationship equation is:

[0055]

[0056] Furthermore, a nonlinear optimization method is used to solve Formula 1. The initial value of T_w2r is solved as T0 under the RANSAC (RANdom Sample Consensus) framework, and the LM (Levenberg-Marquardt) algorithm is used to iteratively solve the first relationship equation, that is, T_w2r is optimized. If the current posture transformation relationship under the current number of iterations reaches a stable state, the current posture transformation relationship under the current number of iterations is used as the third posture transformation relationship.

[0057] Optionally, the calculation formula for optimizing T_w2r using the LM algorithm is:

[0058]

[0059] Among them, x k is the current posture transformation relationship T_w2r at the kth iteration. If k = 0, That is, Δx k is the change in the current posture transformation relationship T_w2r between the kth and k-1th generations, or Δx k is the optimization variable; m is the radius of the confidence interval.

[0060] in, ρ is an evaluation index. If ρ>0.75, m=2m, if ρ<0.25, m=0.5m. And if ρ is greater than the first threshold, it is considered that Δx can be solved based on formula 2 k , let x k =x k -Δx k-1 , solve Δx according to formula 2 k , that is, according to formula 2, the change of the current posture conversion relationship T_w2r under the kth and k-1th iterations is solved. If the change is greater than or equal to the second threshold, it is determined that the current posture conversion relationship under the current iteration has not converged. Let x k =x k+1 -Δx k , recalculate ρ, and solve the change of the current posture transformation relationship T_w2r at the k+1th and kth generations based on formula 2, and determine whether the current posture transformation relationship at the current iteration number (k+1th) converges based on the newly determined change. If not, continue the iterative calculation until the current posture transformation relationship at the current iteration number converges, and then convert the current posture transformation relationship at the current iteration number into the current posture transformation relationship.

[0061] S230: Determine a fourth posture conversion relationship between the world coordinate system and the current frame image coordinate system based on the feature points of the reference object and the projection points of the feature points of the reference object in the current frame image.

[0062] Optionally, the method for determining the fourth pose transformation relationship includes: obtaining an internal parameter matrix pre-calibrated by the shooting device; calculating a second relationship equation based on the internal parameter matrix, the three-dimensional feature points of the reference object, the two-dimensional projection points of the feature points of the reference object in the current frame image, and the current pose transformation relationship between the world coordinate system and the current frame image coordinate system; iteratively solving the second relationship equation, and if the current pose transformation relationship at the current iteration number converges, using the current pose transformation relationship at the current iteration number as the fourth pose transformation relationship.

[0063] Similar to the previous steps, the intrinsic parameter matrix can be determined according to the parameters in the manual of the shooting device, or according to the accuracy of the pose annotation, or by other methods. Specifically, N two-dimensional projection points of the reference object are extracted on the current frame image, and the i-th two-dimensional projection point is recorded as The three-dimensional feature point of the reference object in the world coordinate system corresponding to the i-th two-dimensional projection point is recorded as The current posture transformation relationship between the world coordinate system and the current frame image coordinate system is recorded as T_w2c, and the expression of the second relationship equation is:

[0064]

[0065] Similar to the previous steps, a nonlinear optimization method is used to solve Formula 3. The initial value of T_w2c is solved as T0 under the RANSAC (RANdomSampleConsensus) framework, and the LM (Levenberg-Marquardt) algorithm is used to iteratively solve the second relationship equation, that is, to optimize T_w2c. If the current posture transformation relationship under the current iteration number reaches a stable state, the current posture transformation relationship under the current iteration number is used as the fourth posture transformation relationship. It should be noted that the calculation formula of the LM algorithm to optimize T_w2c is consistent with Formula 2. In this step, x in the LM algorithm k is the current posture conversion relationship T_w2c at the kth iteration number. The second relationship equation is iteratively solved using formula 2. If the current posture conversion relationship at the current iteration number converges, the current posture conversion relationship at the current iteration number is used as the fourth posture conversion relationship.

[0066] S240: Determine the first posture conversion relationship according to the third posture conversion relationship and the fourth posture conversion relationship.

[0067] Optionally, the third posture transformation relationship and the fourth posture transformation relationship are vector-multiplied to obtain the first posture transformation relationship.

[0068] like Figure 3 The figure shows the relationship between the pose transformations of the coordinate systems. Figure 3 The reference object in is the QR code, and the coordinate system where the QR code is located is the world coordinate system. The third pose conversion relationship between the world coordinate system and the reference frame image coordinate system is T_w2r, the fourth pose conversion relationship between the world coordinate system and the current frame image coordinate system is T_w2c, and the first pose conversion relationship between the current frame image coordinate system and the reference frame image coordinate system is T_c2r=T_w2r*T_w2c -1 .

[0069] S250, determining a fifth posture conversion relationship between the target object coordinate system and the reference frame image coordinate system according to the feature points of the target object and the projection points of the feature points of the target object in the reference frame image.

[0070] Optionally, the method for determining the fifth pose transformation relationship includes: obtaining an intrinsic parameter matrix pre-calibrated by a shooting device; calculating a third relationship equation based on the intrinsic parameter matrix, three-dimensional feature points of the target object, and two-dimensional projection points of the feature points of the target object in the reference frame image; iteratively solving the third relationship equation, and if the current pose transformation relationship at the current iteration number converges, using the current pose transformation relationship at the current iteration number as the fifth pose transformation relationship.

[0071] Specifically, M two-dimensional projection points of the target object are extracted on the reference frame image, and the j-th two-dimensional projection point is recorded as The three-dimensional feature point of the target object in the target object coordinate system corresponding to the j-th two-dimensional projection point is recorded as The current pose transformation relationship between the world coordinate system and the reference frame image coordinate system is recorded as T_o2r, and the expression of the third relationship equation is:

[0072]

[0073] Similar to the previous steps, a nonlinear optimization method is used to solve Formula 4. The initial value of T_o2r is solved as T0 under the RANSAC (RANdomSampleConsensus) framework, and the LM (Levenberg-Marquardt) algorithm is used to iteratively solve the third relationship equation, that is, to optimize T_o2r. If the current posture transformation relationship under the current iteration number reaches a stable state, the current posture transformation relationship under the current iteration number is used as the fifth posture transformation relationship. It should be noted that the calculation formula of the LM algorithm to optimize T_o2r is consistent with Formula 2. In this step, x in the LM algorithm k is the current posture conversion relationship T_o2r at the kth iteration number. The third relationship equation is iteratively solved using Formula 2. If the current posture conversion relationship at the current iteration number converges, the current posture conversion relationship at the current iteration number is used as the fifth posture conversion relationship.

[0074] S260: Determine a second posture conversion relationship based on the first posture conversion relationship and the fifth posture conversion relationship.

[0075] Optionally, determining the second pose transformation relationship based on the first pose transformation relationship and the fifth pose transformation relationship includes: performing vector multiplication on the first pose transformation relationship and the fifth pose transformation relationship to obtain a current pose transformation relationship between the target object coordinate system and the current frame image coordinate system; calculating a fourth relationship equation based on an intrinsic parameter matrix obtained by pre-calibration of the camera, three-dimensional feature points of the target object, two-dimensional projection points of the feature points of the target object in the current frame image, and the current pose transformation relationship between the target object coordinate system and the current frame image coordinate system; iteratively solving the fourth relationship equation, and if the current pose transformation relationship at the current iteration number converges, using the current pose transformation relationship at the current iteration number as the second pose transformation relationship.

[0076] Combination Figure 3, the fifth pose conversion relationship between the target object coordinate system and the reference frame image coordinate system is T_o2r, the first pose conversion relationship between the reference frame image coordinate system and the current frame image coordinate system is T_c2r, then the second pose conversion relationship between the current frame image coordinate system and the target object coordinate system is T_o2c=T_o2r*T_c2r -1 .

[0077] Specifically, M two-dimensional projection points of the target object are extracted on the current frame image, and the j-th two-dimensional projection point is recorded as The three-dimensional feature point of the target object in the target object coordinate system corresponding to the j-th two-dimensional projection point is recorded as The current posture conversion relationship between the world coordinate system and the current frame image coordinate system is recorded as T_o2c, and the expression of the fourth relationship equation is:

[0078]

[0079] The expression of the fourth relationship equation can also be in the following form:

[0080]

[0081] Similar to the previous steps, a nonlinear optimization method is used to solve Formula 5 or Formula 6, and the initial value of T_o2c is solved as T0 under the RANSAC (RANdomSample Consensus) framework, and the LM (Levenberg-Marquardt) algorithm is used to iteratively solve the third relationship equation, that is, to optimize T_o2c. If the current posture transformation relationship under the current iteration number reaches a stable state, the current posture transformation relationship under the current iteration number is used as the fifth posture transformation relationship. It should be noted that the calculation formula of the LM algorithm to optimize T_o2c is consistent with Formula 2. In this step, x in the LM algorithm k is the current posture transformation relationship T_o2c at the kth iteration number. The fourth relationship equation is iteratively solved using Formula 2. If the current posture transformation relationship at the current iteration number converges, the current posture transformation relationship at the current iteration number is used as the second posture transformation relationship.

[0082] S270, based on the second posture conversion relationship and the projection point of the target object in any frame image captured by the shooting device, annotate the posture of the target object. Optionally, based on the second posture conversion relationship and the projection point of the target object in any frame image captured by the shooting device, annotate the posture of the target object, including: according to the projection point of the target object in any frame image captured by the shooting device, the second posture conversion relationship and the intrinsic parameter matrix pre-calibrated by the shooting device, obtain the posture of the feature points of the target object in the image captured by the shooting device, and annotate the posture of the feature points of the target object.

[0083] Specifically, any frame image captured by the camera is located in the current frame image coordinate system. In the current frame image coordinate system, the projection point of the target object in the captured image is multiplied by the projection point of the target object in any frame image captured, the second posture conversion relationship, and the intrinsic parameter matrix pre-calibrated by the camera to obtain the posture of the feature point of the target object in the image captured by the camera, that is, the six-degree-of-freedom posture of the feature point of the target object in the target object coordinate system, and the posture of the target object is annotated.

[0084] like Figure 4 The following is a logical diagram of pose annotation. Figure 4 The above process is explained as a whole. The path of the robotic arm is determined in advance and the internal parameter matrix of the shooting device is obtained. The robotic arm drives the shooting device to collect sequence frame images according to the pre-planned path, and the reference frame image and the current frame are selected from the collected sequence frame images; the current frame image and the sequence frame image include the target object and the reference object. According to the projection points of the target object in the current frame image and the sequence frame image and the projection points of the reference object, as well as the feature points of the target object and the feature points of the reference object, the RANSAC framework and the LM algorithm are used to calculate the pose conversion between the target object coordinate system and the current frame image coordinate system; when the shooting device captures a new image, the six-degree-of-freedom pose of the target object is determined according to the projection points, the internal parameter matrix and the pose relationship in the newly captured image, until the shooting device has captured all images and completed the pose annotation.

[0085] The technical solution provided in this embodiment accurately calculates the posture transformation relationship between the target object coordinate system and the current frame image coordinate system based on the intrinsic parameter matrix, the feature points of the reference object, the feature points of the target object and the projection points in the sequence frame images; further based on the projection points of the target object in any frame image captured by the shooting device, according to the intrinsic parameter matrix of the shooting device and the posture transformation, the six-degree-of-freedom posture of the feature points of the target object is accurately calculated and labeled without manual labeling, thereby reducing labor costs; further, when the target object is positioned and analyzed based on the posture of the target object, the positioning accuracy can be improved, especially when applied to the field of robot grasping target objects, the grasping efficiency and grasping accuracy of the target object can be greatly improved.

[0086] Embodiment 3

[0087] Figure 5 This is a schematic diagram of the structure of a posture annotation device provided by Embodiment 3 of the present invention. Figure 5 As shown, the system includes: an image determination module 310 , a first pose conversion relationship determination module 310 , a second pose conversion relationship determination module 330 , and a pose annotation module 340 .

[0088] The image determination module 310 is used to receive a sequence of frame images captured by a shooting device at different angles, and determine a reference frame image and a current frame image in the sequence of frame images, wherein each frame image in the sequence of frame images includes a reference object and a target object;

[0089] A first pose conversion relationship determination module 320 is used to determine a first pose conversion relationship between a current frame image coordinate system and a reference frame image coordinate system based on feature points of the reference object, projection points of the feature points of the reference object in the reference frame image, and projection points of the feature points of the reference object in the current frame image;

[0090] A second posture conversion relationship determination module 330 is used to determine a second posture conversion relationship between a target object coordinate system where the target object is located and a current frame image coordinate system based on feature points of the target object, projection points of the feature points of the target object in the reference frame image, and the first posture conversion relationship;

[0091] The posture annotation module 340 is used to annotate the posture of the target object based on the second posture conversion relationship and the projection point of the target object in any frame image captured by the shooting device.

[0092] The technical solution provided by this embodiment receives a sequence of frame images captured by a shooting device at different angles, each frame of the sequence of frame images includes a reference object and a target object, so that the reference object assists the shooting device in shooting; determines the reference frame image and the current frame image in the sequence of frame images, and determines the first pose transformation relationship between the current frame image coordinate system and the reference frame image coordinate system based on the feature points of the reference object, the projection points of the feature points of the reference object in the reference frame image, and the projection points of the feature points of the reference object in the current frame image; determines the second pose transformation relationship between the target object coordinate system where the target object is located and the current frame image coordinate system based on the feature points of the target object, the projection points of the feature points of the target object in the reference frame image, and the first pose transformation relationship. Since the feature points of the reference object and the feature points of the target object are stable and reliable, the first pose transformation matrix and the second pose transformation matrix can be accurately calculated; when any frame image taken by the shooting device is obtained, the pose of the feature points of the target object is automatically determined based on the calculated second pose transformation relationship and the projection point of the target object in any frame image, and the pose of the feature points of the target object is automatically marked without manual marking, thereby reducing labor costs and improving the reliability of pose calculation; further, when the target object is positioned and analyzed based on the pose of the target object, the positioning accuracy can be improved, especially when it is applied in the field of robot grasping target objects, the grasping efficiency and grasping accuracy of the target object can be greatly improved.

[0093] Optionally, the first pose conversion relationship determination module 320 is further used to determine a third pose conversion relationship between a world coordinate system where the reference object is located and a reference frame image coordinate system based on feature points of the reference object and projection points of the feature points of the reference object in the reference frame image;

[0094] Determining a fourth posture conversion relationship between the world coordinate system and the current frame image coordinate system based on the feature points of the reference object and the projection points of the feature points of the reference object in the current frame image;

[0095] The first posture conversion relationship is determined according to the third posture conversion relationship and the fourth posture conversion relationship.

[0096] Optionally, the first pose conversion relationship determination module 320 is further used to obtain an internal parameter matrix pre-calibrated by the shooting device;

[0097] Calculate a first relationship equation according to the intrinsic parameter matrix, the three-dimensional feature points of the reference object, the two-dimensional projection points of the feature points of the reference object in the reference frame image, and the current posture conversion relationship between the world coordinate system and the reference frame image coordinate system;

[0098] The first relationship equation is iteratively solved, and if the current posture conversion relationship at the current iteration number converges, the current posture conversion relationship at the current iteration number is used as the third posture conversion relationship.

[0099] Optionally, the first pose conversion relationship determination module 320 is further used to obtain an internal parameter matrix pre-calibrated by the shooting device;

[0100] Calculate a second relationship equation according to the intrinsic parameter matrix, the three-dimensional feature points of the reference object, the two-dimensional projection points of the feature points of the reference object in the current frame image, and the current posture conversion relationship between the world coordinate system and the current frame image coordinate system;

[0101] The second relationship equation is iteratively solved, and if the current posture conversion relationship at the current iteration number converges, the current posture conversion relationship at the current iteration number is used as the fourth posture conversion relationship.

[0102] Optionally, the second posture conversion relationship determination module 330 is further used to determine a fifth posture conversion relationship between the target object coordinate system and the reference frame image coordinate system according to the feature points of the target object and the projection points of the feature points of the target object in the reference frame image;

[0103] Based on the first posture conversion relationship and the fifth posture conversion relationship, the second posture conversion relationship is determined.

[0104] Optionally, the second posture conversion relationship determination module 330 is further used to obtain an internal parameter matrix pre-calibrated by the shooting device;

[0105] Calculating a third relationship equation according to the intrinsic parameter matrix, the three-dimensional feature points of the target object, and the two-dimensional projection points of the feature points of the target object in the reference frame image;

[0106] The third relationship equation is iteratively solved, and if the current posture conversion relationship at the current iteration number converges, the current posture conversion relationship at the current iteration number is used as the fifth posture conversion relationship.

[0107] Optionally, the second posture conversion relationship determination module 330 is further used to perform vector multiplication on the first posture conversion relationship and the fifth posture conversion relationship to obtain a current posture conversion relationship between the target object coordinate system and the current frame image coordinate system;

[0108] Calculate a fourth relationship equation according to an intrinsic parameter matrix obtained by pre-calibration of the camera, the three-dimensional feature points of the target object, the two-dimensional projection points of the feature points of the target object in the current frame image, and the current posture conversion relationship between the target object coordinate system and the current frame image coordinate system;

[0109] The fourth relationship equation is iteratively solved, and if the current posture transformation relationship at the current iteration number converges, the current posture transformation relationship at the current iteration number is used as the second posture transformation relationship.

[0110] Optionally, the first posture conversion relationship determination module 320 is further used to perform vector multiplication of the third posture conversion relationship and the fourth posture conversion relationship to obtain the first posture conversion relationship.

[0111] Optionally, the pose annotation module 340 is also used to obtain the pose of the feature points of the target object in the image taken by the shooting device based on the projection points of the target object in any frame image taken by the shooting device, the second pose transformation relationship and the internal parameter matrix pre-calibrated by the shooting device, and to perform pose annotation on the feature points of the target object.

[0112] Optionally, the shooting device includes any one of the following: a camera, a camera head and a laser; the reference object includes any one of the following: a QR code and a barcode.

[0113] Embodiment 4

[0114] Figure 6 A structural diagram of a posture annotation system provided in Embodiment 4 of the present invention. Figure 6 A block diagram of an exemplary robot suitable for use in implementing embodiments of the present invention is shown. Figure 6 The robot shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0115] Figure 6 The pose annotation system shown includes a robot, a target object and a reference object. The robot includes a robotic arm, a shooting device, a memory (not shown in the figure), and a processor (not shown in the figure). The robotic arm drives the shooting device to move so that the shooting device can collect sequence frame images at multiple angles; each frame of the sequence frame image includes a reference object and a target object, the target object includes multiple feature points, and the reference object includes multiple feature points. Figure 6 As shown, the robot further includes: a base and an end tool; the robotic arm is mounted on the base, the end tool is mounted on the end of the robotic arm, and the shooting device is mounted on one side of the end tool.

[0116] The processor executes various functional applications and data processing by running the program stored in the memory, for example, implementing a posture annotation method provided by an embodiment of the present invention, the method comprising:

[0117] Receiving a sequence of frame images captured by a shooting device at different angles, and determining a reference frame image and a current frame image in the sequence of frame images, wherein each frame image in the sequence of frame images includes a reference object and a target object;

[0118] Determine a first pose transformation relationship between a current frame image coordinate system and a reference frame image coordinate system based on feature points of the reference object, projection points of the feature points of the reference object in the reference frame image, and projection points of the feature points of the reference object in the current frame image;

[0119] Determine a second posture conversion relationship between a target object coordinate system where the target object is located and a current frame image coordinate system based on feature points of the target object, projection points of the feature points of the target object in the reference frame image, and the first posture conversion relationship;

[0120] Based on the second posture conversion relationship and the projection point of the target object in any frame image taken by the shooting device, the posture of the target object is marked.

[0121] Of course, those skilled in the art can understand that the processor can also implement the technical solution of a posture annotation method provided by any embodiment of the present invention.

[0122] Embodiment 5

[0123] Embodiment 5 of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, a posture annotation method provided in the embodiment of the present invention is implemented. The method includes:

[0124] Receiving a sequence of frame images captured by a shooting device at different angles, and determining a reference frame image and a current frame image in the sequence of frame images, wherein each frame image in the sequence of frame images includes a reference object and a target object;

[0125] Determine a first pose transformation relationship between a current frame image coordinate system and a reference frame image coordinate system based on feature points of the reference object, projection points of the feature points of the reference object in the reference frame image, and projection points of the feature points of the reference object in the current frame image;

[0126] Determine a second posture conversion relationship between a target object coordinate system where the target object is located and a current frame image coordinate system based on feature points of the target object, projection points of the feature points of the target object in the reference frame image, and the first posture conversion relationship;

[0127] Based on the second posture conversion relationship and the projection point of the target object in any frame image taken by the shooting device, the posture of the target object is marked.

[0128] Of course, the computer-readable storage medium provided by an embodiment of the present invention, and the computer program stored thereon are not limited to the above method operations, but can also execute related operations in a posture labeling method provided by any embodiment of the present invention.

[0129] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, system or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, system or device or used in combination with it.

[0130] The computer-readable signal medium may include a first posture conversion relationship, a second posture conversion relationship, a projection point of a target object in a captured image, etc., which carries a computer-readable program code. This propagation of the first posture conversion relationship, the second posture conversion relationship, the projection point of a target object in a captured image, etc. is in the form of. The computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, system or device.

[0131] Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0132] Computer program code for performing the operations of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0133] It is worth noting that in the embodiment of the above-mentioned posture labeling device, the various modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0134] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A pose labeling method, characterized in that: include: Receiving a sequence of frame images captured by a shooting device at different angles, and determining a reference frame image and a current frame image in the sequence of frame images, wherein each frame image in the sequence of frame images includes a reference object and a target object; Determine a first pose transformation relationship between a current frame image coordinate system and a reference frame image coordinate system based on feature points of the reference object, projection points of the feature points of the reference object in the reference frame image, and projection points of the feature points of the reference object in the current frame image; Determine a second posture conversion relationship between a target object coordinate system where the target object is located and a current frame image coordinate system based on feature points of the target object, projection points of the feature points of the target object in the reference frame image, and the first posture conversion relationship; Based on the second posture conversion relationship and the projection point of the target object in any frame image captured by the shooting device, the target object is marked with a posture; The method of determining the first pose conversion relationship between the current frame image coordinate system and the reference frame image coordinate system based on the feature points of the reference object, the projection points of the feature points of the reference object in the reference frame image, and the projection points of the feature points of the reference object in the current frame image includes: determining the third pose conversion relationship between the world coordinate system where the reference object is located and the reference frame image coordinate system based on the feature points of the reference object and the projection points of the feature points of the reference object in the reference frame image; determining the fourth pose conversion relationship between the world coordinate system and the current frame image coordinate system based on the feature points of the reference object and the projection points of the feature points of the reference object in the current frame image; and performing vector multiplication of the third pose conversion relationship and the fourth pose conversion relationship to obtain the first pose conversion relationship; The method of determining a second posture transformation relationship between a target object coordinate system where the target object is located and a current frame image coordinate system based on the feature points of the target object, the projection points of the feature points of the target object in the reference frame image, and the first posture transformation relationship, includes: determining a fifth posture transformation relationship between the target object coordinate system and the reference frame image coordinate system according to the feature points of the target object and the projection points of the feature points of the target object in the reference frame image; performing vector multiplication of the first posture transformation relationship and the fifth posture transformation relationship to obtain a current posture transformation relationship between the target object coordinate system and the current frame image coordinate system; calculating a fourth relationship equation according to an intrinsic parameter matrix obtained by pre-calibration of the camera, the three-dimensional feature points of the target object, the two-dimensional projection points of the feature points of the target object in the current frame image, and the current posture transformation relationship between the target object coordinate system and the current frame image coordinate system; iteratively solving the fourth relationship equation, and if the current posture transformation relationship at the current iteration number converges, using the current posture transformation relationship at the current iteration number as the second posture transformation relationship.

2. The method according to claim 1, characterized in that The determining, based on the feature points of the reference object and the projection points of the feature points of the reference object in the reference frame image, a third posture conversion relationship between the world coordinate system where the reference object is located and the reference frame image coordinate system comprises: Obtaining an intrinsic parameter matrix obtained by pre-calibration of the camera device; Calculate a first relationship equation according to the intrinsic parameter matrix, the three-dimensional feature points of the reference object, the two-dimensional projection points of the feature points of the reference object in the reference frame image, and the current posture conversion relationship between the world coordinate system and the reference frame image coordinate system; The first relationship equation is iteratively solved, and if the current posture conversion relationship at the current iteration number converges, the current posture conversion relationship at the current iteration number is used as the third posture conversion relationship.

3. The method according to claim 1, characterized in that The determining, based on the feature points of the reference object and the projection points of the feature points of the reference object in the current frame image, a fourth posture conversion relationship between the world coordinate system and the current frame image coordinate system comprises: Obtaining an intrinsic parameter matrix obtained by pre-calibration of the camera device; Calculate a second relationship equation according to the intrinsic parameter matrix, the three-dimensional feature points of the reference object, the two-dimensional projection points of the feature points of the reference object in the current frame image, and the current posture conversion relationship between the world coordinate system and the current frame image coordinate system; The second relationship equation is iteratively solved, and if the current posture conversion relationship at the current iteration number converges, the current posture conversion relationship at the current iteration number is used as the fourth posture conversion relationship.

4. The method according to claim 1, characterized in that: The step of determining a fifth posture conversion relationship between the target object coordinate system and the reference frame image coordinate system according to the feature points of the target object and the projection points of the feature points of the target object in the reference frame image comprises: Obtaining an intrinsic parameter matrix obtained by pre-calibration of the camera device; Calculating a third relationship equation according to the intrinsic parameter matrix, the three-dimensional feature points of the target object, and the two-dimensional projection points of the feature points of the target object in the reference frame image; The third relationship equation is iteratively solved, and if the current posture conversion relationship at the current iteration number converges, the current posture conversion relationship at the current iteration number is used as the fifth posture conversion relationship.

5. The method according to claim 1, characterized in that The step of labeling the target object based on the second posture conversion relationship and the projection point of the target object in any frame image captured by the shooting device includes: According to the projection point of the target object in any frame image taken by the shooting device, the second posture conversion relationship and the internal parameter matrix pre-calibrated by the shooting device, the posture of the feature points of the target object in the image taken by the shooting device is obtained, and the posture of the feature points of the target object is annotated.

6. The method according to claim 1, characterized in that The photographing device includes any one of the following: a camera, a camera head and a laser; and / or the reference object includes any one of the following: a QR code, a barcode.

7. A posture labeling device, characterized in that: include: An image determination module is used to receive a sequence of frame images captured by a shooting device at different angles, and determine a reference frame image and a current frame image in the sequence of frame images, wherein each frame image in the sequence of frame images includes a reference object and a target object; A first pose conversion relationship determination module is used to determine a first pose conversion relationship between a current frame image coordinate system and a reference frame image coordinate system based on feature points of the reference object, projection points of the feature points of the reference object in the reference frame image, and projection points of the feature points of the reference object in the current frame image; A second posture conversion relationship determination module is used to determine a second posture conversion relationship between a target object coordinate system where the target object is located and a current frame image coordinate system based on feature points of the target object, projection points of the feature points of the target object in the reference frame image, and the first posture conversion relationship; A posture annotation module, used for annotating the posture of the target object based on the second posture conversion relationship and the projection point of the target object in any frame image captured by the shooting device; The first posture conversion relationship determination module is further used to determine a third posture conversion relationship between a world coordinate system where the reference object is located and a coordinate system of the reference frame image based on feature points of the reference object and projection points of the feature points of the reference object in the reference frame image; determine a fourth posture conversion relationship between the world coordinate system and the coordinate system of the current frame image based on feature points of the reference object and projection points of the feature points of the reference object in the current frame image; and perform vector multiplication of the third posture conversion relationship and the fourth posture conversion relationship to obtain the first posture conversion relationship; The second posture transformation relationship determination module is also used to determine a fifth posture transformation relationship between the target object coordinate system and the reference frame image coordinate system based on the feature points of the target object and the projection points of the feature points of the target object in the reference frame image; perform vector multiplication on the first posture transformation relationship and the fifth posture transformation relationship to obtain a current posture transformation relationship between the target object coordinate system and the current frame image coordinate system; calculate a fourth relationship equation based on an intrinsic parameter matrix obtained by pre-calibration of the camera, the three-dimensional feature points of the target object, the two-dimensional projection points of the feature points of the target object in the current frame image, and the current posture transformation relationship between the target object coordinate system and the current frame image coordinate system; iteratively solve the fourth relationship equation, and if the current posture transformation relationship at the current iteration number converges, use the current posture transformation relationship at the current iteration number as the second posture transformation relationship.

8. A posture annotation system, comprising a robot, a target object and a reference object, wherein: The robot comprises a mechanical arm, a photographing device, a memory and a processor; the characteristic is that the mechanical arm drives the photographing device to move so that the photographing device collects sequence frame images at multiple angles; wherein each frame of the sequence frame images comprises a reference object and a target object, the target object comprises a plurality of feature points, and the reference object comprises a plurality of feature points; Among them, a computer program exists in the memory and can be run on the processor, and when the processor executes the computer program, the pose labeling method as described in any one of claims 1-6 is implemented.

9. The system according to claim 8, characterized in that The robot also includes: a base and an end tool; The mechanical arm is installed on the base, the end tool is installed at the end of the mechanical arm, and the shooting device is installed on one side of the end tool.

10. A storage medium containing computer executable instructions, characterized in that: When the computer executable instructions are executed by a computer processor, the pose labeling method according to any one of claims 1 to 6 is implemented.

Images