Object pose determination method and apparatus, robot, and storage medium

Abstract

The method for determining the pose of an object provided by the present disclosure, device, robot and storage medium, the method is applied to a robot, by acquiring first pose information of a to-be-identified object in a field of view of a first camera, and acquiring current coordinate system pose information of a calibration object, the current coordinate system pose information is based on the first camera, the first camera identifies the first height of the calibration object is consistent with the second height of the to-be-identified object identified by the first camera, and according to the current coordinate system pose information and the reference coordinate system pose information, the compensation coordinate system pose information is determined, and then the first pose information is adjusted according to the compensation coordinate system pose information, to obtain the second pose information of the to-be-identified object. In the technical solution, the change amount of the reference pose and the current pose is used to correct the pose of the to-be-identified object, so that the determination of the pose of the to-be-identified object is more accurate, and the measurement error caused by the time-varying of the camera and replacement is avoided.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of machine vision, and particularly relates to a method and device for determining the pose of an object, a robot and a storage medium. BACKGROUND

[0002] With the long-term operation of the camera, or the camera is hit, etc., the accuracy of the camera will be deteriorated, and in this case, the camera needs to be re-calibrated.

[0003] When the camera is re-calibrated, the technician needs to adjust according to some teaching templates, and it takes time, and the above-mentioned accuracy deterioration is not immediately discovered, which causes supervision difficulties, affecting the operation stability and efficiency of the corresponding vision system.

[0004] Therefore, how to ensure the accuracy of camera vision when grabbing objects becomes a technical problem to be solved. SUMMARY

[0005] The present disclosure provides a method and device for determining the pose of an object, a robot and a storage medium to solve the problem of how to ensure the accuracy of camera vision when grabbing objects.

[0006] In a first aspect, the present disclosure provides a method for determining the pose of an object, applied to a robot, comprising:

[0007] obtaining first pose information of a to-be-identified object under the field of view of a first camera;

[0008] obtaining current coordinate system pose information of a calibration object, the current coordinate system pose information being obtained based on the first camera, and a first height of the calibration object identified by the first camera being consistent with a second height of the to-be-identified object identified by the first camera;

[0009] determining compensation coordinate system pose information according to the current coordinate system pose information and reference coordinate system pose information, the reference coordinate system pose information being obtained by pre-determining the pose of the calibration object based on a second camera, a third height of the calibration object identified by the second camera being consistent with the first height, and the compensation coordinate system pose information being change information between the reference coordinate system pose information and the current coordinate system pose information;

[0010] adjusting the first pose information according to the compensation coordinate system pose information to obtain second pose information of the to-be-identified object.

[0011] In a possible design, the calibration object includes at least three calibration balls.

[0012] Before the determining the compensation coordinate system pose information according to the current coordinate system pose information and the reference coordinate system pose information, the method further comprises:

[0013] S1, acquiring point cloud information of the at least three calibration balls by the second camera, the first plane formed by the at least three calibration balls being located in a second plane corresponding to a field of view of the second camera;

[0014] S2, determining a spherical cap point cloud of the at least three calibration balls from the point cloud information of the at least three calibration balls;

[0015] S3, for each calibration ball, fitting a ball center pose of the calibration ball according to the spherical cap point cloud of the calibration ball;

[0016] S4, determining first coordinate system pose information of the center of the at least three calibration balls based on the ball center pose of each calibration ball;

[0017] S5, repeating the steps S1-S4 for more than N times within a preset time length, and when the difference between all obtained first coordinate system pose information is less than a first preset threshold, taking the first coordinate system pose information of any time as the reference coordinate system pose information, the N being a positive integer greater than 0.

[0018] As a possible design, the method further comprises:

[0019] During the repeating of the steps S1-S4, if the difference between the obtained first coordinate system pose information is greater than or equal to the first preset threshold, determining a first prompt information, the first prompt information being used to prompt the user to check the point cloud quality and the robot to position repeatability.

[0020] As a possible design, after the acquiring the current coordinate system pose information of the calibration object, the method further comprises:

[0021] determining a drift amount of the calibration object according to the current coordinate system pose information of the calibration object and the reference coordinate system pose information of the calibration object;

[0022] if the drift amount is greater than a second preset threshold and less than a third preset threshold, determining a second prompt information, the second prompt information being used to prompt the user that the calibration object has drift;

[0023] if the drift amount is greater than or equal to the third preset threshold, determining a third prompt information, the third preset threshold being greater than the second preset threshold, the third prompt information being used to prompt the user that the calibration object has too much drift.

[0024] As a possible design, the method further comprises:

[0025] The reference coordinate system pose information is updated according to steps S1-S5 when a preset condition is met.

[0026] The preset condition includes at least one of the following: an operation instruction of a user for updating the reference coordinate system pose information, detection of a drift between the current coordinate system pose information and the reference coordinate system pose information being greater than a second preset threshold, and a time length from a last time when the reference coordinate system pose information is determined reaching a first preset time length.

[0027] As a possible design, the current coordinate system pose information of the calibration object is obtained in the following manner:

[0028] The current coordinate system pose information of the calibration object is obtained based on the first camera within a second preset time length before the first pose information is obtained.

[0029] The current coordinate system pose information of the calibration object is obtained based on the first camera within a third preset time length after the first pose information is obtained.

[0030] When it is detected that the pose information of the last to-be-recognized object is inaccurate after the last to-be-recognized object is compensated for pose information based on the last compensated coordinate system pose information, the current coordinate system pose information of the calibration object is obtained based on the first camera.

[0031] As a possible design, a distance value between the first wide side of the first plane and the second wide side of the second plane is equal to a second wide side corresponding length value of a preset coefficient, and a distance value between the first long side of the first plane and the second long side of the second plane is equal to a second long side corresponding length value of the preset coefficient.

[0032] In a second aspect, an object pose determination apparatus is provided, and is applied to a robot. The apparatus includes:

[0033] The obtaining module is configured to obtain first pose information of a to-be-recognized object in a field of view of a first camera, and obtain current coordinate system pose information of a calibration object, the current coordinate system pose information being obtained based on the first camera, and a first height of the calibration object recognized by the first camera being consistent with a second height of the to-be-recognized object recognized by the first camera.

[0034] The first determining module is configured to determine compensation coordinate system pose information according to the current coordinate system pose information and reference coordinate system pose information, wherein the reference coordinate system pose information is determined based on a second camera and the calibration object, the second camera identifies that the third height of the calibration object is consistent with the first height, and the compensation coordinate system pose information is change information between the reference coordinate system pose information and the current coordinate system pose information.

[0035] The second determining module is configured to adjust the first pose information according to the compensation coordinate system pose information to obtain second pose information of the object to be identified.

[0036] In a possible design, the calibration object includes at least three calibration balls.

[0037] Before the compensation coordinate system pose information is determined according to the current coordinate system pose information and the reference coordinate system pose information, the first determining module is further configured to

[0038] S1, acquiring point cloud information of the at least three calibration balls by using the second camera, wherein a first plane formed by the at least three calibration balls is located in a second plane corresponding to a field of view of the second camera.

[0039] S2, determining a spherical cap point cloud of the at least three calibration balls from the point cloud information of the at least three calibration balls.

[0040] S3, for each calibration ball, fitting a ball center pose of the calibration ball according to the spherical cap point cloud of the calibration ball.

[0041] S4, determining first coordinate system pose information of a center of the at least three calibration balls based on the ball center pose of each calibration ball.

[0042] S5, repeating the steps S1-S4 for more than N times within a preset time length, and when a difference between all obtained first coordinate system pose information is less than a first preset threshold, taking first coordinate system pose information of any one time as the reference coordinate system pose information, wherein N is a positive integer greater than 0.

[0043] In a possible design, the first determining module is further configured to

[0044] During the repeating of the steps S1-S4, if the difference between the obtained first coordinate system pose information is greater than or equal to the first preset threshold, first prompt information is determined, wherein the first prompt information is used to prompt a user to check point cloud quality and a robot to position repeatability.

[0045] As a possible design, after obtaining the current coordinate system pose information of the calibration object, the first determination module is further configured to

[0046] determine a drift amount of the calibration object according to the current coordinate system pose information of the calibration object and the reference coordinate system pose information of the calibration object;

[0047] if the drift amount is greater than a second preset threshold and less than a third preset threshold, determine second prompt information, the second prompt information being used to prompt the user that the calibration object has a drift;

[0048] if the drift amount is greater than or equal to the third preset threshold, determine third prompt information, the third preset threshold being greater than the second preset threshold, and the third prompt information being used to prompt the user that the calibration object has a high drift.

[0049] As a possible design, the first determination module is further configured to

[0050] update the reference coordinate system pose information according to steps S1-S5 when a preset condition is met;

[0051] The preset condition includes at least one of the following: an operation instruction of the user for updating the reference coordinate system pose information, a detection that the drift amount between the current coordinate system pose information and the reference coordinate system pose information is greater than a second preset threshold, and a time length from a time when the reference coordinate system pose information is last determined to a first preset time length.

[0052] As a possible design, the obtaining of the current coordinate system pose information of the calibration object includes any one of the following:

[0053] obtaining the current coordinate system pose information of the calibration object based on the first camera within a second preset time length before the first pose information is obtained;

[0054] obtaining the current coordinate system pose information of the calibration object based on the first camera within a third preset time length after the first pose information is obtained;

[0055] detecting that the pose information of the last to-be-recognized object is inaccurate after the last to-be-recognized object is compensated for pose information based on last compensation coordinate system pose information, and obtaining the current coordinate system pose information of the calibration object based on the first camera.

[0056] As a possible design, a distance value between the first wide side of the first plane and the second wide side of the second plane is equal to a second wide side corresponding length value of a preset coefficient, and a distance value between the first long side of the first plane and the second long side of the second plane is equal to a second long side corresponding length value of the preset coefficient.

[0057] In a third aspect, the present disclosure provides a robot, comprising: a processor, and a memory connected to the processor in communication;

[0058] The memory stores computer-executable instructions.

[0059] The processor executes the computer-executable instructions stored in the memory to implement the method according to the first aspect or any of the manners.

[0060] In a fourth aspect, the present disclosure provides a computer-readable storage medium, which stores computer-executable instructions, and the computer-executable instructions are executed by a processor to implement the method according to the first aspect or any of the manners.

[0061] In a fifth aspect, the present disclosure provides a computer program product, which comprises a computer program, and the computer program is executed by a processor to implement the method according to the first aspect or any of the manners.

[0062] The method for determining the pose of an object provided by the present disclosure is applied to a robot, and the method comprises the following steps: obtaining first pose information of a to-be-identified object in a field of view of a first camera, and obtaining current coordinate system pose information of a calibration object, the current coordinate system pose information being obtained based on a first camera, a first height of the calibration object identified by the first camera being consistent with a second height of the to-be-identified object identified by the first camera, and determining compensation coordinate system pose information according to the current coordinate system pose information and reference coordinate system pose information, the reference coordinate system pose information being obtained by determining the pose of the calibration object based on a second camera in advance, a third height of the calibration object identified by the second camera being consistent with the first height, the compensation coordinate system pose information being change information between the reference coordinate system pose information and the current coordinate system pose information, and then adjusting the first pose information according to the compensation coordinate system pose information to obtain second pose information of the to-be-identified object. In the technical solution, the change amount of the reference pose and the current pose is used to correct the pose of the to-be-identified object, so that the determination of the pose of the to-be-identified object is more accurate, and the measurement error caused by the time-varying of the camera and replacement is avoided. BRIEF DESCRIPTION OF DRAWINGS

[0063] The accompanying drawings, which are incorporated into and form a part of the specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the disclosure.

[0064] Figure 1 Flowchart of the method for determining the pose of an object provided by the present disclosure Figure One ;

[0065] Figure 2 Schematic view of the field of view of the first camera provided by the present disclosure

[0066] Figure 3 A view field diagram of a second camera provided for an embodiment of the present disclosure;

[0067] Figure 4 A flowchart of a method for determining an object pose provided for an embodiment of the present disclosure Figure Two ;

[0068] Figure 5 An installation diagram of a calibration object;

[0069] Figure 6 A flowchart of a method for determining an object pose provided for an embodiment of the present disclosure Figure Three ;

[0070] Figure 7 A structure diagram of a device for determining an object pose provided for an embodiment of the present disclosure;

[0071] Figure 8 A structure diagram of a robot provided for an embodiment of the present disclosure.

[0072] The specific embodiments of the present disclosure have been shown through the above-mentioned drawings, and will be described in more detail hereinafter. These drawings and textual descriptions are not intended to limit the scope of the concept of the present disclosure by any means, but to illustrate the concept of the present disclosure to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0073] In order to make the objects, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be described clearly and completely below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only some of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without any creative work fall within the scope of protection of the present disclosure.

[0074] Before introducing the embodiments of the present disclosure, the application background of the embodiments of the present disclosure is first explained:

[0075] After a camera is operated for a long time, its precision will change due to time-varying phenomena such as temperature drift, or / and the camera needs to be replaced due to collision, and the parameters of the camera need to be recalibrated to improve the precision of the camera.

[0076] When recalibrating the camera, the technician needs to adjust according to some teaching templates, and this debugging method has the following technical problems:

[0077] 1. The change of camera parameters caused by temperature drift and other time-varying phenomena is generally not easy to be found, and may be found when there is a major problem;

[0078] 2. Recalibrating the camera and its teaching template is a large-scale task that results in a significant waste of manpower and financial resources.

[0079] 3. There is a lack of error monitoring mechanisms at the operational site;

[0080] 4. Unable to promptly correct the accurate position of an object when grasping it.

[0081] To address the technical problems existing in the prior art, the inventors of this disclosure have the following concept: During the process of adjusting camera accuracy, the inventors discovered that if the pose of the most standard calibration object can be determined in advance using the camera, then during subsequent measurements, the standard pose and the currently determined pose of the calibration object can be used to determine the possible accuracy error of the camera, thereby correcting the real-time object to be identified. Furthermore, in scenarios where the camera is replaced, the accuracy error can also be determined using the standard pose and the real-time pose under the replaced camera, correcting the object to be identified. Additionally, based on the changes between the current pose of the calibration object and the standard pose, corresponding warning thresholds can be assigned, realizing a monitoring mechanism for the operational site.

[0082] It should be understood that the methods disclosed herein can be used in EyeInHand high-precision 3D vision scenarios.

[0083] The technical solutions of this disclosure will now be described in detail through specific embodiments. It should be noted that the following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0084] It is worth noting that the application fields of the methods, devices, robots and storage media for determining the pose of objects disclosed herein are not limited.

[0085] In this disclosure, the implementing entity is a robot, which may specifically be a control unit in the robot or a controller that controls the robot.

[0086] Figure 1 Flowchart of the method for determining the pose of an object provided in the embodiments of this disclosure Figure One ,like Figure 1 As shown, the method for determining the pose of this object may include the following steps:

[0087] Step 11: Obtain the first pose information of the object to be identified in the field of view of the first camera;

[0088] Step 12: Obtain the current coordinate system pose information of the calibration object;

[0089] The current coordinate system pose information is based on the image captured by the first camera, and the first height of the calibration object identified by the first camera is consistent with the second height of the object to be identified identified by the first camera.

[0090] In the two steps above, the first camera may have accuracy errors due to long-term operation, or the first camera may be a new camera that has been replaced. In this case, it is necessary to use the first camera to measure the actual pose information of the object to be identified.

[0091] First, the first camera is used to obtain the first pose information of the object to be identified in the field of view of the first camera, and the current coordinate system pose information of the calibrated object is obtained.

[0092] Optionally, obtaining the current coordinate system pose information may include any of the following:

[0093] 1. Within a second preset time period before acquiring the first pose information, acquire the current coordinate system pose information of the calibration object based on the first camera;

[0094] 2. Within a third preset time period after acquiring the first pose information, acquire the current coordinate system pose information of the calibration object based on the first camera;

[0095] 3. If the pose information of the object to be identified is inaccurate after the previous pose information of the object to be identified is compensated based on the previous coordinate system pose information, the current coordinate system pose information of the calibrated object is obtained based on the first camera.

[0096] For example, the first camera takes a picture of the target object and then takes a picture of the calibration object, or takes a picture of the target object before taking a picture of the calibration object; the time of taking a picture of the target object is 11:00 on the 2nd, which is within 10 hours before the first camera takes a picture of the target object, or within 10 hours after the first camera takes a picture of the target object; if the recognition error of the previous target object is detected to be too large, the calibration object is photographed, for example, the target object is not accurately captured (this implementation may be that the target object is compensated according to the previous compensated coordinate system pose information, but the target object is still not captured).

[0097] Optionally, the calibration objects can be objects that can be established into a coordinate system, such as at least three calibration spheres, objects with properties that can be used to construct a coordinate system, etc.

[0098] For example, Figure 2 A schematic diagram of the field of view of the first camera provided in an embodiment of this disclosure, such as... Figure 2 As shown, the schematic diagram includes: a first camera (e.g., camera 21), a calibration object 23, an object to be identified 22, a first height H1, and a second height H2.

[0099] That is, the first height of the calibrator 23 identified by the camera 21 is H1, and the second height of the object to be identified by the camera 21 is H2, where H1 = H2.

[0100] It should be understood that the first camera can be a camera that the related system has been using (i.e., the second camera described below, as described below); or can be a camera after the second camera described below is replaced due to some reasons.

[0101] Optionally, in different height scenarios, taking k layers (each layer has different heights) as an example, if the workpieces of the k layers share one shooting point, there will be k different shooting (workpiece / calibration object) distances.

[0102] As an example, the calibration object can be at least three calibration balls. The first height is the distance between the center of the plane formed by the at least three calibration balls and the first camera.

[0103] Optionally, if the first height is less than a height threshold, the size of each calibration ball is 60 mm; if the first height is greater than or equal to the height threshold, the size of each calibration ball is 100 mm.

[0104] As an example, the height threshold can be 2 m, and the size can be the diameter of the calibration ball.

[0105] Optionally, the at least three calibration balls include at least one of the following: matte iron balls, ceramic balls.

[0106] In order to avoid the influence of external factors on the accuracy of the camera, in the selection of the calibration object, matte iron balls or ceramic balls that are friendly to point cloud acquisition can be used. When the distance between the camera and the calibration object (i.e., the first height, which can also be the second height or the third height) is greater than or equal to 2 m, 3 matte iron balls with a size of 100 mm are selected; when the distance between the camera and the calibration object (i.e., the first height) is less than 2 m, 3 matte iron balls with a size of 60 mm are selected.

[0107] For example, the first pose information is (10, 12, 10); the current coordinate system pose information is (5, 6, 5).

[0108] Step 13, determining compensation coordinate system pose information according to the current coordinate system pose information and the reference coordinate system pose information.

[0109] The reference coordinate system pose information is determined in advance based on the pose of the calibration object determined by the second camera, the third height of the calibration object recognized by the second camera is consistent with the first height, and the compensation coordinate system pose information is the change information between the reference coordinate system pose information and the current coordinate system pose information.

[0110] In this step, the reference coordinate system pose information is the standard pose information of the coordinate system determined by the second camera for the calibration object when the second camera does not have drift and the like.

[0111] Optionally, the change information between the reference coordinate system pose information and the current coordinate system pose information can be determined by using the current coordinate system pose information determined by the first camera in real time or the current coordinate system pose information determined by the first camera before that, and the reference coordinate system pose information determined by the second camera in advance.

[0112] For example, according to the above example, the reference coordinate system pose information is (5, 5, 5), and the compensation coordinate system pose information is (0, -1, 0).

[0113] Figure 3 A view of the second camera provided by the embodiment of the present disclosure is shown in the schematic view as shown in the figure, which includes a second camera (for example, camera 31), a calibration object 23, and a third height H 3. Figure 3

[0114] If the camera 31 is the camera 21 (that is, the scene of compensating the pose of the camera), if the camera 31 is not the camera 21 (that is, the scene of replacing the camera).

[0115] For example, when the camera is damaged and needs to be replaced, according to the scheme, only the calibration object needs to be reacquired after the camera is replaced, and the object to be recognized can be accurately grabbed (without other additional operations such as recalibration of external parameters and re-teaching of templates); periodically (such as daily or weekly) acquire data of the calibration object, and correct the grabbing pose of the object to be recognized according to the change of the calibration object in the camera coordinate system.

[0116] It should be understood that the second camera can be the first camera or other cameras, which is determined according to the corresponding scene.

[0117] For example, the reference coordinate system pose information (SphereDatumCoordinatePose); the current coordinate system pose information (SphereRealtimeCoordinatePose).

[0118] Step 14, adjusting the first pose information according to the compensation coordinate system pose information to obtain the second pose information of the object to be recognized.

[0119] In this step, after the compensation coordinate system pose information is determined according to the reference coordinate system pose information and the current coordinate system pose information, the first pose information corresponding to the object to be recognized is adjusted (that is, compensated) by using the compensation coordinate system pose information, so as to obtain the actual pose information of the object to be recognized, that is, the second pose information.

[0120] For example, the first pose information is (10, 12, 10), the compensation coordinate system pose information is (0, -1, 0), and the second pose information is (10, 11, 10).

[0121] ​The object pose determination method provided by the embodiments of the present disclosure is applied to a robot, and the method comprises the following steps: obtaining first pose information of a to-be-identified object in a field of view of a first camera, and obtaining current coordinate system pose information of a calibration object, wherein the current coordinate system pose information is obtained based on shooting of the first camera, a first height of the calibration object identified by the first camera is consistent with a second height of the to-be-identified object identified by the first camera, and compensation coordinate system pose information is determined according to the current coordinate system pose information and reference coordinate system pose information, wherein the reference coordinate system pose information is obtained by determining the pose of the calibration object based on a second camera in advance, a third height of the calibration object identified by the second camera is consistent with the first height, and the compensation coordinate system pose information is change information between the reference coordinate system pose information and the current coordinate system pose information, and then the first pose information is adjusted according to the compensation coordinate system pose information to obtain second pose information of the to-be-identified object. In the technical solution, the change amount of the reference pose and the current pose is used to correct the pose of the to-be-identified object, so that the determination of the pose of the to-be-identified object is more accurate, and the measurement error caused by the time-varying of the camera and replacement is avoided.

[0122] On the basis of the above-mentioned embodiments, the calibration object comprises at least three calibration balls, and the current coordinate system pose information can be pose information of a coordinate system at the center of a plane formed by the at least three calibration balls.

[0123] Figure 4 The flowchart of the object pose determination method provided by the embodiments of the present disclosure is shown in Figure Two As shown in Figure 4 Before the above-mentioned step 13, the object pose determination method can further comprise the following steps:

[0124] That is, in the present solution, before the above-mentioned step 13, the reference coordinate system pose information also needs to be determined:

[0125] S1, obtaining point cloud information of the at least three calibration balls by the second camera.

[0126] The first plane formed by the at least three calibration balls is located in a second plane corresponding to the field of view of the second camera;

[0127] Optionally, a distance value between a first long side of the first plane and a second long side of the second plane is equal to a second long side corresponding length value of the preset coefficient, and a distance value between a first wide side of the first plane and a second wide side of the second plane is equal to a second wide side corresponding length value of the preset coefficient.

[0128] For example, Figure 5 The installation diagram of the calibration object is shown in Figure 5As shown, it comprises: a camera 31, a ball 1, a ball 2, a ball 3 (illustrated as three small balls), a first plane S1, a second plane S2, a center O of the first plane S1, a second wide-side corresponding length value W, and a second long-side corresponding length value L.

[0129] For example, the preset coefficient is 0.1.

[0130] Optionally, the preset coefficient can be adjusted based on the possible size of the to-be-identified object. If the to-be-identified object is small, the preset coefficient can be increased so that the calibration balls are gathered towards the center of the field of view of the camera. If the to-be-identified object is large, the preset coefficient can be decreased so that the calibration balls are away from the center of the field of view of the camera.

[0131] It should be understood that the implementation of this step can also adjust the parameters of the second camera so that the point cloud quality of the calibration balls is optimal, and the hemispherical surface point cloud is complete and smooth.

[0132] S2, determining the hemispherical point cloud of the at least three calibration balls from the point cloud information of the at least three calibration balls.

[0133] That is, for each calibration ball, the hemispherical point cloud is extracted from the point cloud in the 3D Region of Interest (RoI) of the calibration object, that is, the hemispherical point cloud of the calibration object is obtained.

[0134] S3, for each calibration ball, fitting the center pose of the calibration ball according to the hemispherical point cloud of the calibration ball.

[0135] The hemispherical point cloud of the calibration ball is fitted to obtain the center pose of the calibration ball.

[0136] S4, determining the first coordinate system pose information of the centers of the at least three calibration balls based on the center pose of each calibration ball.

[0137] As an example, taking three calibration balls as an example, the center pose of each calibration ball is known, and the coordinates of the three corners of the triangle can be used to obtain the coordinates of the center point of the triangle, that is, the first coordinate system pose information of the centers of the three calibration balls is obtained.

[0138] As another possible implementation, the center poses of a plurality of calibration balls can be determined, and three calibration balls with excellent point cloud data are selected to determine the first coordinate system pose information.

[0139] S5, repeating the steps S1-S4 N times or more within a preset time period, and when the difference between all obtained first coordinate system pose information is less than a first preset threshold, taking the first coordinate system pose information of any time as the reference coordinate system pose information.

[0140] Wherein, N is an integer greater than 0

[0141] Optionally, the higher calibration object (for example, the calibration ball) identifies the repeated accuracy (in a short period of time) as a prerequisite for this scheme, that is, the above-mentioned S1-S4 can be repeated more than once within a preset time period (for example, 3 minutes), and the obtained first coordinate system pose information is compared, and whether the difference between all obtained first coordinate system pose information is less than the first preset threshold (for example, 0.3mm) is compared.

[0142] If the difference between all obtained first coordinate system pose information is less than 0.3mm, any first coordinate system pose information can be used as reference coordinate system pose information.

[0143] For example, the last first coordinate system pose information can be used as the reference coordinate system pose information.

[0144] It should be understood that the reference coordinate system pose information is the calibration ball coordinate system pose recorded at the initial time when the entire system is intact. Once the coordinate system is determined, it will not be modified unless the calibration ball needs to be updated due to the calibration ball being moved or the like. The above-mentioned current coordinate system pose information will generate a new calibration ball coordinate system pose each time the calibration ball is photographed in the subsequent process, and the last current coordinate system pose information will be updated (the update is realized by setting the parameter "maximum file number" to 1 in "save results to file" in Mech_Vision).

[0145] In addition, during the repetition of the above-mentioned steps S1-S4, if the difference between the obtained first coordinate system pose information is greater than or equal to the first preset threshold, a first prompt information is determined, and the first prompt information is used to prompt the user to check the point cloud quality and the repeatability of the robot to position.

[0146] That is, if the difference between the obtained first coordinate system pose information is not less than 0.3mm, the user needs to be reminded to check whether there is a problem with the point cloud quality; and whether the robot carrying the second camera reaches the preset photographing position when repeatedly performing the above-mentioned steps.

[0147] The prompt method of the prompt information involved in the present disclosure can be a light with different colors, voice prompt, panel display, and the like, and the method is not limited.

[0148] In addition, the object pose determination method can further include the following operation: updating the reference coordinate system pose information according to the above-mentioned steps S1-S5 when the preset condition is met.

[0149] The preset condition comprises at least one of the following: in response to an operation instruction of the user for updating the reference coordinate system pose information, detecting that a drift between the current coordinate system pose information and the reference coordinate system pose information is greater than a second preset threshold, and a time length from a time when the reference coordinate system pose information is last determined reaching a first preset time length.

[0150] For example, when the calibration object is accidentally touched by the user, there is an offset on the physical space, etc., the user can input an operation instruction to the robot to instruct the robot to update the reference coordinate system pose information; when the robot detects that the calibration object has a drift between the initial and current pose information, and the drift is greater than 0.2 mm (second preset threshold), the robot is instructed to update the reference coordinate system pose information; it is unknown whether the calibration object has changed, at this time, the reference coordinate system pose information can be updated after 10 days (first preset time length) from the time when the reference coordinate system pose information is last determined; the calibration object is loose and displacement is checked (which can be preliminarily judged by related functions of error monitoring), if the calibration object is loose, the reference coordinate system pose information needs to be updated after being reinforced, etc.

[0151] Optionally, the fluctuation difference value between the current pose and the reference pose of the drift of the calibration object can output the maximum difference value of X, Y and Z respectively for front-end display of error discount graph.

[0152] The object pose determination method provided by the embodiments of the present disclosure comprises the following steps: S1, acquiring point cloud information of at least three calibration balls through a second camera, a first plane formed by the at least three calibration balls being located in a second plane corresponding to a field of view of the second camera; S2, determining a spherical cap point cloud of the at least three calibration balls from the point cloud information of the at least three calibration balls; S3, for each calibration ball, fitting a center pose of the calibration ball according to the spherical cap point cloud of the calibration ball; S4, determining first coordinate system pose information of a center of the at least three calibration balls based on the center pose of each calibration ball; and S5, repeating the steps S1-S4 more than N times within a preset time length, and when a difference between all obtained first coordinate system pose information is less than a first preset threshold, taking the first coordinate system pose information of any time as reference coordinate system pose information, wherein N is a positive integer. In the technical solution, the accurate determination of the reference coordinate system pose is realized, and the pose compensation of the to-be-recognized object due to time variation or replacement of the camera is facilitated.

[0153] Based on the above embodiments, Figure 6 The flowchart of the object pose determination method provided by the embodiments of the present disclosure is shown in Figure Three As shown in Figure 6 After the step 11, the object pose determination method can further have the following steps:

[0154] Step 61, determining the drift of the calibration object according to the current coordinate system pose information of the calibration object and the reference coordinate system pose information of the calibration object;

[0155] In the present scheme, the calibration object can detect the drift in the field of view of the camera to remind the user of the state of the calibration object and whether it affects the accuracy of the related system.

[0156] In the present step, the difference information between the current coordinate system pose information and the reference coordinate system pose information can be used as the drift of the calibration object.

[0157] Taking the calibration object as at least three calibration balls as an example, for any calibration ball, for example, ball 1, the drift between the ball center poses in the two messages is determined by using the method described in the above embodiment. For example, the drift is 0.3, or 0.6, etc.

[0158] Step 62, if the drift is greater than a second preset threshold and less than a third preset threshold, determining second prompt information.

[0159] The second prompt information is used to prompt the user that the calibration object has drift.

[0160] In the present step, between the second preset threshold and the third preset threshold, it can be indicated that the calibration ball has drift, but the influence on the system operation is not large, at this time, the alarm can be given, but the system does not stop.

[0161] For example, the second preset threshold is 0.2, and the third preset threshold is 0.5.

[0162] That is, for example, the drift of ball 1 is 0.3, prompting the user that ball 1 has drift.

[0163] Step 63, if the drift is greater than or equal to the third preset threshold, determining third prompt information.

[0164] The third prompt information is used to prompt the user that the drift of the calibration object is too high, and the third preset threshold is greater than the second preset threshold.

[0165] In the present step, greater than or equal to the third preset threshold can indicate that the calibration ball has drift and has a large influence on the operation of the system, at this time, the alarm can be given and the system operation is stopped.

[0166] That is, for example, the drift of ball 3 is 0.6, prompting the user that ball 3 has drift and the system has been stopped.

[0167] Due to the reasons such as too large drift or serious failure, the drift amount is usually reset to zero after troubleshooting and repair (the current correction method should be excluded in the re-teaching process), and the reference coordinate system pose information needs to be updated after the drift amount is reset to zero.

[0168] It should be understood that the execution of steps 62 and 63 is to choose one to implement based on the actual situation.

[0169] The object pose determination method provided in this disclosure determines the drift amount of the calibrated object based on its current coordinate system pose information and its reference coordinate system pose information. If the drift amount is greater than a second preset threshold and less than a third preset threshold, a second prompt message is determined to alert the user that the calibrated object has drifted. If the drift amount is greater than or equal to the third preset threshold, a third prompt message is determined, where the third preset threshold is greater than the second preset threshold, to alert the user that the calibrated object has drifted too much. This technical solution, by determining the warning threshold, implements corresponding warnings when camera accuracy changes, facilitating real-time camera maintenance by the user and avoiding serious consequences that may arise from camera accuracy issues in existing technologies.

[0170] The following are some implementation details of the technical solutions involved in this disclosure based on the above embodiments (the implementation principles and technical effects are the same as above):

[0171] Firstly, regarding the selection of the calibration object (e.g., the calibration ball):

[0172] In terms of material, it can be a point cloud quality friendly material, such as a matte iron ball or a ceramic ball; in terms of size, it can be the distance from the calibration ball to the camera. If it is less than 2 meters, choose a diameter of 60mm; if it is not less than 2 meters, choose a diameter of 100mm. The distance from the calibration ball to the camera can be determined to be consistent with the distance from the workpiece to the camera. If k layers of workpieces share a single shooting point, then there are k different shooting distances (shooting the workpiece or the calibration ball).

[0173] Secondly, install the calibration object:

[0174] (by Figure 2 (For example) The line connecting sphere 2 and sphere 3 is parallel to the camera baseline direction. The distance from the calibration sphere to the camera's field of view boundary is approximately 0.2 times the field of view size. If the workpiece is small, the three calibration spheres can be appropriately clustered towards the center of the field of view.

[0175] Third, the collection of calibration materials:

[0176] The imaging height of the calibration ball is related to the imaging height of the workpiece. Each workpiece has a corresponding calibration ball of similar height within the camera's field of view (the coordinate system generated by the three calibration balls). The workpiece and the calibration ball are both located at the center of the camera's field of view. When taking pictures, the camera's optical axis should be as perpendicular as possible to the workpiece and the calibration ball. The sampling frequency of the calibration ball can be measured in days or weeks.

[0177] Fourth, adjusting camera parameters:

[0178] Adjust the camera parameters to achieve the best point cloud quality on the calibration sphere, resulting in a complete and smooth point cloud on the hemispherical surface;

[0179] V. Maintenance of the calibration object:

[0180] The calibration ball is firmly installed and is prevented from being touched. The calibration ball is regularly cleaned to prevent dust from accumulating on the surface. If dust is visible on the surface, compressed air is used to blow away the dust, and forceful wiping is not allowed.

[0181] The following is an embodiment of the device of the present disclosure, which can be used to execute the method embodiment of the present disclosure. For details not disclosed in the device embodiment of the present disclosure, please refer to the method embodiment of the present disclosure.

[0182] Figure 7 A structural diagram of a device for determining the pose of an object is provided in an embodiment of the present disclosure. As shown in Figure 7 The device for determining the pose of an object is applied to a robot and includes:

[0183] The acquisition module 71 is configured to acquire first pose information of a to-be-identified object in a field of view of a first camera and acquire current coordinate system pose information of a calibration object, the current coordinate system pose information being obtained based on a first camera shot, and a first height of the calibration object identified by the first camera being consistent with a second height of the to-be-identified object identified by the first camera;

[0184] The first determination module 72 is configured to determine compensation coordinate system pose information according to the current coordinate system pose information and reference coordinate system pose information, the reference coordinate system pose information being obtained by pre-determining the pose of the calibration object based on a second camera, a third height of the calibration object identified by the second camera being consistent with the first height, and the compensation coordinate system pose information being change information between the reference coordinate system pose information and the current coordinate system pose information;

[0185] The second determination module 73 is configured to adjust the first pose information according to the compensation coordinate system pose information to obtain second pose information of the to-be-identified object.

[0186] In a possible design, the calibration object includes at least three calibration balls;

[0187] Before determining the compensation coordinate system pose information according to the current coordinate system pose information and the reference coordinate system pose information, the first determination module 72 is further configured to

[0188] S1, acquiring point cloud information of at least three calibration balls through a second camera, a first plane formed by the at least three calibration balls being located in a second plane corresponding to a field of view of the second camera;

[0189] S2, determining ball crown point cloud of the at least three calibration balls from the point cloud information of the at least three calibration balls;

[0190] S3, for each calibration ball, fitting a ball center pose of the calibration ball according to the ball crown point cloud of the calibration ball.

[0191] S4, determine the first coordinate system pose information of the center of the at least three calibration balls based on the ball center pose of each calibration ball;

[0192] S5, repeat the steps S1-S4 for more than N times within a preset time length, and when the difference between all obtained first coordinate system pose information is less than a first preset threshold, take the first coordinate system pose information of any one time as the reference coordinate system pose information, and N is an integer greater than 0.

[0193] As a possible design, the first determination module 72 is further configured to

[0194] In the process of repeating the steps S1-S4, if the difference between the obtained first coordinate system pose information is greater than or equal to the first preset threshold, determine the first prompt information, and the first prompt information is used to prompt the user to check the point cloud quality and the robot to position the repetition accuracy.

[0195] As a possible design, after obtaining the current coordinate system pose information of the calibration object, the first determination module 72 is further configured to

[0196] According to the current coordinate system pose information of the calibration object and the reference coordinate system pose information of the calibration object, determine the drift amount of the calibration object;

[0197] If the drift amount is greater than a second preset threshold and less than a third preset threshold, determine the second prompt information, and the second prompt information is used to prompt the user that the calibration object has drift;

[0198] If the drift amount is greater than or equal to the third preset threshold, determine the third prompt information, the third preset threshold is greater than the second preset threshold, and the third prompt information is used to prompt the user that the calibration object drifts too high.

[0199] As a possible design, the first determination module 72 is further configured to

[0200] When the preset condition is met, update the reference coordinate system pose information according to the steps S1-S5;

[0201] The preset condition includes at least one of the following: in response to the user's operation instruction for updating the reference coordinate system pose information, detecting that the drift amount between the current coordinate system pose information and the reference coordinate system pose information is greater than the second preset threshold, the time length from the last time the reference coordinate system pose information is determined reaches the first preset time length.

[0202] As a possible design, the acquisition module is specifically configured to any one of the following:

[0203] acquire the current coordinate system pose information of the calibration object based on the first camera within a second preset time length before the first pose information is acquired;

[0204] acquire the current coordinate system pose information of the calibration object based on the first camera within a third preset time length after the first pose information is acquired;

[0205] when it is detected that the pose information of the last to-be-recognized object is inaccurate after the last to-be-recognized object is compensated for the last coordinate system pose information, acquire the current coordinate system pose information of the calibration object based on the first camera.

[0206] As a possible design, the distance value between the first long side of the first plane and the second long side of the second plane is equal to the second long side corresponding length value of the preset coefficient, and the distance value between the first wide side of the first plane and the second wide side of the second plane is equal to the second wide side corresponding length value of the preset coefficient.

[0207] The object pose determination apparatus provided by the embodiments of the present disclosure can be used to execute the object pose determination method in any of the above embodiments, and has similar implementation principles and technical effects, which will not be described here.

[0208] It should be noted that the division of each module of the above apparatus is only a logical functional division, and all or part of the modules can be integrated into one physical entity, or can be physically separated. Moreover, all or part of the modules can be integrated together or independently implemented. The processing element described herein can be an integrated circuit having a signal processing capability. In the implementation process, each step of the above method or each module can be completed by the integrated logic circuit of the hardware in the processor element or the instruction in the form of software.

[0209] Figure 8 The structural schematic diagram of the robot provided by the embodiments of the present disclosure is shown in FIG. 1, which can include a processor 81, a memory 82, and computer program instructions stored in the memory 82 and executable on the processor 81, wherein the processor 81 executes the computer program instructions to implement the method provided by any of the above embodiments. Figure 8

[0210] Optionally, the above various devices of the robot can be connected through a system bus.

[0211] The memory 82 can be a separate storage unit or a storage unit integrated in the processor 81. The number of processors 81 is one or more. ​

[0212] It should be understood that the processor 81 can be a central processing unit (CPU), and can also be other general-purpose processors 81, digital signal processors 81 (DSP), application specific integrated circuits (ASIC), etc. The general-purpose processor 81 can be a microprocessor or can also be any conventional processor 81, etc. The steps of the method disclosed in combination with the present disclosure can be directly embodied as execution completed by the hardware processor 81, or executed by a combination of hardware and software modules in the processor 81.

[0213] The system bus can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The system bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is shown in the figure, but it does not mean that there is only one bus or only one type of bus. The memory 82 can include a random access memory 82 (RAM), and can also include a non-volatile memory 82 (NVM), such as at least one disk memory 82.

[0214] All or part of the steps of the above-mentioned method embodiments can be completed by program instruction related hardware. The aforementioned program can be stored in a readable memory 82. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned memory 82 (storage medium) includes a read-only memory 82 (ROM), a RAM, a flash memory 82, a hard disk, a solid state disk, a magnetic tape (English: magnetic tape), a floppy disk (English: floppy disk), an optical disc (English: optical disc), and any combination thereof.

[0215] The robot provided by the embodiments of the present disclosure can be used to execute the object pose determination method provided by any one of the above-mentioned method embodiments, and has similar implementation principles and technical effects, which will not be described here.

[0216] The embodiments of the present disclosure provide a computer readable storage medium, which stores computer instructions, when the computer instructions run on a computer, the computer executes the above-mentioned object pose determination method.

[0217] The computer readable storage medium described above can be implemented by any type of volatile or nonvolatile storage devices or a combination thereof, such as static random access memory, electrically erasable programmable read-only memory, erasable programmable read-only memory, programmable read-only memory, read-only memory, magnetic storage, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special purpose computer.

[0218] Optionally, the readable storage medium is coupled to the processor, so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be an integral part of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the readable storage medium can also exist as discrete components in the device.

[0219] The embodiments of the present disclosure also provide a computer program product, which comprises a computer program stored in a computer readable storage medium, at least one processor can read the computer program from the computer readable storage medium, and the at least one processor executes the computer program to implement the above-mentioned object pose determination method.

[0220] It should be understood that the present disclosure is not limited to the precise construction that has been described above and shown in the accompanying drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the present disclosure is only limited by the appended claims.

Claims

1. A method for determining the pose of an object, characterized in that, The method is applied to a robot and comprises the following steps: obtaining first pose information of a to-be-recognized object in a field of view of a first camera; obtaining current coordinate system pose information of a calibration object, the current coordinate system pose information being obtained based on the first camera, a first height of the calibration object recognized by the first camera being consistent with a second height of the to-be-recognized object recognized by the first camera, the calibration object comprising at least three calibration balls, the three calibration balls being used to construct a first coordinate system of the calibration object multiple times through corresponding ball center positions thereof, determining compensation coordinate system pose information according to the current coordinate system pose information and reference coordinate system pose information, the reference coordinate system pose information being obtained in advance based on a second camera, a third height of the calibration object recognized by the second camera being consistent with the first height, the compensation coordinate system pose information being change information between the reference coordinate system pose information and the current coordinate system pose information, the reference coordinate system pose information being any one of first coordinate system pose information when a difference between all obtained first coordinate system pose information is less than a first preset threshold value; adjusting the first pose information according to the compensation coordinate system pose information to obtain second pose information of the to-be-recognized object.

2. The method of claim 1, wherein, before the step of determining the compensation coordinate system pose information according to the current coordinate system pose information and the reference coordinate system pose information, the method further comprises the following steps: S1, obtaining point cloud information of the at least three calibration balls through the second camera, a first plane formed by the at least three calibration balls being located in a second plane corresponding to a field of view of the second camera; S2, determining ball crown point cloud of the at least three calibration balls from the point cloud information of the at least three calibration balls; S3, for each calibration ball, fitting a ball center pose of the calibration ball according to the ball crown point cloud of the calibration ball; S4, determining first coordinate system pose information of a center of the at least three calibration balls based on the ball center pose of each calibration ball; S5, repeating the steps S1-S4 more than N times within a preset time period, and when a difference between all obtained first coordinate system pose information is less than a first preset threshold value, taking any one of the first coordinate system pose information as the reference coordinate system pose information, the N being a positive integer. The method further comprises the following steps:

3. The method of claim 2, wherein, during the repeating of the steps S1-S4, if a difference between the obtained first coordinate system pose information is greater than or equal to the first preset threshold value, determining first prompt information, the first prompt information being used to prompt a user to check point cloud quality and a repeat accuracy of the robot to a position. after the step of obtaining the current coordinate system pose information of the calibration object, the method further comprises the following steps:

4. The method of claim 1, wherein, determining a drift amount of the calibration object according to the current coordinate system pose information of the calibration object and reference coordinate system pose information of the calibration object; if the drift amount is greater than a second preset threshold value and less than a third preset threshold value, determining second prompt information, the second prompt information being used to prompt the user that the calibration object has a drift. ​ If the drift is greater than or equal to the third preset threshold, third prompt information is determined, the third preset threshold is greater than the second preset threshold, and the third prompt information is used to prompt the user that the calibration object drifts too high.

5. The method according to claim 2 or 3, characterized in that, The method further comprises: When a preset condition is met, the reference coordinate system pose information is updated according to the above steps S1-S5; The preset condition comprises at least one of the following: in response to an operation instruction of the user for updating the reference coordinate system pose information, detecting that the drift between the current coordinate system pose information and the reference coordinate system pose information is greater than a second preset threshold, and a time length from a time when the reference coordinate system pose information is last determined reaching a first preset time length.

6. The method according to any one of claims 1 to 4, characterized in that, The current coordinate system pose information of the calibration object comprises any one of the following: The current coordinate system pose information of the calibration object is acquired based on the first camera within a second preset time length before the first pose information is acquired; The current coordinate system pose information of the calibration object is acquired based on the first camera within a third preset time length after the first pose information is acquired; When it is detected that the pose information of the last to-be-recognized object is inaccurate after the last to-be-recognized object is compensated for the pose information based on the last compensation coordinate system pose information, the current coordinate system pose information of the calibration object is acquired based on the first camera.

7. The method of claim 2 or 3, wherein, The distance value between the first wide side of the first plane and the second wide side of the second plane is equal to the second wide side corresponding length value of the preset coefficient, and the distance value between the first long side of the first plane and the second long side of the second plane is equal to the second long side corresponding length value of the preset coefficient.

8. An apparatus for determining a pose of an object, the apparatus comprising: The device is applied to a robot and comprises: An acquisition module is configured to acquire first pose information of a to-be-recognized object in a field of view of a first camera and acquire current coordinate system pose information of a calibration object, the current coordinate system pose information being obtained based on shooting of the first camera, a first height of the calibration object recognized by the first camera being consistent with a second height of the to-be-recognized object recognized by the first camera, and the calibration object comprising at least three calibration balls used to construct a first coordinate system of the calibration object multiple times through corresponding ball center positions of the three calibration balls; A first determination module is configured to determine compensation coordinate system pose information according to the current coordinate system pose information and reference coordinate system pose information, the reference coordinate system pose information being obtained in advance based on a second camera for determining a pose of the calibration object, a third height of the calibration object recognized by the second camera being consistent with the first height, and the compensation coordinate system pose information being change information between the reference coordinate system pose information and the current coordinate system pose information; and the reference coordinate system pose information being any one of first coordinate system pose information when a difference between all obtained first coordinate system pose information is less than a first preset threshold. A second determination module is configured to adjust the first pose information according to the compensation coordinate system pose information to obtain second pose information of the to-be-recognized object.

9. A robot, characterized in that The device comprises: A processor and a memory connected to the processor in communication; The memory stores computer execution instructions; and The processor executes the computer execution instructions to perform the following steps. The processor executes computer-executable instructions stored in the memory to implement the method of any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions that, when executed by a processor, implement the method of any one of claims 1-7.

Images