Method and apparatus for determining object pose, and robot and storage medium

Abstract

A method and apparatus for determining an object pose, a robot, and a storage medium. The method is applied to a robot. The method comprises: acquiring first pose information of an object to be subjected to recognition in the field of view of a first camera; acquiring the current coordinate system pose information of a calibration object, wherein the current coordinate system pose information is obtained on the basis of photographing performed by the first camera, and a first height of the calibration object that is recognized by the first camera is consistent with a second height of the object to be subjected to recognition that is recognized by the first camera; determining compensation coordinate system pose information on the basis of the current coordinate system pose information and reference coordinate system pose information; and then adjusting the first pose information on the basis of the compensation coordinate system pose information, so as to obtain second pose information of the object to be subjected to recognition. In the technical solution, the pose of an object to be subjected to recognition is corrected by using the variation between a reference pose and the current pose, such that the pose of the object to be subjected to recognition is determined more accurately, thereby avoiding a measurement error caused by problems such as camera time changing and replacement.

Description

Method, device, robot and storage medium for determining object posture

[0001] This disclosure claims priority to a Chinese patent disclosure filed with the Patent Office of China on July 21, 2023, with application number 202310906733.0 and public name “Method, device, robot and storage medium for determining object posture”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] The present disclosure relates to the field of machine vision technology, and in particular to a method, device, robot, and storage medium for determining an object's posture. Background Art

[0003] As the camera runs for a long time or is hit, the accuracy of the camera will deteriorate. In this case, the camera needs to be recalibrated.

[0004] When recalibrating the camera, technicians need to adjust it according to some teaching templates, which is time-consuming. The above-mentioned accuracy degradation will not be discovered immediately, making supervision difficult and affecting the operational stability and efficiency of the corresponding visual system.

[0005] Therefore, how to ensure the accuracy of camera vision when grasping objects becomes a technical problem that needs to be solved urgently. Technical Solutions

[0006] The present disclosure provides a method, device, robot, and storage medium for determining the posture of an object, so as to solve the problem of how to ensure the accuracy of camera vision when grasping an object.

[0007] In a first aspect, an embodiment of the present disclosure provides a method for determining an object's posture, which is applied to a robot. The method includes:

[0008] Obtain the first pose information of the object to be identified in the field of view of the first camera;

[0009] Acquire current coordinate system pose information of the calibration object, where the current coordinate system pose information is obtained based on the first camera, and a first height of the calibration object recognized by the first camera is consistent with a second height of the object to be recognized by the first camera;

[0010] Determining, based on the current coordinate system pose information and the reference coordinate system pose information, compensation coordinate system pose information, wherein the reference coordinate system pose information is obtained by pre-determining the pose of the calibration object based on a second camera, the second camera recognizes that the third height of the calibration object is consistent with the first height, and the compensation coordinate system pose information is change information between the reference coordinate system pose information and the current coordinate system pose information;

[0011] The first posture information is adjusted according to the compensation coordinate system posture information to obtain second posture information of the object to be identified.

[0012] In a possible design, the calibration object includes at least three calibration balls;

[0013] Before determining the compensation coordinate system pose information based on the current coordinate system pose information and the reference coordinate system pose information, the method further includes:

[0014] S1, acquiring point cloud information of the at least three calibration spheres through the second camera, where a first plane formed by the at least three calibration spheres is located in a second plane corresponding to the field of view of the second camera;

[0015] S2, determining spherical cap point clouds of the at least three calibration spheres from the point cloud information of the at least three calibration spheres;

[0016] S3, for each calibration sphere, fitting the center pose of the calibration sphere according to the spherical crown point cloud of the calibration sphere;

[0017] S4, determining first coordinate system pose information of the centers of the at least three calibration spheres based on the pose of the center of each calibration sphere;

[0018] S5. Repeat steps S1-S4 above N times within a preset time, and when the difference between all the obtained first coordinate system pose information is less than a first preset threshold, use any one of the first coordinate system pose information as the reference coordinate system pose information, where N is an integer greater than 0.

[0019] In a possible design, the method further includes:

[0020] In the process of repeating the above steps S1-S4, if the difference between the obtained first coordinate system pose information is greater than or equal to the first preset threshold, a first prompt information is determined, and the first prompt information is used to prompt the user to check the point cloud quality and the positioning repeatability of the robot.

[0021] In a possible design, after obtaining the current coordinate system pose information of the calibration object, the method further includes:

[0022] Determining a drift amount of the calibration object according to the current coordinate system pose information of the calibration object and the reference coordinate system pose information of the calibration object;

[0023] If the drift amount is greater than a second preset threshold and less than a third preset threshold, determining a second prompt message, wherein the second prompt message is used to prompt the user that the calibration object has drifted;

[0024] If the drift amount is greater than or equal to the third preset threshold, a third prompt message is determined. The third preset threshold is greater than the second preset threshold. The third prompt message is used to prompt the user that the calibration object drifts too high.

[0025] In a possible design, the method further includes:

[0026] When the preset conditions are met, the reference coordinate system pose information is updated according to the above steps S1-S5;

[0027] Among them, the preset conditions include at least one of the following: responding to the user's operation instruction to update the reference coordinate system posture information, detecting that the drift between the current coordinate system posture information and the reference coordinate system posture information is greater than a second preset threshold, and the time from the last time the reference coordinate system posture information was determined reaches a first preset time.

[0028] In a possible design, obtaining the current coordinate system pose information of the calibration object includes any of the following:

[0029] Acquiring the current coordinate system pose information of the calibration object based on the first camera within a second preset time period before acquiring the first pose information;

[0030] Acquiring current coordinate system pose information of the calibration object based on the first camera within a third preset time period after acquiring the first pose information;

[0031] After compensating the pose information of the last object to be identified based on the last compensated coordinate system pose information, if it is detected that the pose information of the last object to be identified is inaccurate, the current coordinate system pose information of the calibration object is acquired based on the first camera.

[0032] As a possible design, the distance value between the first wide side of the first plane and the second wide side of the second plane is equal to the length value corresponding to the second wide side of the preset coefficient, and the distance value between the first long side of the first plane and the second long side of the second plane is equal to the length value corresponding to the second long side of the preset coefficient.

[0033] In a possible design, the method further includes:

[0034] According to the drift amount of the calibration object, the fluctuation difference of the calibration object in each direction in the coordinate system is displayed on the front end.

[0035] In a second aspect, an embodiment of the present disclosure provides a device for determining an object's posture, which is applied to a robot. The device includes:

[0036] an acquisition module, configured to acquire first pose information of the object to be identified within the field of view of the first camera; and acquire current coordinate system pose information of the calibration object, wherein the current coordinate system pose information is obtained based on the first camera, and a first height of the calibration object identified by the first camera is consistent with a second height of the object to be identified by the first camera;

[0037] a first determining module, configured to determine, based on the current coordinate system pose information and the reference coordinate system pose information, compensation coordinate system pose information, wherein the reference coordinate system pose information is obtained by pre-determining the pose of the calibration object based on a second camera, the second camera recognizes that the third height of the calibration object is consistent with the first height, and the compensation coordinate system pose information is change information between the reference coordinate system pose information and the current coordinate system pose information;

[0038] The second determination module is configured to adjust the first posture information according to the compensation coordinate system posture information to obtain second posture information of the object to be identified.

[0039] In a possible design, the calibration object includes at least three calibration balls;

[0040] Before determining the compensation coordinate system pose information based on the current coordinate system pose information and the reference coordinate system pose information, the first determining module is further configured to:

[0041] S1, acquiring point cloud information of the at least three calibration spheres through the second camera, where a first plane formed by the at least three calibration spheres is located in a second plane corresponding to the field of view of the second camera;

[0042] S2, determining spherical cap point clouds of the at least three calibration spheres from the point cloud information of the at least three calibration spheres;

[0043] S3, for each calibration sphere, fitting the center pose of the calibration sphere according to the spherical crown point cloud of the calibration sphere;

[0044] S4, determining first coordinate system pose information of the centers of the at least three calibration spheres based on the pose of the center of each calibration sphere;

[0045] S5. Repeat steps S1-S4 above N times within a preset time, and when the difference between all the obtained first coordinate system pose information is less than a first preset threshold, use any one of the first coordinate system pose information as the reference coordinate system pose information, where N is an integer greater than 0.

[0046] In a possible design, the first determining module is further configured to:

[0047] In the process of repeating the above steps S1-S4, if the difference between the obtained first coordinate system pose information is greater than or equal to the first preset threshold, a first prompt information is determined, and the first prompt information is used to prompt the user to check the point cloud quality and the positioning repeatability of the robot.

[0048] In a possible design, after obtaining the current coordinate system pose information of the calibration object, the first determination module is further configured to:

[0049] Determining a drift amount of the calibration object according to the current coordinate system pose information of the calibration object and the reference coordinate system pose information of the calibration object;

[0050] If the drift amount is greater than a second preset threshold and less than a third preset threshold, determining a second prompt message, wherein the second prompt message is used to prompt the user that the calibration object has drifted;

[0051] If the drift amount is greater than or equal to the third preset threshold, a third prompt message is determined. The third preset threshold is greater than the second preset threshold. The third prompt message is used to prompt the user that the calibration object drifts too high.

[0052] In a possible design, the first determining module is further configured to:

[0053] When the preset conditions are met, the reference coordinate system pose information is updated according to the above steps S1-S5;

[0054] Among them, the preset conditions include at least one of the following: responding to the user's operation instruction to update the reference coordinate system posture information, detecting that the drift between the current coordinate system posture information and the reference coordinate system posture information is greater than a second preset threshold, and the time from the last time the reference coordinate system posture information was determined reaches a first preset time.

[0055] In a possible design, obtaining the current coordinate system pose information of the calibration object includes any of the following:

[0056] Acquiring the current coordinate system pose information of the calibration object based on the first camera within a second preset time period before acquiring the first pose information;

[0057] Acquiring current coordinate system pose information of the calibration object based on the first camera within a third preset time period after acquiring the first pose information;

[0058] After compensating the pose information of the last object to be identified based on the last compensated coordinate system pose information, if it is detected that the pose information of the last object to be identified is inaccurate, the current coordinate system pose information of the calibration object is acquired based on the first camera.

[0059] As a possible design, the distance value between the first wide side of the first plane and the second wide side of the second plane is equal to the length value corresponding to the second wide side of the preset coefficient, and the distance value between the first long side of the first plane and the second long side of the second plane is equal to the length value corresponding to the second long side of the preset coefficient.

[0060] In a possible design, the first determining module is further configured to:

[0061] According to the drift amount of the calibration object, the fluctuation difference of the calibration object in each direction in the coordinate system is displayed on the front end.

[0062] In a third aspect, the present disclosure provides a robot comprising: a processor, and a memory communicatively connected to the processor;

[0063] The memory stores computer-executable instructions;

[0064] The processor executes the computer-executable instructions stored in the memory to implement the method as described in the first aspect or any one of the above methods.

[0065] In a fourth aspect, the present disclosure provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the method described in the first aspect or any one of the above methods.

[0066] In a fifth aspect, the present disclosure provides a computer program product, comprising a computer program, which, when executed by a processor, implements the method described in the first aspect or any one of the above methods.

[0067] The present disclosure provides a method, device, robot, and storage medium for determining the posture of an object. The method is applied to a robot and obtains the first pose information of an object to be identified in the field of view of a first camera, and obtains the current coordinate system pose information of a calibration object. The current coordinate system pose information is obtained based on the first camera. The first height of the calibration object recognized by the first camera is consistent with the second height of the object to be identified by the first camera. The compensation coordinate system pose information is determined based on the current coordinate system pose information and the reference coordinate system pose information. The reference coordinate system pose information is obtained by pre-determining the posture of the calibration object based on the second camera. The third height of the calibration object recognized by the second camera is consistent with the first height. The compensation coordinate system pose information is the change information between the reference coordinate system pose information and the current coordinate system pose information. The first pose information is then adjusted based on the compensation coordinate system pose information to obtain the second pose information of the object to be identified. In this technical solution, the change between the reference pose and the current pose is used to correct the posture of the object to be identified, so as to more accurately determine the posture of the object to be identified and avoid measurement errors caused by problems such as camera time variation and replacement. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0069] FIG1 is a flow chart of a method for determining an object posture according to an embodiment of the present disclosure;

[0070] FIG2 is a schematic diagram of the field of view of a first camera provided in an embodiment of the present disclosure;

[0071] FIG3 is a schematic diagram of the field of view of a second camera provided in an embodiment of the present disclosure;

[0072] FIG4 is a second flow chart of a method for determining an object posture according to an embodiment of the present disclosure;

[0073] Figure 5 is a schematic diagram of the installation of the calibration object;

[0074] FIG6 is a third flow chart of a method for determining an object posture according to an embodiment of the present disclosure;

[0075] FIG7 is a schematic structural diagram of an apparatus for determining an object posture according to an embodiment of the present disclosure;

[0076] FIG8 is a schematic structural diagram of a robot provided by an embodiment of the present disclosure.

[0077] The above drawings illustrate specific embodiments of the present disclosure, which will be described in more detail below. These drawings and textual descriptions are not intended to limit the scope of the present disclosure in any way, but rather to illustrate the concepts of the present disclosure to those skilled in the art by reference to specific embodiments. Modes for Carrying Out the Invention

[0078] To make the objectives, technical solutions, and advantages of the embodiments of the present disclosure more clear, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present disclosure without making any creative efforts shall fall within the scope of protection of the present disclosure.

[0079] Before introducing the embodiments of the present disclosure, the application background of the embodiments of the present disclosure is first explained:

[0080] After long-term operation, the accuracy of the camera may deteriorate due to time-varying phenomena such as temperature drift, or / and collisions, and the camera needs to be replaced. The camera parameters need to be recalibrated to improve the camera's accuracy.

[0081] When recalibrating the camera, technicians need to adjust it according to some teaching templates. However, this debugging method has the following technical problems:

[0082] 1. Changes in camera parameters caused by time-varying phenomena such as temperature drift are generally not easy to detect and may only be discovered when there is a major problem;

[0083] 2. Recalibrating the camera and its teaching template is a heavy workload, resulting in a large loss of manpower and financial resources;

[0084] 3. Lack of error monitoring mechanism at the operation site;

[0085] 4. The accurate position of the object cannot be corrected in time when grasping the object.

[0086] In response to the technical problems existing in the prior art, the inventors of the present invention have the following ideas: in the process of modulating the camera accuracy, the inventors found that if the posture of the calibration object can be determined in advance using the camera according to the most standard calibration object, in subsequent measurements, the standard posture and the currently determined posture of the calibration object can be used at any time to determine the possible accuracy error of the camera, so as to realize the correction of the real-time object to be identified; and in the scenario of camera replacement, the standard posture and the real-time posture of the replaced camera can also be used to determine the accuracy error to correct the object to be identified; in addition, according to the change between the current posture of the calibration object and the standard posture, the corresponding warning threshold can be assigned to realize the supervision mechanism of the operation site.

[0087] It should be understood that the method disclosed herein can be used in EyeInHand high-precision 3D visual scenarios.

[0088] The technical solution of the present disclosure is described in detail below through specific embodiments. It should be noted that the following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

[0089] It is worth noting that the application fields of the object posture determination method, device, robot and storage medium disclosed in the present invention are not limited.

[0090] Among them, the execution subject of the present disclosure is a robot, specifically it can be a control unit in the robot, or a controller that controls the robot, etc.

[0091] FIG1 is a flow chart of a method for determining an object's posture according to an embodiment of the present disclosure. As shown in FIG1 , the method for determining an object's posture may include the following steps:

[0092] Step 11: Obtain the first pose information of the object to be identified in the field of view of the first camera;

[0093] Step 12: Obtain the current coordinate system pose information of the calibration object;

[0094] The current coordinate system pose information is obtained based on the first camera's capture, and the first height of the calibration object recognized by the first camera is consistent with the second height of the object to be recognized by the first camera.

[0095] In the above two steps, the first camera may have accuracy errors due to long-term operation, or the first camera is a new camera that has been replaced. In this case, the first camera needs to be used to measure the actual posture information of the object to be identified.

[0096] First, the first pose information of the object to be identified in the field of view of the first camera is obtained by using the first camera, and the pose information of the calibration object in the current coordinate system is obtained.

[0097] Optionally, obtaining the current coordinate system pose information may include any of the following:

[0098] 1. Acquire the current coordinate system pose information of the calibration object based on the first camera within a second preset time period before acquiring the first pose information;

[0099] 2. Acquire the current coordinate system pose information of the calibration object based on the first camera within a third preset time period after acquiring the first pose information;

[0100] 3. After compensating the pose information of the last object to be identified based on the last compensated coordinate system pose information, if the pose information of the last object to be identified is inaccurate, obtain the current coordinate system pose information of the calibration object based on the first camera.

[0101] For example, the first camera takes a picture of the calibration object after taking a picture of the object to be identified, or takes a picture of the calibration object before taking a picture of the object to be identified; the time of taking the picture of the object to be identified is 11:00 on the 2nd, within 10 hours before the first camera took the picture of the object to be identified, or within 10 hours after the first camera took the picture of the object to be identified; when it is detected that the recognition error of the object to be identified last time is too large, the calibration object is photographed, for example, the object to be identified is not accurately grasped (this implementation can be that the object to be identified is still not grasped after compensating for the position information of the previous compensation coordinate system).

[0102] Optionally, the calibration object may be an object that can be established into a coordinate system, for example, at least three calibration spheres, some objects with properties that can be used to construct a coordinate system, and so on.

[0103] For example, Figure 2 is a schematic diagram of the field of view of the first camera provided in an embodiment of the present disclosure. As shown in Figure 2, the schematic diagram includes: a first camera (for example, camera 21), a calibration object 23, an object to be identified 22, a first height H1, and a second height H2.

[0104] That is, the first height of the calibration object 23 recognized by the camera 21 is H 1, and the second height of the object to be recognized 22 recognized by the camera 21 is H 2, and H 1=H 2.

[0105] It should be understood that the first camera may be a camera that has been used by the relevant system (ie, the second camera described below, which will be described in detail below); or it may be a camera that is replaced by the second camera due to some reasons.

[0106] Alternatively, in different height scenarios, taking k layers (each layer has different heights) as an example, if the workpieces on the k layers share a photo point, there will be k different photo (workpiece / calibration object) distances.

[0107] As an example, the calibration object may be at least three calibration balls, and the first height is the distance between the center of a plane formed by the at least three calibration balls and the first camera.

[0108] Optionally, if the above-mentioned first height is less than the height threshold, the size of each calibration ball is 60 mm; if the first height is greater than or equal to the height threshold, the size of each calibration ball is 100 mm.

[0109] As an example, the height threshold may be 2 m and the dimension may be the diameter of a calibration sphere.

[0110] Optionally, the at least three calibration balls include at least one of the following: a matte iron ball and a ceramic ball.

[0111] To avoid external factors affecting camera accuracy, point cloud acquisition-friendly matte iron balls or ceramic balls can be used for calibration objects. When the distance between the camera and the calibration object (i.e., the first height, the second height, or the third height) is greater than or equal to 2m, three matte iron balls with a size of 100mm are selected; when the distance between the camera and the calibration object (i.e., the first height) is less than 2m, three matte iron balls with a size of 60mm are selected.

[0112] For example, the first pose information is (10, 12, 10); the current coordinate system pose information is (5, 6, 5).

[0113] Step 13: Determine the compensation coordinate system pose information based on the current coordinate system pose information and the reference coordinate system pose information.

[0114] The reference coordinate system pose information is obtained by pre-determining the pose of the calibration object based on the second camera. The second camera recognizes that the third height of the calibration object is consistent with the first height. The compensation coordinate system pose information is the change information between the reference coordinate system pose information and the current coordinate system pose information.

[0115] In this step, the reference coordinate system pose information is the standard pose information of the coordinate system determined by the second camera taking pictures of the calibration object when the second camera does not drift.

[0116] Optionally, the current coordinate system pose information determined in real time by the first camera, or the current coordinate system pose information previously determined by the camera, and the reference coordinate system pose information determined in advance by the second camera can be used to determine the change information between the reference coordinate system pose information and the current coordinate system pose information.

[0117] For example, continuing with the above example, if the reference coordinate system pose information is (5, 5, 5), then the compensation coordinate system pose information is (0, -1, 0).

[0118] FIG3 is a schematic diagram of the field of view of the second camera provided in an embodiment of the present disclosure. As shown in FIG3 , the schematic diagram includes: a second camera (eg, camera 31 ), a calibration object 23 , and a third height H 3 .

[0119] If camera 31 is camera 21 (this scenario is the scenario of performing posture compensation on the camera), if camera 31 is not camera 21 (this scenario is the scenario of replacing the camera).

[0120] For example, when a camera cannot be repaired due to a collision or other reasons and needs to be replaced, based on this solution, after replacing the camera, you only need to re-collect the calibration object to accurately grasp the object to be identified (no other additional operations are required, such as: re-calibrating external parameters, re-teaching templates); periodically (for example, daily or weekly) collect the data of the calibration object, and correct the grasping posture of the object to be identified based on the changes of the calibration object in the camera coordinate system.

[0121] It should be understood that the second camera can be the first camera or other cameras, depending on the corresponding scene.

[0122] For example, reference coordinate system pose information (such as SphereDatumCoordinatePose); current coordinate system pose information (such as SphereRealtimeCoordinatePose).

[0123] Step 14: Adjust the first pose information according to the compensated coordinate system pose information to obtain the second pose information of the object to be identified.

[0124] In this step, after determining the compensated coordinate system pose information based on the reference coordinate system pose information and the current coordinate system pose information, the compensated coordinate system pose information is used to adjust (i.e., compensate) the first pose information corresponding to the object to be identified to obtain the actual pose information of the object to be identified, i.e., the second pose information.

[0125] For example, if the first pose information is (10, 12, 10) and the pose information of the compensated coordinate system is (0, -1, 0), then the second pose information is (10, 11, 10).

[0126] The method for determining the object posture provided by the embodiment of the present disclosure is applied to a robot, by obtaining the first pose information of the object to be identified in the field of view of the first camera, and obtaining the current coordinate system pose information of the calibration object, the current coordinate system pose information is obtained based on the first camera shooting, the first height of the calibration object recognized by the first camera is consistent with the second height of the object to be identified by the first camera, and the compensation coordinate system pose information is determined based on the current coordinate system pose information and the reference coordinate system pose information, the reference coordinate system pose information is obtained in advance based on the pose of the calibration object determined by the second camera, the third height of the calibration object recognized by the second camera is consistent with the first height, the compensation coordinate system pose information is the change information between the reference coordinate system pose information and the current coordinate system pose information, and then the first pose information is adjusted according to the compensation coordinate system pose information to obtain the second pose information of the object to be identified. In this technical solution, the pose of the object to be identified is corrected by using the change between the reference pose and the current pose to more accurately determine the pose of the object to be identified, avoiding measurement errors caused by problems such as camera time variation and replacement.

[0127] On the basis of the above embodiment, the calibration object includes: at least three calibration spheres, and the current coordinate system pose information may be the pose information of the coordinate system of the center of the plane formed by the at least three calibration spheres.

[0128] FIG4 is a second flow chart of a method for determining an object posture according to an embodiment of the present disclosure. As shown in FIG4 , before step 13, the method for determining an object posture may further include the following steps:

[0129] That is, in this solution, before step 13 above, the reference coordinate system pose information needs to be determined:

[0130] S1, acquiring point cloud information of at least three calibration spheres through a second camera.

[0131] Wherein, the first plane formed by the at least three calibration spheres is located in the second plane corresponding to the field of view of the second camera;

[0132] Optionally, the distance between the first wide side of the first plane and the second wide side of the second plane is equal to the length value corresponding to the second wide side of the preset coefficient, and the distance between the first long side of the first plane and the second long side of the second plane is equal to the length value corresponding to the second long side of the preset coefficient.

[0133] For example, Figure 5 is a schematic diagram of the installation of the calibration object. As shown in Figure 5, it includes: camera 31, ball 1, ball 2, ball 3 (illustrated with 3 small balls), a first plane S1, a second plane S2, the center O of the first plane S1, the second wide side corresponding to the length value W, and the second long side corresponding to the length value L.

[0134] For example, the preset coefficient is 0.1.

[0135] Optionally, the preset coefficient can be adjusted based on the possible size of the object to be identified. If the object to be identified is small, the preset coefficient can be increased so that the calibration sphere converges toward the center of the camera's field of view; if the object to be identified is large, the preset coefficient can be decreased so that the calibration sphere moves away from the center of the camera's field of view.

[0136] It should be understood that the implementation of this step can also adjust the parameters of the second camera so that the point cloud quality of the calibration sphere is optimal and the point cloud of the hemispherical surface is complete and smooth.

[0137] S2, determining spherical cap point clouds of at least three calibration spheres from point cloud information of at least three calibration spheres;

[0138] That is, for each calibration sphere, the point cloud of the spherical cap is extracted from the point cloud within the 3D region of interest (RoI) of the calibration object, and the spherical cap point cloud of the calibration object is obtained.

[0139] S3, for each calibration sphere, fitting the center pose of the calibration sphere according to the spherical crown point cloud of the calibration sphere;

[0140] The spherical crown point cloud of the calibration sphere is fitted to obtain the spherical center pose of the calibration sphere.

[0141] S4, determining first coordinate system pose information of the centers of at least three calibration spheres based on the pose of the center of each calibration sphere;

[0142] As an example, taking three calibration spheres as an example, the center position of each calibration sphere is known. Here, the coordinates of the three corners of the triangle can be used to obtain the coordinates of the center point of the triangle, that is, the first coordinate system position information of the center of the three calibration spheres is obtained.

[0143] In another possibility of this implementation, the center poses of multiple calibration spheres can be determined, and three calibration spheres with good point cloud data are selected to determine the pose information of the first coordinate system.

[0144] S5, repeatedly performing steps S1-S4 above N times within a preset time period, and when the difference between all obtained first coordinate system pose information is less than a first preset threshold, taking any first coordinate system pose information as the reference coordinate system pose information;

[0145] Where N is an integer greater than 0

[0146] Optionally, a higher calibration object (e.g., calibration ball) recognition repeatability accuracy (in a short time) is a prerequisite for this solution, that is, a preset time length (e.g., within 3 minutes) can be used to repeat the above S1-S4, which can be repeated more than once, and the obtained first coordinate system pose information is compared to compare whether the difference between all the obtained first coordinate system pose information is less than a first preset threshold (e.g., 0.3 mm).

[0147] If the difference between all the obtained first coordinate system pose information is less than 0.3 mm, any first coordinate system pose information can be used as the reference coordinate system pose information.

[0148] For example, the last first coordinate system pose information may be used as the reference coordinate system pose information.

[0149] It should be understood that the reference coordinate system pose information, which is the calibration sphere coordinate system pose recorded at the initial moment when the entire system is intact, will not be modified once the coordinate system is determined, unless the pose needs to be updated due to reasons such as the calibration sphere being moved. The above-mentioned current coordinate system pose information will generate a new calibration sphere coordinate system pose for each subsequent calibration sphere capture, and will overwrite the previous current coordinate system pose information (overwriting and updating is achieved by setting the "Maximum number of files" parameter to 1 in the "Save results to file" in Mech_Vision).

[0150] In addition, in the process of repeating the above steps S1-S4, if the difference between the obtained first coordinate system pose information is greater than or equal to the first preset threshold, the first prompt information is determined, and the first prompt information is used to prompt the user to check the point cloud quality and the robot's positioning repeatability.

[0151] That is, if the difference between the obtained first coordinate system pose information is not less than 0.3 mm, the user needs to be reminded to check whether there is a problem with the point cloud quality; and whether the robot carrying the second camera reaches the preset photo position when repeatedly performing the above steps.

[0152] The prompting method of the prompting information involved in the present disclosure can be lighting lights of different colors, voice prompts, panel displays, etc., and the method is not limited.

[0153] In addition, the method for determining the object posture may further include the following operation: when a preset condition is met, updating the reference coordinate system posture information according to the above steps S1-S5.

[0154] Among them, the preset conditions include at least one of the following: responding to the user's operation instruction to update the reference coordinate system posture information, detecting that the drift between the current coordinate system posture information and the reference coordinate system posture information is greater than a second preset threshold, and the time since the last determination of the reference coordinate system posture information reaches a first preset time.

[0155] For example, when the calibration object is accidentally touched by the user or there is an offset in the physical space, the user can input operation instructions to the robot to instruct the robot to update the reference coordinate system pose information; when the robot detects that the calibration object's pose information drifts in the initial and current state, and the drift is greater than 0.2mm (the second preset threshold), the robot is instructed to update the reference coordinate system pose information; if it is unknown whether the calibration object has changed, the reference coordinate system pose information can be updated 10 days (the first preset time length) after the reference coordinate system pose information was last determined; the calibration object loose displacement check (which can be preliminarily judged through the relevant error monitoring functions), if the calibration object is loose, the reference coordinate system pose information needs to be updated after reinforcement, etc.

[0156] Optionally, the fluctuation difference between the current posture and the reference posture of the drift amount of the calibration object can output the maximum difference of X, Y, and Z respectively for the front-end display error discount diagram.

[0157] The method for determining the object pose provided by the embodiment of the present disclosure includes: S1, obtaining point cloud information of at least three calibration spheres through a second camera, and the first plane formed by the at least three calibration spheres is located in the second plane corresponding to the field of view of the second camera; S2, determining the spherical crown point clouds of at least three calibration spheres from the point cloud information of the at least three calibration spheres; S3, fitting the spherical center pose of each calibration sphere based on the spherical crown point cloud of the calibration sphere; S4, determining the first coordinate system pose information of the center of at least three calibration spheres based on the spherical center pose of each calibration sphere; S5, repeating the above steps S1-S4 more than N times within a preset time length, and when the difference between all the obtained first coordinate system pose information is less than a first preset threshold, using any first coordinate system pose information as the reference coordinate system pose information, wherein N is an integer greater than 0. In this technical solution, the reference coordinate system pose is accurately determined, which is used to compensate for the pose of the object to be identified due to time change or replacement of the camera.

[0158] Based on the above embodiment, FIG6 is a flow chart of the third method for determining the object posture provided by the embodiment of the present disclosure. As shown in FIG6 , after the above step 11, the method for determining the object posture may further include the following steps:

[0159] Step 61: Determine the drift of the calibration object based on the current coordinate system pose information of the calibration object and the reference coordinate system pose information of the calibration object;

[0160] In this solution, the calibration object can be tested for drift within the camera's field of view to remind the user of the state of the calibration object and whether it affects the accuracy of the relevant system.

[0161] In this step, the difference between the current coordinate system pose information and the reference coordinate system pose information can be used as the drift of the calibration object.

[0162] Taking the calibration objects as at least three calibration balls as an example, for any calibration ball, for example, ball 1, the method involved in the above embodiment is used to determine the drift between the ball center poses in the two messages. For example, the drift is 0.3 or 0.6.

[0163] Step 62: If the drift amount is greater than the second preset threshold and less than the third preset threshold, determine the second prompt information.

[0164] The second prompt information is used to remind the user that the calibration object has drifted;

[0165] In this step, if the value is between the second preset threshold and the third preset threshold, it may indicate that the calibration ball is drifting, but the impact on the operation of the system is not significant. At this time, an alarm may be issued, but the system will not stop.

[0166] For example, the second preset threshold is 0.2, and the third preset threshold is 0.5.

[0167] That is, for example, if the drift amount of ball 1 is 0.3, the user is prompted that ball 1 is drifting.

[0168] Step 63: If the drift amount is greater than or equal to a third preset threshold, determine a third prompt message.

[0169] The third prompt information is used to prompt the user that the calibration object drifts too high, and the third preset threshold is greater than the second preset threshold.

[0170] In this step, if the value is greater than or equal to the third preset threshold, it may indicate that the calibration ball is drifting and has a significant impact on the operation of the system. In this case, an alarm may be issued to stop the system operation.

[0171] That is, for example, if the drift amount of ball 3 is 0.6, the user is prompted that ball 3 is drifting and the system has been stopped.

[0172] If the drift is too large or a serious fault occurs, the drift will generally be reset to zero after the cause is identified and repaired (the current correction method should be eliminated during the re-teaching process). After the drift is reset to zero, the reference coordinate system pose information needs to be updated.

[0173] It should be understood that the execution of step 62 and step 63 is to select one of them according to the actual situation.

[0174] The method for determining the object posture provided by the embodiment of the present disclosure determines the drift of the calibration object based on the current coordinate system posture information of the calibration object and the reference coordinate system posture information of the calibration object. If the drift is greater than a second preset threshold and less than a third preset threshold, a second prompt message is determined, and the second prompt message is used to prompt the user that the calibration object has drifted. If the drift is greater than or equal to the third preset threshold, a third prompt message is determined, and the third preset threshold is greater than the second preset threshold. The third prompt message is used to prompt the user that the calibration object has drifted too much. In this technical solution, by determining the warning threshold, a corresponding warning is achieved when the camera accuracy changes, which facilitates the user to maintain the camera in real time and avoids the serious consequences that may be caused by camera accuracy in the prior art.

[0175] The following are some implementation details of the technical solutions involved in this disclosure based on the above embodiments (the implementation principles and technical effects are the same as above):

[0176] First, the selection of calibration objects (such as calibration spheres):

[0177] The material can be point cloud quality-friendly, for example, matte iron or ceramic spheres. The size can be the distance from the calibration sphere to the camera: for less than 2 meters, a 60mm diameter is selected, and for spheres no less than 2 meters, a 100mm diameter is selected. The distance from the calibration sphere to the camera can be determined to be consistent with the distance from the workpiece to the camera. If k layers of workpieces share a single photographic point, there are k different photographic distances (for photographing the workpiece or the calibration sphere).

[0178] Second, install the calibration object:

[0179] (Take Figure 2 as an example) The line between balls 2 and 3 is parallel to the camera baseline. The distance from the calibration ball to the edge of the camera field of view is approximately 0.2 times the field of view size. If the workpiece is small, the three calibration balls can be appropriately gathered towards the center of the field of view.

[0180] Third, the collection of calibration objects:

[0181] The photographing height of the calibration sphere is related to the photographing height of the workpiece. Each workpiece height within the camera's field of view corresponds to a calibration sphere of similar height (the coordinate system generated by the three calibration spheres). The workpiece and calibration sphere are both located at the center of the camera's field of view. When photographing, the camera's optical axis is as perpendicular to the workpiece and calibration sphere as possible. The acquisition frequency of the calibration sphere can be daily or weekly.

[0182] Fourth, adjustment of camera parameters:

[0183] Adjust the camera parameters to optimize the quality of the calibration sphere point cloud and ensure that the hemispherical surface point cloud is complete and smooth.

[0184] Fifth, maintenance of calibration objects:

[0185] The calibration ball must be securely installed and protected from being touched. The calibration ball must be protected from dust daily. If dust is visible on the surface, it is recommended to blow it away with compressed air instead of wiping it hard.

[0186] The following are embodiments of the apparatus disclosed herein, which can be used to implement the method embodiments disclosed herein. For details not disclosed in the apparatus embodiments disclosed herein, please refer to the method embodiments disclosed herein.

[0187] FIG7 is a schematic diagram of the structure of the device for determining the position and posture of an object provided by an embodiment of the present disclosure. As shown in FIG7 , the device for determining the position and posture of an object is applied to a robot and includes:

[0188] An acquisition module 71 is configured to acquire first pose information of the object to be identified within the field of view of the first camera, and acquire current coordinate pose information of the calibration object, where the current coordinate pose information is obtained based on the first camera's capture, and a first height of the calibration object identified by the first camera is consistent with a second height of the object to be identified by the first camera;

[0189] a first determining module 72 for determining, based on the current coordinate system pose information and the reference coordinate system pose information, the reference coordinate system pose information being pre-determined based on the pose of the calibration object by the second camera, the second camera recognizing that the third height of the calibration object is consistent with the first height, and the compensation coordinate system pose information being the change information between the reference coordinate system pose information and the current coordinate system pose information;

[0190] The second determination module 73 is configured to adjust the first pose information according to the compensated coordinate system pose information to obtain second pose information of the object to be identified.

[0191] As a possible design, the calibration object includes at least three calibration balls;

[0192] Before determining the compensation coordinate system pose information based on the current coordinate system pose information and the reference coordinate system pose information, the first determining module 72 is further configured to:

[0193] S1, acquiring point cloud information of at least three calibration spheres through a second camera, where a first plane formed by the at least three calibration spheres is located in a second plane corresponding to the field of view of the second camera;

[0194] S2, determining spherical cap point clouds of at least three calibration spheres from point cloud information of at least three calibration spheres;

[0195] S3, for each calibration sphere, fitting the center pose of the calibration sphere according to the spherical crown point cloud of the calibration sphere;

[0196] S4, determining first coordinate system pose information of the centers of at least three calibration spheres based on the pose of the center of each calibration sphere;

[0197] S5, within the preset time length, repeat the above steps S1-S4 N times or more, and when the difference between all the obtained first coordinate system pose information is less than the first preset threshold, use any one of the first coordinate system pose information as the reference coordinate system pose information, N is an integer greater than 0.

[0198] In a possible design, the first determining module 72 is further configured to:

[0199] In the process of repeating the above steps S1-S4, if the difference between the obtained first coordinate system pose information is greater than or equal to the first preset threshold, the first prompt information is determined, and the first prompt information is used to prompt the user to check the point cloud quality and the robot's positioning repeatability.

[0200] In a possible design, after obtaining the current coordinate system pose information of the calibration object, the first determination module 72 is further configured to:

[0201] Determine the drift of the calibration object based on the current coordinate system pose information of the calibration object and the reference coordinate system pose information of the calibration object;

[0202] If the drift amount is greater than the second preset threshold and less than the third preset threshold, a second prompt message is determined, where the second prompt message is used to prompt the user that the calibration object has drifted;

[0203] If the drift amount is greater than or equal to a third preset threshold, a third prompt message is determined. The third preset threshold is greater than the second preset threshold. The third prompt message is used to prompt the user that the calibration object drifts too high.

[0204] In a possible design, the first determining module 72 is further configured to:

[0205] When the preset conditions are met, the reference coordinate system pose information is updated according to the above steps S1-S5;

[0206] Among them, the preset conditions include at least one of the following: responding to the user's operation instruction to update the reference coordinate system posture information, detecting that the drift between the current coordinate system posture information and the reference coordinate system posture information is greater than a second preset threshold, and the time since the last determination of the reference coordinate system posture information reaches a first preset time.

[0207] As a possible design, the acquisition module is specifically used for any of the following:

[0208] Acquiring the current coordinate system pose information of the calibration object based on the first camera within a second preset time period before acquiring the first pose information;

[0209] Acquiring the current coordinate system pose information of the calibration object based on the first camera within a third preset time period after acquiring the first pose information;

[0210] After compensating the pose information of the last object to be identified based on the last compensated coordinate system pose information, if it is detected that the pose information of the last object to be identified is inaccurate, the current coordinate system pose information of the calibration object is obtained based on the first camera.

[0211] In one possible design, the distance between the first wide side of the first plane and the second wide side of the second plane is equal to the length value corresponding to the second wide side of the preset coefficient, and the distance between the first long side of the first plane and the second long side of the second plane is equal to the length value corresponding to the second long side of the preset coefficient.

[0212] In a possible design, the first determining module 72 is further configured to:

[0213] According to the drift of the calibration object, the fluctuation difference of the calibration object in each direction of the coordinate system is displayed on the front end.

[0214] The device for determining the object posture provided by the embodiments of the present disclosure can be used to execute the method for determining the object posture in any of the above embodiments. Its implementation principles and technical effects are similar and will not be repeated here.

[0215] It should be noted that it should be understood that the division of the various modules of the above device is merely a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated. Moreover, these modules can all be implemented in the form of software called by processing elements; they can also all be implemented in the form of hardware; some modules can also be implemented in the form of software called by processing elements, and some modules can be implemented in the form of hardware. In addition, these modules can be fully or partially integrated together or implemented independently. The processing element described here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each of the above modules can be completed by the hardware integrated logic circuit in the processor element or by instructions in the form of software.

[0216] Figure 8 is a structural diagram of the robot provided in an embodiment of the present disclosure. As shown in Figure 8, the robot may include: a processor 81, a memory 82, and computer program instructions stored in the memory 82 and executable on the processor 81. When the processor 81 executes the computer program instructions, the method provided in any of the aforementioned embodiments is implemented.

[0217] Optionally, the above-mentioned components of the robot can be connected via a system bus.

[0218] The memory 82 may be a separate storage unit or a storage unit integrated in the processor 81. The number of the processor 81 may be one or more.

[0219] It should be understood that the processor 81 can be a central processing unit (CPU), other general-purpose processors 81, digital signal processors 81 (DSP), application-specific integrated circuits (ASIC), etc. The general-purpose processor 81 can be a microprocessor 81 or any conventional processor 81. The steps of the method disclosed in the present disclosure can be directly implemented by the hardware processor 81 or performed by a combination of hardware and software modules in the processor 81.

[0220] The system bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, among others. A system bus can be divided into an address bus, a data bus, a control bus, and so on. For ease of illustration, the figure uses only a single thick line, but this does not imply a single bus or type of bus. Memory 82 may include random access memory 82 (RAM) and may also include non-volatile memory 82 (NVM), such as at least one disk storage device 82.

[0221] All or part of the steps in the above-described method embodiments can be implemented by hardware associated with program instructions. The aforementioned program can be stored in a readable memory 82. When executed, the program performs the steps of the above-described method embodiments. The aforementioned memory 82 (storage medium) includes: read-only memory 82 (ROM), RAM, flash memory 82, hard disk, solid-state drive, magnetic tape, floppy disk, optical disc, and any combination thereof.

[0222] The robot provided in the embodiments of the present disclosure can be used to execute the method for determining the object posture provided in any of the above-mentioned method embodiments. Its implementation principles and technical effects are similar and will not be repeated here.

[0223] An embodiment of the present disclosure provides a computer-readable storage medium, in which computer instructions are stored. When the computer instructions are executed on a computer, the computer executes the above-mentioned method for determining the position and posture of an object.

[0224] The computer-readable storage medium mentioned above may be implemented by any type of volatile or non-volatile memory device, or a combination thereof, such as static random access memory, electrically erasable programmable read-only memory, erasable programmable read-only memory, programmable read-only memory, read-only memory, magnetic storage, flash memory, magnetic disk, or optical disk. The computer-readable storage medium may be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0225] Optionally, a readable storage medium is coupled to the processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium may also be an integral part of the processor. The processor and the readable storage medium may be located in an application-specific integrated circuit (ASIC). Of course, the processor and the readable storage medium may also exist as discrete components in the device.

[0226] An embodiment of the present disclosure also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium, and when the at least one processor executes the computer program, the above-mentioned method for determining the object posture can be implemented.

[0227] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for determining the position and posture of an object, characterized in that: Applied to a robot, the method comprises: Obtain the first position information of the object to be identified in the field of view of the first camera; Acquire current coordinate system pose information of the calibration object, where the current coordinate system pose information is obtained based on the first camera, and a first height of the calibration object recognized by the first camera is consistent with a second height of the object to be recognized by the first camera; Determine compensation coordinate system pose information according to the current coordinate system pose information and the reference coordinate system pose information, wherein the reference coordinate system pose information is obtained by pre-determining the pose of the calibration object based on a second camera, the second camera recognizes that the third height of the calibration object is consistent with the first height, and the compensation coordinate system pose information is change information between the reference coordinate system pose information and the current coordinate system pose information; The first posture information is adjusted according to the compensation coordinate system posture information to obtain second posture information of the object to be identified.

2. The method according to claim 1, characterized in that The calibration object includes at least three calibration balls; Before determining the compensation coordinate system pose information according to the current coordinate system pose information and the reference coordinate system pose information, the method further includes: S1, acquiring point cloud information of the at least three calibration spheres through the second camera, wherein a first plane formed by the at least three calibration spheres is located in a second plane corresponding to the field of view of the second camera; S2, determining the spherical cap point clouds of the at least three calibration spheres from the point cloud information of the at least three calibration spheres; S3, for each calibration sphere, fitting the sphere center position of the calibration sphere according to the sphere crown point cloud of the calibration sphere; S4, determining first coordinate system pose information of the centers of the at least three calibration spheres based on the sphere center pose of each calibration sphere; S5, within a preset time length, repeat the above steps S1-S4 for more than N times, and when the differences between all the obtained first coordinate system pose information are less than the first preset threshold, use any one of the first coordinate system pose information as the reference coordinate system pose information, where N is an integer greater than 0.

3. The method according to claim 2, characterized in that The method further comprises: In the process of repeating the above steps S1-S4, if the difference between the obtained first coordinate system posture information is greater than or equal to the first preset threshold, a first prompt information is determined, and the first prompt information is used to prompt the user to check the point cloud quality and the repetitive accuracy of the robot in place.

4. The method according to claim 1, characterized in that: After obtaining the current coordinate system pose information of the calibration object, the method further includes: Determining a drift amount of the calibration object according to the current coordinate system pose information of the calibration object and the reference coordinate system pose information of the calibration object; If the drift amount is greater than a second preset threshold and less than a third preset threshold, second prompt information is determined, where the second prompt information is used to prompt the user that the calibration object has drifted; If the drift amount is greater than or equal to the third preset threshold, a third prompt message is determined, the third preset threshold is greater than the second preset threshold, and the third prompt message is used to prompt the user that the calibration object drifts too high.

5. The method according to claim 2 or 3, characterized in that: The method further comprises: When the preset conditions are met, the reference coordinate system pose information is updated according to the above steps S1-S5; Among them, the preset conditions include at least one of the following: responding to a user's operation instruction to update the reference coordinate system posture information, detecting that the drift between the current coordinate system posture information and the reference coordinate system posture information is greater than a second preset threshold, and the time from the last time the reference coordinate system posture information was determined reaches a first preset time.

6. The method according to any one of claims 1 to 4, characterized in that: The obtaining of the current coordinate system pose information of the calibration object includes any of the following: Acquiring the current coordinate system pose information of the calibration object based on the first camera within a second preset time period before acquiring the first pose information; Acquiring the current coordinate system pose information of the calibration object based on the first camera within a third preset time period after acquiring the first pose information; After compensating the posture information of the last object to be identified based on the last compensated coordinate system posture information, if it is detected that the last posture information of the object to be identified is inaccurate, the current coordinate system posture information of the calibration object is acquired based on the first camera.

7. The method according to claim 2 or 3, characterized in that: The distance between the first wide side of the first plane and the second wide side of the second plane is equal to the length value corresponding to the second wide side of the preset coefficient, and the distance between the first long side of the first plane and the second long side of the second plane is equal to the length value corresponding to the second long side of the preset coefficient.

8. The method according to claim 4, characterized in that The method further comprises: According to the drift amount of the calibration object, the fluctuation difference of the calibration object in each direction in the coordinate system is displayed on the front end.

9. A device for determining the position and posture of an object, characterized in that: Applied to a robot, the device comprises: an acquisition module, configured to acquire first pose information of the object to be identified in the field of view of the first camera; and acquire current coordinate system pose information of the calibration object, wherein the current coordinate system pose information is obtained based on the shooting of the first camera, and the first height of the calibration object identified by the first camera is consistent with the second height of the object to be identified by the first camera; A first determination module is used to determine the compensation coordinate system pose information according to the current coordinate system pose information and the reference coordinate system pose information, wherein the reference coordinate system pose information is obtained by determining the pose of the calibration object in advance based on the second camera, the second camera recognizes that the third height of the calibration object is consistent with the first height, and the compensation coordinate system pose information is the change information between the reference coordinate system pose information and the current coordinate system pose information; The second determination module is used to adjust the first posture information according to the compensation coordinate system posture information to obtain second posture information of the object to be identified.

10. A robot, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 8 when executed by a processor.

12. A computer program product, characterized in that The method comprises a computer program, which implements the method according to any one of claims 1 to 8 when executed by a processor.

Images