Coordinate system calibration method, device and equipment and computer storage medium

Abstract

The invention provides a coordinate system calibration method, device and equipment and a computer storage medium. The coordinate system calibration method comprises the steps that target point cloud data, corresponding to a target calibration plate, collected by a three-dimensional camera in the calibration process are obtained, the three-dimensional camera is fixedly installed on a tail end flange of a target robot, and the geometrical shape of the target calibration plate is an asymmetric shape; determining a first conversion relation between a camera coordinate system corresponding to the three-dimensional camera and a calibration plate coordinate system corresponding to the target calibration plate in the calibration process according to the target point cloud data; and calibrating a second conversion relation between a flange coordinate system corresponding to the tail end flange and the camera coordinate system based on the first conversion relation. Through the coordinate system calibration method, the operation precision of the target robot is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of robots of Internet of Things devices, and in particular to a coordinate system calibration method, device, equipment and computer storage medium. BACKGROUND

[0002] In the field of industrial robots, point cloud data is collected by a camera installed on the end flange of the robot to guide the movement of the robot, and therefore, it is crucial to accurately calibrate the pose conversion relationship between the camera coordinate system and the robot flange coordinate system, and the accuracy of calibration directly affects the accuracy of the robot in performing operations such as grasping and assembly based on visual feedback.

[0003] Currently, hand-eye calibration using a calibration board is a common method, among which a rectangular calibration board is widely used due to its simple structure and easy manufacturing. However, the coordinate system of the rectangular calibration board has a significant ambiguity problem, that is, when the robot drives the camera to observe the rectangular calibration board from different poses, multiple different robot poses may correspond to the same pose description of the calibration board, that is, the real pose of the calibration board in the camera coordinate system cannot be uniquely determined.

[0004] This ambiguity makes the calibration result obtained only by the rectangular calibration board have multiple ambiguous solutions, which further affects the accurate solution of the pose conversion relationship between the camera coordinate system and the robot flange coordinate system, and further leads to positioning deviation of the robot in visual guidance-based movement, resulting in low work accuracy. SUMMARY

[0005] To solve the above technical problems, the present application provides a coordinate system calibration method, device, equipment and computer storage medium.

[0006] To solve the above technical problems, the present application provides a coordinate system calibration method, which comprises:

[0007] Obtaining target point cloud data corresponding to a target calibration board collected by a three-dimensional camera in a calibration process, wherein the three-dimensional camera is fixedly installed on an end flange of a target robot, and the target calibration board has an asymmetric shape;

[0008] Determining a first conversion relationship between a camera coordinate system corresponding to the three-dimensional camera and a calibration board coordinate system corresponding to the target calibration board in the calibration process according to the target point cloud data;

[0009] Calibrating a second conversion relationship between a flange coordinate system corresponding to the end flange and the camera coordinate system based on the first conversion relationship.

[0010] The first conversion relationship between the camera coordinate system corresponding to the three-dimensional camera and the calibration board coordinate system corresponding to the target calibration board in the calibration process is determined according to the target point cloud data, and the first conversion relationship comprises:

[0011] A plurality of groups of edge point clouds corresponding to a plurality of edge line segments of the target calibration board are extracted from the target point cloud data.

[0012] A coordinate representation of an origin of the calibration board coordinate system in the camera coordinate system is determined according to the plurality of groups of edge point clouds.

[0013] A plurality of vector representations of direction vectors of a plurality of direction axes in the calibration board coordinate system in the camera coordinate system are determined according to the plurality of groups of edge point clouds and the coordinate representation.

[0014] The first conversion relationship is determined based on the coordinate representation and the plurality of vector representations.

[0015] The coordinate representation of the origin of the calibration board coordinate system in the camera coordinate system is determined according to the plurality of groups of edge point clouds, and the coordinate representation comprises:

[0016] A plurality of straight line equations corresponding to the plurality of edge line segments in the camera coordinate system are determined according to the plurality of groups of edge point clouds.

[0017] A first edge line segment and a second edge line segment satisfying a first preset condition are selected from the plurality of edge line segments according to the plurality of straight line equations.

[0018] An intersection point coordinate of a target intersection point of the first edge line segment and the second edge line segment in the camera coordinate system is determined according to the plurality of straight line equations.

[0019] The intersection point coordinate is determined as the coordinate representation.

[0020] The plurality of vector representations of the direction vectors of the plurality of direction axes in the calibration board coordinate system in the camera coordinate system are determined according to the plurality of groups of edge point clouds and the coordinate representation, and the plurality of vector representations comprise:

[0021] A first unit direction vector corresponding to the first edge line segment is determined according to a first straight line equation corresponding to the first edge line segment, when the target intersection point is taken as a starting point.

[0022] A second unit direction vector corresponding to the first edge line segment is determined according to a second straight line equation corresponding to the second edge line segment, when the target intersection point is taken as the starting point.

[0023] A third unit direction vector is determined according to the first unit direction vector and the second unit direction vector.

[0024] The first unit direction vector, the second unit direction vector and the third unit direction vector determine the plurality of vector representations.

[0025] The geometric shape of the target calibration board is a non-isosceles right triangle.

[0026] According to a plurality of straight line equations, a first edge segment and a second edge segment satisfying a first preset condition are selected from the plurality of edge segments, including:

[0027] Two edge segments are randomly selected from the plurality of edge segments, and whether the two edge segments are perpendicular is determined according to two straight line equations corresponding to the two edge segments.

[0028] In the case that the two edge segments are perpendicular, the selected two edge segments are determined as the first edge segment and the second edge segment satisfying the first preset condition.

[0029] The calibration process includes a first calibration process and a second calibration process, and the first conversion relationship includes a first conversion matrix corresponding to the first calibration process and a second conversion matrix corresponding to the second calibration process.

[0030] The second conversion relationship between the flange coordinate system corresponding to the end flange and the camera coordinate system is calibrated based on the first conversion relationship, including:

[0031] The conversion relationship between the base coordinate system of the target robot and the flange coordinate system in the first calibration process and the second calibration process is obtained respectively, to obtain a third conversion matrix and a fourth conversion matrix.

[0032] Based on the third conversion matrix and the fourth conversion matrix, a first calibration matrix is determined, wherein the first calibration matrix represents the conversion relationship between the flange coordinate system in the first calibration process and the flange coordinate system in the second calibration process.

[0033] Based on the first conversion matrix and the second conversion matrix, a second calibration matrix is determined, wherein the second calibration matrix represents the conversion relationship between the camera coordinate system in the first calibration process and the camera coordinate system in the second calibration process, to obtain a second calibration matrix.

[0034] The target conversion matrix representing the second conversion relationship is determined according to the first calibration matrix and the second calibration matrix, wherein the matrix product of the second calibration matrix and the target conversion matrix is the matrix product of the target conversion matrix and the first calibration matrix.

[0035] The target point cloud data corresponding to the target calibration board collected by the three-dimensional camera in the calibration process is obtained, including:

[0036] obtain initial point cloud data corresponding to the target calibration plate

[0037] traverse each collection point in the initial point cloud data;

[0038] In the process of traversing the current collection point, obtain a near neighbor point set in the initial point cloud data with a distance less than a first preset value from the current collection point;

[0039] In the case where the number of near neighbor points in the near neighbor point set is greater than a preset number threshold, determine the intermediate point of each near neighbor point in the near neighbor point set and the current collection point to obtain an intermediate point set;

[0040] Insert the intermediate point in the intermediate point set that meets the second preset condition into the initial point cloud data to obtain the target point cloud data.

[0041] To solve the above technical problems, the application further provides a coordinate system calibration device, which comprises:

[0042] An acquisition module is configured to acquire target point cloud data corresponding to a target calibration plate collected by a three-dimensional camera in a calibration process, wherein the three-dimensional camera is fixedly installed on an end flange of a target robot, and the target calibration plate has an asymmetric shape;

[0043] A determination module is configured to determine a first conversion relationship between a camera coordinate system corresponding to the three-dimensional camera and a calibration plate coordinate system corresponding to the target calibration plate in the calibration process according to the target point cloud data;

[0044] A calibration module is configured to calibrate a second conversion relationship between a flange coordinate system corresponding to the end flange and the camera coordinate system based on the first conversion relationship.

[0045] To solve the above technical problems, the application further provides a coordinate system calibration device, which comprises a memory and a processor coupled with the memory; wherein the memory is configured to store program data, and the processor is configured to execute the program data to realize the coordinate system calibration method as described above.

[0046] To solve the above technical problems, the application further provides a computer storage medium for storing program data, which is used to realize the coordinate system calibration method as described above when executed by a computer.

[0047] Compared with the prior art, the application has the beneficial effects that: by setting the target calibration board as a non-symmetrical shape, the point cloud data obtained under any observation angle of the three-dimensional camera can obtain a unique conversion relationship between the calibration coordinate system and the camera coordinate system, i.e., a first conversion relationship, the ambiguity of the rectangular calibration board is eliminated, the calibration result has a unique solution, there is no multiple ambiguous solutions, the accurate solution of the conversion relationship between the camera coordinate system and the robot flange coordinate system is ensured, and the working precision of the robot is improved. BRIEF DESCRIPTION OF DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0049] Among them:

[0050] Figure 1 is a schematic diagram of the calibration provided by the present application using a rectangular calibration board for calibration;

[0051] Figure 2 is a flowchart of an embodiment of the coordinate system calibration method provided by the present application;

[0052] Figure 3 is a geometric shape diagram of the target calibration board in an embodiment provided by the present application;

[0053] Figure 4 is a geometric shape diagram of the target calibration board in another embodiment provided by the present application;

[0054] Figure 5 is a schematic diagram of the target robot calibration process in an embodiment provided by the present application;

[0055] Figure 6 is a schematic diagram of the relative position of the two calibration processes of the target robot in another embodiment provided by the present application;

[0056] Figure 7 is a flowchart of the preprocessing of the initial point cloud data provided by the present application;

[0057] Figure 8 is a structural diagram of an embodiment of the coordinate system calibration device provided by the present application;

[0058] Figure 9 is a structural diagram of an embodiment of the coordinate system calibration device provided by the present application;

[0059] Figure 10is a structural schematic diagram of an embodiment of the computer storage medium provided in the present application. DETAILED DESCRIPTION

[0060] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work under the premise should be within the scope of protection of the present application.

[0061] The terms "first", "second", "third", "fourth" and the like (if any) in the specification and claims of the present application and the above drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein, for example, can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not necessarily have to include those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to the process, method, product or device.

[0062] Please refer to Figure 1 , Figure 1 is a schematic diagram of calibration in the related art using a rectangular calibration board, as shown in Figure 1 , taking a rectangular calibration board as a rectangular calibration board for example, when the camera observes the calibration board, the geometric constraint of the feature points (such as corner points) in the point cloud data collected by the camera is used to calculate the pose representation of the calibration board coordinate system in the camera coordinate system, Figure 1 , the coordinate system in (a) is the pose representation corresponding to the calibration board coordinate system calculated according to the point cloud data observed by the camera when the camera observes the rectangular calibration board at the first position, Figure 1The pose representation corresponding to the coordinate system of the calibration board calculated according to the point cloud data observed by the camera when the coordinate system of (b) is used to observe the long rectangular calibration board at a second position. The first position and the second position are two different positions, and the pose of the robot at the first position and the second position is different, for example, the robot controls the movement, and after observing the long rectangular calibration board at the first position, the camera is rotated by 180 degrees around the calibration board to observe the long rectangular calibration board at the second position. At this time, the pose representation of the coordinate system of the calibration board calculated at the two different positions is the same, so that there are multiple ambiguous solutions for the calibration result calculated only by the point cloud data of the rectangular calibration board, which further affects the accurate solving of the pose conversion relationship between the camera coordinate system and the robot flange coordinate system, and further causes the positioning deviation of the robot in the visual guidance-based movement, and the accuracy of the robot operation is not high.

[0063] The coordinate system calibration method of the present application is applied to a coordinate system calibration device. The coordinate system calibration device of the present application can be a server, a terminal device, or a system comprising a server and a terminal device. Accordingly, each part of the coordinate system calibration device, such as each unit, subunit, module, and sub-module, can be provided in the server, the terminal device, or both.

[0064] Further, the server described above can be hardware or software. When the server is hardware, it can be implemented as a distributed server cluster composed of multiple servers, or as a single server. When the server is software, it can be implemented as multiple software or software modules, such as software or software modules for providing a distributed server, or as a single software or software module, without specific limitation.

[0065] Please refer to Figure 2 , Figure 2 is a flowchart of an embodiment of the coordinate system calibration method provided by the present application, as shown in Figure 2 The specific steps are as follows:

[0066] Step S11: Obtain the target point cloud data corresponding to the target calibration board collected by the three-dimensional camera in the calibration process, wherein the three-dimensional camera is fixedly installed on the end flange of the target robot, and the geometric shape of the target calibration board is an asymmetric shape.

[0067] In the embodiment of the present application, the target robot comprises a base, a mechanical arm, and an end flange. The base is usually fixed to the ground or a rack. One end of the mechanical arm is connected to the base, and the other end is installed with the end flange. The end flange is the connection hub of the mechanical arm and the end effector (such as a gripper, a welding gun, a sensor, etc.). The end flange realizes the work function through the connected end effector under the drive of the mechanical arm.

[0068] The three-dimensional camera is fixedly connected on the end flange, moves following the movement of the mechanical arm, collects point cloud data in the movement process, determines the three-dimensional coordinates of the target object in the camera coordinate system according to the processing of the point cloud data, and can determine the coordinates of the target object in the flange coordinate system according to the conversion relationship between the camera coordinate system and the flange coordinate system of the end flange, so as to guide the movement of the mechanical arm of the target robot according to the coordinates.

[0069] In order to accurately guide the target robot to accurately move in the working process, it is necessary to calibrate before the target robot starts to work, specifically, to calibrate the pose conversion relationship between the camera coordinate system corresponding to the three-dimensional camera and the flange coordinate system, that is, the second conversion relationship.

[0070] In the calibration process, the camera observes the target calibration board in a certain pose, and collects the target point cloud data corresponding to the target calibration board, wherein the three-dimensional camera can be a structured light camera or a laser radar, the three-dimensional camera scans the target calibration board, obtains the point set of the three-dimensional space points on the surface of the target calibration board, that is, the point cloud data, and the three-dimensional coordinates of each point in the camera coordinate system in the point cloud data are known.

[0071] In the embodiment, the geometric shape of the target calibration board is an asymmetric shape, that is, the target calibration board has geometric uniqueness, and the shapes or data observed at different angles are different.

[0072] Please refer to Figure 3 and Figure 4 , Figure 3 is a geometric shape schematic diagram of the target calibration board in an embodiment provided by the present application, Figure 4 is a geometric shape schematic diagram of the target calibration board in another embodiment provided by the present application, wherein, as shown in Figure 3 , Figure 3 the target calibration board is a right trapezoid, the straight line where the longer lower base is located (the straight line where the line segment AB in Figure 3 is located) is taken as the horizontal axis (X axis) in the calibration board coordinate system, the right-angled vertex corresponding to the lower base (the point A in Figure 3 ) is taken as the origin (point O) in the calibration board coordinate system, the straight line where the vertical waist is located (the straight line where the line segment AD in Figure 3 is located) is taken as the vertical axis (Y axis) in the calibration board coordinate system, and the direction perpendicular to the target calibration board is taken as the vertical axis (Z axis), so that the pose representation of the calibration board coordinate system obtained by the camera when observing at different positions is different according to the above construction constraint of the calibration board coordinate system.

[0073] As shown in Figure 4 , Figure 4 the target calibration board is a right triangle and the triangle is not an isosceles triangle, the straight line where the longer right angle side is located (the straight line where the line segment AB in Figure 4The line containing the median segment EF is used as the horizontal axis (X-axis) in the calibration plate coordinate system, with the right-angle vertex of the triangle (…). Figure 4 Point E in the calibration plate coordinate system is taken as the origin, and the line containing the shorter right-angled side ( Figure 4 The straight line containing the midline segment EG is the vertical axis (Y-axis) in the calibration plate coordinate system, and the direction perpendicular to the target calibration plate is the vertical axis (Z-axis). According to the above construction of the calibration plate coordinate system, the pose representation of the calibration plate coordinate system obtained by the camera at different positions is different.

[0074] Step S12: Determine the first transformation relationship between the camera coordinate system corresponding to the 3D camera and the calibration plate coordinate system corresponding to the target calibration plate during the calibration process based on the target point cloud data.

[0075] In this embodiment of the application, some feature points of the target calibration board can be filtered out from the target point cloud data. For example, the points on each triangle vertex and / or each side of the triangle in the target calibration board corresponding to the non-isosceles right triangle. Based on the coordinate representation of the scanning points corresponding to these feature points in the camera coordinate system in the target point cloud data, the transformation relationship from the calibration board coordinate system to the camera coordinate system is determined, that is, the first transformation relationship.

[0076] In an optional embodiment, determining a first transformation relationship between the camera coordinate system corresponding to the 3D camera and the calibration board coordinate system corresponding to the target calibration board during the calibration process, based on the target point cloud data, includes: extracting multiple sets of edge point clouds corresponding to multiple edge line segments of the target calibration board from the target point cloud data; determining the coordinate representation of the origin of the calibration board coordinate system in the camera coordinate system based on the multiple sets of edge point clouds; determining multiple vector representations of the direction vectors of multiple direction axes in the calibration board coordinate system in the camera coordinate system based on the multiple sets of edge point clouds and the coordinate representations; and determining the first transformation relationship based on the coordinate representations and the multiple vector representations.

[0077] In this embodiment, the multiple edge segments are the line segments corresponding to multiple sides of the target calibration plate. These edge segments (such as the four sides of a right trapezoidal calibration plate) have explicit geometric constraints with the calibration plate coordinate system in the physical world. The geometric constraints represent the geometric correspondence between the various directional axes of the origin in the calibration plate coordinate system and the target calibration plate. For example, for a non-isosceles right-angled triangle target calibration plate, the line containing the longer right-angled side is taken as the horizontal axis (X-axis) in the calibration plate coordinate system, the right-angled vertex of the triangle is taken as the origin in the calibration plate coordinate system, the line containing the shorter right-angled side is taken as the vertical axis (Y-axis) in the calibration plate coordinate system, and the direction perpendicular to the target calibration plate is taken as the vertical axis (Z-axis).

[0078] Extract the point cloud corresponding to each edge line segment in the target point cloud data, that is, the edge point cloud, and a plurality of edge line segments correspond to a plurality of edge point clouds one by one.

[0079] According to the plurality of edge point clouds, the coordinate representation of the origin and the vector representation of the direction vector of each direction axis in the calibration board coordinate system are determined.

[0080] When determining the above coordinate representation and vector representation, first, the straight line equation of each edge line segment is determined according to the plurality of edge point clouds, and a plurality of straight line equations are obtained. Specifically, the straight line equation corresponding to each edge line segment can be determined by using a straight line fitting algorithm.

[0081] According to the plurality of straight line equations, the first edge line segment and the second edge line segment satisfying the first preset condition are selected from the plurality of edge line segments. Taking a target calibration board in the shape of a non-isosceles right triangle as an example, the geometric constraint is that the straight line where the longer right angle side is located is the horizontal axis (X axis) in the calibration board coordinate system, and the straight line where the shorter right angle side is located is the vertical axis (Y axis) in the calibration board coordinate system. Therefore, the first edge line segment and the second edge line segment satisfying the first preset condition are the edge line segment corresponding to the longer right angle side and the edge line segment corresponding to the shorter right angle side, respectively.

[0082] Therefore, according to the straight line equations of the three edges of the target calibration board recognized from the target point cloud data, first, the two edge line segments perpendicular to each other are determined according to the plurality of straight line equations, that is, two edge line segments are randomly selected from the three edge line segments. If the two edge line segments are perpendicular, the two edge line segments selected are the first edge line segment and the second edge line segment satisfying the first preset condition. That is, the first edge line segment and the second edge line segment satisfying the first preset condition are selected from the plurality of edge line segments according to the plurality of straight line equations, which includes: randomly selecting two edge line segments from the plurality of edge line segments, and determining whether the two edge line segments are perpendicular according to the two straight line equations corresponding to the two edge line segments; in the case where the two edge line segments are perpendicular, the two edge line segments selected are determined as the first edge line segment and the second edge line segment satisfying the first preset condition.

[0083] In addition, if the two edge line segments selected are not perpendicular, the two edge line segments are reselected.

[0084] If the two edge line segments selected are perpendicular, the lengths of the two edge line segments are determined, the longer edge line segment is determined as the first edge line segment, and the shorter edge line segment is determined as the second edge line segment.

[0085] With the right-angle vertex of the triangle (i.e. the intersection of the longer right-angle side and the shorter right-angle side) as the origin in the calibration board coordinate system, the intersection of the first edge line segment and the straight line where the second edge lies (i.e. the target intersection point) is the origin, and the coordinates of the intersection of the first edge line segment and the straight line where the second edge line segment lies in the camera coordinate system are calculated according to the straight line equation corresponding to the first edge line segment and the straight line equation corresponding to the second edge line segment, i.e. the target intersection point coordinates of the first edge line segment and the second edge line segment in the camera coordinate system are determined according to the plurality of straight line equations, and the intersection point coordinates are determined as the coordinate representation.

[0086] Further, the plurality of directional axes in the calibration board coordinate system includes a horizontal axis, a vertical axis and a vertical axis, i.e. an X-axis, a Y-axis and a Z-axis, the straight line where the longer right-angle side lies is the horizontal axis in the calibration board coordinate system, and the straight line where the shorter right-angle side lies is the vertical axis in the calibration board coordinate system. Therefore, the straight line where the first edge line segment lies corresponds to the horizontal axis in the calibration board coordinate system, i.e. the first straight line equation corresponding to the first edge line segment is the straight line equation representation of the calibration board coordinate system in the camera coordinate system; the straight line where the second edge line segment lies corresponds to the vertical axis in the calibration board coordinate system, i.e. the second straight line equation corresponding to the second edge line segment is the straight line equation representation of the calibration board coordinate system in the camera coordinate system.

[0087] A first unit directional vector is determined according to the first straight line equation, and a second unit directional vector is determined according to the second straight line equation. Specifically, when the calibration board coordinate system is constructed, the right-angle vertex is taken as the origin, and the directions of the right-angle vertex pointing to the two vertices are taken as the positive directions of the horizontal axis and the vertical axis, respectively, as shown in FIG. 1. Figure 4 The coordinate representation of the two end points of the first edge line segment in the camera coordinate system is determined according to the first straight line equation, and the two end points are respectively Figure 4 The coordinate representation of the two end points of the second edge line segment in the camera coordinate system is determined according to the second straight line equation, and the two end points are respectively Figure 4 The midpoint E and the point G, wherein the point E corresponds to the target intersection point, and the first unit directional vector and the second unit directional vector are determined according to the coordinate representations of the three end points, wherein the first unit directional vector is represented as:

[0088]

[0089] The second unit directional vector is represented as:

[0090]

[0091] In a three-dimensional coordinate system, if the unit vectors of the horizontal axis and the vertical axis (i.e. the first unit directional vector and the second unit directional vector) are known, the unit vector of the vertical axis (i.e. the third unit directional vector) can be directly determined by vector cross product, i.e. the third unit directional vector is determined according to the first unit directional vector and the second unit directional vector.

[0092] The three unit direction vectors correspond to the plurality of vector representations, i.e., the first unit direction vector, the second unit direction vector, and the third unit direction vector determine the plurality of vector representations.

[0093] Step S13: calibrating a second conversion relationship between the flange coordinate system corresponding to the end flange and the camera coordinate system based on the first conversion relationship.

[0094] In the embodiments of the present application, the second conversion relationship between the flange coordinate system and the camera coordinate system can be determined based on the first conversion relationship between the camera coordinate system and the calibration board coordinate system.

[0095] Please refer to Figure 5 , Figure 5 is a schematic diagram of a target robot calibration process in an embodiment provided by the present application, as Figure 5 shown, the conversion relationship (third conversion relationship) between the base coordinate system and the flange coordinate system of the robot in the calibration process can be directly obtained according to the controller of the robot, the base and the target calibration board are fixed, the conversion relationship (fourth conversion relationship) between the corresponding base coordinate system and the calibration board coordinate system can be measured, and the second conversion relationship can be determined according to the first conversion relationship, the third conversion relationship, the fourth conversion relationship, and the pose conversion transmission relationship.

[0096] In an optional embodiment, the calibration process includes a first calibration process and a second calibration process, and the first conversion relationship includes a first conversion matrix corresponding to the first calibration process and a second conversion matrix corresponding to the second calibration process.

[0097] Calibrating a second conversion relationship between the flange coordinate system corresponding to the end flange and the camera coordinate system based on the first conversion relationship includes: obtaining conversion relationships between the base coordinate system of the target robot and the flange coordinate system in the first calibration process and the second calibration process respectively to obtain a third conversion matrix and a fourth conversion matrix; determining a first calibration matrix based on the first conversion matrix and the second conversion matrix, wherein the first calibration matrix represents a conversion relationship between the flange coordinate system in the first calibration process and the flange coordinate system in the second calibration process; determining a second calibration matrix based on the third conversion matrix and the fourth conversion matrix, wherein the second calibration matrix represents a conversion relationship between the camera coordinate system in the first calibration process and the camera coordinate system in the second calibration process, obtaining a second calibration matrix; and determining a target conversion matrix representing the second conversion relationship according to the first calibration matrix and the second calibration matrix, wherein the matrix product of the second calibration matrix and the target conversion matrix is the same as the matrix product of the target conversion matrix and the first calibration matrix.

[0098] In the embodiment of the present application, the second conversion relationship is determined through two calibration processes, please refer to Figure 6 , Figure 6 is the relative position diagram of the two calibration processes of the target robot in another embodiment provided by the present application, as Figure 6 shown, the two calibration processes are the first calibration process and the second calibration process respectively, the positions of the base and the target calibration board remain unchanged in the two calibration processes, the end flange and the three-dimensional camera are moved to different positions by the mechanical arm and calibrated at different positions.

[0099] Figure 6 The dashed part in the middle is corresponding to the second calibration process, the solid part corresponds to the first calibration process, according to the pose conversion transmission relationship, the following relationship is met in the two calibration processes:

[0100]

[0101] Wherein, i represents the first calibration process, j represents the second calibration process, F i represents the flange coordinate system in the first calibration process, F j represents the flange coordinate system in the second calibration process, C i represents the camera coordinate system in the first calibration process, C j represents the camera coordinate system in the second calibration process, T represents the conversion matrix of the two coordinate systems.

[0102] And because in the calibration process, the three-dimensional camera and the end flange are fixedly connected in rigidity, the relative position is unchanged, the following formula is met:

[0103]

[0104] Therefore, in the two calibration processes, each coordinate system meets the following formula:

[0105]

[0106] Wherein, F T C represents the conversion relationship (the second conversion relationship) of the flange coordinate system and the camera coordinate system, that is, the target calibration matrix.

[0107] In determining the second conversion relationship, that is, calculating the target calibration matrix, it is necessary to calculate the conversion relationship of the flange coordinate system in the first calibration process and the flange coordinate system in the second calibration process, that is, the first calibration matrix And calculate the conversion relationship of the camera coordinate system in the first calibration process and the flange coordinate system in the camera calibration process, that is, the second calibration matrix

[0108] Wherein, the first calibration matrix and the second calibration matrix meet the following formula:

[0109]

[0110]

[0111] wherein B represents a base coordinate system, F i represents a flange coordinate system in the first calibration process, F j represents a flange coordinate system in the second calibration process, C i represents a camera coordinate system in the first calibration process, C j represents a camera coordinate system in the second calibration process, D represents a calibration plate coordinate system; and T represents a conversion matrix of two coordinate systems.

[0112] In determining the first calibration matrix, the conversion relationship between the base coordinate system and the flange coordinate system in the first calibration process, i.e., the third conversion matrix and the conversion relationship between the base coordinate system and the flange coordinate system in the second calibration process, i.e., the fourth conversion matrix The first calibration matrix can be determined according to the third conversion matrix and the fourth conversion matrix.

[0113] In determining the second calibration matrix, the conversion relationship between the calibration plate coordinate system and the camera coordinate system in the first calibration process, i.e., the first conversion matrix, and the conversion relationship between the calibration plate coordinate system and the camera coordinate system in the second calibration process, i.e., the fourth conversion matrix, are obtained, and the second calibration matrix can be determined according to the first conversion matrix and the fourth conversion matrix.

[0114] In the embodiments of the application, by setting the target calibration plate as a calibration plate with an asymmetric geometric shape, the conversion relationship between the unique calibration coordinate system and the camera coordinate system, i.e., the first conversion relationship, can be obtained from the point cloud data collected by the three-dimensional camera at any observation angle, the ambiguity of the rectangular calibration plate is eliminated, the calibration result has a unique solution, there is no multiple ambiguous solutions, the accurate solution of the pose conversion relationship between the camera coordinate system and the robot flange coordinate system is ensured, and the working accuracy of the robot is improved.

[0115] In an optional embodiment, the target point cloud data corresponding to the target calibration plate collected by the three-dimensional camera in the calibration process includes: obtaining initial point cloud data corresponding to the target calibration plate, traversing each collection point in the initial point cloud data; when traversing a current collection point, obtaining a near neighbor point set in the initial point cloud data with a distance less than a first preset value from the current collection point; in the case where the number of near neighbor points in the near neighbor point set is greater than a preset number threshold, determining an intermediate point between each near neighbor point in the near neighbor point set and the current collection point to obtain an intermediate point set; inserting the intermediate points in the intermediate point set that satisfy a second preset condition into the initial point cloud data to obtain the target point cloud data.

[0116] inserting the intermediate points in the intermediate point set satisfying the second preset condition into the initial point cloud data to obtain the target point cloud data, comprising: traversing each intermediate point in the intermediate point set; when traversing a current intermediate point, determining whether there is at least one collection point in the initial point cloud data falling into a target neighborhood of the current intermediate point; if yes, traversing the next; if no, determining the current intermediate point as satisfying the second preset condition.

[0117] In the embodiments of the present application, the initial point cloud data collected by the three-dimensional camera is preprocessed to obtain target point cloud data, so as to improve the matching accuracy and eliminate the errors caused by the uneven density and missing planar point cloud profile of the collected initial point cloud data.

[0118] In the embodiments of the present application, first, whether the current collection point is an isolated point is determined according to whether the number of collection points in the neighborhood of the current collection point is greater than a preset number threshold. In the case that the current collection point is not an isolated point, i.e., the number of near neighbor points in the near neighbor point set is greater than the preset number threshold, the intermediate point between the current collection point and each near neighbor point is determined to obtain an intermediate point set. The intermediate point in the intermediate point set, i.e., the intermediate point satisfying the second preset condition, which has a distance greater than a second preset value from the current collection point, is inserted into the initial point cloud data, the traversal of the current collection point is completed, and the next collection point in the initial point cloud data is traversed until all collection points in the initial point cloud data are traversed.

[0119] The Euclidean distance between the collection point falling into the target neighborhood of the current intermediate point and the current collection point is greater than or equal to the second preset value.

[0120] Please refer to Figure 7 , Figure 7 is a flowchart of preprocessing the initial point cloud data provided by the present application, as shown in Figure 7 , each collection point in the initial point cloud data is traversed in turn, specifically comprising:

[0121] S701, traversing the i-th collection point Pi, searching for near neighbor points in the neighborhood of Pi to obtain a near neighbor point set, wherein the distance between the collection point falling into the neighborhood of Pi and the collection point Pi is less than a first preset value;

[0122] S702, determining whether the data amount of the near neighbor point is greater than a preset number threshold, if yes, executing step S703, if not, executing step S709;

[0123] S703, traversing the j-th near neighbor point Pj in the near neighbor point set;

[0124] S704, judging whether the distance between the collection point Pi and the neighboring point Pj is greater than or equal to a third preset value, wherein the third preset value is twice the second preset value; if yes, executing step S705, and if not, executing step S710;

[0125] S705, determining a middle point between the collection point Pi and the neighboring point Pj;

[0126] S706, judging whether all neighboring points in the neighboring point set have been traversed, if yes, executing step S707, and if not, executing step S710;

[0127] S707, judging whether all collection points in the initial point cloud data have been traversed, if yes, executing step S708, and if not, executing step S709;

[0128] S708, inserting the middle point into the initial point cloud data to obtain target point cloud data;

[0129] S709, executing i+1 to traverse the next collection point in the initial point cloud data;

[0130] S710, executing j+1 to traverse the next neighboring point in the neighboring point set.

[0131] Those skilled in the art can understand that in the above method of the specific embodiment, the writing order of each step does not mean a strict execution order and does not constitute any limitation on the implementation process, and the specific execution order of each step should be determined by its function and possible internal logic.

[0132] To implement the above coordinate system calibration method, the application further provides a coordinate system calibration device, please refer to Figure 8 , Figure 8 is a structural schematic diagram of an embodiment of the coordinate system calibration device provided by the application.

[0133] The coordinate system calibration device 500 of the embodiment comprises:

[0134] An acquisition module 51 is configured to acquire target point cloud data corresponding to a target calibration board collected by a three-dimensional camera in a calibration process, wherein the three-dimensional camera is fixedly installed on an end flange of a target robot, and the target calibration board has an asymmetric shape.

[0135] A determination module 52 is configured to determine a first conversion relationship between a camera coordinate system corresponding to the three-dimensional camera and a calibration board coordinate system corresponding to the target calibration board in the calibration process according to the target point cloud data.

[0136] A calibration module 53 is configured to calibrate a second conversion relationship between a flange coordinate system corresponding to the end flange and the camera coordinate system based on the first conversion relationship.

[0137] To realize the coordinate system calibration method, the application further provides a coordinate system calibration device, please refer to Figure 9 , Figure 9 is a structural schematic diagram of an embodiment of the coordinate system calibration device provided by the application.

[0138] The coordinate system calibration device 400 of the embodiment comprises a processor 41, a memory 42, an input and output device 43 and a bus 44.

[0139] The processor 41, the memory 42 and the input and output device 43 are connected with the bus 44 respectively, the memory 42 stores program data, and the processor 41 is used to execute the program data to realize the coordinate system calibration method described in the above embodiment.

[0140] In the embodiment of the application, the processor 41 can also be called a CPU (Central Processing Unit). The processor 41 can be an integrated circuit chip with signal processing capability. The processor 41 can also be a general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor 41 can also be any conventional processor.

[0141] The application further provides a computer storage medium, please continue to refer to Figure 10 , Figure 10 is a structural schematic diagram of an embodiment of the computer storage medium provided by the application. The computer storage medium 600 stores a computer program 61. The computer program 61 is used to realize the coordinate system calibration method of the above embodiment when executed by a processor.

[0142] The embodiments of the present application are realized in the form of software function units and sold or used as independent products, and can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application or the whole or part of the technical solutions that essentially contribute to the prior art can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the method described in the various embodiments of the present application. The aforementioned storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0143] The above description is only the embodiments of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation using the content of the specification and drawings, or direct or indirect application in other related technical fields, is also included in the patent protection scope of the present application.

Claims

1. A coordinate system calibration method, characterized in that, The coordinate system calibration method includes: During the calibration process, the target point cloud data corresponding to the target calibration plate is acquired by the 3D camera. The 3D camera is fixedly installed on the end flange of the target robot, and the target calibration plate has an asymmetrical geometry. Based on the target point cloud data, determine the first transformation relationship between the camera coordinate system corresponding to the 3D camera and the calibration plate coordinate system corresponding to the target calibration plate during the calibration process; Based on the first transformation relationship, a second transformation relationship between the flange coordinate system corresponding to the end flange and the camera coordinate system is determined.

2. The coordinate system calibration method according to claim 1, characterized in that, Determining the first transformation relationship between the camera coordinate system corresponding to the 3D camera and the calibration board coordinate system corresponding to the target calibration board during the calibration process based on the target point cloud data includes: Extract multiple sets of edge point clouds corresponding to multiple edge line segments of the target calibration board from the target point cloud data; The coordinate representation of the origin of the calibration board coordinate system in the camera coordinate system is determined based on the multiple sets of edge point clouds; Based on the multiple sets of edge point clouds and the coordinate representation, determine the multiple vector representations of the direction vectors of multiple directional axes in the calibration plate coordinate system in the camera coordinate system; The first transformation relationship is determined based on the coordinate representation and the plurality of vector representations.

3. The coordinate system calibration method according to claim 2, characterized in that, Determining the coordinate representation of the origin of the calibration board coordinate system in the camera coordinate system based on the multiple sets of edge point clouds includes: Based on the multiple sets of edge point clouds, determine the multiple straight line equations corresponding to the multiple edge line segments in the camera coordinate system; Based on multiple straight line equations, select the first edge segment and the second edge segment that satisfy the first preset condition from the multiple edge segments; The coordinates of the intersection point of the first edge segment and the second edge segment in the camera coordinate system are determined based on the multiple straight line equations. The coordinates of the intersection point are determined as the coordinate representation.

4. The coordinate system calibration method according to claim 3, characterized in that, Based on the multiple sets of edge point clouds and the coordinate representation, determine the multiple vector representations of the direction vectors of multiple direction axes in the calibration board coordinate system in the camera coordinate system, including: The first unit direction vector corresponding to the first edge line segment is determined based on the first straight line equation corresponding to the first edge line segment, taking the target intersection point as the starting point; The second unit direction vector corresponding to the first edge line segment is determined based on the second straight line equation corresponding to the second edge line segment, with the target intersection point as the starting point; The third unit direction vector is determined based on the first unit direction vector and the second unit direction vector. The first unit direction vector, the second unit direction vector, and the third unit direction vector are used to determine the plurality of vector representations.

5. The coordinate system calibration method according to claim 2, characterized in that, The target calibration plate has a non-isosceles right-angled triangle geometry. Based on multiple straight line equations, a first edge segment and a second edge segment satisfying a first preset condition are selected from the multiple edge segments, including: Two edge segments are randomly selected from the plurality of edge segments, and the two edge segments are determined to be perpendicular based on the equations of the two lines corresponding to the selected two edge segments. When the two edge segments are perpendicular, the selected two edge segments are determined as the first edge segment and the second edge segment that satisfy the first preset condition.

6. The coordinate system calibration method according to claim 1, characterized in that, The calibration process includes a first calibration process and a second calibration process, and the first transformation relationship includes a first transformation matrix corresponding to the first calibration process and a second transformation matrix corresponding to the second calibration process. Based on the first transformation relationship, a second transformation relationship is determined between the flange coordinate system corresponding to the end flange and the camera coordinate system, including: The transformation relationship between the base coordinate system and the flange coordinate system of the target robot is obtained in the first calibration process and the second calibration process, respectively, to obtain the third transformation matrix and the fourth transformation matrix; Based on the third transformation matrix and the fourth transformation matrix, a first calibration matrix is ​​determined, wherein the first calibration matrix represents the transformation relationship between the flange coordinate system in the first calibration process and the flange coordinate system in the second calibration process; Based on the first transformation matrix and the second transformation matrix, a second calibration matrix is ​​determined, wherein the second calibration matrix represents the transformation relationship between the camera coordinate system in the first calibration process and the camera coordinate system in the second calibration process, thus obtaining the second calibration matrix; The target transformation matrix representing the second transformation relationship is determined based on the first calibration matrix and the second calibration matrix, wherein the matrix product of the second calibration matrix and the target transformation matrix is ​​the matrix product of the target transformation matrix and the first calibration matrix.

7. The coordinate system calibration method according to claim 1, characterized in that, Acquire the target point cloud data corresponding to the target calibration board acquired by the 3D camera during the calibration process, including: Obtain the initial point cloud data corresponding to the target calibration board. Iterate through each collection point in the initial point cloud data; When traversing the current collection point, obtain the set of nearest neighbor points in the initial point cloud data whose distance from the current collection point is less than a first preset value; If the number of nearest points in the nearest point set is greater than a preset threshold, determine the midpoint between each nearest point in the nearest point set and the current collection point to obtain the midpoint set. The intermediate points that meet the second preset condition from the intermediate point set are inserted into the initial point cloud data to obtain the target point cloud data.

8. A coordinate system calibration device, characterized in that, The coordinate system calibration device includes: The acquisition module is used to acquire target point cloud data corresponding to the target calibration plate collected by the 3D camera during the calibration process. The 3D camera is fixedly installed on the end flange of the target robot, and the target calibration plate has an asymmetrical geometry. The determination module is used to determine, based on the target point cloud data, a first transformation relationship between the camera coordinate system corresponding to the 3D camera and the calibration plate coordinate system corresponding to the target calibration plate during the calibration process; The calibration module is used to calibrate the second transformation relationship between the flange coordinate system corresponding to the end flange and the camera coordinate system based on the first transformation relationship.

9. A coordinate system calibration device, characterized in that, The coordinate system calibration device includes a memory and a processor coupled to the memory; The memory is used to store program data, and the processor is used to execute the program data to implement the coordinate system calibration method as described in any one of claims 1 to 7.

10. A computer storage medium, characterized in that, The computer storage medium is used to store program data, which, when executed by the computer, is used to implement the coordinate system calibration method as described in any one of claims 1 to 7.

Images