Joint calibration method, joint calibration apparatus and joint calibration system

Abstract

This application provides a joint calibration method, joint calibration device, and joint calibration system. The method includes: acquiring multiple first coordinate vectors and multiple second coordinate vectors when the robot's motion trajectory is a straight line; acquiring the coordinates of each trajectory point of the camera trajectory and the coordinates of each trajectory point of the first radar trajectory when the robot's motion trajectory is a circular arc; fitting the first coordinate vectors and second coordinate vectors using a least squares algorithm to obtain a target rotation matrix; transforming the coordinates of each trajectory point of the first radar trajectory to the camera coordinate system based on the target rotation matrix to obtain the coordinates of multiple transformed trajectory points, which form the second radar trajectory; and calculating the vector between the center of the circle corresponding to the camera trajectory and the center of the circle corresponding to the second radar trajectory to obtain the target translation vector. This method solves the problem in the prior art where the camera and radar often do not share a common field of view, causing the target-based calibration method to fail.

Description

Technical Field

[0001] This application relates to the field of equipment calibration technology, and more specifically, to a joint calibration method, a joint calibration device, a computer-readable storage medium, and a joint calibration system. Background Technology

[0002] Positioning and navigation accuracy has always been a key challenge in industrial robot research, and the quality of extrinsic parameter calibration directly determines the level of positioning accuracy. Camera and radar calibration methods can be broadly categorized into motion-based calibration, target-based calibration, and environment-based calibration. Motion-based calibration typically employs hand-eye calibration theory, requiring two sensors to acquire a path segment and then performing extrinsic parameter calibration based on that path. Target-based calibration methods generally create objects that both the camera and radar can detect simultaneously for calibration, such as affixing visual QR codes (e.g., ArUco codes or AprilTag codes) to a suspended square. Environment-based calibration generally refers to calibration based on edge matching within the environment. This method extracts edge information from the environment using both the camera and radar, constructs edge projection errors, and then iteratively solves for extrinsic parameters based on these errors.

[0003] An existing method for calibrating radar and camera extrinsic parameters involves constructing a standard mesh in a two-dimensional coordinate system based on coarsely extracted 3D corner points. An iterative optimization method is then used to align the standard mesh with the mesh features on the 2D corner points, obtaining the aligned pose of the standard mesh and the 2D corner points. Based on this aligned pose, the aligned 2D corner points are transformed from the two-dimensional coordinate system to the radar coordinate system to obtain finely extracted 3D corner points. 3D-2D matching points are then formed using the finely extracted 3D corner points and the corner points of the two-dimensional image. Based on these 3D-2D matching points, the PnP method is used to solve for the radar and camera extrinsic parameters. However, due to the nature of their operation, cameras and radars mounted on industrial robots often do not share a common field of view, rendering target-based calibration methods ineffective.

[0004] The information disclosed above in the background section is only intended to enhance the understanding of the background art of the art described herein. Therefore, the background art may contain certain information that does not constitute prior art known to those skilled in the art in this country. Summary of the Invention

[0005] The main objective of this application is to provide a joint calibration method, a joint calibration device, a computer-readable storage medium, and a joint calibration system to solve the problem that target-based calibration methods often fail to function due to the lack of shared viewing area between cameras and radars in the prior art.

[0006] According to one aspect of the present invention, a joint calibration method is provided for calibrating an image acquisition device and a radar installed on a robot. The calibration method includes: when the robot's motion trajectory is a straight line, acquiring a plurality of first coordinate vectors and a plurality of second coordinate vectors, wherein the first coordinate vectors correspond one-to-one with the second coordinate vectors; the first coordinate vectors are vectors representing the translation transformation relationship between the camera coordinate system and the world coordinate system; the second coordinate vectors are vectors representing the translation transformation relationship between the radar coordinate system and the world coordinate system; the first coordinate vectors include the x-axis and y-axis coordinates of the camera's trajectory points in the world coordinate system; the second coordinate vectors include the x-axis and y-axis coordinates of the radar's trajectory points in the world coordinate system; and the detection period to which the first coordinate vector belongs is the same as the detection period to which the corresponding second coordinate vector belongs; when the robot's motion trajectory is a circular arc, acquiring the trajectory points of the camera trajectory. The coordinates of each trajectory point of the first radar trajectory are calculated. The camera trajectory is the motion trajectory of the camera located in the plane containing the x-axis and y-axis in the world coordinate system, and the first radar trajectory is the motion trajectory of the radar located in the plane containing the x-axis and y-axis in the world coordinate system. The first coordinate vector and the second coordinate vector are fitted using a least squares algorithm to obtain a target rotation matrix, which is a matrix representing the rotation transformation relationship between the camera coordinate system and the radar coordinate system. Based on the target rotation matrix, the coordinates of each trajectory point of the first radar trajectory are transformed to the camera coordinate system to obtain the coordinates of multiple transformed trajectory points, which form the second radar trajectory. The vector between the center of the circle corresponding to the camera trajectory and the center of the circle corresponding to the second radar trajectory is calculated to obtain the target translation vector, which is a vector representing the translation transformation relationship between the camera coordinate system and the radar coordinate system.

[0007] Optionally, before fitting the first coordinate vector and the second coordinate vector using the least squares algorithm to obtain the target rotation matrix, the method further includes: obtaining formula R based on the hand-eye calibration principle. A ×t X +t A =R X ×t B +t X , where R X Let t be the target rotation matrix. X Let t be the target translation vector. A Let t be the first coordinate vector. B Let R be the second coordinate vector. A Let R be the first rotation matrix. BThe first rotation matrix is ​​a matrix representing the rotational transformation relationship between the camera coordinate system and the world coordinate system, and the second rotation matrix is ​​a matrix representing the rotational transformation relationship between the radar coordinate system and the world coordinate system. When the robot's motion trajectory is a straight-line motion trajectory, the first rotation matrix R is determined. A Let R be the identity matrix; let the first rotation matrix R be... A Substitute into formula R A ×t X +t A =R X ×t B +t X After simplification, we obtain the formula t. A =R X ×t B .

[0008] Optionally, the first coordinate vector and the second coordinate vector are fitted using a least squares algorithm to obtain the target rotation matrix, including: according to formula t A =R X ×t B Multiple preliminary rotation matrices are calculated, where t A Let t be the first coordinate vector. B Let RX be the second coordinate vector, and let RX be the preparatory rotation matrix, which corresponds one-to-one with the first coordinate vector; according to formula e R =||R X ×t B -t A || 2 Determine the target rotation matrix, where t A Let t be the first coordinate vector. B Let R be the second coordinate vector. X For the pre-rotation matrix, e R The error value is a one-to-one correspondence between the error value and the preliminary rotation matrix, and the target rotation matrix is ​​the preliminary rotation matrix corresponding to the smallest error value.

[0009] Optionally, calculating the vector between the center of the circle corresponding to the camera trajectory and the center of the circle corresponding to the second radar trajectory to obtain the target translation vector includes: determining the magnitude and direction of a first vector, the magnitude and direction of a second vector, and the direction of a second vector based on the camera trajectory and the second radar trajectory, wherein the starting point of the first vector is the center of the circle corresponding to the camera trajectory, the ending point of the first vector is the starting point of the camera trajectory, the starting point of the second vector is the center of the circle corresponding to the second radar trajectory, and the ending point of the second vector is the starting point of the second radar trajectory, and the starting point of the camera trajectory is the same as the starting point of the second radar trajectory; and obtaining the target translation vector based on the magnitude and direction of the first vector, the magnitude and direction of the second vector.

[0010] Optionally, determining the magnitude and direction of the first vector, the magnitude and direction of the second vector, and the direction of the second vector based on the camera trajectory and the second radar trajectory includes: determining a first arc length and the camera's motion direction based on the camera trajectory, and determining a second arc length and the radar's motion direction based on the second radar trajectory, wherein the first arc length is the arc length of the camera trajectory, and the second arc length is the arc length of the second radar trajectory; calculating the ratio of the first arc length to an angle value to obtain the magnitude of the first vector, wherein the angle value is the angle of the central angle corresponding to the first arc length, and the angle of the central angle corresponding to the first arc length is the same as the angle of the central angle corresponding to the second arc length; calculating the ratio of the second arc length to the angle value to obtain the magnitude of the second vector; determining the direction of the first vector based on the camera's motion direction; and determining the direction of the second vector based on the radar's motion direction.

[0011] Optionally, the method for obtaining the first coordinate vector includes: pre-laying a QR code on the linear motion trajectory; obtaining the x-axis coordinates and y-axis coordinates of the corner points of the QR code in the world coordinate system to obtain a third coordinate vector; when the robot moves along the linear motion trajectory, obtaining the x-axis coordinates and y-axis coordinates of the corner points of the QR code in the camera coordinate system to obtain multiple fourth coordinate vectors, wherein the third coordinate vectors correspond one-to-one with the fourth coordinate vectors; and determining multiple first coordinate vectors based on each third coordinate vector and the corresponding fourth coordinate vector.

[0012] Optionally, the method for obtaining the coordinates of each trajectory point of the camera trajectory includes: pre-laying a QR code on the circular motion trajectory; obtaining the x-axis coordinates and y-axis coordinates of the corner points of the QR code in the world coordinate system to obtain multiple fifth coordinate vectors; when the robot moves according to the circular motion trajectory, obtaining the x-axis coordinates and y-axis coordinates of the corner points of the QR code in the camera coordinate system to obtain multiple sixth coordinate vectors, wherein the fifth coordinate vectors correspond one-to-one with the sixth coordinate vectors; and determining the coordinates of each trajectory point of the camera trajectory based on each fifth coordinate vector and the corresponding sixth coordinate vector.

[0013] According to another aspect of the present invention, a joint calibration device is also provided, applied to an image acquisition device and a radar installed on a robot. The calibration device includes: a first acquisition unit, configured to acquire a plurality of first coordinate vectors and a plurality of second coordinate vectors when the robot's motion trajectory is a straight line motion trajectory, wherein the first coordinate vectors correspond one-to-one with the second coordinate vectors, the first coordinate vectors being vectors representing the translation transformation relationship between the camera coordinate system and the world coordinate system, and the second coordinate vectors being vectors representing the translation transformation relationship between the radar coordinate system and the world coordinate system, the first coordinate vectors including the x-axis coordinates and y-axis coordinates of the camera's trajectory points in the world coordinate system, and the second coordinate vectors including the x-axis coordinates and y-axis coordinates of the radar's trajectory points in the world coordinate system, wherein the detection period to which the first coordinate vector belongs is the same as the detection period to which the corresponding second coordinate vector belongs; and a second acquisition unit, configured to acquire each trajectory point of the camera trajectory when the robot's motion trajectory is a circular arc motion trajectory. The system comprises: a camera trajectory and a radar trajectory; a camera trajectory being the motion trajectory of the camera located in the plane containing the x and y axes in the world coordinate system; a camera trajectory being the motion trajectory of the radar located in the plane containing the x and y axes in the world coordinate system; a fitting unit for fitting the first coordinate vector and the second coordinate vector using a least squares algorithm to obtain a target rotation matrix, which is a matrix representing the rotation transformation relationship between the camera coordinate system and the radar coordinate system; a transformation unit for transforming the coordinates of each trajectory point of the first radar trajectory to the camera coordinate system according to the target rotation matrix, obtaining the coordinates of multiple transformed trajectory points, which form the second radar trajectory; and a calculation unit for calculating the vector between the center of the circle corresponding to the camera trajectory and the center of the circle corresponding to the second radar trajectory, obtaining a target translation vector, which is a vector representing the translation transformation relationship between the camera coordinate system and the radar coordinate system.

[0014] According to another aspect of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored program, wherein when the program is executed by a processor, the processor performs any of the aforementioned joint calibration methods.

[0015] According to another aspect of the present invention, a joint calibration system is also provided, comprising: one or more processors, a memory, a display device, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include methods for performing any of the joint calibration methods described above.

[0016] In the aforementioned joint calibration method, firstly, when the robot's motion trajectory is a straight line, multiple first coordinate vectors and multiple second coordinate vectors are acquired. The first coordinate vectors correspond one-to-one with the second coordinate vectors. The first coordinate vector represents the translational transformation relationship between the camera coordinate system and the world coordinate system, and the second coordinate vector represents the translational transformation relationship between the radar coordinate system and the world coordinate system. The first coordinate vector includes the x-axis and y-axis coordinates of the camera's trajectory points in the world coordinate system, and the second coordinate vector includes the x-axis and y-axis coordinates of the radar's trajectory points in the world coordinate system. The detection period of the first coordinate vector is the same as the detection period of the corresponding second coordinate vector. Then, when the robot's motion trajectory is a circular arc, the coordinates of each trajectory point of the camera trajectory and the coordinates of each trajectory point of the first radar trajectory are acquired. The first radar trajectory is the motion trajectory of the camera located in the plane containing the x and y coordinate axes in the world coordinate system. Then, a least squares algorithm is used to fit the first and second coordinate vectors to obtain a target rotation matrix, which represents the rotational transformation relationship between the camera coordinate system and the radar coordinate system. Next, based on the target rotation matrix, the coordinates of each trajectory point of the first radar trajectory are transformed to the camera coordinate system, resulting in multiple transformed trajectory points. These transformed trajectory points form the second radar trajectory. Finally, the vector between the center of the circle corresponding to the camera trajectory and the center of the circle corresponding to the second radar trajectory is calculated to obtain the target translation vector, which represents the translational transformation relationship between the camera coordinate system and the radar coordinate system. This method uses camera trajectory points in the camera coordinate system and radar trajectory points in the radar coordinate system. Given that the robot moves along a specific straight-line trajectory, the trajectory points in the camera and radar coordinate systems only have a rotational transformation relationship. In this case, the first and second coordinate vectors are fitted to determine the target rotation matrix. Then, based on the robot's movement along a specific circular arc trajectory, the first radar trajectory in the world coordinate system is transformed to the camera coordinate system according to the determined target rotation matrix, resulting in the second radar trajectory. Here, the trajectory points in the camera coordinate system and the radar trajectory in the camera coordinate system only have a translational transformation relationship. By aligning the determined target translation vectors of the camera and second radar trajectories, this method calibrates the target rotation matrix and target translation vector based on a specific motion trajectory. It does not require the camera and radar to have a shared field of view, thus solving the problem in existing technologies where the lack of a shared field of view between the camera and radar often prevents target-based calibration methods from functioning. Attached Figure Description

[0017] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0018] Figure 1 A flowchart of a joint calibration method according to an embodiment of this application is shown;

[0019] Figure 2 A flowchart of a joint calibration method according to a specific embodiment of this application is shown;

[0020] Figure 3 A flowchart of a joint calibration method according to another specific embodiment of this application is shown;

[0021] Figure 4 A flowchart of a joint calibration method according to another specific embodiment of this application is shown;

[0022] Figure 5 A flowchart of a joint calibration method according to yet another specific embodiment of this application is shown;

[0023] Figure 6 A schematic diagram of a camera trajectory and a second radar trajectory according to one embodiment of this application is shown;

[0024] Figure 7 A flowchart of a joint calibration method according to a specific embodiment of this application is shown;

[0025] Figure 8 A flowchart of a joint calibration method according to another specific embodiment of this application is shown;

[0026] Figure 9 A schematic diagram of a joint calibration apparatus according to an embodiment of this application is shown. Detailed Implementation

[0027] It should be noted that the following detailed descriptions are illustrative and intended to provide further explanation of this application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0028] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this application. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0029] It should be understood that when an element (such as a layer, film, region, or substrate) is described as being "on" another element, the element may be directly on the other element, or there may be an intermediate element present. Furthermore, in the specification and claims, when an element is described as being "connected" to another element, the element may be "directly connected" to the other element, or "connected" to the other element via a third element.

[0030] As mentioned in the background section, in the prior art, cameras and radars often do not share a common field of view, which makes target-based calibration methods ineffective. In order to solve the above problems, in a typical embodiment of this application, a joint calibration method, a joint calibration device, a computer-readable storage medium, and a joint calibration system are provided.

[0031] According to embodiments of this application, a joint calibration method is provided.

[0032] Figure 1 This is a flowchart of the joint calibration method according to an embodiment of this application. For example... Figure 1 As shown, the method includes the following steps:

[0033] Step S101: When the robot's motion trajectory is a straight line, multiple first coordinate vectors and multiple second coordinate vectors are acquired. The first coordinate vectors correspond one-to-one with the second coordinate vectors. The first coordinate vector is a vector representing the translation transformation relationship between the camera coordinate system and the world coordinate system. The second coordinate vector is a vector representing the translation transformation relationship between the radar coordinate system and the world coordinate system. The first coordinate vector includes the x-axis coordinate and y-axis coordinate of the camera's trajectory point in the world coordinate system. The second coordinate vector includes the x-axis coordinate and y-axis coordinate of the radar's trajectory point in the world coordinate system. The detection period to which the first coordinate vector belongs is the same as the detection period to which the corresponding second coordinate vector belongs.

[0034] The second coordinate vector mentioned above contains the x-axis and y-axis coordinates of the radar trajectory points in the world coordinate system after ground compensation. The specific process of ground compensation is as follows: The radar uses ground detection technology. First, it obtains prior information such as the radar installation height and angle. Then, it performs angle compensation on the x-axis and y-axis coordinates of the radar trajectory points in the world coordinate system output by the radar positioning algorithm. The first radar motion trajectory mentioned above is also the radar motion trajectory in the world coordinate system after ground compensation.

[0035] Furthermore, after ground compensation, it is only necessary to solve the rotational transformation relationship between the x-axis and y-axis coordinates of the camera trajectory points and the radar trajectory points when the robot's motion trajectory is a straight line. It is only necessary to solve the translational transformation relationship between the camera trajectory and the radar trajectory located in the plane containing the x-axis and y-axis when the robot's motion trajectory is a circular arc. This simplifies the three-dimensional calibration problem into a two-dimensional calibration problem, improving calibration accuracy. The aforementioned camera motion trajectory is the motion trajectory of the camera coordinate system, and the first and second radar coordinate trajectories are both motion trajectories of the radar coordinate system. Since the distance between the radar and the ground is measurable, the camera coordinate system is the IPM coordinate system, located on the ground. The z-axis coordinates of the radar coordinate system and the z-axis coordinates of the camera coordinate system are fixed and measurable. After completing the two-dimensional calibration, the ground compensation is eliminated based on the distance between the radar coordinate system and the camera coordinate system, and the three-dimensional calibration extrinsic parameters are obtained.

[0036] Optionally, this application does not limit the specific process of obtaining the first coordinate vector, and any feasible method is within the protection scope of this application.

[0037] In one alternative implementation, such as Figure 2 As shown, step S101 above includes:

[0038] Step S1011: Pre-lay QR codes on the above-mentioned linear motion trajectory;

[0039] Step S1012: Obtain the x-axis coordinates and y-axis coordinates of the corner points of the QR code in the world coordinate system to obtain a third coordinate vector. When the robot moves along a straight line trajectory, obtain the x-axis coordinates and y-axis coordinates of the corner points of the QR code in the camera coordinate system to obtain multiple fourth coordinate vectors. The third coordinate vector corresponds one-to-one with the fourth coordinate vector.

[0040] Step S1013: Determine multiple first coordinate vectors based on each of the aforementioned third coordinate vectors and the corresponding fourth coordinate vectors.

[0041] In the above implementation, QR codes are laid out on the straight-line trajectory in advance, and the actual x-axis and y-axis coordinates of the corner points of the QR codes in the world coordinate system are measured to obtain the third coordinate vector. Then, when the robot moves along the straight-line trajectory, the extracted x-axis and y-axis coordinates of the corner points of the QR codes are first transformed to the camera coordinate system to obtain the x-axis and y-axis coordinates of the corner points of the QR codes in the camera coordinate system, and then the fourth coordinate vector is obtained. Then, the actual x-axis and y-axis coordinates of the corner points of the QR codes in the world coordinate system are compared with the x-axis and y-axis coordinates of the corner points of the QR codes in the camera coordinate system to obtain the x-axis and y-axis coordinates of each trajectory point in the world coordinate system in the camera coordinate system, that is, to obtain each of the above-mentioned first coordinate vectors.

[0042] Step S102: When the robot's motion trajectory is a circular arc motion trajectory, obtain the coordinates of each trajectory point of the camera trajectory and the coordinates of each trajectory point of the first radar trajectory. The camera trajectory is the motion trajectory of the camera located in the plane containing the x-axis and y-axis in the world coordinate system. The first radar trajectory is the motion trajectory of the radar located in the plane containing the x-axis and y-axis in the world coordinate system.

[0043] Optionally, this application does not limit the specific process of obtaining the coordinates of each trajectory point of the above-mentioned camera trajectory, and any feasible method is within the protection scope of this application.

[0044] In another alternative implementation, such as Figure 3 As shown, step S102 above further includes:

[0045] Step S1021: Pre-lay QR codes on the above-mentioned circular motion trajectory;

[0046] Step S1022: Obtain the x-axis coordinates and y-axis coordinates of the corner points of the QR code in the world coordinate system to obtain multiple fifth coordinate vectors. When the robot moves along the circular arc trajectory, obtain the x-axis coordinates and y-axis coordinates of the corner points of the QR code in the camera coordinate system to obtain multiple sixth coordinate vectors. The fifth coordinate vectors correspond one-to-one with the sixth coordinate vectors.

[0047] Step S1023: Determine the coordinates of each trajectory point of the camera trajectory based on the aforementioned fifth coordinate vector and the corresponding sixth coordinate vector.

[0048] In the above implementation, QR codes are laid out on the circular motion trajectory in advance, and the true x-axis and y-axis coordinates of the corner points of the QR codes in the world coordinate system are measured to obtain the fifth coordinate vector. Then, when the robot moves along the straight trajectory, the extracted x-axis and y-axis coordinates of the corner points of the QR codes are first transformed to the camera coordinate system to obtain the x-axis and y-axis coordinates of the corner points of the QR codes in the camera coordinate system, thereby obtaining the sixth coordinate vector. Then, the true x-axis and y-axis coordinates of the corner points of the QR codes in the world coordinate system are compared with the x-axis and y-axis coordinates of the corner points of the QR codes in the camera coordinate system to obtain the x-axis and y-axis coordinates of each trajectory point in the camera coordinate system in the world coordinate system, that is, the coordinates of each trajectory point of the camera trajectory are obtained.

[0049] Step S103: The first coordinate vector and the second coordinate vector are fitted using the least squares algorithm to obtain the target rotation matrix. The target rotation matrix is ​​a matrix that represents the rotation transformation relationship between the camera coordinate system and the radar coordinate system.

[0050] In another alternative implementation, such as Figure 4 As shown, prior to step S103, the method further includes:

[0051] Step S201: Based on the hand-eye calibration principle, obtain formula R. A ×t X +t A =R X ×t B +t X , where R X Let t be the target rotation matrix mentioned above. X Let t be the target translation vector mentioned above. A Let t be the first coordinate vector mentioned above. B Let R be the second coordinate vector mentioned above. A Let R be the first rotation matrix. B The first rotation matrix is ​​a matrix that represents the rotational transformation relationship between the camera coordinate system and the world coordinate system, and the second rotation matrix is ​​a matrix that represents the rotational transformation relationship between the radar coordinate system and the world coordinate system.

[0052] Step S202: Given that the robot's motion trajectory is the aforementioned linear motion trajectory, determine the aforementioned first rotation matrix R. A It is the identity matrix;

[0053] Step S203, the first rotation matrix R is... A Substitute into formula R A ×t X +t A =R X ×tB +t X After simplification, we obtain the formula t. A =R X ×t B .

[0054] In the above embodiments, when calibrating the target rotation matrix representing the rotational transformation relationship between the camera coordinate system and the radar coordinate system based on a specific linear motion trajectory, when the robot's motion trajectory is an ideal linear motion trajectory, the robot only has translational motion and no rotational motion. Therefore, the first rotation matrix R representing the rotational transformation relationship between the camera coordinate system and the world coordinate system is... A Let R be the identity matrix. Substituting it into the formula based on the hand-eye calibration principle, we obtain the formula R. A ×t X +t A =R X ×t B +t X Simplify the expression to obtain formula t. A =R X ×t B This formula shows that when the robot's motion trajectory is an ideal straight line motion trajectory, the camera trajectory and the radar trajectory have the same shape. At this time, the camera coordinate system and the radar coordinate system differ only in a rotation transformation relationship. Therefore, under this condition, the target rotation matrix can be accurately determined.

[0055] It should be noted that the classic hand-eye calibration model is... R X Let t be the target rotation matrix. X Let t be the target translation vector mentioned above. A Let t be the first coordinate vector mentioned above. B Let R be the second coordinate vector mentioned above. A Let R be the first rotation matrix mentioned above. B Let be the second rotation matrix mentioned above, where the formula can be decomposed into the following two equations. That is, we get the above formula R. A ×t X +t A =R X ×t B +t X .

[0056] Optionally, this application does not limit the specific process of fitting the first coordinate vector and the second coordinate vector using the least squares algorithm to obtain the target rotation matrix, and any feasible method is within the protection scope of this application.

[0057] For example, in yet another alternative implementation, such as Figure 5 As shown, step S102 above includes:

[0058] Step S1031, according to formula t A =R X ×t B Multiple preliminary rotation matrices are calculated, where t A Let t be the first coordinate vector mentioned above. B Let R be the second coordinate vector mentioned above. X The above-mentioned pre-rotation matrix corresponds one-to-one with the above-mentioned first coordinate vector;

[0059] Step S1032, according to formula e R =||R X ×t B -t A || 2 Determine the target rotation matrix as described above, where t A Let t be the first coordinate vector mentioned above. B Let R be the second coordinate vector mentioned above. X For the above pre-rotation matrix, e R The error value is defined as follows: each error value corresponds one-to-one with the aforementioned preliminary rotation matrix; the target rotation matrix is ​​the aforementioned preliminary rotation matrix corresponding to the smallest error value.

[0060] In the above embodiments, since it is impossible to guarantee that the robot will maintain a completely straight-line motion trajectory during actual robot movement, it is necessary to collect the first and second coordinate vectors when the robot performs multiple segments of relatively straight-line motion trajectories. Then, the least squares algorithm is used to fit the first and second coordinate vectors, and the preliminary rotation matrix R that minimizes the error value is determined. X This is used as the target rotation matrix to improve the accuracy of target rotation matrix calibration, thereby improving the accuracy of camera and radar extrinsic parameter calibration.

[0061] Step S104: Based on the target rotation matrix, the coordinates of each trajectory point of the first radar trajectory are transformed to the camera coordinate system to obtain the coordinates of multiple transformed trajectory points, which together form the second radar trajectory.

[0062] After solving for the target rotation matrix, the target rotation matrix R is... X Substitute into the equation Wherein, the target translation vector t X Substituting the zero vector, we get the formula. According to the formula The first radar trajectory is rotated to the camera coordinate system to obtain the second radar trajectory. At this point, the camera trajectory and the second radar trajectory differ only in translation transformation.

[0063] Step S105: Calculate the vector between the center of the circle corresponding to the camera trajectory and the center of the circle corresponding to the second radar trajectory to obtain the target translation vector. The target translation vector is a vector representing the translation transformation relationship between the camera coordinate system and the radar coordinate system.

[0064] In this case, it is assumed that the robot moves along a circular arc trajectory with an angle of θ, such as... Figure 6 As shown, the starting point of both the camera trajectory and the second radar trajectory is Q. i The endpoint of the camera trajectory is Q. I i+1 The endpoint of the second radar trajectory is Q. L i+1 Because the camera coordinate system and the radar coordinate system are transformed by translation, the center of the circle corresponding to the camera trajectory cannot coincide with the center of the circle corresponding to the second radar trajectory. The second radar trajectory is translated so that the radar's trajectory and the camera trajectory have the same center Q. I The starting point of the radar's trajectory after translation is Q. i’ The endpoint is Q. L i+1’ At this point, the problem of solving for the target translation vector is transformed into solving for Q. i’ With Q i Vectors between The problem, namely, solving for the target translation vector, is transformed into solving Q. I With Q L Vectors between The problem.

[0065] Optionally, this application does not limit the specific process of calculating the vector between the center of the circle corresponding to the camera trajectory and the center of the circle corresponding to the second radar trajectory to obtain the target translation vector, and any feasible method is within the protection scope of this application.

[0066] For example, in one alternative implementation, such as Figure 7 As shown, step S104 above includes:

[0067] Step S1041: Based on the camera trajectory and the second radar trajectory, determine the magnitude and direction of the first vector, the magnitude and direction of the second vector, the starting point of the first vector is the center of the circle corresponding to the camera trajectory, the ending point of the first vector is the starting point of the camera trajectory, the starting point of the second vector is the center of the circle corresponding to the second radar trajectory, the ending point of the second vector is the starting point of the second radar trajectory, and the starting point of the camera trajectory is the same as the starting point of the second radar trajectory.

[0068] Step S1042: Based on the magnitude and direction of the first vector, the magnitude and direction of the second vector, the target translation vector is obtained.

[0069] In the above embodiments, such as Figure 6 As shown, the first vector mentioned above is The second vector mentioned above is First, determine the magnitude R of the first vector based on the camera trajectory. I Based on the direction of the first vector and the second radar trajectory, the magnitude R of the second vector is determined. L And the direction of the second vector, then, according to the magnitude R of the first vector. I The direction of the first vector and the magnitude R of the second vector. L The direction of the second vector can be used to determine the target translation vector.

[0070] Optionally, this application does not limit the specific process of determining the magnitude and direction of the first vector, the magnitude and direction of the second vector based on the above-mentioned camera trajectory and the above-mentioned second radar trajectory, and any feasible method is within the protection scope of this application.

[0071] For example, in another alternative implementation, such as Figure 8 As shown, step S1041 above includes:

[0072] Step S10411: Determine the first arc length and the direction of motion of the camera based on the camera trajectory, and determine the second arc length and the direction of motion of the radar based on the second radar trajectory. The first arc length is the arc length of the camera trajectory, and the second arc length is the arc length of the second radar trajectory.

[0073] Step S10412: Calculate the ratio of the first arc length to the angle value to obtain the magnitude of the first vector. The angle value is the angle of the central angle corresponding to the first arc length. The angle of the central angle corresponding to the first arc length is the same as the angle of the central angle corresponding to the second arc length.

[0074] Step S10413: Calculate the ratio of the second arc length to the angle value to obtain the magnitude of the second vector;

[0075] Step S10414: Determine the direction of the first vector based on the camera's motion direction.

[0076] Step S10415: Determine the direction of the second vector based on the direction of motion of the radar.

[0077] In the above embodiments, such as Figure 6 As shown, according to the formula The size R of the first vector is calculated. I , where θ is the aforementioned angle value, Given the first arc length mentioned above, according to the formula... The size R of the second vector is calculated. L ,in, The second arc length is given above. The direction of the first vector is obtained by rotating the camera's motion direction 90 degrees to the right, and the direction of the second vector is obtained by rotating the radar's motion direction 90 degrees to the right. The camera's motion direction is the starting point Q in the camera's trajectory. i The tangential direction at that point, and the radar's direction of motion, are the starting points Q in the second radar trajectory. i The direction of the tangent at that point.

[0078] In the aforementioned joint calibration method, firstly, when the robot's motion trajectory is a straight line, multiple first coordinate vectors and multiple second coordinate vectors are acquired. The first coordinate vectors correspond one-to-one with the second coordinate vectors. The first coordinate vector represents the translational transformation relationship between the camera coordinate system and the world coordinate system, and the second coordinate vector represents the translational transformation relationship between the radar coordinate system and the world coordinate system. The first coordinate vector includes the x-axis and y-axis coordinates of the camera's trajectory points in the world coordinate system, and the second coordinate vector includes the x-axis and y-axis coordinates of the radar's trajectory points in the world coordinate system. The detection period of the first coordinate vector is the same as the detection period of the corresponding second coordinate vector. Then, when the robot's motion trajectory is a circular arc, the coordinates of each trajectory point of the camera trajectory and the coordinates of each trajectory point of the first radar trajectory are acquired. The first radar trajectory is the motion trajectory of the camera located in the plane containing the x and y coordinate axes in the world coordinate system. Then, a least squares algorithm is used to fit the first and second coordinate vectors to obtain a target rotation matrix, which represents the rotational transformation relationship between the camera coordinate system and the radar coordinate system. Next, based on the target rotation matrix, the coordinates of each trajectory point of the first radar trajectory are transformed to the camera coordinate system, resulting in multiple transformed trajectory points. These transformed trajectory points form the second radar trajectory. Finally, the vector between the center of the circle corresponding to the camera trajectory and the center of the circle corresponding to the second radar trajectory is calculated to obtain the target translation vector, which represents the translational transformation relationship between the camera coordinate system and the radar coordinate system. This method uses camera trajectory points in the camera coordinate system and radar trajectory points in the radar coordinate system. Given that the robot moves along a specific straight-line trajectory, the trajectory points in the camera and radar coordinate systems only have a rotational transformation relationship. In this case, the first and second coordinate vectors are fitted to determine the target rotation matrix. Then, based on the robot's movement along a specific circular arc trajectory, the first radar trajectory in the world coordinate system is transformed to the camera coordinate system according to the determined target rotation matrix, resulting in the second radar trajectory. Here, the trajectory points in the camera coordinate system and the radar trajectory in the camera coordinate system only have a translational transformation relationship. By aligning the determined target translation vectors of the camera and second radar trajectories, this method calibrates the target rotation matrix and target translation vector based on a specific motion trajectory. It does not require the camera and radar to have a shared field of view, thus solving the problem in existing technologies where the lack of a shared field of view between the camera and radar often prevents target-based calibration methods from functioning.

[0079] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0080] This application also provides a joint calibration device. It should be noted that the joint calibration device of this application can be used to execute the joint calibration method provided in this application. The joint calibration device provided in this application is described below.

[0081] Figure 9 This is a schematic diagram of a joint calibration apparatus according to an embodiment of this application. Figure 9 As shown, the device includes:

[0082] The first acquisition unit 10 is used to acquire multiple first coordinate vectors and multiple second coordinate vectors when the robot's motion trajectory is a straight line motion trajectory. The first coordinate vectors correspond one-to-one with the second coordinate vectors. The first coordinate vector is a vector representing the translation transformation relationship between the camera coordinate system and the world coordinate system. The second coordinate vector is a vector representing the translation transformation relationship between the radar coordinate system and the world coordinate system. The first coordinate vector includes the x-axis coordinate and y-axis coordinate of the camera's trajectory point in the world coordinate system. The second coordinate vector includes the x-axis coordinate and y-axis coordinate of the radar's trajectory point in the world coordinate system. The detection period to which the first coordinate vector belongs is the same as the detection period to which the corresponding second coordinate vector belongs.

[0083] The second coordinate vector mentioned above contains the x-axis and y-axis coordinates of the radar trajectory points in the world coordinate system after ground compensation. The specific process of ground compensation is as follows: The radar uses ground detection technology. First, it obtains prior information such as the radar installation height and angle. Then, it performs angle compensation on the x-axis and y-axis coordinates of the radar trajectory points in the world coordinate system output by the radar positioning algorithm. The first radar motion trajectory mentioned above is also the radar motion trajectory in the world coordinate system after ground compensation.

[0084] Furthermore, after ground compensation, it is only necessary to solve the rotational transformation relationship between the x-axis and y-axis coordinates of the camera trajectory points and the radar trajectory points when the robot's motion trajectory is a straight line. It is only necessary to solve the translational transformation relationship between the camera trajectory and the radar trajectory located in the plane containing the x-axis and y-axis when the robot's motion trajectory is a circular arc. This simplifies the three-dimensional calibration problem into a two-dimensional calibration problem, improving calibration accuracy. The aforementioned camera motion trajectory is the motion trajectory of the camera coordinate system, and the first and second radar coordinate trajectories are both motion trajectories of the radar coordinate system. Since the distance between the radar and the ground is measurable, the camera coordinate system is the IPM coordinate system, located on the ground. The z-axis coordinates of the radar coordinate system and the z-axis coordinates of the camera coordinate system are fixed and measurable. After completing the two-dimensional calibration, the ground compensation is eliminated based on the distance between the radar coordinate system and the camera coordinate system, and the three-dimensional calibration extrinsic parameters are obtained.

[0085] Optionally, this application does not limit the specific process of obtaining the first coordinate vector, and any feasible method is within the protection scope of this application.

[0086] In one optional implementation, the first acquisition unit includes:

[0087] The first processing module is used to pre-lay QR codes on the aforementioned linear motion trajectory;

[0088] The first acquisition module is used to acquire the x-axis coordinates and y-axis coordinates of the corner points of the QR code in the world coordinate system to obtain a third coordinate vector. When the robot moves along a straight line trajectory, it acquires the x-axis coordinates and y-axis coordinates of the corner points of the QR code in the camera coordinate system to obtain multiple fourth coordinate vectors. The third coordinate vector corresponds one-to-one with the fourth coordinate vector.

[0089] The first determining module is used to determine multiple first coordinate vectors based on each of the aforementioned third coordinate vectors and the corresponding fourth coordinate vectors.

[0090] In the above implementation, QR codes are laid out on the straight-line trajectory in advance, and the actual x-axis and y-axis coordinates of the corner points of the QR codes in the world coordinate system are measured to obtain the third coordinate vector. Then, when the robot moves along the straight-line trajectory, the extracted x-axis and y-axis coordinates of the corner points of the QR codes are first transformed to the camera coordinate system to obtain the x-axis and y-axis coordinates of the corner points of the QR codes in the camera coordinate system, and then the fourth coordinate vector is obtained. Then, the actual x-axis and y-axis coordinates of the corner points of the QR codes in the world coordinate system are compared with the x-axis and y-axis coordinates of the corner points of the QR codes in the camera coordinate system to obtain the x-axis and y-axis coordinates of each trajectory point in the world coordinate system in the camera coordinate system, that is, to obtain each of the above-mentioned first coordinate vectors.

[0091] The second acquisition unit 20 is used to acquire the coordinates of each trajectory point of the camera trajectory and the coordinates of each trajectory point of the first radar trajectory when the robot's motion trajectory is an arc motion trajectory. The camera trajectory is the motion trajectory of the camera located in the plane containing the x-axis and y-axis in the world coordinate system, and the first radar trajectory is the motion trajectory of the radar located in the plane containing the x-axis and y-axis in the world coordinate system.

[0092] Optionally, this application does not limit the specific process of obtaining the coordinates of each trajectory point of the above-mentioned camera trajectory, and any feasible method is within the protection scope of this application.

[0093] In another optional implementation, the second acquisition unit includes:

[0094] The second processing module is used to pre-lay QR codes on the aforementioned circular arc motion trajectory;

[0095] The second acquisition module is used to acquire the x-axis coordinates and y-axis coordinates of the corner points of the QR code in the world coordinate system to obtain multiple fifth coordinate vectors. When the robot moves along the circular arc trajectory, it acquires the x-axis coordinates and y-axis coordinates of the corner points of the QR code in the camera coordinate system to obtain multiple sixth coordinate vectors. The fifth coordinate vectors correspond one-to-one with the sixth coordinate vectors.

[0096] The second determining module is used to determine the coordinates of each trajectory point of the camera trajectory based on each of the aforementioned fifth coordinate vectors and the corresponding sixth coordinate vectors.

[0097] In the above implementation, QR codes are laid out on the circular motion trajectory in advance, and the true x-axis and y-axis coordinates of the corner points of the QR codes in the world coordinate system are measured to obtain the fifth coordinate vector. Then, when the robot moves along the straight trajectory, the extracted x-axis and y-axis coordinates of the corner points of the QR codes are first transformed to the camera coordinate system to obtain the x-axis and y-axis coordinates of the corner points of the QR codes in the camera coordinate system, thereby obtaining the sixth coordinate vector. Then, the true x-axis and y-axis coordinates of the corner points of the QR codes in the world coordinate system are compared with the x-axis and y-axis coordinates of the corner points of the QR codes in the camera coordinate system to obtain the x-axis and y-axis coordinates of each trajectory point in the camera coordinate system in the world coordinate system, that is, the coordinates of each trajectory point of the camera trajectory are obtained.

[0098] Fitting unit 30 is used to fit the first coordinate vector and the second coordinate vector using the least squares algorithm to obtain the target rotation matrix, which is a matrix representing the rotation transformation relationship between the camera coordinate system and the radar coordinate system.

[0099] In another alternative embodiment, the above-mentioned joint calibration device further includes:

[0100] The processing unit is used to obtain formula R based on the hand-eye calibration principle. A ×t X +t A =R X ×t B +t X , where R X Let t be the target rotation matrix mentioned above. X Let t be the target translation vector mentioned above. A Let t be the first coordinate vector mentioned above. B Let R be the second coordinate vector mentioned above. A Let R be the first rotation matrix. B The first rotation matrix is ​​a matrix that represents the rotational transformation relationship between the camera coordinate system and the world coordinate system, and the second rotation matrix is ​​a matrix that represents the rotational transformation relationship between the radar coordinate system and the world coordinate system.

[0101] The determining unit is configured to determine the first rotation matrix R when the robot's motion trajectory is the aforementioned linear motion trajectory. A It is the identity matrix;

[0102] Simplify the simple element, which is used to transform the first rotation matrix R mentioned above. A Substitute into formula R A ×t X +t A =R X ×t B +t X After simplification, we obtain the formula t. A =R X ×t B .

[0103] In the above embodiments, when calibrating the target rotation matrix representing the rotational transformation relationship between the camera coordinate system and the radar coordinate system based on a specific linear motion trajectory, when the robot's motion trajectory is an ideal linear motion trajectory, the robot only has translational motion and no rotational motion. Therefore, the first rotation matrix R representing the rotational transformation relationship between the camera coordinate system and the world coordinate system is... A Let R be the identity matrix. Substituting it into the formula based on the hand-eye calibration principle, we obtain the formula R. A ×t X +t A =R X ×t B +t X Simplify the expression to obtain formula t. A =R X ×t BThis formula shows that when the robot's motion trajectory is an ideal straight line motion trajectory, the camera trajectory and the radar trajectory have the same shape. At this time, the camera coordinate system and the radar coordinate system differ only in a rotation transformation relationship. Therefore, under this condition, the target rotation matrix can be accurately determined.

[0104] It should be noted that the classic hand-eye calibration model is... R X Let t be the target rotation matrix. X Let t be the target translation vector mentioned above. A Let t be the first coordinate vector mentioned above. B Let R be the second coordinate vector mentioned above. A Let R be the first rotation matrix mentioned above. B Let be the second rotation matrix mentioned above, where the formula can be decomposed into the following two equations. That is, we get the above formula R. A ×t X +t A =R X ×t B +t X .

[0105] Optionally, this application does not limit the specific process of fitting the first coordinate vector and the second coordinate vector using the least squares algorithm to obtain the target rotation matrix, and any feasible method is within the protection scope of this application.

[0106] For example, in yet another alternative implementation, the above-mentioned fitting unit includes:

[0107] The calculation module is used to calculate based on formula t. A =R X ×t B Multiple preliminary rotation matrices are calculated, where t A Let t be the first coordinate vector mentioned above. B Let R be the second coordinate vector mentioned above. X The above-mentioned pre-rotation matrix corresponds one-to-one with the above-mentioned first coordinate vector;

[0108] The third determining module is used to determine the formula e. R =||R X ×t B -t A || 2 Determine the target rotation matrix as described above, where t A Let t be the first coordinate vector mentioned above. B Let R be the second coordinate vector mentioned above. X For the above pre-rotation matrix, e RThe error value is defined as follows: each error value corresponds one-to-one with the aforementioned preliminary rotation matrix; the target rotation matrix is ​​the aforementioned preliminary rotation matrix corresponding to the smallest error value.

[0109] In the above embodiments, since it is impossible to guarantee that the robot will maintain a completely straight-line motion trajectory during actual robot movement, it is necessary to collect the first and second coordinate vectors when the robot performs multiple segments of relatively straight-line motion trajectories. Then, the least squares algorithm is used to fit the first and second coordinate vectors, and the preliminary rotation matrix R that minimizes the error value is determined. X This is used as the target rotation matrix to improve the accuracy of target rotation matrix calibration, thereby improving the accuracy of camera and radar extrinsic parameter calibration.

[0110] The conversion unit 40 is used to convert the coordinates of each trajectory point of the first radar trajectory to the camera coordinate system according to the target rotation matrix, so as to obtain the coordinates of multiple converted trajectory points, which together form the second radar trajectory.

[0111] After solving for the target rotation matrix, the target rotation matrix R is... X Substitute into the equation Wherein, the target translation vector t X Substituting the zero vector, we get the formula. According to the formula The first radar trajectory is rotated to the camera coordinate system to obtain the second radar trajectory. At this point, the camera trajectory and the second radar trajectory differ only in translation transformation.

[0112] The calculation unit 40 is used to calculate the vector between the center of the circle corresponding to the camera trajectory and the center of the circle corresponding to the second radar trajectory, so as to obtain the target translation vector. The target translation vector is a vector representing the translation transformation relationship between the camera coordinate system and the radar coordinate system.

[0113] In this case, it is assumed that the robot moves along a circular arc trajectory with an angle of θ, such as... Figure 6 As shown, the starting point of both the camera trajectory and the second radar trajectory is Q. i The endpoint of the camera trajectory is Q. I i+1 The endpoint of the second radar trajectory is Q. L i+1 Because the camera coordinate system and the radar coordinate system are transformed by translation, the center of the circle corresponding to the camera trajectory cannot coincide with the center of the circle corresponding to the second radar trajectory. The second radar trajectory is translated so that the radar's trajectory and the camera trajectory have the same center Q. I The starting point of the radar's trajectory after translation is Q. i’ The endpoint is Q. Li+1’ At this point, the problem of solving for the target translation vector is transformed into solving for Q. i’ With Q i Vectors between The problem, namely, solving for the target translation vector, is transformed into solving Q. I With Q L Vectors between The problem.

[0114] Optionally, this application does not limit the specific process of calculating the vector between the center of the circle corresponding to the camera trajectory and the center of the circle corresponding to the second radar trajectory to obtain the target translation vector, and any feasible method is within the protection scope of this application.

[0115] For example, in one alternative implementation, the computing unit includes:

[0116] The fourth determining module is used to determine the magnitude and direction of the first vector, the magnitude and direction of the second vector, and the direction of the second vector based on the camera trajectory and the second radar trajectory. The starting point of the first vector is the center of the circle corresponding to the camera trajectory, the ending point of the first vector is the starting point of the camera trajectory, the starting point of the second vector is the center of the circle corresponding to the second radar trajectory, the ending point of the second vector is the starting point of the second radar trajectory, and the starting point of the camera trajectory is the same as the starting point of the second radar trajectory.

[0117] The third processing module is used to obtain the target translation vector based on the magnitude and direction of the first vector, the magnitude and direction of the second vector.

[0118] In the above embodiments, such as Figure 6 As shown, the first vector mentioned above is The second vector mentioned above is First, determine the magnitude R of the first vector based on the camera trajectory. I Based on the direction of the first vector and the second radar trajectory, the magnitude R of the second vector is determined. L And the direction of the second vector, then, according to the magnitude R of the first vector. I The direction of the first vector and the magnitude R of the second vector. L The direction of the second vector can be used to determine the target translation vector.

[0119] Optionally, this application does not limit the specific process of determining the magnitude and direction of the first vector, the magnitude and direction of the second vector based on the above-mentioned camera trajectory and the above-mentioned second radar trajectory, and any feasible method is within the protection scope of this application.

[0120] For example, in another alternative implementation, the fourth determining module described above includes:

[0121] The first determining submodule is used to determine the first arc length and the direction of motion of the camera based on the camera trajectory, and to determine the second arc length and the direction of motion of the radar based on the second radar trajectory, wherein the first arc length is the arc length of the camera trajectory and the second arc length is the arc length of the second radar trajectory.

[0122] The first calculation submodule is used to calculate the ratio of the first arc length to the angle value to obtain the size of the first vector. The angle value is the angle of the central angle corresponding to the first arc length. The angle of the central angle corresponding to the first arc length is the same as the angle of the central angle corresponding to the second arc length.

[0123] The second calculation submodule is used to calculate the ratio of the second arc length to the angle value to obtain the magnitude of the second vector.

[0124] The second determining submodule is used to determine the direction of the first vector based on the motion direction of the camera.

[0125] The third determining submodule is used to determine the direction of the second vector based on the direction of motion of the radar.

[0126] In the above embodiments, such as Figure 6 As shown, according to the formula The size R of the first vector is calculated. I , where θ is the aforementioned angle value, Given the first arc length mentioned above, according to the formula... The size R of the second vector is calculated. L ,in, The second arc length is given above. The direction of the first vector is obtained by rotating the camera's motion direction 90 degrees to the right, and the direction of the second vector is obtained by rotating the radar's motion direction 90 degrees to the right. The camera's motion direction is the starting point Q in the camera's trajectory. i The tangential direction at that point, and the radar's direction of motion, are the starting points Q in the second radar trajectory. i The direction of the tangent at that point.

[0127] In the aforementioned joint calibration device, the first acquisition unit is used to acquire multiple first coordinate vectors and multiple second coordinate vectors when the robot's motion trajectory is a straight line motion trajectory. The first coordinate vectors correspond one-to-one with the second coordinate vectors. The first coordinate vector is a vector representing the translational transformation relationship between the camera coordinate system and the world coordinate system, and the second coordinate vector is a vector representing the translational transformation relationship between the radar coordinate system and the world coordinate system. The first coordinate vector includes the x-axis and y-axis coordinates of the camera's trajectory points in the world coordinate system, and the second coordinate vector includes the x-axis and y-axis coordinates of the radar's trajectory points in the world coordinate system. The detection period to which the first coordinate vector belongs is the same as the detection period to which the corresponding second coordinate vector belongs. The second acquisition unit is used to acquire the coordinates of each trajectory point of the camera trajectory and the coordinates of each trajectory point of the first radar trajectory when the robot's motion trajectory is a circular arc motion trajectory. The first radar trajectory is the motion trajectory of the camera located in the plane containing the x-axis and y-axis in the world coordinate system. The fitting unit is used to fit the first coordinate vector and the second coordinate vector using a least squares algorithm to obtain a target rotation matrix, which represents the rotation transformation relationship between the camera coordinate system and the radar coordinate system. The transformation unit is used to transform the coordinates of each trajectory point of the first radar trajectory to the camera coordinate system based on the target rotation matrix, obtaining the coordinates of multiple transformed trajectory points, which form the second radar trajectory. The calculation unit is used to calculate the vector between the center of the circle corresponding to the camera trajectory and the center of the circle corresponding to the second radar trajectory, obtaining a target translation vector, which represents the translation transformation relationship between the camera coordinate system and the radar coordinate system.The device uses camera trajectory points in the camera coordinate system and radar trajectory points in the radar coordinate system. Given that the robot moves along a specific straight-line trajectory, the trajectory points in the camera and radar coordinate systems only have a rotational transformation relationship. In this case, the first and second coordinate vectors are fitted to determine the target rotation matrix. Then, based on the robot's movement along a specific circular arc trajectory, the first radar trajectory in the world coordinate system is transformed to the camera coordinate system according to the determined target rotation matrix, resulting in the second radar trajectory. Here, the trajectory points in the camera coordinate system and the radar trajectory in the camera coordinate system only have a translational transformation relationship. By aligning the determined target translation vectors of the camera and second radar trajectories, the device calibrates the target rotation matrix and target translation vector based on a specific trajectory. This eliminates the need for the camera and radar to share a common field of view, thus solving the problem in existing technologies where the lack of a common field of view between the camera and radar often renders target-based calibration methods ineffective.

[0128] The aforementioned joint calibration device includes a processor and a memory. The first acquisition unit, the second acquisition unit, the fitting unit, the conversion unit, and the calculation unit are all stored in the memory as program units. The processor executes the aforementioned program units stored in the memory to achieve the corresponding functions.

[0129] The processor contains a kernel, which retrieves the corresponding program units from memory. One or more kernels can be configured, and adjusting kernel parameters can address the problem in existing technologies where cameras and radar often lack a common field of view, rendering target-based calibration methods ineffective.

[0130] The memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0131] This invention provides a computer-readable storage medium including a stored program, wherein when the program is executed by a processor, the processor executes any of the aforementioned joint calibration methods.

[0132] This invention provides a device including a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, it performs at least the following steps:

[0133] Step S101: When the robot's motion trajectory is a straight line, multiple first coordinate vectors and multiple second coordinate vectors are acquired. The first coordinate vectors correspond one-to-one with the second coordinate vectors. The first coordinate vector is a vector representing the translation transformation relationship between the camera coordinate system and the world coordinate system. The second coordinate vector is a vector representing the translation transformation relationship between the radar coordinate system and the world coordinate system. The first coordinate vector includes the x-axis coordinate and y-axis coordinate of the camera's trajectory point in the world coordinate system. The second coordinate vector includes the x-axis coordinate and y-axis coordinate of the radar's trajectory point in the world coordinate system. The detection period to which the first coordinate vector belongs is the same as the detection period to which the corresponding second coordinate vector belongs.

[0134] Step S102: When the robot's motion trajectory is a circular arc motion trajectory, obtain the coordinates of each trajectory point of the camera trajectory and the coordinates of each trajectory point of the first radar trajectory. The camera trajectory is the motion trajectory of the camera located in the plane containing the x-axis and y-axis in the world coordinate system. The first radar trajectory is the motion trajectory of the radar located in the plane containing the x-axis and y-axis in the world coordinate system.

[0135] Step S103: The first coordinate vector and the second coordinate vector are fitted using the least squares algorithm to obtain the target rotation matrix. The target rotation matrix is ​​a matrix that represents the rotation transformation relationship between the camera coordinate system and the radar coordinate system.

[0136] Step S104: Based on the target rotation matrix, the coordinates of each trajectory point of the first radar trajectory are transformed to the camera coordinate system to obtain the coordinates of multiple transformed trajectory points, which together form the second radar trajectory.

[0137] Step S105: Calculate the vector between the center of the circle corresponding to the camera trajectory and the center of the circle corresponding to the second radar trajectory to obtain the target translation vector. The target translation vector is a vector representing the translation transformation relationship between the camera coordinate system and the radar coordinate system.

[0138] The devices mentioned in this article can be servers, PCs, tablets, mobile phones, etc.

[0139] This application also provides a computer program product, which, when executed on a data processing device, is suitable for executing an initialization program having at least the following method steps:

[0140] Step S101: When the robot's motion trajectory is a straight line, multiple first coordinate vectors and multiple second coordinate vectors are acquired. The first coordinate vectors correspond one-to-one with the second coordinate vectors. The first coordinate vector is a vector representing the translation transformation relationship between the camera coordinate system and the world coordinate system. The second coordinate vector is a vector representing the translation transformation relationship between the radar coordinate system and the world coordinate system. The first coordinate vector includes the x-axis coordinate and y-axis coordinate of the camera's trajectory point in the world coordinate system. The second coordinate vector includes the x-axis coordinate and y-axis coordinate of the radar's trajectory point in the world coordinate system. The detection period to which the first coordinate vector belongs is the same as the detection period to which the corresponding second coordinate vector belongs.

[0141] Step S102: When the robot's motion trajectory is a circular arc motion trajectory, obtain the coordinates of each trajectory point of the camera trajectory and the coordinates of each trajectory point of the first radar trajectory. The camera trajectory is the motion trajectory of the camera located in the plane containing the x-axis and y-axis in the world coordinate system. The first radar trajectory is the motion trajectory of the radar located in the plane containing the x-axis and y-axis in the world coordinate system.

[0142] Step S103: The first coordinate vector and the second coordinate vector are fitted using the least squares algorithm to obtain the target rotation matrix. The target rotation matrix is ​​a matrix that represents the rotation transformation relationship between the camera coordinate system and the radar coordinate system.

[0143] Step S104: Based on the target rotation matrix, the coordinates of each trajectory point of the first radar trajectory are transformed to the camera coordinate system to obtain the coordinates of multiple transformed trajectory points, which together form the second radar trajectory.

[0144] Step S105: Calculate the vector between the center of the circle corresponding to the camera trajectory and the center of the circle corresponding to the second radar trajectory to obtain the target translation vector. The target translation vector is a vector representing the translation transformation relationship between the camera coordinate system and the radar coordinate system.

[0145] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0146] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units described above can be a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0147] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0148] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0149] If the aforementioned integrated units are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a computer-readable storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned computer-readable storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0150] As can be seen from the above description, the embodiments of this application achieve the following technical effects:

[0151] 1) In the joint calibration method of this application, firstly, when the robot's motion trajectory is a straight line, multiple first coordinate vectors and multiple second coordinate vectors are acquired. The first coordinate vectors correspond one-to-one with the second coordinate vectors. The first coordinate vector represents the translational transformation relationship between the camera coordinate system and the world coordinate system, and the second coordinate vector represents the translational transformation relationship between the radar coordinate system and the world coordinate system. The first coordinate vector includes the x-axis and y-axis coordinates of the camera's trajectory points in the world coordinate system, and the second coordinate vector includes the x-axis and y-axis coordinates of the radar's trajectory points in the world coordinate system. The detection period of the first coordinate vector is the same as the detection period of the corresponding second coordinate vector. Then, when the robot's motion trajectory is a circular arc, the coordinates of each trajectory point of the camera trajectory and the coordinates of each trajectory point of the first radar trajectory are acquired. The trajectory is the motion trajectory of the camera located in the plane containing the x and y coordinate axes in the world coordinate system, and the first radar trajectory is the motion trajectory of the radar located in the plane containing the x and y coordinate axes in the world coordinate system. Then, the first coordinate vector and the second coordinate vector are fitted using a least squares algorithm to obtain a target rotation matrix, which represents the rotational transformation relationship between the camera coordinate system and the radar coordinate system. Next, based on the target rotation matrix, the coordinates of each trajectory point of the first radar trajectory are transformed to the camera coordinate system, resulting in multiple transformed trajectory points. These transformed trajectory points form the second radar trajectory. Finally, the vector between the center of the circle corresponding to the camera trajectory and the center of the circle corresponding to the second radar trajectory is calculated to obtain the target translation vector, which represents the translational transformation relationship between the camera coordinate system and the radar coordinate system. This method uses camera trajectory points in the camera coordinate system and radar trajectory points in the radar coordinate system. Given that the robot moves along a specific straight-line trajectory, the trajectory points in the camera and radar coordinate systems only have a rotational transformation relationship. In this case, the first and second coordinate vectors are fitted to determine the target rotation matrix. Then, based on the robot's movement along a specific circular arc trajectory, the first radar trajectory in the world coordinate system is transformed to the camera coordinate system according to the determined target rotation matrix, resulting in the second radar trajectory. Here, the trajectory points in the camera coordinate system and the radar trajectory in the camera coordinate system only have a translational transformation relationship. By aligning the determined target translation vectors of the camera and second radar trajectories, this method calibrates the target rotation matrix and target translation vector based on a specific motion trajectory. It does not require the camera and radar to have a shared field of view, thus solving the problem in existing technologies where the lack of a shared field of view between the camera and radar often prevents target-based calibration methods from functioning.

[0152] 2) In the joint calibration device of this application, the first acquisition unit is used to acquire multiple first coordinate vectors and multiple second coordinate vectors when the robot's motion trajectory is a straight line motion trajectory. The first coordinate vectors correspond one-to-one with the second coordinate vectors. The first coordinate vector is a vector representing the translation transformation relationship between the camera coordinate system and the world coordinate system, and the second coordinate vector is a vector representing the translation transformation relationship between the radar coordinate system and the world coordinate system. The first coordinate vector includes the x-axis coordinate and y-axis coordinate of the camera's trajectory point in the world coordinate system, and the second coordinate vector includes the x-axis coordinate and y-axis coordinate of the radar's trajectory point in the world coordinate system. The detection period to which the first coordinate vector belongs is the same as the detection period to which the corresponding second coordinate vector belongs. The second acquisition unit is used to acquire the coordinates of each trajectory point of the camera trajectory and the coordinates of each trajectory point of the first radar trajectory when the robot's motion trajectory is a circular arc motion trajectory. The trajectory is the motion trajectory of the camera located in the plane containing the x-axis and y-axis in the world coordinate system, and the first radar trajectory is the motion trajectory of the radar located in the plane containing the x-axis and y-axis in the world coordinate system. A fitting unit is used to fit the first coordinate vector and the second coordinate vector using a least squares algorithm to obtain a target rotation matrix, which represents the rotation transformation relationship between the camera coordinate system and the radar coordinate system. A transformation unit is used to transform the coordinates of each trajectory point of the first radar trajectory to the camera coordinate system based on the target rotation matrix, obtaining the coordinates of multiple transformed trajectory points, which together form the second radar trajectory. A calculation unit is used to calculate the vector between the center of the circle corresponding to the camera trajectory and the center of the circle corresponding to the second radar trajectory, obtaining a target translation vector, which represents the translation transformation relationship between the camera coordinate system and the radar coordinate system.The device uses camera trajectory points in the camera coordinate system and radar trajectory points in the radar coordinate system. Given that the robot moves along a specific straight-line trajectory, the trajectory points in the camera and radar coordinate systems only have a rotational transformation relationship. In this case, the first and second coordinate vectors are fitted to determine the target rotation matrix. Then, based on the robot's movement along a specific circular arc trajectory, the first radar trajectory in the world coordinate system is transformed to the camera coordinate system according to the determined target rotation matrix, resulting in the second radar trajectory. Here, the trajectory points in the camera coordinate system and the radar trajectory in the camera coordinate system only have a translational transformation relationship. By aligning the determined target translation vectors of the camera and second radar trajectories, the device calibrates the target rotation matrix and target translation vector based on a specific trajectory. This eliminates the need for the camera and radar to share a common field of view, thus solving the problem in existing technologies where the lack of a common field of view between the camera and radar often renders target-based calibration methods ineffective.

[0153] 3) The joint calibration system of this application includes: one or more processors, a memory, a display device, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include methods for performing any of the joint calibration methods described above. In this system, the camera's trajectory points are in the camera coordinate system, and the radar's trajectory points are in the radar coordinate system. Given that the robot moves along a specific straight-line trajectory, the trajectory points in the camera and radar coordinate systems only have a rotational transformation relationship. In this case, the first and second coordinate vectors are fitted to determine the target rotation matrix. Then, based on the robot's movement along a specific circular arc trajectory, the first radar trajectory in the world coordinate system is transformed to the camera coordinate system according to the determined target rotation matrix, resulting in the second radar trajectory. Here, the trajectory points in the camera coordinate system and the radar trajectory in the camera coordinate system only have a translational transformation relationship. By aligning the determined target translation vectors of the camera and second radar trajectories, the system calibrates the target rotation matrix and target translation vector based on a specific motion trajectory. This eliminates the need for the camera and radar to have a shared field of view, thus solving the problem in existing technologies where the lack of a shared field of view between the camera and radar often renders target-based calibration methods ineffective.

[0154] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A joint calibration method, characterized in that, The calibration method, applied to image acquisition devices and radars installed on robots, includes: When the robot's motion trajectory is a straight line, multiple first coordinate vectors and multiple second coordinate vectors are acquired. The first coordinate vectors correspond one-to-one with the second coordinate vectors. The first coordinate vector is a vector representing the translational transformation relationship between the camera coordinate system and the world coordinate system, and the second coordinate vector is a vector representing the translational transformation relationship between the radar coordinate system and the world coordinate system. The first coordinate vector includes the x-axis and y-axis coordinates of the camera's trajectory points in the world coordinate system, and the second coordinate vector includes the x-axis and y-axis coordinates of the radar's trajectory points in the world coordinate system. The detection period to which the first coordinate vector belongs is the same as the detection period to which the corresponding second coordinate vector belongs. The x-axis and y-axis coordinates of the radar's trajectory points in the world coordinate system included in the second coordinate vector are the x-axis and y-axis coordinates of the radar's trajectory points in the world coordinate system after ground compensation. The specific process of ground compensation includes: the radar uses ground detection technology to acquire prior information on the radar's installation height and angle, and provides angle compensation for the x-axis and y-axis coordinates of the radar's trajectory points in the world coordinate system output by the radar positioning algorithm. When the robot's motion trajectory is a circular arc, the coordinates of each trajectory point of the camera trajectory and the coordinates of each trajectory point of the first radar trajectory are obtained. The camera trajectory is the motion trajectory of the camera located in the plane containing the x-axis and y-axis in the world coordinate system. The first radar trajectory is the motion trajectory of the radar located in the plane containing the x-axis and y-axis in the world coordinate system. The first radar trajectory is the motion trajectory of the radar in the world coordinate system after ground compensation. Since the distance between the radar and the ground is measurable, the camera coordinate system is the IPM coordinate system, located on the ground. The z-axis coordinates of the radar coordinate system and the z-axis coordinates of the camera coordinate system are fixed and measurable. After completing the two-dimensional calibration, the ground compensation is eliminated based on the distance between the radar coordinate system and the camera coordinate system to obtain the three-dimensional calibration extrinsic parameters. The first coordinate vector and the second coordinate vector are fitted using the least squares algorithm to obtain the target rotation matrix, which is a matrix representing the rotation transformation relationship between the camera coordinate system and the radar coordinate system; Based on the target rotation matrix, the coordinates of each trajectory point of the first radar trajectory are transformed to the camera coordinate system to obtain the coordinates of multiple transformed trajectory points, which together form the second radar trajectory. Calculate the vector between the center of the circle corresponding to the camera trajectory and the center of the circle corresponding to the second radar trajectory to obtain the target translation vector, which is a vector representing the translation transformation relationship between the camera coordinate system and the radar coordinate system.

2. The calibration method according to claim 1, characterized in that, Before fitting the first coordinate vector and the second coordinate vector using the least squares algorithm to obtain the target rotation matrix, the method further includes: Based on the hand-eye calibration principle, the formula is obtained. , where R X Let t be the target rotation matrix. X Let t be the target translation vector. A Let t be the first coordinate vector. B Let R be the second coordinate vector. A Let R be the first rotation matrix. B The first rotation matrix is ​​a matrix representing the rotational transformation relationship between the camera coordinate system and the world coordinate system, and the second rotation matrix is ​​a matrix representing the rotational transformation relationship between the radar coordinate system and the world coordinate system. When the robot's motion trajectory is the linear motion trajectory, the first rotation matrix R is determined. A It is the identity matrix; The first rotation matrix R A Substitute into the formula After simplification, we obtain the formula. .

3. The calibration method according to any one of claims 1 or 2, characterized in that, The target rotation matrix is ​​obtained by fitting the first coordinate vector and the second coordinate vector using the least squares algorithm, including: According to the formula Multiple preliminary rotation matrices are calculated, where t A Let t be the first coordinate vector. B Let R be the second coordinate vector. X The preparatory rotation matrix is ​​given, and the preparatory rotation matrix corresponds one-to-one with the first coordinate vector; According to the formula Determine the target rotation matrix, where t A Let t be the first coordinate vector. B Let R be the second coordinate vector. X For the pre-rotation matrix, The error value is a one-to-one correspondence between the error value and the preliminary rotation matrix, and the target rotation matrix is ​​the preliminary rotation matrix corresponding to the smallest error value.

4. The calibration method according to claim 1, characterized in that, Calculate the vector between the center of the circle corresponding to the camera trajectory and the center of the circle corresponding to the second radar trajectory to obtain the target translation vector, including: Based on the camera trajectory and the second radar trajectory, the magnitude and direction of the first vector, the magnitude and direction of the second vector are determined. The starting point of the first vector is the center of the circle corresponding to the camera trajectory, and the ending point of the first vector is the starting point of the camera trajectory. The starting point of the second vector is the center of the circle corresponding to the second radar trajectory, and the ending point of the second vector is the starting point of the second radar trajectory. The starting point of the camera trajectory is the same as the starting point of the second radar trajectory. The target translation vector is obtained based on the magnitude and direction of the first vector, the magnitude and direction of the second vector.

5. The calibration method according to claim 4, characterized in that, Based on the camera trajectory and the second radar trajectory, the magnitude and direction of the first vector, the magnitude and direction of the second vector are determined, including: The first arc length and the direction of motion of the camera are determined based on the camera trajectory, and the second arc length and the direction of motion of the radar are determined based on the second radar trajectory. The first arc length is the arc length of the camera trajectory, and the second arc length is the arc length of the second radar trajectory. Calculate the ratio of the first arc length to the angle value to obtain the magnitude of the first vector. The angle value is the angle of the central angle corresponding to the first arc length. The angle of the central angle corresponding to the first arc length is the same as the angle of the central angle corresponding to the second arc length. Calculate the ratio of the second arc length to the angle value to obtain the magnitude of the second vector; The direction of the first vector is determined based on the direction of motion of the camera; The direction of the second vector is determined based on the direction of motion of the radar.

6. The calibration method according to claim 1, characterized in that, The method for obtaining the first coordinate vector includes: QR codes are pre-laid on the linear motion trajectory; The x-axis coordinates and y-axis coordinates of the corner points of the QR code in the world coordinate system are obtained to obtain a third coordinate vector. When the robot moves along a straight line trajectory, the x-axis coordinates and y-axis coordinates of the corner points of the QR code in the camera coordinate system are obtained to obtain multiple fourth coordinate vectors. The third coordinate vector corresponds one-to-one with the fourth coordinate vector. Based on each of the third coordinate vectors and the corresponding fourth coordinate vectors, a plurality of first coordinate vectors are determined.

7. The calibration method according to claim 1, characterized in that, The method for obtaining the coordinates of each trajectory point of the camera trajectory includes: QR codes are pre-laid on the circular motion trajectory; The x-axis coordinates and y-axis coordinates of the corner points of the QR code in the world coordinate system are obtained to obtain multiple fifth coordinate vectors. When the robot moves along the circular arc trajectory, the x-axis coordinates and y-axis coordinates of the corner points of the QR code in the camera coordinate system are obtained to obtain multiple sixth coordinate vectors. The fifth coordinate vectors correspond one-to-one with the sixth coordinate vectors. The coordinates of each trajectory point of the camera trajectory are determined based on the fifth coordinate vector and the corresponding sixth coordinate vector.

8. A joint calibration device, characterized in that, The calibration device, applied to image acquisition equipment and radar installed on a robot, includes: The first acquisition unit is used to acquire multiple first coordinate vectors and multiple second coordinate vectors when the robot's motion trajectory is a straight line trajectory. The first coordinate vectors correspond one-to-one with the second coordinate vectors. The first coordinate vector is a vector representing the translation transformation relationship between the camera coordinate system and the world coordinate system, and the second coordinate vector is a vector representing the translation transformation relationship between the radar coordinate system and the world coordinate system. The first coordinate vector includes the x-axis coordinates and y-axis coordinates of the camera's trajectory points in the world coordinate system, and the second coordinate vector includes the x-axis coordinates and y-axis coordinates of the radar's trajectory points in the world coordinate system. The detection period to which the first coordinate vector belongs is the same as the detection period to which the corresponding second coordinate vector belongs. The x-axis coordinates and y-axis coordinates of the radar's trajectory points in the world coordinate system included in the second coordinate vector are the x-axis coordinates and y-axis coordinates of the radar's trajectory points in the world coordinate system after ground compensation. The specific process of ground compensation includes: the radar uses ground detection technology to acquire prior information on the radar's installation height and angle, and provides angle compensation to the x-axis coordinates and y-axis coordinates of the radar's trajectory points in the world coordinate system output by the radar positioning algorithm. The second acquisition unit is used to acquire the coordinates of each trajectory point of the camera trajectory and the coordinates of each trajectory point of the first radar trajectory when the robot's motion trajectory is an arc motion trajectory. The camera trajectory is the motion trajectory of the camera located in the plane containing the x-axis and y-axis in the world coordinate system. The first radar trajectory is the motion trajectory of the radar located in the plane containing the x-axis and y-axis in the world coordinate system. The first radar trajectory is the motion trajectory of the radar in the world coordinate system after ground compensation. Since the distance between the radar and the ground is measurable, the camera coordinate system is the IPM coordinate system, located on the ground. The z-axis coordinates of the radar coordinate system and the z-axis coordinates of the camera coordinate system are fixed and measurable. After completing the two-dimensional calibration, the ground compensation is eliminated according to the distance between the radar coordinate system and the camera coordinate system to obtain the three-dimensional calibration extrinsic parameters. The fitting unit is used to fit the first coordinate vector and the second coordinate vector using the least squares algorithm to obtain the target rotation matrix, wherein the target rotation matrix is ​​a matrix representing the rotation transformation relationship between the camera coordinate system and the radar coordinate system; The conversion unit is used to convert the coordinates of each trajectory point of the first radar trajectory to the camera coordinate system according to the target rotation matrix, so as to obtain the coordinates of multiple converted trajectory points, and the converted trajectory points form the second radar trajectory. The calculation unit is used to calculate the vector between the center of the circle corresponding to the camera trajectory and the center of the circle corresponding to the second radar trajectory, to obtain the target translation vector, which is a vector representing the translation transformation relationship between the camera coordinate system and the radar coordinate system.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein when the program is executed by a processor, the processor performs the joint calibration method according to any one of claims 1 to 7.

10. A joint calibration system, characterized in that, include: One or more processors, a memory, a display device, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including methods for performing the joint calibration method according to any one of claims 1 to 7.

Images