A camera calibration method

Abstract

The application discloses a camera calibration method, when a robot calibrates internal and external parameters of a camera, the internal parameters of the camera are optimized through images of a calibration board, then the external parameters of the camera are calibrated through the pose of the camera relative to each calibration board obtained according to the images of the calibration board and the pose of the calibration board of a calibration jig relative to the center point of the calibration jig obtained in advance, so that the robot can complete the calibration of the internal and external parameters of the camera in one camera calibration operation, the time for calibrating the internal and external parameters of the camera is shortened, and the production efficiency of the robot is improved; the robot obtains the pose of the calibration board of the calibration jig relative to the center point of the calibration jig before camera calibration, and as long as the center point of the robot with different external parameters coincides with the center point of the calibration jig, the external parameters of the robot can be calculated through the pose of the calibration board of the calibration jig relative to the center point of the calibration jig.

Description

Technical Field

[0001] This invention relates to the field of camera technology, and more specifically to a camera calibration method. Background Technology

[0002] In the fields of computer vision and graphics, applications such as security monitoring, interactive motion-sensing games, autonomous driving, and 3D environmental modeling all require extrinsic parameter calibration of cameras to map the positional relationship between the real physical space and the camera's field of view. For example, for robots, cameras are crucial for acquiring external information. To obtain an accurate position based on its own coordinate system, the robot needs to calibrate both the intrinsic and extrinsic parameters of the camera. Currently, the industry practice for calibrating the intrinsic and extrinsic parameters of robot cameras is to calibrate them separately, requiring two calibration operations, which is time-consuming and reduces mass production efficiency. Summary of the Invention

[0003] To address the above problems, this invention provides a camera calibration method. The specific technical solution of this invention is as follows:

[0004] A camera calibration method includes the following steps: S1: A robot acquires an image of a calibration plate of a calibration fixture using a camera, and then identifies calibration points from the image of the calibration plate; S2: The robot calibrates the intrinsic parameters of the camera and acquires the pose of the camera relative to each calibration plate based on the positions of the calibration points on the calibration plate and the positions of the calibration points in the image; S3: The robot checks the intrinsic parameter calibration results of the camera. If the check result is correct, proceed to step S4; if the check result is incorrect, recalibrate the intrinsic parameters of the camera; S4: The robot calibrates the extrinsic parameters of the camera using the pose of the calibration plate of the calibration fixture relative to the center point of the calibration fixture and the pose of the camera relative to each calibration plate.

[0005] Further, in step S1, the robot acquires an image of several calibration plates through the camera, including the following steps: the robot moves into a calibration fixture composed of several calibration plates, and then the center point of the robot coincides with the center point of the calibration fixture; the robot makes the camera vertically upward, and then acquires an image of all the calibration plates of the calibration fixture through the camera.

[0006] Further, in step S1, the robot identifies calibration points from the image of the calibration board, including the following steps: the robot detects the black area of ​​the quadrilateral in the image of the calibration board; the robot connects the detected black area of ​​the quadrilateral in a clockwise direction; the robot obtains the inner quadrilateral of the black area of ​​the quadrilateral; the robot uses corner detection to take the four endpoints of the inner quadrilateral as the calibration points of the calibration board; wherein, the calibration board is a checkerboard calibration board, and the four sides of the inner quadrilateral are respectively connected to the black area of ​​the quadrilateral.

[0007] Further, in step S2, the robot calibrates the camera's intrinsic parameters and obtains the camera's pose relative to each calibration board based on the positions of the calibration points on the calibration board and the positions of the calibration points in the image. This includes the following steps: the robot obtains the physical coordinates of the calibration points in the physical coordinate system of the calibration board and the pixel coordinates of the calibration points in the image; the robot projects the calibration points on the calibration board onto the image of the calibration board based on the initial pose of each calibration board relative to the robot's center point and the initial intrinsic parameters of the camera, obtaining the projected coordinates of the calibration points on the calibration board; the robot calculates the Euclidean distance between the projected coordinates of the calibration points on the calibration board and the pixel coordinates of the corresponding calibration points in the image using the Euclidean distance formula; the robot optimizes the camera's intrinsic parameters using the obtained Euclidean distance and the iterative formula of the Gauss-Newton method to obtain the optimal camera intrinsic parameters and the camera's pose relative to each calibration board.

[0008] Furthermore, when the robot optimizes the intrinsic parameters of the camera using the iterative formula of the Gauss-Newton method, when it obtains the minimum Euclidean distance during the calculation process, it takes the pose of each calibration plate and the robot's center point corresponding to the minimum Euclidean distance and the intrinsic parameters of the camera as the optimal intrinsic parameters of the camera and the pose of the camera relative to each calibration plate.

[0009] Furthermore, the robot acquires the initial pose of the calibration board relative to the robot's center point, including the following steps: the robot draws a perpendicular line from the upper left corner of the calibration board; the robot uses the angle between the perpendicular line and the robot's forward direction as the angle between the calibration board and the robot; the robot acquires the distance between the robot's center point and the upper left corner of the calibration board; the robot combines the angle between the calibration board and the robot and the distance between the robot's center point and the upper left corner of the calibration board to obtain the initial pose of the calibration board relative to the robot's center point.

[0010] Furthermore, the robot obtains the physical coordinates of the calibration point in the physical coordinate system of the calibration board, including the following steps: the robot constructs a physical coordinate system with the top left corner of each calibration board as the origin, and the two sides of the calibration board connecting the top left corner as the X-axis and Y-axis, respectively; the robot obtains the coordinates of the calibration point in the physical coordinate system based on the position of the calibration point on the calibration board.

[0011] Further, in step S3, the robot detects the intrinsic parameter calibration results of the camera, including the following steps: the robot classifies the calibration points in the image and determines the calibration board to which the identified calibration points belong; the robot converts the pixel coordinates of the identified calibration points belonging to the same calibration board into the physical coordinates of the physical coordinate system of the calibration board through the camera intrinsic parameters and the pose of the camera relative to the calibration board, thereby obtaining the physical coordinates of the calibration points in the image;

[0012] The robot acquires the physical coordinates of the calibration points in the image and the Euclidean distance between the physical coordinates of the calibration points on the corresponding calibration board. If the average of a set number of Euclidean distances is greater than a first set threshold, the calibration board is determined to be calibrated incorrectly; otherwise, it is calibrated correctly. The robot fits the calibration points of the same calibrated calibration board into several straight lines and then compares the perpendicular distance between two adjacent and parallel straight lines. If the perpendicular distance between any two straight lines is greater than a second set threshold, the calibration board is determined to be calibrated incorrectly; otherwise, it is calibrated correctly. The robot deletes the calibrated calibration board and the identified calibration points belonging to that calibration board, and then recalibrates the camera's intrinsic parameters.

[0013] Further, in step S2, the robot classifies the identified calibration points and determines the calibration board to which each calibration point belongs, including the following steps: A1: The robot randomly selects the same number of calibration points as the calibration board as the cluster center; A2: The robot obtains the cluster distance between the calibration point and the center of each cluster, and then assigns the calibration point to the cluster with the smallest cluster distance; A3: The robot obtains the average value of the image coordinates of the calibration points in each cluster, and then uses the average value as the new cluster center; A4: The robot repeats steps A2 and A3 until the average value of the image coordinates of the calibration points in each cluster no longer changes, and then the robot determines the calibration board to which the calibration points of each cluster belong based on the center of each cluster.

[0014] Further, in step S4, the robot calibrates the extrinsic parameters of the camera by using the pose of the calibration plate of the calibration fixture relative to the center point of the calibration fixture and the pose of the camera relative to each calibration plate. This includes the following steps: the robot obtains the rotation matrix of the pose of the calibration plate of the calibration fixture relative to the center point of the calibration fixture; the robot obtains the rotation matrix of the pose of the camera relative to one of the calibration plates; the robot transforms the rotation matrix of the pose of the calibration plate of the calibration fixture relative to the center point of the calibration fixture and the rotation matrix of the pose of the camera relative to one of the calibration plates according to Euler's formula to obtain the extrinsic parameter matrix of the camera.

[0015] Furthermore, before calibrating the camera, the robot first acquires the pose of the calibration plate of the calibration fixture relative to the center point of the calibration fixture, including the following steps: the robot moves the calibration plate within the camera's field of view and then moves it to different preset positions to acquire several images of the calibration plate, and acquires IMU data during the movement; the robot identifies the calibration points from the acquired images of the calibration plate, and then uses a nonlinear optimization algorithm to calculate the pose of the camera relative to the calibration plate at different preset positions based on the identified calibration points; the robot uses a nonlinear optimization algorithm to calculate the center point of the camera relative to the robot based on the IMU data and the pose of the camera relative to the calibration plate at different preset positions. Pose; The robot moves into a calibration fixture consisting of several calibration plates, and then sets the current position of the robot's center point as the center point of the calibration fixture; The robot aligns the camera vertically upwards, and then acquires images of all the calibration plates in the calibration fixture through the camera, and identifies the calibration points from the images of the calibration plates; The robot obtains the pose of the camera relative to each calibration plate based on the positions of the calibration points on the calibration plates and the positions of the calibration points in the images; Based on the pose of the camera relative to each calibration plate and the pose of the camera relative to the robot's center point, the robot uses matrix transformation to obtain the pose of the calibration plates of the calibration fixture relative to the center point of the calibration fixture.

[0016] Compared with existing technologies, the advantages of this application are as follows: When the robot of this application performs camera intrinsic and extrinsic parameter calibration, it first optimizes the camera's intrinsic parameters using images from calibration boards to improve the camera's accuracy; then, it calibrates the camera's extrinsic parameters by using the camera's pose relative to each calibration board obtained from the calibration board images and the pose of the calibration board relative to the center point of the calibration fixture obtained in advance. This allows the robot to complete the calibration of both the camera's intrinsic and extrinsic parameters in a single camera calibration operation, shortening the calibration time and improving the robot's efficiency. The robot's mass production efficiency is improved. Before camera calibration, the robot acquires the pose of the calibration plate relative to the center point of the calibration fixture. By aligning the center point of the robot with the center point of the calibration fixture for different extrinsic parameters, the robot's extrinsic parameters can be calculated from the pose of the calibration plate relative to the center point of the calibration fixture, resulting in high flexibility. After calibrating the camera's intrinsic parameters using the image from the calibration plate, the robot verifies the camera's intrinsic parameters to improve the accuracy of the camera's intrinsic parameter calibration results and the accuracy of the extrinsic parameter calibration results using the same data. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating a camera calibration method in one embodiment of the present invention. Detailed Implementation

[0018] The embodiments of the present invention are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout.

[0019] The following description, in conjunction with the accompanying drawings, further illustrates specific embodiments of the present invention, making the technical solution and its beneficial effects clearer and more explicit. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, but should not be construed as limiting the invention.

[0020] like Figure 1 As shown, a camera calibration method includes the following steps:

[0021] S1: The robot acquires images of the calibration plates on the calibration fixture via a camera, and then identifies calibration points from the images. The calibration fixture consists of several calibration plates, the number and size of which are adjusted according to the robot's needs; there are no limitations on the number or size of the calibration plates. Calibration plates can be checkerboard calibration plates, ArUco calibration plates (ArUco is an open-source library that estimates camera pose based on preset black and white markers (QR codes)), or ChArUco calibration plates (ChArUco calibration plates are a combination of checkerboard and ArUco calibration plates), etc. Depending on the type of calibration plate, calibration points can be corner points, round points, or Charuco corners, etc. The robot can identify calibration points from the images of the calibration plates using a corner detection algorithm. S2: The robot calibrates the camera's intrinsic parameters and obtains the camera's pose relative to each calibration plate based on the positions of the calibration points on the calibration plates and the positions of the calibration points in the images. The calibration points on the calibration board are the points on the calibration board in the actual calibration fixture. The position of the calibration points on the calibration board is determined by the type and size of the calibration board. The position of the calibration points on the calibration board is different for different types and sizes of calibration boards. Users can detect this using auxiliary tools before calibrating the camera. The calibration points in the image are the calibration points identified in the calibration board image. The position of the calibration points in the image is the pixel coordinate of the image, obtained from the number of pixels occupied by the calibration point when the robot identifies the calibration point in the image. S3: The robot checks the intrinsic parameter calibration result of the camera. If the detection result is correct, proceed to step S4. If the detection result is incorrect, process the calibration points and recalibrate the camera's intrinsic parameters through step S2. After the robot determines the camera's intrinsic parameters, the intrinsic parameter calibration result can be checked to improve the accuracy of the camera's intrinsic parameter calibration result. The intrinsic parameters of the camera calibrated by the robot are generally an intrinsic parameter matrix, which includes the focal length and center. In addition, for wide-angle cameras, the intrinsic parameters of the camera are projection functions rather than intrinsic parameter matrices. The projection function includes focal length, center, and camera distortion parameters. S4: The robot calibrates the extrinsic parameters of the camera by using the pose of the calibration plate of the calibration fixture relative to the center point of the calibration fixture and the pose of the camera relative to each calibration plate. The extrinsic parameter of the camera is the pose of the camera relative to the center point of the robot. In this application, when the robot determines the pose of the camera relative to each calibration plate, the pose of the calibration plate of the calibration fixture relative to the center point of the calibration fixture, and the pose of the camera relative to the center point of the robot, the positive direction of the camera is the direction in which the camera shoots forward, the positive direction of the robot is the forward direction of the robot, and the positive direction of the calibration plate of the calibration fixture is determined according to the physical coordinate system of the calibration plate. When calibrating the intrinsic and extrinsic parameters of the camera, the robot described in this application first optimizes the intrinsic parameters of the camera using the image of the calibration plate to improve the accuracy of the camera.Then, by obtaining the camera's pose relative to each calibration board from the image of the calibration board and the pose of the calibration board relative to the center point of the calibration fixture from the pre-obtained image, the extrinsic parameters of the camera are calibrated. This allows the robot to complete the calibration of both intrinsic and extrinsic parameters of the camera in a single calibration operation, shortening the calibration time and improving the robot's mass production efficiency. Before camera calibration, the robot obtains the pose of the calibration board relative to the center point of the calibration fixture. By aligning the center point of the robot with the center point of the calibration fixture for different extrinsic parameters, the robot's extrinsic parameters can be calculated from the pose of the calibration board relative to the center point of the calibration fixture, offering high flexibility. After calibrating the camera's intrinsic parameters using the image of the calibration board, the robot verifies the camera's intrinsic parameters to improve the accuracy of the camera's intrinsic parameter calibration results and the accuracy of the extrinsic parameter calibration results using the same data.

[0022] In one embodiment, step S1, where the robot acquires an image of the calibration plates of the calibration fixture via a camera, includes the following steps: the robot moves into the calibration fixture composed of several calibration plates, and then aligns the robot's center point with the center point of the calibration fixture. This alignment can be on a horizontal plane. The calibration plates of the calibration fixture are generally arranged around the robot. When acquiring images of the calibration plates, the robot aligns the camera vertically upwards and then acquires images of all the calibration plates of the calibration fixture via the camera. To ensure the camera captures all the calibration plates, the robot can rotate to ensure all the calibration plates fall within the camera's field of view. If any calibration plate is not within the camera's field of view, it can be removed. In this application, the robot's center point can be the center point of the robot's horizontal cross-section. The center point of the calibration fixture is set using the robot's center point when the robot acquires the pose of the calibration plates relative to the center point of the calibration fixture, and this setting is not changed afterward. The setting time is before the robot begins calibration work. The robot described in this application acquires images of the calibration plates by positioning the camera vertically upwards, which helps the robot obtain images of all calibration plates with the calibration fixture, thereby improving calibration efficiency.

[0023] In one embodiment, step S1, where the robot identifies calibration points from the image of the calibration board, includes the following steps: The robot detects the black areas of quadrilaterals in the image of the calibration board. The robot connects the detected black areas of quadrilaterals in a clockwise direction. The robot obtains the inner quadrilaterals of the black areas of the quadrilaterals. The robot uses corner detection to use the four endpoints of the inner quadrilaterals as calibration points of the calibration board. The calibration board is a checkerboard calibration board, and the four sides of the inner quadrilaterals are connected to the black areas of the quadrilaterals, meaning the black areas of the quadrilaterals are the black quadrilaterals of the checkerboard, and the inner quadrilaterals of the black areas are the white quadrilaterals of the checkerboard, formed by four black quadrilaterals. By first identifying the inner quadrilaterals from the checkerboard calibration board and then using the four endpoints of the inner quadrilaterals as calibration points, the robot effectively removes calibration points connected to the four sides of the checkerboard, improving the accuracy of the calibration results.

[0024] In one embodiment, step S2 involves the robot calibrating the camera's intrinsic parameters and obtaining the camera's pose relative to each calibration board based on the positions of calibration points on the calibration board and the positions of calibration points in the image. This includes the following steps: The robot obtains the physical coordinates of the calibration points in the physical coordinate system of the calibration board and the pixel coordinates of the calibration points in the image. The robot projects the calibration points on the calibration board onto the image of the calibration board based on the initial pose of each calibration board relative to the robot's center point (which can be calculated by manually measuring the distance between the calibration board and the robot's center point) and the camera's initial intrinsic parameters (estimated from the image (or obtained from the camera manufacturer)). This yields the projected coordinates of the calibration points on the calibration board. The robot then calculates the Euclidean distance between the projected coordinates of the calibration points on the calibration board and the pixel coordinates of the corresponding calibration points in the image, using the Euclidean distance formula. The robot optimizes the camera's intrinsic parameters using the iterative formula of Gauss-Newton's method based on the acquired Euclidean distance, obtaining the optimal camera intrinsic parameters and the camera's pose relative to each calibration board. Assuming the physical coordinates of the calibration point on the calibration board are (x0, y0), the projected coordinates of the calibration point on the calibration board are obtained as (x1, y1) through projection coordinate transformation. The pixel coordinates of the calibration point in the image are (x2, y2). The Euclidean distance between the projected coordinates of the calibration point on the calibration board and the pixel coordinates of the corresponding calibration point in the image is d = sqrt((x1-x2)∧2 - (y1-y2)∧2). Then, the robot optimizes the camera's intrinsic parameters using the iterative formula of the Gauss-Newton method: θn+1 = θn - (JTJ)∧-1*JTd; where θn+1 is the updated intrinsic or extrinsic parameter, θn is the original intrinsic or extrinsic parameter, d is the calculated Euclidean distance, and J is the Jacobian matrix. The robot can also optimize the pose of each calibration board relative to the robot's center point and the camera's intrinsic parameters using the Levenberg-Marquardt algorithm (LM algorithm is the most widely used nonlinear least squares algorithm; it uses gradient maximization to find the maximum (minimum) value). After obtaining the initial pose of each calibration board relative to the robot's center point and the initial intrinsic parameters of the camera, the robot iteratively optimizes the pose of each calibration board relative to the robot's center point and the camera's intrinsic parameters using the Gauss-Newton method, ensuring that the pose of each calibration board relative to the robot's center point and the camera's intrinsic parameters are optimal, thus improving the accuracy of the calculation results.

[0025] In one embodiment, when the robot optimizes the camera's intrinsic parameters using the iterative formula of the Gauss-Newton method, upon obtaining the minimum Euclidean distance during the calculation process, the pose of each calibration board relative to the robot's center point and the camera's intrinsic parameters corresponding to the minimum Euclidean distance are taken as the optimal camera intrinsic parameters and camera pose relative to each calibration board. The Euclidean distance described in this application is the reprojection error between the calibration point on the calibration board and the calibration point in the image. Obtaining the minimum value of the reprojection error to obtain the optimal pose of each calibration board relative to the robot's center point and the camera's intrinsic parameters demonstrates high practicality.

[0026] In one embodiment, the robot acquires the initial pose of the calibration plate relative to its center point, including the following steps: the robot draws a perpendicular line from the top left corner of the calibration plate; the robot uses the angle between the perpendicular line and its forward direction as the angle between the calibration plate and the robot; the robot obtains the distance between its center point and the top left corner of the calibration plate; and the robot combines the angle between the calibration plate and the robot, and the distance between the robot's center point and the top left corner of the calibration plate to obtain the initial pose of the calibration plate relative to its center point. The robot constructs its own coordinate system, using its midpoint as the origin, its forward direction as the X-axis, and its wheel axis as the Y-axis. The robot can calculate the initial pose of each calibration plate relative to its center point based on the angle between the perpendicular line and the robot's forward direction, the distance between the top left corners of the calibration plate, and the height of the top left corner of the calibration plate. The robot calculates the pose of the calibration board relative to the robot's center point based on the angle between the calibration board and the robot, and the distance between the robot's center point and the top left corner of the calibration board, thus improving the calculation speed.

[0027] As one embodiment, the robot obtains the physical coordinates of the calibration point in the physical coordinate system of the calibration board, including the following steps: The robot constructs a physical coordinate system with the top-left corner of each calibration board as the origin, and the two sides connecting the top-left corner of the calibration board as the X-axis and Y-axis, respectively. The robot obtains the coordinates of the calibration point in the physical coordinate system based on the position of the calibration point on the calibration board. The robot constructs the physical coordinate system based on the edges of the calibration board to improve the accuracy of the calculation.

[0028] In one embodiment, step S3 involves the robot detecting the camera's intrinsic parameter calibration results, including the following steps: The robot classifies the calibration points in the image to determine the calibration board to which the identified calibration points belong. The robot converts the pixel coordinates of the identified calibration points belonging to the same calibration board into physical coordinates of the calibration board's physical coordinate system using optimal camera intrinsic parameters and the camera's pose relative to the calibration board, thus obtaining the physical coordinates of the image's calibration points. The robot obtains the Euclidean distance between the physical coordinates of the image's calibration points and the physical coordinates of the calibration points on the corresponding calibration board. If the average of the Euclidean distances between the physical coordinates of a set number of image calibration points and the physical coordinates of the corresponding calibration board calibration points is greater than a first set threshold, the calibration board is judged to be incorrectly calibrated; otherwise, the calibration is correct. The robot can also convert the physical coordinates of the calibration points on the calibration board into pixel coordinates of the image, and then calculate the Euclidean distance between them and the pixel coordinates of the corresponding image calibration points. The robot fits the calibration points of the same calibration board that are correctly calibrated into several straight lines of calibration points. It then compares the perpendicular distance between any two adjacent and parallel calibration point lines. If the perpendicular distance between any two adjacent and parallel calibration point lines is greater than a second preset threshold, the calibration board is determined to be incorrectly calibrated; otherwise, it is correctly calibrated. The robot deletes the incorrectly calibrated calibration board and the identified calibration points belonging to that board, and then recalibrates the camera's intrinsic parameters through step S2. If all calibration boards are incorrectly calibrated, the robot rotates a certain angle and then re-acquires the image of the calibration board of the calibration fixture through the camera to calibrate the camera's intrinsic and extrinsic parameters. After calibrating the camera's intrinsic parameters using the image of the calibration board, the robot verifies the camera's intrinsic parameters to improve the accuracy of the camera's intrinsic parameter calibration results and the accuracy of the extrinsic parameter calibration results using the same data.

[0029] In one embodiment, step S2 involves the robot classifying the identified calibration points and determining the calibration board to which each point belongs. This includes the following steps: A1: The robot randomly selects the same number of calibration points as the calibration board as the center of a cluster, where each cluster is a set of points. A2: The robot obtains the cluster distance between each calibration point and the center of each cluster, and then assigns the calibration point to the cluster with the smallest cluster distance. A3: The robot obtains the average value of the image coordinates of the calibration points in each cluster, and then uses this average value as the center of the new cluster. A4: The robot repeats steps A2 and A3 until the average value of the image coordinates of the calibration points in each cluster no longer changes. Then, the robot determines the calibration board to which each cluster's calibration points belong based on the center of each cluster. The robot uses a clustering algorithm to classify the calibration points, achieving high accuracy.

[0030] In one embodiment, step S4 involves the robot calibrating the extrinsic parameters of the camera using the pose of a preset calibration plate relative to the robot's center and the pose of the camera relative to each calibration plate. This includes the following steps: The robot acquires the rotation matrix of the preset calibration plate's pose relative to the robot's center. The robot acquires the rotation matrix of the camera's pose relative to one of the calibration plates. The robot transforms the preset rotation matrix of the calibration plate's pose relative to the robot's center and the rotation matrix of the camera's pose relative to one of the calibration plates using Euler's formula to obtain the camera's extrinsic parameter matrix. The robot obtains the camera's extrinsic parameters through matrix transformation, resulting in faster calculation speed.

[0031] In one embodiment, before calibrating the camera, the robot first acquires the pose of the calibration plate of the calibration fixture relative to the center point of the calibration fixture. This includes the following steps: The robot positions the calibration plate within the camera's field of view, then moves to different preset positions to acquire several images of the calibration plate and obtains IMU data during the movement. The robot identifies calibration points from the acquired images of the calibration plate and then uses a nonlinear optimization algorithm to calculate the pose of the camera relative to the calibration plate at different preset positions based on the identified calibration points. The robot uses a nonlinear optimization algorithm to calculate the pose of the camera relative to the robot's center point based on the IMU data and the camera's pose relative to the calibration plate at different preset positions. The robot acquires the initial intrinsic parameters of the camera and, based on the IMU data, acquires the initial pose of each calibration plate relative to the robot's center point at different preset positions. The robot identifies calibration points from the images of the calibration plates and obtains the physical coordinates of the calibration points in the physical coordinate system of the calibration plate. The robot projects the physical coordinates of the calibration points on the calibration plate onto the corresponding calibration plate images to obtain the projected coordinates of the calibration points on the calibration plate. The robot calculates the Euclidean distance between the projected coordinates of the calibration points on the calibration boards and the pixel coordinates of the corresponding calibration points in the image, using the Euclidean distance formula. Based on the obtained Euclidean distance, the robot optimizes the camera's intrinsic parameters using an iterative formula of the Gauss-Newton method to obtain the optimal camera pose relative to each calibration board. The robot also obtains the camera's pose relative to its center point using an iterative formula of the Gauss-Newton method, based on the initial pose of each calibration board relative to the robot's center point and the camera's pose relative to each calibration board. The robot moves into a calibration fixture composed of several calibration boards and sets its current center point as the center point of the fixture. Once the center point of the fixture is set, it remains unchanged when the fixture is used to calibrate cameras on other robots. The robot aligns the camera vertically and acquires images of all the calibration boards in the fixture, identifying the calibration points from these images. The robot acquires the camera's pose relative to each calibration board by comparing the positions of the calibration points on the calibration boards with the positions of the calibration points in the image. Then, based on the camera's pose relative to each calibration board and the camera's pose relative to the robot's center point, the robot uses matrix transformation to acquire the pose of the calibration board relative to the center point of the calibration fixture. Since the robot acquires the pose of the calibration board relative to the center point of the calibration fixture before camera calibration, by aligning the robot's center point with the center point of the calibration fixture for different extrinsic parameters, the robot's extrinsic parameters can be calculated from the pose of the calibration board relative to the center point of the calibration fixture, offering high flexibility.

[0032] Compared with existing technologies, the advantages of this application are as follows: When calibrating the intrinsic and extrinsic parameters of a camera, the robot described in this application first optimizes the intrinsic parameters of the camera using images from calibration boards, improving the accuracy of the camera. Then, by obtaining the camera's pose relative to each calibration board based on the images from the calibration boards and the pose of the calibration board relative to the center point of the calibration fixture, the extrinsic parameters of the camera are calibrated. This allows the robot to complete the calibration of both the intrinsic and extrinsic parameters of the camera in a single calibration operation, shortening the calibration time and improving the robot's mass production efficiency. Before camera calibration, the robot obtains the pose of the calibration board relative to the center point of the calibration fixture. As long as the center point of the robot with different extrinsic parameters coincides with the center point of the calibration fixture, the robot's extrinsic parameters can be calculated from the pose of the calibration board relative to the center point of the calibration fixture, providing high flexibility.

[0033] In the description of this specification, the terms "in one embodiment," "preferred," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. The illustrative expressions of the above terms in this specification do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described can be combined in any suitable manner in one or more embodiments or examples. The connection methods linked in the description of this specification have significant effects and practical utility.

[0034] Based on the above description of the structure and principle, those skilled in the art should understand that the present invention is not limited to the specific embodiments described above. Improvements and substitutions made using techniques known in the art based on the present invention all fall within the protection scope of the present invention and should be defined by the claims.

Claims

1. A camera calibration method, characterized in that, The method includes the following steps: S1: The robot acquires an image of the calibration plate of the calibration fixture through a camera, and then identifies the calibration points from the image of the calibration plate; S2: The robot calibrates the camera's intrinsic parameters and obtains the camera's pose relative to each calibration board based on the positions of the calibration points on the calibration board and the positions of the calibration points in the image. S3: The robot checks the intrinsic parameter calibration results of the camera. If the detection result is correct, proceed to step S4. If the detection result is incorrect, recalibrate the intrinsic parameters of the camera. S4: The robot calibrates the extrinsic parameters of the camera by using the pose of the calibration plate of the calibration fixture relative to the center point of the calibration fixture and the pose of the camera relative to each calibration plate. In step S2, the robot calibrates the camera's intrinsic parameters and obtains the camera's pose relative to each calibration board based on the positions of the calibration points on the calibration board and the positions of the calibration points in the image. This includes the following steps: The robot acquires the physical coordinates of the calibration point in the physical coordinate system of the calibration board and the pixel coordinates of the calibration point in the image. The robot projects the calibration points on the calibration board into the image of the calibration board based on the initial pose of each calibration board relative to the robot's center point and the initial intrinsic parameters of the camera. The robot calculates the Euclidean distance between the projected coordinates of the calibration point on the calibration board and the pixel coordinates of the calibration point in the image corresponding to that calibration point, based on the Euclidean distance formula. The robot optimizes the camera's intrinsic parameters using the iterative formula of the Gauss-Newton method based on the acquired Euclidean distance, thereby obtaining the optimal camera intrinsic parameters and the camera's pose relative to each calibration board.

2. The camera calibration method according to claim 1, characterized in that, In step S1, the robot acquires an image with several calibration plates through a camera, including the following steps: The robot moves into a calibration fixture consisting of several calibration plates, and then the center point of the robot is aligned with the center point of the calibration fixture. The robot aligns the camera vertically upwards and then uses the camera to capture images of all the calibration plates in the calibration fixture.

3. The camera calibration method according to claim 2, characterized in that, In step S1, the robot identifies the calibration points from the image of the calibration board, including the following steps: The robot detects the black quadrilateral area in the image of the calibration board; The robot connects the detected black quadrilateral areas in a clockwise direction; The robot obtains the inner quadrilateral of the black area of ​​the quadrilateral; The robot uses corner detection to determine the four endpoints of the inner quadrilateral as the calibration points of the calibration plate. The calibration board is a checkerboard calibration board, and the four sides of the inner quadrilateral are connected to the black areas of the quadrilateral.

4. The camera calibration method according to claim 3, characterized in that, When the robot optimizes the intrinsic parameters of the camera using the iterative formula of the Gauss-Newton method, it obtains the minimum Euclidean distance during the calculation process. Then, it takes the pose of each calibration plate and the robot's center point corresponding to the minimum Euclidean distance and the camera's intrinsic parameters as the optimal camera intrinsic parameters and the camera's pose relative to each calibration plate.

5. The camera calibration method according to claim 3, characterized in that, The robot acquires the initial pose of the calibration board relative to the robot's center point, including the following steps: The robot draws a perpendicular line from the top left corner of the calibration plate to the calibration plate. The robot uses the angle between the perpendicular line to the calibration plate and the robot's direction of travel as the angle between the calibration plate and the robot. The robot obtains the distance between its center point and the top left corner of the calibration board; The robot obtains the initial pose of the calibration board relative to the robot's center point by combining the angle between the calibration board and the robot and the distance between the robot's center point and the top left corner of the calibration board.

6. The camera calibration method according to claim 3, characterized in that, The robot obtains the physical coordinates of the calibration point in the physical coordinate system of the calibration board, including the following steps: The robot uses the top left corner of each calibration board as the origin, and the two sides connecting the top left corner of the calibration board are the X-axis and Y-axis, respectively, to construct a physical coordinate system; The robot obtains the coordinates of the calibration point in the physical coordinate system based on the position of the calibration point on the calibration plate.

7. A camera calibration method according to claim 6, characterized in that, In step S3, the robot checks the intrinsic parameter calibration results of the camera, including the following steps: The robot classifies the calibration points in the image and determines the calibration board to which the identified calibration points belong; The robot converts the pixel coordinates of the identified calibration points belonging to the same calibration board into physical coordinates of the calibration board's physical coordinate system using the camera's intrinsic parameters and the camera's pose relative to the calibration board, thus obtaining the physical coordinates of the image's calibration points. The robot acquires the physical coordinates of the calibration points in the image and the Euclidean distance between the physical coordinates of the calibration points on the calibration board corresponding to the calibration points in the image. If the average value of a set number of Euclidean distances is greater than a first set threshold, the calibration board is judged to be calibrated incorrectly; otherwise, the calibration is correct. The robot fits the calibration points of the same calibration board that are correctly calibrated into several calibration point lines. Then it compares the vertical distance between two adjacent and parallel calibration point lines. If the vertical distance between any two calibration point lines is greater than a second set threshold, the calibration board is judged to be calibrated incorrectly; otherwise, it is calibrated correctly. The robot deletes the incorrectly calibrated calibration board and the calibration points identified as belonging to that calibration board, and then recalibrates the camera's intrinsic parameters.

8. A camera calibration method according to claim 7, characterized in that, In step S2, the robot classifies the identified calibration points and determines the calibration board to which each calibration point belongs, including the following steps: A1: The robot randomly selects the same number of calibration points as the calibration board as the center of the cluster; A2: The robot obtains the cluster distance between the calibration point and the center of each cluster, and then assigns the calibration point to the cluster with the smallest cluster distance; A3: The robot obtains the average value of the image coordinates of the calibration points of each cluster, and then uses the average value as the center of the new cluster; A4: The robot repeats steps A2 and A3 until the average value of the image coordinates of the calibration points of each cluster no longer changes. Then the robot determines the calibration board to which the calibration points of each cluster belong based on the center of each cluster.

9. A camera calibration method according to claim 7, characterized in that, In step S4, the robot calibrates the extrinsic parameters of the camera by using the pose of the calibration plate of the calibration fixture relative to the center point of the calibration fixture and the pose of the camera relative to each calibration plate, including the following steps: The robot acquires the rotation matrix of the pose of the calibration plate of the calibration fixture relative to the center point of the calibration fixture; The robot acquires the rotation matrix of the camera's pose relative to one of the calibration plates; The rotation matrix of the robot's pose relative to the center point of the calibration fixture and the rotation matrix of the camera's pose relative to one of the calibration plates are transformed using Euler's formula to obtain the camera's extrinsic parameter matrix.

10. A camera calibration method according to claim 9, characterized in that, Before calibrating the camera, the robot first obtains the pose of the calibration plate of the calibration fixture relative to the center point of the calibration fixture, including the following steps: The robot positions the calibration board within the camera's field of view, then moves it to different preset positions to acquire several images of the calibration board and obtain IMU data during the movement. The robot identifies calibration points from the acquired images of the calibration board, and then uses a nonlinear optimization algorithm to calculate the pose of the camera relative to the calibration board at different preset positions based on the identified calibration points. The robot uses a nonlinear optimization algorithm to calculate the pose of the camera relative to the center point of the robot based on IMU data and the pose of the camera relative to the calibration plate at different preset positions. The robot moves into a calibration fixture consisting of several calibration plates, and then the current position of the robot's center point is set as the center point of the calibration fixture. The robot aligns the camera vertically upwards, then uses the camera to acquire images of all the calibration plates in the calibration fixture, and identifies the calibration points from the images of the calibration plates; The robot obtains the camera pose relative to each calibration board based on the position of the calibration points on the calibration board and the position of the calibration points in the image. The robot uses matrix transformation to obtain the pose of the calibration plate of the calibration fixture relative to the center point of the calibration fixture, based on the pose of the camera relative to each calibration plate and the pose of the camera relative to the center point of the robot.

Images