Camera calibration method and device, camera calibration equipment and storage medium

Abstract

According to the camera calibration scheme provided by the embodiment of the invention, through a camera which is arranged in a calibration room and carries out automatic shooting according to a preset rule, images of a plurality of checkerboard calibration plates around the camera are collected to obtain a calibration plate image, and the freedom degrees of the checkerboard calibration plates are different; segmenting the plurality of calibration plate images to obtain a plurality of checkerboard images, the checkerboard images including identification information; determining a calibration image set according to the identification information of the plurality of checkerboard images; and performing camera internal reference calibration on the camera according to the calibration image set. According to the scheme, the multiple checkerboard calibration plates with different degrees of freedom arranged around the camera in the calibration room are subjected to image acquisition, and the calibration image set for camera internal reference calibration is determined according to the identification information of the checkerboard images, so that the acquired checkerboard images can cover the dimension of each degree of freedom, and the obtained camera internal reference is more accurate; in image acquisition, automatic shooting is carried out through a camera arranged in a calibration room according to a preset rule, and the acquisition efficiency is high.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of autonomous driving technology, and in particular to a camera calibration method, apparatus, camera calibration device and storage medium. Background Art

[0002] Environmental perception is an important component of autonomous driving technology. Environmental perception mainly uses various types of sensors to perceive the surrounding environment of autonomous driving equipment (such as vehicles, robots, aircraft, etc.). Among them, camera sensors are usually used to detect and track obstacles, identify traffic lights and traffic signs, detect lane lines and semantic segmentation, and other traffic environment perceptions.

[0003] To use a camera for environmental perception, the camera's intrinsic parameters need to be calibrated. The camera's intrinsic parameter calibration mainly obtains the camera's focal length, optical center, distortion coefficient and other parameters. Existing camera intrinsic parameter calibration methods usually use a camera intrinsic parameter calibration board with a chessboard pattern held by a human hand, collect chessboard images of different postures through the camera, and then use the calibration algorithm to process the collected images and output the camera's intrinsic parameters.

[0004] However, the checkerboard images collected by manually holding a checkerboard calibration board usually have duplications or omissions in the dimensions of each degree of freedom, which is not accurate enough, resulting in inaccurate camera intrinsic parameters, affecting the safety of autonomous driving, and low manual operation efficiency. Summary of the invention

[0005] In view of this, an embodiment of the present application provides a camera calibration solution to at least partially solve the above-mentioned problem.

[0006] According to a first aspect of an embodiment of the present application, a camera calibration method is provided, comprising: using a camera arranged in a calibration room to automatically shoot according to preset rules, collecting images of multiple checkerboard calibration plates around the camera to obtain calibration plate images, wherein the multiple checkerboard calibration plates are arranged with different degrees of freedom; segmenting the multiple calibration plate images to obtain multiple checkerboard images, wherein the checkerboard images contain identification information; determining a calibration image set according to the identification information of the multiple checkerboard images; and calibrating the camera intrinsic parameters according to the calibration image set.

[0007] Optionally, a camera is set in the calibration room to automatically shoot according to preset rules, and images of multiple checkerboard calibration plates around the camera are collected to obtain calibration plate images, including: obtaining the setting positions of the multiple checkerboard calibration plates and the preset motion path of the camera, the setting positions of the checkerboard calibration plates are determined by the focal length and field of view of the camera; determining multiple shooting positions and camera directions of the camera on the preset motion path; setting the camera to automatically shoot according to the multiple shooting positions and camera directions on the preset motion path to obtain calibration plate images.

[0008] Optionally, a calibration image set is determined based on the identification information of multiple checkerboard images, including: if the multiple identification information completely covers the preset identification information, then the multiple checkerboard images corresponding to the multiple identification information are determined as the calibration image set; if the multiple identification information does not completely cover the preset identification information, then returning to the step of collecting images of multiple checkerboard calibration plates around the camera through a camera set in the calibration room that automatically shoots according to preset rules and continuing to execute.

[0009] Optionally, if multiple identification information completely covers the preset identification information, then the multiple checkerboard images corresponding to the multiple identification information are determined as the calibration image set, including: if the multiple identification information completely covers the preset identification information, then the multiple checkerboard images are determined as the calibration image set to be screened; if there is duplication of identification information in the calibration image set to be screened, then the calibration image set is determined according to the clarity of each image in the calibration image to be screened.

[0010] Optionally, the identification information is a QR code.

[0011] Optionally, multiple calibration plate images are segmented to obtain multiple checkerboard images, including: segmenting multiple calibration plate images to obtain multiple candidate checkerboard images; determining the clarity of the candidate checkerboard images, deleting the candidate checkerboard images whose clarity is less than a preset clarity threshold, and obtaining multiple checkerboard images according to the deletion results.

[0012] Optionally, according to the calibration image set, the camera is calibrated with camera intrinsic parameters, including: processing the calibration image set through an intrinsic parameter calibration function to obtain camera intrinsic parameters and multiple reprojection errors; if there is a reprojection error greater than a preset error threshold, returning to the step of collecting images of multiple checkerboard calibration plates around the camera through a camera set in a calibration room that automatically shoots according to preset rules and continuing to execute; if the reprojection errors are all less than or equal to the preset error threshold, the camera intrinsic parameter calibration is completed.

[0013] According to a second aspect of an embodiment of the present application, a camera calibration device is provided, including: an image acquisition module, used to acquire images of multiple checkerboard calibration plates around the camera by a camera arranged in a calibration room and performing automatic shooting according to preset rules to obtain calibration plate images, wherein the multiple checkerboard calibration plates are arranged with different degrees of freedom; an image segmentation module, used to segment the multiple calibration plate images to obtain multiple checkerboard images, wherein the checkerboard images contain identification information; a calibration image set determination module, used to determine a calibration image set according to the identification information of the multiple checkerboard images; and a camera intrinsic parameter calibration module, used to perform camera intrinsic parameter calibration on the camera according to the calibration image set.

[0014] According to a third aspect of an embodiment of the present application, a camera calibration device is provided, including: a camera carrying device, a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other through the communication bus; the camera carrying device is used to carry a camera to automatically shoot according to preset rules, and the camera is used to collect images of multiple checkerboard calibration plates around the camera; the memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform an operation corresponding to the method described in the first aspect.

[0015] According to a fourth aspect of an embodiment of the present application, a computer storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the method described in the first aspect is implemented.

[0016] According to the camera calibration scheme provided by the embodiment of the present application, a camera is set in the calibration room and automatically shoots according to preset rules, and images of multiple checkerboard calibration plates around the camera are collected to obtain calibration plate images, and the multiple checkerboard calibration plates are set with different degrees of freedom; the multiple calibration plate images are segmented to obtain multiple checkerboard images, and the checkerboard images contain identification information; according to the identification information of the multiple checkerboard images, a calibration image set is determined; according to the calibration image set, the camera is calibrated for camera intrinsic parameters. The scheme of the embodiment of the present application, by performing image acquisition on multiple checkerboard calibration plates with different degrees of freedom set around the camera in the calibration room, and determining the calibration image set for camera intrinsic parameter calibration of the camera according to the identification information of the multiple checkerboard images, so that the acquired checkerboard images can cover the dimensions of each degree of freedom, then when performing the internal parameter calibration, the coordinate information obtained from the checkerboard images is richer and more comprehensive, so that the obtained camera intrinsic parameters are more accurate; and the image acquisition is automatically shot by the camera set in the calibration room according to preset rules, and the acquisition efficiency is high compared with manual shooting. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of the present application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0018] Figure 1 is a flowchart of the steps of a camera calibration method according to Embodiment 1 of the present application;

[0019] Figure 2A is a schematic diagram of a checkerboard image in the illustrated embodiment;

[0020] Figure 2B is a schematic diagram of the layout of the calibration room in the illustrated embodiment;

[0021] Figure 2C is a schematic diagram of a scene in which an image is captured by a camera in the illustrated embodiment;

[0022] Figure 2D An example table of degree of freedom setting parameters for the checkerboard calibration plate in the illustrated embodiment;

[0023] Figure 3 is a flowchart of another camera calibration method according to the second embodiment of the present application;

[0024] Figure 4 is a structural block diagram of a camera calibration device according to Embodiment 3 of the present application;

[0025] Figure 5 This is a structural schematic diagram of a camera calibration device according to Embodiment 4 of the present application. DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the embodiments of the present application should fall within the scope of protection of the embodiments of the present application.

[0027] The specific implementation of the embodiment of the present application is further explained below in conjunction with the accompanying drawings of the embodiment of the present application.

[0028] Embodiment 1

[0029] Reference Figure 1 , shows a step flow chart of a camera calibration method according to Embodiment 1 of the present application.

[0030] The camera calibration method of this embodiment includes the following steps:

[0031] Step S102: A camera is set in the calibration room to automatically shoot according to preset rules, and images of multiple checkerboard calibration plates around the camera are collected to obtain calibration plate images. The multiple checkerboard calibration plates are set with different degrees of freedom.

[0032] For example, refer to Figure 2B The calibration room is a physical space provided with a camera calibration device and a plurality of checkerboard calibration plates with different degrees of freedom. The camera calibration device may be arranged in the middle of the calibration room, and the structure of the camera calibration device may include a sliding track driven by a motor, and a camera bracket is arranged on the sliding track, and the camera bracket may drive the camera to slide along the sliding track. A motor and a camera fixing device may be arranged at the top of the camera bracket, and the camera fixing device may be driven by the motor to rotate to change the orientation of the camera, and the rotation direction may be any direction, which is not limited in this embodiment.

[0033] A plurality of checkerboard calibration plates can be arranged around the sliding track, that is, around the camera, for example, Figure 2B and 2C , the same specification and different degrees of freedom checkerboard calibration plates (such as Figure 2C In order to meet the needs of cameras with different field of view angles and focal lengths, checkerboard calibration plates with the same ID can be repeatedly placed. A large checkerboard calibration plate (such as Figure 2C The large chessboard calibration plate may be several times the size of the above-mentioned chessboard calibration plate of the same specification, and this embodiment does not impose any limitation on this.

[0034] It should be noted that since the distortion of the four corners of the image taken by the camera is more obvious, if a large chessboard calibration plate is set up, in the subsequent image segmentation process, the four corners of the large chessboard image are each cropped with a chessboard image of the same specification as other chessboard images as the image for camera intrinsic calibration, so as to calculate the distortion coefficient during the camera intrinsic calibration.

[0035] The checkerboard calibration plate can be set up around the sliding track with different degrees of freedom through the bracket. The degrees of freedom include four dimensions: x, y, size, and skew. Among them, x refers to the horizontal coordinate of the checkerboard calibration plate relative to the camera, y refers to the vertical coordinate of the checkerboard calibration plate relative to the camera, and size refers to the size of the checkerboard image in the image captured by the camera. The degree of freedom can be changed by changing the distance between the camera and the checkerboard calibration plate. Skew refers to the change of the plane of the checkerboard calibration plate relative to the camera image plane in the roll, pitch, and yaw dimensions. Roll, pitch, and yaw are three dimensions of the three-dimensional Cartesian coordinate system, among which pitch is the rotation around the X-axis, that is, the pitch angle, yaw is the rotation around the Y-axis, that is, the yaw angle, and roll is the rotation around the Z-axis, that is, the roll angle. For example, you can refer to Figure 2D The degrees of freedom setting parameters of the checkerboard calibration plate shown are used to set the checkerboard calibration plate.

[0036] Step S104: segment the multiple calibration plate images to obtain multiple checkerboard images, where the checkerboard images contain identification information.

[0037] Exemplarily, multiple calibration plate images are segmented, and the complete checkerboard image in the calibration plate image is cut out to obtain multiple checkerboard images. The checkerboard image contains identification information, which can be used as the ID of the checkerboard image. Each checkerboard image corresponds to a checkerboard calibration plate with different degrees of freedom, and each checkerboard image corresponds to an ID serial number.

[0038] In a specific implementation, the identification information is a QR code. For example, an apriltag QR code is embedded in the white grid of the standard black and white chessboard, and the apriltag QR code is used to mark the ID of the chessboard image. AprilTag is a visual reference library that can quickly detect the mark through a specific mark. Figure 2A As shown in the figure, a QR code of the Tag16H5 series is embedded in a standard chessboard. This QR code can be identified as a number by the apriltag recognition program as the ID of the chessboard image, for example Figure 2C In the standard checkerboard, checkerboard calibration plates with IDs 1 to 13 are distributed, that is, checkerboard images with IDs 1 to 13 can be obtained. In the standard checkerboard, the apriltag QR code can be embedded in all white squares or in some white squares, and this implementation does not limit this.

[0039] In this implementation, each chessboard image is identified by a QR code, which is easy to operate and identify, so that each chessboard image has an ID number, so that each chessboard calibration plate with different degrees of freedom can be distinguished and determined by the ID number of each chessboard image.

[0040] In a specific implementation, multiple calibration plate images are segmented to obtain multiple checkerboard images, including: segmenting multiple calibration plate images to obtain multiple candidate checkerboard images; determining the clarity of the candidate checkerboard images, deleting the candidate checkerboard images whose clarity is less than a preset clarity threshold, and obtaining multiple checkerboard images based on the deletion results.

[0041] Exemplarily, multiple calibration plate images are segmented, and the complete checkerboard image in the calibration plate image is cut out to obtain multiple candidate checkerboard images, and the clarity of each candidate checkerboard image is calculated, and the clarity of each candidate checkerboard image is compared with the preset clarity threshold, and the candidate checkerboard images with a clarity less than the preset clarity threshold are deleted, and the preset clarity threshold can be reasonably set according to experiments. Among them, the preset clarity threshold can be flexibly set by those skilled in the art according to actual conditions, and the embodiments of the present application do not limit this.

[0042] In this implementation, the candidate checkerboard images are screened by the clarity of each candidate checkerboard image, and the candidate checkerboard images whose clarity does not meet the preset clarity threshold are deleted, thereby ensuring that the clarity of the checkerboard images for subsequent camera intrinsic calibration is good and avoiding the adverse effects of low-definition checkerboard images on the camera intrinsic calibration results, thereby making the camera intrinsic calibration results more accurate.

[0043] Step S106: Determine a calibration image set according to the identification information of the plurality of checkerboard images.

[0044] Exemplarily, since each checkerboard image corresponds to an identification information, each checkerboard calibration plate has different degrees of freedom, that is, the identification information corresponds to the degrees of freedom of the checkerboard calibration plate, then the degree of freedom information of each checkerboard image can be determined by the identification information of each checkerboard image, and the degree of freedom information of each checkerboard image can be used to screen and obtain a calibration image set.

[0045] In a specific implementation, a calibration image set is determined based on identification information of multiple checkerboard images, including: if the multiple identification information completely covers the preset identification information, then the multiple checkerboard images corresponding to the multiple identification information are determined as the calibration image set; if the multiple identification information does not completely cover the preset identification information, then returning to the step of collecting images of multiple checkerboard calibration plates around the camera through a camera set in the calibration room that automatically shoots according to preset rules and continues to execute.

[0046] Exemplarily, the preset identification information corresponds to the preset degree of freedom information of each checkerboard image. If the identification information of multiple checkerboard images fully covers the preset identification information, that is, the degree of freedom information of multiple checkerboard images fully covers the preset degree of freedom information, it means that there is no situation where a checkerboard image with a certain degree of freedom is omitted in the multiple checkerboard images, and the multiple checkerboard images are determined to be a calibration image set. If the multiple identification information does not fully cover the preset identification information, that is, the degree of freedom information of multiple checkerboard images does not fully cover the preset degree of freedom information, it means that there is a situation where a checkerboard image with a certain degree of freedom is omitted in the multiple checkerboard images, and the process returns to the step of collecting images of multiple checkerboard calibration plates around the camera by using a camera that is set in the calibration room to automatically shoot according to preset rules, and re-collects multiple checkerboard images.

[0047] In this implementation, it is determined whether a chessboard image of a certain degree of freedom is missed by judging whether multiple identification information fully covers the preset identification information. If it is not missed, multiple chessboard images are determined as the calibration image set. If there is an omission, the chessboard images are re-collected. The comprehensiveness of the degrees of freedom of the chessboard images for camera intrinsic calibration is guaranteed, thereby improving the accuracy of camera intrinsic calibration.

[0048] In a specific implementation, if multiple identification information completely covers the preset identification information, multiple checkerboard images corresponding to the multiple identification information are determined as the calibration image set, including: if multiple identification information completely covers the preset identification information, multiple checkerboard images corresponding to the multiple identification information are determined as the calibration image set to be screened; if there is duplication of identification information in the calibration image set to be screened, the calibration image set is determined according to the clarity of each image in the calibration image to be screened.

[0049] Exemplarily, if multiple identification information completely covers the preset identification information, that is, the degree of freedom information of multiple checkerboard images completely covers the preset degree of freedom information, it means that there is no situation where a checkerboard image with a certain degree of freedom is omitted in the multiple checkerboard images, and the multiple checkerboard images can be used as the calibration image set to be screened; it is determined whether the identification information in the calibration image set to be screened is repeated. If there is a repetition, it means that there are multiple checkerboard images with the same degree of freedom. Here, the calibration image set can be determined according to the clarity of each image in the calibration image set to be screened. For example, the clarity of each image in the calibration image set to be screened can be calculated, and the clarity of the checkerboard images with repeated identification information can be compared. The checkerboard image with the highest clarity is retained, and other checkerboard images with lower clarity in the checkerboard images with repeated identification information are deleted to determine the calibration image set.

[0050] In this implementation, the checkerboard images with repeated identification information are further screened by definition, the checkerboard image with the highest definition is retained, and the other checkerboard images in the checkerboard images with repeated identification information are deleted to determine the calibration image set. This avoids multiple checkerboard images with the same degree of freedom from participating in the camera intrinsic calibration calculation, reduces the data processing amount of the camera intrinsic calibration, and avoids the adverse effects of checkerboard images with lower definition, thereby making the camera intrinsic calibration result more accurate.

[0051] Step S108: Calibrate the camera's intrinsic parameters based on the calibration image set.

[0052] For example, the intrinsic calibration function in opencv can be called to process the checkerboard images in the calibration image set to obtain the camera's intrinsic parameters, distortion coefficients, and the reprojection error of each checkerboard image under the current camera's intrinsic parameters and distortion coefficients as the result of the camera intrinsic calibration. Among them, opencv is a cross-platform computer vision and machine learning software library released under the BSD license (open source), providing a wealth of general algorithms in image processing and computer vision.

[0053] In a specific implementation, the camera is calibrated for intrinsic parameters according to the calibration image set, including: processing the calibration image set through an intrinsic parameter calibration function to obtain camera intrinsic parameters and multiple reprojection errors; if there is a reprojection error greater than a preset error threshold, returning to the step of collecting images of multiple checkerboard calibration plates around the camera through a camera set in a calibration room that automatically shoots according to preset rules to continue execution; if the reprojection errors are all less than or equal to the preset error threshold, the camera intrinsic calibration is completed.

[0054] Exemplarily, the checkerboard images in the calibration image set can be processed through the intrinsic parameter calibration function in opencv to obtain the reprojection error of each checkerboard image in the calibration image set under the current camera intrinsic parameters and distortion coefficients, and the reprojection error of each checkerboard image in the calibration image set is compared with a preset error threshold. If there is a checkerboard image with a reprojection error greater than the preset error threshold, it means that the calibration has failed, and the process returns to the step of collecting images of multiple checkerboard calibration plates around the camera through a camera set in the calibration room that automatically shoots according to preset rules and re-executes it; if the reprojection errors are all less than or equal to the preset error threshold, it means that the calibration is successful, and the process of camera intrinsic parameter calibration is completed.

[0055] In this implementation, the reprojection error is used to verify whether the camera intrinsic calibration is successful. If it is unsuccessful, the process returns to the step of collecting images of multiple checkerboard calibration plates around the camera by using a camera set in the calibration room to automatically shoot according to preset rules, and the process is repeated until it succeeds. The accuracy of the camera intrinsic calibration can be guaranteed through the above verification.

[0056] The camera calibration method provided in this embodiment uses a camera that is set in a calibration room to automatically shoot according to preset rules, collects images of multiple checkerboard calibration plates around the camera to obtain calibration plate images, and the multiple checkerboard calibration plates are set with different degrees of freedom; the multiple calibration plate images are segmented to obtain multiple checkerboard images, and the checkerboard images contain identification information; a calibration image set is determined according to the identification information of the multiple checkerboard images; and the camera intrinsic parameters are calibrated according to the calibration image set. The solution of this embodiment is to acquire images of multiple checkerboard calibration plates with different degrees of freedom arranged around the camera in the calibration room, and determine a calibration image set for camera intrinsic parameter calibration of the camera according to the identification information of the multiple checkerboard images, so that the acquired checkerboard images can cover the dimensions of each degree of freedom. When performing camera intrinsic parameter calibration, the coordinate information obtained from the checkerboard images is richer and more comprehensive, so that the obtained camera intrinsic parameters are more accurate; and the image acquisition is automatically shot by the camera arranged in the calibration room according to preset rules, so the acquisition efficiency is high, and labor can be saved, and mass production of camera intrinsic parameter calibration can be realized.

[0057] Embodiment 2

[0058] Reference Figure 3 , shows a flowchart of the steps of another camera calibration method according to the second embodiment of the present application. This embodiment mainly shows the specific implementation scheme of step 102 of the above-mentioned first embodiment, that is, by setting a camera in the calibration room to automatically shoot according to preset rules, images of multiple checkerboard calibration plates around the camera are collected.

[0059] This embodiment includes the following steps:

[0060] Step S202: Obtain the setting positions of multiple checkerboard calibration plates and the preset motion path of the camera.

[0061] The setting position of the checkerboard calibration plate is determined by the focal length and field of view of the camera.

[0062] Exemplarily, the setting positions of multiple checkerboard calibration plates and the preset motion path of the camera are obtained, for example, referring to Figure 2B and Figure 2C , obtain the preset motion path of the camera according to the preset sliding track, and the motion path of the camera can be reciprocating along the preset sliding track. Determine the setting position of the checkerboard calibration plate according to the focal length and field of view of the camera, and the setting position is the position relative to the preset motion path of the camera, for example, referring to Figure 2D The size parameter in the image and the focal length and field of view of the camera can determine the distance between the camera and the checkerboard calibration plate. The distribution density of the checkerboard calibration plate can be set according to experiments, and this embodiment does not limit this.

[0063] Step S204: determining a plurality of shooting positions and camera directions of the camera on a preset motion path.

[0064] Exemplarily, according to the setting positions of the plurality of checkerboard calibration plates and the preset motion path of the camera, a plurality of shooting positions and camera orientations of the camera on the preset motion path are determined, for example, referring to Figure 2C , you can set the camera to move along a preset path ( Figure 2C The preset frequency can be a meters / time, or b minutes / time, where a and b can be determined according to the length of the preset path, the speed of the camera movement, etc., and this embodiment does not limit this. Then, multiple camera orientations are preset according to the preset frequency of camera shooting and the setting positions of multiple checkerboard calibration plates, and automatic shooting is performed according to the preset frequency of camera shooting and the preset camera orientation. The shooting results are evaluated. The evaluation factors can be set to the integrity of the checkerboard image, the image clarity, etc. The shooting position and the corresponding camera orientation corresponding to the best evaluated shooting result are determined as multiple shooting positions and camera orientations of the camera on the preset motion path. For example, Figure 2C The shooting positions of the cameras at p1 to p5 and the corresponding camera orientations.

[0065] It should be noted that the number of shots can be set according to the camera's intrinsic calibration performance and the checkerboard image number curve with different degrees of freedom, and this application does not impose any restrictions on this.

[0066] Step S206: setting the camera to automatically shoot according to multiple shooting positions and camera directions on the preset motion path.

[0067] Exemplarily, the camera bracket is set to drive the camera to move on a preset motion path and stop at each shooting position. At the same time, the motor at the top of the camera bracket drives the camera fixing device to rotate so that the camera rotates to the camera orientation corresponding to each shooting position for automatic shooting.

[0068] Step S208: collecting images of multiple checkerboard calibration plates around the camera to obtain calibration plate images, wherein the multiple checkerboard calibration plates are arranged with different degrees of freedom.

[0069] Exemplarily, the camera is set to automatically shoot according to multiple shooting positions and camera directions on a preset motion path, and images of multiple checkerboard calibration plates with different degrees of freedom around the camera are collected to obtain calibration plate images.

[0070] Step S210: segment the multiple calibration plate images to obtain multiple checkerboard images, where the checkerboard images contain identification information.

[0071] Step S212: determining a calibration image set according to identification information of the plurality of checkerboard images.

[0072] Step S214: calibrate the camera's intrinsic parameters according to the calibration image set.

[0073] It should be noted that steps S210 to S214 of this embodiment can be specifically implemented with reference to steps S104 to S108 in the first embodiment, and will not be described in detail here.

[0074] In this embodiment, multiple shooting positions and camera orientations of the camera on the preset motion path are determined by setting positions of multiple checkerboard calibration plates and the preset motion path of the camera, and images of the checkerboard calibration plate are collected according to the multiple shooting positions and camera orientations of the camera on the preset motion path. Manual collection of images of the checkerboard calibration plate is avoided, which saves labor, improves the efficiency of camera intrinsic calibration, and can achieve mass production of camera intrinsic calibration; in addition, images of checkerboard calibration plates with different degrees of freedom can be pre-arranged to avoid missing the degrees of freedom of the checkerboard calibration plate in manual collection.

[0075] Embodiment 3

[0076] Reference Figure 4 , shows a structural block diagram of a camera calibration device according to embodiment three of the present application.

[0077] The camera calibration device of this embodiment includes: an image acquisition module 302, which is used to acquire images of multiple checkerboard calibration plates around the camera through a camera that is set in a calibration room and automatically shoots according to preset rules to obtain calibration plate images, and the multiple checkerboard calibration plates are set with different degrees of freedom; an image segmentation module 304, which is used to segment the multiple calibration plate images to obtain multiple checkerboard images, and the checkerboard images contain identification information; a calibration image set determination module 306, which is used to determine a calibration image set according to the identification information of the multiple checkerboard images; a camera intrinsic parameter calibration module 308, which is used to perform camera intrinsic parameter calibration on the camera according to the calibration image set.

[0078] Optionally, the image acquisition module 302 is also used to obtain the setting positions of multiple checkerboard calibration plates and a preset motion path of the camera, the setting positions of the checkerboard calibration plates being determined by the focal length and field of view of the camera; determine multiple shooting positions and camera directions of the camera on the motion path; and set the camera to automatically shoot according to the multiple shooting positions and camera directions on the motion path.

[0079] Optionally, the calibration image set determination module 306 is further used to determine the multiple checkerboard images as the calibration image set if the multiple identification information fully covers the preset identification information; if the multiple identification information does not fully cover the preset identification information, re-capture images of the multiple checkerboard calibration plates around the camera.

[0080] Optionally, the calibration image set determination module 306 is also used to determine multiple checkerboard images as the calibration image set to be screened if multiple identification information completely covers the preset identification information; if the identification information in the calibration image set to be screened is repeated, determine the calibration image set according to the clarity of each image in the calibration image to be screened.

[0081] Optionally, the image segmentation module 304 is also used to segment multiple calibration plate images to obtain multiple candidate checkerboard images; determine the clarity of the candidate checkerboard images, delete the candidate checkerboard images whose clarity is less than a preset clarity threshold, and obtain multiple checkerboard images based on the deletion results.

[0082] Optionally, the camera intrinsic parameter calibration module 308 is also used to process the calibration image set through the intrinsic parameter calibration function to obtain the camera intrinsic parameters and multiple reprojection errors; if there is a reprojection error greater than a preset error threshold, then return to the step of collecting images of multiple checkerboard calibration plates around the camera through a camera set in the calibration room that automatically shoots according to preset rules and continues to execute; if the reprojection errors are all less than or equal to the preset error threshold, the camera intrinsic parameter calibration is completed.

[0083] The camera calibration device of this embodiment is used to implement the corresponding camera calibration methods in the aforementioned multiple method embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be described in detail here. In addition, the functional implementation of each module in the camera calibration device of this embodiment can refer to the description of the corresponding parts in the aforementioned method embodiments, which will not be described in detail here.

[0084] Embodiment 4

[0085] Reference Figure 5 , shows a structural schematic diagram of a camera calibration device according to Embodiment 4 of the present application.

[0086] The camera calibration device of this embodiment includes: a camera carrying device, a processor 404, a memory 402, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other through the communication bus; the camera carrying device is used to carry the camera to automatically shoot according to preset rules, and the camera is used to collect images of multiple checkerboard calibration plates around the camera; the memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform operations corresponding to the method described in embodiment 1 or 2. The specific implementation of the above-mentioned camera calibration method can be implemented by referring to the relevant description in the above-mentioned embodiment 1 or 2, and will not be repeated here.

[0087] The camera calibration device also includes a driving device 406 and a control device 408. Then, after the processor 404 determines the corresponding preset rule for automatic shooting, it sends a driving signal to the driving device 406, and the driving device 406 controls the control device 408 in the camera calibration device to perform corresponding control operations on the camera carrying device with the help of the driving signal, thereby realizing automatic calibration of the camera calibration device.

[0088] For the convenience and brevity of description, the specific working process of the camera calibration device described above can refer to the corresponding process description in the aforementioned method embodiment, and will not be repeated here.

[0089] An embodiment of the present application further provides a computer storage medium having a computer program stored thereon, which, when executed by a processor, implements the camera calibration method as described in any one of the above-mentioned method embodiments.

[0090] It should be pointed out that, according to the needs of implementation, the various components / steps described in the embodiments of the present application can be split into more components / steps, or two or more components / steps or partial operations of components / steps can be combined into new components / steps to achieve the purpose of the embodiments of the present application.

[0091] The above-mentioned method according to the embodiment of the present application can be implemented in hardware, firmware, or implemented as software or computer code that can be stored in a recording medium (such as a CD ROM, RAM, floppy disk, hard disk or magneto-optical disk), or implemented as a computer code originally stored in a remote recording medium or a non-temporary machine-readable medium downloaded through a network and to be stored in a local recording medium, so that the method described herein can be stored in such software processing on a recording medium using a general-purpose computer, a special-purpose processor or programmable or special-purpose hardware (such as an ASIC or FPGA). It can be understood that a computer, a processor, a microprocessor controller or programmable hardware includes a storage component (e.g., RAM, ROM, flash memory, etc.) that can store or receive software or computer code, and when the software or computer code is accessed and executed by a computer, a processor or hardware, the method described herein is implemented. In addition, when a general-purpose computer accesses the code for implementing the method shown here, the execution of the code converts the general-purpose computer into a special-purpose computer for executing the method shown here.

[0092] Those of ordinary skill in the art will appreciate that the units and method steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the embodiments of the present application.

[0093] The above implementation methods are only used to illustrate the embodiments of the present application, and are not limitations on the embodiments of the present application. Ordinary technicians in the relevant technical field can make various changes and modifications without departing from the spirit and scope of the embodiments of the present application. Therefore, all equivalent technical solutions also belong to the scope of the embodiments of the present application. The scope of patent protection of the embodiments of the present application should be limited by the claims.

Claims

1. A camera calibration method, characterized in that: include: A camera is set in the calibration room to automatically shoot according to preset rules, and images of multiple checkerboard calibration plates around the camera are collected to obtain calibration plate images, wherein the multiple checkerboard calibration plates are set with different degrees of freedom; Segmenting the plurality of calibration plate images to obtain a plurality of checkerboard images, wherein the checkerboard images contain identification information; Determining a calibration image set according to the identification information of the plurality of checkerboard images; The camera is calibrated with camera intrinsic parameters according to the calibration image set.

2. The method according to claim 1, characterized in that The camera that is set in the calibration room and automatically shoots according to preset rules collects images of multiple checkerboard calibration plates around the camera to obtain calibration plate images, including: Acquire the setting positions of a plurality of the checkerboard calibration plates and the preset motion path of the camera, wherein the setting positions of the checkerboard calibration plates are determined by the focal length and the field of view angle of the camera; Determining a plurality of shooting positions and camera orientations of the camera on the preset motion path; The camera is set to automatically shoot according to a plurality of shooting positions on the preset motion path and the camera orientation to obtain a calibration plate image.

3. The method according to claim 1, characterized in that The step of determining a calibration image set according to the identification information of the plurality of checkerboard images comprises: If the plurality of identification information completely covers the preset identification information, determining the plurality of checkerboard images corresponding to the plurality of identification information as the calibration image set; If the plurality of identification information does not completely cover the preset identification information, the step of returning to the step of collecting images of the plurality of checkerboard calibration plates around the camera by using a camera set in the calibration room to automatically shoot according to preset rules continues to be executed.

4. The method according to claim 3, characterized in that If the plurality of identification information completely covers the preset identification information, determining the plurality of checkerboard images corresponding to the plurality of identification information as the calibration image set includes: If the plurality of identification information completely covers the preset identification information, determining the plurality of checkerboard images corresponding to the plurality of identification information as the calibration image set to be screened; If the identification information in the calibration image set to be screened is repeated, the calibration image set is determined according to the definition of each image in the calibration image to be screened.

5. The method according to claim 1, characterized in that: The identification information is a QR code.

6. The method according to claim 1, characterized in that The step of segmenting the plurality of calibration plate images to obtain a plurality of checkerboard images comprises: Segmenting the plurality of calibration plate images to obtain a plurality of candidate checkerboard images; The clarity of the candidate checkerboard images is determined, and the candidate checkerboard images whose clarity is less than a preset clarity threshold are deleted, and a plurality of the checkerboard images are obtained according to the deletion results.

7. The method according to any one of claims 1 to 6, characterized in that The step of calibrating the camera intrinsic parameters according to the calibration image set includes: Processing the calibration image set by an intrinsic calibration function to obtain camera intrinsic parameters and a plurality of reprojection errors; If the reprojection error is greater than the preset error threshold, returning to the step of collecting images of multiple checkerboard calibration plates around the camera by using a camera set in the calibration room to automatically shoot according to preset rules and continue executing; If the reprojection errors are all less than or equal to the preset error threshold, the camera intrinsic parameter calibration is completed.

8. A camera calibration device, characterized in that: include: An image acquisition module, used to acquire images of a plurality of checkerboard calibration plates around the camera by using a camera arranged in the calibration room to automatically shoot according to preset rules, wherein the plurality of checkerboard calibration plates are arranged with different degrees of freedom; An image segmentation module, used to segment the plurality of calibration plate images to obtain a plurality of checkerboard images, wherein the checkerboard images contain identification information; A calibration image set determination module, configured to determine a calibration image set according to the identification information of the plurality of checkerboard images; A camera intrinsic parameter calibration module is used to calibrate the camera intrinsic parameters of the camera according to the calibration image set.

9. A camera calibration device, characterized in that: include: A camera carrying device, a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The camera carrying device is used to carry the camera to automatically shoot according to preset rules, and the camera is used to collect images of multiple chessboard calibration plates around the camera; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform an operation corresponding to the method according to any one of claims 1 to 7.

10. A computer storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images