Camera external parameter calibration method and device, computer equipment and readable storage medium

Abstract

The invention relates to a camera external parameter calibration method and device, computer equipment and a readable storage medium. The method comprises the following steps: acquiring a calibration image shot by a camera for a calibration target; wherein the outer edge of the calibration target is provided with a mirror surface device, the calibration image comprises the calibration target and a mirror image calibration target of at least part of the calibration target in the mirror surface device, the calibration target comprises a plurality of original calibration points, and the mirror image calibration target comprises a plurality of mirror image calibration points; acquiring a first coordinate of each calibration point in a world coordinate system; feature point detection is carried out based on the calibration image, and second coordinates of a plurality of feature points obtained through detection in the image coordinate system are determined; respectively determining feature points corresponding to the calibration points from the plurality of feature points; and determining the external parameters of the camera according to the first coordinates of the calibration points and the second coordinates of the feature points corresponding to the calibration points. By adopting the method, the accuracy of the camera external parameters obtained through calibration can be improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a camera extrinsic calibration method, device, computer equipment, and computer-readable storage medium. Background Art

[0002] With the development of computer technology, computer vision technology has emerged. Computer vision is a science that studies how to make machines "see". To put it more specifically, it refers to using cameras and computers to replace human eyes to identify, track and measure targets, and further perform graphic processing to make the computer processing into images that are more suitable for human eye observation or transmission to instruments for detection.

[0003] In computer vision technology, camera extrinsic calibration is often required. Camera extrinsic calibration is the process of determining the position and orientation of the camera in the world coordinate system. Based on the extrinsic parameters, a mapping between the image coordinate system and the world coordinate system can be established.

[0004] However, the camera extrinsics obtained by current camera extrinsic calibration methods often have the problem of low accuracy. Summary of the invention

[0005] Based on this, it is necessary to provide a camera extrinsic parameter calibration method, apparatus, computer equipment, computer-readable storage medium and computer program product that can improve the accuracy of camera extrinsic parameters obtained by calibration in order to address the above technical problems.

[0006] In a first aspect, the present application provides a camera extrinsic calibration method, comprising:

[0007] Obtain a calibration image captured by the camera for the calibration target;

[0008] Wherein, a mirror device is provided at the outer edge of the calibration target, the calibration image includes the calibration target and a mirror calibration target of at least part of the calibration target in the mirror device, the calibration target includes original calibration points, the mirror calibration target includes mirror calibration points, and the original calibration points and the mirror calibration points constitute a calibration point set;

[0009] Obtaining the first coordinate of each calibration point in the calibration point set in the world coordinate system;

[0010] Performing feature point detection based on the calibration image to determine second coordinates of feature points corresponding to each calibration point in the calibration point set in the image coordinate system;

[0011] The external parameters of the camera are determined according to the first coordinates of each calibration point and the second coordinates of the feature points corresponding to each calibration point in the image coordinate system.

[0012] In a second aspect, the present application also provides a camera extrinsic calibration device, comprising:

[0013] A calibration image acquisition module, used to acquire a calibration image taken by a camera for a calibration target; wherein a mirror device is provided at the outer edge of the calibration target, the calibration image includes the calibration target and a mirror calibration target of at least part of the calibration target in the mirror device, the calibration target includes a plurality of original calibration points, and the mirror calibration target includes a plurality of mirror calibration points;

[0014] A first coordinate acquisition module, used to acquire the first coordinate of each of the original calibration points in the world coordinate system, and to acquire the first coordinate of each of the mirror calibration points in the world coordinate system;

[0015] A second coordinate acquisition module, used to perform feature point detection based on the calibration image, and determine second coordinates of a plurality of feature points obtained by the detection in the image coordinate system;

[0016] A corresponding relationship determination module, used to determine, from the plurality of feature points, the feature points corresponding to the original calibration points and the feature points corresponding to the mirror calibration points;

[0017] The external parameter determination module is used to determine the external parameters of the camera according to the first coordinates of each original calibration point, the second coordinates of the feature points corresponding to each original calibration point, the first coordinates of each mirror calibration point, and the second coordinates of the feature points corresponding to each mirror calibration point.

[0018] In a third aspect, the present application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above-mentioned camera extrinsic parameter calibration method when executing the computer program.

[0019] In a fourth aspect, the present application also provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the above-mentioned camera extrinsic parameter calibration method are implemented.

[0020] In a fifth aspect, the present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned camera extrinsic parameter calibration method when executed by a processor.

[0021] In the above-mentioned camera extrinsic calibration method, device, computer equipment, computer-readable storage medium and computer program product, a mirror device is provided at the outer edge of the calibration target, the calibration image includes the calibration target and a mirror calibration target of at least part of the calibration target in the mirror device, the calibration target includes a plurality of original calibration points, and the mirror calibration target includes a plurality of mirror calibration points. When performing calibration, the first coordinate of each original calibration point in the world coordinate system can be obtained, and the first coordinate of each mirror calibration point in the world coordinate system can be obtained, feature point detection is performed based on the calibration image, and the plurality of feature points detected are determined in the image coordinate system. The second coordinate under the calibration system, from multiple feature points, respectively determine the feature points corresponding to each original calibration point and the feature points corresponding to each mirror calibration point, according to the first coordinate of each original calibration point, the second coordinate of the feature point corresponding to each original calibration point, the first coordinate of each mirror calibration point and the second coordinate of the feature point corresponding to each mirror calibration point, determine the external parameters of the camera, because the original calibration target can be expanded through the mirror device, so that the range of the external parameter calibration point can be expanded, the accuracy of long-distance projection is improved, and the calibrated external parameters can be applied to a wider range of point projection. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0023] Figure 1 A diagram showing an application environment of a camera extrinsic calibration method in some embodiments;

[0024] Figure 2 is a schematic flow chart of a camera extrinsic calibration method in some embodiments;

[0025] Figure 3 Schematic diagram of the positions of mirror calibration points in some embodiments;

[0026] Figure 4 A schematic diagram of the location of the mirror installed in a factory calibration room in some embodiments;

[0027] Figure 5 A schematic diagram of a world coordinate system in a factory calibration room in some embodiments;

[0028] Figure 6 is a schematic diagram of a calibration image captured by a vehicle-mounted camera in some embodiments;

[0029] Figure 7A schematic diagram of the overall process involved in the calibration process in some embodiments;

[0030] Figure 8 Schematic diagram of checkerboard corner points in some embodiments;

[0031] Fig. 9 A schematic diagram of the coordinate positions of corner points in a calibration image in some embodiments;

[0032] Fig.10 It is a schematic diagram of rearranging the order of corner points in some embodiments;

[0033] Fig.11 is a structural block diagram of a camera extrinsic parameter calibration device in some embodiments;

[0034] Fig.12 is an internal structure diagram of a computer device in some embodiments;

[0035] Fig.13 1 is a diagram of the internal structure of a computer device in some other embodiments. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0037] The camera extrinsic calibration method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. The camera 102 can shoot a calibration image for the calibration target, a mirror device is provided at the outer edge of the calibration target, the calibration image includes the calibration target and a mirror calibration target of at least part of the calibration target in the mirror device, the calibration target includes a plurality of original calibration points, after acquiring the calibration image, the computer device 104 can acquire the first coordinate of each original calibration point in the world coordinate system, and acquire the first coordinate of each mirror calibration point in the world coordinate system, perform feature point detection based on the calibration image, and determine the second coordinates of the plurality of feature points obtained by the detection in the image coordinate system, and respectively determine the feature points corresponding to each original calibration point and the feature points corresponding to each mirror calibration point from the plurality of feature points, and determine the external parameters of the camera according to the first coordinates of each original calibration point, the second coordinates of the feature points corresponding to each original calibration point, the first coordinates of each mirror calibration point, and the second coordinates of the feature points corresponding to each mirror calibration point.

[0038] The computer device 104 may be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, IoT devices, and portable wearable devices. The IoT devices may be smart speakers, smart TVs, smart air conditioners, smart car devices, projection devices, etc. The portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. The head-mounted devices may be virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, etc.

[0039] In some exemplary embodiments, Figure 2 As shown, a camera extrinsic calibration method is provided, which is applied to Figure 1 The computer device 104 in the example is used as an example to illustrate, including the following steps 202 to 210. Among them:

[0040] Step 202, obtaining a calibration image captured by a camera for a calibration target; wherein a mirror device is provided at an outer edge of the calibration target, the calibration image includes the calibration target and a mirror calibration target of at least a portion of the calibration target in the mirror device, the calibration target includes a plurality of original calibration points, and the mirror calibration target includes a plurality of mirror calibration points.

[0041] Among them, the calibration target is used to assist the calibration process, and the calibration target may have known geometric features, such as a checkerboard, a dot pattern or other objects with identifiable features. A mirror device is provided at the outer edge of the calibration target. Exemplarily, the mirror device may be vertically arranged at the outer edge of the calibration target, so that the mirror image calibration target of the calibration target in the mirror device is symmetrical with the calibration target about the mirror device. It is understandable that, in some other embodiments, the mirror device may also be arranged at the outer edge of the calibration target at a preset angle. Exemplarily, the mirror device may be arranged at a certain distance from the outer edge of the calibration target. Exemplarily, the mirror device may also be arranged parallel to the outer edge of the calibration target.

[0042] Since a mirror device is provided at the outer edge of the calibration target, based on the mirror reflection of the mirror device, at least part of the calibration target can form a mirror image in the mirror device, and the mirror image is the mirror calibration target of at least part of the calibration target in the mirror device. When the camera is shooting the calibration target, it can face the mirror device, so that the calibration image obtained by shooting can include the calibration target and the mirror calibration target of at least part of the calibration target in the mirror device. The calibration target includes a plurality of original calibration points, and the original calibration point is usually a point with certain characteristics in the calibration target, such as a corner point of a chessboard. The mirror calibration target includes a plurality of mirror calibration points, and each mirror calibration point corresponds to one of the original calibration points in the calibration target. When the calibration target part where a certain original calibration point is located has a corresponding mirror calibration target in the mirror device, the original calibration point has a corresponding mirror image in the mirror device, and the mirror image is the mirror calibration point of the original calibration point, so that more calibration points can be expanded through the mirror device.

[0043] Specifically, the computer device can obtain a calibration image taken by the camera for the calibration target in a wired or wireless manner, the calibration image including the calibration target and a mirror image calibration target of at least part of the calibration target in the mirror device, and the computer device can further determine the external parameters of the camera based on the calibration image.

[0044] Step 204, obtaining the first coordinate of each original calibration point in the world coordinate system, and obtaining the first coordinate of each mirror calibration point in the world coordinate system.

[0045] Specifically, during the calibration process, the coordinate origin in the world coordinate system is fixed, so the first coordinate of the original calibration point in the world coordinate system can be determined according to the position of the origin. For example, assuming that the calibration target is a checkerboard pattern, the original calibration point is the corner point of the checkerboard, and the size of each checkerboard is n, the first coordinate of the original calibration point in the world coordinate system can be determined based on the distance between the checkerboard pattern and the coordinate origin in the world coordinate system and the position of the checkerboard where the original calibration point is located.

[0046] The first coordinate of the mirror calibration point in the world coordinate system can be determined according to the first coordinate of the original coordinate point corresponding to it and the relative position relationship between the mirror device and the calibration target. For example, when the mirror device is vertically arranged at the outer edge of the calibration target, for each original calibration point with a mirror calibration point, the distance between the original calibration point and the mirror device can be first determined, and the mirror calibration point of the original calibration point is symmetrical with the original calibration point about the mirror device, so the first coordinate of the mirror calibration point in the world coordinate system can be determined.

[0047] Step 206 , performing feature point detection based on the calibration image, and determining second coordinates of a plurality of feature points obtained by the detection in the image coordinate system.

[0048] Among them, in the camera extrinsic calibration, the main purpose of feature point detection is to extract stable and repeatable key points, such as corner points or dots, from the image captured by the camera. These points will be used in subsequent matching and calculation processes to determine the camera's extrinsic parameters.

[0049] Specifically, the computer device may perform feature point detection based on the calibration image to obtain multiple feature points, and then determine the second coordinate of each feature point in the image coordinate system according to the position of each feature point in the calibration image.

[0050] Exemplarily, the computer device can first perform dedistortion processing on the calibration image to obtain the target image, and then use traditional mathematical calculation methods to detect feature points, such as SIFT (Scale-Invariant Feature Transform) algorithm, SURF (Speeded Up Robust Features) algorithm, ORB (Oriented FAST and Rotated BRIEF) algorithm or FAST (features from accelerated segment test) algorithm. Any one of the algorithms can be used for feature point detection.

[0051] Exemplarily, after dedistorting the target image, the computer device can also use deep learning methods to detect feature points. For example, a feature point detection method based on convolutional neural networks (CNN) can be used to automatically learn image features by training the network, which can then be applied to feature point detection of the calibration image.

[0052] Step 208 , determining the feature points corresponding to the original calibration points and the feature points corresponding to the mirror calibration points from the plurality of feature points.

[0053] Specifically, the calibration image is obtained by photographing the calibration target, and the calibration image includes the calibration target and a mirror calibration target of at least part of the calibration target in the mirror device. Therefore, for each original calibration point in the calibration target, there is a corresponding feature point in the calibration image, and for each mirror calibration point, there is also a corresponding feature point in the calibration image. After obtaining the feature points in the calibration image, the computer device can respectively establish a corresponding relationship between each original calibration point and its corresponding feature point, and establish a corresponding relationship between each mirror calibration point and its corresponding feature point.

[0054] Exemplarily, the computer device can sort multiple original calibration points and multiple mirror calibration points in a preset manner, and sort multiple feature points in a preset manner. For each original calibration point, the feature point with the same sorting position as the original calibration point is determined as the feature point corresponding to the original calibration point, and for each mirror calibration point, the feature point with the same sorting position as the mirror calibration point is determined as the feature point corresponding to the mirror calibration point.

[0055] Step 210, determining the external parameters of the camera according to the first coordinates of each original calibration point, the second coordinates of the feature points corresponding to each original calibration point, the first coordinates of each mirror calibration point, and the second coordinates of the feature points corresponding to each mirror calibration point.

[0056] Specifically, after establishing the correspondence between the calibration points and the feature points, including the correspondence between the original calibration points and the corresponding feature points, and the correspondence between the mirror calibration points and the corresponding feature points, the computer device can further perform parameter estimation on these point pairs with corresponding relationships. For example, mathematical models and optimization algorithms (such as the least squares method) can be used to estimate the rotation matrix and the translation vector. The estimated external parameters can be further verified and optimized, such as by comparing the predicted image points with the actually observed points to verify the accuracy of the calibration results, and make necessary adjustments.

[0057] In the above-mentioned camera extrinsic calibration method, a mirror device is arranged at the outer edge of the calibration target, the calibration image includes the calibration target and a mirror calibration target of at least part of the calibration target in the mirror device, the calibration target includes a plurality of original calibration points, and the mirror calibration target includes a plurality of mirror calibration points. When performing calibration, the first coordinate of each original calibration point in the world coordinate system can be obtained, and the first coordinate of each mirror calibration point in the world coordinate system can be obtained, feature point detection is performed based on the calibration image, and the second coordinates of the plurality of feature points obtained by detection in the image coordinate system are determined, and the plurality of feature points are obtained from the plurality of feature points. In the feature point acquisition, the feature points corresponding to each original calibration point and the feature points corresponding to each mirror calibration point are determined respectively. According to the first coordinates of each original calibration point, the second coordinates of the feature points corresponding to each original calibration point, the first coordinates of each mirror calibration point and the second coordinates of the feature points corresponding to each mirror calibration point, the external parameters of the camera are determined. Since the original calibration target can be expanded through the mirror device, the range of the external parameter calibration points can be expanded, the accuracy of long-distance projection is improved, and the calibrated external parameters can be applied to a wider range of point projections.

[0058] In some exemplary embodiments, feature point detection is performed based on a calibration image, and a second coordinate of each feature point detected in an image coordinate system is determined, including: dedistorting the calibration image to obtain a target image corresponding to the calibration image; performing feature point detection on the target image to obtain multiple feature points; and determining the second coordinate of each feature point in the image coordinate system based on the position of each feature point in the calibration image.

[0059] Among them, image dedistortion is to eliminate the distortion caused by camera lens, shooting angle or environmental factors during the image acquisition process through a series of algorithms and technical means, so that the image can more realistically and accurately reflect the appearance of the original scene.

[0060] Specifically, the computer device can establish a corresponding distortion model based on the distortion coefficient obtained by camera calibration. The distortion model usually includes a radial distortion model and a tangential distortion model, which are used to describe the distortion of pixels in the image. Then, the distortion model can be used to remap each pixel in the calibration image, that is, the position of each pixel in the undistorted image is calculated according to the distortion model. This process involves coordinate transformation and interpolation calculation to ensure the quality of the remapped image. The computer device can further generate a dedistorted image based on the remapped pixel coordinates and the pixel values ​​of the original image through an interpolation algorithm (such as bilinear interpolation, bicubic interpolation, etc.). This image is the target image corresponding to the calibration image, where the choice of interpolation algorithm depends on the requirements for image quality and computational efficiency.

[0061] After obtaining the target image, the computer device can further perform feature point detection on the target image to obtain multiple feature points, and determine the second coordinate of each feature point in the image coordinate system according to the position of each feature point in the calibration image.

[0062] In this embodiment, by dedistorting the calibration image to obtain the target image, performing feature point detection on the target image and then determining the second coordinates of the feature points, the accuracy of feature detection can be improved, and a more accurate second coordinate can be obtained, thereby improving the accuracy of camera extrinsic calibration.

[0063] In some embodiments, determining the feature points corresponding to the original calibration points and the feature points corresponding to the mirror calibration points from the plurality of feature points respectively includes:

[0064] A plurality of original calibration points and a plurality of mirror calibration points are sorted in a preset manner; a plurality of feature points are sorted in a preset manner; for each original calibration point, a feature point having the same sorting position as the original calibration point is determined as a feature point corresponding to the original calibration point; for each mirror calibration point, a feature point having the same sorting position as the mirror calibration point is determined as a feature point corresponding to the mirror calibration point.

[0065] Specifically, the computer device can sort all calibration points including the original calibration points and the mirror calibration points according to a preset method, and sort the feature points detected based on the calibration image according to the same preset method, and then for each original calibration point, the feature point with the same sorting position as the original calibration point is determined as the feature point corresponding to the original calibration point, and for each mirror calibration point, the feature point with the same sorting position as the mirror calibration point is determined as the feature point corresponding to the mirror calibration point. It can be understood that in a specific application, the sorting method can be set as needed.

[0066] For example, assuming that there are 25 calibration points including original calibration points and mirror calibration points, after sorting the calibration points and feature points according to a preset method, if the sorting position of an original calibration point is N, then the sorting position of its corresponding feature point is also N; if the sorting position of a mirror calibration point is M, then the sorting position of its corresponding feature point is also M, where N and M are both less than or equal to 25.

[0067] In the above embodiment, since the calibration points including the original calibration points and the mirror calibration points and the feature points can be sorted in the same sorting manner, and then a corresponding relationship is established according to the sorting position, the corresponding relationship can be quickly established, thereby improving the efficiency of external parameter calibration.

[0068] In some embodiments, the calibration target is a preset pattern with known geometric features, and the mirror device is vertically arranged at the outer edge of the preset pattern.

[0069] Specifically, in this embodiment, the calibration target can be a preset pattern, and the preset pattern has known geometric features. The preset image can be, for example, a checkerboard pattern. The overall shape of the preset pattern can be a rectangle and is set on a horizontal ground. The mirror device is vertically set at the outer edge of the preset pattern, and the lower edge of the mirror device can be parallel to one of the sides of the preset pattern and perpendicular to the horizontal ground.

[0070] For example, when the mirror device is vertically arranged at the outer edge of the preset pattern, it can be spaced a certain distance L from the outer edge to ensure that the preset pattern can present a complete mirror image in the mirror device. The specific value of L can be set according to actual needs.

[0071] It is understandable that in a specific application, the preset pattern is not limited to a checkerboard, and any pattern with known geometric features is acceptable, and the present application does not limit the preset pattern.

[0072] In the above embodiment, since a preset image with known geometric features is used as a calibration target, and the mirror device is vertically arranged at the outer edge of the preset image, the convenience of the external parameter calibration process is improved.

[0073] In some embodiments, when the preset pattern is a checkerboard pattern, obtaining the first coordinate of each original calibration point in the world coordinate system, and obtaining the first coordinate of each mirror calibration point in the world coordinate system, includes: for each original calibration point, determining the first coordinate of the original calibration point in the world coordinate system according to the position of the original calibration point in the checkerboard pattern; for each mirror calibration point, obtaining the first coordinate of the original calibration point corresponding to the mirror calibration point in the world coordinate system, and determining the first coordinate of the mirror calibration point in the world coordinate system according to the first coordinate of the original calibration point corresponding to the mirror calibration point in the world coordinate system, the checkerboard square size of the checkerboard pattern, and the distance between the mirror device and the outer edge of the checkerboard pattern.

[0074] Specifically, since the coordinate origin of the world coordinate system is fixed and the size of the checkerboard squares in the checkerboard pattern is also known, for each original calibration point, the first coordinate of the original calibration point in the world coordinate system can be determined based on the distance between the checkerboard pattern and the coordinate origin in the world coordinate system and the position of the checkerboard square where the original calibration point is located.

[0075] For the mirror calibration point, the original calibration point corresponding to the mirror calibration point can be determined first, that is, the original calibration point used to form the mirror calibration point, and then the first coordinate of the original calibration point in the world coordinate system can be obtained. According to the first coordinate of the original calibration point in the world coordinate system, the chessboard square size of the chessboard pattern and the distance between the mirror device and the outer edge of the chessboard pattern, the first coordinate of the mirror calibration point in the world coordinate system is determined.

[0076] For example, see Figure 3 , assuming that an original calibration point in the chessboard pattern is P, the position of P is as follows Figure 3 As shown, its coordinates in the world coordinate system are (x p ,y p ), it is known that the size of the chessboard square where p is located is n*n, and according to the distance between the mirror device 304 and the chessboard pattern 302 is L, the coordinates of the mirror calibration point p′ of p can be inferred to be (x p ,y p +4*n+2*L), the coordinates of other calibrated checkerboard corner points extended by the mirror can be obtained in the world coordinate system according to the same principle.

[0077] In the above embodiment, since the preset pattern is a checkerboard pattern, the first coordinate of the mirror calibration point in the world coordinate system is determined according to the first coordinate of the original calibration point corresponding to the mirror calibration point in the world coordinate system, the checkerboard square size of the checkerboard pattern, and the distance between the mirror device and the outer edge of the checkerboard pattern. The coordinates of the mirror calibration point can be quickly determined, thereby improving the efficiency of external parameter calibration.

[0078] In some embodiments, determining the external parameters of the camera according to the first coordinates of each original calibration point, the second coordinates of the feature points corresponding to each original calibration point, the first coordinates of each mirror calibration point, and the second coordinates of the feature points corresponding to each mirror calibration point includes:

[0079] Parameter estimation is performed according to the first coordinates of each original calibration point, the second coordinates of the feature points corresponding to each original calibration point, the first coordinates of each mirror calibration point, and the second coordinates of the feature points corresponding to each mirror calibration point to obtain an initial rotation matrix and an initial translation vector; the initial rotation matrix and the initial translation vector are evaluated to obtain an evaluation result; when the evaluation result meets the preset calibration accuracy condition, the initial rotation matrix and the initial translation vector are determined as the external parameters of the camera.

[0080] Specifically, after the computer device obtains the first coordinates of each calibration point including the original calibration point and the mirror calibration point and the second coordinates of the feature point corresponding to each calibration point and establishes a corresponding relationship, it can perform parameter estimation based on these corresponding point pairs to obtain the initial extrinsic parameters of the camera, that is, the initial extrinsic parameters, the initial rotation matrix and the initial translation vector. The parameter estimation here can be implemented using the existing extrinsic parameter calculation function, which will not be elaborated in this application.

[0081] The computer device can further evaluate the initial external parameters to obtain an evaluation result. By evaluating the initial external parameters, the accuracy of the initial external parameters of the camera (including the rotation matrix and the translation vector) can be checked to ensure that they can correctly describe the position and posture of the camera in the world coordinate system. The computer device can further determine whether the initial external parameters meet the preset calibration accuracy conditions. When the computer device determines that the evaluation result meets the preset calibration accuracy conditions, the initial rotation matrix and the initial translation vector are determined as the external parameters of the camera. Here, the preset calibration accuracy conditions can be determined according to different evaluation methods.

[0082] Exemplarily, the computer device can calculate the reprojection error based on the initial external parameters. If the reprojection error is small enough, for example, less than a certain threshold, it is considered that the initial camera external parameters meet the preset calibration accuracy conditions and are accurate. In a specific implementation, the points in the world coordinate system can be projected onto the image plane through the camera external parameters, and the differences between these projected points and the points actually detected in the image can be calculated to obtain the reprojection error.

[0083] Exemplarily, the computer device can evaluate the initial extrinsic parameters by scene consistency check. In a specific implementation, the same scene can be photographed at multiple viewing angles, and the images at different viewing angles can be aligned using camera extrinsic parameters. In the aligned images, check whether the same feature points are consistent at different viewing angles. If the average value of the position difference of the feature points at different viewing angles is within an acceptable range, for example, less than a certain threshold, it is considered that the initial camera extrinsic parameters meet the preset calibration accuracy conditions and are accurate.

[0084] Further, when the computer device determines that the evaluation result does not meet the preset calibration accuracy condition, a calibration optimization step is executed.

[0085] Exemplarily, the computer device can adopt any one of the algorithms such as gradient descent method, Newton method, Levenberg-Marquardt algorithm, etc., with the goal of minimizing the reprojection error, and obtain the camera's extrinsic parameters by iteratively adjusting the camera's extrinsic parameters until a certain convergence condition is met or a preset number of iterations is reached.

[0086] Exemplarily, the computer device introduces additional constraints during the optimization process, such as the non-holistic constraints of the camera, the motion characteristics of the camera, etc., to improve the accuracy and robustness of the optimization results. Taking the camera as a vehicle-mounted camera as an example, in a specific application, the motion characteristics of the vehicle can be used to improve the random sampling consensus algorithm (RANSAC) to improve the estimation speed and robustness of the camera external parameters.

[0087] In some other embodiments, deep learning models such as convolutional neural networks (CNN) can also be used to learn the movement and posture of the camera, so as to automatically determine the position and posture of the camera. In addition, technologies such as generative adversarial networks (GAN) can also be combined to improve the quality of the data set and accelerate the training process.

[0088] In the above embodiment, after obtaining initial external parameters such as the initial rotation matrix and the initial translation vector, the initial external parameters can be evaluated. Only when the evaluation results meet the preset calibration accuracy conditions, the initial rotation matrix and the initial translation vector are determined as the external parameters of the camera. When the evaluation results do not meet the preset calibration accuracy conditions, the calibration optimization step is performed to obtain the external parameters of the camera, thereby further improving the accuracy of the calculated camera external parameters.

[0089] In some embodiments, the camera of the present application can be a vehicle-mounted camera, the calibration target is set in a factory calibration room, and the calibration image is taken when the vehicle is moved to the factory calibration room and the coordinate origin of the checkerboard pattern is located at the center point of the vehicle.

[0090] In this embodiment, the camera extrinsic calibration method is applied to the camera extrinsic factory calibration. Different from other types of extrinsic calibration, the camera extrinsic factory calibration uses a mechanical device to move the vehicle to a preset position, and the calibration target is a fixed calibration pattern, so that a relatively accurate calibration point coordinate can be obtained. Its main characteristics are stability, speed, and high accuracy. It is usually applied to the factory calibration of the camera during assembly, such as the extrinsic calibration of the camera on the vehicle after assembly. The factory calibration method is usually based on setting a fixed calibration pattern in a limited space. The calibration pattern covers the range of 1 to 2 meters outward from the vehicle outline, and as the coverage area increases, the construction space requirements and costs increase accordingly.

[0091] The additive extrinsic parameter calibration method provided in the embodiment of the present application can be based on mirror reflection, using a limited space to lay out the calibration pattern, but it is a calibration method that can calibrate accurate extrinsic parameters in a larger range. In this embodiment, it is only necessary to add a mirror at a lower cost on the basis of the original factory calibration to collect calibration data in a larger range than the original calibration room, and by adding corresponding calibration points to the calibration program, high-accuracy camera extrinsics covering a larger range can be calculated.

[0092] In this embodiment, the calibration room is renovated by adding a mirror, so the construction cost is low; the calibration logic is simple, the calibration range can be expanded, and the accuracy of the external reference long-distance projection is significantly improved.

[0093] The following combination Figures 4 to 10 The camera extrinsic parameter calibration method of the present application is specifically described in the camera extrinsic parameter factory calibration scenario.

[0094] At present, the factory calibration room usually consists of the following components: displacement machine and calibration pattern. The displacement machine is used to move the vehicle to the center of the calibration pattern to ensure that the origin of the coordinates of the corner points of the calibration pattern is located at the center of the vehicle; the calibration pattern is a pre-designed pattern with a width of W. Chessboard , the length is H Chessboard The coordinates of the corner points of the calibration pattern are fixed in the calibration room. After the camera shoots the calibration pattern, the coordinates of the corner points of the calibration pattern can be obtained by identifying the corner points, and combined with the preset corner point coordinates in the vehicle coordinate system, the camera extrinsic parameters can be calculated.

[0095] Based on the calibration room mentioned above, it is necessary to fix mirrors on the four walls of the calibration room or on the four outer edges of the calibration pattern. The mirrors are fixed parallel to the calibration pattern at a distance of L. From this, it can be deduced that the length of the front and rear mirrors is (W Chessboard +2*L), the length of the left and right mirrors is (H Chessboard +2*L), the height of the four mirrors is greater than the height of the calibration vehicle (assuming the height of the calibration vehicle is H car ), the installed mirror is strictly vertical to the ground plane (90 degrees). The overall workshop diagram is as follows Figure 4 shown.

[0096] After the workshop mirror is constructed, the calibration pattern in a larger space can be simulated under the imaging of the vehicle camera due to the principle of mirror reflection. Take the right side as an example. Figure 5 As shown in the figure, the ego car is moved to the center of the workshop by the workshop mechanical device. The coordinate system is a preset world coordinate system established with the center of the calibration pattern. The front of the vehicle is the X axis, the left side of the vehicle is the Y axis, and the Z axis is the upward direction. The coordinate system is referred to as O cali The calibration pattern can be expanded through the mirror. For details, please refer to Figure 3 , where p is a known calibration point in the preset calibration pattern (i.e., the original calibration point), assuming that it is cali The coordinates are (x p ,y p ), it is known that the size of the chessboard square where p is located is n*n. According to the mirror installation size, the coordinates of the mirror calibration point p′ of p can be inferred to be (x p ,y p +4*n+2*L), the coordinates of other calibration checkerboard corner points expanded by mirroring can be obtained in the calibration coordinate system according to the same principle. Since the calibration pattern is expanded, the calibration image captured by the on-board camera can include the original calibration pattern and the calibration pattern after mirroring expansion. Taking the right side as an example (after dedistortion), the calibration image captured by the on-board camera is as follows Figure 6 shown.

[0097] Through the above, we can build a basic hardware system for capturing expanded size calibration data in the originally limited calibration space. After building a calibration workshop for installing mirrors, we can calibrate high-precision camera external parameters covering a wider range through the software process system, so as to adapt to more application scenarios. Figure 7 To explain:

[0098] First, since the calibration pattern specifications of the original calibration room are known, the coordinates of all the original calibration points in the calibration coordinate system (i.e., the world coordinate system) are also known. The calibration points here refer to the corner points of the chessboard, i.e., the intersection points of the black and white grids inside the chessboard, such as Figure 8 In the example in , the circled points are the corner points of the chessboard. It is assumed here that there are M original calibration points in the right calibration chessboard between the calibration points. In this example, M = 4*2+6*4.

[0099] Since the distance between the mirror installation position and the calibration pattern is known to be L, and the mirror is strictly perpendicular to the ground, an extended chessboard can be constructed through the mirror. The calibration pattern on each side of the vehicle can be extended outward through the mirror, and the calibration pattern offset of the mirror simulation is 2*L.

[0100] In the specific implementation, the coordinates of the mirror's chessboard corner points (i.e., the mirror calibration points) can be calculated according to the installation specifications, thereby obtaining the coordinates of the M calibration points of the mirror extension. Combined with the coordinates of the existing M original calibration points, the coordinates of 2*M calibration points can be obtained, thereby constituting the actual coordinates of the calibration points required for the camera's extrinsic parameters, that is, the coordinates in the calibration coordinate system; on the other hand, drive the vehicle into the calibration workshop, move the vehicle to the preset position through mechanical equipment, and start the camera to capture the image required for the extrinsic parameter calibration. Here, the camera on the right is taken as an example.

[0101] Since the original image data is distorted, it is necessary to perform a dedistortion operation before identifying the checkerboard corner points on the image. The specific dedistortion process is not described here. After dedistortion is completed, corner point recognition (i.e. feature point detection) can be performed. Specifically, the purpose of corner point recognition is to obtain the coordinates of the checkerboard pattern corner points in the image coordinate system. The specific recognition operation can be completed by calling the corresponding OpenCV function, such as Fig. 9 As shown in the figure, the circled point position is the coordinate position of the identified corner point; the external parameter calculation requires the coordinates of the corner points of the calibration pattern on the image and in the actual calibration coordinate system to form a one-to-one corresponding point pair. Since the order of the points after recognition is not fixed, it is necessary to rearrange the point order according to the preset method, as shown in the example Fig.10 , the calibration points in the corresponding calibration coordinate system are rearranged in the same preset way; after completing the above, the ordered point pairs required for the extrinsic parameter calculation are obtained, so that the camera extrinsic parameters can be calculated using the extrinsic parameter calculation function.

[0102] The camera extrinsic parameter calibration method provided in the embodiment of the present application can be a calibration method for calibrating accurate extrinsic parameters in a larger range in a limited space. By adding a mirror on the basis of the original factory calibration in a relatively low-cost manner, a calibration pattern of a larger range of mirror expansion can be collected, and thus calibration data of a larger range than the original calibration room can be obtained. On this basis, the corresponding calibration points added due to the mirror expansion are added to the calibration program, so that high-accuracy camera extrinsic parameters covering a larger range can be calculated.

[0103] This embodiment has the following advantages:

[0104] 1) Low construction cost: only the front, back, left and right mirrors need to be added to the existing factory calibration construction plan; 2) Simple calibration logic: no major adjustments are required to the original calibration procedure, only the calibration points due to the expansion of the mirror need to be increased; 3) Improved accuracy of external parameters: Since the scheme can expand the original calibration pattern through the mirror, the range of external parameter points can be expanded to twice the original, improving the accuracy of long-distance projection, so that the calibrated external parameters can be applied to a wider range of point projections.

[0105] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0106] Based on the same inventive concept, the embodiment of the present application also provides a camera extrinsic calibration device for implementing the camera extrinsic calibration method involved above. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above method, so the specific limitations in one or more camera extrinsic calibration device embodiments provided below can refer to the limitations of the camera extrinsic calibration method above, and will not be repeated here.

[0107] In some exemplary embodiments, Fig.11 As shown, a camera extrinsic calibration device 1100 is provided, comprising:

[0108] The calibration image acquisition module 1102 is used to acquire a calibration image captured by the camera for the calibration target; wherein a mirror device is provided at the outer edge of the calibration target, and the calibration image includes the calibration target and a mirror calibration target of at least part of the calibration target in the mirror device, wherein the calibration target includes a plurality of original calibration points, and the mirror calibration target includes a plurality of mirror calibration points;

[0109] A first coordinate acquisition module 1104 is used to acquire the first coordinate of each original calibration point in the world coordinate system, and acquire the first coordinate of each mirror calibration point in the world coordinate system;

[0110] A second coordinate acquisition module 1106 is used to perform feature point detection based on the calibration image and determine second coordinates of the plurality of feature points obtained by the detection in the image coordinate system;

[0111] A corresponding relationship determination module 1108 is used to determine, from the plurality of feature points, the feature points corresponding to the original calibration points and the feature points corresponding to the mirror calibration points;

[0112] The external parameter determination module 1110 is used to determine the external parameters of the camera according to the first coordinates of each original calibration point, the second coordinates of the feature points corresponding to each original calibration point, the first coordinates of each mirror calibration point and the second coordinates of the feature points corresponding to each mirror calibration point.

[0113] In the above-mentioned camera extrinsic calibration device, a mirror device is arranged at the outer edge of the calibration target, the calibration image includes the calibration target and a mirror calibration target of at least part of the calibration target in the mirror device, the calibration target includes a plurality of original calibration points, and the mirror calibration target includes a plurality of mirror calibration points. When performing calibration, the first coordinate of each original calibration point in the world coordinate system can be obtained, and the first coordinate of each mirror calibration point in the world coordinate system can be obtained, feature point detection is performed based on the calibration image, and the second coordinates of the plurality of feature points obtained by detection in the image coordinate system are determined, and the plurality of feature points are obtained from the plurality of feature points. In the feature point acquisition, the feature points corresponding to each original calibration point and the feature points corresponding to each mirror calibration point are determined respectively. According to the first coordinates of each original calibration point, the second coordinates of the feature points corresponding to each original calibration point, the first coordinates of each mirror calibration point and the second coordinates of the feature points corresponding to each mirror calibration point, the external parameters of the camera are determined. Since the original calibration target can be expanded through the mirror device, the range of the external parameter calibration points can be expanded, the accuracy of long-distance projection is improved, and the calibrated external parameters can be applied to a wider range of point projections.

[0114] In some embodiments, the second coordinate acquisition module 1106 is further used to: perform dedistortion processing on the calibration image to obtain a target image corresponding to the calibration image; perform feature point detection on the target image to obtain multiple feature points; and determine the second coordinate of each feature point in the image coordinate system according to the position of each feature point in the calibration image.

[0115] In some embodiments, the correspondence determination module 1108 is also used to: sort multiple original calibration points and multiple mirror calibration points according to a preset method; sort multiple feature points according to a preset method; for each original calibration point, determine the feature point with the same sorting position as the original calibration point as the feature point corresponding to the original calibration point; for each mirror calibration point, determine the feature point with the same sorting position as the mirror calibration point as the feature point corresponding to the mirror calibration point.

[0116] In some embodiments, the calibration target is a preset pattern with known geometric features, and the mirror device is vertically arranged at the outer edge of the preset pattern.

[0117] In some embodiments, the preset pattern is a checkerboard pattern, and the first coordinate acquisition module 1104 is further used to: for each original calibration point, determine the first coordinate of the original calibration point in the world coordinate system according to the position of the original calibration point in the checkerboard pattern; for each mirror calibration point, obtain the first coordinate of the original calibration point corresponding to the mirror calibration point in the world coordinate system, and determine the first coordinate of the mirror calibration point in the world coordinate system according to the first coordinate of the original calibration point corresponding to the mirror calibration point in the world coordinate system, the checkerboard square size of the checkerboard pattern, and the distance between the mirror device and the outer edge of the calibration pattern.

[0118] In some embodiments, the camera is a vehicle-mounted camera, the calibration target is set in a factory calibration room, and the calibration image is taken when the vehicle is moved to the factory calibration room and the coordinate origin of the checkerboard pattern is located at the center point of the vehicle.

[0119] In some embodiments, the external parameter determination module 1110 is also used to: perform parameter estimation based on the first coordinates of each original calibration point, the second coordinates of the feature points corresponding to each original calibration point, the first coordinates of each mirror calibration point, and the second coordinates of the feature points corresponding to each mirror calibration point to obtain an initial rotation matrix and an initial translation vector; evaluate the initial rotation matrix and the initial translation vector to obtain an evaluation result; when the evaluation result meets the preset calibration accuracy condition, determine the initial rotation matrix and the initial translation vector as the external parameters of the camera; when the evaluation result does not meet the preset calibration accuracy condition, execute the calibration optimization step to obtain the external parameters of the camera.

[0120] Each module in the above-mentioned camera extrinsic calibration device can be implemented in whole or in part by software, hardware, or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0121] In some exemplary embodiments, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Fig.12As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store calibration images, camera external parameters and other data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the camera external parameter calibration method provided in the embodiment of the present application is implemented.

[0122] In some exemplary embodiments, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Fig.13 As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, near field communication (Near Field Communication, NFC) or other technologies. When the computer program is executed by the processor, the camera extrinsic parameter calibration method provided in the embodiment of the present application is implemented. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0123] Those skilled in the art will understand that Fig.12 , Fig.13The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0124] In some exemplary embodiments, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps of the above-mentioned camera extrinsic parameter calibration method when executing the computer program.

[0125] In some embodiments, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned camera extrinsic parameter calibration method are implemented.

[0126] In some embodiments, a computer program product is provided, including a computer program, which implements the steps of the above-mentioned camera extrinsic parameter calibration method when executed by a processor.

[0127] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0128] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0129] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0130] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A camera extrinsic calibration method, characterized in that: The method comprises: Acquire a calibration image captured by a camera for a calibration target; wherein a mirror device is provided at an outer edge of the calibration target, the calibration image includes the calibration target and a mirror calibration target of at least part of the calibration target in the mirror device, the calibration target includes a plurality of original calibration points, and the mirror calibration target includes a plurality of mirror calibration points; Obtaining the first coordinate of each of the original calibration points in the world coordinate system, and obtaining the first coordinate of each of the mirror calibration points in the world coordinate system; Performing feature point detection based on the calibration image, and determining second coordinates of a plurality of feature points obtained by the detection in the image coordinate system; Determine, from the plurality of feature points, feature points corresponding to the original calibration points and feature points corresponding to the mirror calibration points; The external parameters of the camera are determined according to the first coordinates of each of the original calibration points, the second coordinates of the feature points corresponding to each of the original calibration points, the first coordinates of each of the mirror calibration points, and the second coordinates of the feature points corresponding to each of the mirror calibration points.

2. The method according to claim 1, characterized in that The detecting of feature points based on the calibration image and determining the second coordinates of each feature point obtained by the detection in the image coordinate system includes: Performing a dedistortion process on the calibration image to obtain a target image corresponding to the calibration image; Performing feature point detection on the target image to obtain multiple feature points; According to the position of each feature point in the calibration image, a second coordinate of each feature point in the image coordinate system is determined.

3. The method according to claim 2, characterized in that The step of respectively determining, from the plurality of feature points, feature points corresponding to the original calibration points and feature points corresponding to the mirror calibration points comprises: Sorting the plurality of original calibration points and the plurality of mirror calibration points according to a preset method; Sorting the plurality of feature points according to the preset method; For each original calibration point, determining a feature point having the same sorting position as the original calibration point as the feature point corresponding to the original calibration point; For each mirror calibration point, a feature point having the same sorting position as the mirror calibration point is determined as a feature point corresponding to the mirror calibration point.

4. The method according to claim 1, characterized in that: The calibration target is a preset pattern with known geometric features, and the mirror device is vertically arranged at the outer edge of the preset pattern.

5. The method according to claim 4, characterized in that The preset pattern is a checkerboard pattern, and the obtaining of the first coordinate of each of the original calibration points in the world coordinate system and the obtaining of the first coordinate of each of the mirror calibration points in the world coordinate system include: For each original calibration point, determine a first coordinate of the original calibration point in a world coordinate system according to a position of the original calibration point in the checkerboard pattern; For each mirror calibration point, obtain the first coordinate of the original calibration point corresponding to the mirror calibration point in the world coordinate system, and determine the first coordinate of the mirror calibration point in the world coordinate system according to the first coordinate of the original calibration point corresponding to the mirror calibration point in the world coordinate system, the checkerboard size of the checkerboard pattern, and the distance between the mirror device and the outer edge of the checkerboard pattern.

6. The method according to claim 5, characterized in that The camera is an on-board camera of the vehicle, the calibration target is set in a factory calibration room, and the calibration image is captured when the vehicle is moved to the factory calibration room and the coordinate origin of the checkerboard pattern is located at the center point of the vehicle.

7. The method according to claim 1, characterized in that Determining the external parameters of the camera according to the first coordinates of the original calibration points, the second coordinates of the feature points corresponding to the original calibration points, the first coordinates of the mirror calibration points, and the second coordinates of the feature points corresponding to the mirror calibration points includes: Parameter estimation is performed according to the first coordinates of each of the original calibration points, the second coordinates of the feature points corresponding to each of the original calibration points, the first coordinates of each of the mirror calibration points, and the second coordinates of the feature points corresponding to each of the mirror calibration points to obtain an initial rotation matrix and an initial translation vector; Evaluate the initial rotation matrix and the initial translation vector to obtain an evaluation result; When the evaluation result satisfies a preset calibration accuracy condition, determining the initial rotation matrix and the initial translation vector as external parameters of the camera; The method further comprises: When the evaluation result does not meet the preset calibration accuracy condition, a calibration optimization step is performed to obtain the external parameters of the camera.

8. A camera extrinsic calibration device, characterized in that: The device comprises: A calibration image acquisition module, used to acquire a calibration image taken by a camera for a calibration target; wherein a mirror device is provided at the outer edge of the calibration target, the calibration image includes the calibration target and a mirror calibration target of at least part of the calibration target in the mirror device, the calibration target includes a plurality of original calibration points, and the mirror calibration target includes a plurality of mirror calibration points; A first coordinate acquisition module, used to acquire the first coordinate of each of the original calibration points in the world coordinate system, and to acquire the first coordinate of each of the mirror calibration points in the world coordinate system; A second coordinate acquisition module, used to perform feature point detection based on the calibration image, and determine second coordinates of a plurality of feature points obtained by the detection in the image coordinate system; A corresponding relationship determination module, used to determine, from the plurality of feature points, the feature points corresponding to the original calibration points and the feature points corresponding to the mirror calibration points; The external parameter determination module is used to determine the external parameters of the camera according to the first coordinates of each original calibration point, the second coordinates of the feature points corresponding to each original calibration point, the first coordinates of each mirror calibration point, and the second coordinates of the feature points corresponding to each mirror calibration point.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images