Camera external parameter calibration method and device, computer equipment and readable storage medium
The method generates a bird's-eye view from a distorted image of a closed polygon calibration target to adjust reference angles, improving the accuracy of camera extrinsic calibration by determining optimal angles, thereby enhancing the precision of yaw angle determination.
Patent Information
- Application Number
- CN202411207518.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-30
- Publication Date
- 2024-12-06
AI Technical Summary
Traditional camera extrinsic calibration methods using Apriltag or checkerboard patterns face challenges with high complexity, leading to low accuracy when applied to cameras with large distortion, often failing to detect vertices.
A method involving generating a bird's-eye view from a distorted image of a closed polygon calibration target, adjusting reference roll and pitch angles, and calculating cost values based on vertex distances to determine optimal angles, ensuring high accuracy and minimizing false detections.
Improves the accuracy of camera extrinsic calibration by determining optimal roll and pitch angles through multiple adjustments, enhancing the precision of yaw angle determination and overall calibration results.
Smart Images

Figure CN119090973A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of camera calibration, and in particular to a camera extrinsic calibration method, device, computer equipment and computer-readable storage medium. Background Art
[0002] Since the original images captured by the camera often have certain distortion, in order to make the images captured by the camera consistent with what the human eye observes, camera calibration is required. Camera extrinsics are parameters that describe the position and posture of the camera in the world coordinate system. Camera extrinsic calibration mainly calibrates the roll angle, pitch angle, and yaw angle.
[0003] Traditional camera extrinsic calibration often uses Apriltag (a black and white QR code-like pattern) or checkerboard as the target pattern, and calibrates the camera extrinsic parameters by aligning the surveying points of the target pattern with the image detection points. However, due to the high complexity of the target pattern, when applied to a camera with large distortion, it is easy to fail to detect vertices, so there is a problem of low accuracy of the calibration result. Summary of the invention
[0004] Based on this, it is necessary to provide a camera extrinsic calibration method, device, computer equipment, computer readable storage medium and computer program product to address the above technical problems, which can improve the accuracy of the calibration results.
[0005] In a first aspect, the present application provides a camera extrinsic calibration method, comprising:
[0006] Obtain a distorted image of the calibration target taken by the camera to be calibrated, and generate a bird's-eye view based on the distorted image and the reference external parameters of the camera to be calibrated; the calibration target is a closed polygon;
[0007] Extracting a closed figure in the bird's-eye view, and calculating a cost value based on distances between adjacent vertices and distances between diagonal vertices in the closed figure;
[0008] Adjusting the reference roll angle and the reference pitch angle in the reference extrinsic parameters, and continuing to perform the step of generating a bird's-eye view based on the distorted image and the reference extrinsic parameters of the camera to be calibrated, until the adjusted reference roll angle and the adjusted reference pitch angle meet the adjustment stop condition;
[0009] Based on each adjusted cost value, a target roll angle is determined in each adjusted reference roll angle, and a target pitch angle is determined in each adjusted reference pitch angle; and based on a bird's-eye view corresponding to the target roll angle and the target pitch angle, a target yaw angle is determined.
[0010] In one embodiment, determining a target yaw angle based on a bird's-eye view corresponding to a target roll angle and a target pitch angle includes:
[0011] Extract the target closed figure in the bird's-eye view corresponding to the target roll angle and the target pitch angle;
[0012] A target yaw angle is determined based on coordinates of a plurality of target vertices in a target closed figure.
[0013] In one embodiment, determining a target yaw angle based on coordinates of a plurality of target vertices in a target closed figure includes:
[0014] Calculate the reference coordinates of the reference point according to the coordinates of multiple target vertices in the target closed figure;
[0015] Control the movement of the camera to be calibrated and re-acquire the reference coordinates of the reference point;
[0016] Based on the change in the reference coordinates, the target yaw angle is determined.
[0017] In one embodiment, based on each adjusted cost value, determining a target roll angle from each adjusted reference roll angle, and determining a target pitch angle from each adjusted reference pitch angle, comprises:
[0018] Determining a target cost value among the cost values obtained after each adjustment;
[0019] The adjusted reference roll angle corresponding to the target cost value in each adjusted reference roll angle is used as the target roll angle, and the adjusted reference pitch angle corresponding to the target cost value in each adjusted reference pitch angle is used as the target pitch angle.
[0020] In one embodiment, determining a target cost value from each adjusted cost value includes:
[0021] The minimum cost value among the cost values obtained after the adjustments is taken as the target cost value.
[0022] In one embodiment, the calibration target is a closed equilateral polygon; and the cost value is calculated based on the distance between adjacent vertices and the distance between diagonal vertices in the closed figure, including:
[0023] The distance between adjacent vertices is determined as the length of each side of the closed figure, and the distance between diagonal vertices is determined as the length of each diagonal line of the closed figure;
[0024] The cost value is determined based on the side length differences between the side lengths and the diagonal length differences between the diagonal lengths.
[0025] In a second aspect, the present application also provides a camera extrinsic calibration device, comprising:
[0026] An acquisition module is used to acquire a distorted image of a calibration target taken by the camera to be calibrated, and generate a bird's-eye view based on the distorted image and the reference external parameters of the camera to be calibrated; the calibration target is a closed polygon;
[0027] An extraction module, for extracting a closed figure in the bird's-eye view, and calculating a cost value based on distances between adjacent vertices and distances between diagonal vertices in the closed figure;
[0028] An adjustment module, used for adjusting a reference roll angle and a reference pitch angle in the reference extrinsic parameter, and continuing to execute the step of generating a bird's-eye view based on the distorted image and the reference extrinsic parameter of the camera to be calibrated, until the adjusted reference roll angle and the adjusted reference pitch angle meet an adjustment stop condition;
[0029] The determination module is used to determine the target roll angle in each adjusted reference roll angle based on each adjusted cost value, and determine the target pitch angle in each adjusted reference pitch angle; and determine the target yaw angle based on the bird's-eye view corresponding to the target roll angle and the target pitch angle.
[0030] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:
[0031] Obtain a distorted image of the calibration target taken by the camera to be calibrated, and generate a bird's-eye view based on the distorted image and the reference external parameters of the camera to be calibrated; the calibration target is a closed polygon;
[0032] Extracting a closed figure in the bird's-eye view, and calculating a cost value based on distances between adjacent vertices and distances between diagonal vertices in the closed figure;
[0033] Adjusting the reference roll angle and the reference pitch angle in the reference extrinsic parameters, and continuing to perform the step of generating a bird's-eye view based on the distorted image and the reference extrinsic parameters of the camera to be calibrated, until the adjusted reference roll angle and the adjusted reference pitch angle meet the adjustment stop condition;
[0034] Based on each adjusted cost value, a target roll angle is determined in each adjusted reference roll angle, and a target pitch angle is determined in each adjusted reference pitch angle; and based on a bird's-eye view corresponding to the target roll angle and the target pitch angle, a target yaw angle is determined.
[0035] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:
[0036] Obtain a distorted image of the calibration target taken by the camera to be calibrated, and generate a bird's-eye view based on the distorted image and the reference external parameters of the camera to be calibrated; the calibration target is a closed polygon;
[0037] Extracting a closed figure in the bird's-eye view, and calculating a cost value based on distances between adjacent vertices and distances between diagonal vertices in the closed figure;
[0038] Adjusting the reference roll angle and the reference pitch angle in the reference extrinsic parameters, and continuing to perform the step of generating a bird's-eye view based on the distorted image and the reference extrinsic parameters of the camera to be calibrated, until the adjusted reference roll angle and the adjusted reference pitch angle meet the adjustment stop condition;
[0039] Based on each adjusted cost value, a target roll angle is determined in each adjusted reference roll angle, and a target pitch angle is determined in each adjusted reference pitch angle; and based on a bird's-eye view corresponding to the target roll angle and the target pitch angle, a target yaw angle is determined.
[0040] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:
[0041] Obtain a distorted image of the calibration target taken by the camera to be calibrated, and generate a bird's-eye view based on the distorted image and the reference external parameters of the camera to be calibrated; the calibration target is a closed polygon;
[0042] Extracting a closed figure in the bird's-eye view, and calculating a cost value based on distances between adjacent vertices and distances between diagonal vertices in the closed figure;
[0043] Adjusting the reference roll angle and the reference pitch angle in the reference extrinsic parameters, and continuing to perform the step of generating a bird's-eye view based on the distorted image and the reference extrinsic parameters of the camera to be calibrated, until the adjusted reference roll angle and the adjusted reference pitch angle meet the adjustment stop condition;
[0044] Based on each adjusted cost value, a target roll angle is determined in each adjusted reference roll angle, and a target pitch angle is determined in each adjusted reference pitch angle; and based on a bird's-eye view corresponding to the target roll angle and the target pitch angle, a target yaw angle is determined.
[0045] The camera extrinsic parameter calibration method, device, computer equipment, computer-readable storage medium and computer program product generate a bird's-eye view by using the distorted image obtained by photographing the calibration target with the camera to be calibrated and the reference extrinsic parameters, and then extract the closed figures in the bird's-eye view. Since the calibration target is a closed polygon, the figure is relatively simple and not prone to false detection. Therefore, the closed figure extraction is convenient and has high accuracy. At the same time, the cost value is calculated based on the distance between adjacent vertices and the distance between diagonal vertices in the closed figure, with less calculation and higher accuracy. Then, different bird's-eye views are generated by adjusting the reference roll angle and the reference pitch angle in the reference extrinsic parameters multiple times. Since the bird's-eye view generated each time has a different distortion effect, the adjusted cost values are The value can reflect the degree of de-distortion of the corresponding bird's-eye view. Therefore, based on each adjusted cost value, the target roll angle is determined in each adjusted reference roll angle, and the target pitch angle is determined in each adjusted reference pitch angle. It can ensure that the bird's-eye view corresponding to the target roll angle and the target pitch angle is the bird's-eye view with the best de-distortion effect. This method of determining the target roll angle and the target pitch angle through multiple adjustments is conducive to improving the accuracy of the target roll angle and the target pitch angle in the calibration results. In addition, the target yaw angle in the calibration result is determined based on the bird's-eye view with the best de-distortion effect, which is conducive to eliminating the influence of the deviation of the target roll angle and the target pitch angle on the target yaw angle, and further improves the accuracy of the calibration result. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.
[0047] FIG1 is a diagram showing an application environment of a camera extrinsic calibration method according to an embodiment;
[0048] FIG2 is a schematic diagram of a flow chart of a camera extrinsic calibration method in one embodiment;
[0049] FIG3 is a schematic diagram of a distorted image in one embodiment;
[0050] FIG4 is a schematic diagram of a bird's-eye view corresponding to a target roll angle and a target pitch angle in one embodiment;
[0051] FIG5 is a schematic diagram of steps for determining a target yaw angle in one embodiment;
[0052] FIG6 is a schematic diagram of the principle of determining a target yaw angle in one embodiment;
[0053] FIG7 is a schematic diagram of the overall flow of a camera extrinsic calibration method in one embodiment;
[0054] FIG8 is a schematic diagram of a method for adjusting a reference external parameter in one embodiment;
[0055] FIG9 is a structural block diagram of a camera extrinsic calibration device in one embodiment;
[0056] FIG. 10 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION
[0057] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0058] Since the original images captured by the camera often have certain distortion, in order to make the images captured by the camera consistent with what the human eye observes, camera calibration is required. Camera extrinsics are parameters that describe the position and posture of the camera in the world coordinate system. Camera extrinsic calibration mainly calibrates the roll angle, pitch angle, and yaw angle.
[0059] Traditional camera extrinsic calibration often uses fixed targets or pattern-based camera extrinsic calibration methods. Among them, the fixed target uses the mapping points of the fixed target to align with the image detection points to achieve camera extrinsic calibration. Pattern-based camera extrinsic calibration methods, for example, use Apriltag (a black and white QR code-like pattern) or a checkerboard as a target pattern, detect the vertices of the target pattern, and then use the alignment of the two-dimensional coordinates and three-dimensional coordinates of the vertices to achieve camera extrinsic calibration.
[0060] However, fixed targets usually require a tedious mapping process and expensive mapping costs. In the pattern-based camera extrinsic calibration method, the target pattern contains a combination of multiple horizontal and vertical lines, and the pattern complexity is relatively high. When applied to a camera with large distortion, it is easy to fail to detect the vertex. Both of the above methods require determining the three-dimensional coordinates of the relevant points on the target. The calibration method is complicated and cumbersome, and there is a problem of low accuracy of the calibration results.
[0061] Based on this, an embodiment of the present application proposes a camera extrinsic parameter calibration method, which generates a bird's-eye view by using a distorted image obtained by photographing a calibration target with a camera to be calibrated and a reference extrinsic parameter, and then extracts closed figures in the bird's-eye view. Since the calibration target is a closed polygon, the figure is relatively simple and not prone to false detection. Therefore, the closed figure extraction is convenient and has high accuracy. At the same time, the cost value is calculated based on the distance between adjacent vertices and the distance between diagonal vertices in the closed figure, with less calculation and higher accuracy. Then, different bird's-eye views are generated by adjusting the reference roll angle and the reference pitch angle in the reference extrinsic parameter multiple times. Since the bird's-eye view generated each time has a different distortion effect, each adjusted cost value can reflect The de-distortion degree of the corresponding bird's-eye view, therefore, based on each adjusted cost value, the target roll angle is determined in each adjusted reference roll angle, and the target pitch angle is determined in each adjusted reference pitch angle, which can ensure that the bird's-eye view corresponding to the target roll angle and the target pitch angle is the bird's-eye view with the best de-distortion effect. This method of determining the target roll angle and the target pitch angle through multiple adjustments is beneficial to improving the accuracy of the target roll angle and the target pitch angle in the calibration results; in addition, the target yaw angle in the calibration result is determined based on the bird's-eye view with the best de-distortion effect, which is beneficial to eliminate the influence of the deviation of the target roll angle and the target pitch angle on the target yaw angle, and further improves the accuracy of the calibration result.
[0062] The camera extrinsic calibration method provided in the embodiment of the present application can be applied in the application environment as shown in FIG1 . Among them, the terminal 102 communicates with the camera 104 to be calibrated through a network. The data storage system can store the data that the terminal 102 needs to process. The data storage system can be integrated on the terminal 102, or it can be placed on the cloud or other network servers. This embodiment is illustrated by applying the method to the terminal 102. It can be understood that the method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. The terminal 102 obtains the distorted image obtained by the camera 104 to be calibrated shooting the calibration target, and generates a bird's-eye view based on the distorted image and the reference external parameters of the camera to be calibrated; the calibration target is a closed polygon; the closed figure in the bird's-eye view is extracted, and the cost value is calculated based on the distance between adjacent vertices and the distance between diagonal vertices in the closed figure; the reference roll angle and the reference pitch angle in the reference external parameters are adjusted, and the step of generating the bird's-eye view based on the distorted image and the reference external parameters of the camera to be calibrated is continued until the adjusted reference roll angle and the adjusted reference pitch angle meet the adjustment stop condition; based on each adjusted cost value, the target roll angle is determined in each adjusted reference roll angle, and the target pitch angle is determined in each adjusted reference pitch angle; based on the bird's-eye view corresponding to the target roll angle and the target pitch angle, the target yaw angle is determined. Among them, the terminal 102 can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, Internet of Things devices and portable wearable devices, and the Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car devices, projection devices, etc. Portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. Head-mounted devices may be virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, etc.
[0063] In an exemplary embodiment, as shown in FIG2 , a camera extrinsic calibration method is provided, which is described by taking the method applied to the terminal 102 in FIG1 as an example, and includes the following steps 202 to 208. Among them:
[0064] Step 202, obtaining a distorted image of a calibration target captured by the camera to be calibrated, and generating a bird's-eye view based on the distorted image and a reference external parameter of the camera to be calibrated; the calibration target is a closed polygon.
[0065] The camera to be calibrated refers to a camera to be calibrated with external parameters, which may be a surround camera installed in a vehicle, or a camera installed in a robotic arm or a mobile robot, etc. The camera to be calibrated may be a fisheye camera or other ordinary cameras.
[0066] The calibration target refers to the target pattern for camera calibration. The calibration target is a closed polygon, such as a triangle, quadrilateral, pentagon, etc.
[0067] A distorted image refers to an image with a certain degree of distortion. Since the camera to be calibrated has not been calibrated with external parameters, the distorted image obtained by the camera to be calibrated when shooting the calibration target has a certain degree of distortion.
[0068] The calibration target can be a fixed target or a non-fixed target. A non-fixed target means that the position of the calibration target in the calibration site is not fixed. For example, the calibration target can be randomly placed within the field of view of the camera to be calibrated in the calibration site. A fixed target means that the position of the calibration target in the calibration site is fixed.
[0069] The calibration target of the calibration site may include at least one. FIG3 is a schematic diagram of a distorted image in an embodiment. The calibration target includes four quadrilaterals, and the distorted image includes patterns corresponding to the four calibration targets.
[0070] In some embodiments, the calibration site may be a calibration room, for example, after the vehicle enters the calibration room, the camera to be calibrated in the vehicle takes a picture of the calibration target to obtain a distorted image. In other embodiments, the calibration site may also be a site in a dynamic calibration scene, for example, dynamic calibration is performed during vehicle driving, and the calibration target may be a crosswalk, etc.
[0071] The reference external parameters refer to the external parameters of the camera to be calibrated before calibration, which can come from the digital model of the vehicle on which the camera to be calibrated is installed. The reference external parameters include the reference roll angle, reference pitch angle, and reference yaw angle.
[0072] A bird's-eye view refers to an image of a target viewed from a certain point in a high place using a high viewpoint perspective method based on the perspective principle. For the first generated bird's-eye view, the terminal can generate it based on the distorted image and the reference external parameters. The bird's-eye view is the dedistorted image. In order to further improve the dedistortion effect of the bird's-eye view image, the external parameters can be calibrated. The calibrated external parameters can generate a bird's-eye view with better dedistortion effect.
[0073] Step 204 , extracting a closed figure in the bird's-eye view, and calculating a cost value based on the distances between adjacent vertices and the distances between diagonal vertices in the closed figure.
[0074] Among them, the closed figure is the figure corresponding to the calibration target in the bird's-eye view. Since the calibration target is a closed polygon, the bird's-eye view has a closed figure that corresponds to the calibration target one by one. Referring to Figure 3, the four closed quadrilaterals in the figure are the closed figures corresponding to the four calibration targets, and points A, B, C, and D are the vertices of one of the closed figures. The terminal can use a closed contour detection algorithm to extract closed figures from the bird's-eye view to obtain at least one closed image. For any closed figure, a corner point detection algorithm is used to detect the corner points of the closed figure in the bird's-eye view, thereby extracting multiple vertices in the closed figure. Since the calibration target figure is relatively simple, the bird's-eye view can often contain clearer corner points, which is conducive to improving the accuracy of corner point detection.
[0075] Adjacent vertices are a pair of vertices on a line, and diagonal vertices are a pair of vertices on a diagonal line. Referring to Figure 3, vertex A and vertex B are a pair of adjacent vertices, vertex B and vertex C are a pair of adjacent vertices, vertex C and vertex D are a pair of adjacent vertices, vertex D and vertex A are a pair of adjacent vertices, vertex A and vertex C are a pair of diagonal vertices, and vertex B and vertex D are a pair of diagonal vertices.
[0076] The distance between adjacent vertices can be determined by the coordinates of the adjacent vertices. For example:
[0077]
[0078]
[0079]
[0080]
[0081] The distance between diagonal vertices can be determined by the coordinates of the diagonal vertices. For example:
[0082]
[0083]
[0084] in, Represents the distance between adjacent vertices AB, Represents the distance between adjacent vertices BC, Represents the distance between adjacent vertices DC, represents the distance between adjacent vertices AD, represents the distance between diagonal vertices AC, represents the distance between diagonal vertices BD, , denote the horizontal and vertical coordinates of vertex A respectively, , denote the horizontal and vertical coordinates of vertex B, respectively. , denote the horizontal and vertical coordinates of vertex C respectively, , represent the horizontal and vertical coordinates of vertex D respectively.
[0085] The cost value is calculated by the distance between adjacent vertices and the distance between diagonal vertices. In some embodiments, a cost function can be constructed based on the distance between adjacent vertices and the distance between diagonal vertices, and the cost value is the solution of the cost function. For example, the cost value can be the deviation of the distance between each adjacent vertex and the sum of the deviations of the distance between each diagonal vertex. The cost value can be used to characterize the degree of distortion of the bird's-eye view.
[0086] Step 206, adjust the reference roll angle and the reference pitch angle in the reference extrinsic parameters, and continue to execute the step of generating a bird's-eye view based on the distorted image and the reference extrinsic parameters of the camera to be calibrated until the adjusted reference roll angle and the adjusted reference pitch angle meet the adjustment stop condition.
[0087] For different external parameters, bird's-eye views with different distortion degrees can be generated. The terminal can use a method of adjusting the reference roll angle and the reference pitch angle in the reference external parameters, so that a bird's-eye view with different distortion degrees can be generated each time the adjustment is made, and after each adjustment, the adjusted cost value can be recalculated by extracting the vertices in the adjusted bird's-eye view.
[0088] A feasible adjustment method for the reference external parameter can be: within a preset roll angle range, adjust the reference roll angle by the first step length each time, and within a preset pitch angle range, adjust the reference pitch angle by the second step length each time, until the adjusted reference pitch angle reaches the preset roll angle and the adjusted reference pitch angle reaches the preset pitch angle.
[0089] Another feasible adjustment method with reference to external parameters can be: traversing multiple preset roll angles in the preset roll angle set, and multiple pitch angles in the preset pitch angle set, and using the preset roll angle traversed each time as the roll angle after adjustment, and using the preset pitch angle traversed each time as the pitch angle after adjustment, until the data in the preset roll angle set and the preset pitch angle set are traversed.
[0090] Step 208, based on each adjusted cost value, determine a target roll angle in each adjusted reference roll angle, and determine a target pitch angle in each adjusted reference pitch angle; determine a target yaw angle based on a bird's-eye view corresponding to the target roll angle and the target pitch angle.
[0091] Among them, since the cost value can be used to characterize the degree of distortion removal of the bird's-eye view, the terminal can determine the bird's-eye view with the best degree of distortion removal according to each adjusted cost value, and use the adjusted reference roll angle for generating the bird's-eye view as the target roll angle, and use the adjusted reference pitch angle for generating the bird's-eye view as the target pitch angle. As shown in Figure 4, it is a schematic diagram of a bird's-eye view corresponding to a target roll angle and a target pitch angle in an embodiment. By comparing Figures 3 and 4, it can be seen that after multiple adjustments, the degree of distortion removal of the bird's-eye view is improved.
[0092] The terminal further determines the target yaw angle based on the bird's-eye view corresponding to the roll angle and the target pitch angle, and the calibration result includes the target roll angle, the target pitch angle and the target yaw angle. Since the bird's-eye view corresponding to the target roll angle and the target pitch angle is the bird's-eye view with the best degree of distortion removal, the target yaw angle determined based on the bird's-eye view has a high accuracy, thereby improving the accuracy of the calibration result.
[0093] In the above-mentioned camera extrinsic parameter calibration method, a bird's-eye view is generated by using the distorted image obtained by photographing the calibration target with the camera to be calibrated and the reference extrinsic parameter, and then the closed figure in the bird's-eye view is extracted. Since the calibration target is a closed polygon, the figure is relatively simple and not prone to false detection. Therefore, the closed figure extraction is convenient and has high accuracy. At the same time, the cost value is calculated based on the distance between adjacent vertices and the distance between diagonal vertices in the closed figure, which has less calculation amount and higher accuracy. Then, different bird's-eye views are generated by adjusting the reference roll angle and the reference pitch angle in the reference extrinsic parameter multiple times. Since the bird's-eye view generated each time has a different distortion effect, each adjusted cost value can reflect the corresponding bird's-eye view. Therefore, based on each adjusted cost value, the target roll angle is determined in each adjusted reference roll angle, and the target pitch angle is determined in each adjusted reference pitch angle, which can ensure that the bird's-eye view corresponding to the target roll angle and the target pitch angle is the bird's-eye view with the best de-distortion effect. This method of determining the target roll angle and the target pitch angle through multiple adjustments is conducive to improving the accuracy of the target roll angle and the target pitch angle in the calibration result; in addition, the target yaw angle in the calibration result is determined based on the bird's-eye view with the best de-distortion effect, which is conducive to eliminating the influence of the deviation of the target roll angle and the target pitch angle on the target yaw angle, and further improves the accuracy of the calibration result.
[0094] In an exemplary embodiment, a target yaw angle is determined based on a bird's-eye view corresponding to a target roll angle and a target pitch angle, including: extracting a target closed figure in the bird's-eye view corresponding to the target roll angle and the target pitch angle; and determining the target yaw angle based on coordinates of a plurality of target vertices in the target closed figure.
[0095] The terminal uses a closed contour detection algorithm to extract a closed figure from the bird's-eye view corresponding to the target roll angle and the target pitch angle to obtain a target closed figure. The target closed figure may include at least one, and for any target closed figure, a corner point detection algorithm is used to perform corner point detection on the target closed figure in the bird's-eye view, thereby extracting multiple target vertices.
[0096] Referring to FIG. 4 , since the bird's-eye view corresponding to the target roll angle and the target pitch angle is the bird's-eye view with the best degree of de-distortion, a relatively accurate target yaw angle can be determined based on the coordinates of multiple target vertices.
[0097] In this embodiment, the target closed figure in the bird's-eye view corresponding to the target roll angle and the target pitch angle is extracted, and then the target yaw angle is determined based on the coordinates of multiple target vertices in the target closed figure. Since the bird's-eye view is the one with the best degree of de-distortion, it is helpful to improve the accuracy of the target yaw angle.
[0098] In an exemplary embodiment, a target yaw angle is determined based on the coordinates of multiple target vertices in a target closed figure, including: calculating a reference coordinate of a reference point according to the coordinates of multiple target vertices in the target closed figure; controlling the movement of a camera to be calibrated and reacquiring the reference coordinate of the reference point; and determining the target yaw angle based on a change in the reference coordinate.
[0099] The reference point refers to a point in the target closed figure, and the reference point corresponds to a preset point in the calibration target. For example, the preset point can be the center point of the calibration target. The reference coordinates refer to the coordinates of the reference point in the bird's-eye view.
[0100] For example, if the preset point is the center point of the calibration target and the calibration target is a quadrilateral, then the reference coordinates of the reference point ( , ) can be expressed as:
[0101]
[0102]
[0103] The terminal controls the movement of the camera to be calibrated. If the camera to be calibrated is installed in a vehicle or a mobile robot, the terminal controls the vehicle or the mobile robot to realize the movement of the camera to be calibrated. The moving direction can be the direction approaching the calibration target or the direction away from the calibration target. As shown in FIG5, a schematic diagram of the steps for determining the target yaw angle in one embodiment. After the camera to be calibrated moves, the terminal re-shoots the calibration target to obtain a new distorted image, and generates a new bird's-eye view based on the new distorted image, the reference yaw angle, the target roll angle and the target pitch angle, and then obtains the reference coordinates of the reference point in the new bird's-eye view. Since the distorted image and the roll angle and pitch angle in the external parameters change after the camera to be calibrated moves, the generated new bird's-eye view changes, and the reference coordinates of the reference point also change. The terminal determines the target yaw angle by using the change in the reference coordinates before and after the camera to be calibrated moves. As shown in FIG6, a schematic diagram of the principle of determining the target yaw angle in one embodiment. In the figure, the reference coordinates of the reference point P change as the camera to be calibrated moves, and the change in the reference coordinates can be used to determine the target yaw angle.
[0104] The change amount of the reference coordinate includes the change amount of the horizontal coordinate and the change amount of the vertical coordinate. In some embodiments, the calculation formula of the target yaw angle yaw is as follows:
[0105]
[0106] in, Indicates the change in the horizontal coordinate before and after the camera to be calibrated moves. Indicates the change in the ordinate before and after the camera to be calibrated moves.
[0107] In this embodiment, the target yaw angle is determined based on the change in the reference coordinates of the reference point before and after the movement of the camera to be calibrated by controlling the movement of the camera to be calibrated. Since the calibrated target roll angle and target pitch angle are used when the camera to be calibrated moves, the calibration influence of the roll angle and pitch angle on the yaw angle is avoided, which is beneficial to improving the accuracy of the target yaw angle.
[0108] In an exemplary embodiment, based on each adjusted cost value, a target roll angle is determined in each adjusted reference roll angle, and a target pitch angle is determined in each adjusted reference pitch angle, including: determining a target cost value in each adjusted cost value; using the adjusted reference roll angle corresponding to the target cost value in each adjusted reference roll angle as the target roll angle, and using the adjusted reference pitch angle corresponding to the target cost value in each adjusted reference pitch angle as the target pitch angle.
[0109] The target cost value is any cost value obtained after the adjustments, for example, it can be the minimum cost value among the cost values obtained after the adjustments, or a cost value within a preset range.
[0110] Since the target cost value is calculated after a certain adjustment, the terminal can use the adjusted reference roll angle and adjusted pitch angle corresponding to the adjustment as the calibration result, that is, the adjusted reference roll angle corresponding to the target cost value in each adjusted reference roll angle is used as the target roll angle, and the adjusted reference pitch angle corresponding to the target cost value in each adjusted reference pitch angle is used as the target pitch angle.
[0111] In this embodiment, a target cost value is determined from the cost values obtained after each adjustment. The target cost value corresponds to the bird's-eye view with the best degree of de-distortion in each adjusted bird's-eye view. The adjusted reference roll angle and the adjusted reference pitch angle corresponding to the target cost value are used as calibration results, which is beneficial to improving the accuracy of the calibration results.
[0112] In an exemplary embodiment, determining the target cost value among the adjusted cost values includes: taking the minimum cost value among the adjusted cost values as the target cost value.
[0113] The smaller the cost value, the higher the degree of distortion removal of the corresponding bird's-eye view, and the larger the cost value, the lower the degree of distortion removal of the corresponding bird's-eye view. The terminal uses the minimum cost value among the adjusted cost values as the target cost value, which is conducive to determining the bird's-eye view with the best degree of distortion removal.
[0114] In this embodiment, by taking the minimum cost value among the adjusted cost values as the target cost value, the higher the de-distortion degree of the bird's-eye view corresponding to the target cost value, the more conducive it is to determine the bird's-eye view with the best de-distortion degree.
[0115] In an exemplary embodiment, the calibration target is a closed equilateral polygon; the cost value is calculated based on the distance between adjacent vertices and the distance between diagonal vertices in the closed figure, including: determining the distance between adjacent vertices as the length of each side in the closed figure, and determining the distance between diagonal vertices as the length of each diagonal line in the closed figure; determining the cost value based on the difference between the side lengths and the difference between the diagonal lengths.
[0116] Since a closed equilateral polygon has the characteristics that the lengths of its sides are equal and its diagonals are equal, a method for calculating the cost value can be determined based on this characteristic.
[0117] Since the closed figure is a pattern corresponding to each calibration target in the bird's-eye view, the closed figure has a certain degree of distortion compared to the calibration target. The distance between adjacent vertices is the length of each side of the closed figure, and the distance between diagonal vertices is the length of each diagonal of the closed figure. The terminal subtracts the lengths of each side from each other to obtain multiple side length differences, and subtracts the lengths of each diagonal from each other to obtain multiple diagonal length differences. The cost value can be the sum of the absolute values of each of the multiple side length differences, plus the sum of the absolute values of each of the multiple diagonal length differences. The cost value can also be the sum of the squares of each of the multiple side length differences, plus the sum of the squares of each of the multiple diagonal length differences. For example, the cost value J can be expressed as follows:
[0118]
[0119] In some embodiments, if there are multiple calibration targets, the cost value may be the sum of the cost values corresponding to the multiple closed figures, or a weighted average value.
[0120] In this embodiment, by taking the distance between adjacent vertices as the length of each side of the closed figure, taking the diagonal vertices as the length of each diagonal line in the closed figure, and combining the characteristics of the closed equilateral polygon, subtracting the lengths of each side from each other, and subtracting the lengths of each diagonal line from each other, the smaller the sum of the differences is, the closer the closed figure is to a closed equilateral polygon, and the higher the degree of distortion of the corresponding bird's-eye view is. This cost value calculation method is conducive to determining the bird's-eye view with the best degree of distortion, thereby improving the accuracy of the calibration result.
[0121] To explain the camera extrinsic calibration method and its effect in detail, a most detailed embodiment is given below:
[0122] The external parameter calibration of the camera to be calibrated can be applied to static calibration scenarios and dynamic calibration scenarios. The static calibration scenario is used as an example for explanation. The camera to be calibrated can be installed in a vehicle, and the calibration target can be a non-fixed target, for example, a closed equilateral polygon randomly placed within the camera field of view of the calibration site, for example, an equilateral triangle or a square.
[0123] FIG. 7 is a schematic diagram of the overall process of a camera extrinsic calibration method in one embodiment. After the vehicle arrives at the calibration area, a distorted image of the calibration target taken by the camera to be calibrated is obtained, and a bird's-eye view is generated based on the distorted image and the reference extrinsic parameters of the camera to be calibrated; a closed figure in the bird's-eye view is extracted, and a cost value is calculated based on the distance between adjacent vertices and the distance between diagonal vertices in the closed figure; a reference roll angle and a reference pitch angle in the reference extrinsic parameters are adjusted, and the step of generating a bird's-eye view based on the distorted image and the reference extrinsic parameters of the camera to be calibrated is continued until the adjusted reference roll angle and the adjusted reference pitch angle meet the adjustment stop condition, for example, the adjustment stop condition is within a preset roll angle range, and the traversal within a preset pitch angle range is completed; based on each adjusted cost value, a target roll angle is determined in each adjusted reference roll angle, and a target pitch angle is determined in each adjusted reference pitch angle; a target closed figure in the bird's-eye view corresponding to the target roll angle and the target pitch angle is extracted, the reference coordinates of the reference point are calculated, the camera to be calibrated is controlled to move, and the reference coordinates of the reference point are re-obtained, and the target yaw angle is determined based on the change in the reference coordinates.
[0124] Among them, the terminal can generate a bird's-eye view based on the distorted image and the reference external parameters, internal parameters and distortion parameters of the camera to be calibrated. Since the bird's-eye view stores a certain distortion, the closed figure extracted from the bird's-eye view is an irregular quadrilateral. The terminal can convert the bird's-eye view into a binary image, perform closed contour detection on the binary image, and extract the vertices of the closed figure, for example, vertex A , Vertex B , Vertex C , vertex D The cost value is then calculated based on the distance between adjacent vertices and the distance between diagonal vertices. The cost function can be constructed by utilizing the characteristics of a square, based on the fact that the four sides are equal in length and the diagonals are also equal.
[0125] As shown in FIG8 , a schematic diagram of a method for adjusting reference external parameters in an embodiment is shown. In the figure, a bird's-eye view is generated based on reference external parameters, distorted images, internal parameters of the camera to be calibrated, and distortion coefficients. The reference external parameters include reference roll angle, reference pitch angle, and reference yaw angle. Closed figure extraction and corner point extraction are performed on the bird's-eye view to construct a cost function. The reference roll angle and reference pitch angle can be expanded by an angle range, such as 5 degrees, respectively. The reference roll angle and reference pitch angle are adjusted multiple times with a preset step size, such as 0.1 degrees, until the angle range is traversed. A cycle is completed by converting the distorted graph into a bird's-eye view, then into a binary image, extracting contours, extracting corner points, and calculating cost values. After multiple adjustments are completed, the adjusted reference roll angle corresponding to the minimum cost value is found as the target roll angle roll, and the adjusted reference pitch angle corresponding to the minimum cost value is found as the target pitch angle pitch, which is the roll angle and pitch angle in the optimal calibration result.
[0126] After obtaining the target roll angle roll and target pitch angle pitch, the target roll angle roll and target pitch angle pitch, reference yaw angle, internal parameter and distortion coefficient are converted into a bird's-eye view, then into a binary image, contour extraction, and corner point extraction to obtain the positions of four corner points. The center point of the four corner point positions is used as the position of point P ( ). Move the vehicle forward, and once again convert the distortion graph into a bird's-eye view, then into a binary image, extract the contour, and extract the corner points according to the target roll angle roll and target pitch angle pitch, reference yaw angle, internal parameters, and distortion coefficients, and obtain the new positions of the four corner points. Use the center point of the four corner points to calculate the new center point P ( ), calculate the target yaw angle yaw according to the calculation formula of the target yaw angle. The obtained target yaw angle is the yaw angle in the optimal calibration result. The target roll angle roll, target pitch angle pitch and target yaw angle are the external parameters obtained by calibration, which are used to correct the installation deviation of the camera to be calibrated.
[0127] The camera extrinsic calibration method generates a bird's-eye view by using the distorted image obtained by the camera to be calibrated shooting the calibration target and the reference extrinsic parameters, and then takes the vertices in the bird's-eye view. Since the calibration target is a closed polygon, the figure is relatively simple and it is not easy to cause false detection. Therefore, vertex extraction is convenient and has high accuracy. At the same time, the cost value is calculated based on the distance between adjacent vertices and the distance between diagonal vertices, which has less calculation amount and higher accuracy. Then, different bird's-eye views are generated by adjusting the reference roll angle and the reference pitch angle in the reference extrinsic parameters for multiple times. Since each generated bird's-eye view has a different distortion effect, each adjusted cost value can reflect the de-distortion degree of the corresponding bird's-eye view. Therefore, based on each adjusted cost value, the target roll angle is determined in each adjusted reference roll angle, and the target pitch angle is determined in each adjusted reference pitch angle, which can ensure that the bird's-eye view corresponding to the target roll angle and the target pitch angle is the bird's-eye view with the best de-distortion effect. This method of determining the target roll angle and the target pitch angle through multiple adjustments is conducive to improving the accuracy of the target roll angle and the target pitch angle in the calibration result; in addition, the target yaw angle in the calibration result is determined based on the bird's-eye view with the best de-distortion effect, which is conducive to eliminating the influence of the deviation of the target roll angle and the target pitch angle on the target yaw angle, further improving the accuracy of the calibration result. At the same time, the placement position of the calibration target is flexible, and the calibration target only needs to be placed within the detectable range of the camera, which can improve work efficiency.
[0128] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.
[0129] Based on the same inventive concept, the embodiment of the present application also provides a camera extrinsic calibration device for implementing the camera extrinsic calibration method involved above. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above method, so the specific limitations in one or more camera extrinsic calibration device embodiments provided below can refer to the limitations of the camera extrinsic calibration method above, and will not be repeated here.
[0130] In an exemplary embodiment, as shown in FIG9 , a camera extrinsic calibration device 100 is provided, comprising: an acquisition module 120, an extraction module 140, an adjustment module 160 and a determination module 180, wherein:
[0131] The acquisition module 120 is used to acquire a distorted image obtained by the camera to be calibrated shooting the calibration target, and generate a bird's-eye view based on the distorted image and the reference external parameters of the camera to be calibrated; the calibration target is a closed polygon;
[0132] An extraction module 140, for extracting vertices in the bird's-eye view and calculating cost values based on distances between adjacent vertices and distances between diagonal vertices;
[0133] An adjustment module 160, configured to adjust a reference roll angle and a reference pitch angle in the reference extrinsic parameters, and continue to execute the step of generating a bird's-eye view based on the distorted image and the reference extrinsic parameters of the camera to be calibrated, until the adjusted reference roll angle and the adjusted reference pitch angle meet an adjustment stop condition;
[0134] The determination module 180 is used to determine the target roll angle in each adjusted reference roll angle based on each adjusted cost value, and determine the target pitch angle in each adjusted reference pitch angle; and determine the target yaw angle based on the bird's-eye view corresponding to the target roll angle and the target pitch angle.
[0135] The above-mentioned camera extrinsic parameter calibration device generates a bird's-eye view through the distorted image obtained by shooting the calibration target with the camera to be calibrated and the reference extrinsic parameter, and then takes the vertices in the bird's-eye view. Since the calibration target is a closed polygon, the figure is relatively simple and it is not easy to cause false detection. Therefore, the vertex extraction is convenient and the accuracy is high. At the same time, the cost value is calculated based on the distance between adjacent vertices and the distance between diagonal vertices, which has less calculation amount and higher accuracy. Then, different bird's-eye views are generated by adjusting the reference roll angle and the reference pitch angle in the reference extrinsic parameter for multiple times. Since the bird's-eye view generated each time has a different distortion effect, each adjusted cost value can reflect the de-distortion degree of the corresponding bird's-eye view. Therefore, based on each adjusted cost value, the target roll angle is determined in each adjusted reference roll angle, and the target pitch angle is determined in each adjusted reference pitch angle, which can ensure that the bird's-eye view corresponding to the target roll angle and the target pitch angle is the bird's-eye view with the best de-distortion effect. This method of determining the target roll angle and the target pitch angle through multiple adjustments is conducive to improving the accuracy of the target roll angle and the target pitch angle in the calibration result; in addition, the target yaw angle in the calibration result is determined based on the bird's-eye view with the best de-distortion effect, which is conducive to eliminating the influence of the deviation of the target roll angle and the target pitch angle on the target yaw angle, and further improves the accuracy of the calibration result.
[0136] In one embodiment, the target yaw angle is determined based on the bird's-eye view corresponding to the target roll angle and the target pitch angle, and the determination module 180 is further used to: extract the target closed figure in the bird's-eye view corresponding to the target roll angle and the target pitch angle; and determine the target yaw angle based on the coordinates of multiple target vertices in the target closed figure.
[0137] In one embodiment, the target yaw angle is determined based on the coordinates of multiple target vertices in the target closed figure, and the determination module 180 is also used to: calculate the reference coordinates of the reference point according to the coordinates of multiple target vertices in the target closed figure; control the movement of the camera to be calibrated and re-acquire the reference coordinates of the reference point; and determine the target yaw angle based on the change in the reference coordinates.
[0138] In one embodiment, based on each adjusted cost value, a target roll angle is determined in each adjusted reference roll angle, and a target pitch angle is determined in each adjusted reference pitch angle. The determination module 180 is also used to: determine the target cost value in each adjusted cost value; use the adjusted reference roll angle corresponding to the target cost value in each adjusted reference roll angle as the target roll angle, and use the adjusted reference pitch angle corresponding to the target cost value in each adjusted reference pitch angle as the target pitch angle.
[0139] In one embodiment, the target cost value is determined from the adjusted cost values, and the determination module 180 is further configured to: use the minimum cost value from the adjusted cost values as the target cost value.
[0140] In one embodiment, the calibration target is a closed equilateral polygon; the cost value is calculated based on the distance between adjacent vertices and the distance between diagonal vertices in the closed figure, and the extraction module 140 is also used to: determine the distance between adjacent vertices as the length of each side of the closed figure, and determine the distance between diagonal vertices as the length of each diagonal line of the closed figure; determine the cost value based on the difference between the lengths of each side and the difference between the lengths of each diagonal line.
[0141] Each module in the above-mentioned camera extrinsic calibration device can be implemented in whole or in part by software, hardware, or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.
[0142] In an exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be shown in FIG10. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit, and an input device. The processor, the memory, and the input / output interface are connected via a system bus, and the communication interface, the display unit, and the input device are connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner may be implemented through WIFI, a mobile cellular network, near field communication (NFC) or other technologies. When the computer program is executed by the processor, a camera external parameter calibration method is implemented. The display unit of the computer device is used to form a visually visible picture, which may be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.
[0143] Those skilled in the art will understand that the structure shown in FIG. 10 is merely a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.
[0144] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:
[0145] A distorted image of a calibration target taken by the camera to be calibrated is obtained, and a bird's-eye view is generated based on the distorted image and the reference extrinsic parameters of the camera to be calibrated; the calibration target is a closed polygon; vertices in the bird's-eye view are extracted, and cost values are calculated based on the distances between adjacent vertices and the distances between diagonal vertices; a reference roll angle and a reference pitch angle in the reference extrinsic parameters are adjusted, and the step of generating a bird's-eye view based on the distorted image and the reference extrinsic parameters of the camera to be calibrated is continued until the adjusted reference roll angle and the adjusted reference pitch angle meet the adjustment stop condition; based on each adjusted cost value, a target roll angle is determined in each adjusted reference roll angle, and a target pitch angle is determined in each adjusted reference pitch angle; based on the bird's-eye view corresponding to the target roll angle and the target pitch angle, a target yaw angle is determined.
[0146] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:
[0147] A target closed figure in the bird's-eye view corresponding to the target roll angle and the target pitch angle is extracted; and a target yaw angle is determined based on the coordinates of a plurality of target vertices in the target closed figure.
[0148] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:
[0149] The reference coordinates of the reference point are calculated according to the coordinates of multiple target vertices in the target closed figure; the movement of the camera to be calibrated is controlled, and the reference coordinates of the reference point are re-acquired; and the target yaw angle is determined based on the change in the reference coordinates.
[0150] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:
[0151] A target cost value is determined from each adjusted cost value; an adjusted reference roll angle corresponding to the target cost value in each adjusted reference roll angle is used as the target roll angle, and an adjusted reference pitch angle corresponding to the target cost value in each adjusted reference pitch angle is used as the target pitch angle.
[0152] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:
[0153] The minimum cost value among the cost values obtained after the adjustments is taken as the target cost value.
[0154] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:
[0155] The target is calibrated as a closed equilateral polygon; the distance between adjacent vertices in the closed figure is determined as the length of each side of the closed figure, and the distance between diagonal vertices is determined as the length of each diagonal of the closed figure; the cost value is determined based on the difference between the lengths of each side and the difference between the lengths of each diagonal.
[0156] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:
[0157] A distorted image of a calibration target taken by the camera to be calibrated is obtained, and a bird's-eye view is generated based on the distorted image and the reference extrinsic parameters of the camera to be calibrated; the calibration target is a closed polygon; vertices in the bird's-eye view are extracted, and cost values are calculated based on the distances between adjacent vertices and the distances between diagonal vertices; a reference roll angle and a reference pitch angle in the reference extrinsic parameters are adjusted, and the step of generating a bird's-eye view based on the distorted image and the reference extrinsic parameters of the camera to be calibrated is continued until the adjusted reference roll angle and the adjusted reference pitch angle meet the adjustment stop condition; based on each adjusted cost value, a target roll angle is determined in each adjusted reference roll angle, and a target pitch angle is determined in each adjusted reference pitch angle; based on the bird's-eye view corresponding to the target roll angle and the target pitch angle, a target yaw angle is determined.
[0158] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:
[0159] A target closed figure in the bird's-eye view corresponding to the target roll angle and the target pitch angle is extracted; and a target yaw angle is determined based on the coordinates of a plurality of target vertices in the target closed figure.
[0160] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:
[0161] The reference coordinates of the reference point are calculated according to the coordinates of multiple target vertices in the target closed figure; the movement of the camera to be calibrated is controlled, and the reference coordinates of the reference point are re-acquired; and the target yaw angle is determined based on the change in the reference coordinates.
[0162] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:
[0163] A target cost value is determined from each adjusted cost value; an adjusted reference roll angle corresponding to the target cost value in each adjusted reference roll angle is used as the target roll angle, and an adjusted reference pitch angle corresponding to the target cost value in each adjusted reference pitch angle is used as the target pitch angle.
[0164] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:
[0165] The minimum cost value among the cost values obtained after the adjustments is taken as the target cost value.
[0166] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:
[0167] The target is calibrated as a closed equilateral polygon; the distance between adjacent vertices is determined as the length of each side of the closed figure, and the distance between diagonal vertices is determined as the length of each diagonal of the closed figure; the cost value is determined based on the difference between the lengths of each side and the difference between the lengths of each diagonal.
[0168] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:
[0169] A distorted image of a calibration target taken by the camera to be calibrated is obtained, and a bird's-eye view is generated based on the distorted image and the reference extrinsic parameters of the camera to be calibrated; the calibration target is a closed polygon; vertices in the bird's-eye view are extracted, and cost values are calculated based on the distances between adjacent vertices and the distances between diagonal vertices; a reference roll angle and a reference pitch angle in the reference extrinsic parameters are adjusted, and the step of generating a bird's-eye view based on the distorted image and the reference extrinsic parameters of the camera to be calibrated is continued until the adjusted reference roll angle and the adjusted reference pitch angle meet the adjustment stop condition; based on each adjusted cost value, a target roll angle is determined in each adjusted reference roll angle, and a target pitch angle is determined in each adjusted reference pitch angle; based on the bird's-eye view corresponding to the target roll angle and the target pitch angle, a target yaw angle is determined.
[0170] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:
[0171] A target closed figure in the bird's-eye view corresponding to the target roll angle and the target pitch angle is extracted; and a target yaw angle is determined based on the coordinates of a plurality of target vertices in the target closed figure.
[0172] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:
[0173] According to the coordinates of multiple target vertices, the reference coordinates of the reference point are calculated; the movement of the camera to be calibrated is controlled, and the reference coordinates of the reference point are re-acquired; based on the change in the reference coordinates, the target yaw angle is determined.
[0174] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:
[0175] A target cost value is determined from each adjusted cost value; an adjusted reference roll angle corresponding to the target cost value in each adjusted reference roll angle is used as the target roll angle, and an adjusted reference pitch angle corresponding to the target cost value in each adjusted reference pitch angle is used as the target pitch angle.
[0176] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:
[0177] The minimum cost value among the cost values obtained after the adjustments is taken as the target cost value.
[0178] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:
[0179] The target is calibrated as a closed equilateral polygon; the distance between adjacent vertices is determined as the length of each side of the closed figure, and the distance between diagonal vertices is determined as the length of each diagonal of the closed figure; the cost value is determined based on the difference between the lengths of each side and the difference between the lengths of each diagonal.
[0180] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.
[0181] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.
[0182] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.
[0183] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.
Claims
1. A camera extrinsic calibration method, characterized in that: The method comprises: Obtain a distorted image of a calibration target captured by the camera to be calibrated, and generate a bird's-eye view based on the distorted image and a reference external parameter of the camera to be calibrated; the calibration target is a closed polygon; Extracting a closed figure in the bird's-eye view, and calculating a cost value based on distances between adjacent vertices and distances between diagonal vertices in the closed figure; Adjusting a reference roll angle and a reference pitch angle in the reference extrinsic parameters, and continuing to perform the step of generating a bird's-eye view based on the distorted image and the reference extrinsic parameters of the camera to be calibrated, until the adjusted reference roll angle and the adjusted reference pitch angle meet an adjustment stop condition; Based on each adjusted cost value, a target roll angle is determined in each adjusted reference roll angle, and a target pitch angle is determined in each adjusted reference pitch angle; and based on a bird's-eye view corresponding to the target roll angle and the target pitch angle, a target yaw angle is determined.
2. The method according to claim 1, characterized in that The determining of the target yaw angle based on the bird's-eye view corresponding to the target roll angle and the target pitch angle comprises: Extracting the target closed figure in the bird's-eye view corresponding to the target roll angle and the target pitch angle; A target yaw angle is determined based on coordinates of a plurality of target vertices in the target closed figure.
3. The method according to claim 2, characterized in that The step of determining the target yaw angle based on the coordinates of a plurality of target vertices in the target closed figure comprises: Calculating the reference coordinates of the reference point according to the coordinates of a plurality of target vertices in the target closed figure; Controlling the movement of the camera to be calibrated and reacquiring the reference coordinates of the reference point; A target yaw angle is determined based on the amount of change in the reference coordinates.
4. The method according to claim 1, characterized in that: The step of determining a target roll angle from each adjusted reference roll angle based on each adjusted cost value, and determining a target pitch angle from each adjusted reference pitch angle, comprises: Determining a target cost value among the adjusted cost values; The adjusted reference roll angle corresponding to the target cost value in each adjusted reference roll angle is used as the target roll angle, and the adjusted reference pitch angle corresponding to the target cost value in each adjusted reference pitch angle is used as the target pitch angle.
5. The method according to claim 4, characterized in that Determining the target cost value from the cost values obtained after the adjustments includes: The minimum cost value among the cost values obtained after the adjustments is taken as the target cost value.
6. The method according to claim 1, characterized in that The calibration target is a closed equilateral polygon; the cost value is calculated based on the distance between adjacent vertices and the distance between diagonal vertices in the closed figure, including: The distance between adjacent vertices is determined as the length of each side of the closed figure, and the distance between diagonal vertices is determined as the length of each diagonal line of the closed figure; The cost value is determined based on the side length differences between the side lengths and the diagonal length differences between the diagonal lengths.
7. A camera extrinsic calibration device, characterized in that: The device comprises: An acquisition module is used to acquire a distorted image obtained by the camera to be calibrated shooting a calibration target, and generate a bird's-eye view based on the distorted image and a reference external parameter of the camera to be calibrated; the calibration target is a closed polygon; An extraction module, configured to extract a closed figure in the bird's-eye view and calculate a cost value based on distances between adjacent vertices and distances between diagonal vertices in the closed figure; an adjustment module, configured to adjust a reference roll angle and a reference pitch angle in the reference extrinsic parameter, and continue to execute the step of generating a bird's-eye view based on the distorted image and the reference extrinsic parameter of the camera to be calibrated, until the adjusted reference roll angle and the adjusted reference pitch angle meet an adjustment stop condition; A determination module is used to determine a target roll angle in each adjusted reference roll angle based on each adjusted cost value, and to determine a target pitch angle in each adjusted reference pitch angle; and to determine a target yaw angle based on a bird's-eye view corresponding to the target roll angle and the target pitch angle.
8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.
10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.
