A method and system for calibrating the mounting posture angle of a vehicle-mounted camera

Abstract

The application provides a kind of vehicle camera installation posture angle calibration method and system, the method comprises: after installing calibration board in a predetermined manner, calibration board image is collected by vehicle camera;Based on the pixel coordinates of the corner point in calibration board image and the pixel coordinates of the optical center of vehicle camera, the yaw angle, the pitch angle and the roll angle of vehicle camera are calculated respectively, and the installation posture angle of vehicle camera is calibrated based on the yaw angle, the pitch angle and the roll angle.The scheme can effectively improve the accuracy and reliability of vehicle camera installation posture angle calibration, and avoid the uncertainty of the results caused by the existing solution method.

Description

Technical Field

[0001] This invention belongs to the field of image processing, and in particular relates to a method and system for calibrating the installation attitude angle of a vehicle-mounted camera. Background Technology

[0002] With the rapid development of ADAS (Advanced Driver Assistance Systems) technology, more and more vehicles are equipped with this system to enhance the driving experience. As an important component of ADAS, the perception calibration results of onboard cameras directly affect the vehicle control behavior of the advanced driver assistance system. Therefore, calibrating the installation attitude angle of onboard cameras has a significant impact on the accuracy of vehicle perception and recognition.

[0003] Camera mounting attitude angle calibration refers to the transformation relationship between the camera coordinate system at the actual mounting position (referred to as the camera mounting coordinate system) and the camera coordinate system at the virtual horizontal position (referred to as the camera ideal coordinate system). The general calibration method for vehicle-mounted camera mounting attitude angles is as follows: First, a calibration board is fixed in the calibration field. The vehicle is parked in a suitable position directly in front of the calibration board, and then the vehicle-mounted camera is used to photograph the calibration board. By identifying the 2D information of the calibration board, the pixel position representation of the calibration board in the camera mounting coordinate system is obtained. Then, the 3D information of the calibration board relative to the camera is measured to obtain the representation of the calibration board in the camera ideal coordinate system. Based on the determined 2D-3D matching pair information of the calibration board, the camera attitude angle is calculated using PnP (Perspective-n-Point). However, since the solution obtained using PnP is not a closed-form solution, there is a risk of getting trapped in local optima, which raises questions about the accuracy and reliability of the camera attitude angle calibration. Summary of the Invention

[0004] In view of this, embodiments of the present invention provide a method and system for calibrating the installation attitude angle of a vehicle-mounted camera, which is used to solve the problem of deviation in the accuracy and reliability of the attitude angle calculation results of a vehicle-mounted camera.

[0005] In a first aspect of the present invention, a method for calibrating the mounting attitude angle of a vehicle-mounted camera is provided, comprising:

[0006] After the calibration plate is installed in a predetermined manner, images of the calibration plate are captured by the vehicle-mounted camera.

[0007] Based on the corner pixel coordinates in the calibration board image and the optical center pixel coordinates of the vehicle camera, the yaw angle, pitch angle and roll angle of the vehicle camera are calculated respectively, and the installation attitude angle of the vehicle camera is calibrated based on the yaw angle, pitch angle and roll angle.

[0008] In a second aspect of the present invention, a vehicle-mounted camera mounting attitude angle calibration system is provided, comprising:

[0009] The image acquisition module is used to acquire images of the calibration board via an onboard camera after the calibration board is installed in a predetermined manner.

[0010] The attitude angle calculation module is used to calculate the yaw angle, pitch angle and roll angle of the vehicle camera based on the corner pixel coordinates in the calibration board image and the optical center pixel coordinates of the vehicle camera, and to calibrate the installation attitude angle of the vehicle camera based on the yaw angle, pitch angle and roll angle.

[0011] In a third aspect of the present invention, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable by the processor, wherein the processor executes the computer program to implement the steps of the method as described in the first aspect of the present invention.

[0012] In a fourth aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the method provided in the first aspect of the present invention.

[0013] In this embodiment of the invention, the camera mounting attitude angle is calculated directly using the identified 2D information of the calibration board based on the calibration installed in a predetermined manner. This not only simplifies the process but also ensures calibration accuracy. Furthermore, it avoids the uncertainties in existing attitude angle calculation processes, effectively improving the accuracy and reliability of vehicle-mounted camera attitude angle calibration. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 This is a flowchart illustrating a method for calibrating the mounting attitude angle of a vehicle-mounted camera according to an embodiment of the present invention.

[0016] Figure 2 This is a schematic diagram of a calibration plate provided in one embodiment of the present invention;

[0017] Figure 3 This is a schematic diagram of another plane of a calibration plate provided in one embodiment of the present invention;

[0018] Figure 4 This is a schematic diagram of the calibration plate installation according to an embodiment of the present invention;

[0019] Figure 5This is a schematic diagram illustrating the principle of calculating the attitude angle of a vehicle-mounted camera according to an embodiment of the present invention.

[0020] Figure 6 This is a schematic diagram of a vehicle-mounted camera mounting attitude angle calibration system according to an embodiment of the present invention;

[0021] Figure 7 This is a schematic diagram of the structure of an electronic device provided in one embodiment of the present invention. Detailed Implementation

[0022] To make the objectives, features, and advantages of this invention more apparent and understandable, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described below are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0023] It should be understood that the terms "comprising" and other similar expressions in the specification, claims, and accompanying drawings of this invention are intended to cover a non-exclusive inclusion, such as a process, method, system, or apparatus that includes a series of steps or units and is not limited to the listed steps or units. Furthermore, "first" and "second" are used to distinguish different objects and are not intended to describe a specific order.

[0024] Please see Figure 1 The present invention provides a flowchart illustrating a method for calibrating the mounting attitude angle of a vehicle-mounted camera, comprising:

[0025] S101. After installing the calibration plate in a predetermined manner, the image of the calibration plate is captured by the vehicle-mounted camera.

[0026] The calibration board consists of three parts, all of which are chessboard grids. The center of the middle chessboard grid is equidistant from the left and right chessboard grids, and the relative positions of the left, middle, and right chessboard grids are horizontal. The left and right chessboard grids are both obliquely symmetrical 2×n chessboard grids, where n≥2, and the middle chessboard grid is an obliquely symmetrical 2×2 chessboard grid.

[0027] Calibration plate pattern as follows Figure 2 and Figure 3 As shown, the calibration plate is divided into three parts: left, middle, and right. Figure 2 The central grid of the winning bid board can be square. Figure 3 The chessboard grid in the middle of the winning bid board can be a quarter circle.

[0028] Specifically, during the installation of the calibration plate, such as Figure 4As shown, the distance from the center of the calibration board to the ground is equal to the height of the vehicle-mounted camera;

[0029] The vehicle body is perpendicular to the plane of the calibration plate, and the center of the calibration plate is exactly on the forward coordinate axis of the ideal coordinate system of the camera (i.e., the virtual horizontal position of the camera coordinate system);

[0030] Adjust the horizontal distance between the calibration board and the camera so that the calibration board is located in the field of view (FOV) of the camera, and the horizontal distance between the calibration board and the camera is greater than the predetermined value L;

[0031] Where L = 2 * l * f / H, l is the side length of the calibration board grid, f is the focal length of the camera, and H is the vertical length of the camera's light-sensitive area.

[0032] S102. Based on the corner pixel coordinates in the calibration board image and the optical center pixel coordinates of the vehicle camera, calculate the yaw angle, pitch angle and roll angle of the vehicle camera respectively, and calibrate the installation attitude angle of the vehicle camera based on the yaw angle, pitch angle and roll angle.

[0033] The corner points are the intersections of the chessboard squares in the indicator board, such as... Figure 2 or Figure 3 As shown, the corner points are the points where the black squares intersect. Pixel coordinates refer to the coordinates of the corner pixel in the image; the optical center pixel coordinates of the vehicle-mounted camera can be obtained from the camera's intrinsic parameters.

[0034] Ideally, if the camera is installed without deviation, the camera's installation coordinate system coincides with its ideal coordinate system, all checkerboard corner points should be on the horizontal line, and the optical center 0 should also coincide with P. However, due to deviations in the installation angle, the positional deviations of the corner points can be used to calculate the installation attitude angle.

[0035] Specifically, such as Figure 5 As shown, let the pixel coordinates corresponding to the corner points on the calibration board detected by the vehicle-mounted camera be as follows: P(X P ,Y P ), And the coordinates of the optical center pixel are O(X) O ,Y O Then, calculate the yaw, pitch, and roll angles respectively according to the formulas:

[0036]

[0037]

[0038]

[0039] In the formula, f is the focal length of the camera, pix is ​​the pixel size of the photosensitive area, and atan is the arctangent function;

[0040] Among them, PL1 and PL2 are the left corner points of the chessboard grid in the calibration board, P is the middle corner point of the chessboard grid in the calibration board, and PR1 and PR2 are the right corner points of the chessboard grid in the calibration board.

[0041] In this embodiment, the camera mounting attitude angle is calculated using the known 2D information of the calibration board installed in a predetermined manner, based on the calibration board. This can ensure calibration accuracy while avoiding the uncertainties of existing calculation methods, thus guaranteeing the stability and reliability of the calculation. Furthermore, the implementation process is simple.

[0042] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0043] Figure 6 This is a schematic diagram of a vehicle-mounted camera mounting attitude angle calibration system provided in an embodiment of the present invention. The system includes:

[0044] The image acquisition module 610 is used to acquire images of the calibration board via an on-board camera after the calibration board is installed in a predetermined manner.

[0045] The calibration board consists of three parts, all of which are chessboard grids. The center of the middle chessboard grid is equidistant from the left and right chessboard grids, and the left, middle, and right chessboard grids are horizontal in relative position.

[0046] The left and right parts of the chessboard are both obliquely symmetrical 2×n chessboards, where n≥2, and the middle part of the chessboard is an obliquely symmetrical 2×2 chessboard.

[0047] Specifically, the installation of the calibration plate in a predetermined manner includes:

[0048] The distance from the center of the calibration board to the ground is equal to the height of the vehicle-mounted camera;

[0049] The longitudinal direction of the vehicle body is perpendicular to the plane of the calibration plate, and the center of the calibration plate is exactly on the forward coordinate axis of the ideal coordinate system of the camera;

[0050] Adjust the horizontal distance between the calibration board and the camera so that the calibration board is located in the field of view (FOV) of the camera, and the horizontal distance between the calibration board and the camera is greater than the predetermined value L;

[0051] Where L = 2 * l * f / H, l is the side length of the calibration board grid, f is the focal length of the camera, and H is the vertical length of the camera's light-sensitive area.

[0052] The attitude angle calculation module 620 is used to calculate the yaw angle, pitch angle and roll angle of the vehicle camera based on the corner pixel coordinates in the calibration board image and the optical center pixel coordinates of the vehicle camera, and to calibrate the installation attitude angle of the vehicle camera based on the yaw angle, pitch angle and roll angle.

[0053] Specifically, let the pixel coordinates of the corner points detected by the vehicle-mounted camera on the calibration board be as follows: P(X P ,Y P ), And the coordinates of the optical center pixel are O(X) O ,Y O Then, calculate the yaw, pitch, and roll angles respectively according to the formulas:

[0054]

[0055]

[0056]

[0057] In the formula, f is the focal length of the camera, pix is ​​the pixel size of the photosensitive area, and atan is the arctangent function;

[0058] Among them, PL1 and PL2 are the left corner points of the chessboard grid in the calibration board, P is the middle corner point of the chessboard grid in the calibration board, and PR1 and PR2 are the right corner points of the chessboard grid in the calibration board.

[0059] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the system and modules described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0060] Figure 7 This is a schematic diagram of an electronic device according to an embodiment of the present invention. The electronic device is used for calibrating the mounting attitude angle of a vehicle-mounted camera. Figure 7 As shown, the electronic device 7 of this embodiment includes a memory 710, a processor 720, and a system bus 730. The memory 710 includes an executable program 7101 stored thereon. As those skilled in the art will understand, Figure 7 The electronic device structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0061] The following is combined Figure 7 A detailed introduction to each component of the electronic device:

[0062] The memory 710 can be used to store software programs and modules. The processor 720 executes various functional applications and data processing of the electronic device by running the software programs and modules stored in the memory 710. The memory 710 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device (such as cached data), etc. In addition, the memory 710 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0063] The memory 710 contains an executable program 7101 for a network request method. This executable program 7101 can be divided into one or more modules / units, which are stored in the memory 710 and executed by the processor 720 to perform functions such as camera attitude angle calibration. Each module / unit can be a series of computer program instruction segments capable of performing a specific function, describing the execution process of the computer program 7101 in the electronic device 7. For example, the computer program 7101 can be divided into functional modules such as an image acquisition module and an attitude angle calibration module.

[0064] The processor 720 is the control center of the electronic device. It connects various parts of the electronic device via various interfaces and lines. By running or executing software programs and / or modules stored in the memory 710, and by calling data stored in the memory 710, it performs various functions and processes data, thereby monitoring the overall status of the electronic device. Optionally, the processor 720 may include one or more processing units; preferably, the processor 720 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, application programs, etc., and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 720.

[0065] The system bus 730 is used to connect various functional components within the computer, transmitting data, address, and control information. Its type can be, for example, a PCI bus, an ISA bus, or a CAN bus. Instructions from the processor 720 are transmitted to the memory 710 via the bus, and the memory 710 sends data back to the processor 720. The system bus 730 handles the data and instruction exchange between the processor 720 and the memory 710. Of course, the system bus 730 can also connect to other devices, such as network interfaces and display devices.

[0066] In this embodiment of the invention, the executable program executed by the processing 720 of the electronic device includes:

[0067] After the calibration plate is installed in a predetermined manner, images of the calibration plate are captured by the vehicle-mounted camera.

[0068] Based on the corner pixel coordinates in the calibration board image and the optical center pixel coordinates of the vehicle camera, the yaw angle, pitch angle and roll angle of the vehicle camera are calculated respectively, and the installation attitude angle of the vehicle camera is calibrated based on the yaw angle, pitch angle and roll angle.

[0069] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and modules described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0070] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0071] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for calibrating the mounting attitude angle of a vehicle-mounted camera, characterized in that, include: After the calibration plate is installed in a predetermined manner, images of the calibration plate are captured by the vehicle-mounted camera. The calibration board consists of three parts, all of which are chessboard grids. The center of the middle chessboard grid is equidistant from the left and right chessboard grids, and the left, middle, and right chessboard grids are horizontal in relative position. The left and right parts of the chessboard are both obliquely symmetrical 2×n chessboards, where n≥2, and the middle part of the chessboard is an obliquely symmetrical 2×2 chessboard. Based on the corner pixel coordinates in the calibration board image and the optical center pixel coordinates of the vehicle camera, the yaw angle, pitch angle and roll angle of the vehicle camera are calculated respectively, and the installation attitude angle of the vehicle camera is calibrated based on the yaw angle, pitch angle and roll angle. Specifically, the calculation of the yaw angle, pitch angle, and roll angle of the vehicle-mounted camera based on the corner pixel coordinates in the calibration board image and the optical center pixel coordinates of the vehicle-mounted camera is as follows: Let the pixel coordinates of the corner points on the calibration board detected by the vehicle-mounted camera be PL1( , ), PL2 ( , ), P( , ), PR1 , ), PR2 , ), and the coordinates of the optical center pixel are O( , Then, calculate the yaw, pitch, and roll angles respectively according to the formulas: ； ； ； In the formula, f is the camera focal length, and pix is ​​the pixel size of the photosensitive area. It is the arctangent function; Among them, PL1 and PL2 are the left corner points of the chessboard grid in the calibration board, P is the middle corner point of the chessboard grid in the calibration board, and PR1 and PR2 are the right corner points of the chessboard grid in the calibration board.

2. The method according to claim 1, characterized in that, The installation of the calibration plate in a predetermined manner includes: The distance from the center of the calibration board to the ground is equal to the height of the vehicle-mounted camera; The longitudinal direction of the vehicle body is perpendicular to the plane of the calibration plate, and the center of the calibration plate is exactly on the forward coordinate axis of the ideal coordinate system of the camera; Adjust the horizontal distance between the calibration board and the camera so that the calibration board is located in the field of view (FOV) of the camera, and the horizontal distance between the calibration board and the camera is greater than the predetermined value L; Where L = 2 * l * f / H, l is the side length of the calibration board grid, f is the focal length of the camera, and H is the vertical length of the camera's light-sensitive area.

3. A vehicle-mounted camera mounting attitude angle calibration system, characterized in that, include: The image acquisition module is used to acquire images of the calibration board via an onboard camera after the calibration board is installed in a predetermined manner. The calibration board consists of three parts, all of which are chessboard grids. The center of the middle chessboard grid is equidistant from the left and right chessboard grids, and the left, middle, and right chessboard grids are horizontal in relative position. The left and right parts of the chessboard are both obliquely symmetrical 2×n chessboards, where n≥2, and the middle part of the chessboard is an obliquely symmetrical 2×2 chessboard. The attitude angle calculation module is used to calculate the yaw angle, pitch angle and roll angle of the vehicle camera based on the corner pixel coordinates in the calibration board image and the optical center pixel coordinates of the vehicle camera, and to calibrate the installation attitude angle of the vehicle camera based on the yaw angle, pitch angle and roll angle. Specifically, the calculation of the yaw angle, pitch angle, and roll angle of the vehicle-mounted camera based on the corner pixel coordinates in the calibration board image and the optical center pixel coordinates of the vehicle-mounted camera is as follows: Let the pixel coordinates of the corner points on the calibration board detected by the vehicle-mounted camera be PL1( , ), PL2 ( , ), P( , ), PR1 , ), PR2 , ), and the coordinates of the optical center pixel are O( , Then, calculate the yaw, pitch, and roll angles respectively according to the formulas: ； ； ； In the formula, f is the camera focal length, and pix is ​​the pixel size of the photosensitive area. It is the arctangent function; Among them, PL1 and PL2 are the left corner points of the chessboard grid in the calibration board, P is the middle corner point of the chessboard grid in the calibration board, and PR1 and PR2 are the right corner points of the chessboard grid in the calibration board.

4. The system according to claim 3, characterized in that, The installation of the calibration plate in a predetermined manner includes: The distance from the center of the calibration board to the ground is equal to the height of the vehicle-mounted camera; The longitudinal direction of the vehicle body is perpendicular to the plane of the calibration plate, and the center of the calibration plate is exactly on the forward coordinate axis of the ideal coordinate system of the camera; Adjust the horizontal distance between the calibration board and the camera so that the calibration board is located in the field of view (FOV) of the camera, and the horizontal distance between the calibration board and the camera is greater than the predetermined value L; Where L = 2 * l * f / H, l is the side length of the calibration board grid, f is the focal length of the camera, and H is the vertical length of the camera's light-sensitive area.

5. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the vehicle-mounted camera mounting attitude angle calibration method as described in any one of claims 1 to 2.

6. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed, it implements the steps of the vehicle-mounted camera mounting attitude angle calibration method as described in any one of claims 1 to 2.

Images