A method for calibrating internal and external parameters of an in-vehicle camera and an in-vehicle terminal

Abstract

The present application relates to a method for calibrating internal and external parameters of an in-vehicle camera and an in-vehicle terminal. The method includes: receiving instruction information input by a user to the in-vehicle terminal, and according to the instruction information, controlling a projection device to project multiple first calibration pictures pre-stored in the in-vehicle terminal on a vehicle windshield. The multiple first calibration pictures are obtained by photographing the same reference object from different shooting angles. Obtaining multiple second calibration pictures obtained by the in-vehicle camera photographing the multiple first calibration pictures, and calculating the internal and external parameters of the in-vehicle camera based on the multiple second calibration pictures. There is no need to obtain a specific calibration object or obtain a calibration object from a specific position, and the internal and external parameters of the in-vehicle camera can be flexibly calibrated, making the calibration result more accurate.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and particularly to a method for calibrating internal and external parameters of an in-vehicle camera and an in-vehicle terminal. Background Art

[0002] Camera calibration is a crucial technology in image processing technology. In the imaging geometric model of a camera, there is a corresponding relationship between spatial points and image points on the image plane, and this corresponding relationship is determined by camera parameters. Generally, camera parameters include internal and external parameters of the camera. The internal parameters are parameters related to the characteristics of the camera itself, such as the focal length and pixel size of the camera, and the external parameters are parameters in the world coordinate system, such as the position and rotation direction of the camera.

[0003] In an in-vehicle camera system, lane lines are usually selected as the calibration object for the in-vehicle camera, requiring clear lane lines and high road flatness. Or, the distance between the vehicle and the parallel lines of the lane and the vanishing point are used for calibration, which is only applicable to specific roads. During the daily driving of the vehicle, using the above methods for calibrating the in-vehicle camera will result in inaccurate calibration results. Summary of the Invention

[0004] Based on this, in view of the above technical problems, it is necessary to provide a method for calibrating internal and external parameters of an in-vehicle camera and an in-vehicle terminal, which can flexibly calibrate the internal and external parameters of the in-vehicle camera and make the calibration results more accurate.

[0005] In a first aspect, a method for calibrating internal and external parameters of an in-vehicle camera is provided, and the method includes:

[0006] Receiving instruction information input by a user to the in-vehicle terminal;

[0007] According to the instruction information, controlling a projection device to project multiple first calibration pictures pre-stored in the in-vehicle terminal on the vehicle windshield. The multiple first calibration pictures are obtained by photographing the same reference object from different shooting angles;

[0008] Obtaining multiple second calibration pictures obtained by the in-vehicle camera photographing the multiple first calibration pictures;

[0009] Calculating the internal and external parameters of the in-vehicle camera based on the multiple second calibration pictures.

[0010] In some possible implementation manners, a calibration program is installed in the in-vehicle terminal, and the multiple first calibration pictures are pre-stored in the calibration program; receiving the instruction information input by the user to the in-vehicle terminal includes:

[0011] In response to detecting that the user starts the calibration program, obtaining the instruction information.

[0012] In some possible implementation manners, in response to detecting that a user starts a calibration program, instruction information is obtained, including:

[0013] In response to detecting voice information for starting a calibration program input by the user to the vehicle-mounted terminal or detecting operation information for starting a calibration program input by the user on a display interface of the vehicle-mounted terminal, instruction information is obtained.

[0014] In some possible implementation manners, in response to the user starting a calibration program, instruction information is obtained, including:

[0015] In response to detecting that the user starts a calibration program, the current vehicle speed is obtained;

[0016] If the current vehicle speed is less than or equal to a preset threshold value, instruction information is obtained.

[0017] In some possible implementation manners, the reference object is a black-and-white chessboard grid; based on multiple second calibration pictures, internal and external parameters of the vehicle-mounted camera are calculated, including:

[0018] The coordinates of the intersection points of each adjacent black grid and white grid in the black-and-white chessboard grid on each second calibration picture among multiple second calibration pictures are obtained;

[0019] Based on the coordinates of all the intersection points, the internal and external parameters of the vehicle-mounted camera are calculated.

[0020] In some possible implementation manners, based on the coordinates of all the intersection points, the internal and external parameters of the vehicle-mounted camera are calculated, including:

[0021] Based on the coordinates of all the intersection points, the pixel coordinates corresponding to the coordinates of all the intersection points are obtained;

[0022] The first matrix of all the pixel coordinates in the world coordinate system is calculated;

[0023] Based on the first matrix, the second matrix of all the pixel coordinates in the camera coordinate system is calculated;

[0024] Based on the first matrix and the second matrix, the third matrix of all the pixel coordinates in the world coordinate system is calculated, where the third matrix includes the internal and external parameters of the vehicle-mounted camera.

[0025] In some possible implementation manners, the method further includes:

[0026] The internal and external parameters of the vehicle-mounted camera are saved to the calibration program.

[0027] In some possible implementation manners, the method further includes:

[0028] If it is detected that the user starts the calibration program again to obtain new internal and external parameters of the vehicle-mounted camera, the new internal and external parameters are updated to the calibration program.

[0029] In some possible implementations, the internal and external parameters of the vehicle-mounted camera are saved to the calibration program, including:

[0030] Naming the internal and external parameters of the vehicle-mounted camera as the first internal and external parameters and saving them to the parameter library of the calibration program;

[0031] The method further includes:

[0032] If it is detected that the user starts the calibration program again to obtain new internal and external parameters of the vehicle-mounted camera, naming the new internal and external parameters as the second internal and external parameters and saving them to the parameter library.

[0033] In a second aspect, a vehicle-mounted terminal is provided, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the method for calibrating the internal and external parameters of the vehicle-mounted camera in the first aspect or any possible implementation manner of the first aspect is implemented.

[0034] In a third aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method for calibrating the internal and external parameters of the vehicle-mounted camera in the first aspect or any possible implementation manner of the first aspect is implemented.

[0035] The above method for calibrating the internal and external parameters of the vehicle-mounted camera and the vehicle-mounted terminal, by receiving the instruction information input by the user to the vehicle-mounted terminal, according to the instruction information, controlling the projection device to project multiple first calibration pictures pre-stored in the vehicle-mounted terminal on the vehicle windshield. The multiple first calibration pictures are taken of the same reference object from different shooting angles, obtaining multiple second calibration pictures taken by the vehicle-mounted camera of the multiple first calibration pictures, and calculating the internal and external parameters of the vehicle-mounted camera based on the multiple second calibration pictures. There is no need to obtain a specific calibration object or obtain the calibration object from a specific position, and the internal and external parameters of the vehicle-mounted camera can be flexibly calibrated, making the calibration result more accurate. Description of the Drawings

[0036] Figure 1 The application scenario of the method for calibrating the internal and external parameters of the vehicle-mounted camera in an embodiment of the present application;

[0037] Figure 2 The flowchart of the method for calibrating the internal and external parameters of the vehicle-mounted camera in an embodiment of the present application;

[0038] Figure 3 The schematic diagram of the black and white checkerboard in an embodiment of the present application;

[0039] Figure 4 The flowchart of calculating the internal and external parameters of the vehicle-mounted camera according to the coordinates of all intersection points in an embodiment of the present application;

[0040] Figure 5 The structural block diagram of the in-vehicle camera internal and external parameter calibration device in an embodiment of the present application;

[0041] Figure 6 The internal structure diagram of the in-vehicle terminal in an embodiment of the present application. Detailed implementation manners

[0042] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, but not to limit the present application.

[0043] The camera is an important and commonly used device among the sensors used in vehicles. Accurately calibrated camera parameters play an important role in improving the system accuracy and precision. In the prior art, static calibration methods are usually used to calibrate the camera. When using this calibration method, the vehicle must be parked in a specific environment of a designated place to perform the calibration. The calibration process is cumbersome, and the calibration method has certain limitations.

[0044] To solve the problems in the prior art, the embodiments of the present application provide a method for calibrating the internal and external parameters of an in-vehicle camera and an in-vehicle terminal. First, the method for calibrating the internal and external parameters of the in-vehicle camera provided by the embodiments of the present application will be introduced below.

[0045] The method for calibrating the internal and external parameters of the in-vehicle camera provided by the embodiments of the present application is applied to Figure 1 the scene shown in the figure. This scene includes an in-vehicle camera 110, a projection device 120, a mirror 130, and a windshield 140. Among them, P w represents the coordinate point in the real world, and P c represents the coordinate point of P w in the in-vehicle camera coordinate system, and P i represents the coordinate point of P w in the image coordinate system. When the in-vehicle terminal receives the instruction sent by the user, it turns on the projection device 120 to project multiple stored calibration pictures according to the instruction. The mirror 130 can adjust the projection angle of the calibration pictures on the windshield 140. The in-vehicle camera 110 takes the calibration photos projected on the windshield 140, obtains the specific coordinate points in the calibration photos, and calculates the conversion matrix between the specific coordinate points, P w , P c , P i to obtain the internal and external parameters of the in-vehicle camera.

[0046] Figure 2 shows the flow chart of the method for calibrating the internal and external parameters of the in-vehicle camera provided by an embodiment of the present application. As Figure 2 shown, the method includes the following steps:

[0047] S210, Receive the command information input by the user to the vehicle-mounted terminal.

[0048] The vehicle-mounted terminal is configured with a vehicle-mounted entertainment system. When the user desires to calibrate the internal and external parameters of the vehicle-mounted camera, corresponding operations are performed through the vehicle-mounted entertainment system. The vehicle-mounted entertainment system receives the command information to initiate the calibration process for the internal and external parameters of the vehicle-mounted camera.

[0049] S220, According to the command information, control the projection device to project multiple first calibration pictures pre-stored in the vehicle-mounted terminal on the vehicle windshield. The multiple first calibration pictures are taken of the same reference object from different shooting angles.

[0050] The command information serves as a trigger signal to initiate the calibration process for the internal and external parameters of the vehicle-mounted camera. When the vehicle-mounted entertainment system receives the command information, it controls the projection device to project multiple first calibration pictures pre-stored in the vehicle-mounted terminal on the vehicle windshield. The calibration result is calculated through the least squares fitting method. When the number of input pictures is relatively small, the accuracy of the fitting result is not high. Therefore, the multiple first calibration pictures are taken of the same reference object from different shooting angles, which is beneficial to improving the accuracy of the calibration result.

[0051] Project the first calibration photos taken of the same reference object from different shooting angles on the vehicle windshield, facilitating the vehicle-mounted terminal to obtain the calibration object at any time and place, reducing the conditional restrictions for obtaining the calibration object, and enabling the vehicle to more flexibly calibrate the internal and external parameters of the vehicle-mounted camera.

[0052] S230, Obtain multiple second calibration pictures taken by the vehicle-mounted camera of the multiple first calibration pictures.

[0053] Whenever the projection device projects a first calibration picture at a certain shooting angle, the vehicle-mounted camera only needs to take a picture of the currently projected picture to obtain the second calibration picture. Therefore, as long as the vehicle-mounted camera is fixed at a suitable shooting position, it can take the second calibration pictures containing the first calibration pictures at different shooting angles without manually moving the vehicle-mounted camera, simplifying the calibration process and making the calibration process of the internal and external parameters of the vehicle-mounted camera more convenient.

[0054] The vehicle-mounted entertainment system obtains multiple second calibration pictures taken by the vehicle-mounted camera for calibrating the internal and external parameters of the vehicle-mounted camera.

[0055] S240, Calculate the internal and external parameters of the vehicle-mounted camera based on the multiple second calibration pictures.

[0056] Based on the multiple second calibration pictures, calculate the internal and external parameters of the vehicle-mounted camera through the calibration algorithm.

[0057] In an embodiment of the present application, by receiving instruction information input by a user to an in-vehicle terminal, and according to the instruction information, controlling a projection device to project multiple first calibration pictures pre-stored in the in-vehicle terminal onto a vehicle windshield. The multiple first calibration pictures are obtained by photographing the same reference object from different shooting angles. Multiple second calibration pictures obtained by the in-vehicle camera photographing the multiple first calibration pictures are acquired, and based on the multiple second calibration pictures, the internal and external parameters of the in-vehicle camera are calculated. There is no need to obtain a specific calibration object or obtain a calibration object from a specific position, and the internal and external parameters of the in-vehicle camera can be flexibly calibrated, making the calibration result more accurate.

[0058] In some embodiments, a calibration program is installed in the in-vehicle terminal, and the multiple first calibration pictures are pre-stored in the calibration program; receiving instruction information input by a user to the in-vehicle terminal includes:

[0059] In response to detecting that the user starts the calibration program, the instruction information is obtained.

[0060] A calibration program is installed in the in-vehicle terminal, and this calibration program is specifically installed in the in-vehicle entertainment system. When the in-vehicle entertainment system receives the input information for starting the calibration program from the user, in response to this input information, the instruction information is obtained. Among them, the input information is voice information input by the user to the in-vehicle terminal for starting the calibration program or operation information input by the user on the display interface of the in-vehicle terminal for starting the calibration program.

[0061] When the input information is voice information input by the user to the in-vehicle terminal for starting the calibration program, the voice recognition device in the in-vehicle entertainment system receives the above voice information, and in response to this voice information, the instruction information is obtained.

[0062] When the input information is operation information input by the user on the display interface of the in-vehicle terminal for starting the calibration program, the display touch screen of the in-vehicle entertainment system receives the operation information input by the user, and in response to this operation information, the instruction information is obtained.

[0063] Starting the calibration program in various forms is convenient for the user to select a suitable opening method, improving the user experience while ensuring the driving safety of the user.

[0064] In some embodiments, in response to the user starting the calibration program, obtaining the instruction information includes:

[0065] In response to detecting that the user starts the calibration program, the current vehicle speed of the vehicle is obtained;

[0066] If the current vehicle speed is less than or equal to a preset threshold, the instruction information is obtained.

[0067] For vehicle safety considerations, before starting the calibration program, it is necessary to determine whether the driving state of the vehicle meets the safety range for camera calibration. Therefore, when it is detected that the user starts the calibration program, the current speed of the vehicle is obtained. If the current vehicle speed is less than or equal to the preset threshold, it indicates that the vehicle is in a stationary state or a low-speed driving state at this time. Performing camera calibration in this state can ensure the safety of the vehicle, so the instruction information is obtained and the calibration program is started. If the current vehicle speed is greater than the preset threshold, it indicates that the vehicle is in a high-speed driving state at this time. Performing camera calibration in this state poses a safety risk to the vehicle and it is not suitable to perform camera calibration, so the instruction information is not continued to be obtained and the calibration program is not started temporarily, maximizing the avoidance of safety risks.

[0068] In some embodiments, the reference object is a black and white checkerboard; based on multiple second calibration pictures, the internal and external parameters of the in-vehicle camera are calculated, including:

[0069] Obtain the coordinates of the intersection points of each adjacent black and white grid in the black and white checkerboard on each second calibration picture among multiple second calibration pictures;

[0070] According to the coordinates of all the intersection points, calculate the internal and external parameters of the in-vehicle camera.

[0071] The reference object is Figure 3 The black and white checkerboard shown. Each second calibration picture contains black and white checkerboard images at different shooting angles. Obtain the coordinates of the intersection points of each adjacent black and white grid, for example, the coordinates of point a. According to the coordinates of all the intersection points, use the calibration algorithm to calculate the internal and external parameters of the in-vehicle camera. Since the black and white checkerboard image can make it easier for the terminal to identify the accurate intersection point coordinates, it provides a data basis for accurately calculating the internal and external parameters of the in-vehicle camera.

[0072] In some embodiments, according to the coordinates of all the intersection points, calculate the internal and external parameters of the in-vehicle camera, as Figure 4 shown, including:

[0073] S410, according to the coordinates of all the intersection points, obtain the pixel coordinates corresponding to the coordinates of all the intersection points.

[0074] The coordinates of the intersection points are the coordinates in the image coordinate system, and they are converted into the coordinates in the pixel coordinate system, that is, the pixel coordinates. According to the coordinates of the intersection points, the calculation formula for the pixel coordinates is as follows:

[0075]

[0076] Among them, u represents the abscissa value of the pixel coordinate, v represents the ordinate value of the pixel coordinate, u0 represents the abscissa value of the origin in the image coordinate system in the pixel coordinate system, v0 represents the ordinate value of the origin in the image coordinate system in the pixel coordinate system, x represents the abscissa value of the intersection point in the image coordinate system, y represents the ordinate value of the intersection point in the image coordinate system, and d x represents the physical size of the intersection point in the abscissa in the pixel coordinate system, and d y represents the physical size of the intersection point in the ordinate in the pixel coordinate system.

[0077] S420. Calculate the first matrix of all pixel coordinates in the world coordinate system.

[0078] In order to describe the projection and transmission relationship of an object from the image coordinate system to the camera coordinate system during the imaging process, perform a coordinate transformation between the image coordinate system and the pixel coordinate system, and calculate the first matrix of all pixel coordinates in the world coordinate system. The first matrix is expressed as:

[0079]

[0080] Among them, u represents the abscissa value of the pixel coordinate, v represents the ordinate value of the pixel coordinate, u0 represents the abscissa value of the origin in the image coordinate system in the pixel coordinate system, v0 represents the ordinate value of the origin in the image coordinate system in the pixel coordinate system, x represents the abscissa value of the intersection point in the image coordinate system, and y represents the ordinate value of the intersection point in the image coordinate system.

[0081] S430. Calculate the second matrix of all pixel coordinates in the camera coordinate system according to the first matrix.

[0082] The camera coordinate system is a coordinate system established on the camera and defined to describe the position of an object from the perspective of the camera, serving as an intermediate link between the world coordinate system and the pixel coordinate system. The conversion from the world coordinate system to the camera coordinate system is essentially a conversion from two-dimensional points to three-dimensional points. Calculate the second matrix of all pixel coordinates in the camera coordinate system according to the first matrix. The second matrix can be expressed as:

[0083]

[0084] Among them, x represents the abscissa value of the intersection point in the image coordinate system, y represents the ordinate value of the intersection point in the image coordinate system, f represents the focal length of the vehicle-mounted camera, X c represents the abscissa value of the intersection point in the camera coordinate system, Y c represents the ordinate value of the intersection point in the camera coordinate system, and Z c represents the vertical coordinate value of the intersection point in the camera coordinate system.

[0085] S440. Calculate a third matrix of all pixel coordinates in the world coordinate system according to the first matrix and the second matrix, where the third matrix includes the internal and external parameters of the vehicle-mounted camera.

[0086] The world coordinate system is the coordinate system of the user-defined three-dimensional world, which is introduced to describe the position of the target object in the real world. Any rotation in any dimension can be represented as the product of a coordinate vector and a suitable square matrix. The translation vector is the offset between the first coordinate origin and the second coordinate origin. In the world coordinate system, there are two important parameters: the rotation matrix R and the translation vector T. Calculate the third matrix of all pixel coordinates in the world coordinate system according to the first matrix, the second matrix, the rotation matrix R and the translation vector T in the world coordinate system. The third matrix can be expressed as:

[0087]

[0088] where is the internal parameter of the camera, is the external parameter of the camera.

[0089] where u represents the abscissa value of the pixel coordinate, v represents the ordinate value of the pixel coordinate, u0 represents the abscissa value of the origin in the image coordinate system in the pixel coordinate system, v0 represents the ordinate value of the origin in the image coordinate system in the pixel coordinate system, x represents the abscissa value of the intersection point in the image coordinate system, y represents the ordinate value of the intersection point in the image coordinate system, R represents the rotation matrix, T represents the translation vector, Z c represents the vertical coordinate value of the intersection point in the camera coordinate system, X w represents the abscissa value of the intersection point in the world coordinate system, Y w represents the ordinate value of the intersection point in the world coordinate system, Z w represents the vertical coordinate value of the intersection point in the world coordinate system, f x represents the component of the focal length of the vehicle-mounted camera in the abscissa, f y represents the component of the focal length of the vehicle-mounted camera in the ordinate.

[0090] In some embodiments, the method further includes: saving the internal and external parameters of the vehicle-mounted camera to the calibration program for subsequent use by the vehicle-mounted terminal. For example, during vehicle positioning, the internal and external parameters of the camera obtained from this calibration can be used to detect the target positions in the vehicle's surrounding environment, enabling the vehicle to drive in the correct area and ensuring the safety of vehicle driving.

[0091] In some embodiments, the method further includes: if it is detected that the user starts the calibration program again to obtain new internal and external parameters of the vehicle-mounted camera, updating the new internal and external parameters to the calibration program, so that the vehicle-mounted terminal can call the real-time internal and external parameters, improving the accuracy of the vehicle-mounted terminal in operating device functions.

[0092] In some embodiments, the internal and external parameters of the vehicle-mounted camera are saved to the calibration program, including:

[0093] Name the internal and external parameters of the vehicle-mounted camera as the first internal and external parameters and save them to the parameter library of the calibration program;

[0094] The method further includes:

[0095] If it is detected that the user starts the calibration program again to obtain new internal and external parameters of the vehicle-mounted camera, name the new internal and external parameters as the second internal and external parameters and save them to the parameter library.

[0096] Save the internal and external parameters obtained by each camera calibration to the parameter library to obtain a database of internal and external parameters. Even when the driving conditions do not allow camera calibration, an optimal set of internal and external parameters can be obtained from the parameter library for the vehicle-mounted terminal to call, ensuring safe driving of the vehicle.

[0097] It should be understood that although Figure 2 and 4 the steps in the flowcharts are shown in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this article, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 2 and 4 at least a part of the steps in

[0098] In some embodiments, as Figure 5 shown, a device for calibrating the internal and external parameters of a vehicle-mounted camera is provided, including: a receiving module 510, a control module 520, an acquisition module 530, and a calculation module 540, where:

[0099] The receiving module 510 is configured to receive the instruction information input by the user to the vehicle-mounted terminal;

[0100] The control module 520 is configured to control the projection device to project multiple first calibration pictures pre-stored in the vehicle-mounted terminal on the vehicle windshield. The multiple first calibration pictures are taken of the same reference object from different shooting angles;

[0101] The acquisition module 530 is configured to acquire multiple second calibration pictures obtained by the vehicle-mounted camera shooting the multiple first calibration pictures;

[0102] A calculation module 540, configured to calculate the internal and external parameters of the vehicle-mounted camera based on multiple second calibration pictures.

[0103] In the embodiments of the present application, there is no need to obtain a specific calibration object or obtain the calibration object from a specific position, and the internal and external parameters of the vehicle-mounted camera can be flexibly calibrated, making the calibration result more accurate.

[0104] In some embodiments, a calibration program is installed in the vehicle-mounted terminal, and multiple first calibration pictures are pre-stored in the calibration program; the receiving module 510 is specifically configured to:

[0105] In response to detecting that the user starts the calibration program, obtain instruction information.

[0106] In some embodiments, the receiving module 510 is specifically configured to:

[0107] In response to detecting voice information input by the user to the vehicle-mounted terminal for starting the calibration program or detecting operation information input by the user on the display interface of the vehicle-mounted terminal for starting the calibration program, obtain instruction information.

[0108] In some embodiments, the receiving module 510 is specifically configured to:

[0109] In response to detecting that the user starts the calibration program, obtain the current vehicle speed;

[0110] If the current vehicle speed is less than or equal to a preset threshold, obtain instruction information.

[0111] In some embodiments, the reference object is a black and white checkerboard; the calculation module 540 is specifically configured to:

[0112] Obtain the coordinates of the intersection points of each adjacent black and white grid in the black and white checkerboard on each second calibration picture among multiple second calibration pictures;

[0113] According to the coordinates of all the intersection points, calculate the internal and external parameters of the vehicle-mounted camera.

[0114] In some embodiments, the calculation module 540 is specifically configured to:

[0115] According to the coordinates of all the intersection points, obtain the pixel coordinates corresponding to the coordinates of all the intersection points;

[0116] Calculate a first matrix of all the pixel coordinates in the world coordinate system;

[0117] According to the first matrix, calculate a second matrix of all the pixel coordinates in the camera coordinate system;

[0118] According to the first matrix and the second matrix, calculate a third matrix of all the pixel coordinates in the world coordinate system, where the third matrix includes the internal and external parameters of the vehicle-mounted camera.

[0119] In some embodiments, the device further includes: a storage device 550, configured to:

[0120] Save the internal and external parameters of the vehicle-mounted camera into the calibration program.

[0121] In some embodiments, the storage device 550 is further configured to:

[0122] If it is detected that the user starts the calibration program again to obtain new internal and external parameters of the vehicle-mounted camera, update the new internal and external parameters to the calibration program.

[0123] In some embodiments, the storage device 550 is specifically configured to:

[0124] Name the internal and external parameters of the vehicle-mounted camera as the first internal and external parameters and save them to the parameter library of the calibration program;

[0125] The storage device 550 is further configured to:

[0126] If it is detected that the user starts the calibration program again to obtain new internal and external parameters of the vehicle-mounted camera, name the new internal and external parameters as the second internal and external parameters and save them to the parameter library.

[0127] For the specific limitations on the device for calibrating the internal and external parameters of the vehicle-mounted camera, reference can be made to the limitations on the method for calibrating the internal and external parameters of the vehicle-mounted camera in the foregoing text, which will not be elaborated herein. Each module in the above device for calibrating the internal and external parameters of the vehicle-mounted camera can be implemented in whole or in part by software, hardware, and their combination. The above modules can be embedded in the processor of the computer device in hardware form or be independent of it, or be stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to the above respective modules.

[0128] In some embodiments, a vehicle-mounted terminal is provided. The internal structure diagram of the vehicle-mounted terminal can be as Figure 6 shown. The vehicle-mounted terminal includes a processor, a memory, a network interface, and a database connected through a system bus. Among them, the processor of the vehicle-mounted terminal is used to provide computing and control capabilities. The memory of the vehicle-mounted terminal includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the vehicle-mounted terminal is used to store calibration data. The network interface of the vehicle-mounted terminal is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, it implements a method for calibrating the internal and external parameters of a vehicle-mounted camera.

[0129] Those skilled in the art can understand, Figure 6The structure shown is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the in-vehicle terminal to which the solution of this application is applied. The specific in-vehicle terminal may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0130] In some embodiments, an in-vehicle terminal is provided, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the following steps are implemented:

[0131] Receive instruction information input by the user to the in-vehicle terminal;

[0132] According to the instruction information, control a projection device to project multiple first calibration pictures pre-stored in the in-vehicle terminal on the vehicle windshield. The multiple first calibration pictures are obtained by photographing the same reference object from different shooting angles;

[0133] Obtain multiple second calibration pictures obtained by the in-vehicle camera photographing the multiple first calibration pictures;

[0134] Based on the multiple second calibration pictures, calculate the internal and external parameters of the in-vehicle camera.

[0135] In some embodiments, when the processor executes the computer program, the following steps are further implemented: A calibration program is installed in the in-vehicle terminal, and the multiple first calibration pictures are pre-stored in the calibration program; receive instruction information input by the user to the in-vehicle terminal, including: in response to detecting that the user starts the calibration program, obtain the instruction information.

[0136] In some embodiments, when the processor executes the computer program, the following steps are further implemented: In response to detecting that the user starts the calibration program, obtain the instruction information, including: in response to detecting voice information input by the user to the in-vehicle terminal for starting the calibration program or detecting operation information input by the user on the display interface of the in-vehicle terminal for starting the calibration program, obtain the instruction information.

[0137] In some embodiments, when the processor executes the computer program, the following steps are further implemented: In response to the user starting the calibration program, obtain the instruction information, including: in response to detecting that the user starts the calibration program, obtain the current vehicle speed; if the current vehicle speed is less than or equal to a preset threshold, obtain the instruction information.

[0138] In some embodiments, when the processor executes the computer program, the following steps are further implemented: The reference object is a black and white checkerboard; based on the multiple second calibration pictures, calculate the internal and external parameters of the in-vehicle camera, including: obtain the coordinates of the intersection points of each adjacent black and white grid in the black and white checkerboard in each of the multiple second calibration pictures; according to the coordinates of all the intersection points, calculate the internal and external parameters of the in-vehicle camera.

[0139] In some embodiments, when the processor executes the computer program, the following steps are further implemented: according to the coordinates of all intersection points, calculate the internal and external parameters of the in-vehicle camera, including: according to the coordinates of all intersection points, obtain the pixel coordinates corresponding to the coordinates of all intersection points; calculate the first matrix of all pixel coordinates in the world coordinate system; according to the first matrix, calculate the second matrix of all pixel coordinates in the camera coordinate system; according to the first matrix and the second matrix, calculate the third matrix of all pixel coordinates in the world coordinate system, wherein the third matrix includes the internal and external parameters of the in-vehicle camera.

[0140] In some embodiments, when the processor executes the computer program, the following steps are further implemented: save the internal and external parameters of the in-vehicle camera to the calibration program.

[0141] In some embodiments, when the processor executes the computer program, the following steps are further implemented: if it is detected that the user starts the calibration program again to obtain new internal and external parameters of the in-vehicle camera, update the new internal and external parameters to the calibration program.

[0142] In some embodiments, when the processor executes the computer program, the following steps are further implemented: save the internal and external parameters of the in-vehicle camera to the calibration program, including: name the internal and external parameters of the in-vehicle camera as the first internal and external parameters and save them to the parameter library of the calibration program; the method further includes: if it is detected that the user starts the calibration program again to obtain new internal and external parameters of the in-vehicle camera, name the new internal and external parameters as the second internal and external parameters and save them to the parameter library.

[0143] In some embodiments, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0144] Receive the instruction information input by the user to the in-vehicle terminal;

[0145] According to the instruction information, control the projection device to project multiple first calibration pictures pre-stored in the in-vehicle terminal on the vehicle windshield, and the multiple first calibration pictures are obtained by photographing the same reference object from different shooting angles;

[0146] Obtain multiple second calibration pictures obtained by the in-vehicle camera photographing the multiple first calibration pictures;

[0147] Based on the multiple second calibration pictures, calculate the internal and external parameters of the in-vehicle camera.

[0148] In some embodiments, when the computer program is executed by the processor, the following steps are further implemented: a calibration program is installed in the in-vehicle terminal, and the multiple first calibration pictures are pre-stored in the calibration program; receive the instruction information input by the user to the in-vehicle terminal, including: in response to detecting that the user starts the calibration program, obtain the instruction information.

[0149] In some embodiments, when the computer program is executed by a processor, the following steps are further implemented: in response to detecting that the user starts the calibration program, obtain instruction information, including: in response to detecting voice information input by the user to the vehicle-mounted terminal for starting the calibration program or detecting operation information input by the user on the display interface of the vehicle-mounted terminal for starting the calibration program, obtain the instruction information.

[0150] In some embodiments, when the computer program is executed by a processor, the following steps are further implemented: in response to the user starting the calibration program, obtain instruction information, including: in response to detecting that the user starts the calibration program, obtain the current vehicle speed; if the current vehicle speed is less than or equal to a preset threshold, then obtain the instruction information.

[0151] In some embodiments, when the computer program is executed by a processor, the following steps are further implemented: the reference object is a black and white checkerboard; based on multiple second calibration pictures, calculate the internal and external parameters of the vehicle-mounted camera, including: obtain the coordinates of the intersection points of each adjacent black and white grid in the black and white checkerboard on each second calibration picture among the multiple second calibration pictures; according to the coordinates of all the intersection points, calculate the internal and external parameters of the vehicle-mounted camera.

[0152] In some embodiments, when the computer program is executed by a processor, the following steps are further implemented: according to the coordinates of all the intersection points, calculate the internal and external parameters of the vehicle-mounted camera, including: according to the coordinates of all the intersection points, obtain the pixel coordinates corresponding to the coordinates of all the intersection points; calculate the first matrix of all the pixel coordinates in the world coordinate system; according to the first matrix, calculate the second matrix of all the pixel coordinates in the camera coordinate system; according to the first matrix and the second matrix, calculate the third matrix of all the pixel coordinates in the world coordinate system, where the third matrix includes the internal and external parameters of the vehicle-mounted camera.

[0153] In some embodiments, when the computer program is executed by a processor, the following steps are further implemented: save the internal and external parameters of the vehicle-mounted camera to the calibration program.

[0154] In some embodiments, when the computer program is executed by a processor, the following steps are further implemented: if it is detected that the user starts the calibration program again to obtain new internal and external parameters of the vehicle-mounted camera, update the new internal and external parameters to the calibration program.

[0155] In some embodiments, when the computer program is executed by a processor, the following steps are further implemented: save the internal and external parameters of the vehicle-mounted camera to the calibration program, including: name the internal and external parameters of the vehicle-mounted camera as the first internal and external parameters and save them to the parameter library of the calibration program; the method further includes: if it is detected that the user starts the calibration program again to obtain new internal and external parameters of the vehicle-mounted camera, name the new internal and external parameters as the second internal and external parameters and save them to the parameter library.

[0156] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database, or other medium used in the embodiments provided in the present application can include non-volatile and / or volatile memories. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM), etc.

[0157] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0158] The above-described embodiments merely represent several implementation manners of the present application. The description thereof is relatively specific and detailed, but it should not be construed as a limitation on the scope of the invention patent. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the appended claims.

Claims

1. A method for calibrating the internal and external parameters of a vehicle-mounted camera, characterized in that, The method includes: Receiving instruction information input by a user to an in-vehicle terminal; Controlling, according to the instruction information, a projection device to project multiple first calibration pictures pre-stored in the in-vehicle terminal on a vehicle windshield, where the multiple first calibration pictures are obtained by photographing the same reference object from different photographing angles; Obtaining multiple second calibration pictures obtained by the in-vehicle camera photographing the multiple first calibration pictures; Calculating internal and external parameters of the in-vehicle camera based on the multiple second calibration pictures.

2. The method according to claim 1, characterized in that A calibration program is installed in the in-vehicle terminal, and the multiple first calibration pictures are pre-stored in the calibration program; The receiving instruction information input by a user to an in-vehicle terminal includes: In response to detecting that the user starts the calibration program, obtaining the instruction information.

3. The method according to claim 2, wherein The in response to detecting that the user starts the calibration program, obtaining the instruction information includes: In response to detecting voice information input by the user to the in-vehicle terminal for starting the calibration program or detecting operation information input by the user on a display interface of the in-vehicle terminal for starting the calibration program, obtaining the instruction information.

4. The method according to claim 2, wherein The in response to the user starting the calibration program, obtaining the instruction information includes: In response to detecting that the user starts the calibration program, obtaining the current vehicle speed; If the current vehicle speed is less than or equal to a preset threshold, obtaining the instruction information.

5. The method according to any one of claims 1 to 4, characterized in that The reference object is a black and white chessboard; the calculating internal and external parameters of the in-vehicle camera based on the multiple second calibration pictures includes: Obtaining coordinates of intersection points of each adjacent black grid and white grid in the black and white chessboard on each second calibration picture among the multiple second calibration pictures; Calculating internal and external parameters of the in-vehicle camera according to coordinates of all the intersection points.

6. The method according to claim 5, characterized in that, The calculating internal and external parameters of the in-vehicle camera according to coordinates of all the intersection points includes: According to coordinates of all the intersection points, obtaining pixel coordinates corresponding to coordinates of all the intersection points; Calculating a first matrix of all the pixel coordinates in a world coordinate system; Calculating a second matrix of all the pixel coordinates in a camera coordinate system according to the first matrix; Calculating a third matrix of all the pixel coordinates in the world coordinate system according to the first matrix and the second matrix, where the third matrix includes internal and external parameters of the in-vehicle camera.

7. The method according to any one of claims 2 to 4, characterized in that, The method further includes: Saving the internal and external parameters of the in-vehicle camera to the calibration program.

8. The method according to claim 7, wherein The method further includes: If it is detected that the user starts the calibration program again to obtain new internal and external parameters of the in-vehicle camera, updating the new internal and external parameters to the calibration program.

9. The method according to claim 7, wherein The saving the internal and external parameters of the in-vehicle camera to the calibration program includes: Naming the internal and external parameters of the in-vehicle camera as first internal and external parameters and saving them to a parameter library of the calibration program; The method further includes: If it is detected that the user starts the calibration program again to obtain new internal and external parameters of the in-vehicle camera, naming the new internal and external parameters as second internal and external parameters and saving them to the parameter library.

10. An in-vehicle terminal, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 9.

Images