External parameter automatic calibration method and device for camera

Abstract

The present application relates to a method and device for automatically calibrating the external parameters of a camera. The method includes: obtaining at least two road pictures captured by a vehicle camera under preset conditions; identifying the lane lines in the road pictures, and using the distance between the lane lines in the road pictures and the actual distance between the lane lines to fit and obtain the external parameters of the camera corresponding to each road picture; grouping the obtained road pictures according to the distribution of the lane lines in the road pictures according to a preset rule, and obtaining the optimal external parameters of the camera in each group of road pictures; randomly selecting two groups of optimal external parameters from the optimal external parameters of each group of road pictures for joint calibration, and determining the optimal joint calibration external parameter among all the joint calibration external parameter results as the target calibration external parameter of the camera. The solution provided by the present application can accurately calibrate the external parameters of the camera and improve the calibration efficiency of the external parameters.

Description

Technical Field

[0001] The present application relates to the field of intelligent transportation, and particularly to a method and device for automatically calibrating the external parameters of a camera. Background Art

[0002] In the related art, in the field of intelligent transportation, it is necessary to obtain the external and internal parameters of a camera for the production of high-precision maps or the data collection of autonomous driving. The internal parameters are related to the memory characteristics of the camera and are provided by the manufacturer or have been calibrated. The external parameters of the camera are related to the pose of the camera in physical space, mainly including the rotation angle and the installation height, etc. The external parameters of the camera are affected by the installation position and angle of the camera and will deviate with the driving on the road. Therefore, it is necessary to calibrate regularly. In the prior art, usually, an object with a known size and pose is manually selected and placed, and the pixel coordinates of the object in the video or road picture are recognized, and the external parameters are fitted by using mathematical methods. The automation degree of the traditional method is not high, the work efficiency is low, and human errors are easily introduced. In particular, it cannot be achieved in the usage scenarios of ordinary users (non-professionals).

[0003] Therefore, there is an urgent need for a method for automatically calibrating the external parameters of a camera to automatically calibrate the external parameters of the camera and improve the calibration efficiency. Summary of the Invention

[0004] To solve or partially solve the problems existing in the related art, the present application provides a method and device for automatically calibrating the external parameters of a camera, which can accurately calibrate the external parameters of the camera and improve the calibration efficiency of the external parameters.

[0005] In the first aspect of the present application, a method for automatically calibrating the external parameters of a camera is provided, including:

[0006] Obtaining at least two road pictures taken by a vehicle camera under preset conditions;

[0007] Identifying the lane lines in the road pictures, and using the interval distance between the lane lines in the road pictures and the actual interval distance between the lane lines to fit and obtain the external parameters of the camera corresponding to each road picture;

[0008] Grouping the obtained road pictures according to the distribution of the lane lines in the road pictures according to preset rules, and obtaining the optimal external parameters of the camera in each group of road pictures. The optimal external parameters of the camera are the external parameters with the highest score obtained based on a preset evaluation algorithm;

[0009] Selecting any two groups of the optimal external parameters of the camera in each group of road pictures for joint calibration, and determining the optimal joint calibration external parameter in all the joint calibration external parameter results as the target calibration external parameter of the camera. The optimal joint calibration external parameter is the joint calibration external parameter with the highest score obtained based on a preset evaluation algorithm.

[0010] Optionally, the preconditions include: the heading angle of the vehicle is within a preset heading angle range, the driving speed of the vehicle is within a specified speed range, and the driving length of the vehicle is greater than a target length.

[0011] Optionally, identifying lane lines further includes:

[0012] Obtain the number of lane lines in the current road image. If the number of lane lines converging at the same vanishing point is less than three, delete all the identified lane lines in the current road image.

[0013] Optionally, when identifying the lane lines of a road image, using the distance between lane lines in the road image and the actual distance between lane lines, the external camera parameters corresponding to each road image are obtained by fitting, including:

[0014] Read the distance data between lane lines in each road image and establish the coordinate system equation of the road image corresponding to the lane lines;

[0015] According to the actual distance data between lane lines, establish the coordinate system equation of the world corresponding to the lane lines;

[0016] According to the coordinate system equation of the road image corresponding to the lane lines and the coordinate system equation of the world corresponding to the lane lines, obtain the external camera parameters corresponding to each road image.

[0017] Optionally, obtaining the optimal external camera parameters in each group of road images includes:

[0018] According to a preset evaluation algorithm, score the external camera parameters of each group of road images, and delete the external camera parameters with a score lower than a preset score threshold;

[0019] Select the external camera parameter with the highest score in each group of road images as the optimal external camera parameter in each group of road images.

[0020] 8. Optionally, select any two groups of the optimal external camera parameters from each group of road images for joint calibration, and determine the optimal joint calibration parameter among all the joint calibration parameter results as the target calibration parameter of the external camera parameters, including:

[0021] Select any two groups of road images from each group of road images, and use the road images corresponding to the optimal external camera parameters in the two groups of road images as the calibration road image combination;

[0022] According to preset combination conditions, obtain the valid road image combination in the calibration road image combination;

[0023] Extract the distance data between lane lines in each road image in the valid road image combination, and use the distance between lane lines in the valid road image combination and the actual distance between lane lines to obtain the joint calibration parameter of the valid road image combination;

[0024] According to a preset evaluation algorithm, score all the extrinsic parameters of the joint calibration, and determine the optimal extrinsic parameters of the joint calibration among all the results of the joint calibration of the extrinsic parameters as the target calibration extrinsic parameters of the camera.

[0025] Optionally, according to preset combination conditions, obtain a valid road picture combination in the calibrated road picture combination, including:

[0026] Obtain the total number of lane lines in each road picture in the group;

[0027] Obtain the maximum number of left lane lines in the grouped road pictures and the maximum number of right lane lines in the grouped road pictures;

[0028] If the total number of lane lines in any road picture is not less than the sum of the maximum number of left lane lines and the maximum number of right lane lines, discard the road picture combination.

[0029] The second aspect of the present application provides an automatic calibration device for the extrinsic parameters of a high-precision map, including:

[0030] An automatic calibration device for the extrinsic parameters of a camera, characterized by including:

[0031] A first image processing unit, configured to obtain at least two road pictures taken by a vehicle camera under preset conditions;

[0032] The first image processing unit identifies the lane lines of the road pictures, and uses the interval distance between the lane lines in the road pictures and the actual interval distance of the lane lines to fit and obtain the camera extrinsic parameters corresponding to each road picture;

[0033] An evaluation unit, configured to group the obtained road pictures according to the distribution of the lane lines in the road pictures according to a preset rule, and obtain the optimal camera extrinsic parameters in each group of road pictures, where the optimal camera extrinsic parameters are the camera extrinsic parameters with the highest score obtained based on a preset evaluation algorithm;

[0034] A determination unit, configured to select any two sets of optimal camera extrinsic parameters from the optimal camera extrinsic parameters of each group of road pictures for joint calibration, and determine the optimal joint calibration extrinsic parameters among all the results of the joint calibration of the extrinsic parameters as the target calibration extrinsic parameters of the camera, where the optimal joint calibration extrinsic parameters are the joint calibration extrinsic parameters with the highest score obtained based on a preset evaluation algorithm.

[0035] The third aspect of the present application provides an electronic device, including:

[0036] A processor; and

[0037] A memory, on which executable code is stored, and when the executable code is executed by the processor, the processor is caused to execute the method as described above.

[0038] The fourth aspect of the present application provides a computer-readable storage medium, on which executable code is stored. When the executable code is executed by a processor of an electronic device, the processor is caused to execute the method as described above.

[0039] The technical solution provided by the present application may include the following beneficial effects: By selecting road driving road pictures that meet preset conditions for automatic external parameter calibration, automatically identifying lane lines in the road pictures, and using the lane line interval data in the road pictures and the actual lane line interval data to obtain the external parameter information of the photo, thereby realizing the external parameter calibration of the camera. This method does not require manual participation, can meet the needs of multiple scenarios, and improves the calibration efficiency.

[0040] On the other hand, the present invention takes into account the calibration road conditions, groups and calibrates the road pictures according to the distribution of different lane lines, performs single calibration on the road driving road pictures that meet the single calibration conditions, and performs joint calibration on the road driving road pictures whose lane lines do not meet the single calibration conditions using multiple road driving road pictures, thereby expanding the application range of the calibrated external parameters and improving the calibration accuracy.

[0041] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] By describing the exemplary embodiments of the present application in more detail in conjunction with the drawings, the above and other objects, features, and advantages of the present application will become more obvious. Among them, in the exemplary embodiments of the present application, the same reference numerals generally represent the same components.

[0043] Figure 1 is a schematic flowchart of the method for automatic external parameter calibration of a camera shown in an embodiment of the present application;

[0044] Figure 2 is another schematic flowchart of the method for automatic external parameter calibration of a camera shown in an embodiment of the present application;

[0045] Figure 3 is another schematic flowchart of the method for automatic external parameter calibration of a camera shown in an embodiment of the present application;

[0046] Figure 4 is a schematic structural diagram of the device for automatic external parameter calibration of a camera shown in an embodiment of the present application;

[0047] Figure 5 is a schematic structural diagram of the electronic device shown in an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0048] Embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the embodiments of the present application are shown in the drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to make the present application more thorough and complete, and to fully convey the scope of the present application to those skilled in the art.

[0049] The terms used in the present application are for the purpose of describing specific embodiments only and are not intended to limit the present application. The singular forms "a", "an", and "the" used in the present application and the appended claims are also intended to include the plural forms unless the context clearly dictates otherwise. It should also be understood that the term "and / or" used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0050] It should be understood that although the terms "first", "second", "third", etc. may be used in the present application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the present application, the meaning of "a plurality" is two or more unless otherwise specifically defined.

[0051] In the related art, in the field of intelligent transportation, it is necessary to obtain the extrinsic and intrinsic parameters of a camera for the production of high-precision maps or data collection for autonomous driving. The intrinsic parameters are related to the memory characteristics of the camera and are provided by the manufacturer or have been calibrated. The extrinsic parameters of the camera are related to the pose of the camera in physical space, mainly including the rotation angle and the installation height, etc. The extrinsic parameters of the camera are affected by the installation position and angle of the camera and will deviate with the driving on the road. Therefore, it is necessary to calibrate regularly. In the prior art, usually, known objects of a certain size and pose are manually selected and placed, and the pixel coordinates of the object in the video or road pictures are identified, and the extrinsic parameters are fitted by mathematical methods. The automation degree of the traditional method is not high, the work efficiency is low, and human errors are easily introduced. In particular, it cannot be achieved in the usage scenarios of ordinary users (non-professionals).

[0052] Therefore, there is an urgent need for a method for automatically calibrating the extrinsic parameters of a high-precision map to automatically calibrate the extrinsic parameters of a camera and improve the calibration efficiency.

[0053] In view of the above problems, an embodiment of the present application provides a method for automatically calibrating the extrinsic parameters of a high-precision map, which can automatically calibrate the extrinsic parameters of a camera and improve the calibration efficiency of the extrinsic parameters of the camera.

[0054] The technical solutions of the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0055] Figure 1 It is a schematic flow chart of a method for automatically calibrating the extrinsic parameters of a high-precision map shown in an embodiment of the present application.

[0056] See Figure 1 , this method includes steps S101 to S101, specifically:

[0057] Step S101: Obtain at least two road pictures taken by a vehicle camera under preset conditions.

[0058] In step S101, the road driving road picture is a road driving road picture containing a GPS (Global Positioning System) logo. The GPS logo is used to identify the driving environment of the vehicle and can also be used to judge the driving condition of the road. In the present application, the road driving road picture can be obtained by intercepting the road picture with a GPS logo from the road driving video, or directly receiving the road picture containing the GPS logo.

[0059] In step S101, the preset conditions include: the heading angle of the vehicle is within a preset heading angle range, the driving speed of the vehicle is within a specified speed range, and the driving length of the vehicle is greater than the target length. Among them, the lane heading angle within the preset heading angle range is used to indicate that the vehicle equipped with the camera is driving in a straight line; the driving speed of the vehicle within the specified speed range and the driving length of the vehicle being greater than the target length are used to indicate that the vehicle drives a distance greater than the preset length on the road at the specified speed within the speed range.

[0060] Optionally, it can also be judged whether the vehicle is going straight according to the received GPS signal to judge whether the vehicle is driving in a straight line.

[0061] Optionally, the vehicle analyzes the driving speed and driving length of the vehicle according to the positions of different lane lines photographed by the vehicle in different road pictures received within a unit time.

[0062] Step S102, identify the lane lines of the road picture, and use the interval distance between the lane lines in the road picture and the actual interval distance of the lane lines to fit and obtain the camera extrinsic parameters corresponding to each road picture.

[0063] The lane lines include: guiding lane lines and variable guiding lane lines. The guiding lane lines are lane markings that guide the direction and are used to indicate that the vehicle should drive in the indicated direction at the entrance section of the intersection. Such markings are generally drawn at traffic intersections with heavy traffic. The purpose is to clarify the driving direction, keep each vehicle in its own lane, and relieve traffic pressure. In the present application, mainly guiding lane lines are used. Among them, the actual lane line data obtains the local lane line interval data according to the GPS logo. In the present application, the actual interval distance of the lane lines is 3.5 meters.

[0064] In step S102, after identifying the lane lines, it is also necessary to judge the quality of the lane lines in the road picture. Road pictures with poor image quality of lane lines should be discarded and not participate in subsequent calculations. Specifically, after identifying the lane lines in the road picture, step S102 further includes: obtaining the number of lane lines in the current road picture. If the number of lane lines converging at the same vanishing point is less than three, all the lane lines identified in the current road picture are deleted.

[0065] Specifically, for the calibration of the external parameters, it usually involves manually selecting and placing an object with a known size and pose, identifying the pixel coordinates of the object in the video or road picture, and using mathematical methods to fit the external parameters. The internal parameters are related to the memory characteristics of the camera and are provided by the manufacturer or have been calibrated. The external parameters of the camera involve the pose of the camera in the physical space, mainly including factors such as the rotation angle and the installation height. The external parameters of the camera can be obtained through the conversion relationship between the pixel coordinate system and the road coordinate system. In this application, the internal and external parameters of the camera are represented by matrices. Obtaining the parameters of the monocular camera includes: constructing a transformation matrix between the pixel coordinate system and the road coordinate system, and obtaining the parameter matrix of the camera according to the transformation matrix.

[0066] In one embodiment, as Figure 2 shown, in step S102, using the interval distance between the lane lines in the road picture and the actual interval distance of the lane lines, the external parameters of the camera corresponding to each road picture are obtained by fitting, including:

[0067] Step S201, reading the interval data of the lane lines in each road picture and establishing the equation of the road picture coordinate system corresponding to the lane lines;

[0068] Step S202, establishing the equation of the world coordinate system corresponding to the lane lines according to the actual interval data of the lane lines;

[0069] Step S203, obtaining the external parameters of the camera corresponding to each road picture according to the equation of the road picture coordinate system corresponding to the lane lines and the equation of the world coordinate system corresponding to the lane lines.

[0070] In one embodiment, the automatic calibration of the external parameters of the camera mainly obtains the rotation error angle and displacement of the camera relative to the reference coordinate system, which are usually represented by the rotation matrix and the translation matrix, and is calibrated using the vanishing point of the lane lines (the intersection point of parallel lane lines on the camera image plane).

[0071] Specifically, identify the vanishing point of the lane lines along the road direction, obtain the vanishing point perpendicular to the road direction, and use the two orthogonal vanishing points and the camera installation height to obtain the focal length, pitch angle and yaw angle of the camera. According to the focal length, pitch angle and yaw angle of the camera, the external parameters of the camera are obtained.

[0072] In one embodiment, the calibration of the extrinsic camera parameters mainly involves transforming the coordinates in the world coordinate system into the pixel coordinate system. For example, the point cloud in the world coordinate system can be projected onto the image of a monocular camera for display, and the accuracy of the calibration can also be verified therefrom. For example, by setting the extrinsic parameter matrix camRRoad, the two-dimensional coordinates (u, v) of the pixel are obtained. K is the known correlation coefficient of the monocular camera, and P is the pixel world coordinate system. According to the conversion formula (1) between the world coordinate system and the two-dimensional coordinate system, the extrinsic parameter matrix can be obtained.

[0073]

[0074] In one embodiment, constraints such as the lane lines being parallel to each other, being consistent with the road driving direction, and having a lane line interval of 3.5 meters can be used to fit and solve the extrinsic camera parameters in formula (1). Among them, by taking two pixels at intervals on the lane line, their actual distance and pixel distance can be obtained, and then formula (1) can be solved to obtain the extrinsic parameter matrix.

[0075] After obtaining the extrinsic parameter matrix in one embodiment, further processing of the extrinsic parameter matrix is required, specifically including: setting a preset index for the extrinsic parameter matrix. If the extrinsic parameter matrix meets the preset index, the calibration of the road picture is successful, and the result is scored, and the lane distribution information of the road picture and the corresponding score of the road picture are saved; otherwise, the calibration of the road picture fails, and the road picture is discarded. For the scoring results that do not meet the requirements, this road picture should be discarded, otherwise it will affect the quality of the road picture.

[0076] Optionally, this index is used to obtain the optimal extrinsic camera parameters in each group of road pictures, specifically including, as Figure 3 shown:

[0077] Step S301: Score the extrinsic camera parameters of each group of road pictures according to a preset evaluation algorithm, and delete the extrinsic camera parameters whose evaluation results are less than the preset score threshold;

[0078] Deleting the extrinsic camera parameters less than the preset score threshold in step S301 is to reduce the influence of the extrinsic camera parameters that do not meet the quality requirements on the subsequent process. The preset evaluation algorithm is generally an accuracy algorithm, which scores the road pictures based on the pixel accuracy of the road pictures. Optionally, the pixel accuracy of the road pictures can be obtained according to the pixel interval and the actual interval between the lane lines, and the road pictures can be scored according to the pixel accuracy.

[0079] Step S302: Select the extrinsic camera parameter with the highest score in each group of road pictures as the optimal extrinsic camera parameter in each group of road pictures.

[0080] In one embodiment, step S302 may sort the extrinsic camera parameters in descending order and select the extrinsic camera parameter with the highest ranking as the optimal extrinsic camera parameter.

[0081] Step S103: According to a preset rule, group the obtained road pictures based on the distribution of lane lines in the road pictures, and obtain the optimal extrinsic camera parameter for each group of road pictures. The optimal extrinsic camera parameter is the extrinsic camera parameter with the highest score obtained based on a preset evaluation algorithm.

[0082] The preset rule includes: grouping the road driving pictures according to the lane distribution information, and the lane distribution information includes: the number of lane lines, the left-side distribution of lane lines during road driving, the right-side distribution of lane lines during road driving, and the bilateral distribution of lane lines during road driving.

[0083] Before step S103, it is also necessary to identify the lane lines in the road picture and record their lane line layouts, which specifically includes: determining the central position of the road picture, recording the lane lines to the left of the center of the road picture in the horizontal direction (i.e., the left-side lane lines) and the lane lines to the right of the center of the road picture in the horizontal direction (i.e., the right-side lane lines), and respectively recording the number of left-side lane lines and right-side lane lines. The sum of the left-side lane lines and the right-side lane lines is the number of lane strips in this distribution.

[0084] Group the obtained road pictures according to the preset rule based on the distribution of lane lines in the road pictures, which specifically includes: grouping the road driving pictures according to the lane distribution information.

[0085] In one embodiment, obtain the number of left-side lane lines and the number of right-side lane lines of the road picture, and group the pictures with the same number of left-side lane lines and right-side lane lines into one group.

[0086] In one embodiment, after grouping the road pictures, calculate the weight of each group of road pictures among all road pictures. Only when the weight of each group of road pictures is within the preset weight range, perform step S104 to jointly calibrate multiple groups of pictures. If the weight of a group of road pictures is greater than the preset weight range, select the optimal extrinsic camera parameter of the road picture with the largest weight as the extrinsic camera parameter of the camera.

[0087] Step S104: Arbitrarily select two groups of optimal extrinsic camera parameters from the optimal extrinsic camera parameters of each group of road pictures for joint calibration, and determine the optimal joint calibration parameter among all the joint calibration parameter results as the target calibration parameter of the camera. The optimal joint calibration parameter is the joint calibration parameter with the highest score obtained based on a preset evaluation algorithm.

[0088] Specifically, step S104 includes: randomly selecting two groups of road pictures from each group of road pictures, and using the road pictures corresponding to the optimal external camera parameters in the two groups of road pictures as the calibrated road picture combination; obtaining the effective road picture combination in the calibrated road picture combination according to the preset combination conditions; extracting the lane line interval data of each road picture in the effective road picture combination, and using the lane line interval distance in the effective road picture combination and the actual interval distance of the lane lines to obtain the joint calibrated external camera parameters of the effective road picture combination; scoring all the joint calibrated external camera parameters according to the preset evaluation algorithm, and determining the optimal joint calibrated external camera parameter among all the joint calibrated external camera parameter results as the target calibrated external camera parameter of the camera.

[0089] Among them, obtaining the effective road picture combination in the calibrated road picture combination according to the preset combination conditions includes: obtaining the total number of lane lines in each road picture in the group; obtaining the maximum number of left lane lines in the grouped road pictures and the maximum number of right lane lines in the grouped road pictures; if the total number of lane lines in any road picture is not less than the sum of the maximum number of left lane lines and the maximum number of right lane lines, then discard the road picture combination.

[0090] In one embodiment, it is also necessary to score the optimal external camera parameters of the joint calibration. If the scoring result meets the preset scoring threshold, then select the optimal external camera parameters. If the scoring result does not meet the preset scoring threshold, then discard the optimal external camera parameters. From step S103, select the external camera parameters of the road picture with the highest score among the optimal external camera parameters in each group of road pictures as the external camera parameters of the camera.

[0091] In one embodiment, determine the weight information of each group of road driving pictures according to the number of road driving pictures and the number of road pictures in each group of road driving pictures. That is, obtain the probability of each group of road pictures appearing in this calibrated scene. Determine the joint distribution density function between each group of road driving pictures and the external camera parameters according to the weight information of each group of road driving pictures and the external camera parameters of each group of road driving pictures. Specifically, obtain the joint distribution probability function of each group of road pictures according to the road picture weight and the road picture external camera parameters. In the subsequent application process, when the joint distribution is required, directly obtain the automatic calibration of the external camera parameters of the road pictures according to the joint distribution function and the road picture grouping.

[0092] In one embodiment, for the joint distribution function, multiple extrinsic parameters of road pictures in the joint distribution group can be fitted to obtain the automatic calibration of the extrinsic parameters under this grouping. The joint calibration method is similar to the single road picture calibration method, and the difference is that multiple road pictures are used to perform fitting operations on the camera extrinsic parameters. According to the extrinsic parameters of each group of road driving pictures and the joint distribution density function, the jointly calibrated extrinsic parameters are obtained. Specifically, it includes: for the calibration results that meet the specified indicators, respectively record the highest scoring results of different lane distribution information, the road pictures corresponding to the highest scoring results, and the total number of road pictures solved, and for the highest scoring results corresponding to different lane distribution information obtained, if the number of lane distribution information reaching or exceeding the specified ratio is greater than one, perform joint calibration of different lane distribution information; otherwise, select the highest scoring result as the final result.

[0093] This application selects road driving pictures that meet the preset conditions for automatic calibration of extrinsic parameters. By automatically identifying the lane lines in the road pictures and using the lane line interval data in the road pictures and the actual lane line interval data to obtain the extrinsic parameter information of the photo, the calibration of the camera is realized. This method does not require manual participation, can meet the needs of multiple scenarios, and improves the calibration efficiency.

[0094] On the other hand, the present invention takes into account the calibration road conditions, performs grouped calibration on road pictures according to the distribution of different lane lines, and performs single calibration on pictures whose lane line conditions meet a single grouping. For those whose lane lines do not meet a single grouping, multiple road driving pictures are used for joint calibration, thereby expanding the application range of the calibrated extrinsic parameters.

[0095] Figure 4 It is a schematic structural diagram of the automatic calibration device for the extrinsic parameters of the camera shown in the embodiment of the present application.

[0096] See Figure 4 , Figure 4 It includes a first unit image processing unit 401, a second image processing unit 402, an evaluation unit 403, and a determination unit 404;

[0097] The first image processing unit 401 is used to obtain at least two road pictures taken by the vehicle camera under preset conditions;

[0098] The second image processing unit 402 is used to identify the lane lines of the road pictures, and use the lane line interval distance in the road pictures and the actual interval distance of the lane lines to fit and obtain the camera extrinsic parameters corresponding to each road picture;

[0099] An evaluation unit 403 is configured to group the obtained road pictures according to the distribution of lane lines in the road pictures according to a preset rule, and obtain the optimal extrinsic parameters of the camera in each group of road pictures. The optimal extrinsic parameters of the camera are the extrinsic parameters of the camera with the highest score obtained based on a preset evaluation algorithm;

[0100] A determination unit 404 is configured to randomly select two sets of optimal extrinsic parameters of the camera from the optimal extrinsic parameters of each group of road pictures for joint calibration, and determine the optimal joint calibration extrinsic parameter as the target calibration extrinsic parameter of the camera from all the joint calibration extrinsic parameter results. The optimal joint calibration extrinsic parameter is the joint calibration extrinsic parameter with the highest score obtained based on a preset evaluation algorithm.

[0101] A first unit 401 is configured to obtain road driving road pictures that meet preset conditions and determine the extrinsic parameters of the road driving road pictures.

[0102] Regarding the device in the above embodiments, the specific manners in which each module performs operations have been described in detail in the embodiments related to the method, and will not be elaborated here.

[0103] Figure 5 It is a schematic structural diagram of an electronic device shown in an embodiment of the present application.

[0104] See Figure 5 , the electronic device 500 includes a memory 510 and a processor 520.

[0105] The processor 520 may be a central processing unit (CPU), or may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0106] The memory 510 may include various types of storage units, such as system memory, read-only memory (ROM), and permanent storage devices. Among them, the ROM may store static data or instructions required by the processor 520 or other modules of the computer. The permanent storage device may be a readable and writable storage device. The permanent storage device may be a non-volatile storage device that does not lose the stored instructions and data even when the computer is powered off. In some embodiments, the permanent storage device employs a mass storage device (such as a magnetic or optical disk, flash memory) as the permanent storage device. In some other embodiments, the permanent storage device may be a removable storage device (such as a floppy disk, optical drive). The system memory may be a readable and writable storage device or a volatile readable and writable storage device, such as dynamic random access memory. The system memory may store some or all of the instructions and data required by the processor during operation. In addition, the memory 510 may include any combination of computer-readable storage media, including various types of semiconductor storage chips (such as DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), and magnetic disks and / or optical disks may also be employed. In some embodiments, the memory 510 may include a removable storage device that is readable and / or writable, such as a compact disc (CD), read-only digital versatile disc (such as DVD-ROM, dual-layer DVD-ROM), read-only Blu-ray disc, super density disc, flash memory card (such as SD card, min SD card, Micro-SD card, etc.), magnetic floppy disk, etc. The computer-readable storage media do not include carrier waves and instantaneous electronic signals transmitted wirelessly or wired.

[0107] Executable code is stored on the memory 510, and when the executable code is processed by the processor 520, it may cause the processor 520 to execute some or all of the methods described above.

[0108] In addition, the method according to the present application may also be implemented as a computer program or a computer program product, which includes computer program code instructions for executing some or all of the above steps of the method according to the present application.

[0109] Alternatively, the present application may also be implemented as a computer-readable storage medium (or a non-transitory machine-readable storage medium or a machine-readable storage medium), on which executable code (or a computer program or computer instruction code) is stored. When the executable code (or the computer program or computer instruction code) is executed by a processor of an electronic device (or a server, etc.), it causes the processor to execute some or all of the steps of the above method according to the present application.

[0110] The embodiments of the present application have been described above. The above description is exemplary and not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The choice of terms used herein is intended to best explain the principles of the embodiments, the practical application, or the improvement of the technology in the market, or to enable other ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. An automatic calibration method for the external parameters of a camera, characterized in that, it includes: Obtain at least two road pictures taken by a vehicle camera under preset conditions; Identify the lane lines in the road pictures, and use the interval distance between the lane lines in the road pictures and the actual interval distance of the lane lines to fit and obtain the external parameters of the camera corresponding to each road picture; According to a preset rule, group the obtained road pictures according to the distribution of the lane lines in the road pictures, and obtain the optimal external parameters of the camera in each group of road pictures. The optimal external parameters of the camera are the external parameters of the camera with the highest score obtained based on a preset evaluation algorithm; Select any two groups of optimal external parameters of the camera from each group of road pictures for joint calibration, and determine the optimal joint calibration external parameter among all the joint calibration external parameter results as the target calibration external parameter of the camera. The optimal joint calibration external parameter is the joint calibration external parameter with the highest score obtained based on the preset evaluation algorithm; It includes: Select any two groups of road pictures from each group of road pictures, and use the road pictures corresponding to the optimal external parameters of the two groups of road pictures as the calibration road picture combination. Obtain the effective road picture combination in the calibration road picture combination, and obtain the joint calibration external parameter of the effective road picture combination. Determine the optimal joint calibration external parameter among all the joint calibration external parameter results as the target calibration external parameter of the camera.

2. The method according to claim 1, characterized in that, the preset conditions include: the heading angle of the vehicle is within a preset heading angle range, the driving speed of the vehicle is within a specified speed range, and the driving length of the vehicle is greater than the target length.

3. The method according to claim 1, characterized in that, the identifying of the lane lines further includes: Obtain the number of lane lines in the current road picture. If the number of lane lines that converge at the same vanishing point is less than three, delete all the lane lines identified in the current road picture.

4. The method according to claim 1, characterized in that, the identifying of the lane lines in the road pictures, and using the interval distance between the lane lines in the road pictures and the actual interval distance of the lane lines to fit and obtain the external parameters of the camera corresponding to each road picture includes: Read the interval data of the lane lines in each road picture, and establish the coordinate system equation of the road picture corresponding to the lane lines; According to the actual interval data of the lane lines, establish the coordinate system equation of the world corresponding to the lane lines; According to the coordinate system equation of the road picture corresponding to the lane lines and the coordinate system equation of the world corresponding to the lane lines, obtain the external parameters of the camera corresponding to each road picture.

5. The method according to claim 1, characterized in that, the obtaining of the optimal external parameters of the camera in each group of road pictures includes: According to the preset evaluation algorithm, score the external parameters of the camera in each group of road pictures, and delete the external parameters of the camera whose score results are less than the preset score threshold; Select the external parameter of the camera with the highest score in each group of road pictures as the optimal external parameter of the camera in each group of road pictures.

6. The method according to claim 1, characterized in that, Selecting any two sets of road pictures from each set of road pictures, and using the road pictures corresponding to the optimal extrinsic camera parameters in the two sets of road pictures as the calibrated road picture combination, obtaining the effective road picture combination in the calibrated road picture combination, and obtaining the joint calibrated extrinsic parameters of the effective road picture combination, and determining the optimal joint calibrated extrinsic parameter as the target calibrated extrinsic parameter of the camera among all the joint calibrated extrinsic parameter results, including: Selecting any two sets of road pictures from each set of road pictures, and using the road pictures corresponding to the optimal extrinsic camera parameters in the two sets of road pictures as the calibrated road picture combination; Obtaining the effective road picture combination in the calibrated road picture combination according to the preset combination conditions; Extracting the lane line interval data of each road picture in the effective road picture combination, and using the lane line interval distance in the effective road picture combination and the actual interval distance of the lane lines to obtain the joint calibrated extrinsic parameters of the effective road picture combination; Scoring all the joint calibrated extrinsic parameters according to the preset evaluation algorithm, and determining the optimal joint calibrated extrinsic parameter as the target calibrated extrinsic parameter of the camera among all the joint calibrated extrinsic parameter results.

7. The method according to claim 6, wherein, the obtaining the effective road picture combination in the calibrated road picture combination according to the preset combination conditions includes: Obtaining the total number of lane lines in each road picture in the group; Obtaining the maximum number of left lane lines in the grouped road pictures and the maximum number of right lane lines in the grouped road pictures; If the total number of lane lines in any road picture is not less than the sum of the maximum number of left lane lines and the maximum number of right lane lines, then discard the road picture combination.

8. An automatic calibration device for the extrinsic parameters of a camera, wherein, it includes: A first image processing unit for obtaining at least two road pictures taken by a vehicle camera under preset conditions; A second image processing unit for identifying the lane lines of the road pictures, and using the lane line interval distance in the road pictures and the actual interval distance of the lane lines to fit and obtain the extrinsic camera parameters corresponding to each road picture; An evaluation unit for grouping the obtained road pictures according to the distribution of the lane lines in the road pictures according to preset rules, and obtaining the optimal extrinsic camera parameters in each group of road pictures, and the optimal extrinsic camera parameters are the extrinsic camera parameters with the highest score obtained based on a preset evaluation algorithm; A determination unit for jointly calibrating any two sets of the optimal extrinsic camera parameters in each group of road pictures, and determining the optimal joint calibrated extrinsic parameter as the target calibrated extrinsic parameter of the camera among all the joint calibrated extrinsic parameter results, and the optimal joint calibrated extrinsic parameter is the joint calibrated extrinsic parameter with the highest score obtained based on the preset evaluation algorithm; The determination unit is used to select any two sets of road pictures from each set of road pictures, and use the road pictures corresponding to the optimal extrinsic camera parameters in the two sets of road pictures as the calibrated road picture combination, obtain the effective road picture combination in the calibrated road picture combination, and obtain the joint calibrated extrinsic parameters of the effective road picture combination, and determine the optimal joint calibrated extrinsic parameter as the target calibrated extrinsic parameter of the camera among all the joint calibrated extrinsic parameter results.

9. An electronic device, It is characterized in that including a processor and a memory storing executable code which, when executed by the processor, causes the processor to execute the method according to any one of claims 1-7.

10. A computer-readable storage medium storing executable code which, when executed by a processor of an electronic device, causes the processor to execute the method according to any one of claims 1-7.

Images