Camera-laser relative external parameter quality evaluation method, device, equipment and medium

Abstract

The invention provides a camera-laser relative external parameter quality evaluation method and device, equipment and a medium. The method comprises the following steps: acquiring a camera segmentation result and a laser segmentation result of a vehicle; the camera segmentation result is used for representing a point set of a vehicle shot by a camera in an image, and the laser segmentation result is used for representing a point cloud coordinate set generated after the vehicle is detected by a laser sensor; determining a camera geometric center and a laser geometric center based on the camera segmentation result and the laser segmentation result; constructing an evaluation function according to the camera geometric center and the laser geometric center; and according to the evaluation function, the camera-laser relative external parameter quality is evaluated, and the relative external parameter initial value is optimized by adopting a preset algorithm. According to the scheme, the time cost and the labor cost for checking and correcting the relative external parameters of the camera-laser sensor of the automatic driving vehicle are reduced, and the relative external parameter quality evaluation efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular, to a method, device, equipment and medium for evaluating the quality of the relative external parameters of a camera and a laser. Background Art

[0002] With the continuous development of vehicle intelligent technology, an autonomous driving system has emerged. In the autonomous driving system, the vehicle's environmental perception system has become the core and key part. As important components of it, vehicle cameras and laser sensors each play a unique role. The camera can obtain rich visual texture information and has significant advantages in target recognition, traffic sign and marking detection, etc.; the laser sensor can accurately measure distances and quickly obtain the three-dimensional spatial information of objects, and can effectively construct a point cloud map of the vehicle's surrounding environment.

[0003] During the autonomous driving process, in order to achieve accurate target positioning, path planning and decision-making control, it is necessary to efficiently fuse the data of both the camera and the laser sensor. The relative external parameters of the vehicle camera-laser sensor refer to the parameters used to describe the relative position and attitude relationship between the camera coordinate system and the laser sensor coordinate system on the vehicle, usually including two parts: rotation and translation. Therefore, in order to ensure the safety of vehicle autonomous driving, it is particularly important to evaluate the quality of the relative external parameters of the vehicle's camera-laser sensor.

[0004] Currently, in the related art, a manual calibration method is used to evaluate the quality of the relative external parameters of the vehicle camera-laser sensor. However, this method is not applicable to the inspection and optimization of the relative external parameters of a large number of individual autonomous driving vehicle cameras and laser sensors, and requires high time cost and labor cost, resulting in low efficiency of relative external parameter quality evaluation. Summary of the Invention

[0005] In an embodiment of the present application, a method, device, equipment and medium for evaluating the quality of the relative external parameters of a camera and a laser are provided.

[0006] In a first aspect of the embodiments of the present application, a method for evaluating the quality of the relative external parameters of a camera and a laser is provided. The method includes:

[0007] Obtain the camera segmentation result and the laser segmentation result of the vehicle; the camera segmentation result is used to represent the set of points of the vehicle in the image captured by the camera, and the laser segmentation result is used to represent the set of point cloud coordinates generated after the vehicle is detected by the laser sensor;

[0008] Based on the camera segmentation result and the laser segmentation result, determine the camera geometric center and the laser geometric center;

[0009] Construct an evaluation function based on the camera geometric center and the laser geometric center;

[0010] Evaluate the quality of the relative extrinsic parameters between the camera and the laser according to the evaluation function and optimize the initial value of the relative extrinsic parameters by using a preset algorithm.

[0011] In an alternative embodiment of the present application, the camera geometric center includes a first camera point mean and a second camera point mean, and the laser geometric center includes a first laser point mean and a second laser point mean;

[0012] Based on the camera segmentation result and the laser segmentation result, determining the initial value of the relative extrinsic parameters, the camera geometric center, and the laser geometric center between the camera and the laser sensor of the vehicle includes:

[0013] Perform a mean process on the camera segmentation result to obtain the first camera point mean and the second camera point mean;

[0014] Perform a mean process on the laser segmentation result to obtain the first laser point mean and the second laser point mean.

[0015] In an alternative embodiment of the present application, the initial value of the relative extrinsic parameters includes: yaw angle, pitch angle, and roll angle between the camera and the laser sensor; and constructing the evaluation function according to the camera geometric center and the laser geometric center includes:

[0016] Calculate a first Euclidean distance and a second Euclidean distance according to the first camera point mean, the first laser point mean, the second camera point mean, and the second laser point mean;

[0017] Determine a first residual and a second residual based on the first Euclidean distance and the second Euclidean distance; the first residual is used to constrain the deviation in the yaw angle and pitch angle dimensions; the second residual is used to constrain the threshold of the first Euclidean distance and the second Euclidean distance;

[0018] Calculate a third residual according to the first camera point mean, the first laser point mean, the second camera point mean, and the second laser point mean; the third residual is used to constrain the deviation in the roll angle dimension;

[0019] Sum up the first residual, the second residual, and the third residual to obtain the evaluation function.

[0020] In an alternative embodiment of the present application, calculating the first Euclidean distance and the second Euclidean distance according to the first camera point mean, the first laser point mean, the second camera point mean, and the second laser point mean includes:

[0021] Calculate a first Euclidean distance based on the horizontal and vertical coordinates of the first camera point mean and the horizontal and vertical coordinates of the first laser point mean;

[0022] Calculate a second Euclidean distance based on the horizontal and vertical coordinates of the second camera point mean and the horizontal and vertical coordinates of the second laser point mean.

[0023] In an alternative embodiment of the present application, determining a first residual and a second residual based on the first Euclidean distance and the second Euclidean distance includes:

[0024] Take the sum of the first Euclidean distance and the second Euclidean distance as the first residual;

[0025] Calculate the distance difference between the first Euclidean distance and the second Euclidean distance, and perform a square calculation process on the distance difference to obtain the second residual.

[0026] In an alternative embodiment of the present application, calculating a third residual based on the first camera point mean, the first laser point mean, the second camera point mean, and the second laser point mean includes:

[0027] Construct a first straight line based on the first camera point mean and the second camera point mean;

[0028] Construct a second straight line based on the first laser point mean and the second laser point mean;

[0029] Calculate the included angle between the first straight line and the second straight line, and take the included angle as the third residual.

[0030] In an alternative embodiment of the present application, evaluating the camera-laser relative extrinsic parameter quality according to the evaluation function and optimizing the initial value of the relative extrinsic parameter by using a preset algorithm includes:

[0031] Evaluate the camera-laser relative extrinsic parameter quality and optimize the initial value of the relative extrinsic parameter by using a preset algorithm, and determine whether the evaluation function meets the iterative convergence condition;

[0032] When the evaluation function meets the preset convergence condition, take the relative extrinsic parameter corresponding to the evaluation function as the optimal relative extrinsic parameter value.

[0033] A second aspect of the embodiments of the present application provides a camera-laser relative extrinsic parameter quality evaluation device, including:

[0034] An acquisition module, configured to acquire a camera segmentation result and a laser segmentation result of a vehicle; the camera segmentation result is used to represent a set of points of the vehicle in an image captured by a camera, and the laser segmentation result is used to represent a set of point cloud coordinates generated after the vehicle is detected by a laser sensor;

[0035] A determination module, configured to determine a camera geometric center and a laser geometric center based on the camera segmentation result and the laser segmentation result;

[0036] A construction module, configured to construct an evaluation function according to the camera geometric center and the laser geometric center;

[0037] An optimization module, configured to evaluate the quality of the relative extrinsic parameters between the camera and the laser according to the evaluation function and optimize the initial value of the relative extrinsic parameters by using a preset algorithm.

[0038] In a third aspect of the embodiments of the present application, a computer device is provided, including: a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, the steps of any one of the above methods are implemented.

[0039] In a fourth aspect of the embodiments of the present application, a computer-readable storage medium is provided, on which a computer program is stored, and characterized in that when the computer program is executed by a processor, the steps of any one of the above methods are implemented.

[0040] In the embodiments of the present application, a method for evaluating the quality of relative extrinsic parameters between a camera and a laser is provided. The method includes: acquiring a camera segmentation result and a laser segmentation result of a vehicle, where the camera segmentation result is used to represent a set of points of the vehicle in an image captured by a camera, and the laser segmentation result is used to represent a set of point cloud coordinates generated after the vehicle is detected by a laser sensor; determining a camera geometric center and a laser geometric center based on the camera segmentation result and the laser segmentation result; constructing an evaluation function according to the camera geometric center and the laser geometric center; evaluating the quality of the relative extrinsic parameters between the camera and the laser according to the evaluation function and optimizing the initial value of the relative extrinsic parameters by using a preset algorithm. The technical solution in the present application can provide data guiding information for the subsequent quality evaluation of the relative extrinsic parameters between the camera and the laser sensor by acquiring the camera segmentation result and the laser segmentation result of the vehicle, accurately determine the camera geometric center and the laser geometric center, and quantify the quality of the relative extrinsic parameters between the camera and the laser sensor from all dimensions of Euler angles according to the camera geometric center and the laser geometric center, provide constraint conditions for the automatic optimization of the relative extrinsic parameters between the camera and the laser sensor, thereby constructing an evaluation function to realize the evaluation of the quality of the relative extrinsic parameters between the camera and the laser, and further optimizing the initial value of the relative extrinsic parameters according to the evaluation function, reducing the time cost and labor cost of checking and correcting the relative extrinsic parameters between the camera and the laser sensor of the autonomous driving vehicle, and further improving the efficiency of relative extrinsic parameter quality evaluation. Brief Description of the Drawings

[0041] The drawings described herein are provided to further understand the present application and form a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not unduly limit the present application. In the drawings:

[0042] Figure 1 is a schematic structural diagram of a computer device provided by an embodiment of the present application;

[0043] Figure 2 is a flowchart of a method for evaluating the relative extrinsic parameter quality of a camera-laser sensor provided by an embodiment of the present application;

[0044] Figure 3 is a schematic flowchart of a method for constructing an evaluation function provided by an embodiment of the present application;

[0045] Figure 4 is a schematic diagram of the comparison effect before and after optimization by the method provided by the present application at different distances provided by an embodiment of the present application;

[0046] Figure 5 is a schematic structural diagram of a device for evaluating the relative extrinsic parameter quality of a camera-laser provided by an embodiment of the present application. Detailed Description of the Embodiments

[0047] In the process of implementing the present application, the inventors found that the traditional manual calibration method cannot be applied to the inspection and optimization of the relative extrinsic parameters of the camera-laser sensors of a large number of individual autonomous vehicles, which requires high time and labor costs and results in low efficiency in evaluating the relative extrinsic parameter quality.

[0048] In order to make the technical solutions and advantages in the embodiments of the present application clearer, the following further details the exemplary embodiments of the present application with reference to the drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than an exhaustive list of all embodiments. It should be noted that, without conflict, the embodiments in the present application and the features in the embodiments can be combined with each other.

[0049] As mentioned in the background art, in order to achieve efficient fusion of camera and lidar sensor data in a vehicle, accurately determining the relative extrinsic parameters between the vehicle camera and lidar sensor is an indispensable prerequisite. These relative extrinsic parameters cover rotation parameters and translation parameters, and precisely describe the position and attitude relationship between the camera coordinate system and the lidar sensor coordinate system. Currently, in related technologies, one way is to use the method in a calibration room to evaluate the quality of the relative extrinsic parameters of the vehicle's camera-lidar sensor; another way is to use manual calibration to evaluate the relative extrinsic parameters. However, both of these two methods are not applicable to the inspection and optimization of the relative extrinsic parameters of a large number of individual autonomous vehicle camera-lidar sensors, which requires high time cost and labor cost, resulting in low efficiency of relative extrinsic parameter quality evaluation.

[0050] Based on the above defects, the present application provides a method for evaluating the quality of camera-lidar relative extrinsic parameters. Compared with related technologies, the technical solution in the present application can provide data guidance information for the subsequent quality evaluation of the relative extrinsic parameters between the camera and the lidar sensor by obtaining the camera segmentation result and the lidar segmentation result of the vehicle, and accurately determine the camera geometric center and the lidar geometric center. And according to the camera geometric center and the lidar geometric center, the quality of the relative extrinsic parameters of the camera-lidar sensor is quantified from all dimensions of Euler angles, providing constraint conditions for the automatic optimization of the relative extrinsic parameters of the camera-lidar sensor, thereby constructing an evaluation function, and then evaluating the quality of the camera-lidar relative extrinsic parameters according to the evaluation function, optimizing the initial value of the relative extrinsic parameters, reducing the time cost and labor cost of inspecting and correcting the relative extrinsic parameters of the autonomous vehicle camera-lidar sensor, and further improving the efficiency of relative extrinsic parameter quality evaluation.

[0051] Among them, the solution in the embodiment of the present application can be implemented in various computer languages. For example, object-oriented programming languages such as Java and interpreted scripting languages such as JavaScript.

[0052] Please refer to Figure 1 , a schematic structural diagram of a computer device provided in an embodiment of the present application. As Figure 1As shown in the figure, the computer device includes a processor, a memory, a network interface, a display screen, and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium can be, for example, a disk. Files (which can be files to be processed or processed files), an operating system, computer programs, etc. are stored in the non-volatile storage medium. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, it realizes a method for evaluating the relative external parameters quality of a camera-laser. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a button, a trackball, or a touchpad set on the computer device housing, or an external keyboard, touchpad, or mouse, etc.

[0053] Please refer to Figure 2 , taking the above computer device as the execution subject in the following embodiments, and taking the application of the method for evaluating the relative external parameters quality of a camera-laser provided in the embodiments of the present application to the above computer device for data processing as an example for specific description. The method for evaluating the relative external parameters quality of a camera-laser provided in the embodiments of the present application includes the following steps 201-step 204:

[0054] Step 201, obtain the camera segmentation result and the laser segmentation result of the vehicle; the camera segmentation result is used to characterize the set of points of the vehicle in the image captured by the camera, and the laser segmentation result is used to characterize the set of point cloud coordinates generated after the vehicle is detected by the laser sensor.

[0055] It should be noted that in the application scenario of autonomous driving, the vehicle can be photographed and detected through an environmental perception system. The environmental perception system can include, for example, a camera and a laser sensor. The vehicle is photographed through the camera to obtain an image, and the image is segmented to obtain the camera segmentation result. The camera segmentation result contains the set of points of the vehicle in the image and can be represented by a two-dimensional coordinate point set. And the vehicle is detected through the laser sensor to obtain laser point cloud data. The laser point cloud data can include the set of point cloud coordinates of the vehicle and can be represented by a three-dimensional point cloud coordinate set.

[0056] Optionally, the above-mentioned vehicles may include two vehicles, three vehicles, or more than three vehicles. Since the constraints of a single vehicle target on the relative extrinsic parameters (parameters to be optimized) between the camera and the lidar sensor are insufficient, for example, there is no constraint on the roll angle dimension, and three or more vehicles will bring a problem of large computational complexity. Therefore, in order to reduce the computational complexity while ensuring the binding force, the above-mentioned vehicles may preferably be two vehicle targets. The two vehicle targets need to be located on the left and right sides of the camera's field of view. Among them, when calculating the distance between the vehicle target and the sensor, the camera resolution and the lidar sensor resolution need to be comprehensively considered. For example, the camera resolution can be set to 1920*1080, the lidar sensor resolution can be set to 0.2° horizontally and 0.1° vertically, and the distance from the vehicle can be set to about 20 - 30m.

[0057] Among them, after the vehicle is photographed by the camera to generate an image, the vehicle target in the scene can be extracted and the image is segmented to obtain the camera segmentation result. After the vehicle is detected by the lidar sensor, the lidar point cloud data is obtained, and the lidar point cloud data is filtered to obtain the lidar segmentation result, which may include two vehicle targets.

[0058] Optionally, the above-mentioned camera segmentation result and lidar segmentation result of the vehicle may be obtained by importing through an external device, or obtained from a blockchain or a database, or a data acquisition request may be sent to the camera and the lidar sensor to obtain them in real time.

[0059] Furthermore, in this embodiment, the camera segmentation result of a certain vehicle can be obtained through the above-mentioned camera segmentation result, that is, the point set information of a certain vehicle in the image is confirmed; at the same time, the lidar segmentation result of a certain vehicle can be obtained through the above-mentioned lidar segmentation result, that is, the point cloud coordinate set of a certain vehicle is confirmed. Thus, good data guidance information is provided for the subsequent quality evaluation of the relative extrinsic parameters between the camera and the lidar sensor.

[0060] Step 202: Based on the camera segmentation result and the lidar segmentation result, determine the camera geometric center and the lidar geometric center.

[0061] It should be noted that the relative external parameters between the above camera and the laser sensor refer to the parameters used to describe the relative position and attitude relationship between the camera coordinate system and the laser sensor coordinate system, mainly including translation parameters and rotation parameters. The translation parameters and rotation parameters can be represented in the form of Euler angles, for example, including roll (roll angle), yaw (yaw angle), and pitch (pitch angle). In the calculation formula, they can be represented in matrix form, with the rotation parameter represented by R and the translation parameter represented by t. The camera geometric center refers to the geometric center in the camera segmentation result. When there are two vehicle targets, it can include the first camera point mean and the second camera point mean; the laser geometric center refers to the geometric center in the laser segmentation result. When there are two vehicle targets, it can include the first laser point mean and the second laser point mean.

[0062] Specifically, when there are two vehicle targets, taking vehicle 1 and vehicle 2 as examples respectively, the laser segmentation results of the two determined vehicle targets are the three-dimensional coordinates of the points corresponding to vehicle 1 and vehicle 2 in the laser point cloud data, and the camera segmentation results of the two vehicle targets are the point sets of the segmentation results of vehicle 1 and vehicle 2 on the image. After obtaining the camera segmentation result and the laser segmentation result, the camera geometric centers of vehicle 1 and vehicle 2 can be calculated according to the camera segmentation result, and the laser geometric centers of vehicle 1 and vehicle 2 can be calculated according to the laser segmentation result.

[0063] Step 203: Construct an evaluation function based on the camera geometric center and the laser geometric center.

[0064] It can be understood that the above evaluation function is a parameter for evaluating the quality of the relative external parameters between the vehicle's camera and laser sensor, and can be used as a constraint condition for optimization in the Euler angle dimension. Among them, the Euler angle dimension includes the yaw (yaw angle), pitch (pitch angle) dimension, and roll (roll angle). The constraint conditions for optimization in the Euler angle dimension include: the deviations in the yaw angle, pitch angle, and roll angle dimensions.

[0065] Specifically, after obtaining the camera geometric centers and laser geometric centers of vehicle 1 and vehicle 2, determine the Euclidean distances of vehicle 1 and vehicle 2 in the camera coordinate system and the laser coordinate system based on the camera geometric center and the laser geometric center, and then construct an evaluation function according to the Euclidean distances.

[0066] In this embodiment, based on the camera geometric center and the laser geometric center, an evaluation function can be accurately constructed, reducing the time cost and labor cost of checking and correcting the relative external parameters of the camera-laser sensor of the autonomous vehicle.

[0067] Step 204: Evaluate the quality of the camera-laser relative external parameters according to the evaluation function and optimize the initial value of the relative external parameters using a preset algorithm.

[0068] The above preset algorithm can be an iterative algorithm customized in advance according to actual requirements.

[0069] In this embodiment, after obtaining the evaluation function, Ceres can be used to construct an optimization problem, and the quantity to be optimized is the initial value of the relative extrinsic parameters between the camera and the lidar sensor. Specifically, according to the evaluation function, the Levenberg-Marquardt (LM) algorithm can be used to optimize the initial value of the relative extrinsic parameters, so as to determine the optimal relative extrinsic parameter value to evaluate the quality of the relative extrinsic parameters between the camera and the lidar. The LM algorithm combines the advantages of the gradient descent method and the Gauss-Newton method. In each iteration, a regularized linear equation system is solved to update the relative extrinsic parameters.

[0070] An embodiment of the present application provides a method for evaluating the quality of relative extrinsic parameters between a camera and a lidar, the method including: obtaining the camera segmentation result and the lidar segmentation result of the vehicle, where the camera segmentation result is used to characterize the set of points of the vehicle in the image captured by the camera, and the lidar segmentation result is used to characterize the set of point cloud coordinates generated after the vehicle is detected by the lidar sensor; determining the camera geometric center and the lidar geometric center based on the camera segmentation result and the lidar segmentation result; constructing an evaluation function according to the camera geometric center and the lidar geometric center; evaluating the quality of the relative extrinsic parameters between the camera and the lidar according to the evaluation function, and optimizing the initial value of the relative extrinsic parameters by using a preset algorithm. The technical solution in the present application can provide data guiding information for the subsequent quality evaluation of the relative extrinsic parameters between the camera and the lidar sensor by obtaining the camera segmentation result and the lidar segmentation result of the vehicle, accurately determine the camera geometric center and the lidar geometric center, and quantify the quality of the relative extrinsic parameters between the camera and the lidar sensor from all dimensions of Euler angles according to the camera geometric center and the lidar geometric center, providing constraint conditions for the automatic optimization of the relative extrinsic parameters between the camera and the lidar sensor, thereby constructing an evaluation function to realize the evaluation of the quality of the relative extrinsic parameters between the camera and the lidar, and then optimizing the initial value of the relative extrinsic parameters according to the evaluation function, reducing the time cost and labor cost of checking and correcting the relative extrinsic parameters of the camera-lidar sensor of the autonomous vehicle, and further improving the efficiency of relative extrinsic parameter quality evaluation.

[0071] In an alternative embodiment of the present application, the camera geometric center includes a first camera point mean value and a second camera point mean value, and the lidar geometric center includes a first lidar point mean value and a second lidar point mean value. The above determining the camera geometric center and the lidar geometric center based on the camera segmentation result and the lidar segmentation result includes:

[0072] Performing a mean value process on the camera segmentation result to obtain a first camera point mean value and a second camera point mean value; performing a mean value process on the lidar segmentation result to obtain a first lidar point mean value and a second lidar point mean value.

[0073] Specifically, taking the vehicle including two as an example, namely vehicle 1 and vehicle 2, the laser segmentation result of vehicle 1 obtained is P1, the laser segmentation result of vehicle 2 is P2, and the camera segmentation result of vehicle 1 is P C1 , and the camera segmentation result of vehicle 2 is P C2 , and the internal parameter k and distortion parameter d of the camera are obtained, and the initial value of the relative external parameter is set. The initial value of the relative external parameter includes the rotation parameter R and the translation parameter t. Project the laser segmentation result of vehicle 1 onto the corresponding camera segmentation result, and project the laser segmentation result of vehicle 2 onto the corresponding camera segmentation result, which can be expressed by the following formula:

[0074]

[0075] where P1 is the laser segmentation result of vehicle 1, P2 is the laser segmentation result of vehicle 2, and P C1 is the camera segmentation result of vehicle 1, and P C2 is the camera segmentation result of vehicle 2, k is the internal parameter of the camera, d is the distortion parameter, R is the rotation parameter, and t is the translation parameter.

[0076] Based on the camera segmentation result and the laser segmentation result, calculate the geometric centers of P1, P2, P C1 , and P C2 respectively, that is, perform mean processing on the camera segmentation result P C1 of vehicle 1 to obtain the first camera point mean value, perform mean processing on the camera segmentation result P C2 of vehicle 2 to obtain the second camera point mean value, perform mean processing on the laser segmentation result P1 of vehicle 1 to obtain the first laser point mean value, and perform mean processing on the laser segmentation result P2 of vehicle 2 to obtain the second laser point mean value, which can be expressed by the following formula:

[0077]

[0078] where P1 is the laser segmentation result of vehicle 1, P2 is the laser segmentation result of vehicle 2, P C1 is the camera segmentation result of vehicle 1, P C2 is the camera segmentation result of vehicle 2, h1 is the first laser point mean value of vehicle 1, h2 is the second laser point mean value of vehicle 2, h C1 is the first camera point mean value of vehicle 1, and h C2 is the second camera point mean value of vehicle 2.

[0079] In this embodiment, by projecting the laser segmentation result onto the camera segmentation result and performing mean processing on the camera segmentation result, the first camera point mean and the second camera point mean are obtained. By performing mean processing on the laser segmentation result, the first laser point mean and the second laser point mean are obtained, which can accurately determine the geometric centers of the laser segmentation result and the camera segmentation result, providing more data guidance information for the construction of the subsequent evaluation function and facilitating the optimization of the relative extrinsic parameter initial value.

[0080] In an alternative embodiment of the present application, the relative extrinsic parameter initial value includes: the yaw angle, pitch angle, and roll angle between the camera and the laser sensor. Based on the camera geometric center and the laser geometric center, an evaluation function is constructed. Please refer to Figure 3 shown, including the following method steps:

[0081] Step 301: Calculate the first Euclidean distance and the second Euclidean distance according to the first camera point mean, the first laser point mean, the second camera point mean, and the second laser point mean.

[0082] Step 302: Based on the first Euclidean distance and the second Euclidean distance, determine the first residual and the second residual; the first residual is used to constrain the deviation in the yaw angle and pitch angle dimensions; the second residual is used to constrain the difference threshold between the first Euclidean distance and the second Euclidean distance.

[0083] Step 303: Calculate the third residual according to the first camera point mean, the first laser point mean, the second camera point mean, and the second laser point mean; the third residual is used to constrain the deviation in the roll angle dimension.

[0084] Step 304: Perform a summation process on the first residual, the second residual, and the third residual to obtain the evaluation function.

[0085] It should be noted that when considering the translation parameter and the rotation parameter in terms of the Euler angle dimension, that is, the relative extrinsic parameter initial value includes: the yaw angle, pitch angle, and roll angle between the camera and the laser sensor. The above-mentioned first Euclidean distance is the Euclidean distance between the first camera point mean and the first laser point mean, and the second Euclidean distance is the Euclidean distance between the second camera point mean and the second laser point mean. The purpose of the first residual is to constrain the sum of the first Euclidean distance and the second Euclidean distance and to constrain the deviation in the yaw angle and pitch angle dimensions. The purpose of the second residual is to constrain the difference between the first Euclidean distance and the second Euclidean distance, to avoid the two distances being too different, and to avoid over-optimizing one of the distance residuals. The purpose of the third residual is to constrain the deviation in the roll angle dimension.

[0086] Specifically, after determining the first camera point mean, the first laser point mean, the second camera point mean, and the second laser point mean, calculate the first Euclidean distance and the second Euclidean distance. By obtaining the horizontal and vertical coordinates of the first camera point mean, the horizontal and vertical coordinates of the first laser point mean, the horizontal and vertical coordinates of the second camera point mean, and the horizontal and vertical coordinates of the second laser point mean, then determine the first Euclidean distance based on the horizontal and vertical coordinates of the first camera point mean and the horizontal and vertical coordinates of the first laser point mean, and determine the second Euclidean distance based on the horizontal and vertical coordinates of the second camera point mean and the horizontal and vertical coordinates of the second laser point mean. According to the first Euclidean distance and the second Euclidean distance, determine the first residual and the second residual, and calculate the third residual based on the first camera point mean, the first laser point mean, the second camera point mean, and the second laser point mean. Then sum up the first residual e1, the second residual e2, and the third residual e3 to obtain the evaluation function e.

[0087] In this embodiment, by calculating the first Euclidean distance and the second Euclidean distance according to the first camera point mean, the first laser point mean, the second camera point mean, and the second laser point mean, and accurately determining the first residual and the second residual based on the first Euclidean distance and the second Euclidean distance, and then calculating the third residual according to the first camera point mean, the first laser point mean, the second camera point mean, and the second laser point mean, it is possible to comprehensively consider the factors of the first residual, the second residual, and the third residual, so that the evaluation function can be obtained more comprehensively, and the accuracy of the relative external parameter quality evaluation is improved.

[0088] In an alternative embodiment of the present application, calculating the first Euclidean distance and the second Euclidean distance according to the first camera point mean, the first laser point mean, the second camera point mean, and the second laser point mean includes the following method steps:

[0089] Calculate the first Euclidean distance according to the horizontal and vertical coordinates of the first camera point mean and the horizontal and vertical coordinates of the first laser point mean; calculate the second Euclidean distance according to the horizontal and vertical coordinates of the second camera point mean and the horizontal and vertical coordinates of the second laser point mean.

[0090] It should be noted that after obtaining the horizontal and vertical coordinates of the first camera point mean, the horizontal and vertical coordinates of the first laser point mean, the horizontal and vertical coordinates of the second camera point mean, and the horizontal and vertical coordinates of the second laser point mean, the first Euclidean distance can be calculated according to the horizontal and vertical coordinates of the first camera point mean and the horizontal and vertical coordinates of the first laser point mean, and the second Euclidean distance can be calculated according to the horizontal and vertical coordinates of the second camera point mean and the horizontal and vertical coordinates of the second laser point mean, which is represented by the following formula:

[0091]

[0092] where x_h1 is the abscissa of the first laser point mean, x_h c1Let \(x_{h1}\) be the abscissa of the mean value of the first camera points, \(y_{h1}\) be the ordinate of the mean value of the first laser points, and \(y_{h}\) c1 be the ordinate of the mean value of the first camera points, \(x_{h2}\) be the abscissa of the mean value of the second laser points, and \(x_{h}\) c2 be the abscissa of the mean value of the second camera points, \(y_{h2}\) be the ordinate of the mean value of the second laser points, and \(y_{h}\) c2 be the ordinate of the mean value of the second camera points, \(h1\_h\) c1 \_dis be the first Euclidean distance, \(h2\_h\) c2 \_dis be the second Euclidean distance.

[0093] In this embodiment, according to the abscissa and ordinate of the mean value of the first camera points and the abscissa and ordinate of the mean value of the first laser points, the first Euclidean distance can be calculated more accurately, and according to the abscissa and ordinate of the mean value of the second camera points and the abscissa and ordinate of the mean value of the second laser points, the second Euclidean distance can be accurately calculated, so that the first residual and the second residual can be determined more finely and comprehensively.

[0094] In an alternative embodiment of the present application, determining the first residual and the second residual based on the first Euclidean distance and the second Euclidean distance includes the following method steps:

[0095] Take the sum of the first Euclidean distance and the second Euclidean distance as the first residual; calculate the distance difference between the first Euclidean distance and the second Euclidean distance, and perform a square calculation process on the distance difference to obtain the second residual.

[0096] Specifically, after obtaining the first Euclidean distance and the second Euclidean distance, add the first Euclidean distance and the second Euclidean distance to obtain the first residual \(e1\), which can be represented by the following formula:

[0097] e1 = h1\_h c1 \_dis + h2\_h c2 \_dis;

[0098] where, h1\_h c1 \_dis is the first Euclidean distance, h2\_h c2 \_dis is the second Euclidean distance, and \(e1\) is the first residual.

[0099] Then calculate the distance difference between the first Euclidean distance and the second Euclidean distance, and perform a square calculation process on the distance difference to obtain the second residual, which can be represented by the following formula:

[0100] e2 = (h1\_h c1 -dis - h2\_h c2 -dis) * (h1 - h c1 -dis - h2 - h c2 -dis);

[0101] Among them, h1_h c1 _dis is the first Euclidean distance, h2_h c2 _dis is the second Euclidean distance, and e2 is the second residual.

[0102] It can be understood that the purpose of the above second residual is to constrain the difference between the first Euclidean distance and the second Euclidean distance from being too large, so as to avoid over-optimizing one of the distance residuals.

[0103] In this embodiment, through the first Euclidean distance and the second Euclidean distance, the first residual and the second residual can be accurately determined, providing a constraint condition for the automatic optimization of the relative external parameters of the camera-laser sensor, which is convenient for accurately constructing an evaluation function.

[0104] In an alternative embodiment of the present application, the above-mentioned method for calculating the first camera point mean, the first laser point mean, the second camera point mean, and the second laser point mean includes the following steps:

[0105] Construct a first straight line according to the first camera point mean and the second camera point mean; construct a second straight line according to the first laser point mean and the second laser point mean; calculate the included angle between the first straight line and the second straight line, and use the included angle as the third residual.

[0106] It can be understood that the above third residual can be the included angle between the first straight line and the second straight line. The first straight line is the straight line composed of the first camera point mean and the second camera point mean, and the second straight line is the straight line composed of the first laser point mean and the second laser point mean.

[0107] Specifically, obtain the abscissa and ordinate of the first camera point mean, the abscissa and ordinate of the second camera point mean, the abscissa and ordinate of the first laser point mean, and the abscissa and ordinate of the second laser point mean, and then determine the third residual e3 through the following formula:

[0108]

[0109] e3 = (a1 - a2) * 10

[0110] Among them, x_h1 is the abscissa of the first laser point mean, x_h c1 is the abscissa of the first camera point mean, y_h1 is the ordinate of the first laser point mean, y_h c1 is the ordinate of the first camera point mean, x_h2 is the abscissa of the second laser point mean, x_h c2 is the abscissa of the second camera point mean, y_h2 is the ordinate of the second laser point mean, y_h c2 is the ordinate of the second camera point mean.

[0111] Among them, the third residual is used to constrain the deviation in the roll dimension.

[0112] After determining the first residual, the second residual, and the third residual, the first residual, the second residual, and the third residual can be added together to obtain an evaluation function, which is represented by the following formula:

[0113] e = e1 + e2 + e3

[0114] Among them, e1 is the first residual, e2 is the second residual, e3 is the third residual, and e is the evaluation function.

[0115] In this embodiment, according to the first camera point mean and the second camera point mean, a first straight line is constructed, and according to the first laser point mean and the second laser point mean, a second straight line is constructed, so as to accurately calculate the included angle between the first straight line and the second straight line, and use the included angle as the third residual, improving the accuracy of determining the third residual and facilitating the more comprehensive construction of the evaluation function.

[0116] In an alternative embodiment of the present application, the camera-laser relative extrinsic parameter quality is evaluated according to the evaluation function and the relative extrinsic parameter initial value is optimized using a preset algorithm, including the following steps:

[0117] Evaluate the camera-laser relative extrinsic parameter quality and optimize the relative extrinsic parameter initial value using a preset algorithm, and determine whether the evaluation function meets the iterative convergence condition; when the evaluation function meets the preset convergence condition, use the relative extrinsic parameter corresponding to the evaluation function as the optimal relative extrinsic parameter value.

[0118] It should be noted that the above preset algorithm can be the LM algorithm. The LM algorithm is an optimization algorithm commonly used for nonlinear least squares problems and is very effective in optimizing the relative extrinsic parameters (rotation parameters and translation parameters) between the camera and the laser sensor. The above preset convergence condition can be custom-set according to actual requirements, for example, it can be to reach the maximum number of iterations or the change in the evaluation function is less than a certain threshold.

[0119] As an alternative implementation, the relative extrinsic parameter quality can be evaluated according to the evaluation function first, and the relative extrinsic parameter initial value is set, for example, including the rotation parameter initial value R0 and the translation parameter initial value t0, and a damping factor is set, which can be set to a small positive number, such as 0.001, and a linear equation system is constructed according to the damping factor to update the rotation parameter initial value and the translation parameter initial value, and then the evaluation function is evaluated, for example, to determine whether the updated evaluation function meets the convergence condition. When the convergence condition is not met, continue to perform iterative operations to iteratively update the current rotation parameter and translation parameter, and continue to perform the convergence condition judgment process; when the convergence condition is met, do not perform iterative operations, and use the rotation parameter and translation parameter at this time as the optimal relative extrinsic parameter value.

[0120] After obtaining the optimal relative external parameter values, the laser segmentation results can be projected onto the camera segmentation results using the optimized external parameter values, and then the misaligned area between the projected area of the point cloud in the laser segmentation results and the camera segmentation results can be observed. If the misaligned area is within the acceptable range, it indicates that the effectiveness of the evaluation function is valid; if the misaligned area is not within the acceptable range, it indicates that the effectiveness of the evaluation function is invalid.

[0121] Exemplarily, taking two vehicle targets, with the rotation parameter being a rotation matrix and the translation parameter being a translation matrix as an example, the camera segmentation results are obtained. The camera segmentation results include the regional information of the vehicle targets, and two point cloud sets that can form a cuboid are set, which simulate the laser segmentation results of two actual vehicle targets. The laser segmentation results are the vehicle point cloud sets, and the identity matrix is used as the initial value of the rotation matrix and the initial value of the translation parameter. The setting principle of the translation parameter is to increase the distance between the point cloud and the camera step by step; a fixed camera internal parameter is set, which can be determined using the pinhole imaging model, for example, it can be a 3×3 matrix. Then, using the initial value of the rotation matrix and the initial value of the translation parameter, the vehicle point cloud sets are projected onto the camera segmentation results to obtain the projected area, and the imaging of the vehicle on the image is fitted. On the basis of determining the rotation parameter, different deviations are added to the three dimensions of the Euler angles, and the evaluation function mentioned in this application is used as the optimization constraint condition for experiments, so as to obtain the experimental results. The experimental results can be seen Figure 4 as shown Figure 4 is a schematic diagram of the comparison effect before and after optimization by the method provided in this application at different distances. Among them, the light gray area is the area projected by the vehicle point cloud through the initial value of the rotation matrix; the dark gray area in the first column is the projection effect after adding deviations to the initial value of the rotation matrix; the dark gray area in the second column is the projection effect after optimizing the rotation matrix by the method proposed in this article.

[0122] It can be understood that the above different deviations can refer to the magnitudes that may occur in the normal state of an autonomous vehicle and be appropriately adjusted by expanding the range. It can be seen in the following table, which includes the deviations, optimizations, errors, and the distance between the point cloud and the camera in the Euler angle dimensions. The "Deviation" column in the table represents the deviations added to the corresponding dimensions (roll, yaw, pitch) on the basis rotation matrix; the "Optimization" column represents the correction amount after optimization; the "Error" column represents the sum of the "Deviation" and the "Optimization" amount.

[0123]

[0124] It should be noted that, as can be seen from the above table, within the deviation of adding 3° in the three dimensions of Euler angles, through the optimization scheme in the present application, the deviation can be optimized to within 0.1°, and only individual angles exceed 0.1°, which proves the effectiveness of the optimization method proposed in the application in the optimization process of the external parameters of the camera and laser in the set scenario. In the actual use scenario, the sensors of the autonomous vehicle are protected by a mold, and it is basically impossible for the deviation of the external parameters of the sensors to be greater than 3°. Therefore, this experiment shows that the algorithm proposed in the present application is sufficient to meet the optimization requirements of the camera-laser external parameters of the autonomous vehicle.

[0125] In this embodiment, three types of residuals are used to quantify the relative external parameters of the camera-laser sensor in all dimensions of Euler angles, providing constraint conditions for the automatic optimization of the relative external parameters of the camera-laser sensor, and being able to automatically complete the correction of the relative external parameters of the camera-laser sensor of the autonomous vehicle, reducing the time cost and labor cost of checking and correcting the relative external parameters of the camera-laser sensor of the autonomous vehicle, facilitating the automatic supervision of the quality change of the external parameters of the two sensors, timely detecting the quality change of the camera-laser external parameters of the autonomous vehicle, and ensuring the safety of the external parameter basis for autonomous driving.

[0126] It should be understood that although the steps in the flowchart are shown in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear indication in this article, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, at least a part of the steps in the figure may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same moment, but can be executed at different moments. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed alternately or in turn with at least a part of other steps or sub-steps or stages of other steps.

[0127] In another embodiment provided by the present application, a camera-laser relative external parameter quality evaluation device is also provided. Please refer to Figure 5 as shown, the device includes:

[0128] An acquisition module 810, configured to acquire a camera segmentation result and a laser segmentation result of the vehicle; the camera segmentation result is used to represent the set of points of the vehicle captured by the camera in the image, and the laser segmentation result is used to represent the set of point cloud coordinates generated after the vehicle is detected by the laser sensor;

[0129] A determination module 820, configured to determine the camera geometric center and the laser geometric center based on the camera segmentation result and the laser segmentation result;

[0130] A construction module 830, configured to construct an evaluation function according to the camera geometric center and the laser geometric center;

[0131] An optimization module 840, configured to evaluate the quality of the relative extrinsic parameters between the camera and the lidar according to an evaluation function and optimize the initial values of the relative extrinsic parameters by using a preset algorithm.

[0132] Optionally, a determination module 820 is specifically configured to:

[0133] Perform a mean processing on the camera segmentation result to obtain a first camera point mean and a second camera point mean;

[0134] Perform a mean processing on the lidar segmentation result to obtain a first lidar point mean and a second lidar point mean.

[0135] Optionally, a construction module 830 is specifically configured to:

[0136] Calculate a first Euclidean distance and a second Euclidean distance according to the first camera point mean, the first lidar point mean, the second camera point mean, and the second lidar point mean;

[0137] Determine a first residual and a second residual based on the first Euclidean distance and the second Euclidean distance; the first residual is used to constrain the deviation in the yaw angle and pitch angle dimensions; the second residual is used to constrain the difference threshold between the first Euclidean distance and the second Euclidean distance;

[0138] Calculate a third residual according to the first camera point mean, the first lidar point mean, the second camera point mean, and the second lidar point mean; the third residual is used to constrain the deviation in the roll angle dimension;

[0139] Sum up the first residual, the second residual, and the third residual to obtain an evaluation function.

[0140] Optionally, the construction module 830 is further configured to:

[0141] Calculate a first Euclidean distance according to the abscissa and ordinate of the first camera point mean and the abscissa and ordinate of the first lidar point mean;

[0142] Calculate a second Euclidean distance according to the abscissa and ordinate of the second camera point mean and the abscissa and ordinate of the second lidar point mean.

[0143] Optionally, the construction module 830 is further configured to:

[0144] Take the sum of the first Euclidean distance and the second Euclidean distance as the first residual;

[0145] Calculate the distance difference between the first Euclidean distance and the second Euclidean distance, and perform a square calculation on the distance difference to obtain a second residual.

[0146] Optionally, the construction module 830 is further configured to:

[0147] Construct a first straight line based on the first camera point mean and the second camera point mean;

[0148] Construct a second straight line based on the first laser point mean and the second laser point mean;

[0149] Calculate the angle between the first straight line and the second straight line, and use the angle as the third residual.

[0150] Optionally, the optimization module 840 is specifically configured to:

[0151] Evaluate the quality of the relative extrinsic parameters between the camera and the laser and optimize the initial values of the relative extrinsic parameters using a preset algorithm, and determine whether the evaluation function meets the iterative convergence condition;

[0152] When the evaluation function meets the preset convergence condition, use the relative extrinsic parameters corresponding to the evaluation function as the optimal relative extrinsic parameter values.

[0153] For the specific limitations on the above camera-laser relative extrinsic parameter quality evaluation device, reference can be made to the limitations on the camera-laser relative extrinsic parameter quality evaluation method in the above text, which will not be elaborated here. Each module in the above camera-laser relative extrinsic parameter quality evaluation device can be implemented in whole or in part through software, hardware, and their combination. The above modules can be embedded in the processor of the computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to the above modules.

[0154] In one embodiment, a computer device is provided, and the internal structure diagram of the computer device can be as Figure 1 shown. The computer device includes a processor, a memory, a network interface, and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, it implements a camera-laser relative extrinsic parameter quality evaluation method as described above. It includes: including a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, it implements any step in the above camera-laser relative extrinsic parameter quality evaluation method.

[0155] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by the processor, it can implement any step in the above camera-laser relative extrinsic parameter quality evaluation method.

[0156] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0157] The present application is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram can be implemented by computer program instructions, and the combination of the flows and / or blocks in the flowchart and / or block diagram can also be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the functions specified in one Figure 1 flow or multiple flows and / or blocks Figure 1 or multiple blocks.

[0158] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including instruction means, and the instruction means implement the functions specified in one Figure 1 flow or multiple flows and / or blocks Figure 1 or multiple blocks.

[0159] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process. Thus, the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one Figure 1 flow or multiple flows and / or blocks Figure 1 or multiple blocks.

[0160] Although the preferred embodiments of the present application have been described, those skilled in the art can make additional changes and modifications to these embodiments once they know the basic creative concepts. Therefore, the appended claims are intended to be construed as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0161] Obviously, those skilled in the art can make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalent technologies, this application is also intended to include these modifications and variations.

Claims

1. A method for evaluating the quality of the relative external parameters of a camera and a laser, characterized in that The method includes: Obtaining the camera segmentation result and the lidar segmentation result of the vehicle; the camera segmentation result is used to represent the set of points of the vehicle captured by the camera in the image, and the lidar segmentation result is used to represent the set of point cloud coordinates generated after detecting the vehicle by the lidar sensor; Based on the camera segmentation result and the lidar segmentation result, determining the camera geometric center and the lidar geometric center; Constructing an evaluation function according to the camera geometric center and the lidar geometric center; Evaluating the quality of the relative extrinsic parameters between the camera and the lidar according to the evaluation function and optimizing the initial value of the relative extrinsic parameters by using a preset algorithm.

2. The method according to claim 1, characterized in that The camera geometric center includes a first camera point mean value and a second camera point mean value, and the lidar geometric center includes a first lidar point mean value and a second lidar point mean value; Based on the camera segmentation result and the lidar segmentation result, determining the camera geometric center and the lidar geometric center includes: Performing a mean value process on the camera segmentation result to obtain the first camera point mean value and the second camera point mean value; Performing a mean value process on the lidar segmentation result to obtain the first lidar point mean value and the second lidar point mean value.

3. The method according to claim 2, wherein The initial value of the relative extrinsic parameters includes: the yaw angle, the pitch angle, and the roll angle between the camera and the lidar sensor; constructing the evaluation function according to the camera geometric center and the lidar geometric center includes: Calculating a first Euclidean distance and a second Euclidean distance according to the first camera point mean value, the first lidar point mean value, the second camera point mean value, and the second lidar point mean value; Based on the first Euclidean distance and the second Euclidean distance, determining a first residual and a second residual; the first residual is used to constrain the deviation in the yaw angle and pitch angle dimensions; the second residual is used to constrain the difference threshold between the first Euclidean distance and the second Euclidean distance; Calculating a third residual according to the first camera point mean value, the first lidar point mean value, the second camera point mean value, and the second lidar point mean value; the third residual is used to constrain the deviation in the roll angle dimension; Performing a summation process on the first residual, the second residual, and the third residual to obtain the evaluation function.

4. The method according to claim 3, characterized in that Calculating the first Euclidean distance and the second Euclidean distance according to the first camera point mean value, the first lidar point mean value, the second camera point mean value, and the second lidar point mean value includes: Calculating the first Euclidean distance according to the horizontal and vertical coordinates of the first camera point mean value and the horizontal and vertical coordinates of the first lidar point mean value; Calculating the second Euclidean distance according to the horizontal and vertical coordinates of the second camera point mean value and the horizontal and vertical coordinates of the second lidar point mean value.

5. The method according to claim 3, characterized in that, Based on the first Euclidean distance and the second Euclidean distance, determining the first residual and the second residual includes: Taking the sum of the first Euclidean distance and the second Euclidean distance as the first residual; Calculating the distance difference between the first Euclidean distance and the second Euclidean distance, and performing a square calculation process on the distance difference to obtain the second residual.

6. The method according to claim 3, wherein Calculating the third residual according to the first camera point mean value, the first lidar point mean value, the second camera point mean value, and the second lidar point mean value includes: Construct a first straight line based on the first camera point mean and the second camera point mean; Construct a second straight line based on the first laser point mean and the second laser point mean; Calculate the angle between the first straight line and the second straight line, and use the angle as the third residual.

7. The method according to claim 1, characterized in that, Evaluate the camera-laser relative extrinsic parameter quality according to the evaluation function and optimize the initial value of the relative extrinsic parameter by using a preset algorithm, including: Evaluate the camera-laser relative extrinsic parameter quality and optimize the initial value of the relative extrinsic parameter by using a preset algorithm, and determine whether the evaluation function meets the iterative convergence condition; When the evaluation function meets the preset convergence condition, use the relative extrinsic parameter corresponding to the evaluation function as the optimal relative extrinsic parameter value.

8. A camera-laser relative external parameter quality evaluation device, characterized in that, The device includes: An acquisition module, configured to acquire a camera segmentation result and a laser segmentation result of a vehicle; the camera segmentation result is used to represent a set of points of the vehicle captured by a camera in an image, and the laser segmentation result is used to represent a set of point cloud coordinates generated after the vehicle is detected by a laser sensor; A determination module, configured to determine a camera geometric center and a laser geometric center based on the camera segmentation result and the laser segmentation result; A construction module, configured to construct an evaluation function according to the camera geometric center and the laser geometric center; An optimization module, configured to evaluate the camera-laser relative extrinsic parameter quality according to the evaluation function and optimize the initial value of the relative extrinsic parameter by using a preset algorithm.

9. A computer device, comprising: A memory and a processor, the memory stores a computer program, wherein the processor implements the steps of the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images