External parameter verification method, system and readable storage medium for camera and laser radar

Abstract

The present application discloses a method, system and readable storage medium for verifying external parameters of cameras and laser radars. The method includes: adding disturbances to the current external parameters to obtain multiple groups of external parameter state values; determining whether the quality of each group of external parameter state values ​​meets the standards; and determining whether the current external parameter calibration meets the requirements based on the proportion of the multiple groups of external parameter state values ​​that meet the standards. Through the above methods, the present application can realize online external parameter verification of cameras and laser radars with high accuracy, convenience and efficiency.

Description

Technical Field

[0001] The present application relates to the field of sensor external parameter calibration, and in particular to external parameter verification methods, systems and readable storage media for cameras and lidars. Background Art

[0002] Smart cars use a variety of sensors to realize the perception of environmental objects and the positioning of their own positions. Cameras can produce accurate 2D detection of the environment, and LiDAR can produce accurate 3D detection of the environment. The observation of the same object by both sensors needs to be unified through the external parameters between the two sensors. The accuracy of the external parameters of the two determines the accuracy of the smart car's perception of the environment and the accuracy of the vehicle's positioning, which ultimately affects the overall safety of the vehicle.

[0003] At present, the external parameter calibration method commonly used by smart car manufacturers is offline calibration. This method requires the vehicle to collect data in a pre-arranged calibration workshop, complete the online calibration of the camera and LiDAR offline, and then manually review the results. This method relies on a pre-arranged calibration workshop and requires manual intervention. The calibration process is long, complicated, time-consuming and labor-intensive.

[0004] After the sensor external parameter calibration is completed, the positions of the two sensors will inevitably change over time. At this time, in order to update the vehicle external parameter results, the vehicle needs to be recalibrated in the calibration site. The existing parameter calibration method relies on manual verification, which requires a lot of time and manpower for large-scale vehicle external parameter verification. Therefore, a fast external parameter calibration verification method is needed. Summary of the invention

[0005] The present application mainly provides a method, system and readable storage medium for verifying external parameters of a camera and a lidar, which solves the problem that vehicles in the prior art cannot perform external parameter verification on their own.

[0006] To solve the above technical problems, the first aspect of the present application provides a method for verifying external parameters of a camera and a lidar, comprising: adding disturbances to the current external parameters to obtain multiple groups of external parameter state values; determining whether the quality of each group of external parameter state values ​​meets the standards; and determining whether the current external parameter calibration meets the requirements based on the proportion of the multiple groups of external parameter state values ​​that meet the standards.

[0007] To solve the above-mentioned technical problems, the second aspect of the present application provides a vehicle system, comprising a processor and a memory coupled to each other; a computer program is stored in the memory, and the processor is used to execute the computer program to implement the external parameter verification method of the camera and lidar provided in the first aspect above.

[0008] In order to solve the above-mentioned technical problems, the third aspect of the present application provides a computer-readable storage medium, which stores program data. When the program data is executed by a processor, the external parameter verification method of the camera and lidar provided in the first aspect is implemented.

[0009] The beneficial effect of the present application is that, different from the prior art, the present application obtains multiple groups of external parameter state values ​​by adding disturbances to the current external parameters, and determines whether the quality of each group of external parameter state values ​​meets the standards, so as to determine the accuracy of the current external parameters according to the standard-compliant proportion of the external parameter state values, and then determine whether to recalibrate the current external parameters. This allows the vehicle to verify the external parameters by itself during driving, and the external parameter verification is more convenient, and the verification results are reliable and accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0011] Figure 1 This is a schematic flow chart of an embodiment of an external parameter verification method for a camera and a laser radar of the present application;

[0012] Figure 2 This is a schematic flowchart of an embodiment of step S12 of the present application;

[0013] Figure 3 This is a schematic flow chart of an embodiment of step S21 of the present application;

[0014] Figure 4 This is a schematic flow chart of an embodiment of step S24 of the present application;

[0015] Figure 5 is a schematic structural block diagram of an embodiment of a vehicle system of the present application;

[0016] Figure 6 is a schematic structural block diagram of another embodiment of the vehicle system of the present application;

[0017] Figure 7 It is a schematic block diagram of the structure of an embodiment of a computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0019] The terms "first" and "second" in this application are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features shown. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally also includes other steps or units inherent to these processes, methods, products or devices.

[0020] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0021] See also Figure 1 , Figure 1 1 is a flowchart of an embodiment of the external parameter verification method for cameras and laser radars of the present application. It should be noted that if there are substantially the same results, this embodiment does not Figure 1 The process sequence shown is limited. This embodiment includes the following steps:

[0022] S11: Add disturbance to the current external parameter to obtain multiple groups of external parameter state values.

[0023] The current external parameters are the external parameters between the camera and the radar that were previously calibrated, for example, obtained through offline calibration in a calibration workshop before leaving the factory.

[0024] Optionally, this step adds disturbance to the current external parameter, which may be adding preset disturbance amounts to the translation variable and / or rotation variable of the current external parameter respectively.

[0025] Specifically, the external parameter data can be expressed as state = [Roll, Pitch, Yaw, X, Y, Z], which includes three rotation variables Roll, Pitch, Yaw, and three translation variables X, Y, Z.

[0026] Among them, preset disturbance amounts are added to the translation variables and / or rotation variables of the current external parameters respectively. It is possible to choose to add a disturbance of a preset rotation angle to any one or more dimensions of the rotation variables Roll, Pitch, and Yaw, and to add a disturbance of a preset moving distance to any one or more dimensions of the translation variables X, Y, and Z, so as to obtain multiple sets of external parameter state values.

[0027] Adding a disturbance to a translation variable means adding or subtracting a preset moving distance on its basis, or not changing it; adding a disturbance to a rotation variable means adding or subtracting a preset rotation angle on its basis, or not changing it.

[0028] Among them, the preset moving distance can be, for example, 0.1 meter or 0.2 meter, and the preset rotation angle can be, for example, 1° or 2°, all of which can be set according to actual conditions and are not limited here.

[0029] In one embodiment, a variable to be disturbed may be selected in the external parameter data to obtain a preset number of external parameter state values.

[0030] Specifically, disturbances can be added to any one or more dimensions of the six variables in the above-mentioned current external parameters. For example, you can choose to add disturbances to one of the rotation variables and one of the translation variables, such as adding disturbances to Roll and X respectively, and you can get 9 sets of external parameter state values; for another example, you can choose to add disturbances to the six dimensions of Roll, Pitch, Yaw, X, Y, and Z, and you can get 729 sets of external parameter state values. According to the system processing capacity, you can also choose to add disturbances to only some of the variables, which will not be repeated here.

[0031] It should be noted that the “plurality” mentioned in this article is intended to indicate a quantity of two or more.

[0032] S12: Determine whether the quality of each group of external parameter state values ​​meets the standard.

[0033] Among them, the quality of external parameters can be judged by the following methods:

[0034] See also Figure 2 , Figure 2 is a flowchart of an embodiment of step S12 of the present application. It should be noted that if there are substantially the same results, this embodiment does not necessarily Figure 2 The process sequence shown is limited. Figure 2 As shown, this embodiment may include the following steps:

[0035] S21: Acquire a two-dimensional image including the detection target taken by the camera and a first point cloud obtained by the laser radar sensing the detection target.

[0036] Optionally, in this step, the camera and the laser radar are triggered synchronously, with a time interval of 20 milliseconds. The two-dimensional image contains pixel information of the detection target and the surrounding environment, and the first point cloud contains point data of the appearance surface perceived by the laser radar for the detection target.

[0037] Among them, the external parameter verification method of the present application can be used in a vehicle system. The vehicle is equipped with a camera and a lidar. When the vehicle is driving, the camera and the lidar can be used to obtain pixel information and point cloud information of objects in the surrounding environment. The objects in the surrounding environment can become detection targets. The detection targets are, for example, other vehicles, roadblocks, warning signs, etc. In this embodiment, other vehicles are used as detection targets to obtain a two-dimensional image and a first point cloud.

[0038] See also Figure 3 , Figure 3 is a flowchart of an embodiment of step S21 of the present application. It should be noted that if there are substantially the same results, this embodiment does not necessarily Figure 3 The process sequence shown is limited. This embodiment includes the following steps:

[0039] S211: Obtain the fourth point cloud of the current environment sensed by the laser radar.

[0040] The fourth point cloud obtained in this step includes all point cloud data obtained by the laser radar's perception of the overall environment, including not only the point cloud data of the detection target, but also the point cloud data of other objects in the current environment except the detection target.

[0041] S212: Projecting the fourth point cloud into the two-dimensional image based on the current external parameter to obtain a fifth point cloud.

[0042] Among them, the fourth point cloud is a 3D point in the coordinate system of the laser radar, and the fifth point cloud is projected to a 2D point in the two-dimensional image.

[0043] The fourth point cloud is projected into the two-dimensional image, specifically, a projection matrix is ​​constructed based on the intrinsic parameters of the camera and the current extrinsic parameters, and then the three-dimensional coordinates of each data point in the fourth point cloud in the radar coordinate system are converted into two-dimensional coordinates in the two-dimensional image coordinate system based on the projection matrix.

[0044] Specifically, the data points in the fourth point cloud are projected onto the two-dimensional image in the following manner to obtain the positions of the data points in the fifth point cloud:

[0045]

[0046] in, I is the unit matrix, UV is the position of the fifth point cloud in the two-dimensional image, K is the intrinsic parameter of the camera, is the external parameter between the camera and the lidar, is the rotation matrix of the current external parameter, is the translation vector of the current external parameter, P L is the three-dimensional coordinate vector of the data point in the fourth point cloud.

[0047] S213: Determine a second area based on the first area, wherein the second area includes the first area and the range of the second area is larger than the first area.

[0048] The first area is the area where the detection target is located in the two-dimensional image, and the edge of the first area is the outline of the detection target in the two-dimensional image. If the detection target is, for example, another vehicle, then the area where the detection target is located is the area where the vehicle is located.

[0049] Prior to this step, semantic segmentation or bounding box detection may be performed on the two-dimensional image containing the detection target to determine the first area where the detection target is located.

[0050] The second area includes the first area, and also includes an area outside the first area whose distance from the outer boundary of the first area is within a preset range. In one embodiment, the second area is determined by taking the preset distance outward from the edge of the first area as the edge of the second area. In another embodiment, the range of the second area can also be determined in other ways, as long as it can include the first area.

[0051] S214: taking the data points in the fifth point cloud that fall within the second area as the first point cloud.

[0052] In this step, the first point cloud in the fifth point cloud that falls within the second area is used as the data point obtained by the laser radar to perceive the detection target.

[0053] The first point cloud is a 3D point corresponding to a 2D point of the fifth point cloud that falls within the second area.

[0054] This embodiment first obtains the fourth point cloud of the laser radar sensing the current environment, and then projects the fourth point cloud into a two-dimensional image to obtain a fifth point cloud; and determines the second area where the detection target is approximately located through the first area, and determines that the 3D points corresponding to the points in the fifth point cloud that fall in the second area are the first point cloud, and the first point cloud includes data points of the laser radar sensing the detection target.

[0055] S22: projecting the first point cloud into the two-dimensional image based on the multiple groups of external parameter state values ​​respectively to form a second point cloud corresponding to each group of external parameter test values.

[0056] Similar to step S212, the first point cloud is projected into the two-dimensional image by the following methods 1) to 2):

[0057] 1) Construct a projection matrix based on the camera's intrinsic parameters and each set of extrinsic parameter state values, where each set of extrinsic parameter state values ​​is used to represent the transformation relationship between the camera coordinate system corresponding to the camera and the radar coordinate system corresponding to the laser radar.

[0058] The projection matrix is ​​constructed as follows:

[0059]

[0060] In the above formula, K is the intrinsic parameter of the camera, and R is the rotation matrix t is the translation vector I is the identity matrix, and K×[R|t] is the constructed projection matrix.

[0061] 2) Based on the projection matrix, the three-dimensional coordinates of each data point in the first point cloud in the radar coordinate system are converted into two-dimensional coordinates in the image coordinate system corresponding to the two-dimensional image.

[0062] Project each data point in the first point cloud into a two-dimensional image, and its two-dimensional coordinates are: UV = K[R|t]P L .

[0063] S23: taking the three-dimensional points corresponding to the data points in each second point cloud that fall within the first region where the detection target is located as the third point cloud.

[0064] Among them, the 3D points corresponding to the data points of the second point cloud falling in the first area are the third point cloud. The third point cloud is regarded as the data points obtained by the laser radar sensing the detection target under the external parameter state value. The third point cloud is verified by the following steps to determine whether the quality of the corresponding external parameter state value meets the standard.

[0065] S24: Calculate the entropy value of the third point cloud.

[0066] The entropy value is represented by the degree of aggregation of data points in the third point cloud, wherein the greater the degree of aggregation, the smaller the entropy value.

[0067] See also Figure 4 , Figure 4 is a flowchart of an embodiment of step S24 of the present application. It should be noted that if there is substantially the same result, this embodiment does not necessarily use Figure 4 The process sequence shown is limited. This embodiment includes the following steps:

[0068] S241: Meshing the third point cloud to form a plurality of meshes.

[0069] In this step, the VoxelGrid filter voxelization grid method of the PCL point cloud library can be used to create a three-dimensional voxel grid in the third point to serve as the grid in this embodiment.

[0070] The function syntax: void setLeafSize(floatIx, floatly, floatlz) can be used to set the size of the voxel grid in the three directions of X, Y, and Z respectively through Ix, ly, and lz. The grid size can be, for example, a grid size of 0.02m×0.02m×0.02m. The grid size can also be set in other ways. The grid setting method here is only for illustrative purposes.

[0071] S242: Count the number of grids into which the data points in the third point cloud fall.

[0072] According to the gridding method in the previous step, the data points in the third point cloud fall into some grids. This step counts the number of grids in which the data points in the third point cloud fall, that is, counts the number of grids containing data points of the third point cloud greater than or equal to 1 in all grids.

[0073] S243: The entropy value is represented by the number of grids into which the data point falls.

[0074] Among them, the larger the number of grids that the data point falls into, the larger the entropy value, indicating that the third point cloud is more dispersed.

[0075] In another embodiment, the centroid of the third point cloud may be determined first, and the average distance between all data points in the third point cloud and the centroid is calculated, and the average distance is used as the entropy value of the third point cloud. The centroid is determined as follows:

[0076]

[0077] In the above formula, n is the number of data points in the third point cloud, (xi,yi,zi) is the coordinate of each data point in the radar coordinate system, P c are the centroid coordinates.

[0078] In other embodiments, semantic segmentation or detection may be performed on the two-dimensional image captured by the camera to obtain a first area where the detection target is located, and then semantic segmentation may be performed on the fourth point cloud of the current environment perceived by the lidar to obtain a three-dimensional area where the detection target is located. The three-dimensional area is projected into the two-dimensional image to obtain a third area corresponding to the three-dimensional area, and the distance between the center point of the third area and the center point of the first area is calculated to obtain the entropy value of the third point cloud.

[0079] S25: Determine whether the quality of each group of external parameter state values ​​meets the standard based on the relationship between the entropy value and the entropy value corresponding to the third point cloud under the current external parameter.

[0080] The entropy value corresponding to the third point cloud under the current external parameter can also be calculated using the methods of the above embodiments, which will not be repeated here.

[0081] Among them, the entropy value of the third point cloud under each set of external parameter state values ​​is compared with the entropy value of the third point cloud under the current external parameter. If the entropy value of the third point cloud under the external parameter state value is less than the entropy value of the third point cloud under the current external parameter, it is determined that the quality of the corresponding external parameter state value meets the standard.

[0082] S13: Determine whether the current external parameter calibration meets the requirements according to the proportion of the multiple groups of external parameter status values ​​that meet the quality standards.

[0083] Among them, multiple groups of external parameter state values ​​are obtained by adding disturbances to the current external parameters, and are not the true current external parameters. If the proportion of external parameter state values ​​that meet the standards is higher, it indicates that the accuracy of the current external parameters still needs to be improved and does not meet the requirements; otherwise, it is determined that the current external parameters meet the requirements and do not need to be recalibrated.

[0084] Specifically, the compliance ratio of the external parameter state values ​​can be expressed as: P=C / N*100%, where P is the compliance ratio, C is the number of external parameter state values ​​that meet the standards, and N is the total number of external parameter state values.

[0085] Among them, if the compliance ratio P of the external parameter state value is less than the preset ratio, it is determined that the current external parameter meets the requirements and there is no need to recalibrate the current external parameter; otherwise, it is determined that the current external parameter does not meet the requirements and it is necessary to recalibrate the current external parameter.

[0086] The preset ratio is between 1.5% and 3% (inclusive), for example, 1.5%, 2%, 2.5%, etc.

[0087] Different from the prior art, the present application can obtain multiple external parameter state values ​​by adding disturbances to the current external parameters, and perform synchronous detection based on the camera and lidar to obtain a two-dimensional image and point cloud data containing the detection target, and then project the point cloud data onto the two-dimensional image according to each external parameter state value, and determine whether the quality of each external parameter state value meets the standard according to the entropy value of the point cloud data in the area, and then determine whether the current external parameter needs to be recalibrated according to the standard-compliant proportion of the external parameter state value. This application can achieve external parameter verification without driving the vehicle back to a specific calibration site, with high accuracy, simplicity and efficiency.

[0088] See also Figure 5 , Figure 5 The vehicle system 100 includes a camera 141 and a laser radar 142. The camera 141 is used to capture a two-dimensional image of the current environment, which contains pixel information of the detection target, and the laser radar 142 is used to perceive the current environment.

[0089] Optionally, during the driving of the vehicle, the camera 141 and the lidar 142 use other vehicles as detection targets to acquire a two-dimensional image and a first point cloud.

[0090] The vehicle system 100 further includes an external parameter processing module 110 , a data processing module 120 , and a judgment module 130 .

[0091] Among them, the external parameter processing module 110 is used to add disturbances to the current external parameters to obtain multiple groups of external parameter state values, the data processing module 120 is used to determine whether the quality of each group of external parameter state values ​​meets the standards, and the judgment module 130 is used to determine whether to recalibrate the current external parameters based on the proportion of the external parameter state values ​​that meet the standards.

[0092] The external parameter processing module 110 is further used to add a disturbance amount of a preset movement distance or a preset rotation angle to the translation variable and / or rotation variable of the current external parameter, so as to obtain multiple groups of external parameter state values.

[0093] The vehicle system 100 further includes an acquisition module 150 for acquiring a two-dimensional image containing a detection target captured by the camera 141 and a first point cloud obtained by sensing the detection target by the laser radar 142. The data processing module 120 is also used to project the first point cloud into the two-dimensional image based on multiple groups of external parameter state values ​​to form a second point cloud corresponding to each group of external parameter test values, and take the data points in each second point cloud that fall within the first area where the detection target is located as the third point cloud, calculate the entropy value of the third point cloud, and finally determine whether the quality of each external parameter state value meets the standard based on the relationship between the entropy value and the entropy value corresponding to the third point cloud under the current external parameter.

[0094] Among them, the acquisition module 150 is also used to acquire the fourth point cloud of the current environment perceived by the laser radar 142, and the data processing module 120 is also used to project the fourth point cloud into the two-dimensional image based on the current external parameters to obtain the fifth point cloud, and determine the second area based on the first area, wherein the second area includes the first area and the range of the second area is larger than the first area, and finally the data points in the fifth point cloud that fall within the second area are used as the first point cloud.

[0095] Among them, the data processing module 120 is also used to construct a projection matrix based on the internal parameters of the camera 141 and each group of external parameter state values, wherein each group of external parameter state values ​​is used to characterize the transformation relationship between the camera coordinate system corresponding to the camera 141 and the radar coordinate system corresponding to the laser radar 142, and based on the projection matrix, the three-dimensional coordinates of each data point in the first point cloud in the radar coordinate system are converted into two-dimensional coordinates in the image coordinate system corresponding to the two-dimensional image.

[0096] Among them, the data processing module 120 is also used to grid the third point cloud to form multiple grids, count the number of grids that the data points in the third point cloud fall into, and finally characterize the entropy value with the number of grids that the data points fall into, wherein the larger the number of grids that the data points fall into, the larger the entropy value.

[0097] For the specific methods of executing each step of each processing, please refer to the description of the steps in the above-mentioned embodiment of the external parameter verification method for the camera and lidar of the present application, and will not be repeated here.

[0098] See also Figure 6 , Figure 6 2 is a schematic block diagram of another embodiment of the vehicle system of the present application. The vehicle system 200 includes a processor 210 and a memory 220 coupled to each other, the memory 220 stores a computer program, and the processor 210 is used to execute the computer program to implement the external parameter verification method of the camera and the laser radar described in the above embodiments.

[0099] For the description of each step of the processing execution, please refer to the description of each step of the embodiment of the external parameter verification method for the camera and lidar of the above-mentioned present application, which will not be repeated here.

[0100] The memory 220 can be used to store program data and modules. The processor 210 executes various functional applications and data processing by running the program data and modules stored in the memory 220. The memory 220 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a point cloud data processing function, an image processing function, etc.), etc.; the data storage area may store data created according to the use of the vehicle system 200 (such as image data, point cloud data, external parameter state value, current external parameter, etc.), etc. In addition, the memory 220 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 220 may also include a memory controller to provide the processor 210 with access to the memory 220.

[0101] In the various embodiments of the present application, the disclosed methods and systems can be implemented in other ways. For example, the various embodiments of the vehicle system 200 described above are merely schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of systems or units, which can be electrical, mechanical or other forms.

[0102] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0103] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0104] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium.

[0105] See also Figure 7 , Figure 7 This is a structural schematic block diagram of an embodiment of a computer-readable storage medium of the present application. The computer-readable storage medium 300 stores program data 310. When the program data 310 is executed, the steps of each embodiment of the external parameter verification method for the camera and lidar as described above are implemented.

[0106] For the description of each step of the processing execution, please refer to the description of each step of the embodiment of the external parameter verification method for the camera and lidar of the above-mentioned present application, which will not be repeated here.

[0107] The computer-readable storage medium 300 may be a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc., which may store program codes.

[0108] The above descriptions are merely embodiments of the present application and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for verifying external parameters of cameras and lidars. It is characterized in that The method comprises: Add disturbances to the current external parameters to obtain multiple sets of external parameter state values; Determine whether the quality of each group of external parameter state values ​​meets the standard; Determining whether the current external parameter calibration meets the requirements according to the proportion of the multiple groups of external parameter state values ​​that meet the quality standards; Wherein, the respectively determining whether the quality of each group of the external parameter state values ​​meets the standard comprises: Acquire a two-dimensional image containing a detection target taken by a camera and a first point cloud obtained by sensing the detection target by a laser radar; projecting the first point cloud into the two-dimensional image based on the multiple groups of the extrinsic parameter state values ​​respectively, so as to form a second point cloud corresponding to each group of the extrinsic parameter state values; Taking the three-dimensional points corresponding to the data points in each of the second point clouds that fall within the first region where the detection target is located as the third point cloud; Calculating the entropy value of the third point cloud; Determining whether the quality of each of the external parameter state values ​​meets the standard according to the magnitude relationship between the entropy value and the entropy value corresponding to the third point cloud under the current external parameter; The determining whether the current external parameter calibration meets the requirements according to the proportion of the multiple groups of external parameter state values ​​that meet the quality standards includes: if the proportion of the external parameter state values ​​that meet the quality standards is less than a preset proportion, determining that the current external parameter meets the requirements; otherwise, determining that the current external parameter does not meet the requirements.

2. The method according to claim 1, It is characterized in that The adding of disturbance to the current external parameter to obtain multiple groups of external parameter state values ​​includes: Preset disturbance amounts are added to the translation variables and / or rotation variables of the current external parameters respectively to obtain multiple groups of external parameter state values.

3. The method according to claim 1, It is characterized in that The step of acquiring a two-dimensional image containing a detection target captured by a camera and a first point cloud obtained by sensing the detection target by a laser radar includes: Acquire a fourth point cloud of the current environment sensed by the laser radar; Projecting the fourth point cloud into the two-dimensional image based on the current extrinsic parameter to obtain a fifth point cloud; Determine a second area based on the first area, wherein the second area includes the first area and the second area is larger than the first area; The data points in the fifth point cloud that fall within the second area are used as the first point cloud.

4. The method according to claim 1, It is characterized in that The projecting the first point cloud into the two-dimensional image based on the multiple groups of the external parameter state values ​​respectively includes: Constructing projection matrices based on the intrinsic parameters of the camera and each group of the extrinsic parameter state values, respectively, wherein each group of the extrinsic parameter state values ​​is used to characterize the transformation relationship between the camera coordinate system corresponding to the camera and the radar coordinate system corresponding to the laser radar; The three-dimensional coordinates of each data point in the first point cloud in the radar coordinate system are converted into two-dimensional coordinates in the image coordinate system corresponding to the two-dimensional image based on the projection matrix.

5. The method according to claim 1, It is characterized in that The entropy value is characterized by the degree of clustering of data points in the third point cloud, wherein the greater the degree of clustering, the smaller the entropy value.

6. The method according to claim 1, It is characterized in that The calculating the entropy value of the third point cloud comprises: Meshing the third point cloud to form a plurality of meshes; Counting the number of grids into which the data points in the third point cloud fall; The entropy value is characterized by the number of grids that the data point falls into, wherein the larger the number of grids that the data point falls into, the larger the entropy value.

7. The method according to claim 1, It is characterized in that The step of acquiring a two-dimensional image containing a detection target captured by a camera and a first point cloud obtained by sensing the detection target by a laser radar comprises: During the driving process of the vehicle equipped with the camera and the laser radar, other vehicles are used as the detection targets to obtain the two-dimensional image and the first point cloud.

8. The method according to claim 1, It is characterized in that The calculating the entropy value of the third point cloud comprises: determining the centroid of the third point cloud; An average value of the distance between each data point in the third point cloud and the centroid is calculated, and the average value is used as the entropy value of the third point cloud.

9. A vehicle system, It is characterized in that The system comprises a processor and a memory coupled to each other; a computer program is stored in the memory, and the processor is used to execute the computer program to implement the steps of the method according to any one of claims 1 to 8.

10. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores program data, and when the program data is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images