External Parameter Calibration Method, System and Readable Storage Medium for Camera and LiDAR

Abstract

The present application discloses an external parameter calibration method, system and readable storage medium for a camera and a lidar. The method includes: obtaining a two-dimensional image captured by the camera and including a detection target, and a first point cloud sensed by the lidar for the detection target; respectively projecting the first point cloud into the two-dimensional image based on a selected multiple sets of external parameter test values to form a second point cloud corresponding to each set of external parameter test values; using the three-dimensional points corresponding to the data points in each second point cloud that fall within a first region where the detection target is located as a third point cloud; calculating the entropy value of the third point cloud; and selecting an optimal external parameter value from the multiple sets of external parameter test values based on the entropy value. Through the above manner, the present application can achieve online external parameter calibration of the camera and the lidar, with high accuracy, convenience and efficiency.

Description

Technical Field

[0001] This application relates to the field of external parameter calibration of sensors, in particular to an external parameter calibration method, system and readable storage medium for a camera and a lidar. Background Art

[0002] Intelligent vehicles use multiple sensors in combination to achieve the perception of environmental objects by the vehicle and the positioning of the vehicle's own position. Among them, the camera can perform accurate 2D detection of the environment, and the LiDAR can perform accurate 3D detection of the environment. For the observation of the same object by both, it needs to be unified through the external parameters between the two sensors. Whether the external parameters of the two are accurate determines the accuracy of the intelligent vehicle's environmental perception and the accuracy of vehicle positioning, ultimately affecting the safety of the overall vehicle.

[0003] Currently, the commonly used external parameter calibration method for intelligent vehicle manufacturers is offline calibration. This method requires the vehicle to collect data in a pre-arranged calibration workshop, complete the online calibration of the camera and LiDAR offline, and then manual result review is required. This method relies on a pre-arranged calibration workshop and requires manual intervention. The calibration process is long, complex, time-consuming and laborious.

[0004] After the external parameters of the sensors are calibrated, as time goes by, the positions between the two sensors will inevitably change. At this time, in order to update the vehicle's external parameter results, the vehicle needs to be driven back into the calibration site for calibration again, which has a negative impact on vehicle operation. Summary of the Invention

[0005] This application mainly provides an external parameter calibration method, system and readable storage medium for a camera and a lidar, which solves the problem that the camera and lidar cannot be calibrated online for external parameters in the prior art.

[0006] To solve the above technical problems, the first aspect of this application provides an external parameter calibration method for a camera and a lidar, including: obtaining a two-dimensional image captured by the camera containing a detection target and a first point cloud obtained by the lidar sensing the detection target; projecting the first point cloud into the two-dimensional image respectively based on a selected set of external parameter test values to form a second point cloud corresponding to each set of external parameter test values; using the three-dimensional points corresponding to the data points in each second point cloud that fall within the first region where the detection target is located as a third point cloud; calculating the entropy value of the third point cloud; and selecting the optimal external parameter value from the set of external parameter test values based on the entropy value.

[0007] To solve the above technical problems, the second aspect of this application provides a vehicle system, including a processor and a memory coupled to each other; the memory stores a computer program, and the processor is configured to execute the computer program to implement the external parameter calibration method for a camera and a lidar provided in the first aspect as described above.

[0008] To solve the above technical problems, a third aspect of the present application provides a computer-readable storage medium. The computer-readable storage medium stores program data, and when the program data is executed by a processor, it implements the external parameter calibration method for the camera and lidar provided in the first aspect above.

[0009] The beneficial effects of the present application are as follows: Different from the prior art, the present application uses the camera to capture a two-dimensional image containing the detection target and the first point cloud obtained by the lidar sensing the detection target. Based on a selected set of external parameter test values, the first point cloud is projected into the two-dimensional image respectively to form a second point cloud corresponding to each set of external parameter test values. The data points in each second point cloud that fall within the first region where the detection target is located are used as the third point cloud, and the optimal external parameter value is selected from the set of external parameter test values according to the entropy value of the third point cloud. The above method uses the point cloud data obtained by the lidar's sensing of the detection target, and based on the entropy value of the projection in the corresponding region of the two-dimensional image under each set of external parameter test values, selects the optimal external parameter value from the set of external parameter test values to achieve online calibration of the external parameters between the camera and the lidar, with high accuracy, convenience and efficiency. Description of the Drawings

[0010] To more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0011] Figure 1 It is a flowchart of an embodiment of the external parameter calibration method for the camera and lidar of the present application;

[0012] Figure 2 It is a flowchart of an embodiment of step S11 of the present application;

[0013] Figure 3 It is a flowchart of an embodiment of step S14 of the present application;

[0014] Figure 4 It is a structural schematic diagram of an embodiment of the vehicle system of the present application;

[0015] Figure 5 It is a structural schematic diagram of another embodiment of the vehicle system of the present application;

[0016] Figure 6 It is a structural schematic diagram of an embodiment of the computer-readable storage medium of the present application. Detailed Embodiments

[0017] The following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative efforts belong to the scope of protection of the present application.

[0018] The terms "first" and "second" in the present application are only for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include at least one of such features. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but optionally further includes steps or units not listed, or optionally further includes other steps or units inherent to these processes, methods, products, or devices.

[0019] Referring to "embodiments" herein means that the specific features, structures, or characteristics described in connection with the embodiments can be included in at least one embodiment of the present application. The phrase appears in various places in the specification and does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art will explicitly and implicitly understand that the embodiments described herein can be combined with other embodiments.

[0020] Please refer to Figure 1 , Figure 1 which is a flowchart showing an embodiment of the method for calibrating the external parameters of the camera and lidar in the present application. It should be noted that if there are substantially the same results, this embodiment is not limited to the Figure 1 flow order shown. This embodiment includes the following steps:

[0021] S11: Obtain a two-dimensional image captured by the camera that includes a detection target and a first point cloud obtained by the lidar's perception of the detection target.

[0022] Optionally, in this step, the camera and the lidar are synchronously triggered. In some embodiments, a trigger time interval less than 20 milliseconds can be considered synchronous triggering. In the two-dimensional image, pixel information of the detection target and the surrounding environment is included. In the first point cloud, point data of the appearance surface of the detection target obtained by the lidar's perception is included.

[0023] Among them, the external parameter calibration method of the present application can be used in a vehicle system. The vehicle is equipped with a camera and a lidar. During the driving process of the vehicle, the objects in the surrounding environment can be used by the camera and the lidar to obtain pixel information and point cloud information. The objects in the surrounding environment can then become the detection targets. The detection targets are, for example, other vehicles, roadblocks, warning signs, etc. In this embodiment, other vehicles are used as the detection targets to obtain a two-dimensional image and a first point cloud.

[0024] Please refer to Figure 2 , Figure 2 which is a flowchart of an embodiment of step S11 of the present application. It should be noted that if there are substantially the same results, this embodiment is not limited to Figure 2 the process sequence shown. This embodiment includes the following steps:

[0025] S111: Obtain the fourth point cloud of the current environment sensed by the lidar.

[0026] The fourth point cloud obtained in this step includes all the point cloud data sensed by the lidar for the overall environment, including not only the point cloud data of the detection target but also the point cloud data of other objects in the current environment except the detection target.

[0027] S112: Project the fourth point cloud into the two-dimensional image based on the current external parameter value to obtain the fifth point cloud.

[0028] Among them, the fourth point cloud is a 3D point in the coordinate system of the lidar, and the fifth point cloud is a 2D point projected into the two-dimensional image.

[0029] Projecting the fourth point cloud into the two-dimensional image is specifically to construct a projection matrix based on the internal parameters of the camera and the current external parameter value, and then convert the three-dimensional coordinates of each data point in the fourth point cloud in the radar coordinate system into two-dimensional coordinates in the two-dimensional image coordinate system based on the projection matrix.

[0030] Specifically, the positions of the data points in the fifth point cloud are obtained by projecting the data points in the fourth point cloud into the two-dimensional image in the following manner:

[0031]

[0032] Among them, I is the identity matrix, UV is the position of the fifth point cloud in the two-dimensional image, K is the internal parameter of the camera, is the external parameter between the camera and the lidar, is the rotation matrix of the current external parameter, is the translation vector of the current external parameter, P L is the three-dimensional coordinate vector of each data point in the fourth point cloud.

[0033] S113: Determine a second region based on a first region, where the second region includes the first region and has a larger range than the first region.

[0034] Wherein, the first region is the region where the detection target is located in the two-dimensional image, and the edge of the first region is the contour of the detection target in the two-dimensional image. If the detection target is, for example, another vehicle, then the region where the detection target is located is the region where the vehicle is located.

[0035] Before this step, the first region where the detection target is located can be determined by performing semantic segmentation or bounding box detection on the two-dimensional image containing the detection target.

[0036] Wherein, the second region includes the first region and also includes the region outside the first region whose distance from the outer boundary of the first region is within a preset range. In one embodiment, the second region is determined as follows: the edge of the second region is at a preset distance outward from the edge of the first region. In other embodiments, the range of the second region can also be determined in other ways, as long as the first region can be included.

[0037] S114: Use the data points in the fifth point cloud that fall within the second region as the first point cloud.

[0038] In this step, the first point cloud in the fifth point cloud that falls within the second region is used as the data points obtained by the lidar sensing the detection target.

[0039] Wherein, the first point cloud is the 3D points corresponding to the 2D points in the fifth point cloud that fall within the second region.

[0040] In this embodiment, first obtain the fourth point cloud sensed by the lidar for the current environment, then project the fourth point cloud onto a two-dimensional image to obtain the fifth point cloud; and determine the second region where the detection target is approximately located through the first region, and determine the 3D points corresponding to the points in the fifth point cloud that fall within the second region as the first point cloud, and this first point cloud includes the data points sensed by the lidar for the detection target.

[0041] S12: Project the first point cloud onto the two-dimensional image respectively based on the selected multiple sets of external parameter test values to form a second point cloud corresponding to each set of external parameter test values.

[0042] Wherein, both the external parameter test values and the current external parameter data can be expressed in the form of state = [Roll, Pitch, Yaw, X, Y, Z], which is formed by performing Euler angle transformation on the rotation matrix of the external parameters and then splicing with the translation vector. It includes three rotation parameters Roll, Pitch, Yaw, and three translation parameters X, Y, Z.

[0043] Optionally, before this step, within the optimization space defined by the optimization upper limit and the optimization lower limit, multiple sets of external parameter test values are obtained.

[0044] Among them, the optimization upper limit determines the upper limit of the values of the parameters Roll, Pitch, Yaw, X, Y, Z in each dimension of the external parameter test values, and the optimization lower limit determines the lower limit of the values of the parameters Roll, Pitch, Yaw, X, Y, Z in each dimension of the external parameter test values. Within the optimization space, a finite number of sets of external parameter test values can be determined.

[0045] Specifically, the optimization upper limit can be expressed as [A1, B1, C1, X1, Y1, Z1] for example, and the optimization lower limit can be expressed as [A2, B2, C2, X2, Y2, Z2]. Where A1 and A2 respectively correspond to the upper limit and the lower limit of the value of the parameter Roll, B1 and B2 are respectively the upper limit and the lower limit of the value of the parameter Pitch, C1 and C2 are respectively the upper limit and the lower limit of the value of the parameter Yaw, X1 and X2 are respectively the upper limit and the lower limit of the value of the parameter X, Y1 and Y2 are respectively the upper limit and the lower limit of the value of the parameter Y, and Z1 and Z2 are respectively the upper limit and the lower limit of the value of the parameter Z. In this way, the above optimization upper limit and optimization lower limit form an optimization space. The parameters in each dimension of [Roll, Pitch, Yaw, X, Y, Z] take values within the range of their upper limit and lower limit at a set interval, and there are a finite number of possible values, so a finite number of sets of external parameter test values can be obtained.

[0046] In one embodiment, the optimization upper limit and the optimization lower limit are determined according to the current external parameter value. Specifically, for the values of each dimension in the optimization upper limit, after the current external parameter value is transformed in the above manner, a set angle conversion amount is added to / subtracted from each rotation parameter respectively to obtain the upper limit / lower limit of the value of each rotation parameter of the external parameter state value, and a set distance conversion amount is added to / subtracted from each translation parameter to obtain the optimization upper limit / lower limit of the value of each translation parameter of the external parameter state value. It can be understood that for the rotation parameters in each dimension, the set angle conversion amount added to / subtracted from them can be the same or different. Similarly, for the translation parameters in each dimension, the specific values of the set distance conversion amount added to / subtracted from them can be the same or different. The multiple sets of external parameter test values in this embodiment are generated centered on the current external parameter value. When the change of the current external parameter value is small, the best external parameter test value can be quickly determined within the optimization space to update the current external parameter, with fast processing speed and high accuracy.

[0047] Similar to step S212, the first point cloud is projected onto a two-dimensional image in the following ways 1) - 2):

[0048] 1) Construct the projection matrix based on the internal parameters of the camera and each set of external parameter test values. Among them, each set of external parameter test values is used to characterize the transformation relationship between the camera coordinate system corresponding to the camera and the radar coordinate system corresponding to the lidar.

[0049] The construction method of the projection matrix is as follows:

[0050]

[0051] In the above formula, K is the internal parameter of the camera, and R is the rotation matrix t is the translation vector I is the identity matrix, and the projection matrix is constructed by K×[R|t].

[0052] 2) Based on the projection matrix, convert the three-dimensional coordinates of each data point in the first point cloud in the radar coordinate system into two-dimensional coordinates in the image coordinate system corresponding to the two-dimensional image.

[0053] Project each data point in the first point cloud into the two-dimensional image, and its two-dimensional coordinates are: UV = K[R|t]P L .

[0054] S13: Take the three-dimensional points corresponding to the data points in each second point cloud that fall within the first region where the detection target is located as the third point cloud.

[0055] Among them, the 3D points corresponding to the data points in the second point cloud that fall within the first region are the third point cloud. The third point cloud is regarded as the data points obtained by the lidar sensing the detection target under this set of external parameter test values. By calculating the entropy of the third point cloud through the following steps, the best external parameters can be selected from the determined external parameter test values.

[0056] S14: Calculate the entropy value of the third point cloud.

[0057] Among them, the entropy value is characterized by the degree of aggregation of the data points in the third point cloud, and the greater the degree of aggregation, the smaller the entropy value.

[0058] Please refer to Figure 3 , Figure 3 which is a flowchart of an embodiment of step S14 of this application. It should be noted that if there are substantially the same results, this embodiment is not limited to the Figure 3 shown process sequence. This embodiment includes the following steps:

[0059] S141: Grid the third point cloud to form multiple grids.

[0060] This step can use the method of voxelizing the grid of the VoxelGrid filter in the PCL point cloud library to create a three-dimensional voxel grid in the third point cloud as the grid in this embodiment.

[0061] Among them, the function syntax: void setLeafSize(float Ix, float ly, float lz) can be used to set the sizes of the voxel grid in the X, Y, and Z directions through Ix, ly, and lz respectively. The grid size can be, for example, a grid size of 0.02m × 0.02m × 0.02m. There can also be other ways to set the grid size. The grid setting method here is only for illustrative purposes.

[0062] S142: Count the number of grids into which the data points in the third point cloud fall.

[0063] According to the grid division method in the previous step, the data points in the third point cloud fall into some grids. This step counts the number of grids into which the data points in the third point cloud fall, that is, counts the number of grids in all grids where the number of data points in the third point cloud is greater than or equal to 1.

[0064] S143: Characterize the entropy value by the number of grids into which the data points fall.

[0065] Among them, the larger the number of grids into which the data points fall, the larger the entropy value, indicating that the third point cloud is more dispersed.

[0066] In another embodiment, the centroid of the third point cloud can also be determined first, and the average distance between all data points in the third point cloud and the centroid is calculated, and this average distance is used as the entropy value of the third point cloud. Among them, the method for determining the centroid is as follows:

[0067]

[0068] In the above formula, n is the number of data points in the third point cloud, (xi, yi, zi) are the coordinates of each data point in the radar coordinate system, and P c is the centroid coordinate.

[0069] In other embodiments, the two-dimensional image obtained by camera shooting can also be semantically segmented or detected first to obtain the first region where the detection target is located, and then the fourth point cloud of the current environment sensed by the lidar is semantically segmented to obtain the three-dimensional region where the detection target is located. The three-dimensional region is projected onto the two-dimensional image to obtain the third region corresponding to the three-dimensional region, and the distance between the center point of the third region and the center point of the first region is calculated, and the entropy value of the third point cloud can also be obtained.

[0070] S15: Select the best extrinsic parameter value from multiple groups of extrinsic parameter test values based on the entropy value.

[0071] Among them, the smaller the entropy value, the higher the aggregation degree of the third point cloud, and the higher the accuracy of the corresponding extrinsic parameter test value. Therefore, this step selects the extrinsic parameter test value corresponding to the smallest entropy value as the best extrinsic parameter value to recalibrate the extrinsic parameters between the camera and the lidar.

[0072] Different from the prior art, in the optimization space of the present application, the first point cloud of the detection target obtained under the current external parameter value is respectively projected onto the area of the detection target in the two-dimensional image by using the corresponding external parameter test values, so as to obtain the third point cloud. Taking the minimum entropy value of the third point cloud as the optimization target, the optimal external parameter value is selected in the optimization space and used as the new external parameter to recalibrate the external parameter between the camera and the lidar, realizing the online external parameter calibration of the camera and the lidar, without the need to drive the vehicle back to a specific calibration site for parameter calibration, with high accuracy, convenience and efficiency.

[0073] Please refer to Figure 4 , Figure 4 which is a structural schematic diagram of an embodiment of the vehicle system of the present application. Among them, the vehicle system 100 includes a camera 141 and a lidar 142. The camera 141 is used to capture a two-dimensional image of the current environment, and the pixel information of the detection target is included in the two-dimensional image. The lidar 142 is used to sense the current environment.

[0074] Optionally, during the driving process of the vehicle, the camera 141 and the lidar 142 use other vehicles as detection targets to obtain two-dimensional images and the first point cloud.

[0075] Among them, the vehicle system 100 further includes an acquisition module 110 and a data processing module 120. The acquisition module 110 is used to acquire the two-dimensional image including the detection target captured by the camera 141 and the first point cloud obtained by the lidar 142 sensing the detection target; the data processing module 120 is used to project the first point cloud onto the two-dimensional image respectively based on the selected multiple groups of external parameter test values to form a second point cloud corresponding to each group of external parameter test values; take the three-dimensional points corresponding to the data points in each second point cloud that fall within the first area where the detection target is located as the third point cloud; calculate the entropy value of the third point cloud; and finally select the optimal external parameter value from the multiple groups of external parameter test values based on the entropy value.

[0076] Among them, the acquisition module 110 is further used to acquire the fourth point cloud of the current environment sensed by the lidar 142.

[0077] The data processing module 120 is further used to project the fourth point cloud onto the two-dimensional image based on the current external parameter value to obtain a fifth point cloud, determine a second area based on the first area, where the second area includes the first area and the range of the second area is larger than that of the first area, and take the data points in the fifth point cloud that fall within the second area as the first point cloud.

[0078] Among them, the data processing module 120 is further used to adjust the current external parameter value based on the optimization upper limit and the optimization lower limit to obtain multiple groups of external parameter test values.

[0079] Among them, the data processing module 120 is further configured to construct a projection matrix based on the internal parameters of the camera 141 and each set of external parameter test values, where each set of external parameter test values is used to characterize the transformation relationship between the camera coordinate system corresponding to the camera 141 and the radar coordinate system corresponding to the lidar 142, and then convert the three-dimensional coordinates of each data point in the first point cloud in the radar coordinate system into two-dimensional coordinates in the image coordinate system corresponding to the two-dimensional image based on the projection matrix.

[0080] Among them, the data processing module 120 is further configured to grid the first area to form a plurality of grids; count the number of grids into which the data points in the third point cloud fall; use the number of grids into which the data points fall to characterize the entropy value, where the larger the number of grids into which the data points fall, the larger the entropy value.

[0081] Among them, the data processing module 120 is further configured to determine the centroid of the third point cloud and calculate the average value of the distances between each data point in the third point cloud and the centroid, and use the average value as the entropy value of the third point cloud.

[0082] For the specific manners of the steps executed by each process, please refer to the descriptions of the steps in the embodiments of the external parameter calibration method for the camera and lidar in the present application above, and details are not described herein again.

[0083] Please refer to Figure 5 , Figure 5 FIG. is a structural schematic diagram of another embodiment of the vehicle system of the present application. The vehicle system 200 includes a processor 210 and a memory 220 that are coupled to each other. A computer program is stored in the memory 220, and the processor 210 is configured to execute the computer program to implement the external parameter calibration method for the camera and lidar described in each of the above embodiments.

[0084] For the descriptions of the steps executed by the process, please refer to the descriptions of the steps in the embodiments of the external parameter calibration method for the camera and lidar in the present application above, and details are not described herein again.

[0085] The memory 220 can be used to store program data and modules. The processor 210 executes various functional applications and data processing by running the program data and modules stored in the memory 220. The memory 220 may mainly include a program storage area and a data storage area. Among them, the program storage area can store an operating system, application programs required for at least one function (such as point cloud data processing function, image processing function, etc.); the data storage area can store data created according to the use of the vehicle system 200 (such as image data, point cloud data, external parameter status values, current external parameters, etc.). In addition, the memory 220 may include high-speed random access memory and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage devices. Correspondingly, the memory 220 may also include a memory controller to provide the processor 210 with access to the memory 220.

[0086] In various embodiments of the present application, the disclosed methods and systems can be implemented in other ways. For example, the various embodiments of the vehicle system 200 described above are merely illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections between each other can be through some interfaces. The indirect couplings or communication connections of the systems or units can be in electrical, mechanical or other forms.

[0087] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0088] In addition, in various embodiments of the present application, the functional units can be integrated in one processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above integrated units can be implemented in the form of hardware or in the form of software functional units.

[0089] If the above integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product, and this computer software product is stored in a storage medium.

[0090] Refer to Figure 6 , Figure 6 which is a structural schematic block diagram of an embodiment of the computer-readable storage medium of the present application. The computer-readable storage medium 300 stores program data 310, and when the program data 310 is executed, the steps of the above-described embodiments of the external parameter calibration method for the camera and lidar are implemented.

[0091] For the description of each step of the processing execution, please refer to the description of each step of the above-described embodiments of the external parameter calibration method for the camera and lidar of the present application, and details are not described herein again.

[0092] The computer-readable storage medium 300 may be various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk, or an optical disc.

[0093] The above are only the embodiments of the present application, and do not limit the patent scope of the present application. Any equivalent structural or equivalent process transformation made by using the content of the specification and drawings of the present application, or directly or indirectly applied to other related technical fields, shall be equally included in the patent protection scope of the present application.

Claims

1. An external parameter calibration method for a camera and a lidar, characterized in that, The method includes: Obtaining a two-dimensional image captured by a camera and including a detection target, and a first point cloud obtained by a lidar sensing the detection target; Based on a selected set of multiple external parameter test values, respectively projecting the first point cloud into the two-dimensional image to form a second point cloud corresponding to each set of the external parameter test values; Regarding the three-dimensional points corresponding to the data points in each of the second point clouds that fall within a first region where the detection target is located as a third point cloud; Calculating the entropy value of the third point cloud; Selecting an optimal external parameter value from the set of multiple external parameter test values based on the entropy value; The step of respectively projecting the first point cloud into the two-dimensional image based on a selected set of multiple external parameter test values includes: Based on the internal parameters of the camera and each set of the external parameter test values, respectively constructing a projection matrix, where each set of the external parameter test values is used to represent the transformation relationship between the camera coordinate system corresponding to the camera and the radar coordinate system corresponding to the lidar; Based on the projection matrix, converting the three-dimensional coordinates of each data point in the first point cloud in the radar coordinate system into two-dimensional coordinates in the image coordinate system corresponding to the two-dimensional image.

2. The method according to claim 1, characterized in that, The step of obtaining a two-dimensional image captured by a camera and including a detection target, and a first point cloud obtained by a lidar sensing the detection target includes: Obtaining a fourth point cloud of the current environment sensed by the lidar; Based on a current external parameter value, projecting the fourth point cloud into the two-dimensional image to obtain a fifth point cloud; Determining a second region based on the first region, where the second region includes the first region and the range of the second region is larger than that of the first region; Regarding the data points in the fifth point cloud that fall within the second region as the first point cloud.

3. The method according to claim 2, characterized in that Before the step of respectively projecting the first point cloud into the two-dimensional image based on a selected set of multiple external parameter test values, further includes: Within an optimization space defined by an optimization upper limit and an optimization lower limit, obtaining the set of multiple external parameter test values.

4. The method according to claim 1, wherein The entropy value is characterized by the aggregation degree of the data points in the third point cloud, where the larger the aggregation degree, the smaller the entropy value.

5. The method according to claim 1, wherein The step of calculating the entropy value of the third point cloud includes: Meshing the first region to form a plurality of grids; Counting the number of grids into which the data points in the third point cloud fall; Characterizing the entropy value by the number of grids into which the data points fall, where the larger the number of grids into which the data points fall, the larger the entropy value.

6. The method according to claim 1, wherein The step of obtaining a two-dimensional image captured by a camera and including a detection target, and a first point cloud obtained by a lidar sensing the detection target includes: During the driving of a vehicle equipped with the camera and the lidar, using other vehicles as the detection target to obtain the two-dimensional image and the first point cloud.

7. The method according to claim 1, wherein The step of calculating the entropy value of the third point cloud includes: Determining the centroid of the third point cloud; Calculating the average value of the distances between each data point in the third point cloud and the centroid, and using the average value as the entropy value of the third point cloud.

8. A vehicle system, characterized in that, The vehicle system includes a processor and a memory coupled to each other; a computer program is stored in the memory, and the processor is configured to execute the computer program to implement the steps of the method according to any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program data, and when the program data is executed by a processor, the steps of the method according to any one of claims 1-7 are implemented.

Images