A method and apparatus for calibrating extrinsic parameters of vehicle-mounted and roadside lidar.

Abstract

This application provides a method and apparatus for extrinsic parameter calibration of vehicle-mounted and roadside LiDAR. The method includes: acquiring first point cloud data collected by a vehicle-mounted LiDAR and second point cloud data collected by a roadside LiDAR in a target area; processing the first point cloud data using a target detection model to obtain a first set of three-dimensional target boxes; processing the second point cloud data using the target detection model to obtain a second set of three-dimensional target boxes; constructing an affinity matrix based on the similarity between the first and second three-dimensional target boxes; determining multiple matching target box pairs from a preset number of first and second three-dimensional target boxes based on the affinity matrix; determining first extrinsic parameters of the vehicle-mounted and roadside LiDAR based on the multiple matching target box pairs; and determining second extrinsic parameters of the vehicle-mounted and roadside LiDAR within the range defined by the first extrinsic parameters. This application improves the accuracy of extrinsic parameter calibration for vehicle-mounted and roadside LiDAR.

Description

Technical Field

[0001] This application relates to the field of external parameter calibration technology, and in particular to a method and apparatus for external parameter calibration of vehicle-mounted and road-mounted lidar. Background Technology

[0002] Currently, due to differences in sampling methods and frequencies of vehicle-to-infrastructure (V2I) radars, there are significant differences in point cloud data at the V2I endpoints. These point clouds acquired by different types of LiDARs are called heterogeneous point clouds. Registration of heterogeneous point clouds in V2I multi-sensor fusion is more complex than registration of homogeneous point clouds. Furthermore, existing V2I multi-sensor fusion registration methods primarily rely on perception results to assist in V2I endpoint registration, lacking in the analysis of the perception results themselves, leading to a decline in registration effectiveness. Summary of the Invention

[0003] In view of this, this application provides a method and apparatus for calibrating the extrinsic parameters of vehicle-mounted and road-mounted lidar to solve the above-mentioned technical problems.

[0004] In a first aspect, embodiments of this application provide a method for calibrating the extrinsic parameters of vehicle-mounted and roadside lidar, including:

[0005] Acquire the first point cloud data collected by the vehicle-mounted LiDAR and the second point cloud data collected by the roadside LiDAR in the target area;

[0006] The first point cloud data is processed using an object detection model to obtain a first set of three-dimensional object boxes; the second point cloud data is processed using the same object detection model to obtain a second set of three-dimensional object boxes.

[0007] Based on confidence levels, a preset number of first three-dimensional target boxes are determined from the first set of three-dimensional target boxes, and a preset number of second three-dimensional target boxes are determined from the second set of three-dimensional target boxes.

[0008] An affinity matrix is ​​constructed based on the similarity between a preset number of first 3D bounding boxes and a preset number of second 3D bounding boxes.

[0009] Based on the affinity matrix, multiple matching target box pairs are determined from a preset number of first three-dimensional target boxes and a preset number of second three-dimensional target boxes;

[0010] Based on multiple matched target bounding boxes, the first extrinsic parameters of the vehicle-side and roadside LiDARs are determined; within the range defined by the first extrinsic parameters of the vehicle-side and roadside LiDARs, the second extrinsic parameters of the vehicle-side and roadside LiDARs are determined.

[0011] Further, based on confidence levels, a preset number of first three-dimensional target boxes are determined from the first set of three-dimensional target boxes, and a preset number of second three-dimensional target boxes are determined from the second set of three-dimensional target boxes; including:

[0012] The first three-dimensional target boxes in the first three-dimensional target box set are sorted in descending order of confidence level, and the first preset number of first three-dimensional target boxes in the first three-dimensional target box sequence are taken as the preset number of first three-dimensional target boxes;

[0013] The second three-dimensional target boxes in the second three-dimensional target box set are sorted in descending order of confidence level, and the first preset number of second three-dimensional target boxes in the second three-dimensional target box sequence are taken as the preset number of second three-dimensional target boxes.

[0014] Furthermore, based on the similarity between a preset number of first 3D target boxes and a preset number of second 3D target boxes, an affinity matrix is ​​constructed, including:

[0015] Obtain the 3D coordinates of the eight corner points of each first 3D target box, calculate the 3D coordinates of the center point of the first 3D target box; subtract the 3D coordinates of the corresponding center point from the 3D coordinates of the eight corner points of the first 3D target box to obtain the decentralized 3D coordinates of the eight corner points of the first 3D target box.

[0016] Obtain the 3D coordinates of the eight corner points of each second 3D target box, and calculate the 3D coordinates of the center point of the second 3D target box; subtract the 3D coordinates of the corresponding center point from the 3D coordinates of the eight corner points of the second 3D target box to obtain the decentralized 3D coordinates of the eight corner points of the second 3D target box.

[0017] For the i-th first 3D bounding box and the j-th second 3D bounding box, calculate the correlation matrix H. ij :

[0018] H ij =X i Y j T

[0019] Where 1≤i≤K, 1≤j≤K; K is the preset quantity; X i Y is a 3×8 matrix composed of the decentralized 3D coordinates of the 8 corner points of the i-th first 3D target box; j It is a 3×8 matrix composed of the decentralized 3D coordinates of the 8 corner points of the j-th second 3D target box;

[0020] For matrix H ij Perform SVD decomposition:

[0021] H ij =U ij ΛV ij T

[0022] Among them, Uij and V ij All are unitary matrices;

[0023] Using the i-th first 3D target box and the j-th second 3D target box as a matching pair, the rotation matrix R between the vehicle-side lidar body coordinate system and the road-side lidar body coordinate system is calculated. ij for:

[0024] R ij =V ij U ij T

[0025] The translation vector t between the vehicle-mounted lidar coordinate system and the road-end lidar coordinate system ij for:

[0026]

[0027] in, Let be the 3D coordinates of the center point of the i-th first 3D target bounding box. The coordinates of the center point of the j-th second 3D target box are given.

[0028] This yields a K×K set of rotation matrices and translation vectors;

[0029] Using the rotation matrix R between the vehicle-mounted lidar body coordinate system and the road-end lidar body coordinate system ij Translation vector t ij Transform the K first 3D target boxes into the coordinate system of the roadside lidar to obtain K third 3D target boxes; calculate the intersection-union ratio F between the i-th second 3D target box and the j-th third 3D target box. ij ;

[0030] The intersection-union ratio F of K×K second 3D bounding boxes and j-th third 3D bounding boxes is calculated. ij The sum of the values ​​is used as the similarity between the second and third 3D bounding boxes:

[0031]

[0032] Based on all similarities A ij Construct the affinity matrix A.

[0033] Further, based on the affinity matrix, multiple matching target box pairs are determined from a preset number of first three-dimensional target boxes and a preset number of second three-dimensional target boxes; including:

[0034] Construct the objective function f(W):

[0035]

[0036] Where matrix W represents the decision variable w ij The matrix formed, w ij The value of can be 0 or 1; where w ij =1 indicates that the i-th second 3D bounding box and the j-th third 3D bounding box are matched, w ij =0 indicates that the i-th second 3D bounding box and the j-th third 3D bounding box do not match, and there is

[0037] The maximum value of f(W) is obtained by using the Hungarian algorithm to obtain the second and third 3D target boxes of K pairs of matching;

[0038] Obtain the matching scores of K pairs of matching second and third 3D target boxes from the affinity matrix A, and take the matching pairs with matching scores greater than a preset threshold as the final matching pairs.

[0039] Furthermore, within the range defined by the first extrinsic parameters of the vehicle-side and roadside lidars, the second extrinsic parameters of the vehicle-side and roadside lidars are determined; including:

[0040] When the final number of matching pairs is N, the N first extrinsic parameters obtained from the N matching second 3D target boxes and first 3D target boxes are: L,

[0041] Using the N first extrinsic parameters as the initial positions of the N extrinsic parameter particles in the first iteration, an iterative search is performed within the range of the initial positions. The position of the nth extrinsic parameter particle after the (d+1)th iteration is... for:

[0042]

[0043]

[0044] Where d is the number of iterations, w is the inertial movement factor of the external parameter particle, and 1≤n≤N; This represents the moving speed of the nth extrinsic particle after the d-th iteration; c1 represents the position of the nth extrinsic particle after the d-th iteration; c2 represents the individual trust level of the extrinsic particle, and c3 represents the group trust level of the extrinsic particles. This represents the best position traversed by the nth extrinsic particle after the dth iteration; gbest d This represents the best position reached by the N extrinsic particles after the d-th iteration;

[0045] When the preset total number of iterations is reached, gbest will... d It serves as the second external parameter for vehicle-mounted and roadside LiDAR.

[0046] Secondly, embodiments of this application provide a vehicle-mounted and road-mounted lidar extrinsic parameter calibration device, comprising:

[0047] The acquisition unit is used to acquire the first point cloud data collected by the vehicle-mounted lidar and the second point cloud data collected by the roadside lidar in the target area.

[0048] The detection unit is used to process the first point cloud data using a target detection model to obtain a first set of three-dimensional target boxes; and to process the second point cloud data using the target detection model to obtain a second set of three-dimensional target boxes.

[0049] The first determining unit is configured to determine a preset number of first three-dimensional target boxes from the first three-dimensional target box set and a preset number of second three-dimensional target boxes from the second three-dimensional target box set based on confidence levels.

[0050] The building unit is used to construct an affinity matrix based on the similarity between a preset number of first 3D target boxes and a preset number of second 3D target boxes;

[0051] The second determining unit is used to determine multiple matching target box pairs from a preset number of first three-dimensional target boxes and a preset number of second three-dimensional target boxes based on the affinity matrix.

[0052] The third determining unit is used to determine the first extrinsic parameters of the vehicle-end and road-end LiDAR based on multiple matching target box pairs; and to determine the second extrinsic parameters of the vehicle-end and road-end LiDAR within the range defined by the first extrinsic parameters of the vehicle-end and road-end LiDAR.

[0053] Thirdly, embodiments of this application provide an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method of embodiments of this application.

[0054] Fourthly, an embodiment of this application provides a computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, and the computer program is executed by a processor to implement the method of the embodiment of this application.

[0055] This application improves the accuracy of external parameter calibration for vehicle-mounted and roadside lidar. Attached Figure Description

[0056] To more clearly illustrate the technical solutions in the specific embodiments of this application or the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0057] Figure 1 A flowchart illustrating the extrinsic parameter calibration method for vehicle-side and roadside LiDAR provided in this application embodiment;

[0058] Figure 2 Functional structure diagram of the vehicle-side and road-side lidar extrinsic parameter calibration device provided in the embodiments of this application;

[0059] Figure 3 A functional structure diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0060] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0061] Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0062] First, a brief introduction to the design concept of the embodiments of this application will be given.

[0063] Due to the different sampling methods and frequencies of vehicle-to-infrastructure (V2I) radars, there are significant differences in point cloud data at the V2I endpoints. These point clouds acquired by different types of LiDARs are called heterogeneous point clouds. Therefore, the registration of heterogeneous point clouds in the V2I multi-sensor fusion problem is more complex than the registration of homogeneous point clouds. At the same time, existing V2I multi-sensor fusion registration methods mainly rely on the perception results to assist in the registration of V2I point clouds, lacking the mining of the perception results themselves, which leads to a decline in the registration effect.

[0064] To address the aforementioned technical issues, this application provides a method for extrinsic parameter calibration of vehicle-side and road-side LiDAR. This method integrates target detection results from both vehicle-side and road-side systems, replacing the registration of heterogeneous point clouds at the vehicle and road ends with the registration of perception detection boxes. This allows for real-time and automatic completion of LiDAR point cloud registration tasks at the vehicle and road ends, becoming an important foundation for vehicle-road cooperative perception and vehicle localization tasks.

[0065] The registration task in vehicle-to-infrastructure (V2I) multi-sensor fusion suffers from reduced calibration results due to the combined effects of noise, outliers, density differences, partial overlap, and scale differences in heterogeneous point clouds. The method in this application improves the registration speed and reduces the impact of heterogeneous point cloud noise on the registration results.

[0066] After introducing the application scenarios and design concepts of the embodiments of this application, the technical solutions provided by the embodiments of this application will be described below.

[0067] like Figure 1 As shown, this application provides a method for calibrating the extrinsic parameters of vehicle-mounted and roadside lidar, including steps 101-106:

[0068] Step 101: Acquire the first point cloud data collected by the vehicle-mounted LiDAR and the second point cloud data collected by the roadside LiDAR in the target area;

[0069] Step 102: Process the first point cloud data using the target detection model to obtain a first set of three-dimensional target boxes; process the second point cloud data using the target detection model to obtain a second set of three-dimensional target boxes;

[0070] Each 3D target bounding box contains coordinate and size information.

[0071] Step 103: Based on the confidence level, determine a preset number of first three-dimensional target boxes from the first set of three-dimensional target boxes, and determine a preset number of second three-dimensional target boxes from the second set of three-dimensional target boxes;

[0072] Among these methods, filtering 3D target boxes using confidence scores can yield higher-quality 3D target boxes.

[0073] Step 104: Construct an affinity matrix based on the similarity between a preset number of first 3D bounding boxes and a preset number of second 3D bounding boxes;

[0074] Step 105: Based on the affinity matrix, determine multiple matching target box pairs from a preset number of first three-dimensional target boxes and a preset number of second three-dimensional target boxes;

[0075] Step 106: Based on multiple matched target box pairs, determine the first extrinsic parameters of the vehicle-side and roadside LiDAR; within the range defined by the first extrinsic parameters of the vehicle-side and roadside LiDAR, determine the second extrinsic parameters of the vehicle-side and roadside LiDAR.

[0076] To address the shortcomings of existing heterogeneous point cloud registration methods, this application proposes a registration method for sensing detection boxes based on heterogeneous point clouds, which improves the registration accuracy of vehicle-to-infrastructure (V2I) lidar.

[0077] In some embodiments, determining a preset number of first three-dimensional target boxes from the first set of three-dimensional target boxes and a preset number of second three-dimensional target boxes from the second set of three-dimensional target boxes based on confidence levels; including:

[0078] The first three-dimensional target boxes in the first three-dimensional target box set are sorted in descending order of confidence level, and the first preset number of first three-dimensional target boxes in the first three-dimensional target box sequence are taken as the preset number of first three-dimensional target boxes;

[0079] The second three-dimensional target boxes in the second three-dimensional target box set are sorted in descending order of confidence level, and the first preset number of second three-dimensional target boxes in the second three-dimensional target box sequence are taken as the preset number of second three-dimensional target boxes.

[0080] In some embodiments, based on a preset number of first three-dimensional target boxes and a preset number of second three-dimensional target boxes, the three-dimensional coordinates of the eight corner points of each first three-dimensional target box are obtained, and the three-dimensional coordinates of the center point of the first three-dimensional target box are calculated; the three-dimensional coordinates of the eight corner points of the first three-dimensional target box are subtracted from the three-dimensional coordinates of the corresponding center point to obtain the decentralized three-dimensional coordinates of the eight corner points of the first three-dimensional target box.

[0081] Obtain the 3D coordinates of the eight corner points of each second 3D target box, and calculate the 3D coordinates of the center point of the second 3D target box; subtract the 3D coordinates of the corresponding center point from the 3D coordinates of the eight corner points of the second 3D target box to obtain the decentralized 3D coordinates of the eight corner points of the second 3D target box.

[0082] For the i-th first 3D bounding box and the j-th second 3D bounding box, calculate the correlation matrix H. ij :

[0083] H ij =X i Y j T

[0084] Where 1≤i≤K, 1≤j≤K; K is the preset quantity; X i Y is a 3×8 matrix composed of the decentralized 3D coordinates of the 8 corner points of the i-th first 3D target box; jIt is a 3×8 matrix composed of the decentralized 3D coordinates of the 8 corner points of the j-th second 3D target box;

[0085] For matrix H ij Perform SVD decomposition:

[0086] H ij =U ij ΛV ij T

[0087] Among them, U ij and V ij All are unitary matrices;

[0088] Taking the i-th first 3D target box and the j-th second 3D target box as a matching pair, we need to find a rigid transformation T = (R, t) that aligns these two point sets. This involves calculating the extrinsic parameters of the two 3D target boxes, where R is the rotation matrix and t is the translation vector. In other words, we need to solve for:

[0089]

[0090] The calculated rotation matrix R between the vehicle-mounted lidar body coordinate system and the road-end lidar body coordinate system ij for:

[0091] R ij =V ij U ij T

[0092] The translation vector t between the vehicle-mounted lidar coordinate system and the road-end lidar coordinate system ij for:

[0093]

[0094] in, Let be the 3D coordinates of the center point of the i-th first 3D target bounding box. The coordinates of the center point of the j-th second 3D target box are given.

[0095] This yields a K×K set of rotation matrices and translation vectors;

[0096] Using the rotation matrix R between the vehicle-mounted lidar body coordinate system and the road-end lidar body coordinate system ij Translation vector t ij Transform the K first 3D target boxes into the coordinate system of the roadside lidar to obtain K third 3D target boxes; calculate the intersection-union ratio F between the i-th second 3D target box and the j-th third 3D target box. ij ;

[0097] The intersection-union ratio F of K×K second 3D bounding boxes and j-th third 3D bounding boxes is calculated. ij The sum of the values ​​is used as the similarity between the second and third 3D bounding boxes:

[0098]

[0099] Based on all similarities A ij Construct the affinity matrix A.

[0100] In some embodiments, based on the affinity matrix, multiple matching target box pairs are determined from a preset number of first three-dimensional target boxes and a preset number of second three-dimensional target boxes; including:

[0101] Construct the objective function f(W):

[0102]

[0103] Where matrix W represents the decision variable w ij The matrix formed, w ij The value of can be 0 or 1; where w ij =1 indicates that the i-th second 3D bounding box and the j-th third 3D bounding box are matched, w ij =0 indicates that the i-th second 3D bounding box and the j-th third 3D bounding box do not match, and there is

[0104] The maximum value of f(W) is obtained by using the Hungarian algorithm to obtain the second and third 3D target boxes of K pairs of matching;

[0105] Obtain the matching scores of K pairs of matching second and third 3D target boxes from the affinity matrix A, and take the matching pairs with matching scores greater than a preset threshold as the final matching pairs.

[0106] In some embodiments, within the range defined by the first extrinsic parameters of the vehicle-mounted and roadside lidars, a second extrinsic parameter is determined; including:

[0107] When the final number of matching pairs is N, the N first extrinsic parameters obtained from the N matching second 3D target boxes and first 3D target boxes are: L,

[0108] Using the N first extrinsic parameters as the initial positions of the N extrinsic parameter particles in the first iteration, an iterative search is performed within the range of the initial positions. The position of the nth extrinsic parameter particle after the (d+1)th iteration is... for:

[0109]

[0110] Where d is the number of iterations, w is the inertial movement factor of the external parameter particle, and 1≤n≤N; This represents the moving speed of the nth extrinsic particle after the d-th iteration; c1 represents the position of the nth extrinsic particle after the d-th iteration; c2 represents the individual trust level of the extrinsic particle, and c3 represents the group trust level of the extrinsic particles. This represents the best position traversed by the nth extrinsic particle after the dth iteration; gbest d This represents the best position reached by the N extrinsic particles after the d-th iteration;

[0111] Based on the same inventive concept, this application provides a vehicle-mounted and road-mounted lidar extrinsic parameter calibration device, see reference. Figure 2 As shown, the vehicle-end and road-end lidar extrinsic parameter calibration device 200 provided in this application embodiment includes at least:

[0112] The acquisition unit 201 is used to acquire the first point cloud data collected by the vehicle-mounted lidar and the second point cloud data collected by the road-mounted lidar in the target area.

[0113] The detection unit 202 is used to process the first point cloud data using a target detection model to obtain a first set of three-dimensional target boxes; and to process the second point cloud data using the target detection model to obtain a second set of three-dimensional target boxes.

[0114] The first determining unit 203 is configured to determine a preset number of first three-dimensional target boxes from the first three-dimensional target box set and a preset number of second three-dimensional target boxes from the second three-dimensional target box set based on confidence level.

[0115] Construction unit 204 is used to construct an affinity matrix based on the similarity between a preset number of first three-dimensional target boxes and a preset number of second three-dimensional target boxes;

[0116] The second determining unit 205 is used to determine multiple matching target box pairs from a preset number of first three-dimensional target boxes and a preset number of second three-dimensional target boxes based on the affinity matrix.

[0117] The third determining unit 206 is used to determine the first extrinsic parameters of the vehicle-end and road-end LiDAR based on multiple matching target box pairs; and to determine the second extrinsic parameters of the vehicle-end and road-end LiDAR within the range defined by the first extrinsic parameters of the vehicle-end and road-end LiDAR.

[0118] It should be noted that the principle of the vehicle-end and road-end lidar extrinsic parameter calibration device 200 provided in this application embodiment to solve the technical problem is similar to the method provided in this application embodiment. Therefore, the implementation of the vehicle-end and road-end lidar extrinsic parameter calibration device 200 provided in this application embodiment can refer to the implementation of the method provided in this application embodiment, and the repeated parts will not be described again.

[0119] Based on the same inventive concept, this application also provides an electronic device, see reference. Figure 3 As shown, the electronic device 300 provided in this application embodiment includes at least: a processor 301, a memory 302, and a computer program stored in the memory 302 and executable on the processor 301. When the processor 301 executes the computer program, it implements the vehicle-side and road-side lidar extrinsic parameter calibration method provided in this application embodiment.

[0120] The electronic device 300 provided in this application embodiment may further include a bus 303 connecting different components (including processor 301 and memory 302). The bus 303 represents one or more types of bus structures, including memory bus, peripheral bus, local area bus, etc.

[0121] The memory 302 may include a readable medium in the form of volatile memory, such as random access memory (RAM) 3021 and / or cache memory 3022, and may further include read-only memory (ROM) 3023.

[0122] The memory 302 may also include a program tool 3025 having a set (at least one) of program modules 3024, including but not limited to: an operating subsystem, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0123] Electronic device 300 can also communicate with one or more external devices 304 (e.g., keyboard, remote control, etc.), and with one or more devices that enable a user to interact with electronic device 300 (e.g., mobile phone, computer, etc.), and / or with any device that enables electronic device 300 to communicate with one or more other electronic devices 300 (e.g., router, modem, etc.). This communication can be performed through input / output (I / O) interface 305. Furthermore, electronic device 300 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) through network adapter 306. Figure 3 As shown, network adapter 306 communicates with other modules of electronic device 300 via bus 303. It should be understood that, although... Figure 3 As not shown, other hardware and / or software modules may be used in conjunction with electronic device 300, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, Redundant Arrays of Independent Disks (RAID) subsystems, tape drives, and data backup storage subsystems.

[0124] It should be noted that, Figure 3 The electronic device 300 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0125] This application also provides a computer-readable storage medium storing computer instructions. When executed by a processor, these computer instructions implement the vehicle-side and roadside LiDAR extrinsic parameter calibration method provided in this application. Specifically, the executable program can be built into or installed in the electronic device 300, so that the electronic device 300 can implement the vehicle-side and roadside LiDAR extrinsic parameter calibration method provided in this application by executing the built-in or installed executable program.

[0126] The vehicle-side and road-side lidar extrinsic parameter calibration method provided in this application embodiment can also be implemented as a program product. The program product includes program code. When the program product can run on the electronic device 300, the program code is used to make the electronic device 300 execute the vehicle-side and road-side lidar extrinsic parameter calibration method provided in this application embodiment.

[0127] The program product provided in this application embodiment can be any combination of one or more readable media, wherein the readable media can be a readable signal medium or a readable storage medium, and the readable storage medium can be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination thereof. Specifically, more specific examples of readable storage media (a non-exhaustive list) include: electrical connections with one or more wires, portable disks, hard disks, RAM, ROM, erasable programmable read-only memory (EPROM), optical fibers, portable compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0128] The program product provided in this application embodiment can be a CD-ROM and include program code, and can also run on a computing device. However, the program product provided in this application embodiment is not limited thereto. In this application embodiment, the readable storage medium can be any tangible medium that contains or stores a program, which can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0129] It should be noted that although several units or sub-units of the device have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of this application, the features and functions of two or more units described above can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided and embodied by multiple units.

[0130] Furthermore, although the operations of the method of this application are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all the operations shown must be performed to achieve the desired result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0131] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application and are not intended to limit it. Although this application has been described in detail with reference to the embodiments, those skilled in the art should understand that modifications or equivalent substitutions to the technical solutions of this application do not depart from the spirit and scope of the technical solutions of this application, and should all be covered within the scope of the claims of this application.

Claims

1. A method for calibrating extrinsic parameters of vehicle-mounted and roadside lidar, characterized in that, include: Acquire the first point cloud data collected by the vehicle-mounted LiDAR and the second point cloud data collected by the roadside LiDAR in the target area; The first point cloud data is processed using an object detection model to obtain a first set of three-dimensional object boxes; the second point cloud data is processed using the same object detection model to obtain a second set of three-dimensional object boxes. Based on confidence levels, a preset number of first three-dimensional target boxes are determined from the first set of three-dimensional target boxes, and a preset number of second three-dimensional target boxes are determined from the second set of three-dimensional target boxes. An affinity matrix is ​​constructed based on the similarity between a preset number of first 3D bounding boxes and a preset number of second 3D bounding boxes. Based on the affinity matrix, multiple matching target box pairs are determined from a preset number of first three-dimensional target boxes and a preset number of second three-dimensional target boxes; Based on multiple matching target bounding boxes, the first extrinsic parameters of the vehicle-end and road-end LiDAR are determined; within the range defined by the first extrinsic parameters of the vehicle-end and road-end LiDAR, the second extrinsic parameters of the vehicle-end and road-end LiDAR are determined. Based on the similarity between a preset number of first 3D bounding boxes and a preset number of second 3D bounding boxes, an affinity matrix is ​​constructed, including: Obtain the 3D coordinates of the eight corner points of each first 3D target box, calculate the 3D coordinates of the center point of the first 3D target box; subtract the 3D coordinates of the corresponding center point from the 3D coordinates of the eight corner points of the first 3D target box to obtain the decentralized 3D coordinates of the eight corner points of the first 3D target box. Obtain the 3D coordinates of the eight corner points of each second 3D target box, and calculate the 3D coordinates of the center point of the second 3D target box; subtract the 3D coordinates of the corresponding center point from the 3D coordinates of the eight corner points of the second 3D target box to obtain the decentralized 3D coordinates of the eight corner points of the second 3D target box. For the i-th first 3D bounding box and the j-th second 3D bounding box, calculate the correlation matrix H. ij : H ij =X i Y j T Where 1≤i≤K, 1≤j≤K; K is the preset quantity; X i Y is a 3×8 matrix composed of the decentralized 3D coordinates of the 8 corner points of the i-th first 3D target box; j It is a 3×8 matrix composed of the decentralized 3D coordinates of the 8 corner points of the j-th second 3D target box; For matrix H ij Perform SVD decomposition: H ij =U ij ΛV ij T Among them, U ij and V ij All are unitary matrices; Using the i-th first 3D target box and the j-th second 3D target box as a matching pair, the rotation matrix R between the vehicle-side lidar body coordinate system and the road-side lidar body coordinate system is calculated. ij for: R ij =V ij U ij T The translation vector t between the vehicle-mounted lidar coordinate system and the road-end lidar coordinate system ij for: in, Let be the 3D coordinates of the center point of the i-th first 3D target bounding box. The coordinates of the center point of the j-th second 3D target box are given. This yields a K×K set of rotation matrices and translation vectors; Using the rotation matrix R between the vehicle-mounted lidar body coordinate system and the road-end lidar body coordinate system ij Translation vector t ij Transform the K first 3D target boxes into the coordinate system of the roadside lidar to obtain K third 3D target boxes; calculate the intersection-union ratio F between the i-th second 3D target box and the j-th third 3D target box. ij ; The intersection-union ratio F of K×K second 3D bounding boxes and j-th third 3D bounding boxes is calculated. ij The sum of the values ​​is used as the similarity between the second and third 3D bounding boxes: Based on all similarities A ij Construct the affinity matrix A.

2. The method according to claim 1, characterized in that, Based on confidence levels, a predetermined number of first 3D target boxes are determined from the first set of 3D target boxes, and a predetermined number of second 3D target boxes are determined from the second set of 3D target boxes; including: The first three-dimensional target boxes in the first three-dimensional target box set are sorted in descending order of confidence level, and the first preset number of first three-dimensional target boxes in the first three-dimensional target box sequence are taken as the preset number of first three-dimensional target boxes; The second three-dimensional target boxes in the second three-dimensional target box set are sorted in descending order of confidence level, and the first preset number of second three-dimensional target boxes in the second three-dimensional target box sequence are taken as the preset number of second three-dimensional target boxes.

3. The method according to claim 1, characterized in that, Based on the affinity matrix, multiple matching target box pairs are determined from a preset number of first 3D target boxes and a preset number of second 3D target boxes; including: Construct the objective function f(W): Where matrix W represents the decision variable w ij The matrix formed, w ij The value of can be 0 or 1; where w ij =1 indicates that the i-th second 3D bounding box and the j-th third 3D bounding box are matched, w ij =0 indicates that the i-th second 3D bounding box and the j-th third 3D bounding box do not match, and there is The maximum value of f(W) is obtained by using the Hungarian algorithm to obtain the second and third 3D target boxes of K pairs of matching; Obtain the matching scores of K pairs of matching second and third 3D target boxes from the affinity matrix A, and take the matching pairs with matching scores greater than a preset threshold as the final matching pairs.

4. The method according to claim 3, characterized in that, Within the range defined by the first extrinsic parameters of the vehicle-mounted and roadside lidar, determine the second extrinsic parameters of the vehicle-mounted and roadside lidar; including: When the final number of matching pairs is N, the N first extrinsic parameters obtained from the N matching second 3D target boxes and first 3D target boxes are: Using the N first extrinsic parameters as the initial positions of the N extrinsic parameter particles in the first iteration, an iterative search is performed within the range of the initial positions. The position of the nth extrinsic parameter particle after the (d+1)th iteration is... for: Where d is the number of iterations, w is the inertial movement factor of the external parameter particle, and 1≤n≤N; This represents the moving speed of the nth extrinsic particle after the d-th iteration; c1 represents the position of the nth extrinsic particle after the d-th iteration; c2 represents the individual trust level of the extrinsic particle, and c3 represents the group trust level of the extrinsic particles. This represents the best position traversed by the nth extrinsic particle after the dth iteration; gbest d This represents the best position reached by the N extrinsic particles after the d-th iteration; When the preset total number of iterations is reached, gbest will... d It serves as the second external parameter for vehicle-mounted and roadside LiDAR.

5. A device for calibrating the extrinsic parameters of a vehicle-mounted and roadside lidar, characterized in that, include: The acquisition unit is used to acquire the first point cloud data collected by the vehicle-mounted lidar and the second point cloud data collected by the roadside lidar in the target area. The detection unit is used to process the first point cloud data using a target detection model to obtain a first set of three-dimensional target boxes; and to process the second point cloud data using the target detection model to obtain a second set of three-dimensional target boxes. The first determining unit is configured to determine a preset number of first three-dimensional target boxes from the first three-dimensional target box set and a preset number of second three-dimensional target boxes from the second three-dimensional target box set based on confidence levels. The building unit is used to construct an affinity matrix based on the similarity between a preset number of first 3D target boxes and a preset number of second 3D target boxes; The second determining unit is used to determine multiple matching target box pairs from a preset number of first three-dimensional target boxes and a preset number of second three-dimensional target boxes based on the affinity matrix. The third determining unit is used to determine the first extrinsic parameters of the vehicle-end and road-end LiDAR based on multiple matching target box pairs; and to determine the second extrinsic parameters of the vehicle-end and road-end LiDAR within the range defined by the first extrinsic parameters of the vehicle-end and road-end LiDAR. Building units are specifically used for: Obtain the 3D coordinates of the eight corner points of each first 3D target box, calculate the 3D coordinates of the center point of the first 3D target box; subtract the 3D coordinates of the corresponding center point from the 3D coordinates of the eight corner points of the first 3D target box to obtain the decentralized 3D coordinates of the eight corner points of the first 3D target box. Obtain the 3D coordinates of the eight corner points of each second 3D target box, and calculate the 3D coordinates of the center point of the second 3D target box; subtract the 3D coordinates of the corresponding center point from the 3D coordinates of the eight corner points of the second 3D target box to obtain the decentralized 3D coordinates of the eight corner points of the second 3D target box. For the i-th first 3D bounding box and the j-th second 3D bounding box, calculate the correlation matrix H. ij : H ij =X i Y j T Where 1≤i≤K, 1≤j≤K; K is the preset quantity; X i Y is a 3×8 matrix composed of the decentralized 3D coordinates of the 8 corner points of the i-th first 3D target box; j It is a 3×8 matrix composed of the decentralized 3D coordinates of the 8 corner points of the j-th second 3D target box; For matrix H ij Perform SVD decomposition: H ij =U ij ΛV ij T Among them, U ij and V ij All are unitary matrices; Using the i-th first 3D target box and the j-th second 3D target box as a matching pair, the rotation matrix R between the vehicle-side lidar body coordinate system and the road-side lidar body coordinate system is calculated. ij for: R ij =V ij U ij T The translation vector t between the vehicle-mounted lidar coordinate system and the road-end lidar coordinate system ij for: in, Let be the 3D coordinates of the center point of the i-th first 3D target bounding box. The coordinates of the center point of the j-th second 3D target box are given. This yields a K×K set of rotation matrices and translation vectors; Using the rotation matrix R between the vehicle-mounted lidar body coordinate system and the road-end lidar body coordinate system ij Translation vector t ij Transform the K first 3D target boxes into the coordinate system of the roadside lidar to obtain K third 3D target boxes; calculate the intersection-union ratio F between the i-th second 3D target box and the j-th third 3D target box. ij ; The intersection-union ratio F of K×K second 3D bounding boxes and j-th third 3D bounding boxes is calculated. ij The sum of the values ​​is used as the similarity between the second and third 3D bounding boxes: Based on all similarities A ij Construct the affinity matrix A.

6. An electronic device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the method as described in any one of claims 1-4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed by a processor, implement the method as described in any one of claims 1-4.

Images