Vehicle end and road end laser radar external parameter calibration method and device

Through the target detection model and affinity matrix matching technology of vehicle-road lidar, the problem of heterogeneous point cloud registration complexity is solved, the registration accuracy and speed of vehicle-road lidar are improved, and the basis of vehicle-road collaborative perception is made. .

CN119087408AActive Publication Date: 2024-12-06TSINGHUA UNIVERSITY
View PDF 8 Cites 0 Cited by

Patent Information

Application Number
CN202410964621.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-18
Publication Date
2024-12-06
Estimated Expiration
2044-07-18

AI Technical Summary

Technical Problem

The heterogeneous point cloud registration in vehicle-road multi-sensor fusion is complex. The existing methods rely on perceived results to assist registration, and lack the mining of perceived results, resulting in a decrease in registration effect.

Method used

The target detection model of vehicle-end and road-end lidar is used to process point cloud data, build an affinity matrix, determine external parameters through the matching of the target box, and improve registration accuracy using Hungarian algorithms and particle iterative search.

Benefits of technology

The accuracy and speed of point cloud registration of vehicle-road-side lidar is improved, and the impact of heterogeneous point cloud noise on the registration results is reduced, and an important foundation for vehicle-road collaborative perception and vehicle positioning is realized.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119087408A_ABST
    Figure CN119087408A_ABST
Patent Text Reader

Abstract

The invention provides a vehicle-end and road-end laser radar external parameter calibration method and device. The method comprises the following steps: acquiring first point cloud data acquired by a vehicle-end laser radar and second point cloud data acquired by a road-end laser radar in a target area; processing the first point cloud data by using a target detection model to obtain a first three-dimensional target frame set; processing the second point cloud data by using the target detection model to obtain a second three-dimensional target frame set; constructing an affinity matrix based on the similarity of the first three-dimensional target frame and the second three-dimensional target frame; determining a plurality of matched target frame pairs from a preset number of first three-dimensional target frames and a preset number of second three-dimensional target frames based on the affinity matrix; determining first external parameters of the vehicle end laser radar and the road end laser radar based on the plurality of matched target frame pairs; and determining second external parameters of the vehicle-end laser radar and the road-end laser radar in a range limited by the first external parameters of the vehicle-end laser radar and the road-end laser radar. According to the invention, the precision of vehicle-end and road-end laser radar external parameter calibration is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of external parameter calibration, and in particular to a method and device for calibrating external parameters of vehicle-side and road-side laser radars. Background Art

[0002] At present, due to the different sampling methods and frequencies of vehicle-road radars, there are large differences in the vehicle-road endpoint cloud data. The point clouds obtained by these different types of lidars are called heterogeneous point clouds. The registration of heterogeneous point clouds in the vehicle-road multi-sensor fusion problem is more complicated than the registration of homologous point clouds. At the same time, the existing registration methods for vehicle-road multi-sensor fusion basically rely on the perception results to assist the registration of the vehicle-road endpoint clouds, and lack the mining of the perception results themselves, resulting in a decrease in the registration effect. Summary of the invention

[0003] In view of this, the present application provides a method and device for calibrating external parameters of vehicle-side and road-side lidar to solve the above-mentioned technical problems.

[0004] In a first aspect, an embodiment of the present application provides a method for calibrating external parameters of a vehicle-side and road-side laser radar, including:

[0005] Acquire first point cloud data collected by the vehicle-side laser radar and second point cloud data collected by the road-side laser radar in the target area;

[0006] The first point cloud data is processed using a target detection model to obtain a first three-dimensional target frame set; the second point cloud data is processed using a target detection model to obtain a second three-dimensional target frame set;

[0007] Determining a preset number of first three-dimensional object frames from the first three-dimensional object frame set and a preset number of second three-dimensional object frames from the second three-dimensional object frame set based on the confidence;

[0008] constructing an affinity matrix based on similarities between a preset number of first three-dimensional target frames and a preset number of second three-dimensional target frames;

[0009] Based on the affinity matrix, determining a plurality of matching target frame pairs from a preset number of first three-dimensional target frames and a preset number of second three-dimensional target frames;

[0010] Based on multiple matching target frame pairs, the first external parameters of the vehicle-side and road-side laser radars are determined; within the range limited by the first external parameters of the vehicle-side and road-side laser radars, the second external parameters of the vehicle-side and road-side laser radars are determined.

[0011] Further, based on the confidence, respectively determining a preset number of first three-dimensional object frames from the first three-dimensional object frame set, and determining a preset number of second three-dimensional object frames from the second three-dimensional object frame set; comprising:

[0012] Arrange the first three-dimensional object frames in the first three-dimensional object frame set in descending order according to confidence, and use the first preset number of first three-dimensional object frames in the first three-dimensional object frame sequence as the preset number of first three-dimensional object frames;

[0013] The second 3D object frames in the second 3D object frame set are arranged in descending order according to confidence, and the first preset number of second 3D object frames in the second 3D object frame sequence are used as the preset number of second 3D object frames.

[0014] Furthermore, based on the similarity between a preset number of first three-dimensional target frames and a preset number of second three-dimensional target frames, an affinity matrix is ​​constructed, including:

[0015] Obtaining the three-dimensional coordinates of the eight corner points of each first three-dimensional object frame, and calculating the three-dimensional coordinates of the center point of the first three-dimensional object frame; subtracting the three-dimensional coordinates of the corresponding center point from the three-dimensional coordinates of the eight corner points of the first three-dimensional object frame to obtain decentralized three-dimensional coordinates of the eight corner points of the first three-dimensional object frame;

[0016] Obtain the three-dimensional coordinates of the eight corner points of each second three-dimensional object frame, and calculate the three-dimensional coordinates of the center point of the second three-dimensional object frame; subtract the three-dimensional coordinates of the corresponding center point from the three-dimensional coordinates of the eight corner points of the second three-dimensional object frame to obtain the decentralized three-dimensional coordinates of the eight corner points of the second three-dimensional object frame;

[0017] For the i-th first 3D target frame and the j-th second 3D target frame, calculate the association matrix H ij :

[0018] H ij =X i Y j T

[0019] Where, 1≤i≤K, 1≤j≤K; K is the preset number; X i Y is a 3×8 matrix consisting of the decentralized 3D coordinates of the 8 corner points of the i-th first 3D target box; j A 3×8 matrix consisting of the decentralized 3D coordinates of the 8 corner points of the j-th second 3D target box;

[0020] For the matrix H ij Perform SVD decomposition:

[0021] H ij =U ij ΛV ij T

[0022] Among them, Uij and V ij All are unitary matrices;

[0023] The i-th first 3D target frame and the j-th second 3D target frame are taken as a matching pair, and the rotation matrix R between the vehicle-side laser radar body coordinate system and the road-side laser radar body coordinate system is calculated. ij for:

[0024] R ij =V ij U ij T

[0025] Then the translation vector t between the vehicle-side laser radar body coordinate system and the road-side laser radar body coordinate system is ij for:

[0026]

[0027] in, is the 3D coordinate of the center point of the i-th first 3D target frame, is the three-dimensional coordinate of the center point of the j-th second three-dimensional target frame;

[0028] Thus, we get K×K groups of rotation matrices and translation vectors;

[0029] The rotation matrix R between the vehicle-side laser radar body coordinate system and the road-side laser radar body coordinate system is used ij and the translation vector t ij , transform the K first three-dimensional target frames into the road-side laser radar body coordinate system to obtain K third three-dimensional target frames; calculate the intersection and union ratio F of the i-th second three-dimensional target frame and the j-th third three-dimensional target frame ij ;

[0030] The intersection and union of the K×K i-th second 3D target frame and the j-th third 3D target frame is calculated as F ij The sum of is taken as the similarity between the second 3D target frame and the third 3D target frame:

[0031]

[0032] Based on all similarities A ij Construct affinity matrix A.

[0033] Further, based on the affinity matrix, a plurality of matching target frame pairs are determined from a preset number of first three-dimensional target frames and a preset number of second three-dimensional target frames; including:

[0034] Construct the objective function f(W):

[0035]

[0036] Among them, the matrix W is the decision variable w ij The matrix composed of ij The value of is 0 or 1; ij = 1 means that the i-th second 3D target frame and the j-th third 3D target frame are matched, w ij = 0 means that the i-th second 3D target frame and the j-th third 3D target frame do not match, and there is

[0037] The Hungarian algorithm is used to solve the maximum value of f(W) and obtain K pairs of matching second 3D target frames and third 3D target frames;

[0038] The matching scores of K pairs of matching second 3D target frames and third 3D target frames are obtained from the affinity matrix A, and the matching pairs with matching scores greater than a preset threshold are taken as the final matching pairs.

[0039] Further, within the range defined by the first external parameters of the vehicle-side laser radar and the road-side laser radar, second external parameters of the vehicle-side laser radar and the road-side laser radar are determined; including:

[0040] When the final number of matching pairs is N, the N first external parameters obtained by matching the second 3D target frame and the first 3D target frame are: L,

[0041] The N first external parameters are used as the initial positions of the N external parameter particles in the first iteration, and an iterative search is performed within the limited range of the initial positions. The position of the nth external parameter particle after the d+1th iteration is for:

[0042]

[0043]

[0044] Where d is the number of iterations, w is the inertial movement factor of the external reference particle, 1≤n≤N; represents the moving speed of the nth external reference particle after the dth iteration; represents the position of the nth external reference particle after the dth iteration; c1 represents the individual trust of the external reference particle, and c2 represents the group trust of the external reference particle. gbest represents the best position passed by the nth external reference particle after the dth iteration; d represents the best position passed by N external reference particles after the dth iteration;

[0045] When the preset total number of iterations is reached, gbest d Serves as the second external reference for vehicle-side and road-side lidar.

[0046] In a second aspect, an embodiment of the present application provides a vehicle-side and road-side laser radar extrinsic parameter calibration device, comprising:

[0047] An acquisition unit, used to acquire first point cloud data collected by the vehicle-side laser radar and second point cloud data collected by the road-side laser radar in the target area;

[0048] A detection unit, configured to process the first point cloud data using a target detection model to obtain a first three-dimensional target frame set; and process the second point cloud data using a target detection model to obtain a second three-dimensional target frame set;

[0049] A first determining unit, configured to determine a preset number of first three-dimensional object frames from the first three-dimensional object frame set and a preset number of second three-dimensional object frames from the second three-dimensional object frame set based on confidence;

[0050] A construction unit, configured to construct an affinity matrix based on similarities between a preset number of first three-dimensional target frames and a preset number of second three-dimensional target frames;

[0051] A second determining unit, configured to determine, based on the affinity matrix, a plurality of matching target frame pairs from a preset number of first three-dimensional target frames and a preset number of second three-dimensional target frames;

[0052] The third determination unit is used to determine the first external parameters of the vehicle-side and road-side laser radars based on multiple matching target frame pairs; and determine the second external parameters of the vehicle-side and road-side laser radars within the range limited by the first external parameters of the vehicle-side and road-side laser radars.

[0053] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method of the embodiment of the present application when executing the computer program.

[0054] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, and the computer program implements the method of the embodiment of the present application when executed by a processor.

[0055] This application improves the accuracy of external parameter calibration of vehicle-side and road-side lidar. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the specific implementation methods of the present application or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0057] Figure 1 A flow chart of the vehicle-side and road-side laser radar extrinsic calibration method provided in the embodiment of the present application;

[0058] Figure 2 Functional structure diagram of the vehicle-side and road-side laser radar extrinsic calibration device provided in the embodiment of the present application;

[0059] Figure 3 A functional structure diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0061] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0062] First, the design concept of the embodiments of the present application is briefly introduced.

[0063] Due to the different sampling methods and frequencies of vehicle-road radars, there are large differences in the vehicle-road endpoint cloud data. The point clouds obtained by different types of lidar are called heterogeneous point clouds. Therefore, the heterogeneous point cloud registration in the vehicle-road multi-sensor fusion problem is more complicated than the homologous point cloud registration. At the same time, the existing vehicle-road multi-sensor fusion registration method basically relies on the perception results to assist the registration of the vehicle-road endpoint cloud, and lacks the mining of the perception results themselves, resulting in a decrease in the registration effect.

[0064] In order to solve the above technical problems, the present application provides a method for calibrating external parameters of vehicle-side and road-side lidars. The method integrates the target detection results of the vehicle-side and road-side, and replaces the registration of heterogeneous point clouds on the vehicle-road side by the registration of the perception detection frame. It can automatically complete the vehicle-road side lidar point cloud registration task in real time, becoming an important basis for vehicle-road collaborative perception and vehicle positioning tasks.

[0065] The registration task in vehicle-road multi-sensor fusion results in a decrease in calibration effect due to the combined effects of noise and outliers, density differences, partial overlaps, and scale differences in the vehicle-road heterogeneous point cloud. The method of the present application improves the registration speed and reduces the impact of heterogeneous point cloud noise on the registration results.

[0066] After introducing the application scenarios and design concepts of the embodiments of the present application, the technical solutions provided by the embodiments of the present application are described below.

[0067] like Figure 1 As shown, the present application implements a method for calibrating external parameters of a vehicle-side and road-side laser radar, including steps 101 to 106:

[0068] Step 101: Acquire first point cloud data collected by a vehicle-side laser radar and second point cloud data collected by a road-side laser radar in a target area;

[0069] Step 102: Processing the first point cloud data using a target detection model to obtain a first three-dimensional target frame set; processing the second point cloud data using a target detection model to obtain a second three-dimensional target frame set;

[0070] Each 3D target box contains coordinate information and size information.

[0071] Step 103: determining a preset number of first 3D object frames from the first 3D object frame set and a preset number of second 3D object frames from the second 3D object frame set based on the confidence level;

[0072] Among them, by filtering the three-dimensional target frame by confidence, a higher quality three-dimensional target frame can be obtained.

[0073] Step 104: constructing an affinity matrix based on the similarities between a preset number of first three-dimensional target frames and a preset number of second three-dimensional target frames;

[0074] Step 105: Based on the affinity matrix, determine a plurality of matching target frame pairs from a preset number of first three-dimensional target frames and a preset number of second three-dimensional target frames;

[0075] Step 106: Based on multiple matching target frame pairs, determine the first external parameters of the vehicle-side and road-side laser radars; within the range defined by the first external parameters of the vehicle-side and road-side laser radars, determine the second external parameters of the vehicle-side and road-side laser radars.

[0076] In view of the shortcomings of the existing heterogeneous point cloud registration, this application proposes a registration method for the perception detection frame based on the heterogeneous point cloud, which improves the registration accuracy of the vehicle-road side lidar.

[0077] In some embodiments, based on the confidence level, a preset number of first three-dimensional target frames are determined from the first three-dimensional target frame set, and a preset number of second three-dimensional target frames are determined from the second three-dimensional target frame set; including:

[0078] Arrange the first three-dimensional object frames in the first three-dimensional object frame set in descending order according to the confidence level, and use the first preset number of first three-dimensional object frames in the first three-dimensional object frame sequence as the preset number of first three-dimensional object frames;

[0079] The second 3D object frames in the second 3D object frame set are arranged in descending order according to confidence, and the first preset number of second 3D object frames in the second 3D object frame sequence are used as the preset number of second 3D object frames.

[0080] In some embodiments, based on a preset number of first three-dimensional target frames and a preset number of second three-dimensional target frames, the three-dimensional coordinates of eight corner points of each first three-dimensional target frame are obtained to calculate the three-dimensional coordinates of the center point of the first three-dimensional target frame; the three-dimensional coordinates of the eight corner points of the first three-dimensional target frame are subtracted from the three-dimensional coordinates of the corresponding center point to obtain the decentralized three-dimensional coordinates of the eight corner points of the first three-dimensional target frame;

[0081] Obtain the three-dimensional coordinates of the eight corner points of each second three-dimensional object frame, and calculate the three-dimensional coordinates of the center point of the second three-dimensional object frame; subtract the three-dimensional coordinates of the corresponding center point from the three-dimensional coordinates of the eight corner points of the second three-dimensional object frame to obtain the decentralized three-dimensional coordinates of the eight corner points of the second three-dimensional object frame;

[0082] For the i-th first 3D target frame and the j-th second 3D target frame, calculate the association matrix H ij :

[0083] H ij =X i Y j T

[0084] Where, 1≤i≤K, 1≤j≤K; K is the preset number; X i Y is a 3×8 matrix consisting of the decentralized 3D coordinates of the 8 corner points of the i-th first 3D target box; jA 3×8 matrix consisting of the decentralized 3D coordinates of the 8 corner points of the j-th second 3D target box;

[0085] For the matrix H ij Perform SVD decomposition:

[0086] H ij =U ij ΛV ij T

[0087] Among them, U ij and V ij All are unitary matrices;

[0088] The i-th first 3D target frame and the j-th second 3D target frame are taken as a matching pair. At this time, a rigid transformation T = (R, t) needs to be found to align the two sets of points, that is, to solve the external parameter values ​​of the two 3D target frames, where R is the rotation matrix and t is the translation vector; that is, solve:

[0089]

[0090] The calculated rotation matrix R between the vehicle-side laser radar body coordinate system and the road-side laser radar body coordinate system ij for:

[0091] R ij =V ij U ij T

[0092] Then the translation vector t between the vehicle-side laser radar body coordinate system and the road-side laser radar body coordinate system is ij for:

[0093]

[0094] in, is the 3D coordinate of the center point of the i-th first 3D target frame, is the three-dimensional coordinate of the center point of the j-th second three-dimensional target frame;

[0095] Thus, we get K×K groups of rotation matrices and translation vectors;

[0096] The rotation matrix R between the vehicle-side laser radar body coordinate system and the road-side laser radar body coordinate system is used ij and the translation vector t ij , transform the K first three-dimensional target frames into the road-side laser radar body coordinate system to obtain K third three-dimensional target frames; calculate the intersection and union ratio F of the i-th second three-dimensional target frame and the j-th third three-dimensional target frame ij ;

[0097] The intersection and union of the K×K i-th second 3D target frame and the j-th third 3D target frame is calculated as F ij The sum of is taken as the similarity between the second 3D target frame and the third 3D target frame:

[0098]

[0099] Based on all similarities A ij Construct affinity matrix A.

[0100] In some embodiments, based on the affinity matrix, a plurality of matching target frame pairs are determined from a preset number of first three-dimensional target frames and a preset number of second three-dimensional target frames; including:

[0101] Construct the objective function f(W):

[0102]

[0103] Among them, the matrix W is the decision variable w ij The matrix composed of ij The value of is 0 or 1; ij = 1 means that the i-th second 3D target frame and the j-th third 3D target frame are matched, w ij = 0 means that the i-th second 3D target frame and the j-th third 3D target frame do not match, and there is

[0104] The Hungarian algorithm is used to solve the maximum value of f(W) and obtain K pairs of matching second 3D target frames and third 3D target frames;

[0105] The matching scores of K pairs of matching second 3D target frames and third 3D target frames are obtained from the affinity matrix A, and the matching pairs with matching scores greater than a preset threshold are taken as the final matching pairs.

[0106] In some embodiments, determining second external parameters of the vehicle-side laser radar and the road-side laser radar within a range defined by the first external parameters of the vehicle-side laser radar and the road-side laser radar comprises:

[0107] When the final number of matching pairs is N, the N first external parameters obtained by matching the second 3D target frame and the first 3D target frame are: L,

[0108] The N first external parameters are used as the initial positions of the N external parameter particles in the first iteration, and an iterative search is performed within the limited range of the initial positions. The position of the nth external parameter particle after the d+1th iteration is for:

[0109]

[0110] Where d is the number of iterations, w is the inertial movement factor of the external reference particle, 1≤n≤N; represents the moving speed of the nth external reference particle after the dth iteration; represents the position of the nth external reference particle after the dth iteration; c1 represents the individual trust of the external reference particle, and c2 represents the group trust of the external reference particle. gbest represents the best position passed by the nth external reference particle after the dth iteration; d represents the best position passed by N external reference particles after the dth iteration;

[0111] Based on the same inventive concept, the present application embodiment provides a vehicle-side and road-side laser radar extrinsic calibration device, see Figure 2 As shown, the vehicle-side and road-side laser radar extrinsic parameter calibration device 200 provided in the embodiment of the present application at least includes:

[0112] An acquisition unit 201 is used to acquire first point cloud data collected by a vehicle-side laser radar and second point cloud data collected by a road-side laser radar in a target area;

[0113] A detection unit 202 is used to process the first point cloud data using a target detection model to obtain a first three-dimensional target frame set; and to process the second point cloud data using a target detection model to obtain a second three-dimensional target frame set;

[0114] A first determining unit 203 is used to determine a preset number of first 3D object frames from the first 3D object frame set and a preset number of second 3D object frames from the second 3D object frame set based on confidence;

[0115] A construction unit 204 is used to construct an affinity matrix based on similarities between a preset number of first three-dimensional target frames and a preset number of second three-dimensional target frames;

[0116] A second determining unit 205 is used to determine a plurality of matching target frame pairs from a preset number of first three-dimensional target frames and a preset number of second three-dimensional target frames based on the affinity matrix;

[0117] The third determination unit 206 is used to determine the first external parameters of the vehicle-side and road-side laser radars based on multiple matching target frame pairs; and determine the second external parameters of the vehicle-side and road-side laser radars within the range defined by the first external parameters of the vehicle-side and road-side laser radars.

[0118] It should be noted that the principle of solving the technical problem by the vehicle-end and road-end LiDAR external parameter calibration device 200 provided in the embodiment of the present application is similar to the method provided in the embodiment of the present application. Therefore, the implementation of the vehicle-end and road-end LiDAR external parameter calibration device 200 provided in the embodiment of the present application can refer to the implementation of the method provided in the embodiment of the present application, and the repeated parts will not be repeated.

[0119] Based on the same inventive concept, the present application embodiment also provides an electronic device, referring to Figure 3 As shown, the electronic device 300 provided in the embodiment of the present application includes at least: a processor 301, a memory 302, and a computer program stored in the memory 302 and executable on the processor 301. When the processor 301 executes the computer program, the vehicle-side and road-side lidar external parameter calibration method provided in the embodiment of the present application is implemented.

[0120] The electronic device 300 provided in the embodiment of the present application may further include a bus 303 connecting different components (including the processor 301 and the memory 302). The bus 303 represents one or more of several types of bus structures, including a memory bus, a peripheral bus, a local bus, and the like.

[0121] The memory 302 may include a readable medium in the form of a volatile memory, such as a random access memory (RAM) 3021 and / or a cache memory 3022 , and may further include a read-only memory (ROM) 3023 .

[0122] The memory 302 may also include a program tool 3025 having a set (at least one) of program modules 3024, the program modules 3024 including but not limited to: an operating subsystem, one or more applications, other program modules and program data, each of which or some combination may include the implementation of a network environment.

[0123] The electronic device 300 may also communicate with one or more external devices 304 (e.g., keyboards, remote controls, etc.), and may also communicate with one or more devices that enable a user to interact with the electronic device 300 (e.g., mobile phones, computers, etc.), and / or communicate with any device that enables the electronic device 300 to communicate with one or more other electronic devices 300 (e.g., routers, modems, etc.). Such communication may be performed through an input / output (I / O) interface 305. Furthermore, the electronic device 300 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter 306. Figure 3 As shown, the network adapter 306 communicates with other modules of the electronic device 300 via the bus 303. It should be understood that although Figure 3 Not shown, other hardware and / or software modules may be used in conjunction with the electronic device 300, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, disk arrays (Redundant Arrays of Independent Disks, RAID) subsystems, tape drives, and data backup storage subsystems.

[0124] It should be noted that Figure 3 The electronic device 300 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0125] The embodiment of the present application also provides a computer-readable storage medium, which stores computer instructions, and when the computer instructions are executed by the processor, the vehicle-side and road-side laser radar extrinsic calibration method provided in the embodiment of the present application is implemented. Specifically, the executable program can be built-in or installed in the electronic device 300, so that the electronic device 300 can implement the vehicle-side and road-side laser radar extrinsic calibration method provided in the embodiment of the present application by executing the built-in or installed executable program.

[0126] The vehicle-side and road-side lidar external parameter calibration method provided in the embodiment of the present application can also be implemented as a program product, which includes a program code. When the program product can be run on an electronic device 300, the program code is used to enable the electronic device 300 to execute the vehicle-side and road-side lidar external parameter calibration method provided in the embodiment of the present application.

[0127] The program product provided in the embodiments of the present application may adopt any combination of one or more readable media, wherein the readable medium may be a readable signal medium or a readable storage medium, and the readable storage medium may be but is not limited to an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any combination of the above. Specifically, more specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, RAM, ROM, Erasable Programmable Read Only Memory (EPROM), optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0128] The program product provided in the embodiment of the present application may adopt a CD-ROM and include program code, and may also be run on a computing device. However, the program product provided in the embodiment of the present application is not limited thereto. In the embodiment of the present application, the readable storage medium may be any tangible medium containing or storing a program, and the program may be used by or in combination with an instruction execution system, apparatus, or device.

[0129] It should be noted that, although several units or subunits of the device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more units described above can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided into multiple units to be embodied.

[0130] In addition, although the operations of the method of the present application are described in a specific order in the drawings, this does not require or imply that the operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0131] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present application and are not intended to limit it. Although the present application is described in detail with reference to the embodiments, a person skilled in the art should understand that any modification or equivalent replacement of the technical solution of the present application does not depart from the spirit and scope of the technical solution of the present application and should be included in the scope of the claims of the present application.

Claims

1. A method for calibrating external parameters of vehicle-side and road-side laser radars, characterized in that: include: Acquire first point cloud data collected by the vehicle-side laser radar and second point cloud data collected by the road-side laser radar in the target area; The first point cloud data is processed using a target detection model to obtain a first three-dimensional target frame set; the second point cloud data is processed using a target detection model to obtain a second three-dimensional target frame set; Determining a preset number of first three-dimensional object frames from the first three-dimensional object frame set and a preset number of second three-dimensional object frames from the second three-dimensional object frame set based on the confidence; constructing an affinity matrix based on similarities between a preset number of first three-dimensional target frames and a preset number of second three-dimensional target frames; Based on the affinity matrix, determining a plurality of matching target frame pairs from a preset number of first three-dimensional target frames and a preset number of second three-dimensional target frames; Based on multiple matching target frame pairs, the first external parameters of the vehicle-side and road-side laser radars are determined; within the range limited by the first external parameters of the vehicle-side and road-side laser radars, the second external parameters of the vehicle-side and road-side laser radars are determined.

2. The method according to claim 1, characterized in that Determining a preset number of first three-dimensional object frames from the first three-dimensional object frame set and determining a preset number of second three-dimensional object frames from the second three-dimensional object frame set based on confidence respectively; comprising: Arrange the first three-dimensional object frames in the first three-dimensional object frame set in descending order according to confidence, and use the first preset number of first three-dimensional object frames in the first three-dimensional object frame sequence as the preset number of first three-dimensional object frames; The second 3D object frames in the second 3D object frame set are arranged in descending order according to confidence, and the first preset number of second 3D object frames in the second 3D object frame sequence are used as the preset number of second 3D object frames.

3. The method according to claim 1, characterized in that: Based on the similarity between a preset number of first three-dimensional target frames and a preset number of second three-dimensional target frames, an affinity matrix is ​​constructed, including: Obtaining the three-dimensional coordinates of the eight corner points of each first three-dimensional object frame, and calculating the three-dimensional coordinates of the center point of the first three-dimensional object frame; subtracting the three-dimensional coordinates of the corresponding center point from the three-dimensional coordinates of the eight corner points of the first three-dimensional object frame to obtain decentralized three-dimensional coordinates of the eight corner points of the first three-dimensional object frame; Obtain the three-dimensional coordinates of the eight corner points of each second three-dimensional object frame, and calculate the three-dimensional coordinates of the center point of the second three-dimensional object frame; subtract the three-dimensional coordinates of the corresponding center point from the three-dimensional coordinates of the eight corner points of the second three-dimensional object frame to obtain the decentralized three-dimensional coordinates of the eight corner points of the second three-dimensional object frame; For the i-th first 3D target frame and the j-th second 3D target frame, calculate the association matrix H ij : H ij =X i Y j T Where, 1≤i≤K, 1≤j≤K; K is the preset number; X i Y is a 3×8 matrix consisting of the decentralized 3D coordinates of the 8 corner points of the i-th first 3D target box; j A 3×8 matrix consisting of the decentralized 3D coordinates of the 8 corner points of the j-th second 3D target box; For the matrix H ij Perform SVD decomposition: H ij =U ij ΛV ij T Among them, U ij and V ij All are unitary matrices; The i-th first 3D target frame and the j-th second 3D target frame are taken as a matching pair, and the rotation matrix R between the vehicle-side laser radar body coordinate system and the road-side laser radar body coordinate system is calculated. ij for: R ij =V ij U ij T Then the translation vector t between the vehicle-side laser radar body coordinate system and the road-side laser radar body coordinate system is ij for: in, is the 3D coordinate of the center point of the i-th first 3D target frame, is the three-dimensional coordinate of the center point of the j-th second three-dimensional target frame; Thus, we get K×K groups of rotation matrices and translation vectors; The rotation matrix R between the vehicle-side laser radar body coordinate system and the road-side laser radar body coordinate system is used ij and the translation vector t ij , transform the K first three-dimensional target frames into the road-side laser radar body coordinate system to obtain K third three-dimensional target frames; calculate the intersection and union ratio F of the i-th second three-dimensional target frame and the j-th third three-dimensional target frame ij ; The intersection and union of the K×K i-th second 3D target frame and the j-th third 3D target frame is calculated as F ij The sum of is taken as the similarity between the second 3D target frame and the third 3D target frame: Based on all similarities A ij Construct affinity matrix A.

4. The method according to claim 3, characterized in that Based on the affinity matrix, determining a plurality of matching target frame pairs from a preset number of first three-dimensional target frames and a preset number of second three-dimensional target frames; comprising: Construct the objective function f(W): Among them, the matrix W is the decision variable w ij The matrix composed of ij The value of is 0 or 1; ij = 1 means that the i-th second 3D target frame and the j-th third 3D target frame are matched, w ij = 0 means that the i-th second 3D target frame and the j-th third 3D target frame do not match, and there is The Hungarian algorithm is used to solve the maximum value of f(W) and obtain K pairs of matching second 3D target frames and third 3D target frames; The matching scores of K pairs of matching second 3D target frames and third 3D target frames are obtained from the affinity matrix A, and the matching pairs with matching scores greater than a preset threshold are taken as the final matching pairs.

5. The method according to claim 4, characterized in that Determining the second external parameters of the vehicle-side and road-side laser radars within the range defined by the first external parameters of the vehicle-side and road-side laser radars; including: When the final number of matching pairs is N, the N first external parameters obtained by matching the second 3D target frame and the first 3D target frame are: The N first external parameters are used as the initial positions of the N external parameter particles in the first iteration, and an iterative search is performed within the limited range of the initial positions. The position of the nth external parameter particle after the d+1th iteration is for: Where d is the number of iterations, w is the inertial movement factor of the external reference particle, 1≤n≤N; represents the moving speed of the nth external reference particle after the dth iteration; represents the position of the nth external reference particle after the dth iteration; c1 represents the individual trust of the external reference particle, and c2 represents the group trust of the external reference particle. gbest represents the best position passed by the nth external reference particle after the dth iteration; d represents the best position passed by N external reference particles after the dth iteration; When the preset total number of iterations is reached, gbest d Serves as the second external reference for vehicle-side and road-side lidar.

6. A vehicle-side and road-side laser radar external parameter calibration device, characterized in that: include: An acquisition unit, used to acquire first point cloud data collected by the vehicle-side laser radar and second point cloud data collected by the road-side laser radar in the target area; A detection unit, configured to process the first point cloud data using a target detection model to obtain a first three-dimensional target frame set; and process the second point cloud data using a target detection model to obtain a second three-dimensional target frame set; A first determining unit, configured to determine a preset number of first three-dimensional object frames from the first three-dimensional object frame set and a preset number of second three-dimensional object frames from the second three-dimensional object frame set based on confidence; A construction unit, configured to construct an affinity matrix based on similarities between a preset number of first three-dimensional target frames and a preset number of second three-dimensional target frames; A second determining unit, configured to determine, based on the affinity matrix, a plurality of matching target frame pairs from a preset number of first three-dimensional target frames and a preset number of second three-dimensional target frames; The third determination unit is used to determine the first external parameters of the vehicle-side and road-side laser radars based on multiple matching target frame pairs; and determine the second external parameters of the vehicle-side and road-side laser radars within the range limited by the first external parameters of the vehicle-side and road-side laser radars.

7. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 1 to 5 when executing the computer program.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Citation Information

Patent Citations

  • Calibration method and calibration device for multiple groups of laser radar external parameters and computer storage medium

    CN114080547A

  • Roadside laser radar calibration method and device and electronic equipment

    CN115480235A

  • Laser radar and camera external parameter calibration method for automatic driving system

    CN116299359A

  • Vehicle detection method based on point cloud data

    CN116883988A

  • Laser radar external parameter calibration method and device based on point cloud registration and storage medium

    CN117274400A