Construction of multi-sensor online external parameter calibration device and calibration method

Abstract

The application discloses a kind of multi-sensor online external parameter calibration device construction and calibration method, comprising the following steps: a plurality of internal surface is coated with different nature coating corner reflector is arranged combination, constructs calibration device;Based on the calibration device, the internal vertex point set of corner reflector is obtained using multi-sensor respectively;The internal vertex point set of corner reflector obtained by each sensor is matched two by two, and the external parameter parameter between multi-sensor two by two is obtained;The external parameter parameter is screened, and the optimal external parameter parameter is obtained, and external parameter calibration is realized based on the optimal external parameter parameter.The application can provide the feature point needed in the external parameter calibration of four common sensors of laser radar, millimeter wave radar, camera, infrared camera, contour feature with high precision, can satisfy two by two registration between four types of sensors, and realize external parameter calibration.

Description

Technical Field

[0001] This invention belongs to the technical field of external parameter calibration for multiple sensors, and in particular relates to the construction and calibration method of an online external parameter calibration device for multiple sensors. Background Technology

[0002] The first existing calibration method is a checkerboard-based calibration of the extrinsic parameters of LiDAR and cameras. This method typically uses a checkerboard calibration board made of a non-deformable material as the base, on which a flat, alternating black and white checkerboard pattern is laser-printed. This type of calibration board is mainly used for high-precision calibration of visible light and multi-beam LiDAR. The calibration method involves using lasers to extract the intersections between multiple black and white squares, while visible light cameras also extract corner points by distinguishing the color blocks. The extrinsic parameters are solved by calculating the matching LiDAR-camera corner point pair PnP equation. However, this method is only suitable for extrinsic parameter calibration between high-resolution cameras and multi-beam LiDAR. For low-beam LiDAR, the sparse reflection of laser points prevents the formation of a dense reflective surface, leading to decreased accuracy. Furthermore, the checkerboard pattern can result in unclear edges and distortion due to lighting conditions. This method also cannot meet the calibration requirements of mobile platforms equipped with multiple sensors.

[0003] The second method is a camera and millimeter-wave radar extrinsic parameter calibration method based on corner reflectors. This calibration method typically uses corner reflectors made of materials sensitive to millimeter-wave radar, such as metal or alloys, and employs three isosceles right-angled triangular metal plates joined together to form a triangular pyramid with three mutually perpendicular faces. The reflection principle of this device is based on the reflection of millimeter waves along the incident angle on the reflector. The position information of the central corner point of the corner reflector is extracted, and then compared with the reflector corner point identified by the camera to form a 3D-2D point pair. Multiple point pairs are used to solve the PnP equation, thereby obtaining the extrinsic parameters. This method is suitable for the extrinsic parameter calibration of cameras and millimeter-wave radars, but requires knowledge of the camera's intrinsic parameters. Conventional corner reflectors cannot meet the calibration requirements of lidar and millimeter-wave radars, and cannot be used for the extrinsic parameter calibration of mobile platforms equipped with multiple sensors.

[0004] The third approach is to find point features, line features, and surface features directly from the environment or road without using manually designed calibration objects, and then perform feature matching to obtain extrinsic parameters. However, because environmental information cannot remain absolutely static, this method cannot continuously and stably obtain the features of the same obstacle, resulting in an unstable solution for extrinsic parameters and a relatively large error. It is currently not suitable for application in real-world scenarios.

[0005] The existing technologies have the following technical problems: (1) Multi-sensor calibration requires various dedicated calibration devices, which cannot be solved uniformly. (2) Due to prolonged use, road bumps, or unstable installation, the external parameters of the sensors on various platforms change, requiring recalibration. (3) The calibration process is complex and requires repeated calibration by professionals, which is time-consuming and laborious for product users. (4) Currently, the features used for real-time calibration of sensors without calibration devices, such as line features and obstacle contour features, are unstable, and the parameters calibrated in real time have large errors, which cannot meet the requirements of multi-sensor fusion on daily platforms. (5) During the multi-sensor fusion process of road vehicles, the points fed back by millimeter-wave radar cannot be determined in terms of specific location, resulting in the inability to obtain a definite distance, thereby reducing the millimeter-wave fusion effect.

[0006] In summary, the current online calibration of multiple sensors is limited by the lack of a calibration device that continuously and stably provides high-precision feature points, thus making it impossible to detect a stable and deterministic set of feature points for extrinsic parameter calibration. Summary of the Invention

[0007] The purpose of this invention is to provide a construction and calibration method for a multi-sensor online extrinsic parameter calibration device, applicable to low-beam and high-beam lidar, visible light cameras, infrared cameras and millimeter-wave cameras in various situations, especially during movement or driving, for online extrinsic parameter calibration and online extrinsic parameter monitoring, in order to solve the problems existing in the prior art.

[0008] To achieve the above objectives, the present invention provides a construction and calibration method for a multi-sensor online extrinsic parameter calibration device, comprising the following steps:

[0009] A calibration device is constructed by arranging and combining several corner reflectors with different coatings on their inner surfaces.

[0010] Based on the calibration device, multiple sensors are used to acquire the set of vertex points inside the corner reflector.

[0011] The internal vertex point sets of the corner reflectors acquired by each sensor are matched pairwise to obtain the extrinsic parameters between each pair of sensors.

[0012] The extrinsic parameters are filtered to obtain the optimal extrinsic parameters, and the extrinsic parameter calibration is performed based on the optimal extrinsic parameters.

[0013] Optionally, the intersecting surfaces inside the corner reflector are coated with coatings of different colors, different heat absorption rates, or different laser reflectivities.

[0014] Optionally, the process of acquiring the vertex set inside the corner reflector using multiple sensors includes: simultaneously acquiring the vertex set inside the corner reflector in the form of point cloud using lidar, point cloud using millimeter-wave radar, image using a visible light camera, and thermal radiation imaging using an infrared camera, and recording the timestamp.

[0015] Optionally, the vertex set inside the corner reflector obtained by the lidar and millimeter-wave radar is a three-dimensional point cloud set, and the vertex set inside the corner reflector obtained by the visible light camera and infrared camera is a two-dimensional point cloud set.

[0016] Optionally, the process of obtaining the corresponding vertex set inside the corner reflector in the form of a point cloud based on the lidar includes: collecting point clouds based on the lidar, then performing point cloud intensity range filtering based on the laser reflectivity of the calibration device to obtain the target intensity range; within the target intensity range, using a clustering algorithm or a sample consistency algorithm to filter out stray point clouds, and obtaining the calibration device point cloud based on the color and surface features of the calibration device; based on the calibration device point cloud, performing surface reconstruction using the size information of the corresponding corner reflector and the connecting plane, thereby obtaining the vertex set inside the corner reflector corresponding to the lidar.

[0017] Optionally, the process of obtaining the corresponding vertex set inside the corner reflector in the form of a point cloud based on millimeter-wave radar includes: collecting point clouds based on millimeter-wave radar, taking the point clouds with reflection intensity higher than a preset reflection intensity as the initial three-dimensional point set of the corresponding corner reflector; filtering out the initial three-dimensional point set through the arrangement and spacing information of the calibration device, thereby obtaining the vertex set inside the corner reflector corresponding to the millimeter-wave radar.

[0018] Optionally, the process of acquiring the corresponding vertex set inside the corner reflector in the form of an image based on a visible light camera includes: acquiring image information based on the visible light camera, performing HSV space transformation to obtain the HSV representation of the image, and then performing HSV range filtering based on the color features of the calibration device to obtain the target color gamut of the image; in the target color gamut of the image, filtering out the color blocks of the non-corner reflector based on the internal color splicing features of the calibration device, and then obtaining the corresponding corner reflector line features based on the LSD isolinear feature extraction algorithm, and then obtaining the vertex set inside the corner reflector corresponding to the visible light camera based on the size information of the corresponding corner reflector; or acquiring the vertex dataset inside the corner reflector of the calibration device, and then obtaining the vertex set inside the corner reflector corresponding to the visible light camera based on a neural network.

[0019] Optionally, the process of acquiring the corresponding vertex point set inside the corner reflector in the form of thermal radiation imaging based on the infrared camera includes: acquiring image information based on the infrared camera, performing pixel filtering on the image based on the heat absorption rate of different internal surfaces of the calibration device, then obtaining the corresponding corner reflector edge line features based on the LSD isolinear feature extraction algorithm, and then obtaining the vertex point set inside the corner reflector corresponding to the infrared camera based on the size information of the corresponding corner reflector.

[0020] Optionally, the process of pairwise matching of the vertex point sets inside the corner reflectors corresponding to each sensor includes: obtaining extrinsic parameters between the three-dimensional point cloud point sets based on an iterative nearest point registration algorithm; obtaining extrinsic parameters between any three-dimensional point cloud point set and any two-dimensional point cloud point set based on a perspective n-point positioning method; and obtaining extrinsic parameters between the two-dimensional point cloud point sets based on an image matching algorithm.

[0021] Optionally, the process of filtering the extrinsic parameters includes: filtering the extrinsic parameters based on the correspondence between the vertex point sets inside the corner reflector obtained by each sensor, and obtaining the optimal extrinsic parameters.

[0022] The technical effects of this invention are as follows:

[0023] This invention can provide the feature points and contour features required for the external parameter calibration of four common types of sensors: lidar, millimeter-wave radar, camera, and infrared camera. It can also meet the pairwise registration requirements of the four types of sensors to achieve external parameter calibration.

[0024] This invention enables real-time calibration during movement without the need for additional reference objects or the perception of other scene features.

[0025] The calibration device in this invention can be used to monitor external parameters during movement, avoiding external parameter errors caused by prolonged use of the sensor or slippage of the connector. Attached Figure Description

[0026] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0027] Figure 1 This is a front view of the calibration device for two corner reflectors and a common license plate combination in an embodiment of the present invention;

[0028] Figure 2 This is a schematic diagram of a single side of the corner reflector in an embodiment of the present invention;

[0029] Figure 3 This is a top view of the calibration device for two corner reflectors and a common license plate combination in an embodiment of the present invention;

[0030] Figure 4 This is a front view of the calibration device formed by embedding three corner reflectors into a triangular plane in an embodiment of the present invention. Detailed Implementation

[0031] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0032] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0033] Example 1

[0034] This embodiment provides the construction and calibration method of a multi-sensor online extrinsic parameter calibration device, applicable to dynamic calibration during autonomous vehicle following. Figures 1-3 As shown, the main components are two coated corner reflectors connected to the license plate and embedded in the rear of the vehicle. This device, when applied to automobiles, can increase the perception accuracy of vehicles during autonomous driving, improve the effect of multi-sensor fusion, and reduce the target recognition error rate.

[0035] Step 1: During the autonomous vehicle's operation, upon encountering a vehicle equipped with a calibration device, the external parameter supervised calibration mode is automatically or manually activated. This calibration device consists of two corner reflectors and a common license plate. The dimensions of the corner reflectors and the license plate serve as prior knowledge for identification. The license plate also provides color and shape features, which are beneficial for the camera and LiDAR in recognizing the calibrated target.

[0036] Furthermore, the calibration device is based on conventional millimeter-wave radar calibration metal corner reflectors. The three intersecting surfaces inside are coated with coatings of different properties, such as color, heat absorption rate, and laser reflectivity. For example, a black coating can absorb heat, has low reflectivity, and a bright color. The coating process is not limited, as long as it is smooth. Multiple of these corner reflectors are used in a measurable arrangement based on a vertical plane. The corner reflectors must not overlap, the maximum size of the vertical plane must not exceed 3m, and the base dimension of the corner reflectors must not be less than 10cm. This vertical plane can provide adequate feature information, such as surface features and color features.

[0037] Step 2: The autonomous vehicle's onboard sensors, which operate in a synchronized manner, collect environmental data. The LiDAR captures this data in point cloud format. Location: X L Y L ZL Reflection intensity: I L The millimeter-wave radar is located in point cloud form, at position X. R Y R Z R RCS strength: I R Visible light camera image format, camera coordinate system position: X C Y C Z C Pixel position in image coordinate system: u C v C The infrared camera uses thermal radiation imaging. Its position in the infrared camera coordinate system is: X. I Y I Z I Pixel position in the infrared camera image coordinate system: u I v I Perform, record timestamp T i , i = 0, 1, ... . The position here is a general reference, not specific to the position of each point or each pixel.

[0038] Step 3: Obtain the same timestamp T via soft synchronization or hard triggering. n The set of internal vertices of the corner reflectors under each sensor:

[0039] Millimeter-wave radar: This refers to the radar output intensity I of point cloud targets acquired by millimeter-wave radar. L Point cloud values ​​higher than the predetermined value are selected as the initial 3D point set P for the corner reflector locations acquired by the millimeter-wave radar. R1 Furthermore, by utilizing prior knowledge of the arrangement and spacing established during production, point sets that do not conform to the prior knowledge are filtered out, thus obtaining the vertex point set P inside the millimeter-wave radar corner reflector. R .

[0040] LiDAR: The point cloud target acquired by the LiDAR is filtered according to the reflectivity set during manufacturing, retaining only the target intensity range. This range can be obtained through multiple tests based on the coating of the actual calibration device. Alternatively, stray point clouds can be filtered out using clustering or sample-consistency algorithms. The point cloud representing the calibration device can be obtained using its color and surface features. Furthermore, using prior information about the corner reflectors and connecting plane dimensions, surface reconstruction is performed to obtain the vertex set P inside the corner reflectors. L .

[0041] Visible light camera: The image information acquired by the visible light camera is transformed into HSV space to obtain the HSV representation of the image. HSV range filtering is then performed using known color prior features from the calibration device to retain only the target color gamut. Optionally, other color blocks outside the corner reflector are filtered out using the color gamut stitching features of the three internal surfaces of the corner reflector. Line features of the corner reflector are obtained using the LSD isolinear feature extraction algorithm, and then the pixel set P representing the vertices inside the corner reflector is obtained based on prior information about the corner reflector size. C Alternatively, by creating a dataset of vertices inside the corner reflector of the target calibration device, a neural network can be trained to obtain the pixels representing the vertices inside the corner reflector, thereby obtaining the image target point set P. C .

[0042] Infrared camera: Image information acquired by the infrared camera is filtered out according to a set grayscale threshold. Due to the different heat absorption rates of the internal surfaces of the corner reflector, continuous corner reflector images with different and uniform grayscale are formed. This feature can be used to locate the position of the calibration device, and other irrelevant pixels are filtered out based on prior distance information. Furthermore, the edge line features of the corner reflector are obtained through the LSD isolinear feature extraction algorithm, and then the pixel set P representing the internal vertices of the corner reflector is obtained based on the prior information of the corner reflector size. I .

[0043] Step 4: Based on the timestamp, repeatedly detect the vertex set inside the corner reflector of each sensor in Step 3, save it as an array, and determine the corresponding point pairs between sensors based on the prior position information between the corner reflectors.

[0044] Step 5: After the above steps, the LiDAR, millimeter-wave radar, visible light camera, and infrared camera have obtained their respective vertex point sets inside the corner reflectors. The LiDAR and millimeter-wave radar have 3D point cloud point sets, while the visible light camera and infrared camera have 2D pixel point sets. The calculation of extrinsic parameters between them falls into three categories: 3D point sets are calculated using iterative nearest-point registration algorithms; 3D-2D point sets are calculated using perspective n-point localization methods; and 2D point sets are calculated using image matching algorithms such as image descriptors. It is feasible to ensure that each acquired point pair has redundancy. When calculating the extrinsic parameters between sensors, random iteration can be used to select the extrinsic parameter with the smallest reprojection error. It is also feasible to obtain multiple extrinsic parameters between pairs of points. Because the points detected by each sensor have mutual correspondences, the optimal extrinsic parameter can be selected by verifying the correspondences between points.

[0045] Step 5: Calculate and monitor the error of the external parameters based on the factory parameters of the unmanned vehicle. If the error exceeds a certain threshold, replace the parameters; otherwise, keep the original factory parameters.

[0046] Example 2

[0047] A static marking device for a certain type of road, consisting of multiple corner reflectors embedded in a vertical plane in a known arrangement, can be suspended on static obstacles such as road utility poles and traffic lights, or used by dedicated unmanned vehicles or platforms to mark locations. Figure 4 In one design, three corner reflectors are arranged at intervals and embedded in a triangular plane to form a new calibration device.

[0048] The corner reflectors of static calibration devices on roads must have a base dimension greater than 20cm and an overall side length (X) not exceeding 3m. This requirement stems from the fact that static road calibration cannot be performed at close range, and the overlap range of various sensors is small. Therefore, the corner reflectors of the calibration device need to be relatively large to easily detect line features and internal corner points, while the calibration device itself cannot be too large, exceeding the overlap range. The calibration process can utilize map information to determine the location of the calibration device.

[0049] Step 1: Receive calibration device area information from the map or manually select calibration mode to start.

[0050] Step 2: The autonomous vehicle's onboard sensors, which operate in a synchronized manner, collect environmental data. The LiDAR captures this data in point cloud format. Location: X L Y L Z L Reflection intensity: I L The millimeter-wave radar is located in point cloud form, at position X. R Y R Z R RCS strength: I R Visible light camera image format, camera coordinate system position: X C Y C Z C Pixel position in image coordinate system: u C υ C The infrared camera uses thermal radiation imaging. Its position in the infrared camera coordinate system is: X. I Y I Z I Pixel position in the infrared camera image coordinate system: u I v I Perform, record timestamp T i , i = 0, 1, ...

[0051] Step 3: Obtain the same timestamp T via soft synchronization or hard triggering. n The set of internal vertices of the corner reflectors under each sensor:

[0052] Millimeter-wave radar: This refers to the radar output intensity I of point cloud targets acquired by millimeter-wave radar. LPoint cloud values ​​higher than the predetermined value are selected as the initial 3D point set P for the corner reflector locations acquired by the millimeter-wave radar. R1 Furthermore, by utilizing prior knowledge of the arrangement and spacing established during production, point sets that do not conform to the prior knowledge are filtered out, thus obtaining the vertex point set P inside the millimeter-wave radar corner reflector. R .

[0053] LiDAR: The point cloud target acquired by the LiDAR is filtered according to the reflectivity set during manufacturing, retaining only the target intensity range. This range can be obtained through multiple tests based on the coating of the actual calibration device. Optionally, stray point clouds can be filtered out using clustering or sample consensus algorithms. A point cloud representing the calibration device can be obtained using its color and surface features. Further, using prior information about the corner reflector and connecting plane dimensions, surface reconstruction is performed to obtain the vertex set P inside the corner reflector. L .

[0054] Visible light camera: The image information acquired by the visible light camera is transformed into HSV space to obtain the HSV representation of the image. HSV range filtering is then performed using known color prior features from the calibration device to retain only the target color gamut. Optionally, other color blocks outside the corner reflector are filtered out using the color gamut stitching features of the three internal surfaces of the corner reflector. Line features of the corner reflector are obtained using the LSD isolinear feature extraction algorithm, and then the pixel set P representing the vertices inside the corner reflector is obtained based on prior information about the corner reflector size. C Alternatively, by creating a dataset of vertices inside the corner reflector of the target calibration device, a neural network can be trained to obtain the pixels representing the vertices inside the corner reflector, thereby obtaining the image target point set P. C .

[0055] Infrared camera: Image information acquired by the infrared camera is filtered out according to a set grayscale threshold. Due to the different heat absorption rates of the internal surfaces of the corner reflector, continuous corner reflector images with different and uniform grayscale are formed. This feature can be used to locate the position of the calibration device, and other irrelevant pixels are filtered out based on prior distance information. Furthermore, the edge line features of the corner reflector are obtained through the LSD isolinear feature extraction algorithm, and then the pixel set P representing the internal vertices of the corner reflector is obtained based on the prior information of the corner reflector size. I .

[0056] Step 4: Based on the timestamp, repeatedly detect the vertex set inside the corner reflector of each sensor in Step 3, save it as an array, and form corresponding point pairs between sensors based on the prior position information between the corner reflectors.

[0057] Step 5: After detection, the LiDAR, millimeter-wave radar, visible light camera, and infrared camera each obtained their respective vertex point sets inside the corner reflectors. The LiDAR and millimeter-wave radar are 3D point cloud point sets, while the visible light camera and infrared camera are 2D pixel point sets. The calculation of extrinsic parameters between them falls into three categories: extrinsic parameters between 3D point sets are calculated using iterative nearest-point registration algorithms; extrinsic parameters between 3D and 2D point sets are calculated using perspective n-point localization methods; and extrinsic parameters between 2D point sets are calculated using image matching algorithms such as descriptors. Optionally, since each acquired point pair has redundancy, random iteration can be used to select the extrinsic parameter with the smallest reprojection error when calculating the extrinsic parameters between sensors. Optionally, after obtaining multiple extrinsic parameters between pairs, since the points detected by each sensor have mutual correspondences, the optimal extrinsic parameter can be selected by verifying the correspondence between points.

[0058] Step 5: Calculate and monitor the error of the external parameters based on the factory parameters of the unmanned vehicle. If the error exceeds a certain threshold, replace the parameters; otherwise, keep the original factory parameters.

[0059] Example 3

[0060] With the help of an on-board calibration device, millimeter-wave radar and camera are fused to achieve precise vehicle positioning.

[0061] Step 1: During the operation of the mobile platform, it encounters a vehicle equipped with the calibration device.

[0062] Step 2: Obtain sensor data from millimeter-wave radar and camera respectively. The millimeter-wave radar provides point cloud data, and the camera provides image data.

[0063] Step 3: Using neural networks such as YOLO, identify vehicle obstacles in the image data and obtain the corresponding pixel boxes.

[0064] Step 4: Based on the set point cloud reflection intensity, filter out millimeter-wave point clouds below the set value, and determine the position of the calibration device based on the prior corner reflector spacing of the calibration device.

[0065] Step 5: Project the millimeter-wave radar point cloud onto the image coordinate system using the extrinsic parameter matrix between the millimeter-wave radar and the camera and the intrinsic parameter matrix of the camera.

[0066] Step 6: If the extrinsic parameter matrix is ​​accurate, the point cloud will be projected onto its respective vehicle 2D bounding box, and the two can then perform feature-level fusion of data from the visible light camera and the millimeter-wave radar camera to achieve calibration.

[0067] The above description is merely a preferred embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A construction and calibration method for a multi-sensor online extrinsic parameter calibration device, characterized in that, Includes the following steps: A calibration device is constructed by arranging and combining several corner reflectors with different coatings on their inner surfaces. Based on the calibration device, multiple sensors are used to acquire the set of vertex points inside the corner reflector. The internal vertex point sets of the corner reflectors acquired by each sensor are matched pairwise to obtain the extrinsic parameters between each pair of sensors. The external parameters are filtered to obtain the optimal external parameters, and the external parameters are calibrated based on the optimal external parameters. The intersecting surfaces inside the corner reflector are coated with coatings of different colors, heat absorption rates, or laser reflectivities. The process of acquiring the corresponding vertex set inside the corner reflector using multiple sensors includes: simultaneously acquiring the vertex set inside the corner reflector in the form of point cloud using lidar, point cloud using millimeter-wave radar, image using visible light camera, and thermal radiation imaging using infrared camera, and recording the timestamp. The process of acquiring the vertex set inside the corner reflector in point cloud form based on lidar includes: acquiring point cloud data based on lidar, then filtering the point cloud intensity range based on the laser reflectivity of the calibration device to obtain the target intensity range; within the target intensity range, filtering out stray point cloud data using clustering or sample consensus algorithms, and obtaining the calibration device point cloud based on the color and surface features of the calibration device; and reconstructing the surface based on the calibration device point cloud and the size information of the corner reflector and connecting planes to obtain the vertex set inside the corner reflector corresponding to the lidar. The process of obtaining the internal vertex point set of the corner reflector in the form of point cloud based on millimeter-wave radar includes: collecting point cloud based on millimeter-wave radar, taking the point cloud with reflection intensity higher than the preset reflection intensity as the initial three-dimensional point set of the corresponding corner reflector; filtering the initial three-dimensional point set through the arrangement and spacing information of the calibration device, thereby obtaining the internal vertex point set of the corner reflector corresponding to the millimeter-wave radar. The process of acquiring the internal vertex set of a corner reflector in image form using a visible light camera includes: acquiring image information using the visible light camera, performing HSV space transformation to obtain the HSV representation of the image, and then performing HSV range filtering based on the color features of the calibration device to obtain the target color gamut of the image; within the target color gamut of the image, filtering out color blocks of the non-corner reflector based on the internal color stitching features of the calibration device, and then obtaining the corresponding corner reflector line features based on the LSD isolinear feature extraction algorithm, and finally obtaining the internal vertex set of the corner reflector corresponding to the visible light camera based on the size information of the corner reflector; or acquiring the internal vertex dataset of the corner reflector of the calibration device, and then obtaining the internal vertex set of the corner reflector corresponding to the visible light camera based on a neural network; The process of acquiring the internal vertex set of a corner reflector using an infrared camera in the form of thermal radiation imaging includes: acquiring image information based on the infrared camera; performing pixel filtering on the image based on the heat absorption rate of different internal surfaces of the calibration device; then obtaining the edge line features of the corner reflector based on the LSD isolinear feature extraction algorithm; and finally obtaining the internal vertex set of the corner reflector corresponding to the infrared camera based on the size information of the corner reflector.

2. The construction and calibration method of the multi-sensor online extrinsic parameter calibration device according to claim 1, characterized in that, The vertex set inside the corner reflector obtained by the lidar and millimeter-wave radar is a three-dimensional point cloud set, while the vertex set inside the corner reflector obtained by the visible light camera and infrared camera is a two-dimensional point cloud set.

3. The construction and calibration method of the multi-sensor online extrinsic parameter calibration device according to claim 2, characterized in that, The process of pairwise matching of the vertex point sets inside the corner reflectors corresponding to each sensor includes: obtaining extrinsic parameters between the three-dimensional point cloud point sets based on the iterative nearest point registration algorithm; obtaining extrinsic parameters between any three-dimensional point cloud point set and any two-dimensional point cloud point set based on the perspective n-point positioning method; and obtaining extrinsic parameters between the two-dimensional point cloud point sets based on the image matching algorithm.

4. The construction and calibration method of the multi-sensor online extrinsic parameter calibration device according to claim 1, characterized in that, The process of filtering the extrinsic parameters includes: filtering the extrinsic parameters based on the correspondence between the vertex point sets inside the corner reflector obtained by each sensor, and obtaining the optimal extrinsic parameters.

Images