External parameter calibration method and system for target-free laser event camera based on information distance

Abstract

The present invention relates to the field of image processing technology, and provides a method and system for calibrating the extrinsic parameters of a target-free laser event camera based on information distance. The method comprises: generating a cumulative event image and a laser intensity image, and performing image enhancement processing; performing preliminary optimization on the enhanced cumulative event image and the laser intensity image by minimizing the reprojection error to obtain initial extrinsic parameters; obtaining the probability density function of the laser point cloud intensity, event pixel value and their joint distribution by kernel density estimation, and calculating the normalized information distance based on the information entropy principle; using a rotation and translation extrinsic parameter decoupling optimization method based on sequential quadratic programming, optimizing the initial extrinsic parameters by normalized information distance constraints to obtain the optimal extrinsic parameters. The present invention realizes robust calibration of laser radars and event cameras without the need for targets and manual initial value setting, and can stably, smoothly and accurately achieve extrinsic parameter optimization solutions.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and provides a method and system for calibrating extrinsic parameters of a target-free laser event camera based on information distance. Background Art

[0002] Event cameras are a new type of visual sensor based on pixel brightness changes, showing broad application prospects in tasks such as feature extraction, optical flow estimation, and object recognition. Unlike traditional frame cameras, event cameras do not capture images at a fixed frame rate, but instead record the brightness changes of each pixel asynchronously. This feature gives them significant advantages in several aspects:

[0003] 1) High temporal resolution: Event cameras are able to detect and mark events with microsecond temporal resolution, which enables them to capture very fast motion and avoid the motion blur problem of traditional cameras.

[0004] 2) Low Latency: Each pixel operates independently, without waiting for the global exposure time. Once a change is detected, it is transmitted immediately. As a result, the latency of the event camera is extremely low, approximately sub-millisecond in actual products.

[0005] 3) Low power consumption: Only changes in brightness are transmitted, and power consumption is only used to process the constantly changing pixels.

[0006] 4) High dynamic range: Dynamic range refers to the range of brightness values ​​that an imaging system can capture, that is, the brightness ratio from the darkest part to the brightest part of the scene.

[0007] The fusion of laser radar and event camera can bring out their complementary advantages and enhance the robot's perception capabilities. To achieve accurate fusion of laser radar and event camera, high-precision extrinsic parameter calibration is first required. However, existing research on extrinsic parameter calibration of lidar and event camera still has the following limitations:

[0008] 1) The characteristics of event cameras make target-based methods more complicated, limiting their practical applications.

[0009] 2) Methods based on statistical features such as mutual information rely on relatively accurate initial values ​​of external parameters.

[0010] 3) Methods based on geometric features are difficult to apply directly due to the poor quality of event images. Summary of the Invention

[0011] The present invention aims to address at least one of the technical problems existing in the related art. To this end, it provides a target-free laser event camera extrinsic parameter calibration method based on information distance. This method achieves robust calibration of lidar and event cameras without the need for a target or manual initialization, and achieves stable, smooth, and accurate extrinsic parameter optimization.

[0012] The present invention provides a method for calibrating external parameters of a target-free laser event camera based on information distance, comprising:

[0013] S1: Continuously accumulate event camera data to generate cumulative event images while continuously recording lidar point clouds;

[0014] S2: Projecting the laser radar point cloud onto the accumulated event image to generate a laser intensity image, and performing image enhancement processing on the laser intensity image to obtain an enhanced laser intensity image;

[0015] S3: performing image enhancement processing on the accumulated event image to obtain an enhanced accumulated event image;

[0016] S4: extract matching point pairs between the enhanced cumulative event image and the enhanced laser intensity image, and minimize the reprojection error to obtain the initial extrinsic parameters;

[0017] S5: performing contrast enhancement on the laser point cloud intensity value, calculating a pixel value histogram of the enhanced laser point cloud intensity value, a pixel value histogram of the enhanced cumulative event image, and a pixel value joint distribution histogram;

[0018] S6: Based on the pixel value histogram of the laser point cloud intensity value, the pixel value histogram of the enhanced cumulative event image, and the pixel value joint distribution histogram, a laser point cloud intensity value density function, an event pixel value density function, and a probability density function of the joint distribution are obtained by kernel density estimation, and a normalized information distance is calculated based on the information entropy principle;

[0019] S7: Using the rotation-translation decoupling method based on sequential quadratic programming, the final extrinsic parameters are optimized with the goal of minimizing the normalized information distance.

[0020] According to a method for extrinsic parameter calibration of a target-free laser event camera based on information distance provided by the present invention, in step S1, the accumulation time window for accumulating event camera data is 3 to 5 seconds.

[0021] According to a method for calibrating external parameters of a non-target laser event camera based on information distance provided by the present invention, step S2 includes:

[0022] S21: Projecting the laser point cloud into the accumulated event image through the virtual camera;

[0023] S22: Assigning the reflection intensity of the laser point cloud to the pixel value of the projected cumulative event image to generate a laser intensity image and a back-projection index;

[0024] S23: The laser intensity image is filled with a rectangular dilation kernel, and then the contrast is enhanced by a global histogram equalization method. Finally, the median filter method is used to remove noise to obtain an enhanced laser intensity image.

[0025] According to a method for calibrating external parameters of a non-target laser event camera based on information distance provided by the present invention, step S23 includes:

[0026] S231: Using rectangular dilation kernels of different sizes to perform morphological operations in sequence, filling holes and connecting edges of objects, extracting a mask of the valid pixel area before each dilation, and restoring the pixel values ​​corresponding to the mask position to the state before dilation after each dilation;

[0027] S232: Calculating the histogram and cumulative distribution function of the image and mapping the intensity values ​​according to the cumulative distribution function to obtain a balanced intensity image and a balanced grayscale;

[0028] S233: Use a sliding window to traverse the equalized image, sort the pixels in the window according to the equalized grayscale, and take the median value to replace the original grayscale value of the center pixel of the window to obtain an enhanced laser intensity image.

[0029] According to a method for extrinsic calibration of a target-free laser event camera based on information distance provided by the present invention, step S3 includes: directly applying histogram equalization to the cumulative event image to improve its contrast, and then applying Gaussian blur to smooth the image to eliminate noise caused by equalization and smooth optimization to obtain an enhanced cumulative event image.

[0030] According to a method for calibrating external parameters of a non-target laser event camera based on information distance provided by the present invention, step S4 includes:

[0031] The back-projection index is used to obtain 3D-to-2D corresponding point pairs between the enhanced laser intensity image and the enhanced cumulative event image, and the 3D-to-2D corresponding point pairs are calculated using a re-projection error minimization formula to obtain initial extrinsic parameters.

[0032] According to the present invention, a method for calibrating external parameters of a non-target laser event camera based on information distance, step S5 includes:

[0033] S51: performing histogram equalization on the intensity values ​​of the laser point cloud to enhance the contrast of the laser point cloud and obtain an enhanced laser point cloud;

[0034] S52: traverse each point in the enhanced laser point cloud, project it to the event camera image coordinate system using the initial extrinsic parameters, and obtain a pixel value histogram of the laser point cloud intensity value, a pixel value histogram of the enhanced cumulative event image, and a pixel value joint distribution histogram based on the intensity value and the enhanced cumulative event image pixel value corresponding to the event pixel coordinate position.

[0035] According to a method for calibrating extrinsic parameters of a non-target laser event camera based on information distance provided by the present invention, step S6 includes:

[0036] S61: Calculating the bandwidth for controlling the smoothing degree by using a bandwidth selection rule;

[0037] S62: normalizing the histogram, applying a Gaussian blur convolution operation to the normalized histogram, smoothing the discrete histogram using a Gaussian kernel function to generate a probability density function, and obtaining a laser point cloud intensity value density function, an event pixel value density function, and a probability density function of a joint distribution;

[0038] S63: Calculating the laser point cloud entropy based on the laser point cloud intensity value density function; calculating the event image entropy based on the event pixel value density function; and calculating the joint entropy based on the probability density function of the joint distribution;

[0039] S64: Calculating the normalized information distance between the laser point cloud and the event image according to the laser point cloud entropy, the event image entropy, and the joint entropy.

[0040] According to a method for calibrating extrinsic parameters of a non-target laser event camera based on information distance provided by the present invention, step S7 includes:

[0041] S71: Parameterize the initial extrinsic parameters, impose translation and rotation boundary constraints on the parameterized initial extrinsic parameters, and construct an optimization problem;

[0042] The boundary constraints are obtained by adding or subtracting boundary values ​​from each component of the parameterized initial extrinsic parameters, which represent the boundary constraint thresholds imposed on the translation and rotation extrinsic parameters respectively;

[0043] S72: Solve the optimization problem using bounded sequential quadratic programming to optimize all components of the extrinsic parameters including translation parameters and rotation parameters;

[0044] S73: After the global optimization converges, set a convergence threshold, optimize the rotation parameters separately, and obtain the rotation parameters of the final extrinsic parameters;

[0045] S74: Select the near-field point cloud, optimize the translation parameters separately, and obtain the translation parameters of the final extrinsic parameters;

[0046] S75: The rotation parameter of the final extrinsic parameter and the translation parameter of the final extrinsic parameter are combined to form the final extrinsic parameter.

[0047] The present invention also provides an information distance-based extrinsic parameter calibration system for a target-free laser event camera, comprising:

[0048] Data collection and preprocessing module: continuously accumulates event camera data to generate a cumulative event image, while continuously recording a lidar point cloud; projects the lidar point cloud onto the cumulative event image to generate a laser intensity image, and performs image enhancement processing on the laser intensity image to obtain an enhanced laser intensity image;

[0049] The extrinsic parameter matrix initialization module is used to perform image enhancement processing on the cumulative event image to obtain an enhanced cumulative event image; extract matching point pairs between the enhanced cumulative event image and the enhanced laser intensity image, and minimize the reprojection error to obtain the initial extrinsic parameters;

[0050] An information distance calculation module is used to perform contrast enhancement on the laser point cloud intensity values, calculate a pixel value histogram of the enhanced laser point cloud intensity values, a pixel value histogram of the enhanced cumulative event image, and a pixel value joint distribution histogram; based on the pixel value histogram of the laser point cloud intensity values, the pixel value histogram of the enhanced cumulative event image, and the pixel value joint distribution histogram, obtain a laser point cloud intensity value density function, an event pixel value density function, and a probability density function of the joint distribution through kernel density estimation, and calculate a normalized information distance based on the information entropy principle;

[0051] The optimal extrinsic parameter solving module is used to optimize the final extrinsic parameters using a rotation-translation decoupling method based on sequential quadratic programming, with the goal of minimizing the normalized information distance.

[0052] The above one or more technical solutions in the embodiments of the present invention have at least one of the following technical effects:

[0053] The present invention provides a target-free laser event camera extrinsic parameter calibration method and system based on information distance. The method generates cumulative event images and laser intensity images and performs image enhancement processing. The enhanced cumulative event images and laser intensity images are preliminarily optimized by minimizing the reprojection error to obtain initial extrinsic parameters, thereby realizing robust calibration of the laser radar and event camera without the need for targets and manual initial value setting. The probability density function of the laser point cloud intensity, event pixel value and their joint distribution is obtained by kernel density estimation, and the normalized information distance is calculated based on the information entropy principle. The rotation and translation extrinsic parameter decoupling optimization method based on sequential quadratic programming is used to optimize the initial extrinsic parameters through the normalized information distance constraints to obtain the optimal extrinsic parameters, thereby achieving stable, smooth and accurate extrinsic parameter optimization solution.

[0054] Additional aspects and advantages of the present invention will be set forth in part in the description which follows and, in part, will be obvious from the description which follows, or may be learned by practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0056] Figure 1 It is a flow chart of the method for extrinsic parameter calibration of a target-free laser event camera based on information distance provided by the present invention.

[0057] Figure 2 This is a structural block diagram of the information distance-based extrinsic parameter calibration device for a target-free laser event camera provided by the present invention.

[0058] Reference numerals:

[0059] 101. Data collection and preprocessing module; 102. External parameter matrix initialization module; 103. Information distance calculation module; 104. Optimal external parameter solution module. DETAILED DESCRIPTION

[0060] To make the purpose, technical solutions and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below. Obviously, the embodiments described are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. The following embodiments are used to illustrate the present invention, but are not used to limit the scope of the present invention.

[0061] In the description of this specification, the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" mean that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the embodiment of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0062] The following combination Figures 1 to 2 The present invention is described.

[0063] Example

[0064] like Figure 1 As shown, Figure 1The figure is a flow chart of the extrinsic parameter calibration method for a target-free laser event camera based on information distance provided by the present invention, which specifically includes the following steps:

[0065] S1: Continuously accumulate event camera data to generate cumulative event images while continuously recording lidar point clouds;

[0066] S2: Projecting the laser radar point cloud onto the accumulated event image to generate a laser intensity image, and performing image enhancement processing on the laser intensity image to obtain an enhanced laser intensity image;

[0067] S3: performing image enhancement processing on the accumulated event image to obtain an enhanced accumulated event image;

[0068] S4: extract matching point pairs between the enhanced cumulative event image and the enhanced laser intensity image, and minimize the reprojection error to obtain the initial extrinsic parameters;

[0069] S5: performing contrast enhancement on the laser point cloud intensity value, calculating a pixel value histogram of the enhanced laser point cloud intensity value, a pixel value histogram of the enhanced cumulative event image, and a pixel value joint distribution histogram;

[0070] S6: Based on the pixel value histogram of the laser point cloud intensity value, the pixel value histogram of the enhanced cumulative event image, and the pixel value joint distribution histogram, a laser point cloud intensity value density function, an event pixel value density function, and a probability density function of the joint distribution are obtained by kernel density estimation, and a normalized information distance is calculated based on the information entropy principle;

[0071] S7: Using the rotation-translation decoupling method based on sequential quadratic programming, the final extrinsic parameters are optimized with the goal of minimizing the normalized information distance.

[0072] Specifically, step S1 includes:

[0073] S11: Fix the lidar and event camera in the scene to ensure their stability during scanning so that they can accurately collect environmental information;

[0074] S12: Set the accumulation time window to 3 to 5 seconds to ensure that sufficient event data is included and avoid the blurring effect caused by too long a time. At the same time, ignore the polarity information of all trigger events and accumulate them to obtain the cumulative event image;

[0075] S13: When the device is stationary, the lidar point cloud is continuously recorded.

[0076] In the event camera, each event Contains the following information: the pixel location where the event occurred ; Time when the event is triggered ; Event polarity , Indicates the direction of brightness change, with a value of -1 or 1. By ignoring event polarity information, all event polarities are defaulted to 1, and the events within the event window are accumulated to obtain a cumulative event image.

[0077] Specifically, step S2 includes:

[0078] S21: Project the laser point cloud into the accumulated event image through a virtual camera, where the posture of the virtual camera is the rotation matrix of the current extrinsic parameters;

[0079] S22: Assign the reflection intensity of the laser point cloud to the projected pixel value to generate a laser intensity image and generate a back-projection index at the same time, so as to restore the 3D coordinates of the original point cloud based on the 2D pixels;

[0080] S23: Filling the laser intensity image with a rectangular dilation kernel to ensure image continuity, then using a global histogram equalization method to enhance contrast, and finally using a median filter to remove noise to obtain an enhanced laser intensity image;

[0081] S231: Rectangular expansion kernel filling: use 3 in sequence × 3, 5 × 5 and 7 × A morphological operation is performed using a rectangular dilation kernel of 7, filling holes and connecting edges. Before each dilation, a mask of the valid pixel area is extracted. After each dilation, the pixel values ​​corresponding to the masked position are restored to their pre-dilation state. During this process, the image values ​​in the valid area remain unchanged, preserving original image features such as edges and corners.

[0082] S232: Calculate the histogram and cumulative distribution function of the image:

[0083]

[0084] in, is the cumulative distribution function, is the grayscale value, Grayscale The number of pixels, is the total number of pixels.

[0085] Map the intensity value according to the cumulative distribution function to obtain the grayscale after intensity equalization :

[0086]

[0087] Indicates the grayscale value range of the pixel. In the embodiment of the present invention, it is 255. Indicates rounding down.

[0088] S233: Use a 3 × The sliding window of 3 traverses the image, sorts the pixels in the window according to the grayscale value, performs median filtering, and takes the median value to replace the original grayscale value of the pixel in the center of the window.

[0089] Specifically, step S3 includes:

[0090] Histogram equalization is directly applied to the cumulative event image to improve its contrast. Then Gaussian blur is applied to smooth the image to eliminate the noise caused by equalization and to achieve smooth optimization, thus obtaining an enhanced cumulative event image.

[0091] Among them, Gaussian blur smoothing is to use the Gaussian function to perform convolution operation on the image, which effectively reduces image noise and details while retaining the overall structural characteristics.

[0092]

[0093] in, is a Gaussian smoothing function, is the horizontal coordinate distance from the pixel to the center of the kernel, is the vertical coordinate distance from the pixel to the center of the kernel, is the standard deviation.

[0094] Specifically, step S4 includes:

[0095] S41: A semi-dense matching network is used to iteratively estimate the image matching network to cross-modally estimate the 2D to 2D corresponding point pairs between the enhanced laser intensity image and the enhanced cumulative event image. The 3D to 2D corresponding set between the laser point cloud points and the enhanced cumulative event image pixels is obtained through the back-projection index generated by S22.

[0096] .

[0097] in, For the Laser point cloud points, For the An enhanced cumulative event image, is the laser point cloud ordinal number, is the total number of laser point clouds, is the set of real numbers.

[0098] S42: Optimize the external parameters by minimizing the reprojection error to obtain the initial external parameters , :

[0099]

[0100] in, To obtain the minimum function, is the rotation matrix, is the translation vector, represents the Cauchy robust kernel function, It is the event camera projection function, which projects a 3D point from the event camera coordinate system to the image pixel coordinate system.

[0101] Specifically, step S5 includes:

[0102] S51: performing histogram equalization on the intensity values ​​of the laser point cloud, calculating the histogram and cumulative distribution function of the point cloud intensity values, and mapping the intensity values ​​according to the cumulative distribution function to obtain point cloud data after intensity equalization;

[0103] S52: First initialize the following histogram: pixel value histogram of enhanced laser point cloud intensity value , , used to count the distribution of point cloud intensity values ​​in 256 discrete intervals; enhance the pixel value histogram of the cumulative event image , , used to count the distribution of event pixel values ​​in 256 discrete intervals; pixel value joint distribution histogram , recorded as , used to count the intensity of point cloud and event pixels The joint distribution in discrete intervals.

[0104] S53: Traverse all point clouds: For the Laser point cloud , , where represents the point cloud intensity, is the horizontal coordinate of the laser point cloud, is the vertical coordinate of the laser point cloud, is the vertical coordinate of the laser point cloud, The point cloud intensity value is projected into the image coordinate system according to the initial external parameters and the event camera internal parameters, and its corresponding image pixel coordinates are calculated

[0105]

[0106] in, is the horizontal coordinate of the pixel coordinate, is the pixel coordinate ordinate, Transpose the matrix.

[0107] S54: Yes Round it up and determine whether it is within the range of the event image. If it is within the range, the point cloud intensity value and the event pixel value corresponding to the pixel position are rounded up. Add 1 and accumulate it into each histogram and the joint histogram.

[0108] Specifically, step S6 includes:

[0109] S61: Calculate the bandwidth that controls the degree of smoothing using the Silverman bandwidth selection rule :

[0110]

[0111] in, is the standard deviation calculated from the histogram, is the number of samples, that is, the sum of the data in all intervals in the histogram;

[0112] S62: Histogram 、 、 Normalize and apply bandwidth to the normalized histogram The Gaussian blur convolution operation is used to smooth the discrete histogram through the Gaussian kernel function to generate a probability density function and obtain the laser point cloud intensity value density function. , event pixel value density function and the probability density function of the joint distribution ;

[0113] S63: Calculate laser point cloud entropy based on probability density function , entropy of event image , and joint entropy :

[0114]

[0115] in, is the laser point cloud independent variable, , is the event image independent variable, .

[0116] S64: Calculate the normalized information distance between the laser point cloud and the event image:

[0117]

[0118] in, represents the mutual information between the laser point cloud and the event image, Represents the normalized information distance between the laser point cloud and the event image.

[0119] The Normalized Information Distance (NID) measures the information difference between two variables. It quantifies their similarity or correlation by measuring the degree of information shared between two random variables. A smaller NID value indicates a stronger correlation between the two data sets, and vice versa. Therefore, in the extrinsic parameter calibration problem, the more accurate the extrinsic parameters, the better the alignment between the point cloud and the event image, and the corresponding NID should be smaller. Based on this principle, by constructing an optimization problem with minimizing the normalized information distance as the optimization objective, the optimal extrinsic parameters can be obtained.

[0120] Specifically, step S7 includes:

[0121] S71: Parameterize the initial extrinsic parameters to obtain parameterized initial extrinsic parameters , ,right Apply translation and rotation boundary constraints and formulate the optimization problem:

[0122]

[0123] in, is the boundary constraint vector, , Represents the horizontal coordinate boundary constraint threshold imposed on the translation extrinsic parameter, Represents the vertical coordinate boundary constraint threshold imposed on the translation extrinsic parameter, Represents the vertical coordinate boundary constraint threshold imposed on the translation extrinsic parameter, Represents the first rotation direction boundary constraint threshold imposed by the rotation extrinsic parameter, Represents the second rotation direction boundary constraint threshold imposed by the rotation extrinsic parameter, represents the third rotation direction boundary imposed by the rotation extrinsic parameter, is the constraint threshold, It is the reference value of external parameters.

[0124] S72: Use bounded sequential quadratic programming to solve the optimization problem constructed in S61. First, optimize all components of the extrinsic parameters, including translation parameters and rotation parameters, to preliminarily align the laser point cloud and event data. Use a large convergence threshold to ensure that the optimization process can quickly converge to a more reasonable initial solution.

[0125] S73: After the global optimization converges, a smaller convergence threshold is set, and the rotation parameters are optimized separately. The accuracy of the rotation alignment is further improved through fine optimization of the rotation parameters, and the rotation parameters of the final extrinsic parameters are obtained;

[0126] S74: Select the near-field point cloud, optimize the translation parameters separately, and obtain the translation parameters of the final extrinsic parameters.

[0127] Combining the above steps, through extrinsic parameter initialization, the dependence on manually set initial values ​​of extrinsic parameters is eliminated; by performing histogram equalization and Gaussian blur preprocessing on the point cloud intensity and event image, the amount of information in low-contrast areas is increased and the convergence and smoothness of the optimization process are improved; by using the normalized information distance as the objective function for the optimization of the lidar and event camera extrinsic parameters, introducing boundary constraints during the optimization process and adopting an optimization strategy that decouples the rotation and translation extrinsic parameters, the calibration accuracy and robustness are effectively improved.

[0128] To evaluate the performance of the initialization algorithm used in a target-free laser event camera extrinsic calibration method based on information distance, extrinsic initialization tests were conducted using one, two, and three laser-event data pairs, respectively, based on datasets collected from event cameras and vehicle-mounted lidar. Due to the lack of comparable laser-event extrinsic initialization algorithms, only the proposed extrinsic initialization algorithm was evaluated. Evaluation metrics included rotation error (the axis angle between the calibration rotation matrix and the reference rotation matrix), translation error (the Euclidean distance between the calibration translation vector and the reference translation vector), and success rate. Table 1 shows the experimental results of the extrinsic initialization experiments. The algorithm's performance significantly improves with increasing data volume: even with only one pair of data, the success rate is 72.04%. With two pairs of data, the success rate reaches 90.22%, with rotation and translation errors of 0.476° and 0.034m, respectively. Using three pairs of data further increases the success rate to 98.91%, and the rotation and translation errors are reduced to 0.456° and 0.027m, respectively. The above results show that the S3 initialization method has a high success rate and low initialization error, and its initialization results can provide reliable initial values ​​for subsequent external parameter optimization based on information distance.

[0129] Table 1 Performance evaluation of the external parameter initialization algorithm of the method

[0130]

[0131] In order to evaluate the overall extrinsic parameter calibration performance, the targetless laser event camera extrinsic parameter calibration method based on information distance is compared with the traditional method (L2E). The specific experimental settings are as follows: 50 groups of data containing 1 pair, 5 pairs, 10 pairs, 20 pairs and 40 pairs of point clouds and event images are randomly selected from the data set to test the method, the method without image enhancement and the traditional method. Table 2 shows the results of the comparative experiment. In order to avoid the singularity problem of Euler angles, Rodgers vectors are used to represent extrinsic parameter rotation (the rotation angles of the vectors shown in the table are in radians, which are dimensionless, so the units are not marked in the table). When using 40 pairs of point cloud-event data, the standard deviation of the extrinsic parameter translation calibration of the method is [0 . 006m , 0 .006m , 0 . 004m], the standard deviation of the external parameter rotation Rodrigo vector is [0 . 001 , 0 . 001 , 0 . 001], and achieved highly consistent calibration results.

[0132] Compared to traditional methods, the proposed method reduces extrinsic translation error by 48% and extrinsic rotation error by 20%. The proposed method achieves optimal performance when using both a rotation-translation decoupling optimization strategy and image enhancement, which is the key reason for its significant performance improvement over the L2E method. This result demonstrates that the proposed method offers higher accuracy and robustness in the lidar-event camera extrinsic calibration task.

[0133] Table 2 Performance comparison of lidar-event camera extrinsic calibration algorithms

[0134]

[0135] like Figure 2 As shown, the present invention also provides a target-free laser event camera extrinsic parameter calibration system based on information distance, including the following modules:

[0136] The data collection and preprocessing module 101 is used to continuously accumulate event camera data to generate an accumulated event image, while continuously recording a lidar point cloud; project the lidar point cloud onto the accumulated event image to generate a laser intensity image; and perform image enhancement processing on the laser intensity image to obtain an enhanced laser intensity image;

[0137] The extrinsic parameter matrix initialization module 102 is used to perform image enhancement processing on the cumulative event image to obtain an enhanced cumulative event image; extract matching point pairs between the enhanced cumulative event image and the enhanced laser intensity image, and minimize the reprojection error to obtain initial extrinsic parameters;

[0138] The information distance calculation module 103 is used to perform contrast enhancement on the laser point cloud intensity values, calculate a pixel value histogram of the enhanced laser point cloud intensity values, a pixel value histogram of the enhanced cumulative event image, and a pixel value joint distribution histogram; obtain a laser point cloud intensity value density function, an event pixel value density function, and a probability density function of the joint distribution through kernel density estimation based on the pixel value histogram of the laser point cloud intensity values, the pixel value histogram of the enhanced cumulative event image, and the pixel value joint distribution histogram, and calculate a normalized information distance based on the information entropy principle;

[0139] The optimal extrinsic parameter solving module 104 is configured to optimize the final extrinsic parameters using a rotation-translation decoupling method based on sequential quadratic programming with minimization of the normalized information distance as the optimization goal.

[0140] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0141] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0142] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

[0143] It should be noted that the embodiments of the present disclosure can be implemented by hardware, software, or a combination of software and hardware. The hardware portion can be implemented using dedicated logic; the software portion can be stored in a memory and executed by an appropriate instruction execution system such as a microprocessor or dedicated hardware. Those skilled in the art will understand that the above-mentioned devices and methods can be implemented using computer-executable instructions and / or contained in processor control code, for example, such code is provided on a programmable memory or a data carrier such as an optical or electronic signal carrier.

[0144] In addition, although the operations of the method of the present disclosure are described in a particular order in the accompanying drawings, this does not require or imply that these operations must be performed in this particular order, or that all the operations shown must be performed to achieve the desired result. On the contrary, the steps depicted in the flow chart can change the order of execution. Additionally or alternatively, certain steps can be omitted, multiple steps can be combined into one step, and / or one step can be decomposed into multiple steps. It should also be noted that the features and functions of two or more devices according to the present disclosure can be embodied in one device. Conversely, the features and functions of a device described above can be further divided into being embodied by multiple devices.

[0145] Although the present disclosure has been described with reference to several specific embodiments, it should be understood that the present disclosure is not limited to the specific embodiments disclosed. The present disclosure is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

Claims

1. A method for extrinsic parameter calibration of a target-free laser event camera based on information distance, characterized in that: include: S1: Continuously accumulate event camera data to generate cumulative event images while continuously recording lidar point clouds; S2: Projecting the laser radar point cloud onto the accumulated event image to generate a laser intensity image, and performing image enhancement processing on the laser intensity image to obtain an enhanced laser intensity image, including: S21: Projecting the laser point cloud into the accumulated event image through the virtual camera; S22: Assigning the reflection intensity of the laser point cloud to the pixel value of the projected cumulative event image to generate a laser intensity image and a back-projection index; S23: Filling the laser intensity image with a rectangular dilation kernel, then enhancing the contrast using a global histogram equalization method, and finally removing noise using a median filter method to obtain an enhanced laser intensity image; S3: performing image enhancement processing on the accumulated event image to obtain an enhanced accumulated event image; S4: extract matching point pairs between the enhanced cumulative event image and the enhanced laser intensity image, and minimize the reprojection error to obtain the initial extrinsic parameters; S5: performing contrast enhancement on the laser point cloud intensity value, calculating a pixel value histogram of the enhanced laser point cloud intensity value, a pixel value histogram of the enhanced cumulative event image, and a pixel value joint distribution histogram; S6: Based on the pixel value histogram of the laser point cloud intensity value, the pixel value histogram of the enhanced cumulative event image, and the pixel value joint distribution histogram, a laser point cloud intensity value density function, an event pixel value density function, and a probability density function of the joint distribution are obtained by kernel density estimation, and a normalized information distance is calculated based on the information entropy principle; S7: Using the rotation-translation decoupling method based on sequential quadratic programming, the final extrinsic parameters are optimized with the goal of minimizing the normalized information distance.

2. The method for extrinsic parameter calibration of a target-free laser event camera based on information distance according to claim 1, characterized in that: In step S1, the accumulation time window for accumulating event camera data is 3 to 5 seconds.

3. The method for extrinsic parameter calibration of a non-target laser event camera based on information distance according to claim 1, characterized in that: Step S23 includes: S231: Using rectangular dilation kernels of different sizes to perform morphological operations in sequence, filling holes and connecting edges of objects, extracting a mask of the valid pixel area before each dilation, and restoring the pixel values ​​corresponding to the mask position to the state before dilation after each dilation; S232: Calculating the histogram and cumulative distribution function of the image and mapping the intensity values ​​according to the cumulative distribution function to obtain a balanced intensity image and a balanced grayscale; S233: Use a sliding window to traverse the equalized image, sort the pixels in the window according to the equalized grayscale, and take the median value to replace the original grayscale value of the center pixel of the window to obtain an enhanced laser intensity image.

4. The method for extrinsic parameter calibration of a target-free laser event camera based on information distance according to claim 1, characterized in that: Step S3 includes: directly applying histogram equalization to the cumulative event image to improve its contrast, and then applying Gaussian blur to smooth the image to eliminate noise caused by equalization and perform smoothing optimization to obtain an enhanced cumulative event image.

5. The method for extrinsic parameter calibration of a target-free laser event camera based on information distance according to claim 1, characterized in that: Step S4 includes: The back-projection index is used to obtain 3D-to-2D corresponding point pairs between the enhanced laser intensity image and the enhanced cumulative event image, and the 3D-to-2D corresponding point pairs are calculated using a re-projection error minimization formula to obtain initial extrinsic parameters.

6. The method for extrinsic parameter calibration of a target-free laser event camera based on information distance according to claim 5, characterized in that: Step S5 includes: S51: performing histogram equalization on the intensity values ​​of the laser point cloud to enhance the contrast of the laser point cloud and obtain an enhanced laser point cloud; S52: traverse each point in the enhanced laser point cloud, project it to the event camera image coordinate system using the initial extrinsic parameters, and obtain a pixel value histogram of the laser point cloud intensity value, a pixel value histogram of the enhanced cumulative event image, and a pixel value joint distribution histogram based on the intensity value and the enhanced cumulative event image pixel value corresponding to the event pixel coordinate position.

7. The method for extrinsic parameter calibration of a non-target laser event camera based on information distance according to claim 6, characterized in that: Step S6 includes: S61: Calculating the bandwidth for controlling the smoothing degree by using a bandwidth selection rule; S62: normalizing the histogram, applying a Gaussian blur convolution operation to the normalized histogram, smoothing the discrete histogram using a Gaussian kernel function to generate a probability density function, and obtaining a laser point cloud intensity value density function, an event pixel value density function, and a probability density function of a joint distribution; S63: Calculating the laser point cloud entropy based on the laser point cloud intensity value density function; calculating the event image entropy based on the event pixel value density function; and calculating the joint entropy based on the probability density function of the joint distribution; S64: Calculating the normalized information distance between the laser point cloud and the event image according to the laser point cloud entropy, the event image entropy, and the joint entropy.

8. The method for extrinsic parameter calibration of a non-target laser event camera based on information distance according to claim 7, characterized in that: Step S7 includes: S71: Parameterize the initial extrinsic parameters, impose translation and rotation boundary constraints on the parameterized initial extrinsic parameters, and construct an optimization problem; The boundary constraints are obtained by adding or subtracting boundary values ​​from each component of the parameterized initial extrinsic parameters, which represent the boundary constraint thresholds imposed on the translation and rotation extrinsic parameters respectively; S72: Solve the optimization problem using bounded sequential quadratic programming to optimize all components of the extrinsic parameters including translation parameters and rotation parameters; S73: After the global optimization converges, set a convergence threshold, optimize the rotation parameters separately, and obtain the rotation parameters of the final extrinsic parameters; S74: Select the near-field point cloud, optimize the translation parameters separately, and obtain the translation parameters of the final extrinsic parameters; S75: The rotation parameter of the final extrinsic parameter and the translation parameter of the final extrinsic parameter are combined to form the final extrinsic parameter.

9. A non-target laser event camera extrinsic parameter calibration system based on information distance, used to execute the non-target laser event camera extrinsic parameter calibration method based on information distance according to any one of claims 1 to 8, characterized in that: include: The data collection and preprocessing module is used to continuously accumulate event camera data, generate cumulative event images, and continuously record lidar point clouds; Projecting the laser radar point cloud onto the accumulated event image to generate a laser intensity image, and performing image enhancement processing on the laser intensity image to obtain an enhanced laser intensity image; The extrinsic parameter matrix initialization module is used to perform image enhancement processing on the cumulative event image to obtain an enhanced cumulative event image; extract matching point pairs between the enhanced cumulative event image and the enhanced laser intensity image, and minimize the reprojection error to obtain the initial extrinsic parameters; An information distance calculation module is used to perform contrast enhancement on the laser point cloud intensity values, calculate a pixel value histogram of the enhanced laser point cloud intensity values, a pixel value histogram of the enhanced cumulative event image, and a pixel value joint distribution histogram; based on the pixel value histogram of the laser point cloud intensity values, the pixel value histogram of the enhanced cumulative event image, and the pixel value joint distribution histogram, obtain a laser point cloud intensity value density function, an event pixel value density function, and a probability density function of the joint distribution through kernel density estimation, and calculate a normalized information distance based on the information entropy principle; The optimal extrinsic parameter solving module is used to optimize the final extrinsic parameters using a rotation-translation decoupling method based on sequential quadratic programming, with the goal of minimizing the normalized information distance.

Images