Vehicle distance measurement and calibration method based on laser radar and binocular vision fusion

Abstract

The invention discloses a vehicle distance measurement and calibration method based on laser radar and binocular vision fusion, relates to the technical field of measurement and calibration, and is used for solving the problem that the precision of a vehicle distance calibration result is reduced when a laser radar used for calibrating a highway vehicle distance snapshot system is under the conditions of rain, snow, wet and slippery or waterlogging road surfaces. Joint modeling is carried out on spatial distribution and echo energy characteristics of laser radar low-height point cloud, a point cloud echo coefficient and a low-height number proportion are introduced, a specular reflection state code is constructed, a self-calibration mechanism is adaptively triggered accordingly, and a specular reflection state is obtained. The discrete characteristic of a laser radar vehicle distance time sequence and the degradation trend of binocular vision parallax connectivity are subjected to quantitative analysis, the dominant source of ranging abnormity is judged through symbolic analysis, and an offset calibration or fusion correction strategy is selected according to the dominant source. Reliability and engineering adaptability of vehicle distance measurement under the wet and slippery road surface condition can be improved under the condition of not depending on an additional sensor.

Description

Technical Field

[0001] This invention relates to the field of measurement and calibration technology, and more specifically, to a method for vehicle distance measurement and calibration based on the fusion of lidar and binocular vision. Background Technology

[0002] With the continuous increase in highway traffic flow, vehicle distance-based violation capture systems have been widely used in automatic evidence collection scenarios for traffic violations such as failing to maintain a safe following distance and following too closely. Existing highway vehicle distance capture systems typically employ traffic violation monitoring systems based on speed radar, advanced camera technology, and computer vision. These systems continuously measure the spatial distance between vehicles in the same lane and, combined with a time dimension, determine whether vehicles are consistently below the safe following distance threshold. Since these systems directly serve traffic enforcement and evidence collection, the accuracy and reliability of their vehicle distance measurement results directly affect the fairness and legality of law enforcement. Therefore, the calibration of the system's vehicle distance measurement results not only requires high precision but also needs to maintain the stability and consistency of the calibration values ​​under complex weather and road conditions.

[0003] Under conditions of wet, slippery roads or roads with standing water, the road surface reflection characteristics change from diffuse reflection to specular reflection. During measurement and scanning, the test vehicle-mounted lidar used for calibration is prone to generate abnormally enhanced echo signals and low-altitude false point clouds. This causes the calibration device to misjudge road surface reflections or vehicle chassis reflections as valid targets, resulting in systematic deviations or increased fluctuations in the distance calibration results, thus affecting the stability of the distance calibration.

[0004] The existing technology has the following shortcomings: Currently, when using lidar to scan targets in front of the test vehicle's lane under conditions of rain, snow, slippery surfaces, water accumulation, or high reflectivity, the lidar used in calibrating highway vehicle distance capture systems is susceptible to abnormal echoes and low-height false point clouds due to road surface reflections. This leads to systematic shifts or fluctuations in the vehicle distance calibration results. Furthermore, binocular vision is prone to parallax degradation in areas of water accumulation or low-texture water film, reducing the stability of vehicle distance calculations. The lack of a self-calibration mechanism for these conditions makes it difficult to guarantee the reliability of the vehicle distance calibration results, resulting in decreased accuracy under rain, snow, slippery surfaces, or water accumulation. Therefore, this paper proposes a vehicle distance measurement calibration method based on the fusion of lidar and binocular vision. Summary of the Invention

[0005] To overcome the aforementioned deficiencies of the prior art, embodiments of the present invention provide a vehicle distance measurement calibration method based on the fusion of lidar and binocular vision. This method addresses the problems mentioned in the background art by employing adaptive recognition of road surface reflection states and a joint analysis mechanism of multi-source ranging stability differences.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a vehicle distance measurement and calibration method based on the fusion of lidar and binocular vision, comprising the following steps: Step S1: While the test vehicle is driving normally, monitor the point cloud data of the vehicle-mounted lidar, partition the point cloud data, filter and mark the radar point cloud, count the number of marked radar point clouds, and detect the echo intensity data of the marked radar point clouds. Step S2: Calculate the point cloud echo coefficient based on the echo intensity data, analyze the specular reflection state of the radar point cloud in combination with the number of marked radar point clouds, determine whether to trigger the self-calibration mechanism based on the specular reflection state, and set the ranging period after triggering the self-calibration mechanism. Step S3: Scan the radar distance of the test vehicle within the ranging period and perform fluctuation evaluation. Generate distance dispersion features based on the evaluation results and store them in the self-calibration database. Use binocular vision to obtain the distance target area of ​​the test vehicle, identify the disparity pixels in the distance target area and calculate the pixel connectivity length. Step S4: Generate the disparity degradation trend of the test vehicle based on the pixel connectivity length, perform symbolic analysis processing in combination with the vehicle distance dispersion features in the self-calibration database, and select fusion correction or offset calibration for the radar distance of the test vehicle based on the analysis results.

[0007] In a preferred embodiment, in step S1, when the test vehicle is driving normally, the lidar is activated to scan the environmental space of the test vehicle and collect point cloud data output by the lidar within the current scanning cycle. Point cloud data consists of multiple point cloud sampling points, which are characterized by three-dimensional spatial coordinates in the lidar coordinate system. The three-dimensional spatial coordinates consist of three mutually orthogonal coordinate components, including the longitudinal distance component, the lateral offset component, and the height component. The set of point cloud sampling points composed of three-dimensional spatial coordinates is defined as radar point cloud; After acquiring the radar point cloud, the radar point cloud is partitioned according to the preset spatial partitioning rules. The spatial partitioning rules classify point cloud sampling points with height components below the preset height threshold into the low-height analysis area, otherwise they are classified into the non-analysis area.

[0008] In a preferred embodiment, in step S1, point cloud sampling points located in the low-altitude analysis area are marked and combined into a marked radar point cloud, and point cloud sampling points that do not fall into the low-altitude analysis area are not included in subsequent statistics. The number of points in the marked radar point cloud is obtained by statistically analyzing the point cloud sampling points. The echo intensity of the sampling points in the marked radar point cloud is detected. The echo intensity is output synchronously by the lidar during the sampling process, which characterizes the amount of energy returned by the laser signal.

[0009] In a preferred embodiment, in step S2, the total echo intensity of all point cloud sampling points within the marked radar point cloud is calculated to obtain the marked echo intensity. The echo intensity data corresponding to each point cloud sampling point in the marked radar point cloud are sorted in descending order, and high-energy sampling points with echo intensity within a preset high-intensity range are extracted. The high-energy echo intensity is obtained by summing the echo intensities of the high-energy sampling points, and the point cloud echo coefficient of the marked radar point cloud is obtained by calculating the proportion of the high-energy echo intensity within the marked echo intensity. The ratio of the number of points in the marked radar point cloud to the total number of point cloud sampling points in the radar point cloud acquired by the lidar during the current scanning cycle is calculated to obtain the proportion of low-altitude points.

[0010] In a preferred embodiment, in step S2, the point cloud echo coefficients are compared with preset multi-level echo coefficient thresholds to classify multiple echo level states and represent them with digital codes. The proportion of low-height quantities is compared with the preset multi-level quantity proportion thresholds to divide the quantity into multiple quantity levels and represent them with letter codes; The echo level status and the quantity level status are combined and encoded to generate a specular reflection status code, which is composed of a combination of numeric and alphanumeric codes. The specular reflection status code is compared with the preset specular reflection trigger threshold. When the specular reflection status code exceeds the preset specular reflection trigger threshold, the specular reflection status of the radar point cloud is determined to be an abnormal reflection status, and the self-calibration mechanism is triggered at this time. When the specular reflection status code does not exceed the preset specular reflection trigger threshold, the specular reflection status of the radar point cloud is determined to be a non-abnormal reflection status, and the self-calibration mechanism is not triggered at this time. After triggering the self-calibration mechanism, the ranging period is further set, which is a time window that limits the time range for subsequent acquisition of radar vehicle distance.

[0011] In a preferred embodiment, in step S3, during the ranging period, the radar distance of the test vehicle is obtained through the vehicle-mounted lidar ranging interface, and the radar distances of each vehicle during the ranging period are arranged in chronological order to form a vehicle distance time series. The fluctuation of the vehicle distance time series is evaluated. Specifically, the median of the vehicle distance time series is selected as the median value of the vehicle distance. The absolute value of the difference between each radar vehicle distance and the median value of the vehicle distance is calculated to obtain the vehicle distance deviation. The vehicle distance deviations are sorted and the median value of the vehicle distance deviation is selected as the median value of the vehicle distance deviation. The ratio of the median vehicle distance deviation to the median vehicle distance is used as the vehicle distance dispersion index; The product of the vehicle distance dispersion index and the preset adjustable dispersion coefficient is used as the vehicle distance dispersion feature.

[0012] In a preferred embodiment, in step S3, during the ranging period, the road area in the direction of travel in front of the test vehicle is simultaneously acquired by the binocular camera in the binocular vision, and the left eye image and the right eye image are obtained respectively. The left eye image is processed for forward vehicle target detection based on the image target recognition unit. The forward vehicle target detection processing outputs the target vehicle's distance target area in the left eye image; Using the left-eye image as a reference image, pixels in the left-eye image are selected pixel by pixel within the target area at the distance from the vehicle, and the right-eye pixels corresponding to the left-eye pixels are obtained based on the calibration relationship of the binocular cameras. Based on the coordinate difference between the left and right eye pixels in the image coordinates, the disparity value of the pixel is obtained and mapped to the disparity result image.

[0013] In a preferred embodiment, in step S3, if the disparity value of a pixel in the disparity result image is greater than or equal to a preset lower limit disparity threshold, the pixel is marked as a valid disparity pixel; otherwise, the pixel is not marked. The marked pixels are combined into an effective disparity pixel distribution map. The effective disparity pixel distribution map is scanned row by row. In each row of pixels, the length of adjacent effective disparity pixels is counted, and the maximum value is taken as the continuous pixel span. The maximum consecutive pixel span is selected as the pixel connectivity length from all rows of pixels in the target area.

[0014] In a preferred embodiment, in step S4, multiple pixel connectivity lengths within the ranging period are retrieved, arranged in chronological order, and the connectivity lengths of adjacent pixels are subtracted to obtain connectivity difference values. The number of connectivity differences less than zero is counted to obtain a degradation count, and the disparity degradation trend is calculated using the degradation count. Vehicle distance dispersion features are retrieved from the calibration database, and a disparity degradation threshold and a vehicle distance dispersion threshold are preset respectively.

[0015] In a preferred embodiment, in step S4, symbolic analysis is performed based on the radar discrete coefficients and the parallax discrete coefficients, specifically: The ratio of vehicle distance dispersion characteristics to a preset vehicle distance dispersion threshold is used as the radar dispersion coefficient, and the ratio of disparity degradation trend to a preset disparity degradation threshold is used as the disparity dispersion coefficient. The deviation from the dominant difference is obtained by subtracting the radar dispersion coefficient and the parallax dispersion coefficient, and then the deviation from the dominant difference is compared with the preset dominant difference threshold. If the absolute value of the deviation from the dominant difference is greater than the preset dominant difference threshold and the deviation from the dominant difference is positive, it is marked as a radar dominant anomaly symbol, and the radar distance of the test vehicle is calibrated. If the absolute value of the deviation from the dominant difference is greater than the preset dominant difference threshold and the deviation from the dominant difference is negative, it is marked as a visual dominant anomaly symbol, and the radar distance of the test vehicle is fused and corrected. Otherwise, no marking is performed.

[0016] The technical effects and advantages of this invention are as follows: This invention jointly models the spatial distribution and echo energy characteristics of low-altitude point clouds from a lidar system, introducing point cloud echo coefficients and the proportion of low-altitude points to construct a specular reflection state code, and adaptively triggers a self-calibration mechanism based on this code. After self-calibration is triggered, the discrete characteristics of the lidar vehicle distance time series and the degradation trend of binocular vision disparity connectivity are quantitatively analyzed. Symbolic analysis is used to determine the dominant source of ranging anomalies, and offset calibration or fusion correction strategies are selected accordingly. This avoids misjudgments in complex road conditions caused by traditional fixed-weight fusion, forming a closed-loop processing flow from road surface reflection state identification and ranging stability assessment to self-calibration strategy selection. This improves the reliability and engineering adaptability of vehicle distance measurement on slippery road conditions without relying on additional sensors, and achieves real-time anomaly detection and adaptive correction under complex road conditions, effectively reducing ranging errors. Simultaneously, by fusing the complementary characteristics of lidar and binocular vision, the overall ranging accuracy and stability of the calibration device are improved under slippery and highly reflective road conditions. Attached Figure Description

[0017] Figure 1 This is a process stage diagram of a vehicle distance measurement and calibration method based on the fusion of lidar and binocular vision according to the present invention.

[0018] Figure 2 This is a flowchart illustrating the implementation of a vehicle distance measurement and calibration method based on the fusion of lidar and binocular vision according to the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] This invention jointly models the spatial distribution and echo energy characteristics of low-altitude point clouds in lidar systems, introducing point cloud echo coefficients and the proportion of low-altitude points to construct specular reflection state codes, and adaptively triggers a self-calibration mechanism based on these codes. After self-calibration is triggered, the discrete characteristics of the lidar vehicle-distance time series and the degradation trend of binocular visual parallax connectivity are quantitatively analyzed. Symbolic analysis is used to determine the dominant source of ranging anomalies, and offset calibration or fusion correction strategies are selected accordingly. This avoids misjudgments in complex road conditions caused by traditional fixed-weight fusion, forming a closed-loop processing flow from road surface reflection state identification and ranging stability assessment to self-calibration strategy selection.

[0021] Example 1, such as Figures 1 to 2 As shown, a vehicle distance measurement calibration method based on the fusion of lidar and binocular vision includes the following steps: Step S1: While the test vehicle is driving normally, monitor the point cloud data of the lidar on the test vehicle, partition the point cloud data, filter and mark the lidar point cloud, count the number of marked lidar point clouds and detect the echo intensity data of the marked lidar point clouds. Step S2: Calculate the point cloud echo coefficient based on the echo intensity data, analyze the specular reflection state of the radar point cloud in combination with the number of marked radar point clouds, determine whether to trigger the self-calibration mechanism based on the specular reflection state, and set the ranging period after triggering the self-calibration mechanism. Step S3: Scan the radar distance of the test vehicle within the ranging period and perform fluctuation evaluation. Generate distance dispersion features based on the evaluation results and store them in the self-calibration database. Use binocular vision to obtain the distance target area of ​​the test vehicle, identify the disparity pixels in the distance target area and calculate the pixel connectivity length. Step S4: Generate the disparity degradation trend of the test vehicle based on the pixel connectivity length, perform symbolic analysis processing in combination with the vehicle distance dispersion features in the self-calibration database, and select fusion correction or offset calibration for the radar distance of the test vehicle based on the analysis results.

[0022] The specific implementation is as follows: In step S1, while the test vehicle is driving normally, the lidar is activated to scan the environmental space of the test vehicle and collect point cloud data output by the lidar within the current scanning cycle.

[0023] Point cloud data consists of multiple point cloud sampling points. A point cloud sampling point is the smallest spatial measurement unit formed by a lidar at a specific reflection position in the environment during a single scan. Each point cloud sampling point corresponds to the ranging result formed after a laser beam emitted by the lidar is reflected from the target surface in the environment. It is characterized by three-dimensional spatial coordinates in the lidar coordinate system. The three-dimensional spatial coordinates consist of three mutually orthogonal coordinate components, including the longitudinal distance component, the lateral offset component, and the height component. The longitudinal distance component is used to characterize the spatial distance of the point cloud sampling point relative to the lidar in the front-back direction. The lateral offset component is used to characterize the spatial position of the point cloud sampling point relative to the lidar in the left-right direction. The height component is used to characterize the vertical position of the point cloud sampling point relative to the installation height of the lidar.

[0024] The set of point cloud sampling points composed of three-dimensional spatial coordinates is defined as the radar point cloud to describe the environmental spatial structure perceived by the lidar in the current scanning cycle.

[0025] It should be noted that lidar is an active environmental sensing sensor based on the principle of laser ranging. It measures the spatial position of a target by emitting pulsed or continuously modulated laser beams and receiving the echo signals returned after the laser beams interact with external target objects. The lidar coordinate system is a three-dimensional rectangular coordinate system established with the lidar's own installation position as the origin, used to uniformly represent the spatial positional relationship of lidar measurement results.

[0026] After acquiring the radar point cloud, the radar point cloud is partitioned according to the preset spatial partitioning rules. The spatial partitioning rules take the installation position of the lidar as the origin of the coordinates and perform height layering of the radar point cloud according to the height component of the point cloud sampling points. Point cloud sampling points with height components lower than the preset height threshold are assigned to the low height analysis area, and the remaining point cloud sampling points are assigned to the non-analysis area.

[0027] It should be noted that the preset height threshold is set according to the vehicle chassis height. Its physical meaning is that it can cover the space range of the road surface in front of the vehicle and the lower edge of the front chassis. Under the conditions of rain, snow, slippery road surface or water accumulation, when the laser beam illuminates the road surface or the lower edge area of ​​the front chassis at a small incident angle, it is easy to produce specular reflection or low elevation angle false echo. The above abnormalities are spatially manifested as an abnormal increase in the density of point cloud sampling points in the low height area. Selecting point cloud sampling points below the preset height threshold as the analysis object can reflect the abnormal road surface reflection.

[0028] After completing the altitude partitioning, the radar point cloud is filtered and labeled based on the partitioning results. Specifically, point cloud sampling points located in the low altitude analysis area are labeled and combined into labeled radar point clouds, while other point cloud sampling points not falling into the low altitude analysis area are not included in subsequent statistics.

[0029] After marking is completed, the sampling points of the marked radar point cloud are counted to obtain the number of points in the marked radar point cloud. This reflects the density of the distribution of sampling points in the low-altitude analysis area within the current spatial range. The larger the value, the more spatial reflection points the lidar detects in the area near the road surface or the chassis of the vehicle in front, and the higher the probability of abnormal reflections or false echoes in the low-altitude area.

[0030] Further detection of the echo intensity of the sampling points in the marked radar point cloud is performed. The echo intensity is output synchronously by the lidar during the sampling process. Its value is used to characterize the magnitude of the laser signal return energy. The larger the echo intensity value, the stronger the target surface's ability to reflect the laser signal.

[0031] In step S2, after obtaining the marked radar point cloud and its corresponding echo intensity data, the total echo intensity of all point cloud sampling points in the marked radar point cloud is calculated to obtain the marked echo intensity. The echo intensity data corresponding to each point cloud sampling point in the marked radar point cloud are sorted in descending order, and high-energy sampling points with echo intensity within a preset high-intensity range are extracted. The total echo intensity of the high-energy sampling points is calculated to obtain the high-energy echo intensity. The proportion of high-energy echo intensity in the marked echo intensity is calculated to obtain the point cloud echo coefficient of the marked radar point cloud.

[0032] The point cloud echo coefficient ranges from 0 to 1 and is used to quantify the concentration and enhancement of echo energy in low-altitude areas. The larger the value, the more concentrated the laser echo energy is in a small number of point cloud sampling points, the more obvious the peaking characteristics of the echo energy distribution are, and the closer the reflection characteristics are to specular reflection. This indicates a higher probability that the road surface is slippery or has water accumulation.

[0033] It should be noted that the preset high-intensity range is a preset energy cutoff ratio (e.g., 10% to 20%) based on the echo intensity data after descending order, so that point cloud sampling points with relatively high echo intensity are used as high-energy sampling points.

[0034] After calculating the point cloud echo coefficients, the proportion of low-altitude points is further calculated based on the number of points in the marked radar point cloud. Specifically, the proportion of low-altitude points is calculated by comparing the number of points in the marked radar point cloud with the total number of point cloud sampling points acquired by the lidar in the current scanning cycle.

[0035] The percentage of low-altitude points reflects the relative proportion of point cloud distribution in the overall point cloud. The larger the value, the more reflection points the lidar detects in the area near the road surface or the lower edge of the vehicle chassis. This indicates a higher probability of low-elevation reflections or false echoes, and represents the state of slippery road surface or enhanced specular reflection from the perspective of spatial distribution.

[0036] After obtaining the point cloud echo coefficient and the proportion of low-altitude quantities, a hierarchical state determination process is performed based on the point cloud echo coefficient and the proportion of low-altitude quantities, respectively. For the point cloud echo coefficient, it is compared with a preset multi-level echo coefficient threshold. Based on the comparison result, the echo reflection characteristics are divided into multiple echo level states. The echo level states are represented by numerical codes to reflect the changing trend of the echo energy concentration from weak to strong. For the proportion of low-altitude points, it is compared with the preset multi-level proportion threshold. Based on the comparison results, the spatial reflection density is divided into multiple quantity levels. The quantity level is represented by letter codes to reflect the changing trend of low-altitude anomalous point clouds from sparse to dense.

[0037] The levels of both the numerical and alphanumeric codes increase with the increase of the corresponding parameter values. The higher the level, the more significant the road surface slipperiness or specular reflection characteristics.

[0038] After determining the two levels of status, the echo level status and the quantity level status are combined and encoded to generate a specular reflection status code. The specular reflection status code is composed of a numerical code corresponding to an echo level status and a letter code corresponding to a quantity level status. It is used to comprehensively characterize the coupling state of echo energy characteristics and spatial distribution characteristics in the low-altitude region.

[0039] Based on the specular reflection status code, it is determined whether to trigger the self-calibration mechanism. Specifically, the specular reflection status code generated in the current scanning cycle is compared with a preset specular reflection trigger threshold. When the specular reflection status code exceeds the preset threshold, it indicates that the risk of specular reflection interference to the lidar ranging result has reached a preset level, and the specular reflection state of the lidar point cloud is determined to be an abnormal reflection state, at which point the self-calibration mechanism is triggered. When the specular reflection status code does not exceed the preset threshold, it indicates that the current ranging conditions are within a controllable range, and the specular reflection state of the lidar point cloud is determined to be a non-abnormal reflection state, at which point the self-calibration mechanism is not triggered.

[0040] For example, if the specular reflection trigger threshold is set to 3c, when the numerical code corresponding to the echo level state of the point cloud echo coefficient is level 2 and the letter code corresponding to the quantity level state of the low height quantity ratio is level b, the generated specular reflection state code is 2b; when the numerical code corresponding to the echo level state of the point cloud echo coefficient is level 4 and the letter code corresponding to the quantity level state of the low height quantity ratio is level d, the generated specular reflection state code is 4d. In this example, if the specular reflection state code corresponding to 4d exceeds the preset specular reflection trigger threshold, the self-calibration mechanism is triggered, while if 2b does not exceed the specular reflection trigger threshold, the self-calibration mechanism is not triggered.

[0041] It should be noted that the preset multi-level echo coefficient threshold, the preset multi-level quantity ratio threshold, and the preset specular reflection trigger threshold are all set based on the statistical characteristics of the lidar under different road surface reflection conditions. Specifically, the multi-level echo coefficient threshold is obtained through statistical analysis of historical point cloud echo intensity data collected under various working conditions, such as dry road surfaces, wet road surfaces, and roads with significant water accumulation. The point cloud echo coefficients are classified according to their distribution range and quantile locations under different working conditions, ensuring that the echo energy concentration between adjacent levels has a distinguishable statistical difference. The multi-level quantity proportion threshold is set based on the proportion distribution of low-altitude radar point clouds in the total radar point cloud under the same working conditions. By analyzing the statistical trend of the low-altitude quantity proportion changing with road surface conditions, a classification threshold capable of distinguishing the sparse to dense changes in low-altitude abnormal point clouds is determined. The specular reflection trigger threshold is set based on the above two classification thresholds by jointly statistically analyzing the impact of different echo level and quantity level combinations on radar ranging errors. This ensures that the ranging error corresponding to specular reflection state codes exceeding this threshold in historical samples is significantly higher than the controllable error range, thereby ensuring that when the specular reflection state code exceeds the preset specular reflection trigger threshold, the triggering of the self-calibration mechanism has clear statistical basis and quantifiable engineering rationality.

[0042] After triggering the self-calibration mechanism, the ranging period is further set. The ranging period is the time window for subsequent vehicle distance measurement and fusion calibration processing. It is used to limit the time range for centralized sampling and analysis of radar vehicle distance data and binocular vision vehicle distance data during the period when the specular reflection state is in an abnormal reflection state and continues to exist. This provides a unified and controllable time benchmark for subsequent vehicle distance dispersion feature extraction, disparity degradation analysis and fusion calibration decision-making.

[0043] In step S3, during the ranging period, the radar distance of the test vehicle is obtained through the vehicle-mounted lidar ranging interface. The radar distance is the spatial distance between the target vehicle and the test vehicle obtained by the vehicle-mounted lidar performing point cloud ranging calculation on the target vehicle located in the driving direction in front of the test vehicle. The distances between radar vehicles within the ranging period are arranged in chronological order to form a distance time series. The fluctuation of the distance time series is evaluated. Specifically, the median of the distance time series is selected as the median distance value. Using the median distance between vehicles as a reference, the absolute value of the difference between each radar vehicle distance and the median distance is calculated to obtain the distance deviation. The distance deviations are then sorted, and the median is selected as the median distance deviation to reflect the dispersion level of radar vehicle distances within the ranging period. The ratio of the median vehicle distance deviation to the median vehicle distance is used as the vehicle distance dispersion index; The product of the vehicle distance dispersion index and the preset adjustment dispersion coefficient is used as the vehicle distance dispersion feature, which reflects the fluctuation intensity of the radar vehicle distance within the ranging period. The larger the value, the higher the dispersion of the radar vehicle distance within the ranging period and the lower the ranging stability. The vehicle distance dispersion characteristics are stored in a self-calibration database, which is a data storage unit used to store radar vehicle distance dispersion characteristics and road condition information.

[0044] During the ranging period, the road area in the direction of travel ahead of the test vehicle is simultaneously acquired by the binocular camera in the binocular vision system, and the left eye image and the right eye image are obtained respectively. The left eye image is then processed for forward vehicle target detection based on the image target recognition unit. Among them, the forward vehicle target detection processing is used to identify the target vehicle located in the direction of travel in front of the test vehicle in the left eye image, and output the target vehicle distance target area in the left eye image. The pixel coordinate range in the target vehicle distance target area is determined as the coordinate boundary of the target vehicle distance target area, which is used to represent the spatial position of the forward vehicle in the image coordinate system and to limit the spatial range of subsequent binocular parallax analysis. Binocular vision refers to a visual perception method that simultaneously acquires images of the same scene using a left and right eye camera with a fixed baseline spacing, and obtains target distance information based on the pixel position difference of the same target in the left and right images; a binocular camera refers to an imaging device consisting of a left eye camera and a right eye camera with a fixed baseline spacing. Within the target area at vehicle distance, binocular disparity calculation is performed on the left and right images. Specifically, the left image is used as a reference image, and pixels in the left image are selected pixel by pixel within the target area at vehicle distance. Based on the calibration relationship of the binocular camera, a matching search is performed along the epipolar direction in the right image, and the right pixel that is in the same row of pixel coordinates as the left pixel is selected as the corresponding pixel. Based on the coordinate difference between the left pixel and the corresponding right pixel in the image coordinates, the disparity value of the pixel is obtained. The disparity values ​​of each pixel are mapped according to the pixel position to obtain the disparity result image. Within the disparity result image, if the disparity value of a pixel is greater than or equal to a preset lower limit disparity threshold, the pixel is marked as a valid disparity pixel; otherwise, the pixel is not marked. The marked pixels are combined into an effective disparity pixel distribution map. The effective disparity pixel distribution map is scanned row by row. In each row of pixels, the length of adjacent effective disparity pixels is counted, and the maximum value is taken as the continuous pixel span. The maximum consecutive pixel span is selected from all rows of pixels in the vehicle distance target area as the pixel connectivity length, which is used to reflect the continuity of effective disparity pixels in the disparity direction within the vehicle distance target area.

[0045] It should be noted that the vehicle-mounted LiDAR ranging interface is used to receive target point cloud data or target distance calculation results output by the vehicle-mounted LiDAR during operation; the preset adjustable dispersion coefficient can be set according to the vehicle speed range, LiDAR sampling frequency, and LiDAR range configuration; the image target recognition unit is an image processing function unit used to perform target recognition processing on the acquired left-eye image and output the corresponding vehicle target area; the preset lower limit parallax threshold can be set according to the expected maximum vehicle distance range corresponding to the vehicle distance target area.

[0046] In step S4, the connectivity lengths of multiple pixels within the ranging period are retrieved and arranged in chronological order. The connectivity difference is obtained by subtracting the connectivity lengths of adjacent pixels. The number of connectivity differences less than zero is counted to obtain the degradation count. The ratio of the degradation count to the number of times the connectivity lengths of adjacent pixels are subtracted is used as the disparity degradation trend. The larger the value, the worse the continuity of effective disparity pixels in the target area of ​​the vehicle distance. Vehicle distance dispersion features are retrieved from the calibration database, and a disparity degradation threshold and a vehicle distance dispersion threshold are preset respectively. Symbolic analysis is performed based on radar discrete coefficients and parallax discrete coefficients, specifically: The ratio of vehicle distance dispersion characteristics to a preset vehicle distance dispersion threshold is used as the radar dispersion coefficient, and the ratio of disparity degradation trend to a preset disparity degradation threshold is used as the disparity dispersion coefficient. The deviation from the dominant value is obtained by subtracting the radar dispersion coefficient and the parallax dispersion coefficient. This deviation reflects the relative dominance between radar ranging anomalies and parallax degradation anomalies within the current ranging cycle. The larger the absolute value of the deviation from the dominant value, the more obvious the dominant relationship.

[0047] The deviation from the dominant difference is compared with the preset dominant difference threshold. If the absolute value of the deviation from the dominant difference is greater than the preset dominant difference threshold and the deviation from the dominant difference is positive, it indicates that the lidar has a ranging offset anomaly in the current ranging cycle, and it is marked as a lidar dominant anomaly symbol. If the absolute value of the deviation from the dominant difference is greater than the preset dominant difference threshold and the deviation from the dominant difference is negative, it indicates that there is a disparity degradation abnormality in binocular vision within the current ranging period, and it is marked as a visual dominance abnormality symbol. Otherwise, no marking is performed.

[0048] When the anomaly symbol is marked as radar-dominant, select to perform offset calibration on the radar distance of the test vehicle; when the anomaly symbol is marked as vision-dominant, select to perform fusion correction on the radar distance of the test vehicle. Among them, fusion correction refers to reducing the proportion of vehicle distance involved in the binocular vision in the fusion calculation, using the vehicle distance result output by the lidar as a reference, suppressing the vehicle distance information output by the binocular vision, and performing weighted fusion of the radar vehicle distance and the visual vehicle distance based on the preset fusion weight to output the vehicle distance result; offset calibration refers to using the binocular vision vehicle distance as a constraint benchmark to perform overall offset compensation on the lidar vehicle distance in order to correct the systematic distance drift generated by the radar ranging under the current operating conditions.

[0049] It should be noted that the preset parallax degradation threshold can be set based on the parallax connectivity statistics of the binocular camera under historical normal road conditions; the preset vehicle distance dispersion threshold can be set based on the vehicle distance fluctuation statistics of the lidar under historical stable driving conditions; and the preset dominant difference threshold can be set based on the relative difference distribution between the radar dispersion coefficient and the parallax dispersion coefficient under historical normal operating conditions.

[0050] This invention addresses the problems of abnormal echoes and low-altitude false point clouds generated when the lidar used to scan targets in front of the test vehicle's lane during the calibration of highway vehicle distance capture systems under conditions of rain, snow, slippery surfaces, water accumulation, or high reflectivity, as well as the poor ranging accuracy caused by binocular visual parallax degradation. It establishes a closed-loop self-calibration mechanism for road surface reflection state recognition and accurate vehicle distance measurement, achieving adaptive correction of radar vehicle distance deviation and visual ranging anomalies. This effectively improves the measurement accuracy and stability of the calibration device under complex working conditions, solving the technical problems of decreased vehicle distance calibration accuracy and insufficient ranging stability in existing technologies.

[0051] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, in the form of a computer program product.

[0052] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and inventive constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0053] In addition, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0054] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0055] Finally, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A vehicle distance measurement and calibration method based on the fusion of lidar and binocular vision, characterized in that: Includes the following steps: Step S1: While the test vehicle is driving normally, monitor the point cloud data of the lidar on the test vehicle, partition the point cloud data, filter and mark the lidar point cloud, count the number of marked lidar point clouds and detect the echo intensity data of the marked lidar point clouds. Step S2: Calculate the point cloud echo coefficient based on the echo intensity data, analyze the specular reflection state of the radar point cloud in combination with the number of marked radar point clouds, determine whether to trigger the self-calibration mechanism based on the specular reflection state, and set the ranging period after triggering the self-calibration mechanism. Step S3: Scan the radar distance of the test vehicle within the ranging period and perform fluctuation evaluation. Generate distance dispersion features based on the evaluation results and store them in the self-calibration database. Use binocular vision to obtain the distance target area of ​​the test vehicle, identify the disparity pixels in the distance target area and calculate the pixel connectivity length. Step S4: Generate the disparity degradation trend of the test vehicle based on the pixel connectivity length, perform symbolic analysis processing in combination with the vehicle distance dispersion features in the self-calibration database, and select fusion correction or offset calibration for the radar distance of the test vehicle based on the analysis results.

2. The vehicle distance measurement and calibration method based on the fusion of lidar and binocular vision according to claim 1, characterized in that: In step S1, when the test vehicle is driving normally, the lidar is activated to scan the environmental space of the test vehicle and collect point cloud data output by the lidar within the current scanning cycle; Point cloud data consists of multiple point cloud sampling points, which are characterized by three-dimensional spatial coordinates in the lidar coordinate system. The three-dimensional spatial coordinates consist of three mutually orthogonal coordinate components, including the longitudinal distance component, the lateral offset component, and the height component. The set of point cloud sampling points composed of three-dimensional spatial coordinates is defined as radar point cloud; After acquiring the radar point cloud, the radar point cloud is partitioned according to the preset spatial partitioning rules. The spatial partitioning rules classify point cloud sampling points with height components below the preset height threshold into the low-height analysis area, otherwise they are classified into the non-analysis area.

3. The vehicle distance measurement and calibration method based on the fusion of lidar and binocular vision according to claim 2, characterized in that: In step S1, point cloud sampling points located in the low-altitude analysis area are marked and combined into a marked radar point cloud. Point cloud sampling points that do not fall into the low-altitude analysis area are not included in subsequent statistics. The number of points in the marked radar point cloud is obtained by statistically analyzing the point cloud sampling points. The echo intensity of the sampling points in the marked radar point cloud is detected. The echo intensity is output synchronously by the lidar during the sampling process, which characterizes the amount of energy returned by the laser signal.

4. The vehicle distance measurement and calibration method based on the fusion of lidar and binocular vision according to claim 3, characterized in that: In step S2, the sum of the echo intensities of all point cloud sampling points within the marked radar point cloud is calculated to obtain the marked echo intensity; The echo intensity data corresponding to each point cloud sampling point in the marked radar point cloud are sorted in descending order, and high-energy sampling points with echo intensity within a preset high-intensity range are extracted. The high-energy echo intensity is obtained by summing the echo intensities of the high-energy sampling points, and the point cloud echo coefficient of the marked radar point cloud is obtained by calculating the proportion of the high-energy echo intensity in the marked echo intensity. The ratio of the number of points in the marked radar point cloud to the total number of point cloud sampling points in the radar point cloud acquired by the lidar during the current scanning cycle is calculated to obtain the proportion of low-altitude points.

5. The vehicle distance measurement and calibration method based on the fusion of lidar and binocular vision according to claim 4, characterized in that: In step S2, the point cloud echo coefficients are compared with preset multi-level echo coefficient thresholds to classify multiple echo level states and represent them with numerical codes. The proportion of low-height quantities is compared with the preset multi-level quantity proportion thresholds to divide the quantity into multiple quantity levels and represent them with letter codes; The echo level status and the quantity level status are combined and encoded to generate a specular reflection status code, which is composed of a combination of numeric and alphanumeric codes. The specular reflection status code is compared with the preset specular reflection trigger threshold. When the specular reflection status code exceeds the preset specular reflection trigger threshold, the specular reflection status of the radar point cloud is determined to be an abnormal reflection status, and the self-calibration mechanism is triggered at this time. When the specular reflection status code does not exceed the preset specular reflection trigger threshold, the specular reflection status of the radar point cloud is determined to be a non-abnormal reflection status, and the self-calibration mechanism is not triggered at this time. After triggering the self-calibration mechanism, the ranging period is further set, which is a time window that limits the time range for subsequent acquisition of radar vehicle distance.

6. The vehicle distance measurement and calibration method based on the fusion of lidar and binocular vision according to claim 1, characterized in that: In step S3, during the ranging period, the radar distance of the test vehicle is obtained through the vehicle-mounted lidar ranging interface, and the radar distances of each vehicle during the ranging period are arranged in chronological order to form a vehicle distance time series. The fluctuation of the vehicle distance time series is evaluated. Specifically, the median of the vehicle distance time series is selected as the median value of the vehicle distance. The absolute value of the difference between each radar vehicle distance and the median value of the vehicle distance is calculated to obtain the vehicle distance deviation. The vehicle distance deviations are sorted and the median value of the vehicle distance deviation is selected as the median value of the vehicle distance deviation. The ratio of the median vehicle distance deviation to the median vehicle distance is used as the vehicle distance dispersion index; The product of the vehicle distance dispersion index and the preset adjustable dispersion coefficient is used as the vehicle distance dispersion feature.

7. The vehicle distance measurement and calibration method based on the fusion of lidar and binocular vision according to claim 1, characterized in that: In step S3, during the ranging period, the road area in the direction of travel in front of the test vehicle is simultaneously acquired by the binocular camera in the binocular vision system, and the left eye image and the right eye image are obtained respectively. The left eye image is processed for forward vehicle target detection based on the image target recognition unit. The forward vehicle target detection processing outputs the target vehicle's distance target area in the left eye image; Using the left-eye image as a reference image, pixels in the left-eye image are selected pixel by pixel within the target area at the distance from the vehicle, and the right-eye pixels corresponding to the left-eye pixels are obtained based on the calibration relationship of the binocular cameras. Based on the coordinate difference between the left and right eye pixels in the image coordinates, the disparity value of the pixel is obtained and mapped to the disparity result image.

8. The vehicle distance measurement and calibration method based on the fusion of lidar and binocular vision according to claim 7, characterized in that: In step S3, within the disparity result image, if the disparity value of a pixel is greater than or equal to a preset lower limit disparity threshold, the pixel is marked as a valid disparity pixel; otherwise, the pixel is not marked. The marked pixels are combined into an effective disparity pixel distribution map. The effective disparity pixel distribution map is scanned row by row. In each row of pixels, the length of adjacent effective disparity pixels is counted, and the maximum value is taken as the continuous pixel span. The maximum consecutive pixel span is selected as the pixel connectivity length from all rows of pixels in the target area.

9. The vehicle distance measurement and calibration method based on the fusion of lidar and binocular vision according to claim 8, characterized in that: In step S4, the connectivity lengths of multiple pixels within the ranging period are retrieved, arranged in chronological order, and the connectivity difference between adjacent pixel connectivity lengths is obtained. The number of connectivity differences less than zero is counted to obtain a degradation count, and the disparity degradation trend is calculated using the degradation count. Vehicle distance dispersion features are retrieved from the calibration database, and a disparity degradation threshold and a vehicle distance dispersion threshold are preset respectively.

10. The vehicle distance measurement and calibration method based on the fusion of lidar and binocular vision according to claim 9, characterized in that: In step S4, symbolic analysis is performed based on the radar discrete coefficients and parallax discrete coefficients, specifically: The ratio of vehicle distance dispersion characteristics to a preset vehicle distance dispersion threshold is used as the radar dispersion coefficient, and the ratio of disparity degradation trend to a preset disparity degradation threshold is used as the disparity dispersion coefficient. The deviation from the dominant difference is obtained by subtracting the radar dispersion coefficient and the parallax dispersion coefficient, and then the deviation from the dominant difference is compared with the preset dominant difference threshold. If the absolute value of the deviation from the dominant difference is greater than the preset dominant difference threshold and the deviation from the dominant difference is positive, it is marked as a radar dominant anomaly symbol, and the radar distance of the test vehicle is calibrated. If the absolute value of the deviation from the dominant difference is greater than the preset dominant difference threshold and the deviation from the dominant difference is negative, it is marked as a visual dominant anomaly symbol, and the radar distance of the test vehicle is fused and corrected. Otherwise, no marking is performed.

Images