An obstacle recognition device based on laser point cloud data

Abstract

The present invention discloses an obstacle recognition device based on laser point cloud data, which belongs to the field of recognition and detection technology. The device comprises a laser radar confidence analysis module, which obtains the laser radar confidence based on weather visibility and temperature in environmental data, vibration intensity in operating environment data and laser point cloud data; a camera confidence analysis module, which obtains the camera confidence based on illumination intensity in environmental data, vibration intensity in operating environment data and camera data; a fusion weight allocation module, which allocates dynamic fusion weights based on the laser radar confidence and camera confidence; a fusion position analysis module, which constructs a fusion position model based on the dynamic fusion weights and outputs the fusion position according to the recognition results of the laser radar and the camera for the obstacle; the present invention adjusts the fusion weights in real time through dynamic analysis of the confidence of the laser radar and the camera, thereby significantly improving the obstacle recognition capability in complex environments.

Description

Technical Field

[0001] The present invention belongs to the field of identification and detection technology, and in particular relates to an obstacle identification device based on laser point cloud data. Background Art

[0002] In existing obstacle recognition technologies, the performance of single sensors (such as lidar or cameras) often degrades due to environmental constraints. For example, when operating lidar in inclement weather (such as low visibility or high temperatures) or when the device is vibrating, point cloud data is susceptible to noise, affecting detection accuracy. Cameras experience significant degradation in image contrast and clarity in low-light conditions, high vibration, or haze, leading to misjudgments or missed detections. Traditional multi-sensor fusion methods often employ fixed weight allocation strategies, which are unable to dynamically adapt to complex environmental changes and lack robustness.

[0003] Therefore, there is an urgent need for an obstacle recognition device that can evaluate sensor confidence in real time and dynamically adjust fusion weights to improve detection accuracy and reliability in complex environments. Summary of the Invention

[0004] In view of the shortcomings of the existing technology, the present invention provides an obstacle recognition device based on laser point cloud data to solve the above problems.

[0005] To achieve the above objectives, the present invention is implemented through the following technical solutions: an obstacle recognition device based on laser point cloud data, comprising:

[0006] Data acquisition module, used to obtain environmental data, operating environment data, laser point cloud data and camera data;

[0007] The LiDAR confidence analysis module obtains LiDAR confidence based on weather visibility and temperature in environmental data, vibration intensity in operating environment data, and laser point cloud data;

[0008] The camera confidence analysis module obtains the camera confidence based on the light intensity in the environmental data, the vibration intensity in the operating environment data, and the camera data;

[0009] A fusion weight allocation module allocates dynamic fusion weights based on the lidar confidence and the camera confidence;

[0010] The fusion position analysis module constructs a fusion position model based on the dynamic fusion weight and outputs the fusion position according to the recognition results of the laser radar and the camera on the obstacle.

[0011] On the basis of the above technical solutions, the present invention also provides the following optional technical solutions:

[0012] Further technical solutions: The environmental data includes weather visibility, temperature and light intensity; the operating environment data includes the vibration intensity of the laser radar and the carrier on which the camera is located; the laser point cloud data includes the noise ratio, effective reflectivity and point cloud density index; the point cloud density index is the ratio of the current point cloud density to the nominal maximum density; the camera data includes image contrast, brightness gradient and clarity after defogging.

[0013] Further technical solution: The specific working principle of the lidar confidence analysis module is:

[0014] Obtain the lidar environmental penalty index based on the current vibration intensity, current temperature, and current weather visibility;

[0015] The lidar environment penalty index, noise ratio, effective reflectivity, and point cloud density index are imported into the constructed lidar confidence model to output the lidar confidence. The lidar confidence model is expressed as:

[0016]

[0017] in, represents the lidar confidence, represents the point cloud density index, represents the noise ratio, represents the effective reflectivity, represents the environmental penalty index, represents the weight coefficient and , is the lidar environment penalty factor, Indicates the visibility deviation value, Indicates the strength loss value, Indicates the temperature deviation value.

[0018] Further technical solution: The specific working principle of the camera confidence analysis module is:

[0019] Obtain the camera environment penalty index based on the current vibration index and light intensity;

[0020] The image contrast, brightness gradient, and clarity after defogging are normalized to the minimum and maximum values ​​and then imported into the camera confidence model constructed with the camera environment penalty index to output the camera confidence. The camera confidence model is expressed as:

[0021]

[0022] in, Indicates the camera confidence, represents the contrast of the normalized image, represents the brightness gradient after normalization, Indicates the clarity after defogging after normalization, represents the camera environment penalty index, represents the camera environment penalty factor, represents the weight coefficient and , Indicates the strength loss value, Indicates the lighting deviation value.

[0023] Further technical solution: The specific working principle of the fusion weight distribution module is: import the camera confidence and the lidar confidence into the constructed weight distribution model to obtain the lidar weight coefficient and the camera weight coefficient. The weight distribution model is expressed as:

[0024]

[0025] in, Represents the weight coefficient of the laser radar recognition result, Indicates the weight coefficient of the camera recognition result, Indicates the startup function, represents the lidar confidence, Indicates the camera confidence, represents the sensitivity index, Indicates confidence, Represents the confidence threshold.

[0026] Further technical solution: The specific working principle of the fusion position analysis module is to import the weight coefficient of the laser radar recognition result, the weight coefficient of the camera recognition result, the laser radar's recognition coordinates of the obstacle, and the camera's recognition coordinates of the obstacle into the constructed fusion position model to output the fusion position. The fusion position model is expressed as:

[0027]

[0028] in, Indicates the fusion position, Indicates the laser radar's recognition coordinates of the obstacle, Indicates the camera's recognition coordinates of obstacles. Represents the weight coefficient of the laser radar recognition result, Indicates the weight coefficient of the camera recognition result.

[0029] Further technical solution: Obtain the lidar environmental penalty index based on the current vibration intensity, current temperature, and current weather visibility, specifically:

[0030] The current weather visibility is subtracted from the nominal maximum visibility and then the difference is compared with the nominal maximum visibility to obtain the weather visibility deviation value;

[0031] Ratio the current vibration intensity to the nominal maximum vibration intensity to obtain the intensity loss value;

[0032] The current temperature is processed with the optimal temperature, and the difference between the current temperature and the optimal temperature is processed, and the difference between the nominal maximum temperature and the nominal minimum temperature is processed, and the obtained ratio is processed with the absolute value to obtain the temperature deviation value;

[0033] The visibility deviation value, intensity loss value and temperature deviation value are weighted averaged to obtain the lidar environment penalty index.

[0034] Further technical solution: Obtain the camera environment penalty index based on the current vibration index and light intensity, specifically:

[0035] Ratio the current vibration intensity to the nominal maximum vibration intensity to obtain the intensity loss value;

[0036] The absolute value of the difference between the current light intensity and the camera's optimal light intensity is compared with the absolute value of the difference between the maximum light intensity and the minimum light intensity supported by the camera to obtain the light deviation value.

[0037] The obtained intensity loss value and the light intensity deviation value are weighted averaged to obtain the camera environment penalty index.

[0038] The present invention provides an obstacle recognition device based on laser point cloud data, which has the following advantages compared with the prior art:

[0039] 1. The present invention dynamically analyzes the confidence of the lidar and camera (such as weather visibility, temperature, vibration, lighting and other parameters) to adjust the fusion weight in real time, significantly improving the obstacle recognition ability in complex environments. At the same time, the weighted average-based fusion position model combines the geometric accuracy of the laser point cloud data with the rich texture information of the camera image to output a more accurate obstacle position. By introducing the lidar confidence model and the camera confidence model, the impact of environmental factors on sensor performance can be quantified to ensure the rationality of weight distribution. The confidence can be dynamically corrected through environmental penalty indexes (such as vibration and temperature deviation values), reducing the misjudgment rate under extreme conditions and enhancing the stability and reliability of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0042] The specific implementation of the present invention is described in detail below with reference to specific embodiments.

[0043] See also Figure 1 , provided in one embodiment of the present invention, is an obstacle recognition device based on laser point cloud data, comprising:

[0044] Data acquisition module, used to obtain environmental data, operating environment data, laser point cloud data and camera data;

[0045] The LiDAR confidence analysis module obtains LiDAR confidence based on weather visibility and temperature in environmental data, vibration intensity in operating environment data, and laser point cloud data;

[0046] The camera confidence analysis module obtains the camera confidence based on the light intensity in the environmental data, the vibration intensity in the operating environment data, and the camera data;

[0047] A fusion weight allocation module allocates dynamic fusion weights based on the lidar confidence and the camera confidence;

[0048] A fusion position analysis module, which constructs a fusion position model based on the dynamic fusion weight and outputs a fusion position according to the obstacle recognition results of the laser radar and the camera;

[0049] The environmental data includes weather visibility, temperature and light intensity, and the operating environment data includes the vibration intensity of the carrier where the laser radar and camera are located;

[0050] The laser point cloud data includes noise ratio, effective reflectivity and point cloud density index, where the point cloud density index is the ratio of the current point cloud density to the nominal maximum density;

[0051] The camera data includes image contrast, brightness gradient, and clarity after defogging;

[0052] Specifically, the data acquisition module first collects environmental data such as weather visibility, temperature, and light intensity; operating environment data such as vibration intensity; the noise ratio, effective reflectivity, and point cloud density index from the lidar output; and image contrast, brightness gradient, and dehazing clarity from the camera output. The lidar confidence analysis module calculates an environmental penalty index based on ambient temperature, visibility, and vibration intensity. Combined with point cloud quality parameters, it outputs the lidar confidence score using a mathematical model. The camera confidence analysis module simultaneously calculates illumination deviation and vibration intensity loss values, and combines the normalized image quality parameters to generate the camera confidence score. The fusion weight allocation module compares the two confidence levels. If a sensor's confidence falls below a threshold, its weight is reduced; otherwise, its weight is increased, forming a dynamic allocation strategy. The fusion position analysis module performs a weighted average of the lidar and camera coordinates based on the weight coefficients, outputting optimized obstacle position data.

[0053] Compared with existing technologies, existing methods use fixed-weight fusion strategies that are unable to cope with the dynamic degradation of sensor performance. For example, when strong vibration causes blurring of the camera image, traditional methods still assign a fixed weight to the camera, resulting in increased errors in the fusion result. This solution dynamically adjusts weights through real-time confidence assessment, automatically reducing their weight proportion when sensor performance degrades. For example, when the camera's confidence decreases due to vibration, the system automatically increases the lidar weight to ensure the stability of the fusion result.

[0054] Through the above technical solution, this application can maintain obstacle detection accuracy despite complex environmental changes. For example, when heavy rain causes the density of the lidar point cloud to decrease, the confidence analysis module automatically reduces its weight and relies on camera data for compensation. When strong light causes the camera to overexpose, the system increases the lidar weight to avoid detection errors caused by single sensor failure. This solution effectively solves the problem of reduced detection reliability caused by traditional methods due to insufficient environmental adaptability.

[0055] Secondly, a multidimensional environmental status evaluation system was established by defining weather visibility, temperature, and light intensity as core parameters of environmental data, corresponding to the laser radar's beam attenuation characteristics, sensor temperature stability, and camera sensitivity, respectively. Vibration intensity was used as a key indicator of operating environment data, directly linking the laser radar point cloud jitter error and camera image blur, addressing the drawback of existing technologies that fail to consider the impact of mechanical vibration. The noise ratio, effective reflectivity, and point cloud density index were introduced into the laser point cloud data. The noise ratio reflects the degree of signal interference, the effective reflectivity indicates the reliability of object recognition, and the point cloud density index quantifies data integrity by comparing the actual density to the nominal density. Combined, these three metrics accurately assess the quality of the laser radar data. Image contrast, brightness gradient, and post-dehazing clarity were defined for the camera data, corresponding to image feature discernibility, illumination uniformity compensation capability, and haze interference removal effectiveness, respectively. These metrics provide multidimensional quantitative indicators for camera operating status assessment. The systematic construction of this parameter system provides an accurate input data foundation for the subsequent confidence model, enabling dynamic fusion weight allocation to accurately reflect the actual operating status of each sensor in complex environments.

[0056] Compared with existing technologies, traditional methods usually only use a single environmental parameter to evaluate sensor performance. For example, judging the camera status only by light intensity, or evaluating the performance of lidar only by visibility, resulting in insufficient environmental adaptability. In existing technologies, laser point cloud data often only focuses on the absolute value of point cloud density, without considering the impact of device nominal performance differences on data quality assessment. The introduction of the point cloud density index realizes the normalized evaluation of device performance. Existing camera data analysis mostly relies on the original image clarity index, without considering the image quality restoration effect of the defogging algorithm, resulting in inaccurate assessment in hazy environments. Existing motion environment assessments often ignore the joint impact of vibration intensity on lidar and cameras. This solution realizes cross-sensor stability assessment through the unified definition of vibration intensity parameters. In existing technologies, there is a lack of systematic association between environmental parameters and sensor data parameters, resulting in a lack of data support for dynamic fusion weight allocation. This solution provides structured input for the fusion algorithm by establishing a multi-dimensional parameter system.

[0057] Through the above technical solutions, this application solves the problems of poor environmental adaptability and low fusion accuracy caused by incomplete parameter definition in traditional obstacle recognition technology. By refining the sensor data parameter types, it is possible to accurately quantify the degree of signal attenuation of the lidar under complex meteorological conditions, effectively identify the laser performance fluctuations caused by high temperature environments, and accurately evaluate the impact of vibration on point cloud stability. By defining the clarity parameters after defogging, the accuracy of the camera's state assessment in severe weather such as haze is significantly improved. By introducing the point cloud density index, the performance normalization evaluation of lidar equipment of different models is achieved, avoiding misjudgments caused by equipment differences in traditional methods. By systematically constructing laser point cloud quality parameters such as noise ratio and effective reflectivity, performance degradation caused by sensor aging or contamination can be discovered in a timely manner. Ultimately, it provides an accurate and reliable data foundation for the dynamic fusion of multiple sensors, significantly improving the accuracy of obstacle detection in complex environments.

[0058] Preferably, the specific working principle of the lidar confidence analysis module is:

[0059] Obtain the lidar environmental penalty index based on the current vibration intensity, current temperature, and current weather visibility, specifically:

[0060] The current weather visibility is subtracted from the nominal maximum visibility and then the difference is compared with the nominal maximum visibility to obtain the weather visibility deviation value;

[0061] Ratio the current vibration intensity to the nominal maximum vibration intensity to obtain the intensity loss value;

[0062] The current temperature is processed with the optimal temperature, and the difference between the current temperature and the optimal temperature is processed, and the difference between the nominal maximum temperature and the nominal minimum temperature is processed, and the obtained ratio is processed with the absolute value to obtain the temperature deviation value;

[0063] Perform weighted averaging on the visibility deviation value, intensity loss value, and temperature deviation value to obtain the lidar environment penalty index;

[0064] The laser radar environment penalty index is expressed as:

[0065]

[0066] in, represents the lidar environment penalty index, Indicates the visibility deviation value, Indicates the strength loss value, Indicates the temperature deviation value, is the weight coefficient and ;

[0067] Import the lidar environment penalty index, noise ratio, effective reflectivity, and point cloud density index into the constructed lidar confidence model to output the lidar confidence;

[0068] The lidar confidence model is expressed as:

[0069]

[0070] in, represents the lidar confidence, represents the point cloud density index, represents the noise ratio, represents the effective reflectivity, represents the environmental penalty index, represents the weight coefficient and , is the lidar environment penalty factor, Indicates the visibility deviation value, Indicates the strength loss value, Indicates the temperature deviation value.

[0071] Specifically, the calculation of weather visibility deviation uses the nominal maximum visibility as a reference. For example, for a device with a nominal maximum visibility of 1000 meters, if the measured visibility drops to 200 meters in haze weather, the deviation reaches 0.8, directly indicating the degree of laser beam scattering loss. The intensity loss value is calculated as the ratio of vibration intensity to the device's upper tolerance limit. For example, if the device's nominal maximum vibration intensity is 5g and the measured vibration intensity reaches 3g, the loss value is 0.6, reflecting the risk of mechanical stability degradation of the laser transmitter and receiver. The temperature deviation value is calculated by dividing the difference between the actual temperature and the optimal operating temperature (e.g., 25°C) by the device's allowable operating temperature range (e.g., an 80°C span from -20°C to 60°C), achieving uniform quantification under different temperature conditions. In the weighted averaging process, for example, a weight of 0.5 is set for visibility deviation, 0.3 for intensity loss, and 0.2 for temperature deviation to form a comprehensive environmental penalty index. This index ranges from 0 to 1, with higher values ​​indicating more severe environmental interference.

[0072] Compared with existing technologies, traditional methods typically monitor only a single environmental parameter or use fixed thresholds for judgment, such as determining equipment failure based solely on exceeding vibration intensity standards. This solution utilizes multi-dimensional parameter normalization and a dynamic weighting mechanism to account for the compatibility of parameters of different physical dimensions while also reflecting the varying impacts of various environmental factors on lidar performance through weight allocation. For example, in dusty weather, the weight of visibility deviation is automatically increased, while in high-temperature environments, the weight of temperature deviation is increased, forming an adaptive environmental assessment model.

[0073] Through the above technical solution, this application effectively solves the problem of LiDAR performance degradation caused by the coupling of multiple environmental factors such as vibration, temperature, and visibility. By establishing a standardized deviation value and a weighted fusion mechanism, a quantitative assessment of complex environmental interference is achieved, providing accurate input parameters for subsequent confidence calculations. For example, when there is equipment vibration and a low temperature environment at the same time, this solution can accurately distinguish the combined effects of mechanical offset caused by vibration and the decrease in component sensitivity caused by low temperature, avoiding the problem of one-sided environmental factor assessment in traditional methods.

[0074] Furthermore, real-time vibration intensity, temperature, and weather visibility data are collected and processed through the aforementioned steps, and then a weighted average is used to generate an environmental penalty index. This index, along with the point cloud density index, noise ratio, and effective reflectivity, is then substituted into the confidence model. Weight coefficients are used to adjust the contribution of data quality to environmental interference, ultimately outputting a dynamically adjusted lidar confidence score.

[0075] Compared with existing technologies, which typically assess data quality based solely on a single environmental factor, such as relying solely on point cloud density or reflectivity, while ignoring the coupled influence of multiple factors such as vibration and temperature, this solution introduces an environmental penalty index to transform multidimensional environmental parameters into a unified quantitative metric. This, combined with the noise ratio and point cloud density index, enables a comprehensive assessment of lidar data reliability. Furthermore, while existing technologies typically assess the impact of environmental parameters using fixed thresholds, this solution, through weighted calculations and a dynamic attenuation factor, can more precisely reflect the continuous impact of environmental changes.

[0076] Through the above technical solution, this application can quantify the performance degradation of lidar in complex environments in real time, effectively suppressing the problem of reduced reliability of point cloud data caused by vibration, temperature fluctuations or low visibility. For example, in high temperature or strong vibration scenarios, an increase in the environmental penalty index will reduce the weight of the corresponding item in the confidence model, thereby reducing the impact of noise data and outliers on obstacle recognition. At the same time, the product term of the point cloud density index and the effective reflectivity ensures that high-quality data is prioritized when environmental conditions are stable, ultimately improving the accuracy and robustness of obstacle position recognition.

[0077] Preferably, the specific working principle of the camera confidence analysis module is:

[0078] The camera environment penalty index is obtained based on the current vibration index and light intensity, specifically:

[0079] Ratio the current vibration intensity to the nominal maximum vibration intensity to obtain the intensity loss value;

[0080] The absolute value of the difference between the current light intensity and the camera's optimal light intensity is compared with the absolute value of the difference between the maximum light intensity and the minimum light intensity supported by the camera to obtain the light deviation value.

[0081] The obtained intensity loss value and the light intensity deviation value are weighted averaged to obtain a camera environment penalty index, which is expressed as:

[0082]

[0083] in, represents the camera environment penalty index, Indicates the illumination deviation value, Indicates the strength loss value, represents the weight coefficient and ;

[0084] The image contrast, brightness gradient, and clarity after defogging are normalized to the minimum and maximum values, and then the camera environment penalty index is imported into the constructed camera confidence model to output the camera confidence.

[0085] The camera confidence model is expressed as:

[0086]

[0087] in, Indicates the camera confidence, represents the contrast of the normalized image, represents the brightness gradient after normalization, Indicates the clarity after defogging after normalization, represents the camera environment penalty index, represents the camera environment penalty factor, represents the weight coefficient and , Indicates the illumination deviation value, Indicates the strength loss value.

[0088] Specifically, during vibration intensity analysis, when the vehicle is traveling on bumpy roads, the accelerometer collects vibration data in real time. When the vibration intensity reaches 0.3 as a percentage of the nominal maximum value, it indicates a 30% loss in camera stability. When evaluating lighting conditions, if the camera's optimal operating illumination is 2000-5000 lux and the current ambient illumination is 800 lux, the illumination deviation value of 0.9, calculated as (|800-3500|) / (5000-2000), indicates a significant deviation from the ideal operating range. These two parameters are weighted using coefficients of 0.6 and 0.4, respectively. The resulting environmental penalty index is used to reduce the confidence weight of the camera in the fusion decision.

[0089] Compared to existing technologies, traditional methods typically only detect a single environmental factor or use fixed thresholds to determine camera availability. For example, this approach uses a binary judgment based solely on whether the light intensity exceeds the sensor's range. This approach, however, quantifies the combined effects of vibration and light to construct a continuous penalty index, enabling a more refined characterization of the degree of environmental interference. Existing technologies lack modeling of the synergistic effects of mechanical vibration and optical conditions. This approach establishes a coupled analysis mechanism for multidimensional environmental parameters by weightedly integrating the degree of deviation between the two.

[0090] Through the above technical solution, this application can effectively identify the performance degradation state of the camera under strong vibration or abnormal lighting conditions, and provide a dynamic adjustment basis for multi-sensor fusion by generating a quantitative environmental penalty index. When the vehicle passes over a speed bump and causes high-frequency vibration, the system automatically reduces the camera weight to avoid positioning errors caused by image blur; in the case of sudden changes in lighting at the entrance and exit of a tunnel, the fusion strategy is adjusted through real-time updated lighting deviation values ​​to prevent misjudgments caused by overexposure or low-light images. This confidence assessment mechanism based on real-time perception of environmental parameters significantly improves the adaptability and reliability of the obstacle detection system under complex working conditions.

[0091] Furthermore, the camera environment penalty index is generated by collecting vibration intensity and light intensity data in real time, calculating the intensity loss value and light deviation value respectively, and then taking a weighted sum. This process converts image blur caused by mechanical vibration and color distortion caused by abnormal lighting into quantifiable environmental interference indicators. Image contrast, brightness gradient, and clarity after dehazing are normalized and converted into standardized parameters with a unified dimension to ensure that each quality dimension is comparable in the confidence model. In the confidence model calculation process, the product term of the camera environment penalty factor and the environmental penalty index is used as a penalty term to dynamically adjust the attenuation of environmental interference on the final confidence, while the weight coefficient is used to balance the contribution ratio of each dimension of image quality. The final output is a confidence assessment result that comprehensively reflects the working status of the camera and image quality.

[0092] Compared to existing technologies, traditional methods typically judge camera status based solely on light intensity or vibration thresholds, failing to establish a dynamic correlation between multidimensional environmental parameters and image quality metrics. This solution innovatively transforms the combined environmental interference of vibration and light into a computable penalty index. This, combined with multidimensional normalization of image quality parameters, forms a dynamic confidence assessment mechanism, effectively overcoming the limitations of separate analysis of environmental factors and image features.

[0093] Through the above technical solution, this application can accurately quantify the impact of environmental interference on image quality when the camera is subject to mechanical vibration or abnormal lighting conditions. Combined with enhanced features such as clarity after dehazing, the confidence assessment results can be dynamically adjusted. This technology significantly improves the camera's state perception accuracy in complex working conditions, provides a reliable basis for multi-sensor fusion weight allocation, and thus enhances the environmental adaptability and detection stability of obstacle location recognition.

[0094] Preferably, the specific working principle of the fusion weight distribution module is: importing the camera confidence and the lidar confidence into the constructed weight distribution model to obtain the lidar weight coefficient and the camera weight coefficient. The weight distribution model is expressed as:

[0095]

[0096] in, Represents the weight coefficient of the laser radar recognition result, Indicates the weight coefficient of the camera recognition result, Indicates the startup function, represents the lidar confidence, Indicates the camera confidence, represents the sensitivity index (control weight switching speed), Indicates confidence, Represents the confidence threshold.

[0097] Specifically, when the lidar and camera confidence levels are input into the weight allocation model, the activation function assigns weights to the activation function's output based on the two confidence levels. The sensitivity index uses exponential calculations to amplify or minimize the impact of confidence differences on the weights. For example, when the sensitivity index is greater than 1, the weight of high-confidence sensors is significantly increased, thereby prioritizing more reliable sensors in high-interference environments. Finally, the lidar and camera weight coefficients are normalized to ensure that their sum is 1, achieving dynamic weight allocation and stability in the fusion results.

[0098] Compared with existing technologies, traditional multi-sensor fusion methods typically use fixed weight coefficients and are unable to adjust sensor contributions based on environmental changes. For example, when the lidar's confidence level drops due to haze, the fixed weights will still be assigned the same proportion, resulting in increased errors in the fusion results. However, this solution, through dynamic confidence assessment and a nonlinear weight distribution model, can automatically adjust the weight coefficients when sensor performance fluctuates. For example, when the camera's confidence level drops due to insufficient lighting, its weight coefficient is quickly suppressed by exponential calculations. At the same time, the lidar's high confidence level is amplified by the sensitivity exponent, thereby improving the accuracy of the fusion results.

[0099] Through the above technical solution, this application solves the problem of reduced detection accuracy caused by the inability of traditional methods to adapt to environmental changes due to fixed weight distribution. By starting the function to shield the interference of low-confidence sensors and dynamically adjusting the weight distribution ratio in combination with the sensitivity index, when the performance of the lidar or camera degrades alone, the fusion system can automatically focus on the recognition results of the other sensor, thereby maintaining the reliability and robustness of obstacle detection under complex environmental conditions. For example, when the point cloud noise of the lidar increases due to vibration, its confidence decreases, triggering a reduction in the weight coefficient. At the same time, the high confidence of the camera under sufficient lighting conditions is given a higher weight, thereby ensuring the accuracy of the obstacle position fusion result.

[0100] The specific working principle of the fusion position analysis module is as follows: the weight coefficient of the lidar recognition result, the weight coefficient of the camera recognition result, the lidar's recognition coordinates of the obstacle, and the camera's recognition coordinates of the obstacle are imported into the constructed fusion position model to output the fusion position. The fusion position model is expressed as (weighted average formula):

[0101]

[0102] in, Indicates the fusion position, Indicates the laser radar's recognition coordinates of the obstacle, Indicates the camera's recognition coordinates of obstacles. Represents the weight coefficient of the laser radar recognition result, Indicates the weight coefficient of the camera recognition result.

[0103] Specifically, the system updates the weight coefficients in real time as environmental conditions change. For example, in rainy and foggy weather, the LiDAR point cloud density index decreases, while the camera's defogging clarity decreases due to insufficient light intensity. At this point, the confidence analysis module calculates the LiDAR confidence and the camera confidence. The weight distribution model increases the LiDAR weight coefficient and decreases the camera weight coefficient. The fused position model substitutes the obstacle coordinates of both into a weighted average formula to generate fused coordinates that deviate from the original LiDAR and camera coordinates. This process effectively suppresses detection bias of a single sensor in harsh environments by adaptively adjusting the weight ratio.

[0104] Compared to existing technologies, traditional methods typically use fixed weighting coefficients for data fusion. In the presence of sudden environmental interference, this static allocation results in less reliable sensor data still being given a higher weight. This solution introduces a dynamic weighting mechanism. When the camera experiences strong vibrations, which increases the environmental penalty index, it automatically reduces the weighting coefficient, thereby preventing the impact of vibration-induced image blur on the final positioning accuracy.

[0105] Through the above technical solution, this application solves the error accumulation problem caused by the fixed weight allocation strategy when the environment changes suddenly, so that the fluctuation range of obstacle positioning accuracy in complex working conditions such as rain, fog, strong light, and equipment vibration is reduced.

[0106] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus.

[0107] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. An obstacle recognition device based on laser point cloud data, characterized in that: include: Data acquisition module, used to obtain environmental data, operating environment data, laser point cloud data and camera data; The LiDAR confidence analysis module obtains LiDAR confidence based on weather visibility and temperature in environmental data, vibration intensity in operating environment data, and laser point cloud data; The camera confidence analysis module obtains the camera confidence based on the light intensity in the environmental data, the vibration intensity in the operating environment data, and the camera data; A fusion weight allocation module allocates dynamic fusion weights based on the lidar confidence and the camera confidence; A fusion position analysis module, which constructs a fusion position model based on the dynamic fusion weight and outputs a fusion position according to the obstacle recognition results of the laser radar and the camera; The environmental data includes weather visibility, temperature, and light intensity; the operating environment data includes the vibration intensity of the laser radar and the camera's carrier; the laser point cloud data includes the noise ratio, effective reflectivity, and point cloud density index, which is the ratio of the current point cloud density to the nominal maximum density; and the camera data includes image contrast, brightness gradient, and clarity after defogging. The specific working principle of the lidar confidence analysis module is as follows: Obtain the lidar environmental penalty index based on the current vibration intensity, current temperature, and current weather visibility; The lidar environment penalty index, noise ratio, effective reflectivity, and point cloud density index are imported into the constructed lidar confidence model to output the lidar confidence. The lidar confidence model is expressed as: in, represents the lidar confidence, represents the point cloud density index, represents the noise ratio, represents the effective reflectivity, represents the environmental penalty index, represents the weight coefficient and , is the lidar environment penalty factor, Indicates the visibility deviation value, Indicates the strength loss value, Indicates the temperature deviation value; The specific working principle of the camera confidence analysis module is as follows: Obtain the camera environment penalty index based on the current vibration index and light intensity; The image contrast, brightness gradient, and clarity after defogging are normalized to the minimum and maximum values ​​and then imported into the camera confidence model constructed with the camera environment penalty index to output the camera confidence. The camera confidence model is expressed as: in, Indicates the camera confidence, represents the contrast of the normalized image, represents the brightness gradient after normalization, Indicates the clarity after defogging after normalization, represents the camera environment penalty index, represents the camera environment penalty factor, represents the weight coefficient and , Indicates the strength loss value, Indicates the lighting deviation value.

2. The obstacle recognition device based on laser point cloud data according to claim 1, characterized in that: The specific working principle of the fusion weight distribution module is: import the camera confidence and lidar confidence into the constructed weight distribution model to obtain the lidar weight coefficient and camera weight coefficient. The weight distribution model is expressed as: in, Represents the weight coefficient of the laser radar recognition result, Indicates the weight coefficient of the camera recognition result, Indicates the startup function, represents the lidar confidence, Indicates the camera confidence, represents the sensitivity index, Indicates confidence, Represents the confidence threshold.

3. The obstacle recognition device based on laser point cloud data according to claim 2, characterized in that: The specific working principle of the fusion position analysis module is as follows: the weight coefficient of the laser radar recognition result, the weight coefficient of the camera recognition result, the laser radar's recognition coordinates of the obstacle, and the camera's recognition coordinates of the obstacle are imported into the constructed fusion position model to output the fusion position. The fusion position model is expressed as: in, Indicates the fusion position, Indicates the laser radar's recognition coordinates of the obstacle, Indicates the camera's recognition coordinates of obstacles. Represents the weight coefficient of the laser radar recognition result, Indicates the weight coefficient of the camera recognition result.

4. The obstacle recognition device based on laser point cloud data according to claim 1, characterized in that: The lidar environmental penalty index is obtained based on the current vibration intensity, current temperature, and current weather visibility. Specifically: The current weather visibility is subtracted from the nominal maximum visibility and then the difference is compared with the nominal maximum visibility to obtain the weather visibility deviation value; Ratio the current vibration intensity to the nominal maximum vibration intensity to obtain the intensity loss value; The current temperature is processed with the optimal temperature, and the difference between the current temperature and the optimal temperature is processed, and the difference between the nominal maximum temperature and the nominal minimum temperature is processed, and the obtained ratio is processed with the absolute value to obtain the temperature deviation value; The visibility deviation value, intensity loss value and temperature deviation value are weighted averaged to obtain the lidar environment penalty index.

5. The obstacle recognition device based on laser point cloud data according to claim 1, characterized in that: The camera environment penalty index is obtained based on the current vibration index and light intensity, specifically: Ratio the current vibration intensity to the nominal maximum vibration intensity to obtain the intensity loss value; The absolute value of the difference between the current light intensity and the camera's optimal light intensity is compared with the absolute value of the difference between the maximum light intensity and the minimum light intensity supported by the camera to obtain the light deviation value. The obtained intensity loss value and the light intensity deviation value are weighted averaged to obtain the camera environment penalty index.

Images