Obstacle recognition device based on laser point cloud data

Abstract

The invention discloses an obstacle recognition device based on laser point cloud data, and belongs to the technical field of recognition and detection, and the device comprises a laser radar confidence analysis module which obtains the laser radar confidence based on the weather visibility and temperature in environment data, the vibration intensity in operation environment data, and the laser point cloud data; the camera confidence coefficient analysis module is used for acquiring camera confidence coefficient based on the illumination intensity in the environment data, the vibration intensity in the operation environment data and the camera data; the fusion weight distribution module is used for distributing a dynamic fusion weight based on the laser radar confidence coefficient and the camera confidence coefficient; the fusion position analysis module is used for constructing a fusion position model based on the dynamic fusion weight and outputting a fusion position according to an obstacle recognition result of the laser radar and the camera; according to the method, the fusion weight is adjusted in real time through confidence coefficient dynamic analysis of the laser radar and the camera, and the obstacle recognition capability in a complex environment is remarkably improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of recognition and detection, and particularly relates to an obstacle recognition device based on lidar point cloud data. Background Art

[0002] In the existing obstacle recognition technologies, a single sensor (such as lidar or camera) often has its performance degraded due to environmental conditions. For example, in lidar, in bad weather (such as low visibility, high temperature) or when the device vibrates, the point cloud data is vulnerable to noise interference, affecting the detection accuracy; in a camera, in low light, strong vibration or haze environment, the image contrast and clarity are significantly reduced, resulting in misjudgment or missed detection. Traditional multi-sensor fusion methods mostly adopt a fixed weight allocation strategy and cannot dynamically adapt to complex environmental changes, with insufficient robustness.

[0003] Therefore, there is an urgent need for an obstacle recognition device that can evaluate the confidence of sensors in real time and dynamically adjust the fusion weights to improve the detection accuracy and reliability in complex environments. Summary of the Invention

[0004] Aiming at the deficiencies of the prior art, the present invention provides an obstacle recognition device based on lidar point cloud data, which solves the above problems.

[0005] To achieve the above objectives, the present invention is realized through the following technical solutions: An obstacle recognition device based on lidar point cloud data, comprising: A data acquisition module, configured to acquire environmental data, operating environment data, lidar point cloud data, and camera data; A lidar confidence analysis module, configured to obtain the lidar confidence based on the weather visibility and temperature in the environmental data, the vibration intensity in the operating environment data, and the lidar point cloud data; A camera confidence analysis module, configured to obtain the camera confidence based on the light intensity in the environmental data, the vibration intensity in the operating environment data, and the camera data; A fusion weight allocation module, configured to allocate dynamic fusion weights based on the lidar confidence and the camera confidence; A fusion position analysis module, configured to construct a fusion position model based on the dynamic fusion weights and output a fusion position according to the recognition results of the lidar and the camera on the obstacle.

[0006] Based on the above technical solutions, the present invention also provides the following optional technical solutions: Further technical solution: The environmental data includes weather visibility, temperature, and light intensity. The operating environmental data includes the vibration intensity of the carrier where the lidar and camera are located. The lidar point cloud data includes noise ratio, effective reflectivity, and point cloud density index. The point cloud density index is the ratio of the current point cloud density to the nominal maximum density. The camera data includes image contrast, brightness gradient, and clarity after defogging.

[0007] Further technical solution: The specific working principle of the lidar confidence analysis module is as follows: Obtain the lidar environmental penalty index based on the current vibration intensity, current temperature, and current weather visibility; Input the lidar environmental penalty index, noise ratio, effective reflectivity, and point cloud density index into the constructed lidar confidence model to output the lidar confidence. The lidar confidence model is expressed as: Among them, represents the lidar confidence, represents the point cloud density index, represents the noise ratio, represents the effective reflectivity, represents the environmental penalty index, represents the weight coefficient and , is the lidar environmental penalty factor.

[0008] Further technical solution: The specific working principle of the camera confidence analysis module is as follows: Obtain the camera environmental penalty index based on the current vibration index and light intensity; Perform min-max normalization on the image contrast, brightness gradient, and clarity after defogging, and then input them together with the camera environmental penalty index into the constructed camera confidence model to output the camera confidence. The camera confidence model is expressed as: Among them, represents the camera confidence, represents the image contrast after normalization, represents the brightness gradient after normalization, represents the clarity after defogging after normalization, represents the camera environmental penalty index, represents the camera environmental penalty factor, represents the weight coefficient and .

[0009] Further technical solution: The specific working principle of the fusion weight allocation module is as follows: Import the camera confidence and lidar confidence into the constructed weight allocation model to obtain the lidar weight coefficient and the camera weight coefficient. The weight allocation model is expressed as: Where, Represents the lidar recognition result weight coefficient, Represents the camera recognition result weight coefficient, Represents the activation function, Represents the lidar confidence, Represents the camera confidence, Represents the sensitivity index, Represents the confidence, Represents the confidence threshold.

[0010] Further technical solution: The specific working principle of the fusion position analysis module: Import the lidar recognition result weight coefficient, the camera recognition result weight coefficient, the lidar recognition coordinates of the obstacle, and the camera recognition coordinates of the obstacle into the constructed fusion position model to output the fusion position. The fusion position model is expressed as: Where, Represents the fusion position, Represents the lidar recognition coordinates of the obstacle, Represents the camera recognition coordinates of the obstacle, Represents the lidar recognition result weight coefficient, Represents the camera recognition result weight coefficient.

[0011] Further technical solution: Obtain the lidar environmental penalty index based on the current vibration intensity, current temperature, and current weather visibility. Specifically: Perform a difference operation on the current weather visibility and the nominal maximum visibility, and then perform a ratio operation with the nominal maximum visibility to obtain the weather visibility deviation value; Perform a ratio operation on the current vibration intensity and the nominal maximum vibration intensity to obtain the intensity loss value; Perform a difference operation on the current temperature and the optimal temperature, then perform a ratio operation with the difference between the nominal highest temperature and the nominal lowest temperature, and take the absolute value of the obtained ratio to obtain the temperature deviation value; Perform a weighted average operation on the visibility deviation value, the intensity loss value, and the temperature deviation value to obtain the lidar environmental penalty index.

[0012] Further technical solution: Obtain the camera environmental penalty index based on the current vibration index and light intensity. Specifically: Process the ratio of the current vibration intensity to the nominal maximum vibration intensity to obtain the intensity loss value; Process the ratio of the absolute value of the difference between the current light intensity and the optimal light intensity of the camera to the absolute value of the difference between the maximum light intensity and the minimum light intensity supported by the camera to obtain the light deviation value; Perform a weighted average process on the obtained intensity loss value and the light intensity deviation value to further obtain the camera environment penalty index.

[0013] The present invention provides an obstacle recognition device based on lidar point cloud data, which has the following beneficial effects compared with the prior art: 1. Through the dynamic analysis of the confidence levels of the lidar and the camera (such as parameters like weather visibility, temperature, vibration, and light), the present invention can adjust the fusion weights in real time, significantly improving the obstacle recognition ability in complex environments. At the same time, based on the weighted average fusion position model, combining the geometric accuracy of the lidar point cloud data and the rich texture information of the camera image, it outputs a more accurate obstacle position. Moreover, it can quantify the impact of environmental factors on the sensor performance by introducing the lidar confidence model and the camera confidence model, ensuring the rationality of weight allocation, and can dynamically correct the confidence level through the environmental penalty index (such as vibration and temperature deviation values), reducing the misjudgment rate under extreme conditions and enhancing the stability and reliability of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is a schematic flow chart of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0015] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0016] The following describes the specific implementation of the present invention in detail with specific embodiments.

[0017] Please refer to Figure 1 , an obstacle recognition device based on lidar point cloud data provided by an embodiment of the present invention, includes: A data acquisition module for acquiring environmental data, operating environment data, lidar point cloud data, and camera data; A lidar confidence analysis module for obtaining the lidar confidence based on the weather visibility and temperature in the environmental data, the vibration intensity in the operating environment data, and the lidar point cloud data; A camera confidence analysis module for obtaining the camera confidence based on the light intensity in the environmental data, the vibration intensity in the operating environment data, and the camera data; A fusion weight allocation module that allocates dynamic fusion weights based on the lidar confidence and camera confidence; A fusion position analysis module that constructs a fusion position model based on the dynamic fusion weights and outputs a fusion position according to the recognition results of the lidar and the camera for obstacles; The environmental data includes weather visibility, temperature, and light intensity, and the operating environmental data includes the vibration intensity of the carrier where the lidar and the camera are located; The lidar point cloud data includes noise ratio, effective reflectivity, and point cloud density index, and the point cloud density index is the ratio of the current point cloud density to the nominal maximum density; The camera data includes image contrast, brightness gradient, and clarity after defogging; Specifically, first, the data acquisition module first collects the weather visibility, temperature, and light intensity in the environmental data, the vibration intensity in the operating environmental data, the noise ratio, effective reflectivity, and point cloud density index output by the lidar, and the image contrast, brightness gradient, and clarity after defogging output by the camera. The lidar confidence analysis module calculates the environmental penalty index based on the environmental temperature, visibility, and vibration intensity, combines the point cloud quality parameters, and outputs the lidar confidence through a mathematical model. The camera confidence analysis module synchronously calculates the light deviation value and vibration intensity loss value, and generates the camera confidence in combination with the normalization result of the image quality parameters. The fusion weight allocation module compares the two types of confidences, reduces the weight when the confidence of a certain sensor is lower than the threshold, and increases the weight otherwise, forming a dynamic allocation strategy. The fusion position analysis module performs weighted averaging on the recognition coordinates of the lidar and the camera according to the weight coefficient, and outputs the optimized obstacle position data.

[0018] Compared with the prior art, the existing method adopts a fixed weight fusion strategy and cannot cope with the dynamic attenuation of sensor performance. For example, when strong vibration causes the camera image to be blurred, the traditional method still gives the camera a fixed weight, resulting in an increase in the error of the fusion result. This solution dynamically adjusts the weight through real-time confidence evaluation, automatically reducing its weight ratio when the sensor performance decreases. For example, when the camera confidence decreases due to vibration, the system automatically increases the lidar weight to ensure the stability of the fusion result; Through the above technical solutions, the present application can maintain the obstacle detection accuracy under complex environmental changes. For example, when heavy rain causes the lidar point cloud density to decrease, the confidence analysis module automatically reduces its weight and instead relies on the camera data for compensation; when strong light causes the camera to overexpose, the system increases the lidar weight to avoid detection errors caused by the failure of a single sensor. This solution effectively solves the problem of decreased detection reliability caused by the insufficient environmental adaptability of traditional methods.

[0019] Secondly, by defining weather visibility, temperature, and light intensity as the core parameters of environmental data, which respectively correspond to the beam attenuation characteristics of lidar, the temperature stability of sensors, and the photosensitive performance of cameras, a multi-dimensional environmental state evaluation system is established. The vibration intensity is used as the key index of the operating environmental data, directly related to the jitter error of lidar point clouds and the blur degree of camera images, solving the defect that the existing technology does not consider the influence of mechanical vibration. The noise ratio, effective reflectivity, and point cloud density index are introduced into the lidar point cloud data. The noise ratio reflects the degree of signal interference, the effective reflectivity characterizes the reliability of target recognition, and the point cloud density index quantifies the data integrity through the ratio of the actual density to the nominal density. The combination of the three can accurately evaluate the data quality of lidar. In the camera data, image contrast, brightness gradient, and clarity after defogging are defined, corresponding to the recognizable degree of image features, the ability to compensate for light uniformity, and the effect of eliminating haze interference respectively, providing multi-dimensional quantitative indicators for the evaluation of the camera working state. Through the systematic construction of the above parameter system, an accurate input data basis is provided for the subsequent confidence model, enabling the dynamic fusion weight allocation to accurately reflect the actual working state of each sensor in a complex environment.

[0020] Compared with the existing technology, traditional methods usually only use a single environmental parameter to evaluate the performance of sensors. For example, they only judge the state of the camera by light intensity or only evaluate the performance of lidar by visibility, resulting in insufficient environmental adaptability. In the existing technology, lidar point cloud data often only focuses on the absolute value of point cloud density and does not consider the impact of the difference in the nominal performance of the device on the evaluation of data quality. The introduction of the point cloud density index realizes the normalized evaluation of device performance. The analysis of existing camera data mostly relies on the original image clarity index and does not consider the restoration effect of the defogging algorithm on image quality, resulting in inaccurate evaluation in a haze environment. The existing evaluation of the motion environment often ignores the common influence of vibration intensity on lidar and cameras. This solution realizes the cross-sensor stability evaluation through the unified definition of the vibration intensity parameter. In the existing technology, there is a lack of systematic association between environmental parameters and sensor data parameters, resulting in a lack of data support for dynamic fusion weight allocation. This solution provides a structured input for the fusion algorithm by establishing a multi-dimensional parameter system.

[0021] Through the above technical solutions, this application solves the problems of poor environmental adaptability and low fusion accuracy caused by incomplete parameter definitions in traditional obstacle recognition technologies. By refining the types of sensor data parameters, it is possible to accurately quantify the signal attenuation degree of lidar under complex meteorological conditions, effectively identify the performance fluctuations of lasers caused by high-temperature environments, and precisely evaluate the impact of vibrations on the stability of point clouds. By defining the clarity parameter after defogging, the accuracy of the state evaluation of cameras in harsh weather such as haze is significantly improved. By introducing the point cloud density index, the performance normalization evaluation of different models of lidar devices is realized, avoiding misjudgments caused by device differences in traditional methods. By systematically constructing lidar point cloud quality parameters such as noise ratio and effective reflectivity, it is possible to timely detect performance degradation caused by sensor aging or contamination. Finally, it provides an accurate and reliable data basis for multi-sensor dynamic fusion, significantly improving the obstacle detection accuracy in complex environments.

[0022] Preferably, the specific working principle of the lidar confidence analysis module is as follows: Obtain the lidar environmental penalty index based on the current vibration intensity, current temperature, and current weather visibility, specifically; Perform a difference operation between the current weather visibility and the nominal maximum visibility, and then perform a ratio operation with the nominal maximum visibility to obtain the weather visibility deviation value; Perform a ratio operation between the current vibration intensity and the nominal maximum vibration intensity to obtain the intensity loss value; Perform a difference operation between the current temperature and the optimal temperature, then perform a ratio operation with the difference between the nominal maximum temperature and the nominal minimum temperature, and take the absolute value of the obtained ratio to obtain the temperature deviation value; Perform a weighted average operation on the visibility deviation value, intensity loss value, and temperature deviation value to obtain the lidar environmental penalty index; The lidar environmental penalty index is expressed as: Among them, represents the lidar environmental penalty index, represents the visibility deviation value, represents the intensity loss value, represents the temperature deviation value, is the weight coefficient and ; Import the lidar environmental penalty index, noise ratio, effective reflectivity, and point cloud density index into the constructed lidar confidence model to output the lidar confidence; The lidar confidence model is expressed as: Among them, Indicates the lidar confidence Indicates the point cloud density index Indicates the noise ratio Indicates the effective reflectivity Indicates the environmental penalty index Indicates the weight coefficient and , is the lidar environmental penalty factor

[0023] Specifically, the calculation of the weather visibility deviation value establishes a reference frame based on the nominal maximum visibility. For example, in a device with a nominal maximum visibility of 1000 meters, when encountering fog and haze weather, the measured visibility drops to 200 meters, and the deviation value reaches 0.8, directly representing the degree of laser beam scattering loss. The intensity loss value is calculated by the ratio of the vibration intensity to the upper limit of the device's tolerance. For example, when the nominal maximum vibration intensity of the device is 5g and the measured value reaches 3g, the loss value is 0.6, reflecting the risk of mechanical stability decline of the laser transmitter and receiver. The temperature deviation value is obtained by dividing the difference between the actual temperature and the optimal operating temperature (such as 25°C) by the allowable operating temperature range of the device (such as an 80°C span from -20°C to 60°C), achieving unified quantification under different temperature conditions. During the weighted average process, for example, setting the weight of the visibility deviation value to 0.5, the weight of the intensity loss value to 0.3, and the weight of the temperature deviation value to 0.2, a comprehensive environmental penalty index is formed. The value range of this index is between 0 and 1, and the larger the value, the more serious the environmental interference

[0024] Compared with the prior art, traditional methods usually only monitor a single environmental parameter or use a fixed threshold for judgment. For example, only based on the vibration intensity exceeding the standard, the device is determined to be faulty. This solution, through multi-dimensional parameter normalization processing and a dynamic weighting mechanism, not only considers the compatibility of parameters with different physical dimensions, but also reflects the differences in the influence of various environmental factors on lidar performance through weight allocation. For example, in sandstorm weather, the weight of the visibility deviation value automatically increases, and in a high-temperature environment, the weight of the temperature deviation value increases, forming an adaptive environmental assessment model

[0025] Through the above technical solutions, this application effectively solves the problem of lidar performance attenuation caused by the coupling effect of multiple environmental factors such as vibration, temperature, and visibility. By establishing a standardized deviation value and a weighted fusion mechanism, a quantitative assessment of complex environmental interference is achieved, providing accurate input parameters for subsequent confidence calculation. For example, when there are both device vibration and low-temperature environment, this solution can accurately distinguish the superposition effect of mechanical offset caused by vibration and the decrease in component sensitivity caused by low temperature, avoiding the problem of one-sided environmental factor assessment in traditional methods Furthermore, by collecting vibration intensity, temperature, and weather visibility data in real time, an environmental penalty index is generated through weighted averaging after being processed by the above steps. Subsequently, the environmental penalty index, point cloud density index, noise ratio, and effective reflectivity are substituted into the confidence model, and the contribution ratio of data quality and environmental interference is adjusted through weight coefficients, and finally, the lidar confidence with dynamic adjustment is output.

[0026] Compared with the prior art, the prior art usually only evaluates data quality for a single environmental factor. For example, it only relies on point cloud density or reflectivity for judgment, while ignoring the combined effects of multiple factors such as vibration and temperature. This solution introduces an environmental penalty index, converts multi-dimensional environmental parameters into a unified quantitative index, and combines the noise ratio and point cloud density index to achieve a comprehensive evaluation of the reliability of lidar data. In addition, the influence of environmental parameters in the prior art is usually judged by a fixed threshold, while this solution can more precisely reflect the continuous influence of environmental changes through weighted calculation and dynamic attenuation factors.

[0027] Through the above technical solution, this application can quantitatively measure the performance attenuation degree of lidar in a complex environment in real time, and effectively suppress the problem of the decline in the reliability of point cloud data caused by vibration, temperature fluctuations, or low visibility. For example, in a high-temperature or strong vibration scenario, the increase in the environmental penalty index will reduce the weight of the corresponding item in the confidence model, thereby reducing the influence of noise data and abnormal points on obstacle recognition. At the same time, the product term of the point cloud density index and the effective reflectivity can ensure that high-quality data is preferred when the environmental conditions are stable, and finally improve the accuracy and robustness of obstacle position recognition.

[0028] Preferably, the specific working principle of the camera confidence analysis module is as follows: Obtain the camera environmental penalty index based on the current vibration index and light intensity, specifically: Perform a ratio process on the current vibration intensity and the nominal maximum vibration intensity to obtain the intensity loss value; Perform a ratio process on the absolute value of the difference between the current light intensity and the optimal light intensity of the camera and the absolute value of the difference between the maximum light intensity and the minimum light intensity supported by the camera to obtain the light deviation value; Perform a weighted average process on the obtained intensity loss value and the light intensity deviation value, and then obtain the camera environmental penalty index, which is expressed as; Among them, represents the camera environmental penalty index, represents the light deviation value, represents the intensity loss value, represents the weight coefficient and ; After performing min-max normalization on the image contrast, brightness gradient, and clarity after defogging, and importing the camera environment penalty index into the constructed camera confidence model, the camera confidence is output; The camera confidence model is expressed as: Among them, represents the camera confidence, represents the image contrast after normalization, represents the brightness gradient after normalization, represents the clarity after defogging after normalization, represents the camera environment penalty index, represents the camera environment penalty factor, represents the weight coefficient and .

[0029] Specifically, during the vibration intensity analysis, when the vehicle is driving on a bumpy road, the acceleration sensor collects vibration data in real time. When the proportion of the vibration intensity to the nominal maximum value reaches 0.3, it indicates that the camera has a 30% stability loss. When evaluating the lighting conditions, if the optimal working illuminance of the camera is 2000 - 5000 lux, and the current ambient illuminance is measured as 800 lux, the lighting deviation value is calculated as (|800 - 3500|) / (5000 - 2000) = 0.9, indicating that the current lighting conditions seriously deviate from the ideal working range. The above two parameters are weighted and calculated according to the weight coefficients of 0.6 and 0.4, and the finally obtained environment penalty index is used to reduce the confidence weight of the camera in the fusion decision.

[0030] Compared with the prior art, traditional methods usually only detect a single environmental factor or use a fixed threshold to judge the availability of the camera. For example, only a binary judgment is made based on whether the light intensity exceeds the sensor range. However, this solution can more finely characterize the degree of environmental interference by quantifying the combined effects of vibration and light, and constructing a continuous penalty index. There is a lack of modeling for the synergistic effect of mechanical vibration and optical conditions in the prior art. This solution establishes a coupling analysis mechanism for multi-dimensional environmental parameters by weighted fusion of the deviation degrees of the two.

[0031] Through the above technical solutions, this application can effectively identify the performance degradation state of the camera under strong vibration or abnormal lighting conditions, and provide a basis for dynamic adjustment of multi-sensor fusion by generating a quantified environmental penalty index. When the vehicle passes over a speed bump, causing high-frequency vibration, the system automatically reduces the weight of the camera to avoid positioning errors caused by blurred images. In the scenario of sudden changes in lighting at the tunnel entrance and exit, the fusion strategy is adjusted through the real-time updated lighting deviation value to prevent misjudgment caused by overexposed or low-illumination images. This confidence evaluation mechanism based on real-time perception of environmental parameters significantly improves the adaptability and reliability of the obstacle detection system under complex working conditions.

[0032] Furthermore, the camera environmental penalty index is generated by collecting vibration intensity and lighting intensity data in real time, calculating the intensity loss value and lighting deviation value respectively, and then performing weighted summation. This process converts the image blur caused by mechanical vibration and color distortion caused by abnormal lighting into a quantifiable environmental interference index. The image contrast, brightness gradient, and clarity after dehazing are normalized into standardized parameters with a unified dimension to ensure comparability of each quality dimension in the confidence model. During the calculation of the confidence model, the product term of the camera environmental penalty factor and the environmental penalty index is used as a penalty term to dynamically adjust the attenuation amplitude of environmental interference on the final confidence, while the weight coefficient is used to balance the contribution ratio of each dimension of image quality. Finally, a confidence evaluation result that comprehensively reflects the working state of the camera and the image quality is output.

[0033] Compared with the prior art, traditional methods usually judge the camera state only through a single lighting intensity or vibration threshold, without establishing a dynamic association between multi-dimensional environmental parameters and image quality indicators. This solution innovatively converts the combined environmental interference of vibration and lighting into a computable penalty index, and at the same time combines the multi-dimensional normalization processing of image quality parameters to form a dynamic confidence evaluation mechanism, effectively overcoming the limitation of the separate analysis of environmental factors and image features.

[0034] Through the above technical solutions, this application can accurately quantify the degree of influence of environmental interference on image quality when the camera is affected by mechanical vibration or abnormal lighting conditions, and dynamically adjust the confidence evaluation result in combination with enhanced features such as clarity after dehazing processing. This technology significantly improves the state perception accuracy of the camera under complex working conditions, provides a reliable basis for the weight allocation of multi-sensor fusion, and further enhances the environmental adaptability and detection stability of obstacle position recognition.

[0035] Preferably, the specific working principle of the fusion weight allocation module is as follows: the camera confidence and lidar confidence are imported into the constructed weight allocation model to obtain the lidar weight coefficient and the camera weight coefficient. The weight allocation model is expressed as: Wherein, Represents the weight coefficient of the lidar recognition result, Represents the weight coefficient of the camera recognition result, Represents the start function, Represents the lidar confidence, Represents the camera confidence, Represents the sensitivity index (controlling the weight switching speed), Represents the confidence, Represents the confidence threshold.

[0036] Specifically, when the lidar confidence and the camera confidence are input into the weight assignment model, the start function assigns weights according to the two confidences in the output value of the start function. The sensitivity index amplifies or shrinks the influence of the confidence difference on the weight through exponential operation. For example, when the sensitivity index is greater than 1, the weight of the high-confidence sensor will be significantly enhanced, so that the more reliable sensor is preferred when the environmental interference is strong. Finally, the weight coefficients of the lidar and the camera are normalized to ensure that their sum is 1, realizing the stability of the dynamic weight assignment and the fusion result.

[0037] Compared with the prior art, traditional multi-sensor fusion methods usually adopt fixed weight coefficients and cannot adjust the contribution degree of sensors according to environmental changes. For example, when the confidence of the lidar decreases due to haze, the fixed weight will still assign the same proportion to it, resulting in an increase in the error of the fusion result. However, this solution can automatically adjust the weight coefficients when the sensor performance fluctuates through confidence dynamic evaluation and non-linear weight assignment model. For example, when the confidence of the camera decreases due to insufficient light, its weight coefficient is quickly suppressed by exponential operation, while the high confidence of the lidar is amplified by the sensitivity index, thereby improving the accuracy of the fusion result.

[0038] Through the above technical solution, this application solves the problem of the decrease in detection accuracy caused by the inability of the traditional method to adapt to environmental changes due to fixed weight assignment. By shielding the interference of low-confidence sensors through the start function and dynamically adjusting the weight assignment ratio in combination with the sensitivity index, when the performance of the lidar or the camera degrades alone, the fusion system can automatically focus on the recognition result of the other sensor, so as to maintain the reliability and robustness of obstacle detection under complex environmental conditions. For example, when the lidar has increased point cloud noise due to vibration, its confidence decreases and triggers a decrease in the weight coefficient, while the high confidence of the camera under sufficient light conditions is given a higher weight, thus ensuring the accuracy of the fusion result of the obstacle position.

[0039] Specific working principle of the fusion position analysis module: Import the weight coefficient of the lidar recognition result, the weight coefficient of the camera recognition result, the recognition coordinates of the obstacle by the lidar, and the recognition coordinates of the obstacle by the camera into the constructed fusion position model to output the fusion position. The fusion position model is expressed as (weighted average formula): Where, represents the fusion position, represents the recognition coordinates of the obstacle by the lidar, represents the recognition coordinates of the obstacle by the camera, represents the weight coefficient of the lidar recognition result, represents the weight coefficient of the camera recognition result.

[0040] Specifically, when the environmental conditions change, the system updates the weight coefficients in real time. For example, in rainy and foggy weather, the point cloud density index of the lidar decreases, and at the same time, the clarity of the camera after defogging decreases due to insufficient light intensity. At this time, the confidence analysis module calculates the lidar confidence and the camera confidence. After calculation by the weight distribution model, the lidar weight coefficient increases, and the camera weight coefficient decreases. The fusion position model substitutes the obstacle coordinates of both into the weighted average formula to generate a fusion coordinate that deviates from the original coordinates of the lidar and the original coordinates of the camera. This process effectively suppresses the detection deviation of a single sensor in a harsh environment by adaptively adjusting the weight ratio.

[0041] Compared with the prior art, traditional methods usually use fixed weight coefficients for data fusion. In the case of sudden environmental interference, this static allocation will cause the data of sensors with low reliability to still occupy a large weight. This solution introduces a dynamic weight mechanism. When the camera encounters strong vibration and the environmental penalty index increases, its weight coefficient can be automatically reduced, thereby avoiding the impact of image blur caused by vibration on the final positioning accuracy.

[0042] Through the above technical solution, this application solves the problem of error accumulation caused by the fixed weight allocation strategy during environmental mutations, and reduces the fluctuation range of the obstacle positioning accuracy under complex working conditions such as rainy and foggy weather, strong light, and equipment vibration.

[0043] It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprising", "including" or any other variant thereof are intended to cover non-exclusively, so that a process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device.

[0044] Although the embodiments of the present invention have been shown and described, those of ordinary skill in the art can understand that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An obstacle recognition device based on laser point cloud data, characterized in that Including: A data acquisition module, used to acquire environmental data, operating environment data, lidar point cloud data, and camera data; A lidar confidence analysis module, which obtains the lidar confidence based on the weather visibility and temperature in the environmental data, the vibration intensity in the operating environment data, and the lidar point cloud data; A camera confidence analysis module, which obtains the camera confidence based on the light intensity in the environmental data, the vibration intensity in the operating environment data, and the camera data; A fusion weight allocation module, which allocates dynamic fusion weights based on the lidar confidence and the camera confidence; A fusion position analysis module, which constructs a fusion position model based on the dynamic fusion weights and outputs a fusion position according to the recognition results of obstacles by the lidar and the camera.

2. The obstacle recognition device based on laser point cloud data according to claim 1, wherein The environmental data includes weather visibility, temperature, and light intensity. The operating environment data includes the vibration intensity of the carrier where the lidar and the camera are located. The lidar point cloud data includes noise ratio, effective reflectivity, and point cloud density index. The point cloud density index is the ratio of the current point cloud density to the nominal maximum density. The camera data includes image contrast, brightness gradient, and clarity after defogging.

3. The obstacle recognition device based on laser point cloud data according to claim 2, wherein The specific working principle of the lidar confidence analysis module is: Obtain the lidar environmental penalty index based on the current vibration intensity, current temperature, and current weather visibility; Import the lidar environmental penalty index, noise ratio, effective reflectivity, and point cloud density index into the constructed lidar confidence model to output the lidar confidence. The lidar confidence model is expressed as: Among them, represents the lidar confidence,[[]]END]] represents the point cloud density index,[[]]END]] represents the noise ratio,[[]]END]] represents the effective reflectivity,[[]]END]] represents the environmental penalty index,[[]]END]] represents the weight coefficient and , is the lidar environmental penalty factor.[[]]END]] 4. The obstacle recognition device based on laser point cloud data according to claim 3, wherein The specific working principle of the camera confidence analysis module is: Obtain the camera environmental penalty index based on the current vibration index and light intensity; After performing min-max normalization processing on the image contrast, brightness gradient, and clarity after defogging, import them together with the camera environmental penalty index into the constructed camera confidence model to output the camera confidence. The camera confidence model is expressed as: Among them, represents the camera confidence level, represents the image contrast after normalization processing, represents the brightness gradient after normalization processing, represents the clarity after defogging and normalization processing, represents the camera environment penalty index, represents the camera environment penalty factor, represents the weight coefficient and .

5. The obstacle recognition device based on laser point cloud data according to claim 1 or 4, characterized in that, The specific working principle of the fusion weight allocation module is: Import the camera confidence and the lidar confidence into the constructed weight allocation model to obtain the lidar weight coefficient and the camera weight coefficient. The weight allocation model is expressed as: Among them, represents the weight coefficient of the lidar recognition result, represents the weight coefficient of the camera recognition result, represents the startup function, represents the lidar confidence, represents the camera confidence, represents the sensitivity index, represents the confidence, represents the confidence threshold.

6. The obstacle recognition device based on laser point cloud data according to claim 5, characterized in that The specific working principle of the fusion position analysis module: Import the lidar recognition result weight coefficient, the camera recognition result weight coefficient, the recognition coordinates of obstacles by the lidar, and the recognition coordinates of obstacles by the camera into the constructed fusion position model to output the fusion position. The fusion position model is expressed as: Among them, represents the fusion position, represents the recognition coordinates of the obstacle by the lidar, represents the recognition coordinates of the obstacle by the camera, represents the weight coefficient of the lidar recognition result, represents the weight coefficient of the camera recognition result.

7. The obstacle recognition device based on laser point cloud data according to claim 3, characterized in that, Obtain the lidar environmental penalty index based on the current vibration intensity, current temperature, and current weather visibility. Specifically: Perform a difference operation between the current weather visibility and the nominal maximum visibility, and then perform a ratio operation with the nominal maximum visibility to obtain the weather visibility deviation value; Perform a ratio operation between the current vibration intensity and the nominal maximum vibration intensity to obtain the intensity loss value; Perform a difference operation between the current temperature and the optimal temperature, then perform a ratio operation with the difference between the nominal highest temperature and the nominal lowest temperature, and take the absolute value of the obtained ratio to further obtain the temperature deviation value; Perform weighted average processing on the visibility deviation value, intensity loss value, and temperature deviation value to obtain the lidar environmental penalty index.

8. The obstacle recognition device based on laser point cloud data according to claim 4, characterized in that, Obtain the camera environmental penalty index based on the current vibration index and light intensity, specifically: Perform a ratio process on the current vibration intensity and the nominal maximum vibration intensity to obtain the intensity loss value; Perform a ratio process on the absolute value of the difference between the current light intensity and the optimal illumination intensity of the camera and the absolute value of the difference between the maximum light intensity and the minimum light intensity supported by the camera to obtain the light deviation value; Perform weighted average processing on the obtained intensity loss value and light intensity deviation value to further obtain the camera environmental penalty index.

Images