Influence analysis method for intelligent automobile visual sensor in rain and fog environment

Abstract

The invention relates to an influence analysis method of a visual sensor, in particular to an influence analysis method of an intelligent automobile visual sensor for a rain and fog environment, which comprises the following steps of: constructing a test scene based on a rain and fog meteorological environment simulation platform of a closed site; the method comprises the following steps: acquiring data of [rain and fog meteorological parameters-target object distance-image information influenced by meteorology], and analyzing the influence of the rain and fog environment on the intelligent automobile visual sensor through three levels of original data, image features and identification functions, so as to quantify the influence of the rain and fog environment on the intelligent automobile visual sensor in a full-dimension manner; the performance evaluation of the visual sensor in the rain and fog environment is more comprehensive and accurate; and distance parameters are introduced into an influence evaluation range, so that defect analysis and optimization design of a visual sensor algorithm can be better guided. The attenuation effect of the rain and fog environment on the performance of the visual sensor is revealed, and technical support is provided for robustness optimization and test verification of a visual perception system.

Description

Technical Field

[0001] This invention relates to a method for analyzing the impact of visual sensors, and more particularly to a method for analyzing the impact of visual sensors in intelligent vehicles in rainy and foggy environments. Background Technology

[0002] As a crucial information receiving medium between intelligent vehicles and the driving environment, the robustness and reliability of visual sensors directly impact the safety of intelligent vehicles during operation. However, complex environments, coupled with severe weather conditions and complex traffic flows, can easily degrade visual sensor systems, leading to functional safety or anticipated functional safety issues. Rain and fog are extremely common severe weather conditions in the daily driving of intelligent vehicles. Before the actual deployment of intelligent vehicles, quantitatively analyzing the impact of rain and fog on visual sensors is of great significance for improving the component-level, system-level, and vehicle-level safety of intelligent vehicles. Existing analysis methods mostly consider the image quality dimension, analyzing the degradation of the same target object in different rainy or foggy environments compared to clear weather. Existing analysis methods have problems: they only analyze the impact of rain and fog in a single dimension, making it difficult to comprehensively assess the impact of rain and fog on visual sensors and failing to provide effective contributions to the development and application of intelligent vehicles; furthermore, existing methods ignore the distance parameter between the target object and the vehicle, and analysis based solely on different meteorological parameters cannot help in building application scenarios for intelligent vehicle testing. Summary of the Invention

[0003] To address the aforementioned technical problems, this invention provides a method for analyzing the impact of rain and fog on intelligent vehicle vision sensors, comprising the following steps:

[0004] Step 1: Construct a test scenario based on a closed-site rain and fog meteorological environment simulation platform. The test scenario construction and rain and fog environment visual sensor data acquisition process are as follows: Place a static vehicle target object. A vehicle equipped with a visual sensor gradually approaches the target object from a certain distance at a certain speed. First, the visual sensor acquires images of the target object at different distances without rain and fog. Then, turn on the rain or fog simulation device, record the rain and fog meteorological parameters, and acquire rain and fog image data corresponding to different distances of the target object through the visual sensor. During the acquisition process, the distance between the target object and the vehicle in the calibration image is recorded simultaneously. By changing different rain and fog meteorological parameters, the data pairs of [rain and fog meteorological parameters - target object distance - image information affected by meteorological conditions] are repeatedly acquired.

[0005] Furthermore, the collected rain and fog image data Im s Includes rainfall images, denoted as Im sr Fog image, denoted as Im sf .

[0006] Step 2: Analyze the impact of rain and fog on the vision sensors of intelligent vehicles through three different levels: raw data, image features, and recognition functions. The input is the image Im at different target distances without the influence of rain and fog. r and rain and fog image data Im s A combination of images.

[0007] Among them, the raw data hierarchical impact analysis is used to quantify the impact of rain and fog environments on the raw image information output by the vision sensors of intelligent vehicles, and is divided into full-reference indicators and no-reference indicators; the full-reference indicators require an image Im unaffected by rain and fog. r As a benchmark, two methods were used for calculation: Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index (SSIM).

[0008] The no-reference metric uses the Tenengrad gradient evaluation metric, which consists of the horizontal and vertical gradient values ​​of a pixel, and a threshold is set for the gradient. Adjust the sensitivity of the function;

[0009] Furthermore, the PSNR is defined based on the image mean square error (MSE), given an M*N rain- and fog-free image Im. r and rain and fog image data Im s Its MSE is defined as:

[0010]

[0011]

[0012] In the formula, MAX represents the maximum pixel value in the image, and SSIM is defined as:

[0013]

[0014] In the formula, x and y are Im and y, respectively. r and Im s pixel intensity, These represent the means of x and y, respectively. These represent the variances of x and y, respectively. It is the covariance of x and y; Q is a constant that maintains the stability of the SSIM formula; Q is the dynamic range of the image; k1 and k2 are constants.

[0015] Furthermore, the expression for the Tenengrad gradient evaluation index is as follows:

[0016]

[0017] In the formula, For pixels The gradient at point is specifically expressed as:

[0018]

[0019] In the formula, and These are the gradient values ​​in the horizontal and vertical directions of the pixel, respectively, and their specific expressions are as follows:

[0020]

[0021] In the formula, The input image is *, which represents a two-dimensional convolution operation, and S... x and S y These represent the values ​​of the Sobel operator in the horizontal and vertical directions, respectively.

[0022]

[0023]

[0024] The higher the Tenengrad value of the entire image, the higher the overall edge contrast, the clearer the details, and the more accurate the focus.

[0025] Under the condition of the same target distance, for each group of Im r and Im s Image combination was used to calculate three evaluation indicators: peak signal-to-noise ratio (PSNR), image structure similarity (SSIM), and Tenengrad gradient. Target distance and meteorological parameters were used as independent variables, and PSNR, SSIM, and Tenengrad were used as dependent variables. A linear interpolation method was employed to construct the hierarchical impact analysis results of the original data, intuitively revealing the degradation patterns of images under different target distances and meteorological parameter conditions in rain and fog environments in terms of pixel-level differences, structural fidelity, and edge sharpness.

[0026] In the image feature hierarchy impact analysis, two local feature extraction and description algorithms, SIFT and ORB, are introduced. Combined with the collected rain and fog image data, the impact of rain and fog meteorological conditions on the feature detection and matching performance of intelligent vehicle vision sensors is quantitatively evaluated.

[0027] To quantitatively evaluate the degradation effect of corner detection-based feature extraction algorithms, the ORB algorithm is applied to describe the keypoint number decay rate. First, images Im with the same target distance and no rain or fog effects are selected. r and rain and fog image data Im s Key points were extracted using ORB and SITF algorithms respectively; then, the image Im was recorded as unaffected by rain or fog.r The total number of key points detected in the middle is Im s The total number of key points detected in the middle is The retention rate of ORB key points under different target distances and meteorological parameters was calculated. :

[0028]

[0029] Within the same experimental framework, the keypoint retention rate due to rain and fog weather was calculated using the SIFT algorithm. To evaluate the impact of rain and fog environments on the detection performance of scale- and rotation-invariant features, the expression is:

[0030]

[0031] In the formula, Representing Im r and Im s The corresponding SIFT key point recognition results.

[0032] In the functional hierarchy impact analysis, target recognition and distance estimation, two commonly used functions of intelligent vehicle vision sensors, were selected to analyze the impact of rain and fog.

[0033] In the target recognition function impact analysis, the YOLO series algorithms were selected as the object of analysis for the impact of rain and fog on target recognition function. First, an image Im without the influence of rain and fog was obtained. r The ground truth value (GT) of the bounding box of the target in each frame of the image; using the YOLO algorithm to analyze the rain and fog image data. s The detection is performed to obtain the detection bounding box D, and the target recognition algorithm is evaluated using the IOU method:

[0034]

[0035] This is a metric for evaluating the accuracy of target recognition algorithms. The output is a value between 0 and 1; a larger value indicates a more accurate target recognition algorithm. r and Im s The image is used as input to YOLO-V5, and the corresponding output is... Defined as the image output value without the influence of rain or fog. Rain and fog image data output values The impact of rain and fog on target recognition function was analyzed using the degree of recognition reduction. The expression is:

[0036]

[0037] In the impact analysis of distance estimation function, to analyze the influence of rain and fog on the distance estimation function, the true distance value calibrated by the cone was used. and the output of the distance estimation function Perform difference comparison, using Analysis of the impact of rain and fog on distance estimation function Its expression is:

[0038]

[0039] After obtaining the impact analysis results of rain and fog environment on the raw data layer, image feature layer, and recognition function layer of intelligent vehicle vision sensors, the analysis results obtained separately for each layer are first normalized to normalize their range. Then, the results of the three levels of influence are added together to obtain the analysis results that consider the three influencing factors simultaneously. In order to make the expression clearer, the normalization method is used again to obtain the final multi-level influence analysis results of rain and fog environment on intelligent vehicle vision sensor.

[0040] The beneficial effects of this invention are:

[0041] This invention proposes a novel method for analyzing the impact of rain and fog on intelligent vehicle vision sensors. The method first breaks down the analysis into three levels: raw data, image features, and recognition functions. This allows for a comprehensive quantification of the impact of rain and fog on intelligent vehicle vision sensors, resulting in a more complete and accurate performance evaluation of vision sensors in rain and fog conditions. Furthermore, this invention incorporates a distance parameter into the impact assessment, which can better guide the defect analysis and optimization design of vision sensor algorithms. The multi-level impact analysis results of this invention clearly reveal the attenuation effect of rain and fog on vision sensor performance, providing data and methodological support for the robustness optimization and testing of visual perception systems. Attached Figure Description

[0042] Figure 1 This is a schematic diagram of the overall process of the method of the present invention.

[0043] Figure 2 This is a schematic diagram of a test scenario for analyzing the impact of rain and fog environments according to the present invention.

[0044] Figure 3 This is a schematic diagram of the multi-level influence analysis method of the present invention.

[0045] Figure 4 This is a schematic diagram of some image acquisition results in an embodiment of the present invention.

[0046] Figure 5 The image shows the PSNR value results in a rainfall scenario according to an embodiment of the present invention.

[0047] Figure 6 This is the SSIM value result of an image in a rainfall scenario according to an embodiment of the present invention.

[0048] Figure 7 This is the Tenengrad value result of the image in the rainfall scene of the embodiment of the present invention.

[0049] Figure 8 The image shows the PSNR value results in a foggy scene according to an embodiment of the present invention.

[0050] Figure 9 This is the SSIM value result of an image in a fog scene according to an embodiment of the present invention.

[0051] Figure 10 This is the Tenengrad value result of an image in a fog scene according to an embodiment of the present invention.

[0052] Figure 11 In the rainfall scenario of this embodiment of the invention Value result.

[0053] Figure 12 In the rainfall scenario of this embodiment of the invention Value result.

[0054] Figure 13 In the fog scene of the embodiment of the present invention Value result.

[0055] Figure 14 In the fog scene of the embodiment of the present invention Value result.

[0056] Figure 15 In the rainfall scenario of this embodiment of the invention Value result.

[0057] Figure 16 In the rainfall scenario of this embodiment of the invention Value result.

[0058] Figure 17 In the fog scene of the embodiment of the present invention Value result.

[0059] Figure 18 In the fog scene of the embodiment of the present invention Value result.

[0060] Figure 19 This is the result of a multi-level impact analysis in a rainfall scenario according to an embodiment of the present invention.

[0061] Figure 20 This is the result of a multi-level influence analysis in a fog scene according to an embodiment of the present invention. Detailed Implementation

[0062] like Figure 1 As shown in the figure, this embodiment provides a method for analyzing the impact of rain and fog on intelligent vehicle vision sensors, including the following steps:

[0063] (1) Test scenario design for rain and fog environmental impact analysis:

[0064] To accurately analyze the impact of rain and fog environments on intelligent vehicle vision sensors, this invention constructs test scenarios based on a closed-site rain and fog meteorological environment simulation platform, such as... Figure 2 As shown, the rain and fog meteorological environment simulation platform is an existing facility, equipped with a closed site, simulated roads, and rain and fog meteorological environment simulation equipment. The specific test scenario construction and rain and fog environment visual sensor data acquisition process are as follows: First, a vehicle target is statically placed, and the corresponding rain or fog simulation device is activated; then, the vehicle equipped with a visual sensor gradually approaches the target from a distance of 50 meters at a speed of 5 km / h, simultaneously recording the vehicle target image information acquired by the visual sensor; the distance between the vehicle target and the vehicle in the image is calibrated using cones spaced 10 meters apart; finally, by changing different rain and fog meteorological parameters, the above process is repeated to obtain a data pair of [rain and fog meteorological parameters - target distance - image information affected by meteorological conditions], which is then used for analyzing the impact of rain and fog environments on intelligent vehicle visual sensors.

[0065] During the acquisition of visual sensor data in rainy and foggy environments, all image areas except for vehicle targets remain unchanged throughout the acquisition process, eliminating interference from other image areas on the acquisition results. Furthermore, the influence analysis method proposed in this invention requires an image (Im) free from the effects of rain and fog. r Therefore, the rain and fog image data Im was used as a benchmark for analysis. s Previously, it was necessary to collect the Im corresponding to the distance to different vehicle targets. r The collected rain and fog image data Im s Includes rainfall images, denoted as Im sr Fog image, denoted as Im sf .

[0066] (2) Multi-level impact analysis method of rain and fog environment on intelligent vehicle vision sensor

[0067] The impact analysis of rain and fog environments on intelligent vehicle vision sensors constructed in this invention is divided into three different levels: raw data, image features, and recognition function. Its input is an image (Im) of the vehicle at different target distances without the influence of rain and fog. r and rain and fog image data Im s A combination of images.

[0068] First, we introduce the specific impact analysis methods for the three levels, and then we use the normalization method to integrate the results of the multi-level impact analysis.

[0069] (2.1) Analysis of the impact of the original data hierarchy

[0070] Raw data hierarchical impact analysis is used to quantify the impact of rain and fog environments on the raw image information output by intelligent vehicle vision sensors, and is divided into full-reference indicators and no-reference indicators; the full-reference indicators require an image Im unaffected by rain and fog. r As benchmarks, the methods used include Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index (SSIM). PSNR is defined based on the Mean Square Error (MSE) of the image, given a rain- and fog-free image Im of size M*N. r and rain and fog image data Im s Its MSE is defined as:

[0071]

[0072] In the formula, MAX represents the maximum pixel value in the image, which is 255 in this embodiment. Assuming Im is given... r and Im s The pixel intensities are x and y, respectively, and the corresponding SSIM definition is:

[0073]

[0074] In the formula, These represent the means of x and y, respectively. These represent the variances of x and y, respectively. It is the covariance of x and y; Q is a constant that maintains the stability of formula (3); Q is the dynamic range of the image, and in this embodiment Q=255; k1 and k2 are constants, and in this embodiment k1=0.01 and k2=0.02.

[0075] For evaluation without a reference metric, the Tenengrad gradient metric is used. Tenengrad is constructed by extracting the horizontal and vertical gradient values ​​of pixels using the Sobel operator, and a threshold is set for the gradient. The sensitivity of the adjustment function is expressed as follows:

[0076]

[0077] In the formula, For pixels The gradient at point is specifically expressed as:

[0078]

[0079] In the formula, and These are the gradient values ​​in the horizontal and vertical directions of the pixel, respectively, and their specific expressions are as follows:

[0080]

[0081] In the formula, The input image is *, which represents a two-dimensional convolution operation, and S... x and S y These represent the values ​​of the Sobel operator in the horizontal and vertical directions, respectively.

[0082]

[0083] The higher the Tenengrad value of the entire image, the higher the overall edge contrast, the clearer the details, and the more accurate the focus.

[0084] Under the condition of the same target distance, the present invention applies different methods to each group of Im. r and Im s The three evaluation indices, namely Peak Signal-to-Noise Ratio (PSNR), Image Structure Similarity (SSIM), and Tenengrad, were calculated. With target distance and meteorological parameters as independent variables and PSNR, SSIM, and Tenengrad as dependent variables, a linear interpolation method was used to construct the original data hierarchical influence analysis results. The original data hierarchical influence analysis method proposed in this invention can intuitively reveal the degradation laws of images under different target distances and meteorological parameter conditions in rain and fog environments in terms of pixel-level differences, structural fidelity, and edge sharpness.

[0085] (2.2) Analysis of the influence of image feature hierarchy

[0086] In the image feature hierarchy impact analysis, this invention introduces two local feature extraction and description algorithms, SIFT and ORB, and combines them with the collected rain and fog image data to quantitatively evaluate the impact of rain and fog meteorological conditions on the feature detection and matching performance of intelligent vehicle vision sensors.

[0087] Under rainy and foggy weather conditions, the imaging contrast of visual sensors typically decreases significantly, leading to a decline in the performance of feature extraction algorithms based on corner detection. To quantitatively assess this degradation effect, this invention applies the ORB algorithm to describe the rate of keypoint attenuation. The specific process is as follows: First, select an image Im at the same target distance that is unaffected by rain and fog. r and rain and fog image data Im s Keypoints were extracted using both ORB and SITF algorithms; then, Im was recorded. r The total number of key points detected in the middle is Ims The total number of key points detected in the middle is Through calculation:

[0088]

[0089] This allows us to obtain the ORB keypoint retention rate under different target distances and meteorological parameters. Within the same experimental framework, the SIFT algorithm can also be used to calculate the keypoint retention rate to evaluate the impact of rain and fog environments on the performance of scale- and rotation-invariant feature detection. The specific expression is as follows:

[0090]

[0091] In the formula, The keypoint retention rate due to rain and fog weather, obtained using the SITF algorithm; Representing Im r and Im s The corresponding SIFT key point recognition results.

[0092] (2.3) Analysis of the impact of functional hierarchy identification

[0093] In the analysis of the impact of functional hierarchy, this invention selects two commonly used functions of intelligent vehicle vision sensors, namely target recognition and distance estimation, to analyze the impact of rain and fog.

[0094] (2.3.1) Impact analysis of target recognition function

[0095] This invention selects the YOLO series algorithms as the object of analysis for the impact of rain and fog on target recognition. Using MATLAB's GroundTruthLabeler toolkit and after manual correction, an image Im unaffected by rain and fog was obtained. r The ground truth value (GT) of the bounding box of the target in each frame of the image; using the YOLO algorithm to analyze the rain and fog image data. s The detection is performed to obtain the detection bounding box D, and the target recognition algorithm is evaluated using the IOU method:

[0096]

[0097] This is a metric for evaluating the accuracy of target recognition algorithms. The output is a value between 0 and 1; a larger value indicates a more accurate target recognition algorithm. r and Im s The image is used as input to YOLO-V5, and the corresponding output is... Defined respectively and This invention employs an analysis of the impact of rain and fog on target recognition functionality to reduce the degree of recognition degradation. The specific expression is:

[0098]

[0099] (2.3.2) Impact analysis of distance estimation function

[0100] To analyze the impact of rain and fog conditions on distance estimation functionality, this invention utilizes the true distance values ​​calibrated by a cone. and the output of the distance estimation function To perform difference comparison, this invention employs... and Analysis of the impact of rain and fog on distance estimation function Its expression is:

[0101]

[0102] (2.4) Multi-level impact analysis method

[0103] After obtaining the impact analysis results of rain and fog environment on the raw data layer, image feature layer, and recognition function layer of intelligent vehicle vision sensors, the analysis results obtained separately for each layer are first normalized to normalize their range. Then, the results of the three levels of influence are summed to obtain the analysis results considering all three influencing factors. To make the expression clearer, normalization is applied again to obtain the final multi-level influence analysis results of rain and fog environment on intelligent vehicle vision sensors. The method flow is as follows: Figure 3 As shown.

[0104] Impact Analysis Experiment and Results Explanation

[0105] Images of no rain or fog effects at different distances [10 m, 350 m] were collected using a rain and fog meteorological environment simulation platform. r And set different rainfall intensities or fog visibility levels to collect rain and fog image data at different distances from targets. s Some image acquisition results are as follows: Figure 4 As shown. Then Im r and Im s The results of the analysis at each level obtained by inputting the data into the impact analysis method of this invention are as follows: Figure 5-20 As shown.

[0106] For the analysis of the impact of the original data hierarchy, the three indicators PSNR, SSIM, and Tenengrad comprehensively quantify the trend of visual image quality degradation under rain and fog conditions, such as Figure 5-10As shown, in the rainfall scenario, as the rainfall intensity increased from 30 mm / h to 90 mm / h, and the target distance increased from 10 m to 50 m, the image PSNR value decreased from approximately 16.3 dB to a minimum of around 12.6 dB, indicating a significant deterioration in the image signal-to-noise ratio; the SSIM value gradually decreased from 0.7582 to 0.6686, reflecting a gradual loss of image structural consistency; and the Tenengrad value plummeted from approximately 160,000 to below 70,000, indicating severe degradation of image edge details and sharpness. In the fog scenario, the impact on image quality was even more pronounced. As fog visibility decreased from 200 m to 30 m, and the target distance increased, the PSNR value decreased from approximately 20.0 dB to 12.3 dB, the SSIM value decreased from 0.7965 to 0.6806, and the Tenengrad value decreased from approximately 160,000 to below 60,000. When visibility in fog is less than 100 m, all image quality indicators drop rapidly, especially at the distance of distant targets where image quality degrades drastically.

[0107] In image feature level analysis, rain and fog environments significantly interfere with the local feature extraction capabilities of intelligent vehicle vision sensors, such as... Figure 11-14 As shown. This embodiment uses... and The performance changes of two typical feature extraction algorithms under severe weather conditions were quantified. In the rainfall scenario, as the rainfall intensity increased from 30 mm / h to 90 mm / h, and the target distance increased, The percentage dropped rapidly from a high of approximately 62% to a low of less than 20%, exhibiting a clear non-linear decay trend, indicating that raindrop interference has a significant impact on corner-based fast feature extraction methods; meanwhile, The percentage also dropped from over 40% to approximately 21%, indicating that even the SIFT algorithm, which possesses scale invariance and rotation robustness, struggles to cope with image blurring and reduced contrast caused by heavy rain. In foggy scenes, feature extraction capabilities are even more significantly affected. The retention rate dropped from over 60% to about 22%, and there was a precipitous decline when fog visibility was below 100 m; The similar decrease in accuracy from 41% to around 20% indicates that under dense fog conditions, image contrast and texture information are severely lost, making feature point extraction ineffective. Overall, both feature extraction algorithms exhibit a significant performance degradation trend in rainy and foggy environments, with the degree of degradation intensifying with increasing weather intensity and target distance. These results demonstrate that rainy and foggy environments significantly weaken the stability of local image features, impacting the performance of visual perception systems in specific intelligent vehicle driving tasks.

[0108] In the hierarchical analysis of recognition functions, the interference of rain and fog environments on the target recognition and distance estimation functions of intelligent vehicle vision sensors is particularly obvious. Figure 15-18 Showing and The variation of weather intensity and distance to the target object. In a rainfall scenario, as the rainfall intensity increases from 30 mm / h to 90 mm / h, The value decreased from 0.91 to a low of approximately 0.49, exhibiting a dual attenuation trend with increasing target distance and intensified rainfall. This indicates that target identification capability is significantly weakened under conditions of heavy rain and long distance. Meanwhile, The error exhibits a non-linear, fluctuating upward trend, with a significant increase observed during moderate rain. Under heavy rain and distant target conditions, the maximum error approaches 0.9, indicating that visual sensors struggle to provide reliable distance information. In foggy scenarios, as visibility decreases from 200 m to 30 m... The value dropped sharply, from 0.31 to approximately 0.07, with the downward trend being particularly steep when visibility in fog was below 100 m, indicating that dense fog had a suppressive effect on target detection capabilities. Meanwhile, The error increases rapidly, especially under the combined conditions of low visibility and target distance greater than 30 m, where the error value exceeds 0.5. Overall, rain and fog environments severely impact the functionality of visual sensors. The analysis results indicate that in real-world driving scenarios, rain and fog directly threaten the functional safety of visual perception systems, necessitating compensation and optimization through multi-sensor fusion or robust algorithms.

[0109] Figures 19-20 This paper presents the results of a multi-level impact analysis of intelligent vehicle vision sensors under two typical environmental conditions: rainfall and fog. The figure comprehensively reflects the overall performance degradation trend of the vision sensors under different meteorological intensities and target distances. In the rainfall scenario, the performance score shows a significant overall downward trend with increasing rainfall intensity and target distance. The normalized index drops rapidly from an ideal state close to 1.0 as rainfall intensifies, and under heavy rain and target distances exceeding 40 m, the overall performance drops below 0.1. In the fog scenario, the overall performance is highly correlated with visibility. As fog visibility decreases from 200 m to 50 m, the overall index gradually decreases from 0.9 to below 0.25; when the target distance further increases to over 50 m, the performance drops to a minimum of approximately 0.05. The overall surface exhibits a steep slope from the upper right to the lower left, demonstrating the combined attenuation effect of dense fog and long distance on the vision sensor.

[0110] Overall, the multi-level impact analysis results of this invention clearly reveal the attenuation effect of rain and fog environments on the performance of visual sensors. This method effectively integrates three dimensions: raw data, image features, and specific functions, making the performance evaluation of visual sensors in rain and fog environments more comprehensive and accurate. It provides data support and methodological backing for the robustness optimization and testing verification of visual perception systems.

Claims

1. A method for analyzing the impact of rain and fog on intelligent vehicle vision sensors, characterized by the following steps: include: Step 1: Construct a test scenario based on a closed-site rain and fog meteorological environment simulation platform. Place a static vehicle target object. A vehicle equipped with a vision sensor gradually approaches the target object from a certain distance at a certain speed. Collect rain and fog images and rain and fog images corresponding to different vehicle target object distances through the vision sensor. At the same time, record the distance between the vehicle target object and the vehicle in the calibration image. Repeat the collection of data pairs [rain and fog meteorological parameters - target object distance - image information affected by meteorological conditions] by changing different rain and fog meteorological parameters. Step 2: Quantify the impact of rain and fog environment on the raw image information output by intelligent vehicle vision sensor through raw data hierarchical impact analysis, including full reference index and no reference index; The full reference index uses images free from rain and fog as a benchmark and is calculated using two methods: peak signal-to-noise ratio (PSNR) and image structure similarity index (SSIM). The no-reference index uses the Tenengrad gradient evaluation index, which consists of the horizontal and vertical gradient values ​​of pixels. Linear interpolation is used to construct the original data hierarchical impact analysis results. Then, through image feature hierarchy influence analysis, the ORB algorithm is applied to describe the keypoint number decay rate, and the ORB keypoint number retention rate under different target object distances and meteorological parameters is calculated; the SIFT algorithm is used to calculate the keypoint number retention rate caused by rain and fog. Then, through the hierarchical impact analysis of the identification function, the impact of rain and fog on the target identification function is analyzed by selecting two functions: target identification and distance estimation. Among them, the YOLO series algorithm is used as the object of the impact analysis of rain and fog on the target identification function. The impact analysis of the distance estimation function uses the difference between the true distance value calibrated by the cone and the output value of the distance estimation function to analyze the impact of the rain and fog environment on the distance estimation function. After obtaining the impact analysis results of rain and fog environment on the raw data layer, image feature layer and recognition function layer of intelligent vehicle vision sensor, the analysis results obtained separately for each layer are first normalized. Then, the impact results of the three layers are added together to obtain the analysis result that considers the three influencing factors at the same time. The normalization method is used again to obtain the final multi-level impact analysis results of rain and fog environment on intelligent vehicle vision sensor.

2. The method for analyzing the impact of rain and fog on intelligent vehicle vision sensors according to claim 1, characterized in that: In step 1, the collected rain and fog image data includes rainfall images and fog images.

3. The method for analyzing the impact of rain and fog on intelligent vehicle vision sensors according to claim 1, characterized in that: In step 2, the original data hierarchy influence analysis, the PSNR is defined based on the image mean square error (MSE). Given a rain- and fog-free image Im of size M*N... r and rain and fog image data Im s Its MSE is defined as: In the formula, MAX represents the maximum pixel value in the image, and SSIM is defined as: In the formula, x and y are Im and y, respectively. r and Im s pixel intensity, , These represent the means of x and y, respectively. and These represent the variances of x and y, respectively. It is the covariance of x and y; Q is a constant that maintains the stability of the SSIM formula; Q is the dynamic range of the image; k1 and k2 are constants.

4. The method for analyzing the impact of rain and fog on intelligent vehicle vision sensors according to claim 1, characterized in that: In step 2, the original data hierarchical impact analysis is performed, given a rain-free and fog-free impact image Im of size M*N. r and rain and fog image data Im s The Tenengrad gradient evaluation index is expressed as follows: In the formula, For pixels The gradient at point is specifically expressed as: In the formula, and These are the gradient values ​​in the horizontal and vertical directions of the pixel, respectively, and their specific expressions are as follows: In the formula, For the input image, S represents a two-dimensional convolution operation. x and S y These represent the values ​​of the Sobel operator in the horizontal and vertical directions, respectively. The higher the Tenengrad value of the entire image, the higher the overall edge contrast, the clearer the details, and the more accurate the focus.

5. The method for analyzing the impact of rain and fog on intelligent vehicle vision sensors according to claim 1, characterized in that: In step 2, the hierarchical impact analysis of the original data, under the condition of the same target distance, the three indices PSNR, SSIM, and Tenengrad are calculated for each image combination. The target distance and meteorological parameters are used as independent variables, and PSNR, SSIM, and Tenengrad are used as dependent variables. The results of the hierarchical impact analysis of the original data are constructed by linear interpolation.

6. The method for analyzing the impact of rain and fog on intelligent vehicle vision sensors according to claim 1, characterized in that: In step 2, the image feature hierarchy influence analysis first selects an image Im with no rain or fog influence at the same distance from the target object. r and rain and fog image data Im s Key points were extracted using ORB and SITF algorithms, respectively. Then, record the image Im without the influence of rain and fog. r The total number of key points detected in the middle is Im s The total number of key points detected in the middle is The ORB keypoint retention rate was calculated under different target distances and meteorological parameters. : Within the same experimental framework, the keypoint retention rate due to rain and fog weather was calculated using the SIFT algorithm. To evaluate the impact of rain and fog environments on the detection performance of scale- and rotation-invariant features, the expression is: In the formula, and Representing Im r and Im s The corresponding SIFT key point recognition results.

7. The method for analyzing the impact of rain and fog on intelligent vehicle vision sensors according to claim 1, characterized in that: In step 2, the impact analysis of the target recognition function first obtains the image Im, which is free from the influence of rain and fog. r The ground truth value (GT) of the bounding box of the target in each frame of the image; using the YOLO algorithm to analyze the rain and fog image data. s The detection is performed to obtain the detection bounding box D, and the target recognition algorithm is evaluated using the IOU method: This is a metric for evaluating the accuracy of target recognition algorithms. The output is a value between 0 and 1; a larger value indicates a more accurate target recognition algorithm. r and Im s The image is used as input to YOLO-V5, and the corresponding output is... Defined as the image output value without the influence of rain or fog. Rain and fog image data output values ; The impact of rain and fog on target recognition function was analyzed using the degree of recognition reduction. The expression is: In the impact analysis of the distance estimation function, the difference between the true distance value calibrated by the cone and the output value of the distance estimation function was compared. and Analysis of the impact of rain and fog on distance estimation function Its expression is: in, The true value of the distance calibrated for the cone. Output value for the distance estimation function.

Images