Safety processing method and device based on vehicle-mounted rainfall measurement

Abstract

The invention provides a safety processing method and device based on vehicle-mounted rainfall measurement, and relates to the technical field of aided driving, and the method comprises the steps: obtaining a sampling image covering a front windshield sampling region after cleaning of a windscreen wiper, and carrying out the cutting and the pixel-level feature analysis of brightness, gradient, variance and the like on the image, so as to obtain the image; the coverage type of each pixel is jointly judged based on the brightness offset, the local gradient and the neighborhood brightness variance; forming an equivalent coverage pixel number through coverage type conversion, and obtaining a raindrop coverage proportion according to a ratio of the equivalent coverage pixel number to a total pixel point; and mapping the coverage proportion into rainfall information according to the calibration relationship, and comparing the rainfall information with a rainfall boundary threshold step by step to determine the operation state of the auxiliary driving function so as to realize real-time linkage of rainfall assessment and an automatic driving safety strategy. According to the invention, tight coupling of a rainfall assessment mechanism and an automatic driving safety domain can be realized.

Description

Technical Field

[0001] This invention relates to the technical field of driver assistance, and specifically to a safety processing method and device based on vehicle-mounted rain measurement. Background Technology

[0002] As the penetration rate of autonomous vehicles continues to increase, the reliability of vehicles' environmental perception capabilities has become a key factor in ensuring driving safety. Current autonomous driving systems rely on the fusion of multiple sensors, such as cameras, millimeter-wave radar, and lidar, to perceive the surrounding environment. However, their performance is significantly affected by adverse weather conditions such as rain, snow, and fog. Especially in rainy scenarios, camera image quality deteriorates and optical distortion intensifies, potentially reducing target recognition capabilities. The industry generally defines the operational environment for autonomous driving by limiting the boundaries of the Operational Design Domain (ODD). However, accurately defining the safety boundaries in rainy scenarios and enabling intelligent driving systems to recognize boundary changes in real time has become a significant challenge in current autonomous driving safety assurance systems.

[0003] Current mainstream rainfall sensing and measurement technologies include piezoelectric, infrared, and weighing sensors. While these sensors can provide effective measurements under low rainfall conditions, they are still geared towards traditional meteorological or windshield wiper control scenarios. They are insufficient to meet the requirements of autonomous driving, which demand high sensitivity to rainfall changes, fast response times, and deep correlation with the vehicle's perception attenuation. Under conditions of rapidly changing rainfall or complex road conditions, traditional rainfall detection methods alone cannot accurately reflect the true impact of the environment on the sensor's perception capabilities. This makes it difficult to support intelligent driving systems in executing real-time ODD boundary recognition and corresponding safety control strategies. Therefore, a novel rainfall assessment mechanism is urgently needed to achieve close coupling with the autonomous driving safety domain. Summary of the Invention

[0004] This invention provides a safety processing method and device based on vehicle-mounted rainfall measurement, which can achieve close coupling between the rainfall assessment mechanism and the autonomous driving safety domain.

[0005] A first aspect of the present invention provides a safety processing method based on vehicle-mounted rain measurement, the method comprising: After each wiper completes its wiping action, a sampled image covering the sampling area of ​​the front windshield is acquired; Perform pixel-level brightness information parsing and cropping operations on the sampled image to obtain the total number of pixels in the bounded sampling area and extract the brightness information, local gradient and neighborhood brightness variance of the corresponding pixels; Based on the changes in the brightness information and the combined features of the local gradient and the neighborhood brightness variance, the coverage type of each pixel within the defined sampling area is determined; The number of pixels is calculated and statistically analyzed based on the coverage type to form an equivalent number of covered pixels; The ratio of the equivalent coverage pixel count to the total number of pixels is calculated to obtain the raindrop coverage ratio corresponding to the current sampling period; The raindrop coverage ratio is mapped to the rainfall information corresponding to the current sampling period based on a preset calibration relationship; The system compares the rainfall information with a preset rainfall boundary threshold to determine the current safety status of the driver assistance function, and performs driver assistance processing when the rainfall information reaches the corresponding level threshold.

[0006] Based on the above technical solutions, preferably, the step of performing pixel-level brightness information parsing and cropping operations on the sampled image to obtain the total number of pixels in the bounded sampling area and extract the brightness information, local gradient, and neighborhood brightness variance of the corresponding pixels specifically includes: The sampled image is cropped based on the preset coordinates of the bounding sampling area to obtain a cropped image that includes the front windshield sampling area. The total number of pixels in the defined sampling area is obtained by multiplying the number of pixels corresponding to the horizontal width and vertical height within the cropped image. Within the cropped image, the brightness information, the local gradient, and the neighborhood brightness variance are extracted for each pixel. The brightness information is the weighted brightness value obtained by performing a weighted summation on the color values ​​of the pixel. The local gradient is the brightness change amplitude obtained by performing a difference operation on the brightness information in the horizontal and vertical directions, summing the squares of the difference intensities in the two directions, and then performing a square root operation. The neighborhood brightness variance is the brightness dispersion obtained by averaging the brightness information of all pixels in the neighborhood window centered on the pixel, calculating the square of the brightness deviation from the average value, and then averaging the average value.

[0007] Based on the above technical solutions, preferably, the step of determining the coverage type of each pixel within the defined sampling area based on the changes in the brightness information and the combined features of the local gradient and the neighborhood brightness variance specifically includes: A reference brightness background is constructed at the position corresponding to the defined sampling area based on historical sampling images under multiple rainless conditions. Within the current sampling period, the brightness information of each pixel within the defined sampling area is compared with the reference brightness background to obtain the brightness offset. The brightness offset, the local gradient, and the neighborhood brightness variance are used together as feature parameters of the current pixel coverage state. A coverage index is obtained by performing a weighted linear combination on the feature parameters and then applying a nonlinear mapping. A first coverage threshold and a second coverage threshold are set based on the numerical range of the coverage index. Pixels with a coverage index greater than the first coverage threshold are determined as first covered pixels, pixels with a coverage index between the first and second coverage thresholds are determined as second covered pixels, and pixels with a coverage index less than the second coverage threshold are determined as third covered pixels.

[0008] Based on the above technical solutions, preferably, the step of calculating and statistically analyzing pixels according to the coverage type to form an equivalent number of covered pixels specifically includes: Associate the coverage type of each pixel within the defined sampling area with the corresponding coverage index; Assign equivalent coverage weight parameters to each pixel based on the coverage index; For each pixel within the defined sampling area, a pixel-by-pixel traversal process is performed. During the traversal, the coverage type, coverage degree index, and equivalent coverage weight parameter of each pixel are read simultaneously. The equivalent coverage weight parameter is used as the contribution value of the pixel to the number of equivalent covered pixels. The contribution values ​​of all pixels are summed in the spatial dimension to form the number of equivalent covered pixels corresponding to the current sampling period.

[0009] Based on the above technical solutions, preferably, the step of calculating the ratio of the equivalent coverage pixel count to the total number of pixels to obtain the raindrop coverage ratio corresponding to the current sampling period specifically includes: The total number of pixels is determined by multiplying the number of horizontal pixels and the number of vertical pixels within the defined sampling area. The equivalent coverage pixel count is compared with the total number of pixels within the same sampling period to obtain the raindrop coverage ratio corresponding to the current sampling period.

[0010] Based on the above technical solutions, preferably, the step of mapping the raindrop coverage ratio to the rainfall information corresponding to the current sampling period according to a preset calibration relationship specifically includes: Based on multiple calibration scenarios with different physical rainfall intensities, corresponding real rainfall measurement data and raindrop coverage ratio data within the corresponding sampling period are collected, and a calibration relationship is established between the real rainfall measurement data and the raindrop coverage ratio data. Substitute the raindrop coverage ratio into the calibration relationship, and perform multiplication and addition calculations by combining the ratio parameters and offset correction parameters obtained through the calibration process to form the rainfall information corresponding to the current sampling period, so that the rainfall information can characterize the rainfall intensity based on the equivalent coverage pixel number and the total number of pixels.

[0011] Based on the above technical solutions, preferably, the step of comparing the rainfall information with a preset rainfall boundary threshold to determine the current safety status of the assisted driving function, and performing assisted driving processing when the rainfall information reaches the corresponding level threshold, specifically includes: The rainfall information is compared step by step with multiple rainfall boundary thresholds set in ascending order of rainfall amount. When the rainfall information is less than the first rainfall boundary threshold, the current normal operation of the assisted driving function is maintained so that the vehicle continues to work according to the existing functional configuration. When the rainfall information enters the second rainfall boundary threshold range, the assisted driving function is downgraded and a downgrade prompt is output through the human-machine interface. When the rainfall information reaches or exceeds the third rainfall boundary threshold, the driver assistance function exit process is executed and the driver is reminded to take over vehicle control.

[0012] In a second aspect of the invention, a safety processing apparatus based on vehicle-mounted rainfall measurement is provided. The apparatus is used to perform a safety processing method based on vehicle-mounted rainfall measurement as described in any of the preceding embodiments. The apparatus includes an acquisition module, a processing module, and an output module, wherein: The acquisition module is used to acquire a sampled image covering the sampling area of ​​the windshield after each wiper operation. The acquisition module is used to perform pixel-level brightness information parsing and cropping operations on the sampled image, obtain the total number of pixels in the bounded sampling area and extract the brightness information, local gradient and neighborhood brightness variance of the corresponding pixels; The processing module is used to determine the coverage type of each pixel within the bounded sampling area based on the changes in the brightness information and the combined features of the local gradient and the neighborhood brightness variance. The processing module is used to perform statistical calculations on the pixels according to the coverage type to form an equivalent number of covered pixels. The processing module is used to calculate the ratio of the equivalent coverage pixel count to the total number of pixels to obtain the raindrop coverage ratio corresponding to the current sampling period. The processing module is used to map the raindrop coverage ratio to the rainfall information corresponding to the current sampling period according to a preset calibration relationship; The output module is used to compare the rainfall information with a preset rainfall boundary threshold to determine the current safety status of the assisted driving function, and to perform assisted driving processing when the rainfall information reaches the corresponding level threshold.

[0013] In a third aspect of the invention, an electronic device is provided, including a processor, a memory, a user interface, and a network interface, wherein the memory is used to store instructions, the user interface and the network interface are both used to communicate with other devices, and the processor is used to execute the instructions stored in the memory to cause the electronic device to perform the method as described in any of the preceding embodiments.

[0014] In a fourth aspect of the invention, a non-transitory computer-readable storage medium is provided, the computer-readable storage medium storing instructions that, when executed, perform the method as described in any of the preceding claims.

[0015] In summary, one or more technical solutions provided in the embodiments of the present invention have at least the following technical effects or advantages: 1. This invention assesses rainfall directly based on the raindrop coverage characteristics of the windshield sampling area within the vehicle's visual perception link. This establishes a pixel-level continuous correlation between the acquired image, brightness changes, raindrop coverage type determination, equivalent coverage pixel number calculation, and raindrop coverage ratio formation. The mapped rainfall information serves as the direct trigger for assisted driving safety strategies, and is compared step-by-step with safety operation boundary thresholds set according to increasing rainfall amounts. This allows the rainfall assessment results to reflect the degree of camera image degradation and changes in environmental perception capabilities in real time. By closely linking the rainfall measurement output with assisted driving function degradation or exit strategies, this invention achieves deep coupling between the rainfall assessment mechanism and the autonomous driving safety domain, effectively ensuring the safe operation of vehicles in rainy conditions.

[0016] 2. By cropping the sampled image, performing pixel-level brightness analysis, and extracting local texture features, the impact of raindrops can be accurately captured at the pixel scale, avoiding interference from irrelevant areas of the entire image and improving the spatial resolution of raindrop coverage recognition. The multi-dimensional features composed of brightness information, local gradient, and neighborhood brightness variance can simultaneously reflect the brightness attenuation, edge structure changes, and local texture dispersion caused by raindrops, making subsequent coverage type judgment more accurate and robust.

[0017] 3. By establishing a normal pre-rain brightness benchmark based on the reference brightness background and calculating the brightness offset within the current sampling period, and then combining the local gradient and the neighborhood brightness variance to form coverage feature parameters, it is possible to effectively distinguish between real raindrop interference and ambient light changes, so that the coverage index can accurately reflect the degree of raindrop occlusion; the threshold segmentation judgment mechanism based on the coverage index maps continuous features to discrete coverage types, making the coverage status classification more stable, thus providing a reliable basis for subsequent area conversion.

[0018] 4. By assigning equivalent coverage weight parameters to pixels of different coverage types and accumulating them according to weight during pixel-by-pixel traversal to form the equivalent coverage pixel count, the area contribution of fully covered, partially covered, and uncovered areas can be reflected proportionally, achieving a fine quantification of raindrop coverage area; the equivalent coverage weight is associated with the coverage degree index, allowing complex areas such as raindrop edge areas and thin water film areas to participate in the calculation according to the true coverage degree, improving the authenticity of raindrop coverage area statistics.

[0019] 5. By calculating the ratio of the equivalent coverage pixels to the total number of pixels to form the raindrop coverage ratio, the coverage area statistics are converted into a standardized index that is independent of the size of the sampling area. This ratio is comparable across different sampling periods and can reflect the dynamic changes in raindrop coverage over time. As a direct input for rainfall estimation, it ensures the continuity and consistency of rainfall intensity assessment.

[0020] 6 By setting multiple rainfall boundary thresholds and comparing them step by step with rainfall information, the autonomous driving function can automatically enter normal operation, function downgrade or function exit state according to different rainfall levels, realizing dynamic safety management in rainy weather. This mechanism enables rainfall assessment to directly drive the switching of assisted driving strategies, ensuring that the vehicle takes timely safety measures when its perception ability is reduced or severely limited, effectively improving the overall safety of autonomous driving in rainy weather scenarios. Attached Figure Description

[0021] Figure 1 This is a flowchart illustrating a safety processing method based on vehicle-mounted rain measurement disclosed in an embodiment of the present invention; Figure 2 This is a schematic diagram of a safety processing device based on vehicle-mounted rain measurement disclosed in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of an electronic device disclosed in an embodiment of the present invention.

[0022] Explanation of reference numerals in the attached drawings: 201, acquisition module; 202, processing module; 203, output module; 301, processor; 302, communication bus; 303, user interface; 304, network interface; 305, memory. Detailed Implementation

[0023] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0024] In the description of the embodiments of the present invention, words such as "for example" or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "for example" or "for instance" in the embodiments of the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Rather, the use of words such as "for example" or "for instance" is intended to present the relevant concepts in a specific manner.

[0025] In the description of the embodiments of the present invention, the term "multiple" means two or more. For example, multiple systems means two or more systems, and multiple screen terminals means two or more screen terminals. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the indicated technical features. Thus, a feature defined with "first" or "second" may explicitly or implicitly include one or more of that feature. The terms "comprising," "including," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0026] Autonomous vehicles face challenges such as reduced camera image quality and diminished perception capabilities when driving in rainy weather. While traditional piezoelectric, infrared, and weighing rain sensors can measure rainfall data, they lack a deep connection with the safety requirements of intelligent driving systems. They are unable to accurately reflect the impact on environmental perception capabilities under rapidly changing rainfall or adverse road conditions. Therefore, they cannot effectively support real-time identification and strategy adjustment of the ODD safety boundary in rainy weather for autonomous driving. There is an urgent need for a rainfall assessment method that is closely linked to the autonomous driving safety domain and has high sensitivity and rapid response capabilities.

[0027] This embodiment discloses a safety processing method based on vehicle-mounted rain measurement, referring to... Figure 1 This includes the following steps S110-S170: S110: After each wiper operation, acquire a sampled image covering the sampling area of ​​the windshield.

[0028] This invention discloses a security processing method based on vehicle-mounted rainfall measurement, which is applied to a server. The server includes, but is not limited to, electronic devices such as mobile phones, tablets, wearable devices, and PCs (Personal Computers), and can also be a backend server running the security processing method based on vehicle-mounted rainfall measurement. The server can be implemented using a standalone server or a server cluster composed of multiple servers.

[0029] After a complete wiping action by the windshield wipers, most of the existing raindrops, water film, and optical scattering caused by raindrops on the windshield surface are removed, providing a clean and stable initial background for subsequent imaging of new raindrops related to the current rainfall. At this moment, a simultaneous image acquisition operation is performed based on a vision sensor installed in the windshield's field of view, ensuring that the acquired sampled image accurately represents the latest raindrop adhesion state after the wiper action. The vision sensor is typically a high dynamic range (HDR) camera used in autonomous driving systems. This sensor images with a fixed exposure configuration and outputs a complete image containing the windshield area in RAW or YUV format to the image processing unit. To ensure the stability of the sampling area's position, during the calibration phase, fixed coordinates of the sampling area in the image are determined based on the camera's installation position, the vehicle's windshield geometry, and the imaging optical path. This ensures that the image acquisition action is performed on the same windshield area in each sampling cycle, thereby guaranteeing strict spatial consistency between the sampling area and subsequent pixel-level raindrop coverage analysis.

[0030] S120 performs pixel-level brightness information parsing and cropping operations on the sampled image to obtain the total number of pixels in the bounded sampling area and extract the brightness information, local gradient, and neighborhood brightness variance of the corresponding pixels.

[0031] In one possible implementation, pixel-level brightness information parsing and cropping operations are performed on the sampled image to obtain the total number of pixels in the defined sampling area and extract the brightness information, local gradient, and neighborhood brightness variance of the corresponding pixels. Specifically, this includes: cropping the sampled image based on the preset coordinates of the defined sampling area to obtain a cropped image containing the windshield sampling area; counting the product of the number of pixels corresponding to the horizontal width and vertical height in the cropped image to obtain the total number of pixels in the defined sampling area; extracting brightness information, local gradient, and neighborhood brightness variance for each pixel in the cropped image, where the brightness information is the weighted brightness value obtained by performing a weighted summation of the color values ​​of the pixel; the local gradient is the brightness change amplitude obtained by performing a difference operation on the brightness information in the horizontal and vertical directions respectively, summing the squares of the difference in the two directions, and then performing a square root operation; and the neighborhood brightness variance is the brightness dispersion obtained by averaging the brightness information of all pixels in the neighborhood window centered on the pixel, calculating the square of the brightness deviation from the average value, and then taking the average value.

[0032] Specifically, during the vehicle calibration and algorithm configuration phase, based on the installation posture, imaging resolution, and field of view of the vision sensor relative to the windshield, the coordinates of the bounding sampling area corresponding to the windshield sampling region are determined in the camera imaging coordinate system. The bounding sampling area coordinates are defined as a combination of the upper left corner pixel coordinates and the region width and height. For example, the upper left corner reference point is represented by row and column coordinates, and the region size is represented by the number of horizontal and vertical pixels. This combination parameter is then stored in the algorithm configuration file or the non-volatile memory of the electronic control unit. When a complete sampling image is acquired after each wiper operation, the image processing unit calls the preset bounding sampling area coordinates in the image coordinate system and performs a rectangular cropping operation on the sampling image, retaining only the image sub-block covering the windshield sampling area. This image sub-block is defined as the cropped image, thus ensuring that all subsequent pixel-level brightness information parsing, local gradient calculation, and neighborhood brightness variance statistics are limited to the same physical area. This avoids irrelevant pixels introduced by camera shake or field of view edge interference from participating in the calculation, ensuring that the "bounding sampling area" and "cropped image" maintain spatial consistency throughout the entire processing flow.

[0033] After obtaining the cropped image, in order to use a unified spatial reference in subsequent equivalent coverage pixel count and raindrop coverage ratio calculation, the image processing unit first parses the resolution parameters of the cropped image, reads the number of pixels horizontally and vertically in the cropped image from the image buffer or image header information, and records the number of pixels horizontally as the pixel length of the bounding sampling region in the horizontal direction, and the number of pixels vertically as the pixel length of the bounding sampling region in the vertical direction; then, the total number of pixels in the bounding sampling region is obtained through a product operation, and the total number of pixels is recorded as:

[0034] in, This represents the number of pixels in the cropped image horizontally, and its value is a positive integer. The value represents the number of pixels in the cropped image in the vertical direction and is a positive integer; N represents the total number of pixels in the bounded sampling area, which is equal to the number of all pixels in the area that can participate in brightness analysis and coverage determination. This product operation makes the "total number of pixels" used in the subsequent raindrop coverage ratio calculation correspond one-to-one with the actual set of pixels participating in the statistics, thereby establishing a clear quantitative relationship between the preceding and following steps.

[0035] Within the cropped image, to construct luminance information, local gradient, and neighborhood luminance variance features for each pixel that can be used to determine the coverage type, the image processing unit first analyzes the luminance information of the color value of each pixel. When the camera output format is YUV, the Y component is directly read as luminance information. When the output format is RGB, a weighted summation operation is performed on the red, green, and blue components of the pixel to construct a weighted luminance value, for example, at pixel coordinates. The brightness information at that location can be written as:

[0036] in, , , These represent the pixel coordinates of the cropped image. The red, green, and blue component values ​​at a given location are determined by the image quantization bit width; for example, in a common eight-bit scenario, the value range is from zero to 255. , , This represents the weighting coefficients used to map the three-channel color values ​​to luminance information. Their values ​​can be set to a proportional relationship that makes the human eye more sensitive to the green channel; for example, making... Greater than and The specific values ​​of the weighting coefficients are set through camera calibration or industry standards; Indicates pixel coordinates The weighted brightness value obtained is used to characterize the brightness intensity of the pixel in the current sampling period. By performing a weighted summation on the three-channel color values, the brightness information reflects both the overall lighting conditions and is sensitive to the brightness attenuation caused by raindrops, thus providing a unified brightness basis for subsequent gradient analysis and variance analysis.

[0037] After obtaining the brightness information, in order to characterize the spatial brightness variation features caused by the raindrop edges, local gradient calculations are performed on the brightness information of each pixel within the cropped image. The image processing unit calculates the intensity of brightness variation in the horizontal and vertical directions using horizontal and vertical differential operators, respectively, and then calculates the intensity in the pixel coordinates. The intensity of the change in brightness in the horizontal direction is denoted as The intensity of the change in brightness in the vertical direction is denoted as Then, the local gradient magnitude is obtained by summing the squares of the changes in intensity in both directions and taking the square root.

[0038] in, Indicates pixel coordinates The brightness difference result along the horizontal direction can be obtained by performing convolution kernel operation or difference operation between the pixel and its left and right adjacent pixels. The larger the value, the more drastic the change in brightness in the horizontal direction. Indicates pixel coordinates The brightness difference result along the vertical direction can be obtained by performing convolution kernel operation or difference operation between the pixel and its upper and lower adjacent pixels. The larger the value, the more obvious the brightness change in the vertical direction. Indicates pixel coordinates The local gradient magnitude is a scalar of brightness change magnitude obtained by synthesizing the brightness change intensity in two directions. The larger the value, the more the pixel is located in a region of abrupt brightness change, usually corresponding to the edge of a raindrop or a high-texture region. The smaller the value, the more the pixel is located in a region of gradual brightness change, usually corresponding to a uniformly covered region or a background region. The spatial change structure of brightness information is introduced into the coverage type determination process through this local gradient magnitude.

[0039] After constructing the brightness information and local gradient, to further characterize the brightness fluctuations caused by raindrops in the spatial neighborhood, a neighborhood window centered on each pixel is established within the cropped image. The average and variance of the neighborhood brightness are calculated for all pixels within the neighborhood window, and then... (The sentence is incomplete and requires further context to be translated accurately.) Let the neighborhood window set be denoted as The average brightness of the neighborhood is denoted as The neighborhood brightness variance is denoted as The average brightness of the neighborhood can be expressed as:

[0040] The neighborhood brightness variance can be expressed as:

[0041] in, Represented in pixel coordinates A set of all pixel coordinates within a neighborhood window centered on a fixed size. The window size can be chosen based on the algorithm's trade-off between spatial smoothness and detail preservation, such as 3x3 or 5x5. It represents the number of pixels within the neighborhood window and is a positive integer. Represents the pixel coordinates within the neighborhood window Brightness information at the location; Represents the coordinates around the pixel The average level of brightness information within the neighborhood window, which numerically describes the overall brightness intensity of the local area; This indicates the degree of dispersion of brightness information within the neighborhood window relative to the average brightness of the neighborhood. A larger value indicates more drastic changes in brightness within the neighborhood, which is commonly seen at the edges of raindrops or in mixed coverage areas. A smaller value indicates a more uniform distribution of brightness within the neighborhood, which is commonly seen inside fully covered areas or in uncovered background areas. By using the neighborhood brightness variance as a feature of neighborhood brightness dispersion, it is possible to distinguish between uniformly darkened areas and edge transition areas in coverage type determination.

[0042] S130 determines the coverage type of each pixel within the bounded sampling area based on the changes in brightness information and the combined characteristics of local gradient and neighborhood brightness variance.

[0043] In one possible implementation, the coverage type of each pixel within the defined sampling area is determined based on the changes in brightness information and the combined features of local gradient and neighborhood brightness variance. Specifically, this includes: constructing a reference brightness background at the position corresponding to the defined sampling area based on historical sampling images under multi-frame rainless conditions; comparing the brightness information of each pixel within the defined sampling area with the reference brightness background to obtain a brightness offset within the current sampling period; using the brightness offset, local gradient, and neighborhood brightness variance as feature parameters of the current pixel coverage state; performing a weighted linear combination on the feature parameters and obtaining a coverage index through nonlinear mapping; setting a first coverage threshold and a second coverage threshold based on the numerical range of the coverage index, such that pixels with a coverage index greater than the first coverage threshold are determined as first covered pixels, pixels with a coverage index between the first and second coverage thresholds are determined as second covered pixels, and pixels with a coverage index less than the second coverage threshold are determined as third covered pixels.

[0044] Specifically, when the vehicle is in a rain-free condition or experiencing only very light rain before raindrops have significantly adhered to the windshield, the vision sensor acquires historical sampling images covering a defined sampling area over multiple consecutive sampling cycles. The image processing unit then establishes a reference brightness background for each pixel location within the defined sampling area. Specifically, a time-averaged calculation is performed on the brightness information of the same pixel coordinate location across multiple frames of historical sampling images. The brightness information of each frame is accumulated along the time axis, resulting in a long-term stable reference brightness value for each pixel location. For example, at pixel coordinates... The reference brightness background value can be obtained using the following formula:

[0045]

[0046] in, This indicates the pixel coordinates under rainless conditions. The established reference brightness background value, where M represents the number of historical sampled images included in the statistics. This represents the pixel coordinates of the historical sampled image in frame k. The brightness information at the location; in order to adapt the reference brightness background to slow changes in ambient light such as changes in sunlight and streetlights, a recursive update method can be used to make minor adjustments to the reference brightness background during long-term system operation. For example, at the t-th update time, the reference brightness background is updated to:

[0047] in, This represents the reference brightness background value at the current update moment. This represents the reference brightness background value at the last update time. Indicates the current sampling period in pixel coordinates Brightness information at the location, This represents the background update coefficient, which typically ranges from zero to one and is relatively small to ensure that the reference brightness background changes slowly. The reference brightness background matrix obtained in the above way provides a normal brightness benchmark for each pixel position in the entire bounded sampling area when it is not affected by raindrops, so that subsequent offset analysis of brightness information has a unified reference level.

[0048] Within each new sampling period, after obtaining the brightness information for the current sampling period for the defined sampling area, the image processing unit compares the current brightness information with the reference brightness background pixel by pixel to calculate the brightness offset describing the degree of raindrop disturbance. For any pixel coordinate within the defined sampling area... First, calculate the difference between the current brightness and the reference brightness and then normalize it to reduce the impact of different brightness levels on the offset value. For example, a normalized brightness offset can be constructed:

[0049]

[0050] in, Indicates pixel coordinates The brightness offset at a given point describes the degree of relative change in the current brightness relative to the reference brightness. The larger the value, the more significant the impact of raindrop occlusion, refraction, or scattering on the brightness of that pixel. Indicates the current sampling period in pixel coordinates The brightness information at that location is either a weighted brightness value obtained by weighting the color components or a brightness component directly read from the YUV format; This represents the reference brightness value for the corresponding pixel position in the reference brightness background; This represents a tiny positive number to prevent numerical instability caused by an excessively small denominator. It can be set to a constant much smaller than one, depending on the image quantization range. By constructing a brightness offset, sensitivity to local brightness changes can be maintained under different ambient light conditions, allowing the distinction between local brightness attenuation caused by raindrops and overall illumination changes.

[0051] After obtaining the brightness offset, in order to comprehensively utilize the temporal variation characteristics and spatial structure characteristics of brightness information, the image processing unit, for each pixel within the defined sampling area, combines the brightness offset, local gradient, and neighborhood brightness variance to construct feature parameters describing the current pixel's coverage state. For any pixel coordinate... The three features can be combined into a single feature vector:

[0052]

[0053] in, Indicates pixel coordinates The feature parameter vector at the location; This represents the brightness offset, used to reflect the intensity of the brightness change caused by raindrops; The local gradient magnitude is used to characterize the intensity of brightness change of the pixel in the horizontal and vertical directions. It is obtained by performing spatial difference on the brightness information. The neighborhood brightness variance represents the dispersion of brightness distribution within the neighborhood centered on a given pixel. It is obtained by averaging the squared deviations of the neighborhood brightness information relative to the average brightness of the neighborhood. If necessary, to improve the comparability of different components at the same scale, the three features can be normalized or standardized separately when constructing the feature parameters, ensuring they fall within similar numerical ranges. This avoids excessive weight bias for any component during subsequent weighted combination. By combining brightness offset, local gradient, and neighborhood brightness variance, the feature parameters can reflect both temporal brightness changes and spatial edge strength and local texture fluctuations, providing comprehensive input for subsequent coverage index calculations.

[0054] After obtaining the feature parameters, in order to convert the multidimensional feature parameters into scalar quantization results that can directly characterize the degree of raindrop coverage of the current pixel, a weighted linear combination of the feature parameters is performed in the image processing unit, and compression is carried out through a nonlinear mapping function to unify and integrate the contributions of different features in coverage determination. For any pixel coordinate We can first construct intermediate quantities for linear combinations:

[0055]

[0056] Then, by using a nonlinear mapping function, the intermediate quantity is mapped to a value between zero and one, thus obtaining the coverage index:

[0057] in, The intermediate result of the linear combination is the result of summing the brightness offset, local gradient and neighborhood brightness variance according to weights and adding a bias term. , , The weight parameters for the corresponding brightness offset, local gradient, and neighborhood brightness variance are determined through offline calibration or training. Their values ​​reflect the importance of each feature in coverage determination. b represents the bias parameter, which is used to adjust the baseline level of the linear combination output. To represent a nonlinear mapping function, a mapping form with S-shaped curve characteristics can be selected. For example, the output is close to zero when the input is small and close to one when the input is large, so that the intermediate quantity is compressed into a finite range. Indicates pixel coordinates The coverage index at a given location quantifies the degree to which a pixel is covered by raindrops. A value closer to one indicates near-complete coverage, while a value closer to zero indicates near-uncovered coverage. Values ​​in between correspond to partial coverage. This weighted linear combination and non-linear mapping method allows for the transformation of different feature parameters into a single-dimensional coverage index while preserving their physical meaning, facilitating subsequent threshold segmentation.

[0058] After obtaining the coverage index, in order to classify continuous coverage index values ​​into discrete coverage types, a first coverage threshold and a second coverage threshold are set in the image processing unit according to the numerical range of the coverage index, thereby dividing each pixel into a first covered pixel, a second covered pixel, or a third covered pixel. Specifically, two threshold parameters that satisfy a preset size relationship are selected, the higher threshold is recorded as the first coverage threshold, and the lower threshold is recorded as the second coverage threshold. Typically, the first coverage threshold is greater than the second coverage threshold, and both are between zero and one. For any pixel coordinate... When the coverage index When the coverage value exceeds the first coverage threshold, the pixel is identified as a first-coverage pixel, indicating that the pixel is highly likely to be within the area completely covered by raindrops; when the coverage index... When the pixel's value falls between the first and second coverage thresholds, it is classified as a second-coverage pixel, indicating that the pixel is located at the edge of a raindrop or within a thin film of water, representing a partial coverage state. When the coverage index... If the value is less than the second coverage threshold, the pixel is classified as a third-coverage pixel, indicating that the pixel is basically not covered by raindrops and is only affected by overall changes in ambient light. The first and second coverage thresholds can be determined through statistical analysis of labeled data for different rainfall scenarios. The goal is to ensure that the region corresponding to the first-coverage pixel has characteristics of significant brightness attenuation, small local gradient, and small neighborhood brightness variance; the region corresponding to the second-coverage pixel has characteristics of significant brightness shift and large local gradient and neighborhood brightness variance; and the region corresponding to the third-coverage pixel has characteristics of small brightness shift and neither local gradient nor neighborhood brightness variance meets the raindrop characteristic conditions.

[0059] S140, calculates and statistically analyzes the pixels according to the coverage type to form an equivalent number of coverage pixels.

[0060] In one possible implementation, the pixels are converted and statistically analyzed based on the coverage type to form the equivalent coverage pixel count. Specifically, this includes: associating the coverage type of each pixel within the defined sampling area with the corresponding coverage index; assigning an equivalent coverage weight parameter to each pixel based on the coverage index; performing a pixel-by-pixel traversal process for each pixel within the defined sampling area, simultaneously reading the coverage type, coverage index, and equivalent coverage weight parameter of each pixel during the traversal, using the equivalent coverage weight parameter as the contribution value of that pixel to the equivalent coverage pixel count, and summing the contribution values ​​of all pixels in the spatial dimension to form the equivalent coverage pixel count corresponding to the current sampling period.

[0061] Specifically, given a determined coverage type, the first step is to establish a binding relationship between coverage type and coverage degree index for each pixel within the defined sampling area in the storage structure. This involves recording the corresponding coverage type marker and coverage degree index value for each pixel's coordinates. The coverage type marker indicates whether the pixel belongs to the first, second, or third covered pixel, while the coverage degree index quantifies the degree to which the pixel is covered by raindrops, with a value ranging from zero to one. In implementation, a set of data can be stored for each pixel location in a two-dimensional array with the same resolution as the cropped image. This data includes a coverage type marker field and a coverage degree index field, allowing for simultaneous acquisition of both the coverage type and coverage degree index when accessing any pixel during subsequent calculations and statistics. This ensures that the allocation of the equivalent coverage weight parameter is strictly based on the previous coverage determination results, avoiding misalignment between the coverage type and coverage degree index.

[0062] After associating the coverage type with the coverage index, an equivalent coverage weight parameter is assigned to each pixel based on the coverage index. This parameter maps the coverage state to the degree of contribution to the number of pixels with equivalent coverage. Specifically, a first weight is assigned to the pixels of the first covered pixel, a second weight to the pixels of the second covered pixel, and a third weight to the pixels of the third covered pixel. The first weight is usually set to a constant close to one, indicating that the first covered pixel is equivalent to a full pixel in the equivalent area statistics. The third weight is usually set to a constant close to zero, indicating that the third covered pixel does not contribute effectively to the equivalent area statistics. The second weight is adaptively adjusted based on the coverage index, so that the contribution of the second covered pixel in the equivalent area statistics is between full coverage and no coverage, which can be used to construct pixel positions. The equivalent coverage weight parameter is a piecewise function:

[0063]

[0064] in, Indicates pixel coordinates Equivalent coverage weight parameters at the location; This represents the first weight, which is a preset constant, usually with a value close to one, used to ensure that the first covered pixel is counted according to the complete coverage area; This represents the third weight, which is a preset constant, usually taking a value of zero or close to zero, used to ensure that the contribution of the third covered pixel in the statistics is zero; This is an indicator representing the coverage level of a pixel, used to quantify the coverage level. and This represents the linear adjustment coefficient used to construct the second weight. It can be set according to calibration data to a value between zero and one. The larger the coverage index, the closer the second weight is to the first weight; the smaller the coverage index, the closer the second weight is to the third weight. Through this allocation method, the equivalent coverage weight parameter of each pixel corresponds one-to-one with its coverage type and coverage index.

[0065] After the equivalent coverage weight parameters are assigned, a pixel-by-pixel traversal is performed for each pixel within the defined sampling area. During the traversal, the coverage type, coverage degree index, and equivalent coverage weight parameters are read simultaneously for each pixel's coordinates. The equivalent coverage weight parameter is then used as the contribution of that pixel to the total number of pixels with equivalent coverage. The traversal can be performed in row-major or column-major order. The equivalent coverage weight parameters for each pixel are accumulated, and the accumulated result is the number of pixels with equivalent coverage corresponding to the current sampling period. This allows all pixels within the defined sampling area to be grouped into a set. Then the equivalent number of pixels covered in the current sampling period can be expressed as:

[0066]

[0067] in, It represents the number of equivalent coverage pixels corresponding to the current sampling period, which is the spatial summation result of the equivalent coverage weight parameters of all pixels within the defined sampling area; This represents the set of all pixel coordinates within the defined sampling area, where each element is a pair of pixel coordinates; Indicates pixel coordinates The equivalent coverage weight parameter at each location. By traversing and accumulating pixels one by one, discrete pixel-level coverage states can be converted into an equivalent number of pixels to describe the overall coverage area.

[0068] During the accumulation process, all first-coverage pixels are fully included in the equivalent coverage pixel count according to their first weight. This means the equivalent coverage weight parameter of the first-coverage pixels is always equal to the first weight, ensuring that pixels located within the complete raindrop projection area contribute an area close to that of a full pixel to the equivalent coverage pixel count. All second-coverage pixels are partially included in the equivalent coverage pixel count according to their second weight, adjusted by the coverage index. This means the equivalent coverage weight parameter of the second-coverage pixels continuously changes between zero and the first weight as the coverage index changes, ensuring that pixels located at the raindrop edge or in a thin water film region participate in the area statistics proportionally to their coverage level, thus reflecting the contribution of partially covered areas to the equivalent coverage area. All third-coverage pixels, because their equivalent coverage weight parameter is the third weight, which is preset to zero or close to zero, do not make a substantial contribution to the equivalent coverage pixel count during accumulation and exist only as background areas. Through this method, the equivalent coverage pixel count retains the area information of the completely covered area while incorporating the contribution of partially covered areas through weight adjustment, establishing an approximate correspondence between the equivalent coverage pixel count and the actual raindrop coverage area.

[0069] S150 calculates the ratio of the equivalent coverage pixel count to the total number of pixels to obtain the raindrop coverage ratio corresponding to the current sampling period.

[0070] In one possible implementation, the ratio of the equivalent coverage pixel count to the total number of pixels is calculated to obtain the raindrop coverage ratio corresponding to the current sampling period. Specifically, this includes: determining the total number of pixels by multiplying the number of horizontal pixels and the number of vertical pixels within the defined sampling area; and performing a ratio calculation between the equivalent coverage pixel count and the total number of pixels within the same sampling period to obtain the raindrop coverage ratio corresponding to the current sampling period.

[0071] Specifically, after cropping the windshield sampling area, the image processing unit determines the number of pixels in the horizontal and vertical directions of the defined sampling area based on the resolution parameters of the cropped image. The horizontal pixel count corresponds to the number of columns in the cropped image, and the vertical pixel count corresponds to the number of rows; both are positive integers used to describe the discrete sampling density of the defined sampling area along the horizontal and vertical directions. To obtain the total number of pixels in the defined sampling area, the horizontal and vertical pixel counts are multiplied, treating the area as a two-dimensional pixel array divided by a regular grid. Each discrete pixel unit in the array is considered an independent sampling point, thus obtaining the size of the complete pixel set participating in the raindrop coverage analysis. The total number of pixels can be represented as:

[0072]

[0073] Where N represents the total number of pixels in the defined sampling area, which is the number of reference pixels used in the subsequent calculation of raindrop coverage ratio; This indicates the number of horizontal pixels in the cropped image, corresponding to the pixel length of the defined sampling region in the horizontal direction; This represents the vertical pixel count of the cropped image, corresponding to the pixel length of the defined sampling area in the vertical direction. Through this product relationship, the total number of pixels is made completely consistent with the spatial area actually involved in brightness information analysis, coverage type determination, and equivalent coverage statistics, providing a unified reference for normalizing the number of equivalent coverage pixels.

[0074] After obtaining the equivalent coverage pixel count and total pixels, to convert the pixel-level area statistics into a proportional parameter usable for subsequent rainfall calibration and safety policy judgment, a ratio calculation is performed on the equivalent coverage pixel count and total pixels within the same sampling period to obtain the raindrop coverage ratio corresponding to the current sampling period. The equivalent coverage pixel count is obtained by summing the first, second, and third coverage pixels under the action of the equivalent coverage weight parameter, and is used to characterize the comprehensive coverage area of ​​the raindrops on the bounded sampling area within the current sampling period; the total pixels represent the total number of pixel units contained in the bounded sampling area and are used as a normalization benchmark. The raindrop coverage ratio can be expressed as:

[0075]

[0076] Where r represents the raindrop coverage ratio corresponding to the current sampling period, and the value ranges from zero to one. When r is close to one, it means that the raindrops cover a large area within the framed sampling area. When r is close to zero, it means that the raindrops cover a small area. The equivalent coverage pixel count is a statistical value obtained by summing each pixel according to its corresponding equivalent coverage weight parameter, reflecting the combined contribution of the fully covered and partially covered areas; N represents the total number of pixels in the bounded sampling area, which is the pixel base obtained by multiplying the number of horizontal pixels by the number of vertical pixels. Through this ratio calculation, the raindrop coverage in sampling areas with the same spatial resolution between different frames is mapped to a comparable proportional quantity.

[0077] S160 maps the raindrop coverage ratio to the rainfall information corresponding to the current sampling period based on the preset calibration relationship.

[0078] In one possible implementation, the raindrop coverage ratio is mapped to the rainfall information corresponding to the current sampling period based on a preset calibration relationship. Specifically, this includes: collecting corresponding real rainfall measurement data and raindrop coverage ratio data within the corresponding sampling period based on multiple calibration scenarios with different physical rainfall intensities, and establishing a calibration relationship between the real rainfall measurement data and the raindrop coverage ratio data; substituting the raindrop coverage ratio into the calibration relationship, and performing multiplication and addition calculations in combination with the ratio parameters and offset correction parameters obtained through the calibration process to form the rainfall information corresponding to the current sampling period, so that the rainfall information can characterize the rainfall intensity based on the equivalent coverage pixel number and the total number of pixels.

[0079] Specifically, when collecting corresponding real rainfall measurement data and raindrop coverage ratio data within the corresponding sampling period based on multiple calibration scenarios with different physical rainfall intensities, the vehicle to be calibrated is parked or driven in a test site with deployed standard rain gauges. Under natural or artificial rainfall conditions, the rainfall intensity is covered to multiple conditions including light rain, moderate rain, heavy rain, and near-limit rainfall. For each condition, the real rainfall measurement data output by the meteorological rain gauge or experimental rain measurement device and the raindrop coverage ratio data calculated by this method within the corresponding sampling period are recorded simultaneously. The real rainfall measurement data can be quantified using physical rainfall intensity such as hourly rainfall, while the raindrop coverage ratio data is derived from the ratio of equivalent coverage pixels to total pixels. For paired data samples obtained from multiple calibration scenarios, each set of real rainfall measurement data is paired with the corresponding raindrop coverage ratio data in the data processing unit to construct a calibration sample set between raindrop coverage ratio and physical rainfall intensity. Based on this sample set, a linear mapping model was selected as the expression for the calibration relationship. The undetermined parameters in the linear model were solved through a fitting process, enabling the calibration relationship to approximate the correspondence between actual rainfall measurements and raindrop coverage ratios over the entire rainfall range. The linear calibration relationship can be written as:

[0080]

[0081] Where F represents the nominal value of rainfall information corresponding to the actual rainfall measurement data, which is used to characterize the intensity of physical rainfall; r represents the raindrop coverage ratio within the corresponding sampling period, which is the coverage ratio obtained by the ratio of the equivalent coverage pixels to the total number of pixels; 'b' represents the scaling parameter, used to describe the sensitivity of changes in raindrop coverage ratio to changes in rainfall information; 'b' represents the offset correction parameter, used to compensate for system biases introduced by factors such as the camera optical system, windshield film formation effect, and imaging distortion. The scaling parameter and offset correction parameter can be solved by fitting multiple sets of calibration samples using the least squares method. This minimizes the sum of squared errors between the actual rainfall measurement data and the linear model output at all sample points, thus obtaining a set of calibration parameters with the smallest overall error under various rainfall conditions, ensuring that the calibration relationship remains stable and effective throughout the entire operating range.

[0082] After the calibration relationship is established and the proportional and offset correction parameters are determined, for each sampling period in the actual operation, the raindrop coverage ratio obtained in the current sampling period is substituted into the calibration relationship, and a multiplication-addition calculation is performed using the proportional and offset correction parameters obtained through the calibration process to form the rainfall information corresponding to the current sampling period. The substitution operation can be written as:

[0083]

[0084] in, This represents the rainfall information corresponding to the current sampling period, and is the quantitative result of rainfall intensity output by this method for subsequent safety strategy judgment; This represents the proportion of raindrop coverage within the defined sampling area during the current sampling period. It is calculated as the ratio of the equivalent coverage pixels during the current sampling period to the total number of pixels during the current sampling period. 'b' represents the proportional parameter obtained during the calibration phase, which remains consistent with the calibration data; 'b' represents the offset correction parameter obtained during the calibration phase. This multiply-accumulate calculation step maps the raindrop coverage ratio from the image domain to rainfall information consistent with traditional rainfall measurement methods, giving the rainfall information numerically an interpretability comparable to physical rainfall intensity such as millimeters per hour. Since the raindrop coverage ratio itself is composed of the equivalent coverage pixel count and the total number of pixels, and the equivalent coverage pixel count is obtained by accumulating the first and second coverage pixels according to the equivalent coverage weight parameter, the rainfall information in the current sampling period is essentially a physical domain mapping of the "equivalent coverage pixel count and total number of pixels pair of pixel statistics" under the constraints of the calibration parameters. Therefore, this rainfall information can characterize the rainfall intensity formed based on pixel-level coverage statistics and can be directly used as an input quantity for comparison with rainfall boundary thresholds in subsequent security policy processing.

[0085] S170 compares rainfall information with preset rainfall boundary thresholds to determine the current safety status of the assisted driving function, and performs assisted driving processing when the rainfall information reaches the corresponding level threshold.

[0086] In one possible implementation, the safety status of the current driver assistance function is determined by comparing rainfall information with preset rainfall boundary thresholds. Driver assistance processing is then executed when rainfall information reaches the corresponding threshold level. Specifically, this includes: comparing rainfall information with multiple rainfall boundary thresholds set progressively according to rainfall amount; maintaining the normal operation of the current driver assistance function when rainfall information is less than the first rainfall boundary threshold, allowing the vehicle to continue operating according to its existing functional configuration; downgrading the driver assistance function when rainfall information enters the second rainfall boundary threshold range and outputting a downgrading prompt through the human-machine interface; and exiting the driver assistance function when rainfall information reaches or exceeds the third rainfall boundary threshold and prompting the driver to take over vehicle control.

[0087] Specifically, after obtaining the rainfall information corresponding to the current sampling period, the safety policy processing link uses this rainfall information as the core basis for judging the operational capability of the environment. Multiple rainfall boundary thresholds, progressively increasing in magnitude, are pre-configured in the electronic control unit. Each rainfall boundary threshold corresponds to a different level of operational capability of the autonomous driving system. The safety policy processing link performs a step-by-step comparison based on the relative relationship between the numerical position of the rainfall information and each rainfall boundary threshold, directly linking rainfall information to the operational safety of the autonomous driving function. Rainfall boundary thresholds are typically divided into three main levels: the first rainfall boundary threshold distinguishes between normal operating environments and environments with slight rain interference; the second rainfall boundary threshold identifies environments where perception capabilities are significantly reduced and functional degradation is required; and the third rainfall boundary threshold identifies environments where the perception system is severely affected by rain and the autonomous driving function cannot be safely maintained. This step-by-step comparison process establishes a real-time, continuous linkage mechanism between rainfall information and the autonomous driving system's operational strategy.

[0088] When rainfall information is below the first rainfall threshold, it indicates that raindrop coverage has minimal impact on camera imaging quality and environmental perception capabilities. The safety strategy processing link maintains the assisted driving functions at their highest operating level, allowing the vehicle to execute full autonomous driving functions according to its original longitudinal and lateral control capabilities. In this scenario, the system provides full-capacity safety control parameters to the vehicle control module through the electronic control unit, eliminating the need for function reduction or driver intervention prompts. This ensures the vehicle's autonomous driving performance remains unaffected, while simultaneously monitoring subsequent rainfall changes to respond promptly in case of worsening rainfall.

[0089] When rainfall information falls within the second rainfall threshold range, it indicates that raindrop coverage has impacted environmental perception to the point where it may reduce target recognition stability and positioning reliability. The safety policy processing chain will then implement a downgrade of assisted driving functions, causing the vehicle control module to restrict certain driving functions that rely on high-precision visual perception. This could include reducing the trigger range of automatic lane changing capabilities, lowering the intensity of lateral control response with high perception dependence, or disabling some advanced intelligent driving capabilities. Simultaneously, a downgrade warning will be displayed to the driver in real-time through the human-machine interface, ensuring the driver is fully aware of the changes in autonomous driving capabilities, maintains appropriate attention, and is prepared to take over at any time, thereby effectively reducing the safety risks caused by decreased environmental perception.

[0090] When rainfall reaches or exceeds the third rainfall threshold, it indicates that raindrop coverage has severely obstructed the camera's field of view or that raindrop film formation has exacerbated optical distortion, rendering environmental perception unable to meet minimum safe operating conditions. In this situation, the safety policy processing chain triggers the driver assistance function exit procedure, proactively disabling all autonomous driving functions and outputting a forced takeover warning through the human-machine interface. Simultaneously, it implements emergency safety-related restrictions to support driver takeover control, including enhancing brake standby status, improving steering wheel tactile feedback, or briefly retaining lane centering assist during transition phases. This exit procedure ensures that the vehicle promptly returns to direct driver control in cases of excessive rainfall, preventing potential traffic accidents caused by environmental perception failure.

[0091] This embodiment also discloses a safety processing device based on vehicle-mounted rain measurement, referring to... Figure 2 The device includes an acquisition module 201, a processing module 202, and an output module 203. It is used to execute any of the above-described safety processing methods based on vehicle-mounted rainfall measurement, wherein: The acquisition module 201 is used to acquire a sampled image covering the sampling area of ​​the windshield after each wiper operation. The acquisition module 201 is used to perform pixel-level brightness information parsing and cropping operations on the sampled image, obtain the total number of pixels in the bounded sampling area, and extract the brightness information, local gradient and neighborhood brightness variance of the corresponding pixels. Processing module 202 is used to determine the coverage type of each pixel within the bounded sampling area based on the changes in brightness information and the combined features of local gradient and neighborhood brightness variance; Processing module 202 is used to perform calculation and statistics on pixels according to the coverage type to form an equivalent number of covered pixels; Processing module 202 is used to calculate the ratio of the equivalent coverage pixel count to the total number of pixels to obtain the raindrop coverage ratio corresponding to the current sampling period; The processing module 202 is used to map the raindrop coverage ratio to the rainfall information corresponding to the current sampling period according to the preset calibration relationship; The output module 203 is used to compare the rainfall information with the preset rainfall boundary threshold to determine the current safety status of the assisted driving function, and to perform assisted driving processing when the rainfall information reaches the corresponding level threshold.

[0092] It should be noted that the above embodiments of the apparatus are only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0093] This embodiment also discloses an electronic device, as shown in the reference. Figure 3 The electronic device may include: at least one processor 301, at least one communication bus 302, user interface 303, network interface 304, and at least one memory 305.

[0094] The communication bus 302 is used to enable communication between these components.

[0095] The user interface 303 may include a display screen and a camera. Optionally, the user interface 303 may also include a standard wired interface and a wireless interface.

[0096] The network interface 304 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0097] The processor 301 may include one or more processing cores. The processor 301 connects to various parts of the server using various interfaces and lines, and performs various server functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in memory 305, and by calling data stored in memory 305. Optionally, the processor 301 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 301 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required for display; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 301 and may be implemented as a separate chip.

[0098] The memory 305 may include random access memory (RAM) or read-only memory. Optionally, the memory may include a non-transitory computer-readable storage medium. The memory 305 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 305 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-described method embodiments, etc.; the data storage area may store data involved in the above-described method embodiments, etc. Optionally, the memory 305 may also be at least one storage device located remotely from the aforementioned processor 301. As a computer storage medium, the memory 305 may include an operating system, a network communication module, a user interface 303 module, and an application program for a safety processing method based on vehicle-mounted rain measurement.

[0099] exist Figure 3In the electronic device shown, the user interface 303 is mainly used to provide an input interface for the user and to obtain the user input data; while the processor 301 can be used to call an application program stored in the memory 305 that is a safety processing method based on vehicle-mounted rain measurement. When executed by one or more processors 301, the electronic device performs one or more methods as described in the above embodiments.

[0100] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, as some steps can be performed in other orders or simultaneously according to the present invention. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to the present invention.

[0101] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0102] In the several embodiments provided by this invention, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some service interface; the indirect coupling or communication connection between apparatuses or units may be electrical or other forms.

[0103] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0104] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0105] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory 305 and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned memory 305 includes various media capable of storing program code, such as a USB flash drive, external hard drive, magnetic disk, or optical disk.

[0106] The present invention also discloses a non-transitory computer-readable storage medium storing instructions. When executed by one or more processors 301, these instructions cause an electronic device to perform one or more methods as described in the above embodiments.

[0107] The above are merely exemplary embodiments of this disclosure and should not be construed as limiting the scope of this disclosure. Any equivalent changes and modifications made in accordance with the teachings of this disclosure shall still fall within the scope of this disclosure. Those skilled in the art will readily conceive of other embodiments of this disclosure upon considering the specification and the disclosure of practical truths. This invention is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not described in this disclosure. The specification and embodiments are to be considered exemplary only, and the scope and spirit of this disclosure are defined by the claims.

Claims

1. A safety processing method based on on-board rain measurement, characterized in that, The method comprises: After each wiper completes a wiping action, a sampling image covering a front windshield sampling area is acquired; Pixel-level brightness information analysis and clipping operations are performed on the sampling image to obtain total pixel points of the framed sampling area and extract brightness information, local gradient and neighborhood brightness variance of corresponding pixel points; The coverage type of each pixel point in the framed sampling area is determined according to the change of the brightness information and the characteristics of the local gradient and the neighborhood brightness variance; The pixel points are statistically converted according to the coverage type to form an equivalent coverage pixel number; The equivalent coverage pixel number and the total pixel points are calculated by ratio to obtain a raindrop coverage ratio corresponding to the current sampling period; The raindrop coverage ratio is mapped to rainfall information corresponding to the current sampling period according to a preset calibration relationship; The safety state of the current auxiliary driving function is determined by associating and comparing the rainfall information with a preset rainfall boundary threshold, and the auxiliary driving processing is performed when the rainfall information reaches a corresponding level threshold.

2. The safety processing method based on the measurement of the rainfall on the vehicle according to claim 1, characterized in that, The pixel-level brightness information analysis and clipping operations performed on the sampling image to obtain total pixel points of the framed sampling area and extract brightness information, local gradient and neighborhood brightness variance, specifically include: The sampling image is clipped based on preset framed sampling area coordinates to obtain a clipping image containing the front windshield sampling area; The product of the pixel numbers corresponding to the horizontal width and the vertical height in the clipping image is counted to obtain the total pixel points of the framed sampling area; The brightness information, the local gradient and the neighborhood brightness variance are extracted for each pixel point in the clipping image, wherein the brightness information is a weighted brightness value obtained by performing weighted summation on the color value of the pixel point, the local gradient is a brightness change amplitude obtained by performing square summation on the difference intensities of the brightness information in the horizontal direction and the vertical direction and then performing square root operation, and the neighborhood brightness variance is a brightness dispersion degree obtained by calculating the square of the brightness deviation from the average value and taking the average value after averaging all pixel brightness information in the neighborhood window centered on the pixel point.

3. The safety processing method based on the measurement of the rainfall on the vehicle according to claim 1, characterized in that, The coverage type of each pixel point in the framed sampling area is determined according to the change of the brightness information and the characteristics of the local gradient and the neighborhood brightness variance, specifically including: A reference brightness background is constructed at a position corresponding to the framed sampling area based on historical sampling images in a plurality of frames of rain-free working conditions; The brightness information of each pixel point in the framed sampling area is compared with the reference brightness background to obtain a brightness offset in the current sampling period; The brightness offset, the local gradient and the neighborhood brightness variance are jointly used as feature parameters of the current pixel point coverage state; A coverage degree index is obtained by performing weighted linear combination on the feature parameters and through nonlinear mapping; The first coverage threshold and the second coverage threshold are set according to the value range of the coverage degree index, so that a pixel point with the coverage degree index greater than the first coverage threshold is determined as a first coverage pixel, a pixel point with the coverage degree index between the first coverage threshold and the second coverage threshold is determined as a second coverage pixel, and a pixel point with the coverage degree index less than the second coverage threshold is determined as a third coverage pixel.

4. The safety processing method based on the measurement of the rainfall on the vehicle according to claim 1, characterized in that, The pixel points are converted and counted according to the coverage type to form the number of equivalent coverage pixels, specifically including: associating the coverage type of each pixel point in the framed sampling region with the corresponding coverage degree index; assigning an equivalent coverage weight parameter to the pixel point according to the coverage degree index; performing a pixel-by-pixel traversal process on each pixel point in the framed sampling region, simultaneously reading the coverage type, coverage degree index and equivalent coverage weight parameter of each pixel point in the traversal process, taking the equivalent coverage weight parameter as the contribution value of the pixel point to the number of equivalent coverage pixels, and accumulating and summing the contribution values of all pixel points in the spatial dimension to form the number of equivalent coverage pixels corresponding to the current sampling period.

5. The safety processing method based on the measurement of the rainfall on the vehicle according to claim 1, characterized in that, The number of equivalent coverage pixels is calculated by ratio with the total number of pixels to obtain the raindrop coverage ratio corresponding to the current sampling period, specifically including: determining the total number of pixels by multiplying the number of horizontal pixels and the number of vertical pixels in the framed sampling region; performing ratio operation on the number of equivalent coverage pixels and the total number of pixels in the same sampling period to obtain the raindrop coverage ratio corresponding to the current sampling period.

6. The safety processing method based on the measurement of the rainfall on the vehicle according to claim 1, characterized in that, The raindrop coverage ratio is mapped to the rainfall information corresponding to the current sampling period according to a preset calibration relationship, specifically including: Based on a plurality of calibration scenes with different physical rainfall intensities, corresponding real rainfall measurement data and raindrop coverage ratio data in corresponding sampling periods are collected, and a calibration relationship is established between the real rainfall measurement data and the raindrop coverage ratio data; The raindrop coverage ratio is substituted into the calibration relationship, and a multiplication-addition calculation is performed in combination with the proportion parameter and the offset correction parameter obtained through the calibration process to form the rainfall information corresponding to the current sampling period, so that the rainfall information can represent the rainfall intensity formed based on the number of equivalent coverage pixels and the total number of pixels.

7. The safety processing method based on the measurement of the rainfall on the vehicle according to claim 1, characterized in that, The rainfall information is associated and compared with a preset rainfall boundary threshold to determine the safety state of the current auxiliary driving function, and auxiliary driving processing is performed when the rainfall information reaches a corresponding level threshold, specifically including: gradually comparing the rainfall information with a plurality of rainfall boundary thresholds set in ascending order of rainfall size; when the rainfall information is less than a first rainfall boundary threshold, maintaining the normal running state of the current auxiliary driving function to make the vehicle work continuously according to the existing function configuration; when the rainfall information enters a second rainfall boundary threshold interval, performing auxiliary driving function degradation processing and outputting a degradation prompt through a human-machine interaction interface; When the rain information reaches or exceeds a third rain boundary threshold, an auxiliary driving function exit process is executed and the driver is prompted to take over control of the vehicle.

8. A safety processing device based on a vehicle-mounted rain measurement, characterized by comprising: The device is used to execute a safety processing method based on vehicle-mounted rain measurement as claimed in any one of claims 1-7, and comprises an acquisition module, a processing module, and an output module, wherein: The acquisition module is configured to acquire a sampling image of a sampling area on the front windshield after each wiper completes a wiping action; The acquisition module is configured to perform pixel-level brightness information analysis and clipping operations on the sampling image to acquire total pixel points of the framed sampling area and extract brightness information, local gradient, and neighborhood brightness variance of the corresponding pixel points; The processing module is configured to determine the coverage type of each pixel point in the framed sampling area based on the change in the brightness information and the features of the local gradient and neighborhood brightness variance; The processing module is configured to perform equivalent coverage pixel number conversion and statistics based on the coverage type of the pixel points; The processing module is configured to perform ratio calculation of the equivalent coverage pixel number and the total pixel points to obtain a raindrop coverage ratio corresponding to the current sampling period; The processing module is configured to map the raindrop coverage ratio to rain information corresponding to the current sampling period based on a preset calibration relationship; The output module is configured to compare the rain information with preset rain boundary thresholds to determine the safety state of the current auxiliary driving function, and execute an auxiliary driving process when the rain information reaches a corresponding level threshold.

9. An electronic device, comprising: The electronic device includes a processor, a communication bus, a user interface, a network interface, and a memory, the memory is configured to store instructions, the user interface and the network interface are configured to communicate with other devices, the communication bus is configured to realize connection and communication between components in the electronic device, and the processor is configured to execute the instructions stored in the memory to enable the electronic device to execute the method as claimed in any one of claims 1-7.

10. A non-transitory computer-readable storage medium, comprising: The computer-readable storage medium stores instructions that, when executed, perform the method as claimed in any one of claims 1-7.

Images