Automatic windscreen wiper control snow removal system and device based on image snowfall recognition

Abstract

The invention provides an automatic windscreen wiper control snow removal system and device based on image snowfall recognition, relates to the technical field of vehicle intelligence, and particularly relates to an intelligent windscreen wiper control system and method based on multi-stage detection. The method aims at improving the automatic coping capacity of the vehicle under the complex weather conditions such as rain, snow and ice. The system comprises three stages: in the first stage, a classification model is adopted to carry out weather category detection on an input image; in the second stage, pixel-level detection is carried out on snow or water by adopting a segmentation model, and the coverage range and magnitude of the snow or water are determined; in the third stage, a windscreen wiper, a water spraying system and an air conditioning system are intelligently controlled in combination with the detection result, and precise snow removal, water removal or ice removal is achieved. The windscreen wiper strategy can be dynamically adjusted according to the actual weather and the snow (water) amount, invalid wiping or dry wiping is avoided, the service life of the windscreen wiper is prolonged, and meanwhile the driving safety is improved.

Description

Technical Field

[0001] The present invention relates to the field of vehicle intelligent technology, and in particular to an automatic windshield wiper control snow removal system and device based on image snow amount recognition. Background Art

[0002] With the rapid development of intelligent automobile technology, automated driving assistance systems (ADAS) and in-vehicle intelligent software have been widely integrated into modern vehicles, significantly improving driving comfort and safety. However, the snowy environment poses a severe challenge to driving safety: snow covering the windshield will seriously obstruct the driver's vision, and the rain sensors (such as optical or capacitive) that traditional wiper systems rely on have problems such as insufficient sensitivity and delayed response when detecting the amount of snow, resulting in the inability to start the snow removal function in time. According to research, the traffic accident rate in snowy days is about 30% higher than that in normal weather, of which traffic accidents caused by obstructed vision also account for a certain proportion, so stable wiper control technology that can automatically remove snow is particularly important.

[0003] The current mainstream automatic windshield wiper and snow removal systems mainly rely on the following two traditional technologies, but they perform poorly in snowy scenes:

[0004] 1. Traditional sensor technology:

[0005] 1) Optical sensor: It detects rainfall through changes in infrared reflection. However, due to the small detection range, irregular snowflake shapes, and large coverage area, it is easy to misjudge the reflection signal and cannot accurately distinguish the amount of snow.

[0006] 2) Capacitive sensors: They rely on changes in the dielectric constant of the glass surface, but the uniform coverage of snow may make the capacitance value fluctuate insignificantly, resulting in missed detection or delayed response.

[0007] 2. Mechanical snow removal relies on manual intervention: The driver needs to frequently adjust the wiper mode manually, which distracts the driver's attention, especially in snowy weather with poor driving visibility, increasing driving safety risks.

[0008] All of the above methods are difficult to adapt to the complex and changeable snow accumulation conditions on snowy days. Summary of the invention

[0009] The purpose of the present invention is to provide an automatic windshield wiper control snow removal system and device based on image snow amount recognition to solve the technical problems existing in the background technology.

[0010] To achieve the above object, the present invention provides the following technical solutions:

[0011] An automatic wiper control snow removal system based on image snow amount recognition includes the following modules:

[0012] The first stage category detection module is used to process the input image and detect the weather category, and output the classification results of rain, snow and water;

[0013] The second stage snow or water volume detection module is used to process the input image and perform pixel-level detection of snow or water, and output snow or water volume and location information;

[0014] The third stage wiper control module controls the wiper, water spray and air conditioning systems to perform corresponding actions according to the detection results of the first stage category detection module and the second stage snow or water detection module.

[0015] On the basis of the above technical solution, the present invention also provides the following optional technical solution:

[0016] In an optional solution: the first-stage category detection module includes the following units:

[0017] An input image preprocessing unit is used to crop and scale the original image obtained by the driving recorder or the front-view camera, crop the driver's front field of view area, and adjust it to a preset size;

[0018] The category detection model unit uses ResNet, VGGNet or Mobilenet series classification models to classify the preprocessed images, output the confidence scores of rain, snow and ice, and calculate the category with the highest probability through softmax;

[0019] Detection result output unit: transmits the classification result to the second-stage snow or water detection module and the third-stage wiper control module.

[0020] In an optional solution: in the input image preprocessing unit, the original input image size is 1920*1080 pixels, and after a rectangular area of ​​1450*920 pixels is cropped, it is scaled to 512*256 pixels or 1024*512 pixels.

[0021] In an optional solution: the second stage snow or water volume detection module includes the following units:

[0022] The input image processing unit crops and scales the original image to a preset size to meet the input requirements of the segmentation model;

[0023] Snow and water detection model unit: Use SAM, Mask R-CNN, SegNet or EfficientSAM segmentation model to detect the pixel-level distribution and location of snow or water;

[0024] Detection result output unit: integrates multiple frame detection results and outputs snow or water amount and location information to the third-stage wiper control module.

[0025] 5. In an optional solution: the third stage wiper control module includes the following units:

[0026] Wiper control unit: controls the wipers to clear the snow or water on the windshield at different scraping speeds according to the snow or water amount detection results;

[0027] Water spray control unit: When the snow or water position does not change after the wiper wipes, the water spray system is controlled to spray water, and then the wiper control unit controls the wiper wiper;

[0028] Air conditioning control unit: When the detection result is ice, the air conditioning is controlled to blow hot air to the windshield until the ice melts into water, and then the wiper control unit starts the wipers.

[0029] In an optional solution: the execution logic of the wiper control unit is:

[0030] If the detection result is snow and the vehicle speed is less than 40km / h, adjust the scraping speed according to the amount of snow;

[0031] If the detection result is water and it affects the driver's field of vision, the wipers are directly controlled to wipe it away.

[0032] An automatic windshield wiper control snow removal device includes the automatic windshield wiper control snow removal system based on image snow amount recognition as described above.

[0033] By adopting the above technical solution, the present invention has the following beneficial effects:

[0034] The present invention adopts a two-stage detection mechanism, first classifying the weather type, then accurately detecting the amount of snow (water), reducing misjudgment and improving system reliability. The wiper strategy is dynamically adjusted according to the vehicle speed, snow (water) amount and weather type to avoid residual snow at low speed or ineffective wiping at high speed, and optimize the wiper efficiency. When it is detected that the wiper is ineffective (such as mud stains), it automatically triggers water spray cleaning to avoid dry brushing and damaging the wiper blades; when ice is detected, it is first heated to melt and then scraped off to reduce mechanical wear.

[0035] The automatic wiper control and snow removal system based on image snow amount recognition provided by the present invention adopts deep learning algorithms (such as convolutional neural networks) to analyze real-time images collected by the on-board camera, and can identify the distribution and amount of snow accumulation and whether it is frozen. The accuracy and scene adaptability far exceed those of traditional sensors. The system can comprehensively judge the degree of snow accumulation and automatically control the wiper frequency, thereby improving the driver's field of vision and driving experience, and ensuring driving concentration and safety in severe weather. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0037] Figure 1 The figure is a flow chart of the automatic windshield wiper control snow removal system based on image snow amount recognition of the present invention.

[0038] Figure 2 It is a schematic diagram of the electrical principle of the present invention. DETAILED DESCRIPTION

[0039] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0040] The left, right, up, and down positions of the various components given in the drawings are only one arrangement method, and the specific positions are set according to specific needs.

[0041] An automatic wiper control snow removal system based on image snow amount recognition includes the following modules:

[0042] The first stage category detection module is used to process the input image and detect the weather category, and output the classification results of rain, snow and water;

[0043] The second stage snow or water volume detection module is used to process the input image and perform pixel-level detection of snow or water, and output snow or water volume and location information;

[0044] The third stage wiper control module controls the wiper, water spray and air conditioning systems to perform corresponding actions according to the detection results of the first stage category detection module and the second stage snow or water detection module.

[0045] The first stage category detection module includes the following units:

[0046] An input image preprocessing unit is used to crop and scale the original image obtained by the driving recorder or the front-view camera, crop the driver's front field of view area, and adjust it to a preset size;

[0047] The category detection model unit uses ResNet, VGGNet or Mobilenet series classification models to classify the preprocessed images, output the confidence scores of rain, snow and ice, and calculate the category with the highest probability through softmax;

[0048] Detection result output unit: transmits the classification result to the second-stage snow or water detection module and the third-stage wiper control module.

[0049] In the input image preprocessing unit, the original input image size is 1920*1080 pixels. After a rectangular area of ​​1450*920 pixels is cropped, it is scaled to 512*256 pixels or 1024*512 pixels.

[0050] The second stage snow or water volume detection module includes the following units:

[0051] The input image processing unit crops and scales the original image to a preset size to meet the input requirements of the segmentation model;

[0052] Snow and water detection model unit: Use SAM, Mask R-CNN, SegNet or EfficientSAM segmentation model to detect the pixel-level distribution and location of snow or water;

[0053] Detection result output unit: integrates multiple frame detection results and outputs snow or water amount and location information to the third-stage wiper control module.

[0054] The third stage wiper control module includes the following units:

[0055] Wiper control unit: controls the wipers to clear the snow or water on the windshield at different scraping speeds according to the snow or water amount detection results;

[0056] Water spray control unit: When the snow or water position does not change after the wiper wipes, the water spray system is controlled to spray water, and then the wiper control unit controls the wiper wiper;

[0057] Air conditioning control unit: When the detection result is ice, the air conditioning is controlled to blow hot air to the windshield until the ice melts into water, and then the wiper control unit starts the wipers.

[0058] The execution logic of the wiper control unit is:

[0059] If the detection result is snow and the vehicle speed is less than 40km / h, adjust the scraping speed according to the amount of snow;

[0060] If the detection result is water and it affects the driver's vision, the wipers are directly controlled to wipe away the water.

[0061] The specific embodiments of the present invention are as follows: Figure 1 and2 As shown,

[0062] The automatic wiper control snow removal system includes three modules: a first-stage category detection module, a second-stage snow (water) amount detection module and a third-stage wiper control module.

[0063] The first stage category detection module includes three parts: input image, ClsDetectionModel category detection model and detection results; among them:

[0064] The input image is obtained through the vehicle's existing driving recorder camera (DVR) or front-view camera. For example, the original image size obtained by the DVR is 1920*1080 pixels. In order to avoid the influence of the trim in the vehicle's center console, the original image needs to be cropped and scaled. The image of the field of view directly in front of the driver is first cropped out to a rectangle of 1450*920 pixels, and then scaled to 512*256 pixels (it can also be 1024*512 pixels, the main consideration is the computing power of the on-board controller, the computing power is more redundant, and the input image size can be appropriately expanded) to adapt to the input shape required by the ClsDetectionModel category detection model.

[0065] The ClsDetectionModel category detection model adopts general classification model architectures such as ResNet, VGGNet, and Mobilenet series. It uses real vehicle shooting data to artificially divide it into three categories: rain, snow, and ice, and performs image preprocessing operations to form a data set; it uses data aggregation to train the model, and uses the validation set to evaluate and tune the model.

[0066] The model with excellent performance is deployed to the first-stage classification module. After the ClsDetectionModel category detection model receives the input data, it calculates the confidence score of each category of the current image through the model, and calculates the classification result with the highest probability through the softmax operator.

[0067] Model training calculation process:

[0068] The preprocessed input data is passed to the input layer, and the neurons in the input layer pass the data directly to the hidden layer. In the hidden layer, each neuron performs a weighted summation of the input and performs a nonlinear transformation through the activation function. For example, for the i-th hidden layer neuron, its input Z i Z i =∑ j *W ij *X j +B i , where W ij is the weight connecting the jth neuron in the input layer and the ith neuron in the hidden layer, Xj is the value of the jth neuron in the input layer, B i is the bias of the i-th neuron in the hidden layer. Then, the output H of the hidden layer neuron is obtained through the activation function F (common activation functions include ReLU, Sigmoid, Tanh, etc.) i =F(Z i ). Repeat the above process, use the output of the previous hidden layer as the input of the next hidden layer, calculate the output of each hidden layer in turn, and the output of the last hidden layer is the model output result.

[0069] The second stage snow (water) detection module includes three parts: input image, SnowDetectionModel snow (water) detection model, and detection results; among them:

[0070] The input image is obtained through the vehicle's existing driving recorder camera (DVR) or front-view camera. For example, the original image size obtained by the DVR is 1920*1080 pixels. In order to avoid the influence of the center console of the vehicle, the original image needs to be cropped and scaled. First, the image of the driver's field of view is cropped out in the shape of a 1450*920 pixel rectangle, and then scaled to 1024*5126 pixels (compared to the ClsDetectionModel category detection model, the input image size of the SnowDetectionModel is usually larger because the amount of snow needs to be determined through pixel-level detection technology) to adapt to the input shape required by the SnowDetectionModel snow (water) detection model.

[0071] The SnowDetectionModel snow (water) volume detection model adopts general segmentation model architectures such as SAM, Mask R-CNN, SegNet, and EfficientSAM. It uses real-car shooting data sets to mark the size and location of snow (water), and uses the validation set to evaluate and tune the model. The model with excellent performance can then be deployed to the second-stage snow (water) volume detection module. After the SnowDetectionModel snow (water) volume detection model receives the input data, it calculates the size of the snow (water) volume in the current image through the model.

[0072] The detection result is a comprehensive judgment of the amount of snow (water) in the current environment based on the results (such as position and pixel value) of the SnowDetectionModel snow (water) detection model calculation in multiple frames of images (see the model training calculation process of the first stage category detection module for the specific calculation process).

[0073] The third-stage wiper control module includes three parts: wiper control, water spray control, and air conditioning control. According to actual vehicle tests, snow cannot remain on the windshield when driving at high speed (speed>80km / h), but can remain on the vehicle windshield when driving at low speed (speed<40km / h), and light snow can melt into water quickly, medium snow melts into water slowly, and heavy snow and above will remain on the vehicle windshield in the form of snowflakes.

[0074] therefore:

[0075] When the classification result of the first-stage category detection module is water or snow, and the amount of snow (water) detected by the second-stage snow (water) amount detection module affects the driver's field of vision (the so-called impact on the driver's field of vision requires manual calibration of the actual vehicle), send different wiper speed control wiper instructions according to the snow (water) amount value to scrape away the snow;

[0076] If the snow (water) volume position detected by the second-stage snow (water) volume detection module does not change after the windshield wipers are used, it means that there is interference from foreign objects such as mud stains that cannot be wiped off (which will also affect the driver's field of vision). It is necessary to control the water spray before wipers are used to avoid long-term dry brushing and damage to the wipers.

[0077] When the classification result of the first-stage category detection module is ice, the air conditioner needs to be controlled to blow hot air to the windshield (the heating time also needs to be manually calibrated by the actual vehicle) until the classification result of the first-stage category detection module is water, and then the wipers can be controlled to wipe it off to avoid damage to the wipers.

[0078] The automatic wiper control and snow removal system based on image snow amount recognition provided by the present invention realizes corresponding functions completely based on software. With the subsequent iteration of data, it can perform technical upgrades through OTA to optimize functional experience, effectively overcoming the shortcomings of traditional sensor functions that are hard-coded into one key and hardware costs. It is based on the image form and has a larger sensing range (usually 100-200 times that of traditional sensors, depending on the camera model and installation location), and can more accurately identify the types of water, snow, and ice and the amount of snow (water), and provide different wiper strategies based on the results to avoid damage to the wipers while improving driving comfort and safety.

[0079] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.

Claims

1. An automatic windshield wiper control snow removal system based on image snow amount recognition, characterized in that: Includes the following modules: The first stage category detection module is used to process the input image and detect the weather category, and output the classification results of rain, snow and water; The second stage snow or water volume detection module is used to process the input image and perform pixel-level detection of snow or water, and output snow or water volume and location information; The third stage wiper control module controls the wiper, water spray and air conditioning systems to perform corresponding actions according to the detection results of the first stage category detection module and the second stage snow or water detection module.

2. The automatic windshield wiper control snow removal system based on image snow amount recognition according to claim 1, characterized in that: The first stage category detection module includes the following units: An input image preprocessing unit is used to crop and scale the original image obtained by the driving recorder or the front-view camera, crop the driver's front field of view area, and adjust it to a preset size; The category detection model unit uses ResNet, VGGNet or Mobilenet series classification models to classify the preprocessed images, output the confidence scores of rain, snow and ice, and calculate the category with the highest probability through softmax; Detection result output unit: transmits the classification result to the second-stage snow or water detection module and the third-stage wiper control module.

3. The automatic windshield wiper control snow removal system based on image snow amount recognition according to claim 1, characterized in that: In the input image preprocessing unit, the original input image size is 1920*1080 pixels. After a rectangular area of ​​1450*920 pixels is cropped, it is scaled to 512*256 pixels or 1024*512 pixels.

4. The automatic windshield wiper control snow removal system based on image snow amount recognition according to claim 1, characterized in that: The second stage snow or water volume detection module includes the following units: The input image processing unit crops and scales the original image to a preset size to meet the input requirements of the segmentation model; Snow and water detection model unit: Use SAM, Mask R-CNN, SegNet or EfficientSAM segmentation model to detect the pixel-level distribution and location of snow or water; Detection result output unit: integrates multiple frame detection results and outputs snow or water amount and location information to the third-stage wiper control module.

5. The automatic windshield wiper control snow removal system based on image snow amount recognition according to claim 1, characterized in that: The third stage wiper control module includes the following units: Wiper control unit: controls the wipers to clear the snow or water on the windshield at different scraping speeds according to the snow or water detection results; Water spray control unit: When the snow or water position does not change after the wiper wipes, the water spray system is controlled to spray water, and then the wiper control unit controls the wiper wiper; Air conditioning control unit: When the detection result is ice, the air conditioning is controlled to blow hot air to the windshield until the ice melts into water, and then the wiper control unit starts the wipers.

6. The automatic windshield wiper control snow removal system based on image snow amount recognition according to claim 1, characterized in that: The execution logic of the wiper control unit is: If the detection result is snow and the vehicle speed is less than 40km / h, adjust the scraping speed according to the amount of snow; If the detection result is water and it affects the driver's field of vision, the wipers are directly controlled to wipe it away.

7. An automatic wiper control snow removal device, characterized in that: It comprises an automatic windshield wiper control and snow removal system based on image snow amount recognition as described in any one of claims 1 to 6.

Images