A control method and system for an automatic wiper

Abstract

The application belongs to the field of wiper control, and specifically discloses a control method and system of an automatic wiper, which collects image videos of obstacles in front of a windshield of a vehicle, uses a deep learning algorithm to train video frame sequences to distinguish the types and area sizes of the obstacles, and determines the speed and cleaning mode of the automatic wiper according to the area and type of the obstacles; in the scheme, when the types of the obstacles are distinguished, the types of the obstacles are divided in detail through analysis of the properties of the obstacles, so that different obstacles can be recognized with higher efficiency and controlled by using different control modes.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of wiper control, and particularly relates to a control method and system of an automatic wiper. BACKGROUND

[0002] The wiper is a basic component of the automobile, which is used to clean the front windshield glass to prevent rain and other dirty objects from affecting the driver's vision. The traditional wiper needs the driver to manually start and control the movement of the wiper according to the clarity of the windshield and the rain intensity, which is easy to distract the driver and cause traffic accidents. At the same time, the existing electric vehicles are equipped with ADAS advanced auxiliary systems, which have high requirements for the clarity of the vehicle-mounted camera.

[0003] Therefore, the market has appeared an automatic wiper control system based on a rain sensor. This automatic wiper control system mainly relies on the following principle: detecting the rainfall through a rain sensor, converting the detection value into an electrical signal, and then controlling the time interval of the wiper movement according to the size of the electrical signal, thereby realizing the automatic control of the wiper. At present, the control system sensors applied to the automatic wiper mainly include the following three types: piezoelectric sensor, electrostatic capacitance sensor, and light intensity change sensor. The piezoelectric sensor and the electrostatic capacitance sensor are installed outside the vehicle, and the raindrops directly detect the rainfall on the sensor; the light intensity change sensor installs the rain sensor inside the windshield of the cockpit, and realizes the rainfall detection by the light intensity change caused by the rain falling on the glass. Through actual use and analysis, it is found that these rain sensors have defects such as limited measurement range and environmental influence on the detection of key factors.

[0004] In view of the above defects, a patent with publication number CN114312672A proposes a wiper control method and system. The front camera of the vehicle recorder collects the video image of the front windshield of the vehicle; the image analysis of the video image is performed to obtain the area of the obstacle; the rain wiper oscillation frequency required to remove the obstacle is obtained according to the area of the obstacle; and the rain wiper driving motor is controlled to control the rain wiper oscillation at the rain wiper oscillation frequency. Another patent with publication number CN112109664A proposes a control method, which collects the image of the front windshield; processes the image to obtain the shielding degree and the type of the obstacle of the front windshield; and cleans the front windshield based on the shielding degree and the type of the obstacle. The above-mentioned scheme can solve the limitation of the prior art that only the rainfall sensed by the rain sensor is used to control the wiper, and can automatically control the wiper to oscillate at a suitable frequency to improve the intelligence of the wiper control, but the above-mentioned scheme does not consider the nature of the obstacle, and only uses the area and the approximate type of the obstacle to control the oscillation frequency of the wiper. When multiple types of obstacles appear on the front windshield, it is not possible to finely process the corresponding type of obstacle.

[0005] Therefore, an automatic wiper control scheme with high obstacle detection accuracy, high efficiency and effectively reducing irrelevant environmental impact is necessary. SUMMARY

[0006] To solve the above problems, the present application discloses an automatic wiper control method, which comprises the following steps:

[0007] Collecting image videos of the front windshield of the vehicle and forming a video frame sequence;

[0008] Detecting the area and type of the obstacle in sequence according to the video frame sequence, wherein the type of the obstacle includes cleanable obstacle and non-cleanable obstacle;

[0009] Determining the speed and cleaning mode of the automatic wiper according to the area and type of the obstacle.

[0010] Further, before detecting the area and type of the obstacle in sequence according to the video frame sequence, the method further comprises judging whether the obstacle exists according to the video frame sequence;

[0011] If the obstacle exists, the obstacle category is judged; if the obstacle does not exist, the image videos of the front windshield of the vehicle are continuously collected.

[0012] Further, the cleanable obstacle includes rain, snow and / or leaves;

[0013] The non-cleanable obstacle includes mud and / or large-area bird droppings.

[0014] Further, forming the video frame sequence comprises: setting a certain time interval to extract the pictures in the image videos, and forming the video frame sequence with the pictures extracted in the interval time.

[0015] Further, detecting the area and type of the obstacle in sequence according to the video frame sequence comprises the following steps:

[0016] Calculating the area of the obstacle in the video frame sequence and judging whether the area of the obstacle exceeds a first area threshold;

[0017] If the area of the obstacle exceeds the first area threshold, inputting the current frame of the video frame sequence into a deep learning model;

[0018] Judging the type of the obstacle based on the deep learning model.

[0019] Further, if the area of the obstacle does not exceed the first area threshold, discarding the video of the current frame and continuously collecting the image videos of the front windshield of the vehicle.

[0020] Further, determining the speed and cleaning mode of the automatic wiper according to the area and type of the obstacle comprises the following steps:

[0021] judging the type of the obstacle, if the obstacle is a non-cleanable obstacle, not starting the wiper movement, and warning the driver to stop and manually remove the obstacle;

[0022] if the obstacle is a cleanable obstacle, calculating the area of the obstacle, and determining the movement frequency required for the wiper to remove the obstacle according to the area of the obstacle.

[0023] Further, determining the movement frequency required for the wiper to remove the obstacle according to the area of the obstacle comprises the following steps:

[0024] if the area of the obstacle exceeds a first area threshold and is less than or equal to a second area threshold, the wiper speed is set to F1;

[0025] if the area of the obstacle exceeds the second area threshold and is less than or equal to a third area threshold, the wiper speed is set to F2.

[0026] Further, the control method further comprises further judging whether the obstacle exists after the wiper movement exceeds a certain time;

[0027] if the obstacle still exists, re-adjusting the movement speed of the wiper according to the relationship between the remaining obstacle area and the first area threshold, the second area threshold and the third area threshold;

[0028] if the obstacle does not exist, continue to collect the image video of the front windshield of the vehicle.

[0029] In another aspect, the present application provides an automatic wiper control system, the control system comprising:

[0030] a collection unit for collecting image videos of the front windshield of the vehicle and forming a video frame sequence;

[0031] a detection unit for sequentially detecting the area and type of the obstacle according to the video frame sequence, the type of the obstacle including cleanable obstacles and non-cleanable obstacles;

[0032] a determination unit for determining the speed and cleaning mode of the automatic wiper according to the area and type of the obstacle.

[0033] Further, the detection unit sequentially detects the area and type of the obstacle by performing the following logic:

[0034] calculating the area of the obstacle in the video frame sequence and judging whether the area of the obstacle exceeds a first area threshold;

[0035] if the obstacle area exceeds the first area threshold, input the current frame of the video frame sequence into a deep learning model; if the obstacle area does not exceed the first area threshold, discard the video of the current frame and continue to collect the image video of the front windshield of the vehicle.

[0036] determine the type of the obstacle based on the deep learning model.

[0037] Further, the following logical determination is performed in the determining unit to determine the speed and cleaning mode of the automatic wiper:

[0038] determine the type of the obstacle, if the obstacle is a non-removable obstacle, do not start the wiper movement, and warn the driver to stop and manually remove the obstacle;

[0039] if the obstacle is a removable obstacle, calculate the obstacle area, determine the movement frequency required for the wiper to remove the obstacle according to the obstacle area, if the obstacle area exceeds the first area threshold and is less than or equal to the second area threshold, set the wiper speed to F1, and if the obstacle area exceeds the second area threshold and is less than or equal to the third area threshold, set the wiper speed to F2.

[0040] Further, the control system further comprises a subsequent determining unit for further determining whether the obstacle exists after the wiper movement exceeds a certain time, if the obstacle still exists, re-adjust the movement rate of the wiper according to the relationship between the remaining obstacle area and the first area threshold, the second area threshold and the third area threshold, and if the obstacle does not exist, continue to collect the image video of the front windshield of the vehicle.

[0041] The beneficial effects of the present application are:

[0042] The present application collects the image video of the obstacle on the front windshield of the vehicle, and then uses a deep learning algorithm to train and distinguish the type and area size of the obstacle, determines the speed and cleaning mode of the automatic wiper according to the area and type of the obstacle, and in the present application, the type of the obstacle is divided in detail through analysis of the properties of the obstacle itself, which can more efficiently identify different obstacles and control the automatic wiper by using different control methods.

[0043] Other features and advantages of the present application will be set forth in the following description, and in part will become apparent to those skilled in the art from the description, or can be learned by practice of the present application. The objects and other advantages of the present application can be achieved and obtained by the structures indicated in the specification, claims and drawings. BRIEF DESCRIPTION OF DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, below the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0045] Figure 1 The main flow chart of the control method in the embodiments of the present application is shown;

[0046] Figure 2 The detailed flow chart of the control method in the embodiments of the present application is shown. DETAILED DESCRIPTION

[0047] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, below the technical solutions in the embodiments of the present application will be clearly and completely described with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the protection scope of the present application.

[0048] Based on the fact that the prior art does not consider the properties of the obstacles themselves, only the area and approximate type of the obstacles are used to control the swing frequency of the wipers, and when multiple types of obstacles appear on the front windshield, the corresponding processing cannot be intelligently performed according to the type of the obstacles. The present application proposes an automatic wiper control method from the type of the obstacles and the properties of each type of obstacle, as shown in Figure 1 The specific scheme is as follows:

[0049] Collect the image video of the front windshield of the vehicle and form a video frame sequence.

[0050] Detect the area and type of the obstacles in sequence according to the video frame sequence, and the type of the obstacles includes cleanable obstacles and uncleanable obstacles.

[0051] Determine the speed and cleaning mode of the automatic wipers according to the area and type of the obstacles.

[0052] The specific flow is shown in Figure 2

[0053] S1, collect the image video of the front windshield of the vehicle through the front-view camera, and obtain a video frame sequence.

[0054] For the collected image video, a certain time interval can be used to extract the pictures to form a video frame sequence. The specific time interval can be designed by the staff, and the better collection effect is used as the criterion.​

[0055] S2, obstacle detection is performed on the video frame sequence.

[0056] The detection on the collected video frame sequence includes detection of the area and type of the obstacle. The specific steps are as follows:

[0057] S21, judging whether the obstacle exists;

[0058] S22, if the obstacle exists, judging the obstacle category; if the obstacle does not exist, continuing to collect the image video of the front windshield of the vehicle through the front-view camera.

[0059] In step S21, to judge whether the obstacle exists, the present application is implemented by the following method: based on image features such as definition, contrast and brightness, a deep learning algorithm locates the obstacle in the frame image, calculates the area of the obstacle, and judges whether the area of the obstacle exceeds a first area threshold. The first area threshold can be set by the staff. If the area of the obstacle exceeds the first area threshold, it is considered that the obstacle exists.

[0060] In step S22, the deep learning model can detect the obstacle into two categories: cleanable obstacle and non-cleanable obstacle.

[0061] The cleanable obstacle includes but is not limited to rain, snow and leaves, etc. which can be removed by the wiper. For this kind of obstacle, the wiper should be started to clean.

[0062] The non-cleanable obstacle includes but is not limited to obstacles such as mud and large-area bird droppings. The image features of the non-cleanable obstacle may include: high blur, low brightness, large contrast with the background, color features, etc. In contrast to the cleanable obstacle, these image features change less with the wiping action of the wiper, i.e. the wiper cannot remove the obstacle by wiping action to make the picture clear. This kind of obstacle is difficult to clean by the wiper, and even the movement of the wiper will cause its occlusion area to expand, so the movement of the wiper should be stopped immediately, and the driver should be warned to stop the car and manually remove the obstacle.

[0063] The deep learning algorithm is trained by inputting an image data set containing obstacles such as raindrops, rain streaks, mud and resin. The image of each obstacle can be taken by the front-view camera placed behind the front windshield. After training, the obstacle can be intelligently classified.

[0064] Further, weather, environment and other factors can be added in the process of constructing the image data set. For example, the wind direction when the raindrops fall, the position of the obstacle, etc. can be considered.

[0065] S3, obtaining a corresponding processing scheme according to the type of the obstacle.

[0066] Specifically, if the obstacle is a cleanable obstacle, the wiper works normally. At this time, the speed of the wiper can be selected according to the area of the obstacle. For example, if the area of the obstacle exceeds the first area threshold and is less than or equal to the second area threshold, the speed of the wiper can be set to F1; if the area of the obstacle exceeds the second area threshold and is less than or equal to the third area threshold, the speed of the wiper can be set to F2. The speed of the wiper gear and the specific speed can be set by the staff according to the actual situation.

[0067] After the wiper moves for a period of time, it is further determined whether the obstacle exists, and the movement speed of the wiper is adjusted again according to the relationship between the remaining obstacle area and the first area threshold, the second area threshold and the third area threshold.

[0068] If the obstacle is an uncleanable obstacle, the warning unit of the system should be activated. The warning unit first voice broadcasts to warn the driver to manually clean the obstacle, and then sends a request to the BCM (system brake unit) to stop driving the wiper motor. At this time, the driver can intervene the warning unit through voice command or manual lever to control the wiper to continue to open. Among them, the voice command can be: "continue to open the wiper". After the system receives the intervention operation of the driver, the frame picture with the obstacle and the corresponding operation are transmitted to the cloud platform, which is convenient for subsequent optimization work of the researchers.

[0069] At the same time, the driver is warned to stop and manually remove the obstacle.

[0070] Based on the above method, the application provides an automatic wiper control system, which comprises a collection unit, a detection unit and a determination unit;

[0071] Among them,

[0072] The collection unit is used for collecting image videos of the front windshield of the vehicle and forming a video frame sequence; similar to the front-view camera on the front windshield;

[0073] The detection unit is used for detecting the area and type of the obstacle in sequence according to the video frame sequence;

[0074] The determination unit is used for determining the speed and cleaning mode of the automatic wiper according to the area and type of the obstacle;

[0075] The warning unit is used for warning the driver to manually clean the obstacle.

[0076] The control system further comprises a subsequent judgment unit for further judging whether the obstacle exists after the wiper moves for a period of time;

[0077] If the obstacle still exists, the movement speed of the wiper is adjusted again according to the relationship between the remaining obstacle area and the first area threshold, the second area threshold and the third area threshold.

[0078] If the obstacle is not present, then continue to capture the image video of the vehicle front windshield.

[0079] Although the present application has been described in detail with reference to the foregoing embodiments, it should be understood that modifications can be made to the foregoing embodiments, or additional implementations can be implemented, without departing from the spirit and scope of the inventive subject matter. Accordingly, the present application is not limited to the implementations described herein, but is intended to be defined by the claims set forth below, and equivalents thereof.

Claims

1. A control method of an automatic wiper, characterized by, The control method comprises the following steps: Collecting image videos of the front windshield of the vehicle and forming a video frame sequence; According to the video frame sequence, the area and type of the obstacle are detected in sequence, and the type of the obstacle includes cleanable obstacles and non-cleanable obstacles; according to the video frame sequence, the area and type of the obstacle are detected in sequence, which comprises: calculating the area of the obstacle in the video frame sequence and judging whether the area of the obstacle exceeds a first threshold; if the area of the obstacle exceeds the first threshold, the current frame of the video frame sequence is input into a deep learning model; the type of the obstacle is determined based on the deep learning model; wherein the deep learning model is trained by inputting an image data set containing raindrops, rain streaks, mud, and branch obstacles, and the image of each obstacle is taken by a front-view camera placed behind the front windshield; after training, the obstacle can be intelligently classified; weather and environment are added during the construction of the image data set; According to the area and type of the obstacle, the speed and cleaning mode of the automatic wiper are determined.

2. The control method of the automatic wiper according to claim 1, characterized by, Before detecting the area and type of the obstacle according to the video frame sequence, it is further determined whether the obstacle exists according to the video frame sequence; If the obstacle exists, the obstacle category is determined; if the obstacle does not exist, the image video of the front windshield of the vehicle is continuously collected.

3. The control method of the automatic wiper according to claim 1, wherein Forming a video frame sequence comprises: setting a certain time interval to extract the pictures in the image video, and forming a video frame sequence with multiple pictures extracted within the interval time; The cleanable obstacles include rainwater, snow and / or leaves; The non-cleanable obstacles include mud and / or large-area bird droppings.

4. The control method of the automatic wiper according to claim 1, wherein If the area of the obstacle does not exceed the first threshold, the video of the current frame is discarded and the image video of the front windshield of the vehicle is continuously collected.

5. The control method of the automatic wiper according to claim 1, wherein According to the area and type of the obstacle, the speed and cleaning mode of the automatic wiper are determined, which comprises the following steps: If the obstacle is a non-cleanable obstacle, the wiper motion is not started, the driver is warned to stop and manually remove the obstacle; If the obstacle is a cleanable obstacle, the area of the obstacle is calculated, and the motion frequency required for the wiper to remove the obstacle is determined according to the area of the obstacle.

6. The control method of the automatic wiper according to claim 5, wherein According to the area of the obstacle, the motion frequency required for the wiper to remove the obstacle is determined, which comprises the following steps: If the area of the obstacle exceeds the first area threshold and is less than or equal to the second area threshold, the wiper speed is set to F1; If the area of the obstacle exceeds the second area threshold and is less than or equal to the third area threshold, the wiper speed is set to F2.

7. The control method of the automatic wiper according to any one of claims 1-6, wherein The control method further comprises further judging whether the obstacle exists after the wiper motion exceeds a certain time. If the obstacle still exists, the moving speed of the wiper is readjusted according to the relationship between the remaining obstacle area and the first area threshold, the second area threshold and the third area threshold; If the obstacle does not exist, the image video of the front windshield of the vehicle is continuously collected.

8. A control system for an automatic wiper, characterized in that The control system comprises: a collection unit configured to collect an image video of a front windshield of a vehicle and form a video frame sequence; a detection unit configured to sequentially detect an area and a type of the obstacle according to the video frame sequence, the type of the obstacle comprising a cleanable obstacle and a non-cleanable obstacle; the detection unit performs the following logic to sequentially detect the area and the type of the obstacle: calculating the area of the obstacle in the video frame sequence and determining whether the area of the obstacle exceeds a first threshold; if the area of the obstacle exceeds the first threshold, inputting a current frame of the video frame sequence into a deep learning model; if the area of the obstacle does not exceed the first threshold, discarding the video of the current frame and continuously collecting the image video of the front windshield of the vehicle; determining the type of the obstacle based on the deep learning model; a determination unit configured to determine a speed and a cleaning mode of an automatic wiper according to the area and the type of the obstacle.

9. The control system of the automatic wiper according to claim 8, wherein the determination unit performs the following logic to determine the speed and the cleaning mode of the automatic wiper: determining the type of the obstacle, if the obstacle is a non-cleanable obstacle, the wiper is not started to move, and a driver is warned to stop the vehicle and manually remove the obstacle; if the obstacle is a cleanable obstacle, calculating the area of the obstacle, determining a moving frequency required for the wiper to remove the obstacle according to the area of the obstacle; if the area of the obstacle exceeds the first area threshold and is less than or equal to a second area threshold, the speed of the wiper is set to F1; if the area of the obstacle exceeds the second area threshold and is less than or equal to a third area threshold, the speed of the wiper is set to F2.

10. The control system of the automatic wiper according to any one of claims 8-9, wherein the control system further comprises a subsequent determination unit configured to further determine whether the obstacle exists after the wiper moves for a certain time; if the obstacle still exists, the moving speed of the wiper is readjusted according to the relationship between the remaining obstacle area and the first area threshold, the second area threshold and the third area threshold; if the obstacle does not exist, the image video of the front windshield of the vehicle is continuously collected.

Images