VISION OBSTRUCTION DETECTION FOR WINDSCREEN WIPER WEAR MONITORING

Abstract

A vehicle includes a camera configured to capture images through a window of the vehicle, a window wiping system operable between a wiping mode and a cleaning mode, and includes a windshield wiper configured to move along the window in the wiping mode and the cleaning mode, and an interface for manually engaging the wiping mode. A control circuit configured to detect an obstruction on the window based on the images, classify the obstruction as water or debris, detect manual operation of the windshield wiper, determine a wear condition during manual operation of the windshield wiper and based on the classification of the obstruction as debris, and communicate a signal indicating the wear condition.

Description

AREA OF REVELATION

[0001] The present disclosure relates generally to image processing in a vehicle environment and, more particularly, to obstruction detection for windshield wiper wear monitoring. GENERAL STATE OF THE ART

[0002] Conventional image processing techniques for detecting moisture conditions on or around a vehicle can be limited. SUMMARY OF REVELATION

[0003] According to a first aspect of the present disclosure, a vehicle includes a camera configured to capture images through a window of the vehicle, a window wiping system operable between a wiping mode and a cleaning mode, and includes a windshield wiper configured to move along the window in the wiping mode and the cleaning mode, and an interface for manually engaging the wiping mode. A control circuit configured to detect a visual obstruction on the window based on the images, classify the visual obstruction as water or debris, detect manual operation of the windshield wiper, determine a wear condition during manual operation of the windshield wiper and based on the classification of the visual obstruction as debris, and communicate a signal indicating the wear condition.

[0004] Embodiments of the first aspect of the present disclosure may include any one or a combination of the following features: - the window cleaning system includes a spray device configured to apply cleaning fluid to the window during the cleaning operation; - the control circuit includes a memory configured to store wear information, including a usage time of the windshield wiper; - the period of use includes a duration of the wear condition; - the window wiping system can be operated in an automatic mode in which the cleaning and wiping operations are automatically activated in response to the detection of the obstruction of visibility; - the window cleaning system includes a pump for pressurising the cleaning fluid to be applied to the window during the cleaning operation; - the cleaning operation and the wiping operation in the automatic mode are determined based on the classification of the visual obstruction as water or dirt; - the window wiping system is configured to operate the wiper in response to the classification of the visual obstruction as water and to operate the spray device in response to the classification of the visual obstruction as dirt; - the control circuit is configured to control the window wiping system to switch from wiping operation to cleaning operation in response to the wear condition; - the control circuit is configured to classify the visual obstruction with a degree of optical distortion, wherein the classification of the visual obstruction as water or dirt is based on the degree of optical distortion; - the control circuit is configured to capture a subsequent image of the window following the manual operation; and to classify the wear condition with a modifier based on the subsequent image; - the modifier is a multiplier of a calculation for the wear condition, and wherein the control circuit is configured to select the multiplier from a plurality of modifiers corresponding to environmental conditions; - the control circuit is configured to communicate a signal to the modifier for applying cleaning fluid to the window in response to the classification of the wear condition; and - a notification device configured to indicate the wear condition in response to the signal.

[0005] According to a second aspect of the present disclosure, a vehicle includes a camera configured to capture images through a window of the vehicle, a window wiping system operable between a wiping mode and a cleaning mode, and including a spray device configured to apply cleaning fluid to the window in the cleaning mode, a windshield wiper configured to move along the window in the wiping mode and the cleaning mode; and an interface for manually operating the windshield wiper.A control circuit configured to detect an obstruction on the window based on the images, classify the obstruction as water or dirt, detect manual operation of the wiper, determine a wear condition during manual operation of the wiper and based on the classification of the obstruction as dirt, and communicate a signal to indicate the wear condition.

[0006] Embodiments of the second aspect of the present disclosure may include any one or a combination of the following features: - the window wiping system can be operated in an automatic mode in which the cleaning and wiping operations are automatically activated in response to the detection of the obstruction of visibility; - the window cleaning system includes a pump for pressurising the cleaning fluid to be applied to the window during the cleaning operation; - a selection between cleaning operation and wiping operation is determined in the automatic mode based on the classification of the visual obstruction as water or dirt; and - the window wiping system is configured to operate the wiper in response to the visual obstruction being classified as water and to operate the spray device in response to the visual obstruction being classified as dirt.

[0007] According to a third aspect of the present disclosure, a vehicle includes a camera configured to capture images through a window of the vehicle, a window wiping system operable between a wiping mode and a cleaning mode, and includes a spray device configured to apply cleaning fluid to the window in the cleaning mode, a windshield wiper configured to move along the window in the wiping mode and the cleaning mode, and an interface for manually operating the windshield wiper, a notification device configured to indicate a wear condition for the windshield wiper in response to a signal.A control circuit configured to detect an obstruction on the window based on the images, classify the obstruction as water or dirt, detect manual operation of the windshield wiper, determine the wear condition during manual operation of the windshield wiper and based on the classification of the obstruction as dirt, and communicate a signal to indicate the wear condition.

[0008] These and other features, advantages, and objects of the present disclosure will be better understood and appreciated by those skilled in the art by reference to the following description, claims, and accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The following applies in the drawings: Fig. 1 is a perspective view of a vehicle incorporating a moisture detection system according to one aspect of the present disclosure; Fig. 2 is a functional block diagram of a moisture detection system for a vehicle according to one aspect of the present disclosure; Fig. 3A is an image of dust on a window to be cleaned by a window cleaning system of the present disclosure; Fig. 3B is an image of biological residue from an insect on a window to be cleaned by a window cleaning system of the present disclosure; Fig. 4 is a flowchart of an automatic mode of a window wiping system that cleans and / or wipes a window of a vehicle according to one aspect of the present disclosure; Fig. 5 is an exemplary cross-sectional view of a vehicle including an imaging device adjacent a windshield of the vehicle within an interior of the vehicle for detecting humidity conditions in the region outside the vehicle; Fig. 6A is an exemplary image captured by an imaging device in an interior of the vehicle positioned away from the windshield, resulting in detectable water droplets on the windshield; Fig. 6B is an exemplary image captured by an imaging device positioned within an interior of the vehicle and proximate the windshield, illustrating detection of water streaks on the windshield; Fig. 7 is an illustration of a fully convolutional data description network (FCDD network) that processes an image acquired by an imaging device positioned near the windshield and generates image data indicating a visual obstruction in the image; Fig. 8A-8C are captured images alongside image data showing obstructions on a vehicle windshield after processing by an FCDD network; Fig. 9A-9B are exemplary images taken by an imaging device for the vehicle showing spray events along a roadway for the vehicle, wherein Fig. 9B illustrates splash zones overlaying splash events; Fig. 10A is an exemplary illustration of a radar scan of an environment outside the vehicle detecting spray conditions; Fig. 10B is an exemplary illustration of a radar scan of an environment outside the vehicle detecting spray conditions; Fig. 11 is a block diagram illustrating an exemplary detection of an overtaking condition based on splash monitoring in a vehicle environment; and Fig. 12 is an exemplary process performed by a moisture detection system according to one aspect of the present disclosure. DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

[0010] Reference will now be made in detail to the presently preferred embodiments of the disclosure, examples of which are illustrated in the accompanying drawings. Wherever possible, the same reference numerals will be used throughout the drawings to designate the same or similar parts. In the drawings, the illustrated structural elements are not to scale, and certain components are enlarged relative to other components for emphasis and clarity.

[0011] As required, detailed embodiments of the present disclosure are disclosed herein; however, it should be understood that the disclosed embodiments are merely exemplary of the disclosure that may be embodied in various and alternative forms. The figures do not necessarily depict a detailed embodiment; some schematic representations may be exaggerated or reduced to show a functional overview. Therefore, specific structural and functional details disclosed herein are not to be interpreted as limiting, but merely as a representative basis for teaching one skilled in the art to variously employ the present disclosure.

[0012] For the purposes of the description herein, the terms “top,” “bottom,” “right,” “left,” “back,” “front,” “vertical,” “horizontal,” and derivatives thereof refer to the concepts in their orientation in Fig. 1. However, it is to be understood that the concepts may assume various alternative orientations unless expressly stated otherwise. It is also to be understood that the specific devices and processes illustrated in the accompanying drawings and described in the following description are merely exemplary embodiments of the inventive concepts defined in the appended claims. Thus, specific dimensions and other physical characteristics are not to be considered limiting with respect to the embodiments disclosed herein unless the claims expressly state otherwise.

[0013] The presently illustrated embodiments consist primarily of combinations of method steps and device components related to visibility obstruction detection for windshield wiper wear monitoring. Accordingly, the device components and method steps have been represented in the drawings by conventional symbols where appropriate, showing only those specific details relevant to an understanding of the embodiments of the present disclosure, so as not to obscure the disclosure with details that would be readily apparent to one of ordinary skill in the art in light of the present description. Furthermore, like reference numerals represent like elements in the description and the drawings.

[0014] With general reference to the Fig. 1-12, a moisture detection system 10 uses image processing to detect conditions of a region exterior 12 of a vehicle 14, such as water on an exterior surface 16 of a windshield 18 of the vehicle 14 or water in front of or around the vehicle 14. In general, the present systems and methods can provide improved intensity classification for splashes in the region exterior 12 of the vehicle 14 by incorporating depth detection or density detection using radio detection and ranging (RADAR). Further, the present systems and methods can provide improved space utilization within the vehicle 14 by allowing an imaging device 20 of the moisture detection system 10 to be positioned close to or far from the windshield 18 while still detecting and classifying obstructions to view on the windshield 18.Furthermore, the moisture detection system 10 may provide improved visibility by optimizing the wiping of the windshield 18 and / or maneuvering the vehicle 14 to a destination or suggesting maneuvering thereof to improve visibility and / or control. The moisture detection system 10 may also provide more accurate detection of weather conditions, such as rain, humidity, fog, or other wet conditions, in the region exterior 12 of the vehicle 14, thereby enabling improved responsiveness for other vehicle systems. Furthermore, the present systems and methods may provide increased durability for parts (e.g., window wiping parts), efficient use of window cleaning fluid, and overall improved automatic window wiping control.

[0015] With reference to Fig. 1, the moisture detection system 10 for the vehicle 14 may include an imaging device 20 positioned within a cabin 22 of the vehicle 14. For example, the imaging device 20 may be positioned adjacent the windshield 18 of the vehicle 14 within a passenger compartment of the vehicle 14 and oriented in a vehicle-forward orientation. At least one windshield wiper 26 is disposed on the outer surface 16 of the windshield 18 to remove water, dirt, or other substances on the windshield 18. As further described with respect to Fig. 2, a window cleaning system may be employed for cleaning the windshield 18 or any of a plurality of windows of the vehicle 14 where moisture or other substances from the region exterior 12 of the vehicle 14 may accumulate on the window. The window cleaning system may include a nozzle 28 positioned adjacent the windshield wipers 26 between a hood 30 of the vehicle 14 and the windshield 18 of the vehicle 14 and configured to spray cleaning fluid onto the windshield 18. The cleaning fluid may include a cleaning agent, such as methanol, glycol, or other fluids, to remove substances on the exterior surface 16 of the windshield 18 when used with the windshield wipers 26.It is contemplated that the cleaning fluid may include water, which in some examples may be heated to assist in a defrosting operation of the windshield 18.

[0016] With further reference to Fig. 1, a plurality of distance sensors 34, 36, 38 are integrated into the vehicle 14 to detect other objects, such as other vehicles 14, in the region 12 outside the vehicle 14. The detection devices may include ultrasonic and / or infrared detectors, as well as cameras 38, such as the imaging device, configured to detect distances from the vehicle 14 to surrounding objects in the region outside 12 of the vehicle 14. Accordingly, the images from the imaging device 20 may be used to detect a distance from the vehicle 14 to objects within a field of view of the imaging device.

[0017] As further stated with reference to Fig. 2, the proximity sensors 34, 36, 38 may include any of the radio detection and ranging sensors (RADAR 34), the light detection and ranging sensors (LIDAR 36), and the cameras 38. In some examples, ultra-wideband (UWB) sensors are used to detect objects in the region outside 12. In general, the proximity sensors 34, 36, 38 may be configured to detect cross-traffic events and / or detect objects in blind spots of the vehicle 14 to assist a user of the vehicle 14 in maneuvering the vehicle 14. In the present example, the data collected by the detection sensors is used by the moisture detection system 10 to enable the moisture detection system 10 to determine a distance (e.g.a following distance 120) from the vehicle 14 to spray events 118 and classifies the importance, relevance or priority of the spray events 118 based on the distance.

[0018] As further described herein, the proximity sensors 34, 46, 38 may coordinate detection techniques using one or more of the cameras 38 to detect a front of a splash event 118 and one or more of the radars 34 to detect a depth D and / or density of the splash event 118. For example, image-based detection may be limited if a significant visual obstruction is present in a captured image. However, the one or more radars 34 may be used to transmit / receive radio waves or microwaves (e.g., via radar transmitters and receivers) reflected from water droplets in the splash event 118 at a more precise level than visible light waves received by the cameras 38.For example, a control circuit in communication with the radars 34 may use information from the radars 34 to determine the depth D, density, intensity, span, or other characteristic of the splash event 118 based on the information. Generally, the radars 34 use Doppler weather detection techniques to generate a map or distribution of water in the splash event 118. For example, the radars 34 may emit microwave or radio wave energy and measure a reflected wave from the splash event 118. Such measurements may include signals having different frequencies than the signals emitted by the radars 34, resulting in a frequency shift. The frequency shift may be directly related to the movement (speed) of raindrops or other droplets 108 in the splash event 118 or precipitation.Accordingly, the heavier the rain or splashing, the more water droplets 108 are present and a stronger return signal is detected by the RADARs 34.

[0019] With further reference to Fig. 1, the vehicle 14 includes a plurality of wheels 40, each having a tire 42 that interacts with a driving surface 44 for the vehicle 14. Friction between each tire 42 and the driving surface 44 may be affected by moisture conditions between the tires 42 and the driving surface 44. Accordingly, the moisture detection system 10 may be employed to control travel of the wheels 40 to enhance maneuverability of the vehicle 14 along the driving surface 44.

[0020] The vehicle 14 includes at least one lighting assembly 46, such as a headlight assembly having headlights 48 configured to illuminate the region exterior 12 of the vehicle 14. For example, the lighting assemblies may be configured to illuminate the region exterior 12 of the vehicle 14 with a variety of illumination levels (e.g., high beams, low beams, etc.). Control of the lighting assemblies and the power levels and / or illumination levels thereof may be enhanced by the moisture detection system 10. For example, moisture conditions in the region exterior 12 detected by the moisture detection system 10 may cause the moisture detection system 10 to control the power levels of the lighting assemblies due to reduced visibility due to the moisture conditions.

[0021] With reference to Fig. 2, the moisture detection system 10 includes an imaging system 50, a distance detection system 52, and a response control system 54 in communication with the imaging system 50 and the distance detection system 52. Generally, data from the imaging system 50 and / or the distance detection system 52 is communicated to the response control system 54, and the response control system 54 may control one or more vision features and / or motion control features of the vehicle 14. For example, the response control system 54 includes one or more controllers 56, 72 having at least one processor and memory in communication with the processor.The memory may store instructions that, when executed by the processor(s) of the response control system 54, cause the response control system 54 to perform various tasks related to enhancing the view of the region outside 12 of the space of the vehicle 14 and / or motion control for the vehicle 14. Generally, one or more of the controllers 56, 72 and / or other electrical components that provide decision making via software may be referred to as control circuitry.

[0022] For example, a reaction controller 56 of the reaction control system 54 may include a motion control unit 58 and a vision control unit 60. The vision control unit 60 may be configured to control the window wiper system, a light control system 62, and / or any other system that affects the view through one or more of the windows of the vehicle 14. The motion control unit 58 may control communication with one or more vehicle systems, such as a powertrain 64 of the vehicle 14, a braking system 66 of the vehicle 14, or any other motion control system for the vehicle 14.For example, the motion control unit 58 may include a speed sensor in the driveline 64 that detects rotations of gears in the driveline 64, but other speed sensors may be used to detect or infer the speed of the vehicle 14 (RF waves, inductive sensing, capacitive sensing, etc.). Further, the response control system 54 may include and / or be in communication with a display 68, such as a human-machine interface (HMI) 70, within the compartment of the vehicle 14. The display 68 is configured to present messages to a user and / or allow the user to control the window wiper system, the light control system 62, or any other aspect related to the visibility and / or movement of the vehicle 14.In general, the response control system 54 may be configured to actively control visual and / or motion features for the vehicle 14 or to passively present messages on the display 68 to indicate visual and / or motion target actions for the user to initiate.

[0023] With continued reference to Fig. 2, the window wiper system may include a window wiper controller 72 that controls the operation of a pump 74 that pressurizes the cleaning fluid to spray the cleaning fluid onto the windshield 18 via the nozzle 28, as previously described. A valve 76 may be fluidly connected between the pump 74 and the nozzle 28 to selectively allow the cleaning fluid to enter the nozzle 28.

[0024] The window wiper controller 72 may further be in communication with a motor 78 that drives the windshield wipers 26. For example, the motor 78 may be configured to rotate the windshield wipers 26 in response to signals from the window wiper controller 72 across the windshield 18. In some examples, the speed of the vehicle 14, as detected by the speed sensor, may be compared to detected splash events 118, and in response to this detection, the motor 78 may be energized to operate at a specific number of revolutions per minute (RPM). At least one switch 80 is in communication with the window wiper controller 72 and / or directly in communication with the motor 78, the pump 74, and / or the valve 76 to control dispensing the cleaning fluid and / or driving the windshield wipers 26 via manual interaction.For example, the at least one switch 80 may include a first mechanism 82 that causes the cleaning fluid to be dispensed onto the windshield 18 and a second mechanism 84 that controls the operation of the wipers 26. For example, the first mechanism 82 may be a button that, when pulled or pushed, causes the cleaning fluid to be dispensed, and the second mechanism 84 may be a button or knob that causes the wipers 26 to move across the windshield 18. It is contemplated that the window wiper control 72 may be omitted in some examples and that the response control system 54 may directly control the window wiper operations. In such an example, at least one of the switches 80 is located between the valve 76 and the nozzle 28 (e.g., the first mechanism 82) and at least one of the switches 80 is located between the motor 78 and the wipers 26 (e.g.,the second mechanism 84). In both examples, the user may manually control the dispensing of the cleaning fluid and / or the operation of the windshield wipers 26, and such operations may additionally or alternatively be automatically controlled by the response control system 54. In another example, instructions for initiating wiping of the windshield 18 and / or cleaning of the windshield 18 are depicted at the MMS 70, and the automatic control of the window wiping system is omitted.

[0025] The vision control unit 60 may also or alternatively be configured to control the lighting assemblies of the vehicle 14 based on moisture conditions detected by the imaging system 50, as previously described. For example, if it is raining in the region outside 12 of the vehicle 14, the headlights 48 may be automatically turned on, or in response to the detection of moisture conditions in the region outside 12 of the vehicle 14, the response control system 54 may communicate an instruction to present a message on the display 68 for the user to turn on the headlights 48. Further, brightness levels (e.g., binary high / low beam or fine brightness controls) may be actively or passively controlled (e.g., displaying messages on the display 68) by the response control system 54.

[0026] In general, the motion control unit 58 may control the various systems of the vehicle 14 related to motion control, such as driving the wheels 40 (e.g., torque values), braking the braking system 66 (e.g., traction control, anti-lock brakes (ABS)), steering the vehicle 14 (e.g., maneuvering along the driving surface 44), or any other motion control unit 58 for the vehicle 14. Similar to the operation of the vision control unit 60, features of the vehicle 14 related to motion control may be presented on the display 68 in addition to or alternatively to automatically controlling the maneuverability of the vehicle 14.For example, in an at least partially semi-autonomous control mode for the vehicle 14, the vision control unit 60 may communicate an indication to the user on the display 68 to reduce speed, maneuver the vehicle 14 left or right, or the like, and in the alternative may control the speed, maneuverability left or right, etc. in response to the detection of the moisture conditions.

[0027] With continued reference to Fig. 2, the various responses communicated by the response control system 54 may be based on outputs from one or both of the imaging system 50 and the proximity detection system 52. With specific reference to the imaging system 50, the imaging device 20 captures one or more images (captured images 86) of the region exterior 12 of the vehicle 14, which are then processed by an image processor 88 to detect moisture conditions on or around the vehicle 14 and generate one or more output images 90. The image processor 88 is in communication with a memory 91 that stores instructions that, when executed, cause the image processor 88 to detect water streaks, water droplets 108, splash events 118, splash sources 116, or any other optical distortion associated with moisture condition detection and / or obstruction detection.Included within or in communication with the memory 91 is a fully convolutional data description network (FCDDNN 92), which may be a neural network that segments the image data and detects optical distortions in the acquired images 86. The FCDDNN 92 may be trained by a training module 94 of the imaging system 50 that provides sample images and / or historical image data demonstrating optical distortion (e.g., optical distortion caused by moisture in an image). The FCDDNN 92 is employed to detect portions of the image that exhibit distortion due to moisture conditions and is described with reference to FIG. Fig. 7 described in more detail.

[0028] In addition to detecting moisture conditions, the image processor 88 is configured to detect any other visual obstruction within the field of view of the imaging device 20. For example, the visual obstruction may be environmental debris, such as animal droppings, leaves, sticks, non-aquatic material, or any other substance that may adhere to or land on the exterior surface 16 of the windshield 18. The imaging system 50 can therefore distinguish between dirt and water. For example, the FCDDNN 92 can be trained to rate the visual obstruction with a degree of opacity, light transmission, light distortion, or the like. For example, dirt may be associated with opacity, whereas water and / or other moisture conditions may be associated with the diffraction of light by the visual obstruction.

[0029] In general, distance information from the distance detection system 52 and moisture classification and detection from the image processing system may be used by the response control system 54 to initiate the response of the vehicle 14. For example, if moisture above a certain threshold (e.g., a low visibility threshold) is detected on the windshield 18, the imaging system 50 may communicate an output to the response control system 54 indicative of environmental conditions of the outside region. For example, the imaging system 50 may determine that it is raining, snowing, or otherwise precipitation is present in the outside region 12 and communicate an indication to the response control system 54 to start the windshield wipers 26.The image processing system may further distinguish between different forms of precipitation and / or obstructions on the windshield 18 to enable an individualized response initiated by the control system based on the ambient conditions. For example, and as described with reference to the previous figures, the imaging system 50 may classify types of obstructions as moisture-related or light-related, and the signal communicated by the imaging system 50 to the response control system 54 may depend on the type of obstruction.

[0030] For example, the imaging system 50 may detect animal excrement on the windshield 18 and communicate an output to the response control system 54 to control the pump 74 of the window wiper system to spray the cleaning fluid onto the windshield 18. In another example, the imaging system 50 detects spray onto the windshield 18 from another vehicle 14 in front of the vehicle 14 and, in response, communicates a signal to the response control system 54 to control the vehicle 14 to slow down or adjust the positioning of the vehicle 14, or communicates a message to the HMI 70 for the user to perform this control.The lighting assemblies may be further controlled in response to detecting the humidity conditions to illuminate the region outside 12 of the vehicle 14 at a particular power / illumination level and / or to turn the headlights 48 on or off. These responses are exemplary and not limiting, so any combination of responses may be performed by the humidity detection system 10 in response to the classification by the imaging system 50. With continued reference to FIG. Fig. 2, feedback from the window wiping system, the humidity control system, the HMI 70, or other vehicle systems may be monitored by the response control system 54 to optimize the response by the humidity detection system 10. For example, upon classification by the imaging system 50 of the ambient conditions in the region outside 12 of the vehicle 14, the imaging system 50 may communicate an instruction or signal to the response control system 54 that it is lightly raining in the region outside 12 of the vehicle 14. In response to detecting the light rain, the response control system 54 may start the windshield wiper control at a low speed. However, manual control of the windshield wipers 26 to stop wiping the windshield 18 via the second switch 80 may terminate the automatic control of the windshield wipers 26. The response control may detect the feedback (e.g.,that the user manually turns off the windshield wipers 26) and update or optimize future responses when light rain is detected. In this way, the detection, classification, and response to environmental conditions, including humidity conditions, may be enhanced to promote optimized responses by the user. It is contemplated that other examples relating to other environmental conditions may be similarly optimized using feedback (e.g., manual adjustment of lighting assemblies, manual steering adjustments, braking adjustments, etc.). In some examples, the user's dismissal of the messages presented on the HMI 70 may indicate that the response specified in the message is not preferred by the user. Other examples of feedback and control are described with reference to subsequent figures.

[0031] Referring to the Fig. 3A-4 and with further reference to Fig. 2, in some aspects, the vehicle 14 includes the camera configured to capture images through a window (e.g., the windshield 18) of the vehicle 14. The window wiping system is operable between a wiping mode and a cleaning mode. The window wiping system includes one or more of the wipers configured to move along the window in the wiping mode and the cleaning mode. The window wiping system also includes an interface (e.g., the at least one switch 80) for manually enabling the wiping mode.The control circuit is configured to detect a visual obstruction on the window based on the images, classify the visual obstruction as water or dirt, detect manual operation of the windshield wiper 26, determine a wear condition during manual operation of the windshield wiper 26 and classification of the visual obstruction as dirt, and communicate a signal to indicate the wear condition.

[0032] In general, the window wiping system can operate in an automatic mode and a manual mode. In the automatic mode, the wiping system works in conjunction with the control circuitry (e.g., the image processor 88, the controller 56, the controller 72, and / or other control circuitry), the imaging system 50, and / or the distance detection system 52 to operate the wipers 26, the nozzles 28, the valve 76, the pump 74, the motor(s) 78, and the like in response to the detection of obstructions on the windshield 18, as identified by the imaging device 20. For example, the imaging system 50 can detect water droplets 108, streaks or other moisture conditions, dust ( Fig. 3A) or other dirt, biological residues, such as impacting insects ( Fig. 3B), or other deposits or occlusions on the windshield 18. Thus, the image processor 88 may process the images 86 and apply the FCDDNN 92 or other neural network to classify the obstruction. Based on the classification of the obstruction, the specific operation performed by the window wiping system is determined. For example, a wiping operation may be determined in response to humid conditions, and a cleaning operation may be determined in response to non-humidity obstructions. Such operations may be performed automatically in the automatic mode, similar to the operation of automatically activating headlights in response to darkness. In this way, manual intervention may be limited.

[0033] In the cleaning mode, a cleaning fluid spray device is controlled to apply the cleaning fluid to the windshield 18. For example, the controller 56 controls the pump 74 and / or the valves 76 to dispense the cleaning fluid to the windshield 18 while (or just before) signals are communicated to one or more of the motors 78 to drive rotation of the windshield wipers 26 across the windshield 18. The cleaning fluid may serve to reduce adhesion of the obstruction to the windshield 18 to allow the wipers 26 to remove the obstruction. The wiping mode may not involve activation of the spray and may involve only operation of the wipers 26.

[0034] It is contemplated that, in addition to selecting between wiping and cleaning operations, the window wiping system may determine the speed of the wipers 26 (e.g., rpm of the motors 78), the timing of the wipers 26 (e.g., fast initially, then slower, duration of application, etc.), a speed or distribution of the cleaning fluid, or other more specific aspects of the window wiping system. For example, the imaging system 50 may more specifically classify conditions of the windshield 18, such as "very dirty," "dirty," "lightly dirty," etc., based on the degree of obstruction or the location of the obstruction through the windshield 18. In one example, the imaging system 50 detects obstructions on only a portion of the windshield 18 and activates only the appropriate portion of the window wiping system to remove the obstruction (e.g.,only one windscreen wiper 26, one of the nozzles 28, etc.).

[0035] The window cleaning system may also or alternatively be manually controlled by a user via an interface, such as switches 82, 84. For example, switches 82, 84 may be knobs on a control lever that, when turned, initiate cleaning or wiping operation depending on which switch 82, 84 is turned on. Manual activation or deactivation of switches 82, 84 may be monitored by the window cleaning system. Upon detecting a change in operation, controller 56 may, for example, update an algorithm to optimize the timing and conditions for activating or deactivating the wipers 26 and / or the spray device.For example, if the wipers 26 are automatically operated in response to moisture on the windshield 18 in the automatic mode, but the user manually turns off the wipers 26, the response control system 54 may update itself based on this manual feedback to delay the activation of the wipers 26 for a longer time in the automatic mode. In another example, elapsed usage time for the cleaning or wiping operation may be tracked (e.g., until the user manually turns off the cleaning or wiping operation) to optimize intervals for operating the window wiping system. Thus, the present system 10 may utilize learning techniques, such as those described with respect to the training module 94, for optimized responses.

[0036] The controller 56 or another portion of the control circuit may include a database or other memory 91 (e.g., the memory 91) that stores wear information, including a usage time of the windshield wiper(s) 26. For example, an elapsed usage time for the windshield wiper 26 since a blade change for the windshield wiper(s) 26 may be tracked and compared to an elapsed usage time for the windshield wiper(s) 26 at the wear condition. For example, if the user manually engages wiping when dry or non-wet particles are present on the windshield 18 (as detected by the imaging system 50), the system 10 may track a duration of how long the windshield wipers 26 are used at that wear condition.For example, a blade of the windshield wiper 26 may employ natural or synthetic polyisoprene, butadiene, ethylene propylene diene rubber, neoprene, or blends thereof, which may wear over time, particularly more rapidly when used on dry or rough surfaces, where a solvent may be useful to limit blade wear. Thus, the windshield wiper 26 may wear in the form of wear of the windshield wiper blade, overspeed of the motor 78 due to the drag caused by the windshield wipers 26 capturing debris, etc. The wear condition may be communicated to the user via a notification device, such as the MMS 70, audible speakers, light indicators, or any other visible or audible notification.

[0037] As set out herein with respect to Fig. 6A-9B, the imaging system 50 may classify moisture conditions on the windshield 18 based on the degree of optical distortion or occlusion. For example, the imaging system 50 may utilize the image processor 88 to distinguish between moisture conditions and non-moisture conditions on the windshield 18 based on a degree of transparency, opacity, color, or the like. Therefore, drops 108, streaks, or the like of moisture conditions may be distinguished from dirt.

[0038] With reference to Fig. 4, an exemplary method 400 for an automatic wiping mode of the window wiping system includes activating the automatic mode at step S402. In the automatic mode, the system 10 recursively classifies the obstruction(s) on the windshield 18, including the degree of obstruction and the type of obstruction, at S404. If an obstruction is detected (e.g., wipe = yes), the window wiping system activates the windshield wiper(s) 26 at S406. Simultaneously with or in temporal proximity to the detection of the obstruction, the obstruction is classified as moisture-based or non-moisture-based (e.g., primarily dirt), and the system 10 determines whether to apply the cleaning fluid at step S408.For example, primary dry dirt may warrant the cleaning operation, and at step S410, the spray device sprays the cleaning fluid onto the windshield 18 to loosen the dirt and allow the wipers 26 to remove the dirt. It is contemplated that the method 400 presented herein is merely exemplary, and that another modification (e.g., manual override) may limit the automatic mode, and, as previously described, timing and / or classification standards may be adjusted based on manual feedback to optimize the response of the window wiping operation.

[0039] The wear condition may be determined based on a function of usage cycles and / or environmental conditions, such as outside climate, air quality, etc. For example, the system 10 may include one or more temperature sensors, humidity sensors, wind speed sensors, wind direction sensors, or other weather detection sensors that can detect the environmental conditions. The system 10 may also use a geospatial location (e.g., GPS, as discussed below) to determine environmental conditions for the vehicle 14. For example, the vehicle 14 may be used primarily in a geographic region that has one or more common environmental conditions. A vehicle 14 used in a dry climate with high temperatures may be exposed to sandy conditions that typically result in sand on the windshield 18 (e.g., the front window), similar to the dust found in Fig. 3A. Use of the windshield wipers 26 during such conditions may result in greater wear of the windshield wipers 26 than use of the windshield wipers 26 during humid conditions. As another example, use of the windshield wipers 26 in cold or icy conditions may result in greater wear than during typical humid conditions. The control circuit may determine operation of the vehicle 14 in snowy climates and / or in northern geographic regions where salt is typically used to apply to the roads. Classifying the environment as containing road salt may affect the wear condition estimate. For example, road salt may increase blade wear because the salt mixes with moisture to provide salt water, which may increase wear of the windshield wipers 26. Other conditions, such as muddy conditions (e.g.,Humidity conditions combined with dirty conditions can affect wear conditions. Accordingly, the control circuit can update the wear condition in response to the environmental conditions.

[0040] The system 10 may also, or alternatively, provide multiple wear classifications for the response control system 54. Some environmental conditions may be more likely to cause wear or a high degree of wear on the windshield wipers 26 than other environmental conditions. Accordingly, the interaction between the windshield wipers 26 and the windshield 18 may be classified by the control circuitry. For example, the imaging system 50 may detect visual clarity through the windshield 18 before and after a window wiping operation occurs. In a first classification, the wear classification is minimized while allowing suboptimal windshield wiping. For example, if insect debris is present on the windshield 18, the imaging system 50 may determine sufficient humidity conditions for a non-spray wiping operation.However, streaks can be observed during operation. Accordingly, the control circuitry can communicate an instruction to alert the user (visually or audibly) to suboptimal cleaning and recommend (or automatically initiate) spraying in response to streaking. In this classification, wear is classified as minimal wear and wear conditions are minimized.

[0041] In another example, the system 10 classifies the interaction with a second wear classification in which a result of the performed wiping operation has led to increased wear of the windshield wipers 26. For example, if sand is detected on the windshield 18 (similar to the dust in Fig. 3B), interaction of the wipers 26 with the dry sand without spray may cause significant friction between the blades and the windshield. Scraping noises or the like may be detected by the system 10 via microphones. Based on the classification of the visual obstruction and / or the audible noises from the interaction with the wipers 26, the control circuit may classify the wear and update the wear condition accordingly. Further, the control circuit may automatically turn on the spray device in response to the scraping or otherwise communicate an instruction to the user to turn on the application of cleaning fluid. In yet another example, a third wear classification may be determined by the control circuit.The third wear classification may correspond to interactions that result in at least a portion of the blades of the windshield wiper 26 becoming separated or lost from the body of the windshield wiper 26. Such classifications may be detected via image / video analysis via the imaging system 50 of the windshield 18 and thus of the windshield wipers 26. For example, a trailing blade may be classified by the imaging system 50.

[0042] Other classifications may be determined. Thus, each interaction may be given greater or lesser weight to influence the calculation of the condition of the windshield wipers 26. Outside temperature, humidity, wind speed, or other environmental conditions may be taken into account by the control circuit. For example, in freezing conditions (temperatures around freezing or around 0 degrees Celsius), it may be determined that the obstructions on the windshield are due to solid ice conditions, while visually representing them as typical humid conditions. If the windshield wipers 26 are nevertheless turned on, the obstruction may still be present due to the obstruction being solid ice and not having been removed by the wiping operation. In such an example, the control circuit may classify the interaction as significantly increasing the wear condition of the windshield wipers 26.The control circuit may be configured to communicate a signal to indicate to the user that the windshield wipers 26 should not be operated under such conditions. In another example, the control circuit interrupts operation of the windshield wipers 26. Accordingly, the classification step in step 404 of method 400 may include the various wear classifications illustrated and result in different responses by the response control system 54.

[0043] It is contemplated that each wear classification may correspond to a modifier, variable, or multiplier applied to each wear condition estimate. For example, the control circuit may modify the calculation or algorithm for determining the wear condition (e.g., multiplying the usual wear condition by two, three, ten, etc.). In this way, the normal wear of the blades of the windshield wiper 26 may be increased. The service life of the windshield wipers 26 may therefore be re-estimated by the control circuit. For example, using the windshield wipers 26 when a large modifier is present (e.g., very cold temperatures with ice, very hot temperatures on the windshield 18, etc.) may result in a significantly reduced service life of the windshield wipers 26.

[0044] With reference to Fig. 5, an exemplary arrangement of the imaging device 20 relative to the windshield 18 is illustrated to demonstrate the proximity of the imaging device 20 relative to the windshield 18 and the improved optical and spatial properties of the arrangement. Although illustrated as being disposed adjacent to an upper portion 96 of the windshield 18, it is contemplated that the imaging device 20 may be disposed along any portion of any window of the vehicle 14 that allows the imaging device 20 to capture a view of the region exterior 12 of the vehicle 14. In general, the software employed by the moisture detection system 10 of the present disclosure enables the positioning of the imaging device 20 to be at least partially independent of the distance between a lens 98 of the imaging device 20 and the window.For example and as with respect to . Fig. 6A and Fig. 6B, water on the windshield 18 (e.g., condensation, liquid water, frost) may appear differently in the images depending on a distance of the imaging device 20 relative to the windshield 18. For example, the water droplets 108 may appear as individual circles, dots, or other geometric shapes that are more clearly defined along the edges 110 of the shapes when the imaging device 20 is spaced farther from the windshield 18 than when it is adjacent to the windshield 18 or very close (e.g., between 0 and 35 mm) to the windshield 18. The processes and methods performed by the moisture detection system 10 of the present disclosure may include detecting moisture conditions at one or both of a first position 102 of the imaging system 50 (e.g., at a first distance) and a second position 104 of the imaging device 20 (e.g.,at a second distance that is smaller than the first distance). For example, due to the imaging device 20 being close to the water on the windshield 18, the moisture on the windshield 18 may appear as blurry streaks or less defined optical distortions when an imaging device 20 is at the first distance from the windshield 18 than when it is positioned at the second distance. By providing a distance-independent arrangement, the moisture detection system 10 enables improved spacing within the space, a larger and more detailed field of view for the imaging device, and universal application to accommodate imaging devices at different distances from the windshield 18.

[0045] With continued reference to Fig. 5, the windshield 18 may extend at an oblique angle 106 relative to a vertical and / or horizontal orientation. For example, the windshield 18 may be tilted upward relative to the driving surface 44. The oblique angle 106 of the windshield 18 may further distort or otherwise affect how humidity conditions appear in the images captured by the imaging device. The distortion may be more pronounced in examples where the imaging device 20 is in the second position 104 relative to the first position 102.Accordingly, the challenges of capturing humidity conditions of the environment outside the vehicle 14 from images captured by an imaging device 20 inside the compartment may be greater than or different from image processing for humidity detection from images captured by imaging devices outside the compartment. For example, water droplets 108 on the windshield 18 may expand as they land on the windshield 18, making them more elongated than droplets 108 on a vertical or more sloped surface, such as the surface of a passenger window, a rear window, an exterior camera cover, etc.

[0046] With reference to Fig. 6A and Fig. 6B, the differences in the captured images 86 from the first position 102 relative to the second position 104 of the imaging device 20 in Fig. 5. As shown in Fig. 6A (the first position 102), rainwater may appear on the outer surface 16 of the windshield 18 as drops 108 with individual shapes or edges 110 that are more clearly defined than a blur 112 due to water conditions, as in Fig. 6B (the second position 104). Using the current algorithm and processes by the moisture detection system 10, such optical distortion, even if slight, may be detected. For example, a drop detection algorithm may be suitable for detecting moisture conditions when the imaging device 20 is positioned far from the windshield 18, and a blur detection algorithm 112 may be employed by the imaging system 50 to detect moisture conditions on the windshield 18 when the imaging device 20 is positioned closer to the windshield 18 ( Fig. 6B). Accordingly, the moisture detection system 10 may provide improved standoff within the vehicle 14 by providing the imaging device 20 near the windshield 18.

[0047] In some examples, the second position 104 is located within 50 mm of the windshield 18. In some examples, the second position 104 is located between 0 mm and 35 mm from the windshield 18. In some examples, the second position 104 is positioned between 0 mm and 10 mm from the windshield 18. In each of these examples, the second position 104 is located near the windshield 18 to provide improved field of view and clearance within the space. In these examples, the first position 102 is farther from the windshield 18 than the second position 104.

[0048] With reference to Fig. 7 shows how the FCDDNN 92 converts the captured image from Fig. 6B to generate the corresponding output image. The FCDDNN 92 is a deep learning model designed for the purpose of unsupervised irregularity detection in data and operates by learning the underlying patterns and structures within the image data to distinguish normal patterns from anomalous patterns. The FCDD may include a plurality of layers 114, each layer being a convolutional layer. Because each layer may be convolutional, the FCDDNN 92 can capture spatial dependencies and preserve the spatial structure of the input data.

[0049] In the FCDDN, the input data is passed through a series of convolutional layers 114 that extract relevant features at different levels of abstraction. These convolutional layers 114 may be followed by pooling layers 114 to downsample feature maps and reduce spatial dimensions of the feature maps. The output of the convolutional layers 114 may then be flattened and fed into fully connected layers 114, which perform further feature extraction and map the learned features to irregularity scores. Irregularity detection may be achieved by comparing the computed irregularity scores to a predefined threshold, with scores above the threshold indicating anomalous cases. In the present examples, the predefined thresholds may correspond to edge continuity of shapes in the image to detect distortions caused by moisture (e.g.,blurriness). These thresholds can be actively set based on the user feedback described above (e.g., the user manually turns on the windshield wipers 26 and / or the wiping process, the user manually turns on the headlights 48, etc.).

[0050] With reference to the Fig. 8A-8C depict exemplary captured images 86 from the imaging device 20 directed toward the windshield 18, alongside filtered image data (output images 90) indicative of moisture conditions on the windshield 18. The imaging system 50 may process the captured images 86 in the image processor 88, including processing the captured images 86 in one or more machine learning models and / or neural networks. The imaging system 50 may employ edge detection techniques, histogram equalization, linear filters, image segmentation, convolution, or any combination of any image processing techniques to detect at least a portion of the captured image that has a distortion or obstruction. For example, and with reference to Fig. 8A and Fig. 8B, the imaging system 50 may detect the blur 112 on the windshield 18 corresponding to humidification conditions (e.g., moisture on the windshield 18). In another example, the imaging system 50 detects non-humidity conditions, such as one of the windshield wipers 26 moving across the windshield 18 ( Fig. 8C). In these examples, detection may be applied to the captured images 86 to associate groups of pixels of the captured images 86 with surrounding pixels to classify the image data as moisture-related or non-moisture-related. For example, lane detection may be employed by the imaging system 50 to identify one or more lane lines defining the plurality of lanes 126. In general, object classification may be performed by the imaging system 50, such as classifying other vehicles 14, streetlights, trees, or any other object captured in the captured images 86.

[0051] In general, the FCDDNN 92, previously described in relation to Fig. 7, to provide a wetness estimate. Based on the wetness estimate, which may be a continuously changing value or a binary wet or non-wet value, the response control system 54 may determine the response of the moisture detection system 10. By employing the various layers 114 of the FCDDNN 92 in combination with manual feedback from the user, as previously described with respect to Fig. 2, the techniques employed by the moisture detection system 10 may provide an improved response. For example, based on the training of the FCDDNN 92, the windshield wipers 26 may be turned on when the user would normally turn on the windshield wipers 26 or before the user would normally turn on the windshield wipers 26. Further, the speed of the windshield wipers 26 and / or the turning on of the pump 74 to clean the windshield 18 may be optimized based on the moisture level and / or previous turning ons of the windshield wipers 26 or the pump 74. The headlights 48 of the vehicle 14 may also, or alternatively, be turned on with limited misturns. For example, the FCDDNN 92 detects and classifies the image data as oncoming headlights 48 and distinguishes oncoming headlights 48 from moisture conditions or non-moisture conditions.Furthermore, objects with pronounced moiré effects (e.g., fences) may be distinguished by the FCDDNN 92. In some examples, the manual feedback includes manually operating the windshield wiper 26 (e.g., via the at least one switch 80) and / or maneuvering the vehicle 14 away from a splash event 118. Further, and as described below, adjusting the vehicle 14 to a different lane or maintaining positions in a current lane may be manual feedback (e.g., ignoring a lane change recommendation or manually overriding the move to a different lane).

[0052] With reference to Fig. 9A and Fig. 9B, the identification of at least one splash source 116 of splash events 118 may be performed by the imaging system 50 of the moisture detection system 10. For example, the splash sources 116 may include vehicles 14 and non-vehicles 14. The splash source 116 may be tires 42, a vehicle body, bridges, overpasses, construction equipment, fire hydrants, or any other source. Accordingly, the imaging system 50 may be configured to classify the sources using any of the previously described imaging techniques.

[0053] With continued reference to Fig. 9A and Fig. 9B, the imaging system 50 may further or alternatively determine a location of the spray source 116 relative to the vehicle 14. Based on the location of the spray source 116 (e.g., a source lane 126a), the response control system 54 may control any of the window cleaning system, the lighting system, the HMI 70, and / or the motion control system. For example, the imaging system 50 may locate the spray source 116 in a different one of the plurality of lanes 126 than the lane of the vehicle 14 and, based on the location of the spray source 116 in a different lane, recommend maintaining course in the current lane of the vehicle 14. Thus, the moisture detection system 10 may generally control the vehicle 14 or provide a message to control it to adjust lanes based on the location of the spray source 116. In the example, as shown in Fig. 9B, multiple splash sources 116 are detected by the imaging system 50, and the response control system 54 may recommend one of the plurality of travel lanes 126 based on the proximity of the vehicle 14 to one or more of the plurality of splash events 118 (e.g., based on a following distance 120 of the vehicle 14 to the other vehicles 14, based on a relative position of other vehicles 14 to other sources of splash, such as a pothole). In other words, the moisture detection system 10 may detect a first splash source 116 and a second splash source 116, classify each splash source 116 with a relevance score, and recommend a travel lane or other location for the vehicle 14 based on the relevance scores of the first and second splash sources 116.As described below, the moisture detection system 10 may further recommend a target following distance 120, a target speed, or the like in response to the relevance values.

[0054] With specific reference to Fig. 9B, at least one splash zone 122 may be determined or calculated by the imaging system 50 based on image data from the initial images. For example, any of the previously described techniques, including the application of the FCDDNN 92, may enable the imaging system 50 to estimate dimensions of the splash zone 122, such as a depth D, a width W, and a height H of the splash. The dimensions of the splash zones 122 may be determined based on the following distance 120 from the source of the splash to the vehicle 14 and the captured images 86. For example, the following distance 120 may be determined using the distance sensors 34, 36, 38 previously described with respect to Fig. 1 and Fig. 2, and the leading edge of the splash may be detected based on image processing of the captured images 86. The difference between the following distance 120 and the leading edge of the splash zone 122 may be calculated by the control circuitry to determine the depth D of the splash zone 122. In this manner, the distance detection and location detection of the system 10 may provide for determining the origin point of the splash, the size of the splash, the probability of water contact of the splash based on distance and / or speed, and the like.

[0055] In the present example, three splash zones 122 are detected by the imaging system 50, each of the splash zones 122 having a different size. The size of the splash zones 122 may be caused by the sizes of the tires 42, the speeds of the vehicles 14, the lanes in which the vehicles 14 are located (e.g., road roughness, potholes, etc.), or any other factor that may affect the size of the tire splash. For example, uneven road surfaces that may cause puddles, curvatures in the road (e.g., a drop in the road, curves), or the like may further affect the size of the splash. Based on the size of the splash, the imaging system 50 may determine the following distance 120 from any or all of the splash sources 116 (e.g., the other vehicles 14).In general, splash sizes, splash density, splash duration, or any other aspect related to the magnitude of the splash may be detected by the imaging system 50 and used to classify a priority or ranking (e.g., the relevance values) of the splash events 118 to determine a target lane from the plurality of lanes 126, a target following distance 120 (e.g., a minimum following distance 120) for the vehicle 14 relative to other vehicles 14, activation of the window wiper system, presentation of messages to the HMI 70, control of the vehicle 14, or any of the previously described responses. The direction of the splash may also be detected, which may be influenced by wind speed and direction, to enhance the estimation of the splash zones 122 and further enhance the response determined by the response control system 54.Such wind speed and direction can be detected using weather sensors or a global positioning system (GPS) in communication with the moisture detection system 10.

[0056] The moisture detection system 10 may further or alternatively estimate the density of moisture conditions, such as the density of the splash zone 122 or the density of rain. For example, the moisture detection system 10 may estimate the density of rain based on the amount, distribution, or pattern of blurs 112 or other moisture patches detected on the windshield 18 and, in response to the amount, distribution, or pattern exceeding a threshold or matching a target distribution or pattern, turn on the windshield wipers 26 or communicate an indication to the user to operate the wipers 26. For example, if more than 25%, 50%, or 90% of the windshield 18 in the field of view of the imaging device 20 has blurs 112, the imaging system 50 may communicate a signal to the response control system 54 to start the wipers 26.In other examples, the size of the water droplets 108 and / or blurs 112 may be categorized by the FCDDNN 92 and compared to stored humidity conditions to determine the response of the humidity detection system 10.

[0057] In some examples, the moisture detection system 10 may access the position of the vehicle 14 relative to the roadway type. For example, the GPS may provide the position of the vehicle 14, thereby providing the roadway type (e.g., highway, city driving, etc.). In this way, the number of lanes, direction of travel, construction, and / or operation of the roadway (e.g., one-way streets, highways with medians, expressways without medians) may be determined. Accordingly, the moisture detection system 10 may prioritize some lanes over others or adjust following distances 120 further based on roadway types.For example, since the typical speed of a vehicle 14 on one highway lane is greater than on another lane, the moisture detection system 10 may suggest a “slow lane” of two lanes based on moisture conditions detected on the highway lane.

[0058] Although the source lane 126a of the plurality of lanes 126 is illustrated in the previously described features as having the same traffic direction as the other of the plurality of lanes 126, in some examples, the plurality of lanes 126 includes lanes having a first traffic direction and lanes having a second traffic direction. In this example, the imaging system 50 is configured to classify the spray source 116 as being in a lane having the first traffic direction and / or the spray source 116 as being in a lane having the second traffic direction. For example, the imaging system 50 may detect oncoming headlights 48 in an adjacent lane and, in response to that detection, classify the adjacent lane as having an opposite traffic direction.Thus, splashing from oncoming traffic can be compared to splashing from preceding traffic, and the moisture detection system 10 can provide an improved response determination for a target lane for the vehicle 14 based on one or both of the splashing events 118 in the first and second directions of traffic. It is also contemplated that the previously described distance detection system 52 can be used to further determine the target lane for the vehicle 14 by tracking the following distance 120 from the vehicle 14 to the preceding vehicle 14.

[0059] With reference to Fig. 10A and Fig. 10B shows visual representations of RADAR assignments as they relate to the Fig. 9A and 9B, respectively. In these examples, the density of the hatching corresponds to the density of the splash events 118. The control circuit can generate the density map via the distance detection system 52 and compare such a density map to predefined thresholds to classify the splash events 118 with an intensity level.

[0060] According to one example of the present disclosure, a detection system for a target vehicle 14 (e.g., a following vehicle 14) includes a camera (e.g., camera(s) 38) that captures images of an area exterior 12 of the target vehicle 14. The detection system further includes a RADAR module (e.g., one or more of the RADARs 34) that scans the region 12 to detect a depth D of a splash event 118 in the region 12. An actuator, such as one or more actuators of the response control system 54, is configured to operate in response to a response signal. The control circuit is configured to determine a distance between the target vehicle 14 and a front of the spray event 118 based on the images, compare the front of the spray event 118 to the depth D to determine an intensity of the spray event 118, and communicate the response signal based on the intensity of the spray event 118.

[0061] The image processing and depth analysis techniques employed by the control circuitry to determine the intensity levels of the splash events 118 may include any of the methods or components (e.g., neural networks) previously described for moisture detection. For example, Doppler effect analysis may be employed by the control circuitry in conjunction with the radars 34 to detect the density of the splash events 118 and / or the depth of the splash events 118. The density may refer to the ratio of the volume of liquid to the volume of space (e.g., air) within the splash event. The front of the splash events 118 may be detected using pixel analysis (e.g., edge detection, neural networks, such as the FCDDNN 92) to determine positions of water droplets / humidity conditions relative to the vehicle 14.The distance between the front of the splash event 118 and the vehicle 14, as determined based on the images (e.g., the captured images 86 or the output images 90), and the depth D of the splash event 118, as determined based on the scans by the radars 34, can be synthesized at the control circuit. For example, a processor of the distance detection system 52, the imaging system 50, or the controller 56 of the response control system 54 can process the distance information and the depth information to calculate a location of the splash source 116. Further, the height H and width W of the splash events 118 can be used to estimate the splash zones 122 with improved accuracy compared to image-based methods alone. Thus, by employing the radars 34, accurate intensity levels can be determined.

[0062] With continued reference to Fig. 10A and Fig. 10B, the density of each splash zone 122 can be determined by the control circuit using the information from the radars 34 and is illustrated by cross-hatching (i.e., dense cross-hatching corresponds to high moisture density). Using the density information, the location and identity of the splash source 116 can be estimated / determined by the control circuit. Accordingly, in addition to a volume of the splash event 118 calculated using the enhanced splash zones 122, an origin point of the splash event 118 can be determined based on the density.

[0063] The size and density of the splash events 118 may correspond to the intensity of the splash event 118. For example, large splashes may have higher intensity levels compared to small splashes. The intensity levels may be weighted based on a speed of the vehicle 14 and / or a distance from the target vehicle 14 to the splash source 116 and / or the origin of the splash (e.g., a puddle, a pothole, a fire hydrant, or any of the previously described splash sources). For example, the splash intensity classification may be used by the response control system 54 to recommend or actively control a gap between the following vehicle 14 (e.g., the target vehicle 14) and a leading vehicle 14.For example, the information may be used by the system to recommend increasing a following distance 120 from a leading vehicle 14 based on the intensity of the splashing. In one example, a passing condition is determined by the control circuitry and communicated to the driver or another user via the display 68 or another notification device (e.g., an audio prompt). For example, if a driver is considering whether the driver should control the vehicle 14 to pass a leading vehicle 14 when splashing events 118 are present, the system may recommend "pass" or "do not pass." In this example, the controller 56 may compare the splashing events 118 from the leading vehicle 14 to splashing events 118 in a passing lane due to another leading vehicle 14 in an adjacent lane.

[0064] With reference to Fig. 11 illustrates a three-time instance of a method performed using radars 34 and camera(s) 38 to recommend a passing condition. At the first time (t=1), a target vehicle V1 incorporating the present moisture detection system 10 measures / tracks splash events 118 from a leading vehicle V2 in a current lane L1 and an overtaking vehicle V3 in a passing lane L2 using information from radars 34 and camera 38 as the overtaking vehicle V3 passes the leading vehicle V2. During this time, the following vehicle V1 optimizes a following distance 120 from the leading vehicle V2 in response to the intensity level of the leading vehicle V2 via a control over one or more of the vehicle systems' actuators (e.g., braking, motion control, etc.).Alternatively, the system 10 recommends the following distance 120 and the user manually controls the following vehicle V1.

[0065] Based on the previously measured splash event 118 generated by the passing vehicle 14 and an estimated speed of the overtaking vehicle V3 (as determined using speed detection via the radars 34 or other methods), the system 10 estimates a leading end F of the splash event 118 at a second time (t=2). It is contemplated that the speed estimate for the other vehicles 14 may be based on image analysis, a comparison with the speed of the following vehicle V1, information from the radars 34, or any other speed detection method. Based on the location of the leading end F of the splash event 118 relative to the following vehicle V1, the system 10 steers or provides an indication of steering the following vehicle V1 into the passing lane L2 at a third time (t=3).For example, if a distance between the leading end F and the following vehicle 14 exceeds a target following distance 120 (set for the leading end F of the splash event 118) or another threshold distance, the system 10 may indicate that appropriate passing conditions exist. Of course, other measures, such as traffic from behind or elsewhere, as well as other aspects, may be considered when consenting to pass. As described herein, the recommendation or consent to pass is relevant to moisture condition detection and not to other factors that may affect consent to the passing condition.

[0066] In general, the use of the radars 34 relative to the imaging can limit the impact of visual obstructions (sunlight, other lighting, other moisture conditions that obstruct a view of the camera(s) 34) on the depth detection for determining and / or executing the optimal following distance 120. By combining the image-based detection with the radar-based detection, more accurate determinations of splash intensity can be tracked to enable improved response (e.g., activation of the wipers 26, control of the vehicle 14, etc.). For example, the previously described aspects related to optimal window cleaning or wiping can be further optimized by having the system 10 more accurately estimate when moisture conditions on the windshield 18 should be removed (e.g., when the wipers 26 should be activated).

[0067] With reference to Fig.12, an exemplary process 800 performed by the moisture detection system 10 is shown for use with splash detection and moisture condition detection on the windshield 18. At step 802, the imaging device 20 captures images of the region exterior 12 of the vehicle 14. At step 804, the image processor 88 detects an event in the region next interior with respect to the vehicle 14. For example, the event may be moisture conditions, such as splash events 118, water on the exterior surface 16 of the windshield 18, objects on the windshield 18, or any other event related to obstructing or distorting vision. At step 806, the event is classified as associated with the windshield 18 (e.g., on the windshield 18) or remote from the windshield 18.For example, if the event is a splash event 118 related to other vehicles 14 in front of the vehicle 14, as opposed to water splashing on the windshield 18 itself. It is contemplated that the splash event 118 may be classified as both an on-windshield 18 event and an off-windshield 18 event depending on the following distance 120 between the splash source 116 and the vehicle 14. If the event is a visual obstruction on the outer surface 16 of the windshield 18, the image processor 88 may classify the visual obstruction at step 808. For example, the imaging system 50 may classify the visual obstruction as dirt or water. Based on the classification of the visual obstruction, the moisture detection system 10 determines a response at step 810.For example, the response may be initiating the application of wiper fluid to the windshield 18 in the event that the obstruction to vision is debris, such as wet debris, or activating or adjusting the speed of one or more of the wipers 26 on the windshield 18 in the event of wet conditions. At step 812, an output is communicated to initiate the response determined at step 810. It is contemplated that the response may alternatively be a more passive response, such as presenting a message on the display 68 to instruct a user to perform one or more of the functions, which may alternatively be performed automatically.

[0068] At step 814, feedback in the form of manual adjustment or non-operation (e.g., the user does not follow a recommendation) is communicated to the response control system 54 and / or the imaging system 50 to further refine the response determined in future events. For example, if the user is instructed to turn on the windshield wipers 26 and the user does not turn on the windshield wipers 26, the target humidity levels for determining wiper turn-on by the imaging system 50 may be increased to a threshold to limit incorrect responses for future calculations. Such a threshold may be the threshold for the previously described FCDDNN 92 or another threshold.If the event is not related to the conditions of the windshield 18, as determined in step 806, the process may continue at step 816 to determine a location of a splash event 118. For example, using the captured images 86, the imaging system 50 may detect the source and / or location (e.g., splash lane 126a) of the splash events 118 caused around the vehicle 14 (e.g., tire splash). At step 818, the response is determined by the response control system 54. For example, the response may be to adjust the following distance 120 between the splash source 116 and the vehicle 14 by reducing the speed of the vehicle 14. In other examples, the response includes maneuvering the vehicle 14 to another lane of a plurality of lanes 126.In other examples, the response includes presenting instructions via messaging on the HMI 70 to indicate to the user to maneuver the vehicle 14, adjust the speed of the vehicle 14, or the like. Other examples of the response include window cleaning system settings, such as turning on the windshield wiper 26, adjusting the speed of the windshield wiper 26, turning on the pump 74 to apply cleaning fluid, or the like. At step 820, the output may be communicated to initiate the response. Similar to step 812, at step 822, feedback in the form of an action or inaction by the user to reverse the response communicated by the response control system 54 is returned to the moisture detection system 10 to improve response determination in future conditions where splash events 118 are detected.

[0069] In general, the present moisture detection system 10 improves responses for the vehicle 14 to limit obstruction and / or distortion of the view of the region exterior 12 of the vehicle 14. The image processing techniques employed by the moisture detection system 10 can improve the space within the interior of the vehicle 14 by allowing the imaging device 20 to be positioned near the windshield 18. Further, the image processing techniques employed herein can provide improved detection of splash sources 116 and / or moisture conditions on the exterior surface 16 of the windshield 18. Based on the detection of these moisture events, rapid response times for wiping the windshield 18 and / or optimizing the maneuvering of the vehicle 14 can be provided by the moisture detection system 10.

[0070] As used herein, the term "and / or," when used in a list of two or more elements, means that any of the listed elements may be employed individually, or any combination of two or more of the listed elements may be employed. For example, if a composition is described as containing components A, B, and / or C, the composition may contain A alone; B alone; C alone; A and B in combination; A and C in combination; B and C in combination; or A, B, and C in combination.

[0071] Herein, reference terms such as first and second, upper and lower, and the like are used merely to distinguish one entity or act from another entity or act, without necessarily requiring or implying any actual such relationship or order between such entities or acts. The terms "comprises," "comprising," or any other variation thereof are intended to cover a non-exclusive inclusion, such that a process, procedure, article, or device that includes an enumeration of elements not only includes those elements, but may include other elements not expressly listed or inherent in such a process, procedure, article, or device. An element that "comprises...a" does not exclude, without further qualification, the presence of additional identical elements in the process, procedure, article or facility incorporating that element.

[0072] As used herein, the term "about" means that amounts, sizes, formulations, parameters, and other quantities and properties are not and need not be exact, but may be approximate and / or greater or lesser if desired, and may reflect tolerances, conversion factors, rounding off, measurement errors, and the like, and other factors known to those skilled in the art. When the term "about" is used to describe a value or an endpoint of a range, the disclosure should be understood to include the specific value or endpoint being referenced. Regardless of whether a numerical value or an endpoint of a range includes "about" in the description, the numerical value or endpoint of a range is intended to include two embodiments: one modified by "about" and one not modified by "about."It is further understood that the endpoints of each of the domains are significant both relative to the other endpoint and independently of the other endpoint.

[0073] The terms "substantially," "substantially," and variations thereof, as used herein, are intended to indicate that a described characteristic is equal to or approximately equal to a value or description. For example, a "substantially planar" surface is intended to mean that a surface is flat or approximately flat. Furthermore, "substantially" is intended to mean that two values are equal to or approximately equal. In some embodiments, "substantially" may refer to values within about 10% of each other, such as within about 5% of each other, or within about 2% of each other.

[0074] As used herein, the terms "the," "the," "a," or "a" or "an" mean "at least one" and are not intended to be limited to "only one" unless expressly stated otherwise. Thus, for example, a reference to "a component" includes embodiments having two or more such components, unless the context clearly indicates otherwise.

[0075] It is to be understood that variations and modifications may be made to the foregoing structure without departing from the concepts of the present disclosure, and it is further understood that such concepts are intended to be covered by the following claims unless these claims expressly state otherwise by their language.

[0076] According to the present invention, a vehicle is provided comprising: a camera configured to capture images through a window of the vehicle; a window wiping system operable between a wiping mode and a cleaning mode, including: a windshield wiper configured to move along the window in the wiping mode and the cleaning mode; and an interface for manually engaging the wiping mode; and a control circuit configured to: detect an obstruction on the window based on the images; classify the obstruction as water or dirt; detect manual operation of the windshield wiper; determine a wear condition during manual operation of the windshield wiper and based on the classification of the obstruction as dirt;and communicating a signal to indicate the wear condition.;

[0077] According to one embodiment, the window wiping system includes a spray device configured to apply cleaning fluid to the window during the cleaning operation. According to one embodiment, the control circuit includes a memory configured to store wear information, including a usage time of the windshield wiper.

[0078] According to one embodiment, the usage time includes a duration of the wear condition.

[0079] According to one embodiment, the window wiping system is operable in an automatic mode in which the cleaning and wiping operations are automatically activated in response to the detection of the obstruction to visibility.

[0080] According to one embodiment, the window wiping system includes a pump for pressurizing the cleaning fluid to be applied to the window in the cleaning operation.

[0081] According to one embodiment, a selection between the cleaning operation and the wiping operation in the automatic mode is determined based on the classification of the visual obstruction as water or dirt.

[0082] According to one embodiment, the window wiping system is configured to operate the wiper in response to the classification of the visual obstruction as water and to operate the spray device in response to the classification of the visual obstruction as dirt.

[0083] According to one embodiment, the control circuit is configured to control the window wiping system to switch from the wiping mode to the cleaning mode in response to the wear condition.

[0084] According to one embodiment, the control circuit is configured to classify the visual obstruction with a degree of optical distortion, wherein the classification of the visual obstruction as water or dirt is based on the degree of optical distortion.

[0085] According to one embodiment, the control circuit is configured to: capture a subsequent image of the window following the manual operation; and classify the wear condition with a modifier based on the subsequent image.

[0086] According to one embodiment, the modifier is a multiplier of a calculation for the wear condition, and wherein the control circuit is configured to select the multiplier from a plurality of modifiers corresponding to environmental conditions. According to one embodiment, the control circuit is configured to communicate with the modifier a signal for applying cleaning fluid to the window in response to the classification of the wear condition.

[0087] According to one embodiment, the invention is further characterized by: a notification device configured to indicate the wear condition in response to the signal.

[0088] According to the present invention, a vehicle is provided comprising: a camera configured to capture images through a window of the vehicle; a window wiping system operable between a wiping mode and a cleaning mode and including: a spray device configured to apply cleaning fluid to the window in the cleaning mode; a windshield wiper configured to move along the window in the wiping mode and the cleaning mode; and an interface for manually operating the windshield wiper; and a control circuit configured to: detect an obstruction on the window based on the images; classify the obstruction as water or dirt; detect manual operation of the windshield wiper;Determining a wear condition during manual operation of the windshield wiper and based on the classification of the obstruction as dirt; and communicating a signal to indicate the wear condition.;

[0089] According to one embodiment, the window wiping system is operable in an automatic mode in which the cleaning and wiping operations are automatically activated in response to the detection of the obstruction to visibility.

[0090] According to one embodiment, the window wiping system includes a pump for pressurizing the cleaning fluid to be applied to the window in the cleaning operation.

[0091] According to one embodiment, a selection between the cleaning operation and the wiping operation in the automatic mode is determined based on the classification of the visual obstruction as water or dirt.

[0092] According to one embodiment, the window wiping system is configured to operate the wiper in response to the classification of the visual obstruction as water and to operate the spray device in response to the classification of the visual obstruction as dirt.

[0093] According to the present invention, a vehicle is provided comprising: a camera configured to capture images through a window of the vehicle; a window wiping system operable between a wiping mode and a cleaning mode, including: a spray device configured to apply cleaning fluid to the window in the cleaning mode; a windshield wiper configured to move along the window in the wiping mode and the cleaning mode; and an interface for manually operating the windshield wiper; a notification device configured to indicate a wear condition for the windshield wiper in response to a signal; and a control circuit configured to: detect an obstruction on the window based on the images; classify the obstruction as water or dirt;Detecting manual operation of the windshield wiper; determining the wear condition during manual operation of the windshield wiper and based on the classification of the obstruction as dirt; and communicating the signal to indicate the wear condition.

Claims

[1] Vehicle comprising: a camera configured to capture images through a window of the vehicle; a window wiping system operable between a wiping operation and a cleaning operation and comprising: a windshield wiper configured to move along the window in the wiping operation and the cleaning operation; and an interface for manually switching on the wiping operation; and a control circuit configured to: Detecting an obstruction to the view on the window based on the images; Classifying the obstruction as water or dirt; Detecting manual operation of the windshield wiper; Determining a wear condition during manual operation of the windshield wiper and based on the classification of the obstruction as dirt; and Communicate a signal to indicate the wear condition. [2] The vehicle of claim 1, wherein the window wiping system includes a spray device configured to apply cleaning fluid to the window in the cleaning operation. [3] The vehicle of claim 1 or claim 2, wherein the control circuit includes a memory configured to store wear information, including a usage time of the windshield wiper. [4] The vehicle of claim 3, wherein the usage time includes a duration of the wear condition. [5] The vehicle of claim 2, wherein the window wiping system is operable in an automatic mode in which the cleaning and wiping operations are automatically activated in response to the detection of the obstruction to visibility. [6] The vehicle of claim 5, wherein the window wiping system includes a pump for pressurizing the cleaning fluid to be applied to the window in the cleaning operation. [7] A vehicle according to claim 6, wherein a selection between the cleaning operation and the wiping operation in the automatic mode is determined based on the classification of the obstruction as water or dirt. [8] The vehicle of claim 7, wherein the window wiping system is configured to operate the windshield wiper in response to classifying the visual obstruction as water and to operate the spray device in response to classifying the visual obstruction as dirt. [9] The vehicle of claim 8, wherein the control circuit is configured to control the window wiping system to switch from the wiping mode to the cleaning mode in response to the wear condition. [10] The vehicle of claim 9, wherein the control circuit is configured to classify the visual obstruction with a degree of optical distortion, wherein the classification of the visual obstruction as water or dirt is based on the degree of optical distortion. [11] The vehicle of claim 10, wherein the control circuit is configured to: Taking a subsequent image of the window following manual operation; and Classify the wear condition with a modifier based on the following image. [12] The vehicle of claim 11, wherein the modifier is a multiplier of a wear condition calculation, and wherein the control circuit is configured to select the multiplier from a plurality of modifiers corresponding to environmental conditions. [13] The vehicle of claim 11, wherein the control circuit is configured to communicate a signal to the modifier to apply cleaning fluid to the window in response to the classification of the wear condition. [14] Vehicle according to any one of claims 1-13, further comprising: a notification device configured to indicate the wear condition in response to the signal.

Images