VISUALLY AID DETECTION OF WET ROAD CONDITIONS USING TIRE TRACKS

Abstract

Method for determining a wet surface condition of a road (12), the method comprising the following steps: capturing (50) an image of a roadway by an image capture device of the carrier vehicle, wherein the image capture device is mounted on one side of the carrier vehicle and captures the image in a downward direction; identifying (51) by a processor a relevant area in the captured image behind a tire of a carrier vehicle, wherein the relevant area is representative of a tire track such as that produced by a tire on a wet roadway;Converting (52) the image of the relevant area into a polarized image, wherein the polarized image assists in identifying a vertical boundary line of the tire track in the polarized image, wherein the conversion of the image into a polarized image comprises the following steps: identifying a pole in the relevant area, wherein the pole represents a tire-road contact point; identifying a polar axis originating from the pole; and generating the polarized image as a function of radii and angles, wherein each radius in the polarized image is determined as a function of a distance between the pole and the individual image pixels, and each angle in the polarized image is determined as an angle between the polar axis and the individual image pixels by using a tire-road contact point as the pole of the polarized image;Identifying (53) a vertical edge line of the tire track that is representative of water on the road in the polarized image; determining (55) whether water is present in the relevant area based on the vertical edge line of the tire track; and generating (56) a wet road warning signal in response to the detection of water in the relevant area.

Description

BACKGROUND OF THE INVENTION One embodiment generally relates to the detection of a wet road surface by means of tire track detection. Precipitation on a road surface causes various problems for a vehicle. For example, water on a road reduces the coefficient of friction between the vehicle's tires and the road surface, leading to vehicle stability issues. Precipitation on a road is typically detected by vehicle-mounted rain sensors, which register precipitation on the road when it is already affecting driving behavior, such as by detecting wheel slip. This means the vehicle must monitor its own operating conditions (e.g., wheel slip) and compare them to operating conditions on a dry road surface to determine if precipitation is occurring. Therefore, such systems either wait for this condition to arise or send control signals to the vehicle to determine whether the condition exists (e.g.,the generation of sudden acceleration at the drive wheels, which causes wheel slippage in falling precipitation). German patent application DE 10 2013 223 367 A1 discloses a method and a device for determining road surface conditions using a vehicle camera system. German patent application US 2014 / 0 081 507 A1 describes a method for detecting road weather conditions. German patent application US 2014 / 0 307 247 A1 discloses methods and systems for recording weather conditions. German patent application US 2005 / 0 172 526 A1 describes a method for determining the surface properties of a road. One of the purposes of this disclosure is to provide a method for determining the wet surface condition of a road. BRIEF SUMMARY OF THE INVENTION This problem is solved by the subject matter of the independent claim. Advantageous further developments are specified in the dependent claims. One advantage of this embodiment is the detection of water on a road using a vision-based imaging device that detects precipitation based on the tire tracks left by a vehicle driving over a wet road surface. The technique described here does not require any control signals from the vehicle or driver to trigger a determination of whether precipitation is present. Instead, precipitation is detected in response to monitoring the tire tracks directly created by the tires on the road surface as they roll and displace water. The technique preferably captures an image that includes the tire tracks in the water on the road surface left by the vehicle tires rolling on the road surface. The technique uses a polarized image of the captured scene and applies edge filtering to identify a boundary line.The polarized image is aligned to identify a vertical edge of the tire track in the image. The filter creates a high peak along this edge in a filtered graph. If a track is present, a filtered graph shows a large peak; in comparison, the curve is relatively flat if no track is present. A classifier is trained by teaching it data with a separation threshold; subsequently, the trained classifier is implemented for real-time detection of water on the road based on active monitoring of the road surface from the vehicle. BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 is a perspective view of a vehicle, captured by a camera on a wet surface. Fig. 2 shows a block diagram of a wet road surface detection system. Fig. 3 is an exemplary perspective view of a vehicle with its surroundings. Fig. 4 shows a flowchart of a method for detecting a wet surface. Fig. 5 shows an image captured by the image acquisition device. Fig. 6 shows an exemplary polarized image. Fig. 7 shows an exemplary filter representation of a wet road surface. Fig. 8 shows an exemplary filter representation of a dry road surface. Fig. 9 shows an exemplary feature space for characterizing sampled data on wet and dry road surfaces. DETAILED DESCRIPTION Figure 1 shows a vehicle 10 traveling on a road 12. Precipitation 19 in the form of water, shown on the road 12, is frequently displaced by the vehicle wheel 14 and a tire 16 mounted on a rim 18 of the wheel 14, which rolls on the wet road surface of the road 12. It is often advantageous to know when the vehicle will be traveling on a wet road 12 so that water-related problems, such as drive slippage or impaired engine performance caused by water entering the outside air intake openings, can be neglected or at least reduced. Precipitation 19 on road 12 can lead to a reduction in traction when driving on the wet road surface. The precipitation 19 on road 12 lowers the coefficient of friction between the vehicle's tires and the road 12. This reduces the traction between the vehicle's tires and the road 12. A loss of traction can be mitigated by various techniques, including: warning the driver to reduce vehicle speed according to the ambient conditions; activating an automatic application of the vehicle's brakes to use a very low braking force to minimize precipitation buildup on the brake component surfaces; deactivating or limiting the function of cruise control when precipitation is detected; or notifying the driver to increase the distance to the vehicle ahead.It should be noted that the embodiments described herein can be used for other types of systems outside the automotive sector where the detection of wet road conditions is desired. Examples of vehicles other than automobiles that can utilize this system include, but are not limited to, rail vehicles, aircraft, sport and off-road vehicles, robotic vehicles, motorcycles, bicycles, agricultural vehicles, and construction machinery. Fig. 2 shows a block diagram of a wet road detection system. A multitude of vehicle-mounted image capture devices 20 are mounted on the vehicle to capture images all around the vehicle. The multitude of vehicle-mounted image capture devices 20 can be mounted on the front, rear, and sides of the vehicle. Fig. 3 shows an exemplary 360° surround-view camera system for capturing objects around the vehicle. The individual image capture devices work together for detection and object recognition on all sides of the vehicle. The image capture devices 20 include, among other things, a front camera 22, mounted on the front of the vehicle to capture images in front of and partially to the sides of the vehicle. A driver-side camera 24 captures images on the driver's side of the vehicle. A passenger-side camera 26 captures images on the passenger side of the vehicle. A rear-facing camera 28 captures images behind and to the sides of the vehicle. Also shown in Fig. 2, a processor 30 processes the images captured by the image acquisition devices 20. The processor 30 analyzes images and data to determine the presence of water on the road surface based on the tire marks left by the vehicle's tires rolling on the road surface. When the processor 30 detects a wheel of the vehicle 10, it identifies a relevant area and evaluates the area immediately behind the tire where tire marks are likely to be found if there is water on the road. The processor 30 can be part of an existing system, such as a traction control system or another system, or it can be a standalone processor for analyzing the data from the image acquisition devices 22. The processor 30 can be coupled with one or more output devices, such as a control unit 32, to trigger or execute a control action when water is present in the relevant area. One or more countermeasures can be taken to mitigate the effects of the water on the operation of the vehicle. The control unit 32 can be part of the vehicle subsystem or can be used to enable a vehicle subsystem to neutralize the effects of water. For example, in response to the detection that the road surface is wet, the control unit 32 can activate an electric or electro-hydraulic braking system 34, or a similar system, in which a braking strategy is prepared in case of loss of traction. In addition to preparing a braking strategy, the braking system can autonomously apply a slight braking force, without driver intervention, to remove water from the vehicle brakes as soon as the vehicle enters the water. Removing water from the wheels and brakes maintains the expected coefficient of friction between the vehicle brake actuators and the wheel braking surface when the driver manually applies the braking force. The control unit 32 can control a traction control system 36 that individually transfers power to each wheel to reduce wheel slippage when water is detected on the road surface. The control unit 32 can control a cruise control system 38, which can restrict the activation of the speed control or deactivate the speed control when water is detected on the road surface. The control unit 32 can control a driver information system 40 to issue warnings to the vehicle driver when water is detected on the road. Such a warning issued by the control unit 32 can alert the driver to the amount of water on the road surface and recommend that the driver reduce the vehicle speed to a speed appropriate to the current environmental conditions, or the control unit 32 can issue a warning to maintain a safe distance from the vehicle ahead. It should be noted that the control unit 32, as described herein, can include one or more control modules, each controlling an individual function or a combination of functions. The control unit 32 can also control the automatic opening and closing function of air deflectors 42 to prevent water from entering the vehicle engine. Under such conditions, the control unit 32 automatically activates the closing function of the air deflectors 42 when water is detected on the road in front of the vehicle and can reopen the air deflectors when it is determined that there is no more water on the road surface. The control unit 32 can also activate a wireless communication device 44 for autonomous communication of the wet road condition to other vehicles by means of a vehicle-to-vehicle or vehicle-to-infrastructure communication system. The control system can also issue warning signals to the driver of the vehicle to prevent the use of automated functions, such as adaptive cruise control, lane change assist, steering / evasive maneuver assist, automated emergency braking, etc. The advantage of the techniques described herein is that no control signals from the vehicle or driver are required to trigger the detection of water. Previous techniques, on the other hand, required a strong control signal from the vehicle—be it through braking, increased acceleration, steering, etc.—to detect the presence of surface water. Based on the response (e.g., wheel slip, yaw), such a technique determined whether the vehicle was driving on water. In contrast, the techniques described herein for detecting water on the road do not require any control signals from the driver. Figure 4 shows a flowchart for detecting a wet road surface. In step 50, an image is taken of the area next to one of the vehicle's wheels. The image is analyzed to extract information from the scene, with various features in the scene being evaluated to determine whether water is present. Figure 5 shows an image taken by the image capture device mounted on the side of the vehicle. The image can be processed to capture a downward-facing view (i.e., looking down at the road). The lens used by the image capture device can be a so-called fisheye lens, which captures a wide-angle view (e.g., 180 degrees). Additionally, image processing can be used to change the camera's pose so that the pose is downward-facing, as seen in the image.For example, if an image from each of the side cameras is used, then a specific point in the image, other than the focal point of the camera pose, can be used to generate a virtual pose to represent the scene as if the camera were pointing directly downwards to capture the wheel 14, the road surface 12, and the water 19. To change the camera pose, a virtual camera model can be used with the captured image, so that a virtual pose is used to reorient the image, creating a virtual image as if the camera were being reoriented and pointing in a different direction (for example, directly downwards). Reorienting the image to generate the virtual image involves identifying the virtual camera pose and mapping the individual virtual points on the virtual image to a corresponding point on the real image.As used herein, the term camera pose refers to a camera viewing angle (of both the real and the virtual camera) from a camera location, defined by the camera coordinates and the orientation of a camera Z-axis. As used herein, the term virtual camera refers to a simulated camera with simulated camera model parameters and a simulated imaging surface, in addition to another simulated camera. The camera modeling performed by the processor is described herein as the generation of a virtual image, which is a synthesis image of the scene produced by means of the virtual camera modeling. Referring again to Fig. 4, in step 51 a relevant area is determined in the real or virtual image. This technique locates the relevant area, which identifies a region where tire marks are expected when the tires roll through water on the road surface. The relevant area for tire marks left by the wheel extends immediately rearward from the tire when the vehicle is moving forward. As shown in Fig. 5, the relevant area labeled 49 is the relevant area in the image. Referring again to Fig. 4, in step 52 a polar coordinate transformation is applied to the image. The polar coordinate transformation converts the original relevant area image into a polarized image. The polar coordinate transformation uses a polar coordinate system, which is a two-dimensional coordinate system in which each point on a plane is defined by its distance from a reference point and its angle from a specific reference direction. The reference point is usually called the pole, represented as element 60 in Fig. 5. The reference direction is generally referred to as the polar axis. The distance from the pole is referred to here as the radius. Each pixel within the relevant area is mapped to a polarized image using its radius and angle. Figure 6 illustrates an example of a polarized image. The y-axis represents the radius (r) to each pixel from the pole. The x-axis represents the angle (θ) in degrees for a reference direction. Polar image conversion is used to identify the edge of the tire track left by the tire rotating on the water's surface. Depending on the vehicle speed and water depth, the edge of the tire track may not be exactly parallel to the vehicle's sideline. This can create a small angle between the edge of the tire track and the vehicle's sideline (generated at the contact point between the tire and the road). Since it can be difficult to capture this angle in real time, the tire-road contact point can be used as the pole / reference point.Therefore, a polar coordinate transformation can be performed from the pole / reference point, whereby the edges that deviate slightly in other directions from the vehicle's sideline are unified in a vertical direction in the polarized image. As shown in Fig. 6, the vertical stripes represent the upper edge of the tire track. Referring again to Fig. 4, step 53 performs a tire track edge detection analysis to determine whether water is present in the relevant area. Detection is performed using a filter, such as a Gabor filter. Gabor filters are a special class of bandpass filters typically used for edge detection. The 2D Gabor filter used here is essentially a Gaussian kernel function modulated by a sinusoidal plane wave. The filter can be customized by setting its parameters, such as orientation, scale, and modulation frequency. Consequently, the Gabor filter is an orientation-sensitive filter. The orientation of the Gabor filter is fixed in the vertical direction. As the filter scans the polarized image, it exhibits a strong response to image areas whose structures are oriented in the same direction.Therefore, it aids in the detection of vertical edge lines in the polarized image. The detected edges assist in the determination of the edges in the tire track. Edge detection highlights linear motion of textures within the captured image, such as those generated by the top edge of the tire track. The texture is represented by many discontinuous short vertical lines in an image where edges are present. In contrast, if the surface is either dry or snow-covered, then the texture of the dry or snow-covered surface does not exhibit linear motion patterns. It should be noted that, in addition to Gabor filtering, other filter types can be used to detect tire track edges in the image without deviating from the scope of disclosure. The filter response of a polarized image on a wet surface is shown in Fig. 7. The filter response is represented as a 3-dimensional diagram with the angle (θ) on the x-axis, the radius (r) on the y-axis, and the filter response on the z-axis. As shown in the filter response, an edge between the water trail and the splash water exhibits larger peaks than the filter response of a dry surface, as shown in Fig. 8. As can be seen from the diagram in Fig. 8, the filter response on the dry surface is essentially flat (e.g., around zero with only small peaks). In contrast, the filter response shown in Fig. 7 includes large peaks 62 extending along a top edge (e.g., a ridge).As shown, several peaks are present, representing the edges between a tire track where water is retained near the track in one direction and outside the tire track where water is flung into the air in various directions. It should be noted that although a Gabor filter is used here, other filter types can be used, including, but not limited to, Sobel or LM filters. Furthermore, it should be noted that the area where the tire tracks are present on the road surface appears darker in the polarized image than the areas where water is flung into the air. Referring again to Fig. 4, step 54 employs a feature analysis to analyze the filter response of the polarized image, aiming to extract distinctive features that can effectively differentiate between various road surfaces. As shown in Fig. 7, when a water trail is present, a large peak value can be detected at each corresponding radius plane around the water trail edge, which is associated with the respective radius plane. The mean of the peak values ​​from all radius planes should be much larger compared to dry surfaces or surfaces covered with snow. Additionally, the standard deviation of the filter response values ​​across all image pixels on a water-covered surface should be greater than the filter response values ​​across all pixels on the dry surface.It is understood that a classifier is first trained to use feature extraction and feature calculation to build a pattern database. Feature extraction involves extracting features from various images to generate this database. A scan is performed at each radius (e.g., 0-20 degrees), and an average of the peak response values ​​is calculated. Once the pattern database is trained, the pattern database or comparator (e.g., the separation threshold) is implemented online with the processor in the vehicle to calculate a feature value from a real-time image. The calculated feature value is compared in feature space to the separation threshold to detect the presence of water in the captured image. An example of a feature value calculation for an average of the peak filter response values ​​is shown below. An example of a feature value calculation of the variance of filter response values ​​across all image pixels of the image is presented as follows: where N is the total number of radius planes, M is the total number of extent planes, Ipij is the filter response value of a pixel on radius plane i and extent plane j, and µROI is a mean of the filter response values ​​across all image pixels of the polarized image. It is understood that the statistical analysis is only one example of feature analysis and that other techniques may be applied without deviating from the scope of disclosure. Figure 9 shows an exemplary feature space for characterizing sampled data on wet and dry road surfaces. The x-axis represents a first feature, such as the mean of the filter response peak values, and the y-axis represents the variance of the filter response across all pixels of the image. The symbol "x" indicates the presence of water, while the symbol "o" represents an ideal dry surface. Each of these features is represented in the feature space. In a trained classifier, a cutoff threshold is established in the feature space to distinguish between water represented by the edges of the tire track and the ideal dry surface. Referring again to Fig. 4, in step 55, a determination is made as to whether a water surface property is captured by the feature analysis. If it is determined that the water surface feature is captured, then the routine continues with step 56; otherwise, the routine continues with step 57. In step 56, in response to a determination that a water surface feature is detected, a wet surface indicator is set to show water in the relevant area. A signal is transmitted to a control unit that can activate various vehicle functions, as described above, including but not limited to: brake control, traction control, steering control, cruise control, driver warning, air deflector control, and vehicle-to-vehicle communication. If step 55 establishes that no water is present in the relevant area, the routine continues with step 57, in which additional techniques may be used to check whether or not water is present.

Claims

Method for determining a wet surface condition of a road (12), the method comprising the following steps: capturing (50) an image of a roadway by an image capture device of the carrier vehicle, wherein the image capture device is mounted on one side of the carrier vehicle and captures the image in a downward direction; identifying (51) by a processor a relevant area in the captured image behind a tire of a carrier vehicle, wherein the relevant area is representative of a tire track such as that produced by a tire on a wet roadway;Converting (52) the image of the relevant area into a polarized image, wherein the polarized image assists in identifying a vertical boundary line of the tire track in the polarized image, wherein the conversion of the image into a polarized image comprises the following steps: identifying a pole in the relevant area, wherein the pole represents a tire-road contact point; identifying a polar axis originating from the pole; and generating the polarized image as a function of radii and angles, wherein each radius in the polarized image is determined as a function of a distance between the pole and the individual image pixels, and each angle in the polarized image is determined as an angle between the polar axis and the individual image pixels by using a tire-road contact point as the pole of the polarized image;Identifying (53) a vertical edge line of the tire track that is representative of water on the road in the polarized image; determining (55) whether water is present in the relevant area based on the vertical edge line of the tire track; and generating (56) a wet road warning signal in response to the detection of water in the relevant area. The method of claim 1, further comprising the step of applying a filter to the polarized image to identify the vertical edge lines of the tire track in the image. The method of claim 2, wherein the application of a filter includes the application of an edge detection filter. Method according to claim 3, wherein the edge detection filter generates the response of the filter, and the response of the filter generates data indicating the vertical line separation between the tire track and the water thrown into the air. The method of claim 1, wherein the taking of an image in a downward direction includes a real, downward-facing image of the road surface. The method of claim 1, wherein capturing an image in a downward direction includes generating a virtual image in a downward direction based on the actual image, wherein a virtual image is generated by reorienting the image so that the virtual image is generated as if a camera were pointing downwards. The method of claim 6, wherein the realignment of the image to generate the virtual image comprises the following steps: identifying the virtual camera pose; mapping the individual virtual points on the virtual image to a corresponding point on the real image.

Images