Method for determining the snow-covered surface condition of a roadway

Abstract

Method for determining a snow-covered surface condition of a roadway (12), comprising the following steps: emitting a light beam by a light source (21) onto a surface of the roadway (12); capturing an image of the road surface by an image capturing device (20), wherein the image capturing device (20) is mounted on a vehicle (10) and captures an image in a downward direction, and wherein the captured image captures the light beam emitted onto the road surface; identifying an examination area (49) in the captured image which includes the light beam emitted onto the roadway (12); analyzing by a processor (30) a hidden light scatter in the examination area (49), comprising converting image components into a Lab space to separate color components from brightness components of the light beam and applying feature extraction to at least one color component in the Lab space, wherein the application of feature extraction comprises applying a filtering technique to detect edges; determining whether snow (14) is present in the examination area (49) on the roadway (12), comprising a binary conversion of the Lab image and a mean-variance analysis of the binary image; and generating a snow-covered road surface signal in response to detecting snow (14) on the roadway (12).

Description

TECHNICAL FIELDThis invention relates to a method for detecting a snow-covered surface condition of a roadway.BACKGROUND OF THE INVENTIONAn embodiment relates generally to detecting a snow-covered roadway surface using hidden light scattering.Precipitation on a roadway causes several different performance problems for vehicles or people travelling on the roadway. For example, snow on a roadway reduces the coefficient of friction between the tires of the vehicle and the roadway surface, leading to vehicle stability issues. The detection of snow on a roadway is typically performed by a host vehicle capable of measuring the snow on the roadway using a sensing action that affects success when the snow is already affecting operation of the vehicle, such as wheel slip detection. Therefore, the vehicle must monitor its own operating conditions (such as wheel slip) and compare it with the dry roadway operating conditions to determine whether snow is present. As a result, said systems may wait for such a condition to occur or subject the vehicle to an excitation force to determine whether the condition is currently true (e.g., by creating a sudden acceleration of the drive wheels, thereby causing wheel slip in the event of a precipitation occurring).Another advantage is that the method described herein can attenuate the influence of ambient light conditions, as an active light source is used, at the same time erroneous detection of white-coloured dry soil, such as a salt-damaged dry roadway in winter, can be ruled out.DE 10 2010 025 719 A1 discloses a method and a device for outputting a signal when a snow-covered roadway is detected. To recognize a snow-covered roadway, a light emitted by a light source and reflected by the roadway is analyzed. US 2013 / 0 015 946 A1 discloses a method for authenticating a user of a computer device using facial data. US 2009 / 0 290 807 A1 discloses an image processing method for improving a visual appearance of images containing snow. An image processing device which generates color image data and monochrome image data by a scanning process is known from US 2006 / 0 132 855 A1. WO 2015 / 072 217 A1 discloses a method for determining a position of a vehicle, wherein filter methods are used for detecting edges in image data. DE 10 2004 018 088 A1 discloses a method for determining a roadway state based on a comparison of filtered image data of the roadway with reference data. DE 197 30 414 A1 discloses a method for predictive assessment of a roadway, wherein a laser light pattern is evaluated in the triangulation method.The object of the invention is to specify a method which enables an improved determination of a snow-covered surface state of a roadway.This object is achieved by a method having the features of claim 1 for determining a snow-covered surface state of a roadway.SUMMARYAn advantage of an embodiment is the detection of snow on a roadway using an optics-based image capture device with a concentrated light source, such as a laser LED, that identifies snow based on obscured light scattering, wherein a wide turbidity of the scattered light across a snow-covered roadway is generated as opposed to a relatively small spot of light on non-snow-covered roadway surfaces, such as dry and wet roadway surfaces. The method described herein does not require excitation forces by the vehicle or driver to determine whether precipitation is present. Rather, snow is detected in response to the analysis of light scattered on the snow surface of the roadway. The method captures an image that includes the light beam scattered onto the surface. An RGB image is converted into Lab space. Either a Laplace-Gaussian (LOG) filtering method or binary conversion is applied to the lab space image to detect the wide turbidity pattern of the stray light on a snow-covered surface. In the event of a wide haze, the LOG filter response at the location of a wide haze range produces a large increase as opposed to a relatively flat filter response on non-snowy roadway surfaces, such as dry and wet roadway surfaces. Alternatively, an average analysis of variance of the binary image may also identify snow by having the average ratio and the variation ratio be a value greater than 1 between the stray light area and the total area as opposed to the ratio values of nearly 1 on non-snowy roadway surfaces, such as dry and wet roads. A trained classifier is formed and implemented online within the vehicle. A similar process is used within the vehicle to detect and process the light beam. It is determined whether snow is present on the roadway surface depending on the online classifier implemented within the vehicle to detect the snow in real-time on the roadway based on the active monitoring of the roadway surface.One embodiment relates to a method for determining a snow-covered surface condition of a roadway. A light beam is emitted onto a surface of the roadway with the aid of a light source. An image of a road surface is captured by an image capture device. The image capturing apparatus is mounted on the vehicle and captures the image in a downward direction. The captured image captures the light beam emitted onto the roadway surface. An examination region is identified in the captured image, wherein the examination region comprises the light beam emitted onto the roadway. Light scattering generated in the examination region is analyzed by a processor. The analyzing includes converting image components into a Lab space to separate color components from brightness components of the light beam and applying feature extraction to at least one color component in the Lab space. The use of feature extraction involves the use of a filtering technique for detecting edges. A determination is made as to whether snow is present in the examination area on the roadway. The determination includes binary conversion of the Lab image and mean variance analysis of the binary image. In response to detecting snow on the roadway, a snow-covered roadway surface signal is generated.BRIEF DESCRIPTION OF THE DRAWINGSFIG. 1 is a perspective view of a vehicle traveling on a snow-covered surface. FIG. 2 shows a block diagram of a detection system for snow-covered road surfaces. FIG. 3 is an exemplary perspective view of a vehicle with all-round view. FIG. 4 is an exemplary perspective view of a vehicle having an image capture device and a light source. FIG. 5 shows a flow diagram of a method for detecting a snow-covered surface. FIG. 6 shows an example image captured by the image capture device. FIG. 7 a illustrates an RGB image of the dry surface inspection area. FIG. 7 b illustrates an RGB image of the snow-covered surface examination region. FIG. 8a is an exemplary dry surface filtration reaction. FIG. 8 bis an exemplary filter response for a snow-covered surface. FIG. 9 illustrates an example feature space that identifies snow-covered and non-snow-covered surfaces for a construction of the classifier. FIG. 10 shows a flow chart of a second method for detecting the snow-covered road surface. FIG. 11 a illustrates an exemplary RGB image of the dry surface examination region. FIG. 11 b illustrates an exemplary RGB image of the snow-covered surface examination region. FIG. 12 a illustrates exemplary responses for a dry road surface in a Lab color space. FIG. 12 b illustrates exemplary responses for a snow-covered roadway surface in a Lab color space. FIG. 13 a illustrates an example binarized image for a dry surface. FIG. 13 b illustrates an example binarized image for a snow-covered surface. FIG. 14 shows an exemplary pattern similar to a square in which 4 spots form a square pattern. FIG. 15 shows an exemplary line pattern in which the line point is elongated and resemble an oval pattern. FIG. 16 shows a line pattern for all light spots which are formed linearly offset from one another. FIG. 17 illustrates an example real-time tracking sequence for a non-snowed surface. FIG. 18 illustrates an example real-time tracking sequence for a snow-covered surface.DETAILED DESCRIPTIONFIG. 1 shows a vehicle 10 traveling on a roadway 12, such as a road. It should be understood that the term roadway may include any surface used by motor vehicles, bicycles, or pedestrians. For purposes of illustration, the term roadway is used in the sense of a road; however, it should be understood that the term roadway is not limited to a road being traveled by vehicles, and the system and method described herein may be implemented on both moving and stationary objects. Snow 14 located on roadway 12 causes a slippery condition when the tires rotate over the snow-covered surface of roadway 12. It is often advantageous to know when the vehicle is on a snow-covered roadway 12 so that problems caused due to snow, such as loss of traction, can be prevented.The snow 14 on the roadway 12 reduces the coefficient of friction between the vehicle tires and the roadway 12 As a result, traction between the vehicle tires and the roadway 12 is reduced. loss of traction can be mitigated by various mitigation methods, including warning the driver to reduce vehicle speed to a level appropriate for the environmental conditions; by actuating automatic application of the vehicle brake using a very low braking force to minimize precipitation formed on the braking surfaces of the braking components; by disabling cruise control while detecting precipitation; or by notifying the driver to maintain a greater braking distance (distance) to a preceding vehicle. It should be appreciated that the embodiments described herein may be applied to other types of systems in addition to motor vehicles in which the detection of a snow-covered roadway surface is appropriate. Vehicles that may utilize this system include, among all motor vehicles, rail vehicles, aircraft, off-road sports vehicles, robotic vehicles, motor cycles, bicycles, agricultural equipment, and construction machinery, among others.FIG. 2 shows a block diagram of a detection system for snow-covered road surfaces. A plurality of vehicle-based image capture devices 20 and light sources include, among other things, LED lasers mounted on the vehicle for capturing images around the vehicle that help to timely recognize the snow. Obscured light scattering can occur in various light sources (e.g., flashlights, laser LEDs). In order to distinguish the snow from other road conditions, a concentrated light source, such as a laser LED, is preferably used, which provides hidden light scattering on the snow surface, but also a concentrated light spot on othersThis is because it is possible to produce types of roads, such as dry or wet roads. The plurality of vehicle-based image capturing devices 20 and light sources 21 may be mounted on the front, rear, and sides of the vehicle. FIG. 3 shows an example 360 degree surround view for detecting objects in the vehicle environment. All image-based detection devices are used in combination with one another for detecting and recognizing objects on all sides of the vehicle. The image-based capture devices 20 include, among other things, a front camera 22 mounted to the front of the vehicle that captures images forward and partially on the sides of the vehicle. A driver side camera 24 captures images on the driver side of the vehicle. A passenger side camera 26 captures images on the passenger side of the vehicle. A rear-facing camera 28 captures images rearward and to the side of the vehicle.FIG. 4 shows a perspective view of the vehicle 10 traveling on the snow-covered road surface. The light source 21 emits a light beam onto the roadway surface 12, and the light emitted onto the roadway surface is captured by a respective image capture device of the vehicle. The light beam detected by the image detection device is evaluated in order to determine whether snow 14 is present on the road surface. The method employed herein uses hidden scattered light analysis to determine whether snow is present. Hidden light scattering is a method in which the light penetrates a surface of a transparent object (e.g., snow) and is scattered by interaction with the material.The light leaves the snow at various points. The light will generally penetrate the surface of the snow and be reflected a plurality of times at irregular angles within the snow before returning from the snow, at an angle not corresponding to the particular angle that the light would have if it were reflected directly from the surface. Snow is composed of relatively loosely packed ice crystals. The loosely packed ice crystals sometimes make up just 5% of the total volume of space in a snow area. The ice crystals in the snow produce hidden light scattering. The light generated on a dry surface (e.g., a laser pointer bright on a dry road) exhibits substantially no light scattering and exhibits a relatively small spot of light that is substantially uniform compared to a snow-covered surface. The snow-covered surface produces a wide turbidity of the stray light due to the ice crystals diffusing the light through the snow. In evaluating the light on the surface, image conversion into Lab color space is used. This involves the conversion of an RGB image (i.e., a red-green-blue image) to a Lab color space to better detect the broad haze pattern of the color changes with the light while the light signal scatters over the entire ground of the snow. The Lab space contains several components having a dimension L for brightness and "A" and "B" for the counter-colored dimensions. The Lab color space contains all perceivable colors, meaning that its extent exceeds that of the RGB color models. An important attribute of a Lab model is device independence, in which colors are defined regardless of their type of generation. Lab color space is used when graphics must be converted for printing RGB. While the space itself is a three-dimensional real number space, which in reality can contain an infinite number of possible color representations, the space for device-independent digital representation is generally mapped onto a three-dimensional integer space. In the embodiments described herein, the color channels are represented as follows: a brightness component is represented along the L axis, a green to red component is represented along the A axis, and a blue-yellow component is represented along the B axis.Referring again to FIG. 2, a processor 30 processes the images captured by the image capture devices 20. The processor 30 analyzes images and data to determine whether snow is present on the roadway surface proximate the vehicle. Once the image is acquired, the processor 30 identifies an examination area to determine whether snow is present within the examination area based on light scattering. The examination region can contain a square shape, a circular shape or another shape depending on the light source and the problem formulation. The processor 30 can be part of an existing system, such as a traction control system or another system or a self-contained processor, which is provided for evaluating the data 20 recorded by the image recording devices.The processor 30 may be connected to one or more output devices, such as a controller 32, to initiate or actuate a control action if snow is found in the examination area. One or more countermeasures may be initiated to mitigate the impact that the snow may have on the operation of the vehicle.The controller 32 may be part of the vehicle subsystem or may be used to activate a vehicle subsystem to counteract the effects of the snow. For example, in response to a determination that the roadway is snow covered, the controller 32 may activate an electric or electro-hydraulic brake system 34 or the like when a braking strategy is provided in the event that a loss of traction occurs. In addition to preparing a braking strategy, the brake system may independently apply, without driver awareness, a light braking force to remove snow from the vehicle brakes once the vehicle encounters a snow-covered roadway. Removing the snow from the wheels and brakes by melted snow maintains an expected coefficient of friction between the vehicle brake actuators and the braking surface of the wheels when the brake is manually operated by the driver.The controller 32 may control a traction control system 36 that distributes power individually to each respective wheel to reduce wheel slip through the respective wheels once snow is detected on the roadway surface.The control unit 32 can control a speed control system 38 which can deactivate the speed control or restrict the activation of the speed control as soon as snow is detected on the road surface.The control unit 32 may control a driver information system 40 to provide alerts to the driver of the vehicle to alert the driver of the snow that was detected on the roadway. Such a warning initiated by the controller 32 may indicate to and recommend to the driver the imminent snow on the road surface to reduce the vehicle speed to a value appropriate for the current environmental conditions, or the controller 32 may initiate a warning to maintain a safe distance to the preceding vehicle. It should be appreciated that the controller 32, as described herein, may include one or more controllers that may control a single function or a combination of functions.The controller 32 may further manipulate the automatic opening and closing of the baffles 42 to prevent snow from entering the engine of the vehicle. Under the above conditions, the control unit 32 automatically manipulates the closing of the baffles 42 when it is determined that snow is present on the road surface in front of the vehicle, and may open the baffles again when it is determined that snow is no longer present on the road surface.Further, using a communication system that communicates vehicle-to-vehicle or vehicle-to-infrastructure, controller 32 may control actuation of a combination wireless device 44 to independently communicate the snow-covered roadway condition to other vehicles.The control unit can furthermore direct signal warnings to a driver in the case of snow-covered road surfaces, which warnings are to use automated functions, such as, for example, adaptive skill control, lane following, lane changing, auxiliary steering / avoidance maneuvers, and automatic emergency braking.The advantage of the methods described herein is that no excitation forces are required by the vehicle or driver to initiate a determination as to whether snow is present. That is, the prior art methods of detecting snow on the road surface require a considerable energizing force from the vehicle, either through a braking maneuver, an increased acceleration, or a steering maneuver. Based on the response (e.g., wheel slip, yaw), such a method determines whether the vehicle is currently traveling on snow. In contrast, the methods described herein do not require driver excitation forces to detect snow on the roadway. In addition, by the method described herein, since an active light source is used, the influence of the ambient light conditions can be attenuated and erroneous recognition of a white-colored dry soil, such as a salt-damaged dry roadway in winter, can be simultaneously excluded. A common disadvantage for camera-based solutions is the high sensitivity to ambient light. There are many camera-based approaches to detecting snow, but these rely solely on image evaluations of the camera and can react very sensitively to ambient light changes without active illumination and also lead to incorrect decisions in white-coloured dry roads. The method of hidden light scattering with a laser light source described herein can exclude the probability of an erroneous identification of a dry surface with a relatively white color as a snow-covered surface.FIG. 5 shows a flow chart of a first method for detecting the snow-covered road surface. Steps 50-55 represent a training phase for creating a database for road surface patterns, while steps 55-64 represent an online classifier that is used in the vehicle to determine whether snow is present on the road surface.In step 50, an image of an area adjacent a respective side of the vehicle is captured. The image is analyzed to obtain scene information whereby light scattering properties in the scene can be evaluated to determine if snow is present in the image. FIG. 6 shows an image captured by the image capturing apparatus mounted on the side of the vehicle. The image may be processed to capture a downward view (i.e., looking at the roadway). A lens used by a respective image capture device may use a fisheye lens in which a wide image portion (e.g., 180 degrees) is captured. In addition, the image processing can be used to change the pose of the camera such that the pose is directed downward as seen in the image. For example, if an image from one of the respective side cameras is used, a corresponding location in the image, although not the focal point of the camera pose, may be used to generate a virtual pose that the scene is represented as if the camera were directed directly downward to capture the roadway 12 and the snow 14. To change the pose, a virtual camera model may be used with the captured image, such that a virtual pose is used to re-align the image and a virtual image is generated as if the camera were re-aligned (e.g., looking down) in a different direction. The re-alignment of the image to generate the virtual image includes identifying the virtual pose and assigning all virtual points on the virtual image to a corresponding point on the real image. The term pose, as used herein, refers to a camera viewing angle of a camera location (in both real and virtual cameras) defined by the camera coordinates and the orientation of a camera Z-axis. The term virtual camera as used herein refers to, in addition to a simulated camera pose, a simulated camera having simulated camera model parameters and simulated imaging surface. Here, camera modeling performed by the processor to obtain a virtual image that is a synthesized image of the scene using the virtual camera modeling will be described.Referring back to FIG. 5, in step 51, an examination region is identified from the real image or the virtual image. This method locates the examination region identifying each region in which the laser beam is emitted on the road surface relative to the vehicle, so that when snow is detected in this region, it can be considered that the vehicle is traveling on snow. As shown in FIG. 6, the exemplary area indicated by 49 represents an examination area in the image, while the element 51 represents the light transmitted from the light source on the snow 14 of the roadway 12.Referring back to FIG. 5, in step 52, image conversion is performed using a Lab color model. The image converted from RGB to Lab color space easily acquires relevant color information of the light beam in the image for evaluation by isolating the influence of noise signals of the environment such as illumination and scattering salted surfaces, thereby enhancing recognition reliability.In step 53, feature extraction is applied to the converted image (e.g., the Lab image) using a respective filtering technique. Preferably, the feature extraction employs a Laplace Gaussian (LOG) filter technique. The LOG operator calculates a second spatial derivative of the image. The filter highlights areas of rapid intensity variation and can therefore be used for edge detection. Various filters of different scales may be used to filter "A" and "B" channel images of the respective frames. For example, if 6 LOG filters are used on each image, a maximum response of the examination region is considered a feature, resulting in 12 features in a feature set. It should be appreciated that the method described herein may filter filtering other than Laplace Gaussian (LOG) filtering.FIG. 7 a illustrates the RGB image of the dry surface inspection area. As shown in FIG. 7 a, the respective light beam is detected as a substantially defined point with a substantially uniform color. In contrast, FIG. 7 b illustrates the RGB image of the snow-covered surface examination region. The respective light beam emitted onto the snow-covered surface is scattered. The respective light beam has color transitions that change from a center of the light to a radially outward scattering of the light beam in the snow.FIGS. 8 aand 8 b illustrate exemplary filter responses after the LOG filter is applied to the lab images in the examination region in the "A" channel. That is, respective colors may be focused such that color changes in light as the light signal scatters over the entire surface of the snow may be detected, thereby identifying the color component changes that are indicative of light scattering in the snow. In this exemplary case, green to red color images are the focus of evaluation in the Lab color space. The LOG filter is applied to lab images with pixel values from the "A" channel. FIGS. 8 aand 8 b illustrate three-dimensional graphs, wherein the filter response (R) is illustrated along the z-axis, the image height is illustrated along the x-axis, and the image width is illustrated along the y-axis. As shown in Figure 8a, the dry surface filter response is substantially zero. In contrast, FIG. 8 bshows snow that is recognized in the filter image due to the intensity change by a larger light image shape. As a result, the filter response for an image of the snow-covered surface is substantially greater than zero, as shown in FIG. 8 b.Referring again to FIG. 5, in step 54, a classifier is created based on the results in step 53. The respective features representing the sample distributions are recorded in the feature space. FIG. 9 illustrates an example feature space that identifies snow-covered and non-snow-covered surfaces used to create the classifier. An x-axis represents a first feature, while the y-axis represents a second feature. The first feature is an exemplary maximum filter response after the LOG filter with a scale σ equal to 4 is applied to a Lab image with pixel values from the "A" channel. The second feature is an exemplary maximum filter response after the LOG filter with a scale σ equal to 12 is applied to a Lab image with pixel values from the "A" channel. The symbols "◯" represent the snow present in the image, while symbol "×" represents a dry surface. The trained classifier is used to identify a separation plane that can be used to distinguish between snow and the ideal dry surface. If there is snow, a large peak (i.e., substantially greater than zero) can be detected on the image.In step 55, a classifier is trained using the feature set extracted in step 53 to create a database of road surface patterns that includes prestored data associated with the feature set and representing typical patterns of the various road surface conditions. Once the database of road surface patterns is created, it can be implemented online during production in a vehicle.Steps 56-59 relate to the real-time processing of the captured images during the evaluation of the images of the vehicle. The respective steps are the same as steps 50-53 as described above, except that the processing refers to the evaluation of the real-time images, as opposed to training the classifier.In step 59, a set of features extracted from step 53 is calculated depending on the LOG filtering based on the respective targeted features, the lab space channel, and the scales applied.In step 60, the set of features calculated in step 59 is communicated to an online classifier to determine whether snow is present in the examination region by comparing the calculated feature values to the prestored data associated with the same set of features in the road surface pattern database.If the determination of snow on the road surface is confirmed in step 61, the routine proceeds to step 62; otherwise, the routine proceeds to step 63.In step 62, in response to a determination that snow is present on the road surface, a snow-covered surface flag indicating that snow is present in the examination area is set. A signal is transmitted to a control unit by which, as described above, various vehicle operations such as brake control, traction control, cruise control, steering control, driver warning, air deflector control and vehicle-to-vehicle communication, among others, may be actuated.If the determination in step 61 confirms that no snow is present in the examination area, the routine proceeds to step 63 where other additional methods may be used to check whether or not there is snow.FIG. 10 illustrates a flow chart of a second method for detecting the snow-covered roadway surface. Steps 50-55 represent a training phase to create a database of road surface patterns, steps 55-63 representing the online detection of snow on the road surface in a vehicle.In step 70, an image of an area adjacent a respective side of the vehicle is captured. The image is evaluated to obtain scene information in which the detected light beam in the scene can be analyzed to determine whether snow is present in the image. Image acquisition and processing may be used herein as described above.In step 71, noise removal and / or distortion techniques are applied to the input image.In step 72, an examination region is determined from the real or the virtual image. This method locates the examination region identifying each region in which the laser beam is emitted on the road surface relative to the vehicle, so that when snow is detected in this region, it can be considered that the vehicle is traveling on snow.In step 73, the image conversion is performed using the Lab color model. The image converted from RGB to Lab color space acquires relevant color information of the light image for evaluation by isolating the influence of noise signals of the environment such as illumination and surfaces provided with scattering salt, thereby enhancing recognition reliability.In step 74, the binary conversion is applied to the Lab-converted image. Binary conversion results in a pattern structure of snow analysis in the image in the presence of snow, while absence of snow in the LAB converted image does not have any pattern on a non-snow covered surface. An average value and a variance ratio between the stray light area and an overall inspection area on the binary converted image includes values that are more than 1 in contrast to ratio values of about 1 on non-snowy road surfaces.In step 75, feature extraction is applied to the converted binary image. Feature extraction involves the use of a corresponding statistical analysis to evaluate and identify features associated with stray light on the ground surface. FIG. 11 a illustrates the RGB image of the dry surface inspection area. As shown in FIG. 11 a, the respective light beam is detected as a substantially defined point with a substantially uniform color. In contrast, FIG. 11 b illustrates the RGB image of the snow-covered surface examination region. The respective light beam emitted onto the snow-covered surface is scattered. The respective light beam is represented as a color transition which changes from the center of the light beam to the radially outer scattering of the light beam in the snow. Figures 12a and 12b illustrate responses to dry and snow-covered roadway surfaces, respectively, after the RBG images are converted to a Lab color space.Figures 13a and 13b illustrate binary images converted from Lab images. FIG. 13 ashows an exemplary dry surface, while FIG. 13 bshows an exemplary snow-covered surface. As can be seen from the binary images, a set of features can be easily extracted based on a mean analysis of variance at the examination region and the total area. An intensity average can be determined between the light region and the total surface. An average value greater than 1 indicates snow, while an average value of about 1 indicates a dry surface.In step 76, a classifier is created based on the results in step 75. The respective features representing the sample distributions are recorded in the feature space. An example of a classifier may be, but is not limited to, a support vector machine; however, other methods may be used. The classifier is trained using feature extraction to create a database of road surface patterns in step 77. Feature extraction involves extracting from various images to generate a pattern database. Once the database of road surface patterns is created, the pattern database may be implemented online during production in a vehicle.Steps 78- 82 relate to real-time processing of captured images in the evaluation of images in the vehicle. The respective steps are the same as steps 70-74 as described above, except that processing refers to the evaluation of the real-time images as opposed to training the classifier.In step 83, feature calculation is performed in real time, with known features being calculated based on new image inputs.In step 84, the extracted features based on the binary converted image are communicated to an online classifier to analyze the presence of snow in the examination region.In step 85, it is determined whether snow on the road surface has been detected. If it is determined that there is snow on the road surface, the routine proceeds to step 86; otherwise, the routine proceeds to step 87.In step 86, in response to a determination that snow is present on the road surface, a snow-covered surface flag indicating that snow is present in the examination area is set. A signal is transmitted to a control unit by which, as described above, various vehicle operations such as brake control, traction control, cruise control, steering control, driver warning, air deflector control and vehicle-to-vehicle communication, among others, may be actuated.If it has been previously determined in step 85 that no snow is present in the examination zone, the routine proceeds to step 87 in which other additional methods may be used to check whether or not there is snow.FIGS. 14-16 show improved structured light patterns emitted from the light source. The light source may form a desired patterned light pattern instead of a single light spot to detect a larger surface area and improve sensing reliability and robustness. Fig. 14 shows a pattern resembling a square where four light spots form a square pattern. Fig. 15 shows a line pattern in which the line patch is elongated and resembled an oval pattern. FIG. 16 shows a line pattern for each light spot formed linearly offset from each other.Moreover, an improved evaluation can be performed with an image tracking analysis in which images are tracked continuously in real time. A determination of the status state of the roadway surface is based on the evaluation of an image sequence within a moving time window frame instead of a single image at a single point in time. The movable window may be of different sizes and reset at different times. The movable window reduces the noise and provides higher surface recognition reliability. FIG. 17 illustrates an exemplary sequence illustrating a non-snow-covered surface, while FIG. 18 illustrates an image sequence on a snow-covered surface. In each of the figures, a respective number of time cases may be sampled to determine whether snow is present on the road surface.

Claims

A method for detecting a snow-covered surface condition of a roadway (12), comprising the steps of: emitting a beam of light by a light source (21) onto a surface of the roadway (12); acquiring an image of the roadway surface by an image acquisition device (20), wherein the image acquisition device (20) is mounted on a vehicle (10) and acquires an image in a downward direction, and wherein the acquired image acquires the beam of light emitted onto the roadway surface; identifying an examination region (49) in the acquired image comprising the beam of light emitted onto the roadway (12); analyzing, by a processor (30), hidden light scattering in the examination region (49) comprising converting image components into a lab space to separate color components from brightness components of the light beam and applying feature extraction to at least one color component in the lab space, wherein applying feature extraction comprises applying a filtering technique to detect edges; determining whether snow (14) is present in the examination region (49) on the roadway (12) comprising binary converting the lab image and mean variance analysis of the binary image; and generating a snow-covered roadway surface signal in response to detecting snow (14) on the roadway (12).The method of claim 1, wherein the filtering technique comprises Laplace-Gaussian filtering, and wherein a respective number of Laplace-Gaussian filters comprising different scales are applied to filter color components of the scattered light beam in each image and generate a filter response.The method of claim 2, wherein a wide turbidity pattern of stray light representing the snow-covered surface is detected when the filter response indicates peaks that are relatively larger compared to filter response data from a non-snow-covered roadway (12).The method of claim 1, wherein the emitted light beam on the roadway surface comprises square pattern beams emitted onto the roadway surface.The method of claim 1, wherein the emitted light beam is tracked continuously in real time at the roadway surface, wherein an image sequence is tracked within a moving temporal frame to identify the light beam in each respective frame, wherein a moving window is used to compare a plurality of emitted light beams to identify snow (14) on the roadway (12).

Images