Apparatus and method for detecting road surface conditions
The apparatus and method enhance road condition detection by analyzing spectral absorption and polarization of light using multiple sensor recordings, offering reliable and efficient classification of road surfaces, improving vehicle safety through proactive detection of hazardous conditions.
Patent Information
- Application Number
- JP2022143927
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2022-02-14
- Filing Date
- 2022-09-09
- Publication Date
- 2024-11-20
- Estimated Expiration
- 2042-09-09
AI Technical Summary
Current road condition detection systems using RGB cameras struggle to reliably distinguish between wet and dirty road areas due to similar dark appearances, and are affected by environmental variables, leading to unreliable localization and classification results.
An apparatus and method utilizing an input interface, image segmenter, parameter calculator, and classifier to analyze road surface conditions by evaluating spectral absorption and polarization of light, employing multiple sensor recordings with different optical bandwidths and polarization filters to classify road surfaces on a window-by-window basis, reducing computational effort and enhancing resilience to image element errors.
Provides reliable and robust detection of road surface conditions, enabling proactive identification of wet areas, puddles, and ice or snow layers, thereby enhancing vehicle safety by reducing the risk of traction loss.
Smart Images

Figure 0007587556000013 
Figure 0007587556000014 
Figure 0007587556000015
Abstract
Description
[Technical field]
[0001] The present application relates to an apparatus and method for detecting road surface conditions. [Background technology]
[0002] Detecting road conditions is essential for safe driving, and modern vehicles use sensors, including optical sensors, to estimate the general risk of road conditions.
[0003] For example, the situation ahead of a vehicle is recorded by a camera. For example, in a red, green, and blue (RGB) image, it is difficult to distinguish between wet and dirty areas of the road surface because both are just dark. Although water is transparent, it is easily visible in an RGB image because various effects change the path of light rays. Current object detection methods use RGB data to detect objects such as pedestrians. However, objects with simple shapes and complex optical behaviors such as reflection and transparency, such as puddles, are difficult to detect. Environmental variables such as sunlight, background, and subsurface can drastically change the appearance of weather-induced accumulations or deposits, further complicating detection.
[0004] RGB cameras or RGB image sensors are not able to determine various effects such as absorption and / or polarization, therefore the results of localization and classification methods are highly dependent on each case and are less reliable. Summary of the Invention [Problem to be solved by the invention]
[0005] It is an object of the present invention to provide a reliable and robust concept for detecting road surface conditions.
[0006] This object is solved by an apparatus according to claim 1, a method according to claim 19 or a computer program according to claim 20. [Means for solving the problem]
[0007] An embodiment of the present invention provides an apparatus for detecting a road surface condition, the apparatus comprising an input interface, an image segmenter, a parameter calculator, and a classifier.
[0008] The input interface is configured to obtain recorded information of the road surface, such as a camera recording.
[0009] The parameter calculator is configured to calculate at least two parameters for each window by using at least two different pieces of information for each image element within the window.
[0010] The classifier is configured to classify the window based on a tuple of at least two parameters of the window, and the classifier is configured to detect a state of the window as an input value based on the tuple of at least two parameters of the window, and to output the state of the window as an output value.
[0011] In other words, the device of the invention uses a window-by-window or elementary classification that is based on the extraction of the optical properties of the road surface from the sensor recording, in particular by evaluation of the spectral absorption of the road surface and / or the polarization of the reflected light.
[0012] Basic or window-wise classification makes the system more resilient to image element errors since in the case of window-wise classification, image element errors only have a local effect.
[0013] Window-wise classification allows for window-wise evaluation, and the quality of the classification can be determined through a comparison of the windows and the classification results.
[0014] In addition, compared to current image sensing for detecting road surface conditions, the image sensor of the present invention uses reduced computational effort.
[0015] In this way, the image sensor provides reliable and robust detection of road surface conditions while at the same time providing a simple window-by-window assessment.
[0016] In an embodiment, the input interface is configured to obtain output information from the sensor or to obtain several sensor records and to assemble record information from the obtained sensor records, the record information of the road surface comprising at least two sensor records from a group of at least four sensor records per image element.
[0017] The group of sensor recordings comprises a first sensor recording of a road surface having a first optical bandwidth in which water has a first absorptivity.
[0018] The group of sensor recordings comprises a second sensor recording of a road surface having a second optical bandwidth in which water has a second absorptivity that is greater than the first absorptivity.
[0019] The group of sensor recordings includes a third sensor recording having a first polarizing filter with a first polarization angle.
[0020] The group of sensor recordings further includes a fourth sensor recording having a second polarizing filter having a second polarization angle different from the first polarization angle.
[0021] The recorded information of the road surface comprises at least two sensor records per image element.
[0022] The absorption information allows one to more easily find specific materials with particular absorption characteristics in the recording.
[0023] For example, detection of water on the road surface may be enabled by using first and second sensor recordings having different optical bandwidths where water has a different absorption coefficient.
[0024] The sensor recording with a polarizing filter makes it possible to analyze the light beam reflected by the surface. For example, the reflection of liquid water shows a high polarization ratio of light mainly in the horizontal direction, while ice crystals scatter the light beam, causing wave rotation. Ice crystals result in a more strongly scattered polarization in a slightly shifted direction.
[0025] For example, by using sensor recordings with polarizing filters with different polarization angles, it is possible to distinguish between water, snow, and black ice.
[0026] In an embodiment, the first optical bandwidth comprises a bandwidth selected from the spectral range between 400 nm and 900 nm, while the second optical bandwidth comprises a bandwidth selected from the spectral range between 900 nm and 1200 nm.
[0027] The above-mentioned selection of the first and second optical bandwidths allows for optimal intensity difference between recordings in regions containing water.
[0028] In an embodiment, the first optical bandwidth comprises a value between 820 nm and 870 nm at half maximum, and the second optical bandwidth comprises a value between 920 nm and 970 nm at half maximum.
[0029] An optimal selection of the first and second optical bandwidths allows for the largest possible intensity difference between recordings in the water-containing region.
[0030] In an embodiment, the polarization angle of the first polarizing filter and the polarization angle of the second polarizing filter are dimensioned to be shifted 90° relative to each other.
[0031] Evenly distributed polarization angles provide the maximum amount of information about the light reflected from a surface. Two polarizing filters provide sufficient information about the polarization plane of the light.
[0032] In an embodiment, the classifier comprises a memory that stores, for each input tuple of the plurality of possible tuples, associated state information of a plurality of different state information, the plurality of different state information comprising, for example, a dry state, a wet state, a snowy state, and a black ice state.
[0033] Using a classifier that stores state information for possible states such as "wet", "dry", "snow", and "black ice", and compares the state information with input tuples, reduces the amount of computation required, as opposed to current methods such as image detection.
[0034] In an embodiment, the classifier is configured to round at least two parameters of the tuple to values of an existing input tuple, and then access the storage by using the existing input tuple to obtain the state of the window.
[0035] In another embodiment, the classifier is configured to select, based on the tuples of the window, an input tuple that is closest to the tuple of the window according to a criterion, for example, the criterion being the minimum additive squared deviation of at least two parameters of the tuple of the window to an existing input tuple.The classifier is then configured to access the memory with the selected tuple to obtain the state of the window.
[0036] The classifier's memory cannot contain all state information of possible states. If at least two parameters of a tuple are not found in the classifier's memory, the classifier may slightly modify the parameters before the comparison. Thus, the modified parameters of the tuple can be found in the storage. This allows the state of the window to still be found.
[0037] In an embodiment, the classifier is configured to use a separate tuple for each of a plurality of windows to detect a state of each window based on each tuple and output the state of each window of the recorded road surface information as an output value, thereby performing a window-by-window or basic classification.
[0038] Classification by window makes the system more resilient to image element errors: image elements of a window can only affect the state of the window; the effect of an image element or image element error is limited to the window.
[0039] Window-wise classification also allows for window-wise evaluation, and the quality of the classification can be determined by comparing the window with the classification result.
[0040] In an embodiment, the classifier is configured to classify contiguous regions of several adjacent windows block-wise by using the states of the adjacent windows.
[0041] Next, a contiguous region is defined, which is made up of a number of adjacent windows. Determining the boundaries of the contiguous region is very useful for avoiding various hazards in driving situations.
[0042] In an embodiment, the parameter calculator is configured to calculate at least two parameters from the group of an absorption contrast parameter, a polarization contrast parameter, and a reflection intensity parameter.
[0043] For example, the absorption contrast parameter is calculated such that the value of the absorption contrast parameter comprises a number within ±20% of the result of the following formula:
[0044]
number
[0045] In the above formula, abs contrast is the result, n is the image element number within the window, i is the index of the image element in the window, I 850(i) is the intensity value of the first sensor recording of the i-th image element in the window; I 950 (i) is the intensity value of the second sensor recording of the i-th image element within the window.
[0046] For example, the polarization contrast parameter is calculated such that the value of the polarization contrast parameter has a number within ±20% of the result of the following formula:
[0047]
number
[0048] In the above formula, pol contrast is the result, n is the image element number within the window, i is the index of the image element in the window, I pol1 (i) is the intensity value of the third sensor recording of the i-th image element in the window; I pol2 (i) is the intensity value of the fourth sensor recording of the i-th image element within the window.
[0049] For example, the reflected intensity parameter is calculated such that the value of the reflected intensity parameter has a numerical value within ±20% of the result of the following formula:
[0050]
number
[0051] In the above formula, ref mean is the result, n is the image element number within the window, i is the index of the image element in the window, I pol1 (i) is the intensity value of the third sensor recording of the i-th image element in the window; I pol2(i) is the intensity value of the fourth sensor recording of the i-th image element in the window; I 850 (i) is the intensity value of the first sensor recording of the ith image element in the window.
[0052] In an embodiment, the image divider is configured to divide the recorded information into a number of non-overlapping windows.
[0053] The number of non-overlapping windows is significantly less than the number of potentially overlapping windows. Reducing the number of windows to the number of non-overlapping windows saves computational effort.
[0054] In an embodiment, the image splitter is configured to split the recorded information into a number of windows having position-dependent window sizes, the windows representing closer road surface portions being larger than the windows representing more distant road surface portions.
[0055] In the case of position-dependent or dynamic window size, the window size is automatically controlled based on the window position. An optimal window size makes the window more resilient to possible image element errors. A large number of image elements in the window makes it possible to ignore erroneous image elements. At the same time, an optimal image window size reduces the chance of misinterpretation of the state. Distant objects are smaller in the image and can blend into the background more easily than nearby objects.
[0056] In an embodiment, the parameter calculator is configured to apply an angle correction to each window of the recorded information prior to calculation of the parameters, each image element of the window being corrected according to the angle of the incident light relative to the road surface, the correction for a window representing a near road surface portion being different from the correction for a window representing a far road surface portion.
[0057] The camera angle with respect to the road can be determined using a known window position in the image. For example, if the camera aperture angle is 22°, pixels at the top edge of the image will be tilted less than 1° with respect to the road surface, and at the bottom edge they will be tilted about 22°. This results in a large difference in reflection behavior. Angle correction can minimize these differences.
[0058] In an embodiment, the classifier is configured to associate each tuple with a state depending on the angle of the incident light with respect to the road surface.
[0059] The angle of the camera relative to the road can be determined using known window positions in the image. For example, if the camera has a 22° aperture angle, pixels at the top edge of the image will be inclined by less than 1° to the road surface, while at the bottom they will be inclined by about 22°. This makes a big difference in reflectance behavior. By utilizing much more training data, incorporating this angle information could lead to more accurate classification.
[0060] An embodiment of the present invention provides a vehicle or vehicle with an apparatus and an interface, the interface configured to notify a driver of the vehicle and / or affect control of the vehicle when the apparatus detects a water solid condition.
[0061] The vehicle's device enables safe driving through proactive localization and detection of wet areas, puddles, and ice or snow layers to derive an actual danger or actual risk of traction loss. Proactive risk detection before actual traction loss allows the system to inform the driver via an interface or to affect the control of the vehicle.
[0062] Further embodiments according to the invention provide corresponding methods or computer program products.
[0063] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiments of the present invention provide an apparatus and method for detecting water and ice on a road surface by incorporating the unique optical properties of water.
[0064] Subsequently, embodiments according to the invention will be described in more detail with reference to the accompanying drawings. [Brief description of the drawings]
[0065] [Figure 1] FIG. 1 shows a schematic diagram of an embodiment of an apparatus having an input interface, an image segmenter, a parameter calculator, and a classifier. [Diagram 2] FIG. 2 is a schematic perspective view showing a road surface and a frame representing recorded information of the road surface, with two windows having different sizes. [Diagram 3] It is a three-dimensional diagram in which each dot represents a window and the x, y, z coordinates represent three parameters of the window: the absorption contrast parameter, the polarization contrast parameter, and the reflection intensity parameter. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0066] 1 shows a schematic diagram of an embodiment of an apparatus for detecting road surface conditions 100. The apparatus comprises an input interface 110, an image segmenter 120, a parameter calculator 130 and a classifier 140.
[0067] The input interface 110 is configured to obtain road surface recording information 150 and transfer it to the image segmenter 120 .
[0068] The image divider 120 divides the recorded information 150 into a number of windows and transfers the windows, for example windows 123, 126 and 129, to the parameter calculator .
[0069] In addition to the predefined window size, the image divider 120 may also use a dynamic window size, where the window size is automatically controlled according to the window position in the image. An example of the dynamic window size is shown in FIG.
[0070] Each window comprises a number of image elements 135, and each image element 135 comprises at least two different pieces of information.
[0071] The parameter calculator 130 is configured to create one tuple 143 per window, the tuple comprising at least two parameters, the at least two parameters being calculated by using at least two different pieces of information for each image element. The tuple 143 is forwarded to the classifier 140.
[0072] The classifier 140 includes a memory 145 and / or a database 145 having different possible state information. The classifier 140 is configured to compare each tuple 143 to a plurality of state information stored in the memory 145 and / or the database 145 and detect a state 160 of the window 126 or 129 of the recorded information 150 based on the comparison.
[0073] The state information stored in memory 145 and / or database 145 may be visualized, for example, in a 3D view similar to 3D view 300 of FIG.
[0074] The overall state 170 of the recorded information 150 is assembled from the individual window states 160, where the classifier 140 may mutually classify several adjacent windows of the same state.
[0075] Fig. 2 shows a schematic perspective view 200 of a road surface 210 with a grid pattern. Each box of the grid pattern represents a square area of the road of 1m x 1m. Fig. 2 further shows a frame 220 representing recorded information 220 of the road surface. The recorded information 220 is similar to the recorded information 150 of Fig. 1. The recorded information 220 comprises two windows 226 and 229 with different window sizes.
[0076] 1, the recorded information 220 is transferred to the image divider 120 via the input interface 110. The image divider is configured to divide the recording 220 into a number of windows. The recorded information within the recorded image, or frame 220, is analyzed with the help of the windows.
[0077] Here, a window, such as window 226 or 229, is selected to analyze a recorded image or a portion of a recorded image. The size of the window is the number of pixels or image elements n in the x and y directions. x ×n y A larger number of pixels can prevent errors or misinterpretation of individual pixels by averaging, but this increases the computational power required to allow operations such as parameter calculation and / or classification to be performed fast enough. However, if the selected window is significantly larger than the object to be identified, the object will be blended with the background by averaging.
[0078] In addition to predefined window sizes, e.g., 8x4, 8x8, 16x16, 32x32, the image divider 120 of Figure 1 may use dynamic window sizes, where the window size is automatically controlled depending on the window position within the image.
[0079] FIG. 2 shows two images 226 and 229 of different sizes. The window 226 in the upper region of the image is much smaller than the window 229 in the lower region of the image. The size of the windows 226, 229 is determined in an image position dependent or dynamic manner. The distance of the surface area relative to the camera varies with the height, or y coordinate, of the image position. The far surface areas are smaller in the image than the near surface areas and are located in the upper region of the image. Therefore, the optimal window size in the upper region of the image is smaller than the optimal window size in the lower region of the image. The dynamic window size ensures the required accuracy for the distant parts or windows and allows a good averaging of closer objects.
[0080] The sensor recording is scanned pixel by pixel or image element by image element using a selected or dynamic window, and data from the sub-image or window is processed. The number of pixels in the window is determined by the formula n = n x ·n y In case of dynamic window size, the number of pixels of the window is dynamically defined depending on the window position.
[0081] Each window includes a number of image elements, such as n image elements, with each image element comprising at least two different pieces of information. By using the at least two different pieces of information, a parameter calculator, such as parameter calculator 130 of FIG. 1, creates at least two parameters for each window from the group of an absorption contrast parameter, a polarization contrast parameter, and a reflection intensity parameter. For example, the parameter calculator may calculate the at least two parameters such that the values of the at least two parameters are within ±20% of the result of the following equation:
[0082]
number
[0083]
number
[0084]
number
[0085] In the above formula, abs contrast is the absorption contrast parameter, pol contrast is the polarization contrast parameter, ref mean is the reflectance parameter, n is the image element number within the window, i is the index of the image element in the window, I 850 (i) is the intensity value of the first sensor recording of the i-th image element in the window; I 950 (i) is the intensity value of the second sensor recording of the i-th image element in the window, I pol1 (i) is the intensity value of the third sensor recording of the i-th image element in the window; I pol2 (i) is the intensity value of the fourth sensor recording of the i-th image element within the window.
[0086] At least two parameters of a window may be represented as a tuple, and each tuple may be shown in a 3D diagram, similar to 3D diagram 300 of FIG.
[0087] Figure 3 is a three-dimensional diagram, or 3D plot 300, in which each dot represents a window, similar to windows 226 and 229 in Figure 2. The x, y, and z coordinates of the dot in the 3D diagram represent three parameters of the window: an absorption contrast parameter, a polarization contrast parameter, and a reflection intensity parameter.
[0088] The 3D diagram 300 was created from several windows representing different road surfaces in different conditions. For example, a condition file in the database of the classifier can be visualized using the 3D diagram 300. The window parameters and associated conditions are stored in the memory or database of the classifier. In the 3D diagram 300, the conditions "ice", "wet", "dry" and "snow" are represented using colors or different shading.
[0089] The classifier adds new windows or new dots to this 3D drawing 300 and detects the state of the window based on the location or area of the new dot.
[0090] It immediately becomes clear that measuring areas of the same road condition results in tuples located in the same area. This allows metrological separation, i.e. classification of the states. If clear clouds, or clearly separated areas, become apparent, it is indeed possible to separate the states from each other in a metrological way. A particular advantage is that the method is not based on object detection methods of, for example, multi-stage convolutional neural networks, which first search for general objects and then mark them with anchors and in a further step mark or label them, but evaluate individual windows without regard to the entire window. This makes the inventive concept very robust against interferences such as obscuring objects, but also very easy to understand and therefore to verify in terms of its functionality.
[0091] The quality of the detection depends on the amount of data. With additional data, the state can be used more finely. This can be real data recorded with other cameras or filters, or information extracted in various ways from existing camera images.
[0092] In addition, the classifier may also consider the position of the window, or the distance to the camera. Using the known window position in the image, the angle of the camera relative to the road can be determined. For example, the camera's aperture angle in this embodiment is 22°. Thus, pixels at the top of the image are inclined at less than 1° to the road surface, while pixels at the bottom are inclined at about 22°, resulting in a significant difference in reflection behavior. Incorporating this information may lead to more accurate classification, but will require more training data.
[0093] Although some aspects are described in the context of a device, it will be understood that said aspects also represent a description of a corresponding method, such that blocks or structural components of the device are also understood as corresponding method steps or as functions of method steps. By analogy therewith, aspects described in the context of or as method steps also represent a description of the corresponding blocks, or details or functions of the corresponding device. Some or all of the method steps may be performed by (or using) a hardware device. In some embodiments, some or some of the most important method steps may be performed by such a device.
[0094] Depending on the requirements of a particular implementation, the embodiments of the present invention can be implemented in hardware or software. The implementations may be carried out using digital storage media, such as, for example, floppy disks, DVDs, Blu-ray disks, CDs, ROMs, PROMs, EPROMs, EEPROMs or flash memories, hard disks, or any other magnetic or optical memory on which electronically readable control signals are stored that can or will cooperate with a programmable computer system so that the respective methods are executed. This is why the digital storage media can be computer readable.
[0095] Thus, some embodiments according to the present invention comprise a data carrier comprising electronically readable control signals capable of cooperating with a programmable computer system to perform any of the methods described herein.
[0096] In general, embodiments of the present invention may be implemented as a computer program product having program code which is effective to perform any of the methods when the computer program product is run on a computer.
[0097] The program code may also be stored on a machine readable medium, for example.
[0098] Other embodiments comprise the computer program for performing any of the methods described herein, said computer program stored on a machine readable carrier.
[0099] In other words, an embodiment of the inventive method is, a computer program having a program code for performing any of the methods described herein, when the computer program runs on a computer.
[0100] A further embodiment of the inventive method is, therefore, a data carrier (or digital storage medium or computer readable medium) having recorded thereon the computer program for performing any of the methods described herein.
[0101] A further embodiment of the inventive method is therefore a data stream or a sequence of signals representing the computer program for performing any of the methods described herein, The data stream or the sequence of signals may for example be adapted to be transmitted via a data communication link, for example via the Internet.
[0102] Further embodiments comprise a processing unit, for example a computer or a programmable logic device, configured or adapted to perform any of the methods described herein.
[0103] A further embodiment comprises a computer having installed thereon the computer program for performing any of the methods described herein.
[0104] Further embodiments according to the invention include a device or system configured to transmit a computer program for performing at least one of the methods described herein to a receiver. The transmission may be, for example, electronic or optical. The receiver may be, for example, a computer, a mobile device, a memory device, or a similar device. The device or system may, for example, include a file server for transmitting the computer program to the receiver.
[0105] In some embodiments, a programmable logic device (e.g., a field programmable gate array, FPGA) may be used to perform some or all of the functions of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor to perform any of the methods described herein. In general, the methods are performed in some embodiments by any hardware device. The hardware device may be any universally applicable hardware, such as a computer processor (CPU), or may be method-specific hardware, such as an ASIC.
[0106] The above-described embodiments merely represent an example of the principles of the present invention. It is understood that others skilled in the art will appreciate modifications and variations of the configurations and details described herein. This is why it is intended that the present invention be limited only by the scope of the following claims and not by the specific details presented herein by way of the description and discussion of the embodiments. [Explanation of symbols]
[0107] 100 devices 110 Input Interface 120 Image Splitter 123 Window 126 Windows 129 Windows 130 Parameter Calculator 135 Image Elements 140 Classifier 143 Tuples 145 Memory 145 Databases 150 Record Information 160 Status 170 Status 200 Perspective View 210 Road surface 220 Record Information 220 frames 226 Window 229 Window 300 3D figures
Claims
1. A device (100) for detecting a condition (160, 170) of a road surface (210), comprising: an input interface (110) for acquiring recorded information (150, 220) of a road surface; an image divider (120) for dividing the recorded information into a plurality of windows (123, 126, 129, 226, 229), each window comprising a plurality of image elements (135), each image element comprising at least two different pieces of information; a parameter calculator (130) for calculating at least two parameters for each window by using the at least two different information of each image element within the window; a classifier (140) for classifying the window based on the tuple (143) of the at least two parameters of the window, the classifier being configured to detect a state of the window based on the tuple (143) of the at least two parameters of the window as an input value and to output the state (160, 170) of the window as an output value (160, 170); Equipped with the classifier (140) comprises a memory (145) for storing, for each input tuple (143) of a plurality of possible tuples (143), associated state information (160, 170) of a plurality of different state information (160, 170); the classifier (140) is configured to round the at least two parameters of the tuple (143) to a value of an existing input tuple (143) and then access the memory by using the existing input tuple (143); or the classifier (140) is configured to select, based on the tuple (143) of the window (123, 126, 129, 226, 229), an input tuple (143) that is closest to the tuple (143) of the window (123, 126, 129, 226, 229) according to a criterion, and then access the memory (145) using the selected tuple (143) to obtain the state of the window (123, 126, 129, 226, 229). Equipment (100).
2. the input interface (110) is configured to obtain the recorded information (150, 220) from a sensor or to obtain several sensor records and to assemble the recorded information (150, 220) from the obtained sensor records; The recorded information (150, 220) of the road surface (210) is divided into the following groups of sensor records for each image element (135): a first sensor recording of the road surface (210) having a first optical bandwidth in which water has a first absorptivity; a second sensor recording of the road surface (210) having a second optical bandwidth where water has a second absorptivity higher than the first absorptivity; a third sensor recording with a first polarizing filter having a first polarization angle; a fourth sensor recording having a second polarizing filter having a second polarization angle, the first polarizing filter and the second polarizing filter having different polarization angles; and The apparatus (100) of claim 1, comprising at least two sensor records from
3. 3. The apparatus (100) of claim 2, wherein the first optical bandwidth comprises a bandwidth selected from a spectral range between 400 nm and 900 nm, and the second optical bandwidth comprises a bandwidth selected from a spectral range between 900 nm and 1200 nm.
4. 3. The apparatus (100) of claim 2, wherein the first optical bandwidth has a value between 820 nm and 870 nm at half maximum and the second optical bandwidth has a value between 920 nm and 970 nm at half maximum.
5. 3. The apparatus (100) of claim 2, wherein the polarization angle of the first polarizing filter and the polarization angle of the second polarizing filter are dimensioned to be shifted 90 degrees with respect to each other.
6. The apparatus (100) of claim 1, wherein the plurality of different condition information (160, 170) comprises a dry condition, a wet condition, a snowy condition, and a black ice condition.
7. The apparatus (100) of claim 1, wherein the criterion is the minimum additive squared deviation of the at least two parameters of the tuples (143) of the window (123, 126, 129, 226, 229) relative to the existing input tuples (143).
8. A device (100) for detecting a condition (160, 170) of a road surface (210), comprising: an input interface (110) for acquiring recorded information (150, 220) of a road surface; an image divider (120) for dividing the recorded information into a plurality of windows (123, 126, 129, 226, 229), each window comprising a plurality of image elements (135), each image element comprising at least two different pieces of information; a parameter calculator (130) for calculating at least two parameters for each window by using the at least two different information of each image element within the window; a classifier (140) for classifying the window based on the tuple (143) of the at least two parameters of the window, the classifier being configured to detect a state of the window based on the tuple (143) of the at least two parameters of the window as an input value and to output the state (160, 170) of the window as an output value (160, 170); Equipped with the classifier (140) is configured to use a separate tuple (143) for each of the plurality of windows (123, 126, 129, 226, 229) to detect a state (160, 170) of each window (123, 126, 129, 226, 229) based on each tuple (143) and to output the state (160, 170) of each window (123, 126, 129, 226, 229) of the recorded information (150, 220) of the road surface (210) as an output value (160, 170); the classifier (140) is configured to classify, block by block, consecutive regions of several adjacent windows (123, 126, 129, 226, 229) by using states (160, 170) of the adjacent windows (123, 126, 129, 226, 229); Equipment (100).
9. A device (100) for detecting a condition (160, 170) of a road surface (210), comprising: an input interface (110) for acquiring recorded information (150, 220) of a road surface; an image divider (120) for dividing the recorded information into a plurality of windows (123, 126, 129, 226, 229), each window comprising a plurality of image elements (135), each image element comprising at least two different pieces of information; a parameter calculator (130) for calculating at least two parameters for each window by using the at least two different information of each image element within the window; a classifier (140) for classifying the window based on the tuple (143) of the at least two parameters of the window, the classifier being configured to detect a state of the window based on the tuple (143) of the at least two parameters of the window as an input value and to output the state (160, 170) of the window as an output value (160, 170); Equipped with the parameter calculator (130) is configured to calculate at least two parameters from the group of an absorption contrast parameter, a polarization contrast parameter, and a reflection intensity parameter; Equipment (100).
10. A device (100) for detecting a condition (160, 170) of a road surface (210), comprising: an input interface (110) for acquiring recorded information (150, 220) of a road surface; an image divider (120) for dividing the recorded information into a plurality of windows (123, 126, 129, 226, 229), each window comprising a plurality of image elements (135), each image element comprising at least two different pieces of information; a parameter calculator (130) for calculating at least two parameters for each window by using the at least two different information of each image element within the window; a classifier (140) for classifying the window based on the tuple (143) of the at least two parameters of the window, the classifier being configured to detect a state of the window based on the tuple (143) of the at least two parameters of the window as an input value and to output the state (160, 170) of the window as an output value (160, 170); Equipped with the input interface (110) is configured to obtain the recorded information (150, 220) from a sensor or to obtain several sensor records and to assemble the recorded information (150, 220) from the obtained sensor records; The recorded information (150, 220) of the road surface (210) is divided into the following groups of sensor records for each image element (135): a first sensor recording of the road surface (210) having a first optical bandwidth in which water has a first absorptivity; a second sensor recording of the road surface (210) having a second optical bandwidth where water has a second absorptivity higher than the first absorptivity; a third sensor recording with a first polarizing filter having a first polarization angle; a fourth sensor recording having a second polarizing filter having a second polarization angle, the first polarizing filter and the second polarizing filter having different polarization angles; and at least two sensor records from Where: the parameter calculator (130) is configured to calculate an absorption contrast parameter such that a value of the absorption contrast parameter comprises a number within ±20% of the result of the following formula: [0010] In the above formula, abs contrast is the result, n is the image element number within said window (123, 126, 129, 226, 229), i is the index of the image element within said window (123, 126, 129, 226, 229), I 850 (i) is the intensity value of the first sensor recording of the i-th image element (135) within the window (123, 126, 129, 226, 229); I 950 (i) is the intensity value of the second sensor recording of the i-th image element (135) within the window (123, 126, 129, 226, 229), or the parameter calculator (130) is configured to calculate the polarization contrast parameter such that the value of the polarization contrast parameter is within ±20% of the result of the following formula: [0025] In the above formula, pol contrast is the result, n is the image element number within said window (123, 126, 129, 226, 229), i is the index of the image element within said window (123, 126, 129, 226, 229), I pol1 (i) is the intensity value of the third sensor recording of the i-th image element (135) within the window (123, 126, 129, 226, 229); I pol2 (i) is the intensity value of the fourth sensor recording of the i-th image element (135) within the window (123, 126, 129, 226, 229), or the parameter calculator (130) is configured to calculate the reflectance parameter such that the value of the reflectance parameter is within ±20% of the result of the following formula: [0030] In the above formula, ref mean is the result, n is the image element number within said window (123, 126, 129, 226, 229), i is the index of the image element within said window (123, 126, 129, 226, 229), I pol1 (i) is the intensity value of the third sensor recording of the i-th image element (135) within the window (123, 126, 129, 226, 229); I pol2 (i) is the intensity value of the fourth sensor recording of the i-th image element (135) within the window (123, 126, 129, 226, 229); I 850 (i) is the intensity value of the first sensor recording of the i-th image element (135) within the window (123, 126, 129, 226, 229); Equipment (100).
11. A device (100) for detecting a condition (160, 170) of a road surface (210), comprising: an input interface (110) for acquiring recorded information (150, 220) of a road surface; an image divider (120) for dividing the recorded information into a plurality of windows (123, 126, 129, 226, 229), each window comprising a plurality of image elements (135), each image element comprising at least two different pieces of information; a parameter calculator (130) for calculating at least two parameters for each window by using the at least two different information of each image element within the window; a classifier (140) for classifying the window based on the tuple (143) of the at least two parameters of the window, the classifier being configured to detect a state of the window based on the tuple (143) of the at least two parameters of the window as an input value and to output the state (160, 170) of the window as an output value (160, 170); Equipped with the image divider (120) is configured to divide the recorded information (150, 220) into a plurality of non-overlapping windows (123, 126, 129, 226, 229); or the image divider (120) is configured to divide the recorded information (150, 220) into a plurality of windows (123, 126, 129, 226, 229) having a position-dependent window size, the windows (123, 126, 129, 226, 229) representing closer road surface portions being larger than the windows representing more distant road surface portions, Equipment (100).
12. A device (100) for detecting a condition (160, 170) of a road surface (210), comprising: an input interface (110) for acquiring recorded information (150, 220) of a road surface; an image divider (120) for dividing the recorded information into a plurality of windows (123, 126, 129, 226, 229), each window comprising a plurality of image elements (135), each image element comprising at least two different pieces of information; a parameter calculator (130) for calculating at least two parameters for each window by using the at least two different information of each image element within the window; a classifier (140) for classifying the window based on the tuple (143) of the at least two parameters of the window, the classifier being configured to detect a state of the window based on the tuple (143) of the at least two parameters of the window as an input value and to output the state (160, 170) of the window as an output value (160, 170); Equipped with the parameter calculator (130) is configured to apply an angle correction to each window (123, 126, 129, 226, 229) of the recorded information (150, 220) before calculating the parameters, such that each image element (135) of the window (123, 126, 129, 226, 229) is corrected depending on the angle of incidence of the light with respect to the road surface (210), the correction for the windows (123, 126, 129, 226, 229) representing closer road surface parts being different from the correction for the windows representing more distant road surface parts, Equipment (100).
13. A device (100) for detecting a condition (160, 170) of a road surface (210), comprising: an input interface (110) for acquiring recorded information (150, 220) of a road surface; an image divider (120) for dividing the recorded information into a plurality of windows (123, 126, 129, 226, 229), each window comprising a plurality of image elements (135), each image element comprising at least two different pieces of information; a parameter calculator (130) for calculating at least two parameters for each window by using the at least two different information of each image element within the window; a classifier (140) for classifying the window based on the tuple (143) of the at least two parameters of the window, the classifier being configured to detect a state of the window based on the tuple (143) of the at least two parameters of the window as an input value and to output the state (160, 170) of the window as an output value (160, 170); Equipped with the classifier (140) is configured to associate each tuple (143) with a state (160, 170) depending on an angle of incident light with respect to the road surface (210); Equipment (100).
14. A vehicle comprising a device (100) according to any one of claims 1, 8 to 13 and an interface, the interface being configured to inform a driver of the vehicle and / or to affect control of the vehicle when the device detects a solid state of water.
15. A method for detecting a condition (160, 170) of a road surface (210), comprising: obtaining recorded information (150, 220) of the road surface (210); Dividing the recorded information (150, 220) into a plurality of windows (123, 126, 129, 226, 229), each window (123, 126, 129, 226, 229) comprising a plurality of image elements (135), each element (135) comprising at least two different pieces of information; calculating at least two parameters for each window (123, 126, 129, 226, 229) by using said at least two different pieces of information for each image element within said window (123, 126, 129, 226, 229); classifying said window (123, 126, 129, 226, 229) based on said tuple (143) of at least two parameters of said window (123, 126, 129, 226, 229) to obtain a state (160, 170) of said window (123, 126, 129, 226, 229) based on said tuple (143) of at least two parameters of said window (123, 126, 129, 226, 229); Equipped with The step of classifying (140) includes using a memory (145) for storing, for each input tuple (143) of a plurality of possible tuples (143), associated state information (160, 170) of a plurality of different state information (160, 170), and the step of classifying includes rounding the at least two parameters of the tuple (143) to values of an existing input tuple (143), and then accessing the memory by using the existing input tuple (143). or selecting, based on the tuples (143) of the window (123, 126, 129, 226, 229), an input tuple (143) that is closest to the tuple (143) of the window (123, 126, 129, 226, 229) according to a criterion, and then accessing the memory (145) using the selected tuple (143) to obtain the state of the window (123, 126, 129, 226, 229); or the calculating step includes calculating at least two parameters from the group of an absorption contrast parameter, a polarization contrast parameter, and a reflection intensity parameter; or the dividing step includes dividing the recorded information (150, 220) into a plurality of non-overlapping windows (123, 126, 129, 226, 229) or dividing the recorded information (150, 220) into a plurality of windows (123, 126, 129, 226, 229) having position-dependent window sizes, the windows (123, 126, 129, 226, 229) representing closer road surface portions being larger than the windows representing farther road surface portions; or the calculating step comprises, prior to the calculation of the parameters, applying an angle correction to individual windows (123, 126, 129, 226, 229) of the recorded information (150, 220), in which each image element (135) of the window (123, 126, 129, 226, 229) is corrected according to an angle of incidence of light with respect to the road surface (210), the correction for the window (123, 126, 129, 226, 229) representing a near road surface portion being different from the correction for a window representing a far road surface portion; or the classifying step comprises associating each tuple (143) with a state (160, 170) depending on an angle of incident light with respect to the road surface (210); or The step of acquiring includes acquiring the record information (150, 220) from a sensor or acquiring several sensor records and assembling the record information (150, 220) from the acquired sensor records, wherein the record information (150, 220) of the road surface (210) is divided into the following groups of sensor records for each image element (135): a first sensor recording of the road surface (210) having a first optical bandwidth in which water has a first absorptivity; a second sensor recording of the road surface (210) having a second optical bandwidth where water has a second absorptivity higher than the first absorptivity; a third sensor recording with a first polarizing filter having a first polarization angle; a fourth sensor recording having a second polarizing filter having a second polarization angle, the first polarizing filter and the second polarizing filter differing in their polarization angles; and and wherein the calculating step includes calculating an absorption contrast parameter such that the value of the absorption contrast parameter comprises a number within ±20% of the result of the following formula: [0045] In the above formula, abscontrast is the result, n is the image element number within said window (123, 126, 129, 226, 229), i is the index of the image element within said window (123, 126, 129, 226, 229), I 850 (i) is the intensity value of the first sensor recording of the i-th image element (135) within the window (123, 126, 129, 226, 229); I 950 (i) is the intensity value of the second sensor recording of the i-th image element (135) within the window (123, 126, 129, 226, 229); or the calculating step includes calculating the polarization contrast parameter such that the value of the polarization contrast parameter is within ±20% of the result of the following formula: [0050] In the above formula, pol contrast is the result, n is the image element number within said window (123, 126, 129, 226, 229), i is the index of the image element within said window (123, 126, 129, 226, 229), I pol1 (i) is the intensity value of the third sensor recording of the i-th image element (135) within the window (123, 126, 129, 226, 229); I pol2 (i) is the intensity value of the fourth sensor recording of the i-th image element (135) within the window (123, 126, 129, 226, 229), or the calculating step includes calculating the reflectance parameter such that the value of the reflectance parameter is within ±20% of the result of the following formula: [006] In the above formula, ref mean is the result, n is the image element number within said window (123, 126, 129, 226, 229), i is the index of the image element within said window (123, 126, 129, 226, 229), I pol1 (i) is the intensity value of the third sensor recording of the i-th image element (135) within the window (123, 126, 129, 226, 229); I pol2 (i) is the intensity value of the fourth sensor recording of the i-th image element (135) within the window (123, 126, 129, 226, 229); I 850 (i) is the intensity value of the first sensor recording of the i-th image element (135) within the window (123, 126, 129, 226, 229); method.
16. 16. A computer program for carrying out the method according to claim 15 when said computer program is executed on a computer or processor.
Citation Information
Patent Citations
Road black ice intelligent detection and early warning system and method
CN113257000A
Device and method for determining road surface condition
JP2007232652A
On-vehicle sensor, vehicle lighting fixture, vehicle and road surface state sensor
JP2017083352A
Method and apparatus for evaluating a vehicle travel surface
US20190188495A1
