Road condition detection system and method for detecting a surface condition of a road ahead of an ego vehicle

Abstract

The invention relates to a road condition detection system (100) for detecting a surface condition of a road (16) ahead of an ego vehicle (10) currently taking the road, wherein the road condition detection system comprises: signal acquisition means (20) configured to acquire input signals resulting from a detection with respect to a surface condition of the road ahead of the ego vehicle and resulting from a detection of a traffic sign (12) ahead of the ego vehicle, processing means configured to process the acquired input signals and to obtain, from the processing, road surface condition information on whether the surface condition of the road ahead of the ego vehicle is abnormal or normal, and traffic sign information on whether the detected traffic sign indicates that the surface condition of the road ahead of the ego vehicle is abnormal, judging means (40) configured to make a judgement, the judgement indicating that the surface condition of the road ahead of the ego vehicle is abnormal if the traffic sign information indicates that the surface condition of the road ahead of the ego vehicle is abnormal, and abnormal road condition setting means (50) configured to set at least one control signal for changing at least one dynamic parameter of the ego vehicle, relating to the dynamics of the ego vehicle, based on the judgement indicating that that the surface condition of the road ahead of the ego vehicle is abnormal. Moreover, the invention is directed to a corresponding computer-implemented method.

Description

[0001] Road condition detection system and method for detecting a surface condition of a road ahead of an ego vehicle

[0002] Description

[0003] TECHNICAL FIELD

[0004] The invention relates to a road condition detection system for detecting a surface condition of a road ahead of an ego vehicle, preferably an autonomous or driver-assisted vehicle, currently taking the road.

[0005] Furthermore, the invention relates to a computer-implemented method for detecting a surface condition of a road ahead of such an ego vehicle currently taking the road.

[0006] Moreover, the invention is directed to a driver assistance system for said vehicle, comprising such a road condition detection system, a vehicle comprising such a driver assistance system, a corresponding computer program for carrying out the computer-implemented method and a respective non-transitory computer- readable medium storing said computer program.

[0007] BACKGROUND

[0008] Conventional Driver Assistance Systems or Advanced Driver Assistance Systems (ADAS) for vehicles are widely used, for example, in connection with conventional driver-assisted vehicles as well as autonomous vehicles, and usually comprise applications such as an Adaptive Cruise Control (ACC), Traffic-Sign-Assist (TSA), Speed-Limit-Assist (SLA) and so on, which applications aim to avoid vehicle accidents that may have been caused by human errors.

[0009] For example, the Traffic Sign Assist application ensures that the current speed limit and other road signs are displayed to the driver on an ongoing basis, for example, on a screen inside the vehicle.

[0010] Automatic recognition functions via a link between images captured by camera or image sensors, for example, a front camera mounted on a front of the vehicle, and the speed limit information stored in the navigation system. In this way, even speed limits that are not explicitly visible, like within a city, may be displayed to the driver.

[0011] Optionally, the Speed-Limit-Assist application can be extended to an Intelligent Speed Assist (ISA) by adapting the cruise control with speed limit information from traffic sign recognition.

[0012] ADAS applications are also used in connection with the detection of road surface conditions.

[0013] Roads are designed such that vehicles can be driven comfortably and safely. However, at certain locations like sharp curves, congested or accident-prone zones, residential streets and at school zones, additional vehicle speed control is necessary for the safety of people. This is taken care by using numerous methods such as placing speed limit signs, road markings and in certain situations vertical deflections of the surface of the road like road humps, speed bumps, and so on are placed.

[0014] Speed bumps (also called traffic thresholds, speed breakers or sleeping policemen) are a class of traffic calming devices that use a vertical deflection of the road to slow motor-vehicle traffic in order to improve safety conditions. Variations of these are also known as speed humps, speed cushions, speed tables, etc.

[0015] Usually, such a speed bump is a bump in a roadway with vertical heights typically ranging between 8 and 10 centimeters. The traverse distance of a speed bump is typically less than or near to 0.3 m; contrasting with the wider speed humps, which typically have a traverse distance of 3.0 to 4.3 m. Speed bumps vary in length, but it is typical to leave space between the bump and either edge of an enclosed road (i.e. with curbs and gutters) to allow for drainage. Spaces on either side may also allow more expedient passage for emergency vehicles, though effectiveness will depend on the type of vehicle and specific road design.

[0016] A speed hump, which is also called a road hump, or undulation and speed ramp, is a rounded traffic calming device used to reduce vehicle speed and thus sound volume on residential streets. Humps are vertical deflections of the road, are placed across the road to slow traffic and are often installed in a series of several humps to prevent cars from speeding before and after the hump. Common speed hump shapes are parabolic, circular, and sinusoidal. Generally, speed humps have a traverse distance of about 3.7 to 4.3 m and span the width of the road. The height of each hump ranges from 8 to 10 cm. The traverse distance and height of each hump determines the speed at which traffic will travel over the devices. Shorter traverse lengths and greater heights slow cars most drastically. Usually, speed humps are used in locations where low speeds are desired and suitable for the surrounding traffic environment. Speed humps are typically placed on residential roads and are not used on major roads, bus routes, or primary emergency response routes.

[0017] However, placing of such road humps on roads may not always serve the main purpose to reduce the vehicle’s speed but might, in some cases, lead to damage of the vehicle and, in few cases, road humps have induced accidents and even loss of life.

[0018] That is, generally speaking, when vehicles are traveling on a road with abnormal road surface conditions in the form of vertical deflections of the surface of the road exceeding a specific vertical height like speed bumps, road humps, and the like, bottoms of the vehicles may be scraped or safety accidents may occur.

[0019] In view of this, for example, Electronic Control Suspension systems (ECSs) have been employed, which perform suspension and damping control at the point of time when the vehicle passes the speed bump, in order to avoid at least damages of the vehicle.

[0020] Road Safety Assist Systems or Advanced Driver Assistance Systems according to the prior art are configured such that the detection of speed bumps or road humps is highlighted on a screen inside the vehicle as described above, using object detection and image processing techniques. For example, according to US 2021197829 A1 , a road hump is detected as one car passes the road hump using vehicle characteristic changes or sensors and this information is stored in cloud or Wi-Fi is used to inform the upcoming other vehicles on the same road. However, this measure lacks the real time detection of road humps and may lead to usage redundant / false information.

[0021] Even if a road hump is detected in order to assist the driver, however, a level of severity of the road hump with respect to the current vehicle is not determined before assisting the driver. However, severity determines how much the road hump affects the current moving vehicle, which is essential for assisting the driver.

[0022] For example, according to the prior art, a system and method for detecting road surface conditions are known from US 9355319 B2, and an adaptive active suspension system with road preview is know from US 8788146 B1. Moreover, US 2016253566 A1 discloses a road vertical contour detection using a stabilized coordinate frame. Particularly, this document discloses using the lateral slope of a road surface to find a speed hump.

[0023] SUMMARY OF THE INVENTION

[0024] A road condition detection system according to the invention is for detecting a surface condition of a road ahead of an ego vehicle, which is preferably an autonomous or driver-assisted vehicle, currently taking the road. The road condition detection system according to the invention comprises: signal acquisition means configured to acquire input signals resulting from a detection with respect to a surface condition of the road ahead of the ego vehicle and resulting from a detection of a traffic sign ahead of the ego vehicle, processing means configured to process the acquired input signals and to obtain, from the processing, road surface condition information on whether the surface condition of the road ahead of the ego vehicle is abnormal or normal, and traffic sign information on whether the detected traffic sign indicates that the surface condition of the road ahead of the ego vehicle is abnormal, judging means configured to make a judgement indicating that the surface condition of the road ahead of the ego vehicle is abnormal if the traffic sign information indicates that the surface condition of the road ahead of the ego vehicle is abnormal, and abnormal road condition setting means configured to set at least one control signal for changing at least one dynamic parameter of the ego vehicle, relating to the dynamics of the ego vehicle, based on the judgement indicating that the surface condition of the road ahead of the ego vehicle is abnormal.

[0025] Specific road surface conditions may be localized vertical deflections of the road. If these vertical deflections exceed a specific threshold height value, these vertical deflections are considered as abnormal surface conditions of the road. Examples of abnormal surface conditions of the road are the aforementioned road humps and speed bumps.

[0026] Accordingly, one advantageous effect achieved by the invention is that the road condition detection system is able to detect an abnormal surface condition of the road like a road hump or road speed breaker in real time and, thus, is able to further assist the driver in real time, for example, based on the severity of the detected road hump. That is, as the decision as to whether the surface condition of the road ahead of the ego vehicle is abnormal, for example, is a vertical deflection in the form of a road hump is present on the road currently taken by the ego vehicle and ahead of the vehicle is made dependent on traffic sign recognition, speed control of the vehicle is performed irrespective of whether image detection of the road hump leads to speed assist control of the vehicle or not.

[0027] This means that the driver can be assisted at an early stage, even if mere image detection of the road hump would result in the cruise control remaining inactive and no notification of a road hump on the road ahead of the vehicle.

[0028] According to the invention, a severity based speed reduction for less damage to vehicle and passenger safety can be performed. Additionally, enhancement of road database for better navigation is possible by updating the database accordingly upon detection of the traffic sign indicating presence of an upcoming road hump, for example.

[0029] In other words, the road condition detection system according to the invention carries out a method to detect road humps or road speed breakers, in order to further assist the driver of the vehicle based on a severity of the road hump. After detecting the road hump, the driver is assisted in accordance with a severity of the road hump with respect to the vehicle, which determines how much the vehicle is affected by the road hump.

[0030] The road condition detection system according to the invention can be further configured in such a manner that the judging means is configured to make the judgment indicating that the surface condition of the road ahead of the ego vehicle is abnormal if the traffic sign information indicates that the surface condition of the road ahead of the ego vehicle is abnormal, irrespective whether the road surface condition information indicates a vertical deflection of the surface of the road exceeding or being below a predetermined vertical height value.

[0031] This is particularly advantageous in the case where, due to detection inaccuracies, the image detection of the road hump would lead to the result that no road hump is present, but, in fact, such a road hump is present as confirmed by the traffic sign.

[0032] Moreover, the road condition detection system according to the invention can be further designed such that the abnormal road condition setting means is configured to set the control signal to change the dynamic parameter of the ego vehicle such that the speed of the ego vehicle changes, setting of the control signal being dependent on an amount, i.e. height, of vertical deflection of a road surface and / or the distance between the position of the ego vehicle and position of the vertical deflection of the road surface. Accordingly, the invention advantageously evaluates severity of the vertical deflection, i.e. for example, of the detected road hump or road speed breaker, in order to further assist the driver of the vehicle in controlling the speed of the vehicle based on a severity of the road hump, i.e. based on a distance of the position of the vehicle to the road hump and a respective height of the road hump, for example. For example, in order to detect / find the distance between the ego vehicle and the vertical deflection, a generic approach of camera projections is used, which is known from the prior art. In more concrete terms, initially, the vertical deflection is detected in top view image coordinates of a top view image. The top view image coordinates are converted to front view image coordinates of a front view image, using camera parameters. Finally, the front view image coordinates are then converted to world coordinates. The world coordinate x direction is then used for detecting the distance between the ego vehicle and the vertical deflection.

[0033] Furthermore, the road condition detection system according to the invention can be further realized such that the abnormal road condition setting means is configured to classify the amount of the vertical deflection of the road surface in height ranges including a low height range, a moderate height range and a tall height range, the moderate height range lying between the low height range and the tall height range, classify the distance between the position of the ego vehicle and the position of the vertical deflection of the road surface in distance ranges including a short distance range, mid distance range and a long distance range, the mid distance range lying between the short distance range and the long distance range, and set the control signal so that the dynamic parameter of the ego vehicle effects that the ego vehicle is

[0034] - linearly decelerated if the height of the vertical deflection of the road surface is judged to be in the tall height range and the distance between the position of the ego vehicle and the position of the vertical deflection of the road surface is judged to be in the long distance range, and / or the height of the vertical deflection of the road surface is judged to be in the tall height range and the distance between the position of the ego vehicle and the position of the vertical deflection of the road surface is judged to be in the mid distance range, and / or the height of the vertical deflection of the road surface is judged to be in the moderate height range and the distance between the position of the ego vehicle and the position of the vertical deflection of the road surface is judged to be in the mid distance range, and / or the height of the vertical deflection of the road surface is judged to be in the low height range and the distance between the position of the ego vehicle and the position of the vertical deflection of the road surface is judged to be short distance range, and / or

[0035] - immediately decelerated if the height of the vertical deflection of the road surface is judged to be in the tall height range and the distance between the position of the ego vehicle and the position of the vertical deflection of the road surface is judged to be in the short distance range, and / or the height of the vertical deflection of the road surface is judged to be in moderate height range and the distance between the position of the ego vehicle and the position of the vertical deflection of the road surface is judged to be short distance range, and / or

[0036] - slightly decelerated if the height of the vertical deflection of the road surface is judged to be in the moderate height range and the distance between the position of the ego vehicle and the position of the vertical deflection of the road surface is judged to be in the long distance range, and / or the height of the vertical deflection of the road surface is judged to be in the low height range and the distance between the position of the ego vehicle and the vertical deflection of the road surface is judged to be in the mid distance range, and / or

[0037] - not decelerated or to drive in constant speed if the height of the vertical deflection of the road surface is judged to be low height range and the distance between the position of the ego vehicle and the vertical deflection of the road surface is judged to be in the long distance range.

[0038] In this connection, low height range of the vertical deflection means a height equal or lower than 4cm, moderate height range of the vertical deflection means a height between 4cm and 8cm, and tall height range of the vertical deflection means a height equal or greater than 8cm.

[0039] Moreover, short distance range means a range equal or less than 20m, mid distance range means a range between 20m and 30m, and a long distance range means a range equal or greater than 40m.

[0040] Furthermore, a linear deceleration of the vehicle means to decelerate or to deaccelerate slowly to a vehicle speed of less than 3.6 - 6 kmph (km / h). A slight deceleration means to decelerate or deaccelerate moderately to a vehicle speed of 4 - 10 kmph. An immediate deceleration means to decelerate or deaccelerate directly to a vehicle speed of less than 10-16 kmph.

[0041] By means of such a setting of the control signal, a severity of a road hump is evaluated for several relevant cases and the assistance for vehicle speed control can be performed in an adequate manner.

[0042] Additionally, the road condition detection system according to the invention can be further modified in such a way that the processing means is configured to obtain, from the processing, the road surface condition information to indicate that the surface condition of the road is abnormal if the processing of the input signals resulting from the detection with respect to a surface condition of the road ahead of the ego vehicle results in the vertical deflection of the surface of the road exceeding a predetermined height. That is, even if no traffic sign is being detected, indicating that the surface condition of the road ahead of the ego vehicle is abnormal, i.e. a road hump is existing ahead of the vehicle, it is nevertheless checked in real time whether a vertical reflection of the surface of the road exceeds a predetermined height value for determining whether the surface condition of the road is abnormal, i.e. a road hump exceeding a specific height is present. If so, the road surface condition information is processed to indicate that the surface condition of the road is abnormal. Otherwise, if the vertical deflection of the surface of the road does not exceed the predetermined height, i.e. is below the predetermined height, the road surface condition information is processed to indicate that the surface condition of the road is normal.

[0043] Beyond that, the road condition detection system according to the invention can be further configured in such a way that the processing means is configured to process the acquired input signals by performing a road segmentation on a detected top view image resulting from a detection with respect to a surface condition of the road ahead of the ego vehicle and analyzing a number of stripes on the road surface and / or length of a potential abnormal road surface region in the longitudinal direction of the road, in which region the surface condition of the road may be abnormal. For example, if a specific number of stripes is detected and the detected stripes are arranged within the potential abnormal road surface region and have a specific minimum length, the surface condition of the road within the potential abnormal road surface region is further examined.

[0044] In this connection, the road condition detection system according to the invention can be further modified such that the processing means is configured to process the acquired input signals and proceed to further processing if the number of stripes exceeds a predetermined number and / or the length of the potential abnormal road surface region exceeds a predetermined length. For example, if more than two stripes are detected and the length requirement of the vertical deflection of the road, i.e. the potential road hump, is fulfilled, further checks of the potential abnormal road surface region are carried out.

[0045] Moreover, the road condition detection system according to the invention can be further implemented in such a way that the processing means is configured to further process the acquired input signals by

[0046] - setting the potential abnormal road surface region in the top view image, the potential abnormal road surface region being located in a predetermined minimum distance ahead of the ego vehicle,

[0047] - selecting a centroid in the potential abnormal road surface region as a first reference point in the top view image,

[0048] - setting second and third reference points neighboring the first reference point in the top view image so that the first reference point is located between the second and third reference points, the second and third reference points being located near the boundary of the potential abnormal road surface region in the longitudinal direction of the road, - setting fourth and fifth reference points before and after the potential abnormal road surface region in the longitudinal direction of the road in the top view image,

[0049] - converting 2D positions of the first to fifth reference points in the in the top view image to 3D positions in a perspective front view image from which the top view image is obtained by image conversion processing,

[0050] - detecting a height difference between the road surface outside and inside the potential abnormal road surface region by means of a calculation using the 3D positons of the respective first to fifth reference points.

[0051] In more concrete terms, the detection of a surface condition of the road to be abnormal, i.e. the road hump detection, is carried out via a front view camera mounted on the vehicle that captures images. Road segmentation is performed on the top view image and the number of stripes on the road is analyzed. If the number of stripes is greater than two and if the length of the road hump is in the defined range, i.e. in the potential abnormal road surface region, then the region in the image is selected for further checks. The height of the selected region is then calculated using an road hump height calculation algorithm as described in the following:

[0052] For calculating the height of the road hump, one reference point before the road hump start and one reference point after the road hump end are considered. Moreover, three reference points in between road hump start and hump end are selected. Afterwards, 2D positions of the selected reference points are converted into 3D positions. Thereafter, the height difference between 3D position of each reference point within and outside the selected region is considered, wherein the maximum height calculated is determined to be the height of the road hump.

[0053] Moreover, the Euclidian distance from the position of the vehicle to the detected road hump, for example, the reference point in the centroid of the road hump, is calculated using the 3D positions of the vehicle and of the detected road hump. Based on the height of the hump and the distance between the car and hump, severity is assigned to the road hump. Seventy levels, distance between the vehicle and the road hump and the height of the road hump are passed to ACC (Adaptive Cruise Control). With this information, the ACC maintains or decelerates the vehicle for enhanced driver assistance functionality. Also, road hump information is also stored to a road database for improved navigation, for example, for sharing this information in a cloud-based database so that other vehicles equipped with such a road condition detection system can make use of it.

[0054] Apart from this, the road condition detection system according to the invention can be further configured such that the judging means is configured to make the judgement indicating that the surface condition of the road ahead of the ego vehicle is abnormal if the road surface condition information indicate the vertical deflection of the surface of the road, exceeding a predetermined height, to be a road hump.

[0055] A driver assistance system according to the invention is for a subject vehicle and comprises: detecting means configured to detect the surface condition of the road ahead of the ego vehicle and a traffic sign ahead of the ego vehicle by capturing front images by means of at least one sensor, for example an image sensor or a camera sensor, mounted at the front of the ego vehicle, said road condition detection system according to the invention, and controlling means configured to output control signals for controlling the at least one dynamic parameter of the ego vehicle relating to the current dynamics of the ego vehicle to a motor control of the ego vehicle based on signals received from the abnormal road condition setting means.

[0056] Accordingly, the properties and advantages explained in connection with the road condition detection system according to the invention arise in the same or similar manner in respect of the driver assistance system according to the invention, which is why, in order to avoid repetitions, reference is made to the respective explanations with respect to the road condition detection system according to the invention.

[0057] The driver assistance system according to the invention can be further modified in such a way that the detecting means comprises at least one camera or image sensor, configured to capture the front images. For example, one front camera mounted at the front of the vehicle and which captures the front images of the ego vehicle is sufficient.

[0058] A computer-implemented method according to the invention is for detecting a surface condition of a road ahead of an ego vehicle, preferably an autonomous or driver-assisted vehicle, particularly for controlling said road condition detection system according to the invention, wherein the method comprises the following steps: acquiring input signals resulting from a detection with respect to a surface condition of the road ahead of the ego vehicle and resulting from a detection of a traffic sign ahead of the ego vehicle, processing the acquired input signals and to obtain, from the processing, road surface condition information on whether the surface condition of the road ahead of the ego vehicle is abnormal or normal, and traffic sign information on whether the detected traffic sign indicates that the surface condition of the road ahead of the ego vehicle is abnormal, making a judgement, the judgement indicating that the surface condition of the road ahead of the ego vehicle is abnormal if the traffic sign information indicates that the surface condition of the road ahead of the ego vehicle is abnormal, and setting at least one control signal for changing at least one dynamic parameter of the ego vehicle, relating to the dynamics of the ego vehicle, based on the judgement indicating that that the surface condition of the road ahead of the ego vehicle is abnormal.

[0059] Accordingly, the properties and advantages explained in connection with the road condition detection system according to the invention arise in the same or similar manner in respect of the computer-implemented method according to the invention, which is why, in order to avoid repetitions, reference is made to the respective explanations with respect to the road condition detection system according to the invention.

[0060] A computer program according to the invention comprises instructions which, when the program is executed by a computer, cause the computer to carry out the computer-implemented method according to the invention.

[0061] A non-transitory computer-readable medium according to the invention has stored thereon the computer program according to the invention.

[0062] A vehicle according to the invention is preferably an autonomous or driver- assisted vehicle and comprises the driver assistance system according to the invention.

[0063] BRIEF DESCRIPTION OF THE DRAWINGS

[0064] A preferred embodiment and advantages of the invention will become more apparent upon reading the following detailed description along with the accompanying drawings in which:

[0065] FIG. 1a shows one embodiment of a vehicle according to the invention provided with a driver assistance system according to the invention in a concrete driving situation;

[0066] FIG. 1b shows a top view image obtained by conversion processing; FIG. 1c shows a front view image captured by detecting means of the vehicle according to the invention;

[0067] FIG. 2 shows the vehicle according to the invention in the concrete driving situation of FIG. 1 from a top view;

[0068] FIG. 3 shows a flowchart of a computer-implemented method for detecting a surface condition of a road ahead of the vehicle, carried out by the driver assistance system according to the invention of FIG. 1 ;

[0069] FIG. 4 shows a severity table used for controlling the speed of the vehicle of FIG. 1 ; and

[0070] FIG. 5 shows an example of the configuration of the driver assistance system according to the invention including a road condition detection system according to the invention.

[0071] DETAILED DESCRIPTION

[0072] In this context, the road condition detection system as described in this description may include a memory which is, for example, used in the processing carried out in the road condition detection system. A memory used in the embodiments may be a volatile memory, for example a DRAM (Dynamic Random Access Memory) or a non-volatile memory, for example a PROM (Programmable Read Only Memory), an EPROM (Erasable PROM), EEPROM (Electrically Erasable PROM), or a flash memory, e.g., a floating gate memory, a charge trapping memory, an MRAM (Magnetoresistive Random Access Memory) or a PCRAM (Phase Change Random Access Memory).

[0073] In this context, the road condition detection system as described in this description may include a processor or a "circuit". A “circuit” may be understood as any kind of a logic implementing entity, which may be special purpose circuitry or a processor executing software stored in a memory, firmware, or any combination thereof. Thus, in an embodiment, a "circuit" may be a hard-wired logic circuit or a programmable logic circuit such as a programmable processor, e.g. a microprocessor (e.g. a Complex Instruction Set Computer (CISC) processor or a Reduced Instruction Set Computer (RISC) processor). A "circuit" may also be a processor executing software, e.g. any kind of computer program, e.g. a computer program using a virtual machine code such as e.g. Java. Any other kind of implementation of the respective functions which will be described in more detail below may also be understood as a "circuit" in accordance with an alternative embodiment.

[0074] With reference to the drawings, one preferred embodiment of the present invention will now be specifically described for illustrative purposes. It is to be understood that components or elements in this embodiment will be shown and described by way of examples only, but are not intended to limit this invention to them.

[0075] FIG. 1a shows one embodiment of a vehicle 10 (also referred to as “ego vehicle”) according to the invention provided with a driver assistance system 1 according to the invention in a concrete driving situation. In this connection, the configuration of the vehicle 10 and the driver assistance system 1 will be described with reference to FIG. 5.

[0076] FIG. 5 shows an example of the configuration of the driver assistance systeml according to the invention including a road condition detection system 100 according to the invention, both of which are, in turn, included in the (ego) vehicle 10.

[0077] In particular, the road condition detection system 100 is configured to detect a road condition being a surface condition of a road 16 ahead of an (ego) vehicle 10 currently taking the road 16, to be described below.

[0078] In this embodiment, the vehicle 10 is a driver-assisted vehicle which comprises the driver assistance system 1 being an ADAS (Advanced Driver Assistance System). The driver assistance system 1 comprises detecting means 60 configured to detect the surface condition of the road 16 ahead of the ego vehicle 10 and a traffic sign 12 ahead of the ego vehicle 10, as shown in FIG. 1 , which will be described in more detail below, by capturing front images by means of at least one front image sensor or camera sensor mounted at the front of the ego vehicle 10.

[0079] Moreover, the driver assistance system 1 further comprises controlling means 70 configured to output control signals for controlling at one dynamic parameter of the ego vehicle relating to the current dynamics of the ego vehicle, i.e. in this case for controlling the speed of the vehicle, to a motor control of the ego vehicle 10 based on signals received from abnormal road condition setting means 50 of the road condition detection system 100, described below in more detail. As can be further gathered from FIG. 5, the road condition detection system 100 comprises signal acquisition means 20, processing means 30, judging means 40 and abnormal road condition setting means 50.

[0080] The signal acquisition means 20 is configured to acquire input signals from the detecting means 60, resulting from a detection, in this case an image detection, with respect to a surface condition of the road 16 ahead of the vehicle 10 and resulting from a detection of a traffic sign 12 ahead of the ego vehicle 10.

[0081] That is, the camera or image sensor of the detecting means 60, not shown, are configured to capture front images from the perspective of the vehicle 10, i.e. front view images, and output these front view images to the signal acquisition means 20.

[0082] The processing means 30 is configured to process the acquired input signals and to obtain, from the processing, road surface condition information on whether the surface condition of the road 16 ahead of the ego vehicle 10 is abnormal or normal, and traffic sign information on whether a detected traffic sign 12 indicates that the surface condition of the road 16 ahead of the ego vehicle 10 is abnormal. In this embodiment, the surface condition of the road 16 ahead of the ego vehicle 10 is considered abnormal if a vertical deflection of the surface of the road, i.e. a potential road hump 14, is detected, which exceeds a predetermined height to be described below.

[0083] Moreover, the judging means 40 is configured to make a judgement, wherein the judgement indicates that the surface condition of the road 16 ahead of the ego vehicle 10 is abnormal if the traffic sign information indicates that the surface condition of the road 16 ahead of the ego vehicle 10 is abnormal.

[0084] Furthermore, the abnormal road condition setting means 50 is configured to set at least one control signal for changing at least one dynamic parameter of the ego vehicle 10, relating to the dynamics of the ego vehicle 10, based on the judgement indicating that the surface condition of the road 16 ahead of the ego vehicle 10 is abnormal.

[0085] In more concrete terms, in this embodiment, the abnormal road condition setting means 50 sets the at least one control signal and outputs the same to the controlling means 70 which outputs control signals for controlling the speed of the vehicle by decelerating the vehicle 10 or maintaining the speed of the vehicle 10. The detailed operation procedure of the road condition detection system 100 according to a computer-implemented method for detecting the surface condition of the road 16 ahead of an ego vehicle 10 and further details of the road condition detection system 100 will be described below with reference to FIG. 3.

[0086] In this respect, the method may be realized by a computer program comprising instructions or a plurality of algorithms, which, when the program is executed by a computer, cause the computer to carry out the computer-implemented method. In this connection, for example, the program may be stored on a non- transitory computer-readable medium.

[0087] The method according to the invention is based on or starts from the initial situation in which the vehicle 10 is traveling on the road 16 at a certain speed, as shown in FIGS. 1a and 2, wherein FIG. 2 shows the vehicle according to the invention in the concrete driving situation of FIG. 1 from a top view.

[0088] In step S1 according to the flowchart of Fig. 3, the signal acquisition means 20 acquires input signals resulting from the detecting means 60 being the front camera mounted in the front of the vehicle. The input signals result from the detection with respect to a surface condition of the road 16 ahead of the ego vehicle 10 and resulting from a detection of a traffic sign 12 ahead of the ego vehicle 10 by capturing front images.

[0089] FIG. 1c shows a front view image captured by the front camera of the detecting means 60 as an example. As can be further gathered from Fig. 1c, a potential abnormal surface condition being a vertical deflection in the form of a potential road hump 14 is captured.

[0090] In more concrete terms, road segmentation is performed by the processing means 30 on the front view image and, in a next step, a number of stripes on the road 16 is analyzed. If the number of stripes detected is greater than two stripes detected, a further check whether the potential abnormal surface condition in the form of the road hump 14 has a required length I in the longitudinal direction of the road 16 is performed by the processing means 30, as shown in Fig. 2.

[0091] If the potential abnormal surface condition has the required length I, which means it is within a potential abnormal road surface region 19, which is a detection area for detecting / scanning surface conditions of the road, this potential abnormal road surface region 19 is selected for further checks. Thus, if this potential abnormal road surface region 19 is selected for further checks, the front view image being the image captured from the perspective of the front camera is converted in a top view image as shown in FIG. 1b by a known conversion process.

[0092] In step S2 and S3, the processing means 30 further processes the acquired input signals by placing reference points 18 on the top view image of FIG. 1 b in a centroid position of the potential abnormal road surface region 19 which should correspond to a centroid position of the potential road hump 14 and along the longitudinal direction of the road 16 inside and outside the potential abnormal road surface region 19.

[0093] In particular, in step S2, in the top view image of FIG. 1 b, the centroid on the road hump 14 is selected as a first reference point, wherein two neighboring second and third reference points on the road hump 14 within the potential abnormal road surface region 19 are further select. Thus, three first to third reference points are aligned in a longitudinal direction of the road 16 intersecting the middle of the potential road hump 14.

[0094] In step S3, further fourth and fifth reference points are selected in alignment with the first to third reference points but outside the potential abnormal road surface region 19, i.e. one fourth reference point before the road hump start (closer to the vehicle 10) and one fifth reference point after the hump end outside the potential abnormal road surface region 19 (further away from the vehicle) are selected.

[0095] Next, in step S4, the 2D coordinates of the first to fifth reference points 18 in the top view image, which is a 2D image, are converted into 3D coordinates (world coordinates) in the front view image by a known 2D to 3D conversion process as indicated in FIG. 1C showing the first to fifth reference points 18.

[0096] In step S5, each height difference is calculated between the respective fourth and fiftth reference points outside the potential abnormal road surface region 19 and the respective first to third reference points inside the potential abnormal road surface region 19. As a result, the one maximum height difference between respective two reference points outside and inside the road hump 14 is used as the representative height of the potential road hump 14.

[0097] In parallel to the aforementioned steps S1 to S5, road hump traffic sign recognition is performed in step S6. That is, a traffic sign detection is carried out as to whether a “Road hump ahead” directional traffic sign that a road hump is placed ahead of the vehicle is recognized.

[0098] Subsequently, after the processes carried out in steps S1 to S5 and S6, it is decided in step S7 whether a respective traffic sign has been detected which indicates that the surface condition of the road 16 ahead of the ego vehicle 10 is abnormal due to existence of a road hump 14.

[0099] If positive (YES in step S7), the procedure moves to step S9, and otherwise (NO in step S7), the procedure moves to step S8, where it is decided whether the calculated representative height of the road hump 14 exeeds a predetermined threshold value or not. If positive (YES in step S8), the procedure moves to step S9. Otherwise (NO in step S8), the procedure moves to step S11 .

[0100] In step S11 , it is determined that there is no road hump 14 detected in the potential abnormal road surface region 19, thus, there is no abnormal road surface condition, and the procedure ends for this detection cycle. Thus, there is no need for any speed control assistance of the ego vehicle 10 in this case.

[0101] In step S9, at first, a distance (Euclidian distance) d from the positon of the vehicle 10 to the positon of the detected road hump 14 existence of which has either been confirmed by detection of the respective traffic sign in step S7 or by the determination in step S8, i.e. to the position of the road hump 14 where its height is maximum, is calculated by a known processing using the vehicle’s 3D position and the road hump’s 3D position.

[0102] Moreover, in step S9, severity (i.e. a classification) of the detected road hump 14 is determined. In more detail, based on the detected height of the hump and the detected distance d between position of the vehicle 10 and the position of the road hump 14, a severity will be assigned to the road hump 14 in accordance with the table T shown in FIG. 4.

[0103] FIG. 4 shows a severity table used for controlling the speed of the vehicle of FIG. 1 . On closer look at FIG. 4, it can be seen that the abnormal road condition setting means 50 is configured to classify the amount of the vertical deflection of the road surface, i.e. the height of the road hump 14, in height ranges including a low height range (“Low” in FIG. 4), a moderate height range (“Moderate” in FIG. 4) and a tall height range (“Tall” in FIG. 4), which height ranges are predetermined ranges. In this connection, the moderate height range lies, of course, between the low height range and the tall height range. The low height range of the vertical deflection means a height equal or lower than 4cm, moderate height range of the vertical deflection means a height between 4cm and 8cm, and tall height range of the vertical deflection means a height equal or greater than 8cm.

[0104] Moreover, the abnormal road condition setting means 50 is configured to classify the distance d between the position of the ego vehicle 10 and the position of the vertical deflection of the road surface, i.e. the road hump 14, in distance ranges including a short distance range (“Short” in FIG. 4), mid distance range (“Mid” in FIG. 4) and a long distance range (“Long” in FIG. 4). In the same way, the mid distance range is, of course, between the short distance range and the long distance range. The short distance range means a range equal or less than 20m, mid distance range means a range between 20m and 30m, and a long distance range means a range equal or greater than 40m.

[0105] As can be further gathered from the severity table of FIG. 4, each respective combination of a respective height range of the hump (“Tall”, ’’Moderate”, ’’Low”) with a respective distance (“Long”, ’’Mid”, “Short”) is assigned a respective Severity level (Level 1 , 2 or 3) which, in turn, is related with a respect control action (referred to as “Action Taken” in FIG. 4).

[0106] That is, based on the aforementioned combinations, the abnormal road condition setting means 50 is configured to set the control signal so that the dynamic parameter of the ego vehicle, which is the change or maintainace of the vehicle speed in this case, effects that the ego vehicle 10 is

[0107] - linearly decelerated if the height of the vertical deflection of the road surface is judged to be in the tall height range and the distance between the position of the ego vehicle 10 and the position of the vertical deflection of the road surface is judged to be in the long distance range, or the height of the vertical deflection of the road surface is judged to be in the tall height range and the distance between the position of the ego vehicle 10 and the position of the vertical deflection of the road surface is judged to be in the mid distance range, or the height of the vertical deflection of the road surface is judged to be in the moderate height range and the distance between the position of the ego vehicle 10 and the position of the vertical deflection of the road surface is judged to be in the mid distance range, or the height of the vertical deflection of the road surface is judged to be in the low height range and the distance between the position of the ego vehicle 10 and the position of the vertical deflection of the road surface is judged to be short distance range, or

[0108] - immediately decelerated if the height of the vertical deflection of the road surface is judged to be in the tall height range and the distance between the position of the ego vehicle 10 and the position of the vertical deflection of the road surface is judged to be in the short distance range, or the height of the vertical deflection of the road surface is judged to be in moderate height range and the distance between the position of the ego vehicle 10 and the position of the vertical deflection of the road surface is judged to be short distance range, or

[0109] - slightly decelerated if the height of the vertical deflection of the road surface is judged to be in the moderate height range and the distance between the position of the ego vehicle 10 and the position of the vertical deflection of the road surface is judged to be in the long distance range, or the height of the vertical deflection of the road surface is judged to be in the low height range and the distance between the position of the ego vehicle 10 and the vertical deflection of the road surface is judged to be in the mid distance range, or

[0110] - not decelerated if the height of the vertical deflection of the road surface is judged to be low height range and the distance between the position of the ego vehicle 10 and the vertical deflection of the road surface is judged to be in the long distance range.

[0111] In this connection, a linear deceleration of the vehicle means to decelerate or to deaccelerate slowly to a vehicle speed of less than 3.6 - 6 kmph (km / h). A slight deceleration means to decelerate or deaccelerate moderately to a vehicle speed of 4 - 10 kmph. An immediate deceleration means to decelerate or deaccelerate directly to a vehicle speed of less than 10-16 kmph. That is, after the control signal is set according to the table of FIG. 4, the control signal is output to the controlling means 70 which outputs control signals for controlling the speed of the vehicle by decelerating the vehicle 10 or not decelerating the vehicle 10 in accordance with the table of FIG. 4. For example, the control signals according to the severity levels as outlined in

[0112] FIG. 4 are passed to an ACC, i.e. an Adaptive Cruise Control, of the vehicle. With this information, the ACC will accelerate or deaccelerate the vehicle for enhanced driver assistance functionality.

[0113] Moreover, the road hump information is also stored to a road database for improved navigation of with an GPS navigation system, for example.

[0114] The procedure is then ended for this detection cycle and is started again from the beginning.

[0115] The features of the invention disclosed in the foregoing description, in the drawings as well as in the claims may be essential for the realization of the invention both individually and in any combination.

[0116] LIST OF REFERENCE SIGNS

[0117] 1 driver assistance system

[0118] 10 vehicle (ego vehicle)

[0119] 12 traffic sign indicating that the surface condition of the road ahead of the ego vehicle is abnormal (road hump)

[0120] 14 abnormal surface condition (road hump)

[0121] 16 road

[0122] 18 first to fifth reference points

[0123] 19 potential abnormal road surface region

[0124] 20 signal acquisition means

[0125] 30 processing means

[0126] 40 judging means

[0127] 50 abnormal road condition setting means

[0128] 60 detecting means

[0129] 70 controlling means

[0130] 100 road condition detection system

[0131] T severity table

[0132] I length d distance

Claims

Claims1 . A road condition detection system (100) for detecting a surface condition of a road (16) ahead of an ego vehicle (10), preferably an autonomous or driver- assisted vehicle, currently taking the road (16), wherein the road condition detection system (100) comprises: signal acquisition means (20) configured to acquire input signals resulting from a detection with respect to a surface condition of the road (16) ahead of the ego vehicle (10) and resulting from a detection of a traffic sign (12) ahead of the ego vehicle (10), processing means (30) configured to process the acquired input signals and to obtain, from the processing, road surface condition information on whether the surface condition of the road (16) ahead of the ego vehicle (10) is abnormal or normal, and traffic sign information on whether the detected traffic sign (12) indicates that the surface condition of the road (16) ahead of the ego vehicle (10) is abnormal, judging means (40) configured to make a judgement indicating that the surface condition of the road (16) ahead of the ego vehicle (10) is abnormal if the traffic sign information indicates that the surface condition of the road (16) ahead of the ego vehicle (10) is abnormal, and abnormal road condition setting means (50) configured to set at least one control signal for changing at least one dynamic parameter of the ego vehicle (10), relating to the dynamics of the ego vehicle (10), based on the judgement indicating that that the surface condition of the road (16) ahead of the ego vehicle (10) is abnormal.

2. The road condition detection system (100) according to claim 1 , wherein the judging means (40) is configured to make the judgment indicating that the surface condition of the road ahead of the ego vehicle is abnormal if the traffic sign information indicates that the surface condition of the road (16) ahead of the ego vehicle (10) is abnormal, irrespective whether the road surface condition information indicates a vertical deflection of the surface of the road exceeding or being below a predetermined vertical height value.

3. The road condition detection system (100) according to claim 1 or 2, wherein the abnormal road condition setting means (50) is configured to set the control signal to change the dynamic parameter of the ego vehicle (10) such that the speed of the ego vehicle changes, setting of the control signal being dependent on an amount of vertical deflection of a road surface and / or the distance between the position of the ego vehicle (10) and position of the vertical deflection of the road surface.

4. The road condition detection system (100) according to claim 3, wherein the abnormal road condition setting means (50) is configured to classify the amount of the vertical deflection of the road surface in height ranges including a low height range, a moderate height range and a tall height range, the moderate height range lying between the low height range and the tall height range, classify the distance between the position of the ego vehicle (10) and the position of the vertical deflection of the road surface in distance ranges including a short distance range, mid distance range and a long distance range, the mid distance range lying between the short distance range and the long distance range, and set the control signal so that the dynamic parameter of the ego vehicle effects that the ego vehicle (10) is- linearly decelerated if the height of the vertical deflection of the road surface is judged to be in the tall height range and the distance between the position of the ego vehicle (10) and the position of the vertical deflection of the road surface is judged to be in the long distance range, and / or the height of the vertical deflection of the road surface is judged to be in the tall height range and the distance between the position of the ego vehicle (10) and the position of the vertical deflection of the road surface is judged to be in the mid distance range, and / or the height of the vertical deflection of the road surface is judged to be in the moderate height range and the distance between the position of the ego vehicle (10) and the position of the vertical deflection of the road surface is judged to be in the mid distance range, and / orthe height of the vertical deflection of the road surface is judged to be in the low height range and the distance between the position of the ego vehicle (10) and the position of the vertical deflection of the road surface is judged to be short distance range, and / or- immediately decelerated if the height of the vertical deflection of the road surface is judged to be in the tall height range and the distance between the position of the ego vehicle (10) and the position of the vertical deflection of the road surface is judged to be in the short distance range, and / or the height of the vertical deflection of the road surface is judged to be in moderate height range and the distance between the position of the ego vehicle (10) and the position of the vertical deflection of the road surface is judged to be short distance range, and / or- slightly decelerated if the height of the vertical deflection of the road surface is judged to be in the moderate height range and the distance between the position of the ego vehicle (10) and the position of the vertical deflection of the road surface is judged to be in the long distance range, and / or the height of the vertical deflection of the road surface is judged to be in the low height range and the distance between the position of the ego vehicle (10) and the vertical deflection of the road surface is judged to be in the mid distance range, and / or- not decelerated if the height of the vertical deflection of the road surface is judged to be low height range and the distance between the position of the ego vehicle (10) and the vertical deflection of the road surface is judged to be in the long distance range.

5. The road condition detection system (100) according to any of the preceding claims, wherein the processing means (30) is configured to obtain, from the processing, the road surface condition information to indicate that the surface condition of the road (16) is abnormal if the processing of the input signals resulting from the detection with respect to a surface condition of the road (16) ahead of theego vehicle (10) results in the vertical deflection of the surface of the road exceeding a predetermined height.

6. The road condition detection system (100) according to any of the preceding claims, wherein the processing means (30) is configured to process the acquired input signals by performing a road segmentation on a detected top view image resulting from a detection with respect to a surface condition of the road (16) ahead of the ego vehicle (10) and analyzing a number of stripes on the road surface and / or length (I) of a potential abnormal road surface region (19) in the longitudinal direction of the road (16), in which region the surface condition of the road may be abnormal.

7. The road condition detection system (100) according to claim 6, wherein the processing means (30) is configured to process the acquired input signals and proceed to further processing if the number of stripes exceeds a predetermined number and / or the length of the potential abnormal road surface region (19) exceeds a predetermined length.

8. The road condition detection system (100) according to any of the preceding claims 6 or 7, wherein the processing means (30) is configured to further process the acquired input signals by- setting the potential abnormal road surface region in the top view image, the potential abnormal road surface region being located in a predetermined minimum distance ahead of the ego vehicle,- selecting a centroid in the potential abnormal road surface region as a first reference point in the top view image,- setting second and third reference points neighboring the first reference point in the top view image so that the first reference point is located between the second and third reference points, the second and third reference points being located near the boundary of the potential abnormal road surface region in the longitudinal direction of the road,- setting fourth and fifth reference points before and after the potential abnormal road surface region in the longitudinal direction of the road in the top view image,- converting 2D positions of the first to fifth reference points in the in the top view image to 3D positions in a perspective front view image from which the top view image is obtained by image conversion processing,- detecting a height difference between the road surface outside and inside the potential abnormal road surface region by means of a calculation using the 3D positons of the respective first to fifth reference points.

9. The road condition detection system (100) according to any of the preceding claims, wherein the judging means (40) is configured to make the judgement indicating that the surface condition of the road (16) ahead of the ego vehicle (10) is abnormal if the road surface condition information indicate the vertical deflection of the surface of the road (16), exceeding a predetermined height, to be a road hump.

10. A driver assistance system (1 ) for a subject vehicle (50), comprising: detecting means (60) configured to detect the surface condition of the road(16) ahead of the ego vehicle (10) and a traffic sign (12) ahead of the ego vehicle (10) by capturing front images by means of at least one sensor mounted at the front of the ego vehicle (10), said road condition detection system (100) according to any of claims 1 to 9, and controlling means (70) configured to output control signals for controlling the at least one dynamic parameter of the ego vehicle relating to the current dynamics of the ego vehicle to a motor control of the ego vehicle (10) based on signals received from the abnormal road condition setting means (50).11 . The driver assistance system (1) according to claim 10, wherein the detecting means comprises at least one camera or image sensor configured to capture the front images.

12. A computer-implemented method for detecting a surface condition of a road (16) ahead of an ego vehicle (10), preferably an autonomous or driver-assisted vehicle, particularly for controlling a road condition detection system according to any one of claims 1 to 10, wherein the method comprises the following steps:acquiring input signals resulting from a detection with respect to a surface condition of the road (16) ahead of the ego vehicle (10) and resulting from a detection of a traffic sign (12) ahead of the ego vehicle (10), processing the acquired input signals and to obtain, from the processing, road surface condition information on whether the surface condition of the road (16) ahead of the ego vehicle (10) is abnormal or normal, and traffic sign information on whether the detected traffic sign (12) indicates that the surface condition of the road (16) ahead of the ego vehicle (10) is abnormal, making a judgement, the judgement indicating that the surface condition of the road (16) ahead of the ego vehicle (10) is abnormal if the traffic sign information indicates that the surface condition of the road (16) ahead of the ego vehicle (10) is abnormal, and setting at least one control signal for changing at least one dynamic parameter of the ego vehicle (10), relating to the dynamics of the ego vehicle (10), based on the judgement indicating that that the surface condition of the road (16) ahead of the ego vehicle (10) is abnormal.

13. A computer program comprising instructions which, when the program is executed by a computer, cause the computer to carry out the computer-implemented method of claim 12.

14. A non-transitory computer-readable medium having stored thereon the computer program of claim 13.

15. Vehicle (10), preferably an autonomous or driver-assisted vehicle, comprising the driver assistance system (1) according to claim 10 or 11 .

Images