METHOD AND SYSTEM FOR ACTIVE PREDICTION OF ROAD SURFACE AND FOR PROACTIVE ACTIVATION OF THE VEHICLE STABILITY SYSTEM

Abstract

A system (100) for actively predicting road surfaces and proactively activating a vehicle stability system comprises receiving sensor data from multiple sensors (104, 106, 108, 110), a controller (112) configured to determine a weighting matrix (WT) based on driving conditions, and determining the presence of a low-friction surface using a decision matrix (DN), where the decision matrix (DN) is a product of the weighting matrix (WT) and a vector (X) representing the output data from each of the sensors. The system (100) further comprises an electronic stability program (ESP) (114) that is activated based on the detected low-friction surface. ESP activation includes calculating trajectory data, providing warnings to the driver, and activating vehicle control systems (116) if necessary.The system enables improved driving performance and greater passenger comfort through early detection and reaction to low-friction surfaces.

Description

AREA OF INVENTION

[0001] The present invention relates generally to vehicle safety systems and in particular to a proactive system for risk assessment and reduction for vehicle stability under adverse weather conditions. BACKGROUND

[0002] Vehicle stability is a crucial aspect of road safety, especially in adverse weather conditions. When a vehicle encounters a wet or snowy road surface, the risk of aquaplaning increases significantly. Aquaplaning occurs when a layer of water or snow accumulates between the vehicle's tires and the road surface, leading to a loss of traction and control. This phenomenon can cause accidents, injuries, and fatalities, and is therefore a major concern for both drivers and vehicle manufacturers.

[0003] Current advanced driver assistance systems (ADAS) and autonomous driving (AD) systems typically rely on reactive measures to resolve stability issues in adverse weather conditions. These systems often only activate stability protocols when the vehicle encounters hazardous situations, such as on wet or snowy roads. This reactive approach can lead to sudden steering or braking adjustments, which can be inconvenient or even dangerous for vehicle occupants. Furthermore, the sudden activation of these protocols may not give the vehicle sufficient time to effectively adapt to the changing road conditions.

[0004] Existing solutions for dealing with wet road conditions and the risk of aquaplaning have their limitations. Some systems use vehicle-to-everything (V2X) communication to inform target vehicles about road conditions based on previously recorded data. While this approach can provide some degree of early warning, it may not account for rapidly changing weather conditions or local variations in road wetness. Other solutions use sensors near the vehicle to detect water accumulation on the road and provide real-time feedback to the electronic stability program (ESP). However, these systems still rely on reactive measures once the vehicle encounters wet conditions.

[0005] The limitations of current solutions highlight the need for a more proactive approach to risk assessment and mitigation for vehicle stability in adverse weather conditions. Reactively activating evasive mechanisms to stabilize the vehicle can lead to discomfort or potentially dangerous situations for the occupants. Furthermore, relying on historical data or localized sensor information may prevent a comprehensive understanding of upcoming road conditions.

[0006] To meet these challenges, there is a growing need for innovative solutions that can proactively assess and mitigate the risks posed by wet roads, snow, and other adverse weather conditions. Such solutions should be able to integrate multiple data sources, including real-time sensor information, weather forecasts, and historical data, to enable a more accurate and timely assessment of road conditions. Furthermore, these systems should be able to proactively adjust vehicle control strategies to ensure a smoother and safer transition through challenging road conditions.

[0007] Patent documents such as EP3456597B1 and US20180060674A1 have attempted to address these issues. EP3456597B1 discloses a system for detecting weather conditions using onboard sensors. US2022227328A1 discloses a method for determining wetness on a road surface using an image acquisition device. Several wet surface detection methods are applied to the image in real time, with each method independently analyzing whether the road surface is wet. The results of each method are fed into a fusion and decision-making module, where they are weighted based on weather information, geological information, and vehicle movements. However, these solutions do not fully address the issues of proactive risk assessment and mitigation for vehicle stability in rain, wet road conditions, snow, and other adverse weather conditions.

[0008] Therefore, the aforementioned problems must be addressed through the development of advanced systems and methods for proactive risk assessment and mitigation in ADAS and AD systems. These solutions should focus on improving vehicle stability in rain, wet road conditions, snow, and other adverse weather conditions. This will enhance overall vehicle safety, reduce the likelihood of aquaplaning accidents, and provide a more comfortable and safer driving experience for vehicle occupants. TASK OF INVENTION

[0009] The primary object of the present invention is to provide a method and a system for actively predicting road surfaces and for proactively controlling a vehicle stability system in order to improve driving performance and passenger comfort.

[0010] Another object of the present invention is the use of multiple sensors for detecting low-friction surfaces on the roadway in front of the vehicle, thereby enabling a timely response to potential hazards.

[0011] Another object of the present invention is to implement a dynamic weighting system for sensor data that adapts to different driving conditions and environmental factors in order to improve the accuracy of road surface prediction.

[0012] Another object of the present invention is to provide a multi-stage driver warning system that effectively warns the driver of possible low-friction surfaces and gives him enough time to react and adjust his driving behavior. SUMMARY

[0013] According to one aspect of the present invention, a method for actively predicting road surfaces and proactively controlling a vehicle stability system is provided. The method comprises receiving sensor output data from a plurality of sensors, including an image sensor, a LiDAR sensor, a radar sensor, and an infrared sensor, configured to detect road surface conditions. The method further comprises determining a weighting matrix that includes weights for each of the sensor output data based on the current driving state of the vehicle. Subsequently, the presence of a low-friction surface on the road is determined based on a decision matrix, wherein the decision matrix is ​​a product of the weighting matrix and a vector representing the output data from each of the sensors.The procedure then involves activating the vehicle's electronic stability program (ESP) based on the determined low-friction surface on the road. The driving conditions include at least one of the following situations: urban environment, highway driving, adverse weather conditions, and night driving.

[0014] The weighting matrix has a size of 1x4 and comprises the weights W1, W2, W3, and W4, corresponding to the image, LiDAR, RADAR, and infrared sensors, respectively. The sum of these weights W1, W2, W3, and W4 is equal to 1. The weight for each sensor is calculated using a modified Bayesian method based on a base value of the corresponding sensor, an environmental factor function, corresponding sensor values ​​for low friction and normal surfaces, and a corresponding sensor correction factor.

[0015] The method further includes determining the presence of a low-friction surface based on the decision matrix by calculating a low-friction index (LFI) using the weighting matrix (WT) and a vector (X), where X is the 4x1 vector containing the individual sensor output data [C, L, R, I] corresponding to the image, LiDAR, RADAR, and infrared sensors, respectively. The presence of a low-friction surface is determined if the LFI is greater than or equal to a predefined threshold.Furthermore, the individual sensor output data [C, L, R, I] are determined as follows: C is 1 if the standard deviation of the pixel intensity is greater than or equal to a pixel intensity threshold, otherwise C is equal to 0; L is 1 if the LiDAR reflection intensity is greater than or equal to a reflection intensity threshold, otherwise L is equal to 0; R is 1 if the backscatter is greater than or equal to a backscatter coefficient threshold, otherwise R is equal to 0; and I is 1 if the emissivity is greater than or equal to an emissivity threshold, otherwise I is equal to 0.

[0016] The procedure further includes specific steps for controlling the ESP based on the detected low-friction road surface. These steps include calculating the distance, time, location, and vehicle trajectory data to the low-friction surface; providing the calculated data to the ESP; triggering a Level 1 warning to the driver (including audible, visual, and haptic warnings) indicating the presence of the low-friction surface; triggering a Level 2 warning, which includes partial braking, if no feedback is received from the driver; and activating one or more vehicle control systems, including the anti-lock braking system (ABS), traction control, engine braking, the braking system, or stability control, when the vehicle enters the detected low-friction surface.

[0017] According to a further aspect of the present invention, a system for actively predicting road surfaces and proactively controlling a vehicle stability system is provided. The system comprises a plurality of sensors, including a camera, a LiDAR sensor, a radar sensor, and an infrared sensor, which are mounted on the vehicle and configured to detect road surface conditions. The system also includes a control unit configured to determine a weighting matrix based on the current driving state of the vehicle, which includes weights for the output data of each of the sensors; to determine the presence of a low-friction surface on the road based on a decision matrix; and to activate an electronic stability program (ESP) of the vehicle based on the determined low-friction surface on the road.The driving conditions considered by the system include at least one of the following situations: urban environment, motorway driving, bad weather conditions and night driving.

[0018] The weighting matrix is ​​1x4 in size and includes the weights W1, W2, W3 and W4, each of which the camera, LiDAR, radar and infrared sensors The weights correspond to the values ​​where the sum of these weights equals 1. The weighting matrix is ​​calculated using a modified Bayesian method based on a base value of the corresponding sensor, an environmental factor function, corresponding sensor measurements for low friction and normal surfaces, and a corresponding sensor correction factor.

[0019] The controller in the system is further configured to calculate a Low Friction Index (LFI) using the weighting matrix (WT) and a vector (X), where X is a 4x1 vector containing the individual sensor output data [C, L, R, I] corresponding to the camera, lidar, radar, and infrared sensors, respectively. The controller confirms the presence of the low-friction surface when the LFI is greater than or equal to a predefined threshold. The individual sensor output data [C, L, R, I] are determined according to the same criteria as described in the method aspect of the invention. The control unit is further configured to control the ESP based on the detected low-friction surface on the road by performing the same steps described in the method aspect of the invention.

[0020] The present invention offers significant advantages for vehicle safety and handling. By actively predicting road conditions and proactively activating the vehicle stability system, the invention improves the vehicle's ability to react to potential hazards, thereby enhancing overall safety and occupant comfort. The multi-sensor approach, in conjunction with the dynamic weighting system, enables more accurate and reliable detection of low-friction surfaces under various driving conditions.

[0021] The preceding sections serve as a general introduction and are not intended to limit the scope of the following claims. The described embodiments and further advantages are best understood by referring to the following detailed description in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS Fig.1: Exemplary block diagram of a system for the active prediction of road surfaces and for the proactive control of a vehicle stability system. Fig. 2: Exemplary flowchart of a method for actively predicting road surfaces and proactively controlling a vehicle stability system Fig. 3: An exemplary flowchart illustrating the activation process of the ESP based on the determined low-friction road surface. DETAILED DESCRIPTION OF THE INVENTION

[0022] Aspects of the present invention are best understood by reference to the description contained herein. All aspects described herein will be better appreciated and understood when considered in conjunction with the following descriptions. However, it should be understood that the following descriptions, although they indicate preferred aspects and numerous specific details thereof, serve only for illustration and should not be considered as limitations. Changes and modifications may be made within the scope of the present description without departing from the spirit and scope of the description, and the present invention includes all such modifications.

[0023] The present invention relates to a system and method for actively predicting road surfaces and proactively controlling a vehicle stability system. The system comprises a multitude of sensors for accurately predicting road conditions and reacting to potential hazards. Data from various sensors are considered based on their reliability and relevance under different driving conditions. If a low-friction surface, such as a wet or snowy surface, is detected, the vehicle's electronic stability program (ESP) is activated. This proactive approach to vehicle stability control significantly improves safety and handling in challenging road conditions.

[0024] Fig.Figure 1 shows an example block diagram of a system 100 for the active prediction of road surfaces and the proactive control of a vehicle stability system. The system 100, integrated into a vehicle 102, comprises a variety of sensors, including an image sensor 104, a LiDAR sensor 106, a radar sensor 108, an infrared sensor 110, as well as a control unit 112, an ESP 114, and one or more vehicle control systems 116.

[0025] The image sensor (104), the LiDAR sensor (106), the radar sensor (108), and the infrared sensor (110) are a set of sensors strategically positioned on the vehicle to accurately detect the condition of the road surface. Together, these sensors collect data on the road conditions and transmit this data to the control unit 112 for further processing.

[0026] The image sensor 104 is specifically designed to capture visual data of the road surface. It provides a real-time image that can be analyzed to detect potential hazards or changes in the road surface, such as water, ice, or gravel. The LiDAR sensor 106 uses laser technology to create a detailed 3D map of the vehicle's surroundings. It can accurately measure the distance between the vehicle and obstacles, thus helping to predict road conditions. For example, the LiDAR sensor 106 could detect a low-friction area on the road in advance, allowing the system 100 to proactively activate the vehicle's stability control system. The radar sensor 108 and the infrared sensor 110 work together with the image sensor 104 and the LiDAR sensor 106 to provide a comprehensive picture of road conditions.The RADAR sensor 108 uses radio waves to detect the presence of objects or changes in the road surface, even in poor visibility conditions. The infrared sensor 110, on the other hand, measures the heat radiation emitted by the objects or the road surface, which can be particularly useful for detecting the presence of heat-emitting objects or temperature changes in the road surface.

[0027] The control unit 112 acts as the central processing unit of the system. The control unit 112 receives the output data from the sensors: image sensor 104, LiDAR sensor 106, radar sensor 108, and infrared sensor 110. Based on the current driving state of the vehicle, the control unit 112 determines a weighting matrix for the output data of each sensor. The control unit 112 then uses this weighting matrix and the sensor output data to determine the presence of a low-friction surface on the road.

[0028] The ESP 114 is activated by the control unit 112 based on the detected low-friction surface on the road. The control unit 112 is configured to calculate the distance, time, location, and vehicle trajectory data to the low-friction surface. The calculated data is then transmitted to the ESP 114. The control unit 112 is further configured to issue a Level 1 warning to the driver, including audible, visual, and haptic warnings indicating the presence of the low-friction surface. In one embodiment, the audible warning is issued via the vehicle's audio system, the visual warning is displayed on the instrument cluster or head-up display, and the haptic warning is triggered by vibrations in the steering wheel or driver's seat. The multimodal warnings ensure that the driver is effectively alerted to the potential hazard ahead.

[0029] Furthermore, the control unit 112 is configured to trigger a Stage 2 warning in response to the Stage 1 warning to the driver if no feedback is received from the driver. The Stage 2 warning involves partial braking by one or more of the vehicle control systems 116. Driver feedback can include any physical interaction with the vehicle's controls, such as steering input or pressure on the brake pedal, indicating that the driver acknowledges and is responding to the warning. The vehicle control systems 116 are configured to activate various braking mechanisms to safely stop the vehicle on low-friction surfaces. These braking mechanisms include activating the brake booster to increase the hydraulic pressure in the brake lines, thereby increasing the overall braking force.Simultaneously, engine braking can be activated by downshifting or engaging a lower gear to reduce vehicle speed without relying solely on the wheel brakes. The braking system can also modulate the pressure of individual wheel brakes to maintain stability and prevent skidding when the vehicle comes to a stop on a slippery surface.

[0030] When the vehicle 102 enters the low-friction surface detected by the ESP 114, the ESP 114 also activates one or more vehicle control systems 116. The one or more vehicle control systems 116 may include an anti-lock braking system, traction control, engine braking, a braking system, or stability control, which help to mitigate the effects of the low-friction surface on the vehicle's stability and handling.

[0031] In one embodiment, the control unit 112 comprises one or more processors, memory and storage units, and communication modules. The processors may include one or more central processing units (CPUs), graphics processing units (GPUs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or a combination thereof. The processors may execute machine-executable programs stored in the memory and storage units to perform the procedures or instructions described herein.The processors can refer, for example, to one or more GPUs and CPUs configured to perform operations such as receiving and processing sensor output data from a variety of sensors, determining a weight matrix (WT), determining the presence of a low-friction surface on the road based on a decision matrix (DN), and actuating the vehicle's ESP. The memory and storage units can include volatile and non-volatile memory or storage. Examples of memory and storage units include flash memory or storage such as one or more solid-state drives, dynamic random-access memory (DRAM), or synchronous dynamic random-access memory (SDRAM) such as LPDDR-SDRAM (low-power double-data-rate), and embedded multimedia controllers (eMMC).The memory and storage units can store software, firmware, data (including image data), databases, or a combination thereof. The communication module in the vehicle connects various components via in-vehicle communication protocols such as CAN, LIN, FlexRay, or Ethernet to enable seamless data exchange and system integration. The communication module allows the control unit 112 to communicate with the sensors, the ESP 114, and the vehicle control systems 116 via a network.

[0032] In one embodiment, an image sensor module can be used that receives image sensor data and, in addition, employs advanced machine learning algorithms to analyze visual data and detect low-friction areas on the road surface. The image sensor module has a range of over 100 meters, enabling early detection and proactive response. The module processes reflection and refraction patterns of the road surface and identifies changes that may indicate the presence of water, ice, or other low-friction materials. The module is also configured to analyze the surface texture and gloss, which can provide valuable information about the road's condition.Furthermore, the module monitors changes in pixel intensity across successive images, enabling the detection of subtle variations in road surface properties that may indicate low-friction areas. This multi-layered approach allows the image sensor module to accurately identify and classify potential hazards, contributing to the overall safety and stability of the vehicle. Additionally, the image sensor module can be configured to calculate the standard deviation of pixel intensity (σ_pixel) across captured images. This standard deviation is then compared to a predefined pixel intensity threshold (Th_σ) to detect significant variations in the road surface texture.If σ_pixel exceeds the threshold Th_σ, this indicates a potential change in surface texture, requiring further analysis and potentially contributing to the detection of low-friction areas. In one embodiment, the image sensor module can be implemented in the control unit 112.

[0033] In one embodiment, a RADAR sensor module can be configured to use advanced machine learning algorithms to analyze RADAR sensor data and detect low-friction areas on the road surface. With a range of more than 300 meters, the RADAR Sensor 108 enables long-range detection and early response to potential hazards. The RADAR sensor module processes reflections from the road surface and identifies changes in signal strength and phase that may indicate water, ice, or other low-friction materials. It also analyzes variations in the road surface's dielectric constant, which can provide important information about the surface's condition.Furthermore, the module monitors the backscatter coefficients (B_R) across successive radar sweeps, which can reveal subtle changes in road surface properties that indicate low-friction areas. The backscatter coefficient is compared to a predefined threshold (Th_R) to detect significant changes in surface conditions. If B_R exceeds the Th_R threshold, this indicates a potential change in surface properties, triggering further analysis and contributing to the identification of low-friction areas. Thanks to this comprehensive approach, the radar sensor module can accurately detect and classify potential hazards even in poor visibility conditions, thus enhancing the overall safety and stability of the vehicle. In one embodiment, the radar sensor module can be implemented in the control unit 112.

[0034] In one embodiment, the LiDAR sensor module and the infrared sensor module work together to enable comprehensive short-range detection of low-friction areas on the road surface. With a range of up to 50 meters, the sensors 106, 110 provide detailed near-range analysis. A LiDAR module uses advanced machine learning algorithms to process point cloud data and analyze the absorption and attenuation properties of the laser pulses reflected from the road surface. The LiDAR module can be configured to measure the reflection intensity (R_LiDAR) and compare it to a predefined reflection intensity threshold (Th_LiDAR). If R_LiDAR exceeds the Th_LiDAR threshold, this indicates a possible change in the surface properties. An infrared sensor module, in turn, analyzes the thermal emission properties of the road surface.The infrared sensor module calculates the surface's emissivity (E_IR) and compares it to a predefined emissivity threshold (Th_IR). If E_IR exceeds the Th_IR threshold, this indicates a possible change in surface temperature or composition, which could suggest low friction. The LiDAR and infrared sensor modules also analyze the time delay of the returned signals, which can provide information about the surface structure and composition. In one embodiment, the LiDAR and infrared sensor modules can be implemented in the control unit 112.

[0035] In Fig. Figure 2 shows an exemplary flowchart of a method 200 for the active prediction of road surfaces and the proactive control of a vehicle stability system. The method 200 is configured so that it is carried out by the in Fig.The system 100 shown in Figure 1 is implemented. The procedure begins with step 202, in which sensor output data is received from a variety of sensors, including an image sensor 104, a LiDAR sensor 106, a radar sensor 108, and an infrared sensor 110. The sensors are configured to detect the condition of the road surface and provide valuable information about the road on which the vehicle 102 is traveling.

[0036] In step 204, a weighting matrix (WT) is determined. The weighting matrix (WT) includes weights for each of the sensor output data and is based on the current driving state of the vehicle 102. The weighting matrix (WT) is 1x4 and includes the weights W1, W2, W3, W4, each of which the image, LiDAR, RADAR and infrared sensors The weighting matrix (WT) is given as: WT=[W1 W2 W3 W4] where, W1+W2+W3+W4=1.

[0037] The weighting matrix is ​​calculated using a modified Bayesian method, which is expressed as follows: WT(n)=Wf*Wibase*F_env(light,time)*exp(−(Si−ui)2 / (2*F2)) where n is an integer in the range of 1 to 4; WT is the sensor weighting based on driving conditions; Wibase is the sensor's base value; F_env(light, time) is an environmental factor function; Si is the sensor measurement value for a surface with low friction; ui is the sensor measurement value for a normal surface; and F is a sensor correction factor.

[0038] In the case of the image sensor 104, the weight W1 is calculated for an urban environment, for example, based on the following example values: Wibase = 0.8 (manually set threshold) F_env(light, time) = 1 Illumination factor (output from sensor) Si = 0.6 additional road measurement value (output from sensor) Ui = 0.5 current value of the dry road surface (sensor output) F = 0.2 sensor correction factor (manually set threshold) Wf = 0.7 (manually set threshold) WT(1)=0.7*0.8*1*exp(−(0.6−0.5)2) / (2*1.22)=∼0.5

[0039] The parameters in the weight matrix calculation play a crucial role in determining the sensor's significance under different conditions. Wibase, for example, is the initial significance assigned to each sensor. F_env(light, time) considers environmental factors such as lighting and time of day. Si and ui represent the sensor values ​​for low-friction and normal surfaces, respectively, enabling the system to distinguish between these conditions. F serves as a correction factor for fine-tuning the sensor output based on known distortions or limitations. Together, these parameters ensure a dynamic and context-aware weighting system that adapts to changing driving scenarios and environmental conditions.

[0040] In one embodiment, Table I shows exemplary weight values ​​for different sensors under different driving conditions. Table I Driving conditions Image (W1) LiDAR(W2) RADAR(W3) Infrared (W4) Urban Environment 0.5 0.4 0 0.1 Driving on the motorway 0.5 0.4 0 0.1 Bad weather 0.1 0.1 0.4 0.4 dim light 0.1 0.5 0.2 0.2

[0041] As shown in the example table, the image sensor 104 is given greater weight in urban environments and during highway driving because it can clearly detect visual cues in well-lit, structured environments. However, its weight decreases in poor weather and at night when visual clarity is compromised. Conversely, the radar sensor 108 is given more importance in poor weather and at night because it is able to penetrate atmospheric obstacles and operate effectively regardless of ambient brightness, making it more reliable in these challenging environments.

[0042] In step 206, the presence of a low-friction surface on the road is determined based on a decision matrix (DN). The decision matrix is ​​a product of the weight matrix and a vector X representing the output data from each of the sensors. The decision matrix (DN) is expressed as: LFI=WT*X where, LFI = Low Friction Index (LFI), WT=[W1 W2 W3 W4], and X=[C,L,R,I]. X is the 4x1 vector containing the individual sensor output data [C, L, R, I] corresponding to the image, LiDAR, RADAR, and infrared sensors, respectively. Furthermore, the presence of the low-friction surface is determined if a Low Friction Index (LFI), calculated using the formula LFI = WT * X, is greater than or equal to a predefined threshold, expressed as: If WRI >= Th_WRI, then the presence of low µ is high. If WRI <Th_WRI, dann ist das Vorhandensein von niedrigem µ gering.

[0043] Furthermore, the individual sensor output data [C, L, R, I] are determined as follows: C = 1 if σ_pixel (standard deviation of pixel intensity) >= Th_σ (pixel intensity threshold), otherwise C = 0; L = 1 if R_LiDAR (reflection intensity) >= Th_LiDAR (reflection intensity threshold), otherwise L = 0; R = 1 if B_R (backscattering) >= Th_R (threshold for the backscattering coefficient), otherwise R = 0; and I = 1 if E_IR (emissivity) >= Th_IR (threshold for emissivity), otherwise I = 0.

[0044] The individual sensor output data [C, L, R, I] are determined by specific threshold values ​​for each sensor type. For the image sensor 104, C is set to 1 when the standard deviation of the pixel intensity (σ_pixel) exceeds a predefined pixel intensity threshold (Th_σ), indicating a significant deviation in the road surface texture. The LiDAR sensor 106, L, is set to 1 when the reflectance intensity (R_LiDAR) is above a predefined threshold (Th_LiDAR), indicating a change in the surface reflectance. For the RADAR sensor 108, R becomes 1 when the backscatter coefficient (B_R) exceeds a predefined threshold (Th_R), indicating a change in the electromagnetic reflectance of the road.Finally, the infrared sensor 110, I, becomes 1 when the emissivity (E_IR) exceeds its threshold (Th_IR), indicating a change in the thermal properties of the road surface. This binary classification approach for each sensor allows for a clear distinction between normal and potentially low-friction surfaces.

[0045] Finally, in step 208, the ESP 114 of vehicle 102 is activated based on the identified low-friction surface on the road. Activating the ESP 114 involves calculating data on the distance, width, time, location, and vehicle trajectory to the low-friction surface and providing this data to the ESP 114. The distance calculation is crucial because it determines how much time the system has to react and take safety measures. The width of the low-friction area is important for assessing whether the vehicle can safely navigate around it or must prepare for full contact. The time until reaching the area allows the ESP 114 to precisely time its interventions. The location data helps to map the hazard and potentially warn other vehicles through V2V communication.Information about the vehicle's trajectory is important to determine whether the vehicle can safely maneuver around the low-friction area, allowing ESP 114 to plan steering adjustments if evasive action is possible or to prepare for traction control intervention if contact is unavoidable. For example, if system 100 detects a low-friction surface 50 meters ahead of vehicle 102 with an estimated arrival time of 3 seconds, ESP 114 receives the information to prepare appropriate measures.

[0046] Upon detecting a low-friction surface on the road, the ESP 114 system issues a Level 1 warning to the driver, which may include audible, visual, and haptic cues indicating the presence of the detected low-friction surface. If there is no response from the driver, the ESP 114 system proceeds to a Level 2 warning, which includes partial braking. If, despite these warnings, the vehicle still enters the detected low-friction surface, one or more vehicle control systems 116, including the anti-lock braking system (ABS), traction control, engine braking, the braking system, or stability control, are activated via the ESP 114 system to ensure the safety of the vehicle and its occupants.

[0047] Fig.Figure 3 shows an example flowchart illustrating the activation process 300 of the ESP of vehicle 102 based on the determined low-friction surface on the road. Process 300 is implemented by system 100.

[0048] In step 302, system 100 detects a low-friction surface on the road surface based on the processed sensor data. This detection is performed using the previously described method 200, including the calculation of the weight matrix and the decision matrix.

[0049] In step 304, the control unit 112 transmits the information about the detected low-friction surface on the road to the ESP 114. This transmission ensures that the ESP 114 is informed about the potential hazard ahead and can react accordingly.

[0050] In step 306, a Level 1 warning is triggered. The Level 1 warning is designed to alert the driver to the detected low-friction surface. The warning can include visual, audible, and haptic signals to ensure the driver's attention is captured. The visual, audible, and haptic warnings can be delivered, for example, via a head-up display 318, an infotainment system 320, a loudspeaker 322, or another display device 324.

[0051] In step 308, procedure 300 includes the step of verifying the driver's response to the Level 1 warning. Specifically, system 100 determines whether the driver has reacted by reducing the vehicle's speed. This step is crucial for assessing whether the driver has acknowledged the warning and is taking appropriate action.

[0052] If the driver has reacted by reducing speed, the process proceeds to step 310. In step 310, the system assesses whether the braking initiated by the driver is sufficient to safely drive over the low-friction surface. If the braking is deemed sufficient, the ESP is informed of the driver's appropriate response.

[0053] However, if the driver does not reduce speed in step 308, or if it is determined in step 310 that the braking effect is insufficient, the process continues with step 312. In this step, a Stage 2 warning is triggered, which includes partial braking controlled by the ESP 114. The partial braking is performed to further reduce the vehicle's speed in preparation for the low-friction surface. The various braking mechanisms include the activation of a brake booster 326 to increase the hydraulic pressure in the brake lines, thus increasing the overall braking force. Simultaneously, an engine braking mechanism 328 can be activated by downshifting or engaging a lower gear to reduce the vehicle's speed without relying solely on the wheel brakes.A 330 braking system can also modulate the pressure of individual wheel brakes to maintain stability and prevent skidding when stopping on slippery surfaces.

[0054] Following the Stage 2 warning with partial braking, step 314 checks whether the vehicle speed has been reduced to a predefined limit deemed safe for the detected road conditions. If the speed has been reduced to a predefined limit, ESP 114 is informed.

[0055] If the vehicle speed has not been reduced to the predefined limit, the process proceeds to step 316. In this step, the ESP 114 is informed that the vehicle has entered the low-friction surface, which is detected by vehicle sensors such as wheel speed sensors, yaw rate sensor, steering angle sensor, etc. Subsequently, one or more vehicle control systems 116 are activated. These one or more vehicle control systems 116 may include additional warning indicators 332, stability control 334, traction control 336, engine braking 328, braking system 330, and power steering 338. The activation of the vehicle control systems 116 aims to maintain vehicle stability and control while the vehicle traverses the low-friction surface.

[0056] Process 300 ensures a graduated response to the detected low-friction surface, starting with driver warnings and escalating to autonomous intervention in vehicle control if necessary. This approach strikes a balance between the need to raise driver awareness and involve the driver and the requirement of vehicle safety in challenging road conditions.

[0057] The present invention provides a method and a system for actively predicting road surface conditions and proactively controlling a vehicle stability system. The system comprises a plurality of sensors, including an image sensor, a LiDAR sensor, a radar sensor, and an infrared sensor, configured to detect road surface conditions. The system includes receiving sensor output data from the aforementioned sensors and determining a weighting matrix (WT) that assigns weights to the output data of each sensor based on the current driving state of the vehicle. This approach enables a dynamic adjustment of the significance of the sensors based on the current conditions, thereby improving the system's adaptability to different driving scenarios.As soon as the system detects the presence of a low-friction surface, it activates the vehicle's Electronic Stability Program (ESP). The ESP warns the driver of the low-friction surface with an audible, visual, and haptic signal (Stage 1). If there is no response from the driver within 1.5 seconds, the ESP advances to a Stage 2 warning, which includes partial braking. For example, 30% of the vehicle's maximum braking force is applied to reduce the speed before the vehicle encounters the low-friction surface. If the vehicle still encounters the detected low-friction surface despite these warnings, the ESP 114 activates one or more vehicle control systems. These may include the anti-lock braking system (ABS), traction control, engine braking, the braking system, or stability control.

[0058] The present invention offers significant advantages over prior art systems. Unlike conventional solutions, the present invention enables more accurate detection of low-friction surfaces over a wider range of environmental conditions. Furthermore, the present system differs from reactive stability control solutions due to its proactive nature. By predicting low-friction surfaces before the vehicle encounters them, the system can initiate preventive measures, thus reducing the risk of accidents compared to systems that only react after detecting a loss of traction. This can reduce the risk of accidents. The multi-stage warning system also gives the driver more time to react, which increases the effectiveness of human intervention. The present invention is particularly advantageous in areas with rapidly changing weather conditions or in situations where the sensors of a single vehicle could be obstructed or impaired.

[0059] The system is important for both self-driving and fully autonomous vehicles. In self-driving cars, it improves the vehicle's ability to detect and react to changing road conditions, providing an additional layer of safety when human intervention might be necessary. In fully autonomous vehicles, the system's proactive approach to detecting low-friction surfaces enables sophisticated and predictive driving behavior, allowing these vehicles to handle challenging road conditions with greater safety and efficiency than human drivers.

[0060] The present invention significantly enhances vehicle safety by providing early warnings of potentially hazardous road conditions. This proactive approach can reduce the likelihood of accidents, particularly in adverse weather conditions. The system's ability to adapt to different driving environments makes it suitable for use in diverse geographical and climatic locations. Furthermore, the invention contributes to improved handling and vehicle stability. By alerting the driver to low-friction surfaces and automatically adjusting the vehicle control systems, it enhances the overall driving experience and reduces driver fatigue. This is especially beneficial on long journeys or in areas with frequently changing road conditions.Integrating the system into existing vehicle safety systems such as ABS and traction control creates a comprehensive safety package.

[0061] These exemplary embodiments serve only to illustrate the inventive concepts contained herein. Other embodiments and modifications can be made to the compositions and processes without departing from the spirit and scope of the invention. Therefore, the scope of the present invention should not be limited to the embodiments described herein, but should be defined by the appended claims and their equivalents. QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature

[0000] EP 3456597B1

[0007] US 20180060674A1

[0007] US 2022227328A1

[0007]

Claims

Method (200) for actively predicting road surfaces and proactively actuating a vehicle stability system, comprising: Receiving sensor output data from a variety of sensors, including an image sensor (104), a LiDAR sensor (106), a RADAR sensor (108), and an infrared sensor (110) configured to detect road surface conditions; Determining a weighting matrix (WT) comprising weights for each of the sensor output data based on a current driving state of the vehicle (102); Determining the presence of a low-friction road surface based on a decision matrix (DN), wherein the decision matrix (DN) is a product of the weighting matrix (WT) and a vector (X) representing the output data from each of the sensors;and activation of an electronic stability program (ESP) (114) of the vehicle (102) based on the determined low-friction surface on the road; wherein the driving conditions include at least one of the following: urban environment, motorway driving, adverse weather conditions and night driving conditions.; Method (200) according to claim 1, wherein the weighting matrix (WT) has the size 1x4 and comprises the weights W1, W2, W3, W4, which correspond to the image (104), LiDAR (106), RADAR (108) and infrared (110) sensors respectively; wherein the sum of W1, W2, W3 and W4 is equal to 1; and wherein the weighting for each of the sensors is calculated using a modified Bayesian method based on a base value of the corresponding sensor, an environment factor function and corresponding sensor measurements for low friction and normal surfaces, and a corresponding sensor correction factor. Method (200) according to claim 2, wherein the determination of the presence of a low friction surface based on the decision matrix (DN) comprises: calculating a low friction index (LFI) using the weight matrix (WT) and a vector (X); where X is the 4x1 vector containing the individual sensor output data [C, L, R, I] corresponding to the image (104), LiDAR (106), RADAR (108) and infrared sensors (110), respectively; and determining the presence of the low friction surface when the LFI is greater than or equal to a predetermined threshold. Method (200) according to claim 3, wherein the individual sensor output data [C, L, R, I] are determined as follows: C is 1 if the standard deviation of the pixel intensity is greater than or equal to a pixel intensity threshold, otherwise C is 0; L is 1 if the LiDAR reflection intensity is greater than or equal to a reflection intensity threshold, otherwise L is 0; R is 1 if the backscatter is greater than or equal to a backscatter coefficient threshold, otherwise R is 0; and I is 1 if the emissivity is greater than or equal to a threshold for the emissivity, otherwise I is 0. Method (200) according to claim 1, wherein the activation of the ESP (114) based on the detected low-friction surface on the road comprises: calculating the data for distance, time, location and vehicle trajectory to the low-friction surface; providing the calculated distance, time, location and vehicle trajectory data to the ESP (114); triggering a Level 1 warning to the driver, including audible, visual and haptic warnings indicating the presence of the low-friction surface; triggering a Level 2 warning, which includes partial braking, if no feedback is received from the driver; and activating one or more vehicle control systems (116), including anti-lock braking system (ABS), traction control, engine braking, braking system or stability control, when the vehicle (102) enters the detected low-friction surface. System (100) for actively predicting road surfaces and proactively actuating a vehicle stability system, comprising: a plurality of sensors, including an image sensor (104), a LiDAR sensor (106), a RADAR sensor (108), and an infrared sensor (110), mounted on the vehicle (102) and configured to detect road surface conditions; a control unit (112) configured to: determine a weighting matrix (WT) comprising weights for the output data of each sensor based on a current driving state of the vehicle (102); determine the presence of a low-friction surface on the road based on a decision matrix (DN), wherein the decision matrix (DN) is a product of the weighting matrix (WT) and a vector (X) representing the output data from each of the sensors;and to activate the vehicle's (102) electronic stability program (ESP) (114) based on the determined low-friction surface on the road; where the driving conditions include at least one of the following: urban environment, motorway driving, adverse weather conditions and night driving.; System (100) according to claim 6, wherein the weighting matrix (WT) has the size 1x4 and comprises the weights W1, W2, W3, W4, which correspond to the image sensor (104), the LiDAR (106), the RADAR (108) and the infrared sensor (110), respectively; wherein the sum of W1, W2, W3 and W4 is equal to 1; and wherein the weighting matrix (WT) is calculated using a modified Bayesian method. System (100) according to claim 7, wherein the control unit (112) is further configured to: calculate a Low Friction Index (LFI) using the weight matrix (WT) and a vector (X); where X is the 4x1 vector containing the individual sensor output data [C, L, R, I] corresponding to the image (104), LiDAR (106), RADAR (108) and infrared sensors (110), respectively; and detect the presence of the low friction surface when the LFI is greater than or equal to a predetermined threshold. System (100) according to claim 8, wherein the individual sensor output data [C, L, R, I] are determined as follows: C is 1 if the standard deviation of the pixel intensity is greater than or equal to a pixel intensity threshold, otherwise C is 0; L is 1 if the LiDAR reflection intensity is greater than or equal to a reflection intensity threshold, otherwise L is 0; R is 1 if the backscatter is greater than or equal to a backscatter coefficient threshold, otherwise R is 0; and I is 1 if the emissivity is greater than or equal to a threshold for the emissivity, otherwise I is 0. The system (100) according to claim 9, wherein the control unit (112) for controlling the ESP (114) is further configured to: calculate distance, time, location and vehicle trajectory data to the low-friction surface; transmit the calculated distance, time, location and vehicle trajectory data to the ESP (114); issue a level 1 warning to the driver, including audible, visual and haptic warnings indicating the presence of the low-friction surface; initiate a level 2 warning with partial braking by the ESP (114) if no feedback is received from the driver; and activate one or more vehicle control systems (116) including anti-lock braking system (ABS), traction control, engine braking, braking system or stability control via ESP (114) when the vehicle (102) enters the detected low-friction surface.

Images