Warning system for warning vehicle driver and method for warning vehicle driver

Abstract

To provide an additional warning to a driver of an emergency vehicle when the driver is in the vicinity of the own vehicle, and a corresponding warning system and warning method.SOLUTION: A warning system 100 for warning a driver of a vehicle comprises: sensors 3a and 3b for capturing peripheral information; a processing unit 4 for processing the information captured by the sensors; and output units 5, 7, 8, and 9 for outputting warning signals to the driver of the vehicle, wherein: the sensors 3a, 3b are adapted to detect information about other road users; the processing unit 4 analyzes the information regarding the presence of one or more specific features of other road users; and the processing unit outputs a signal to the output unit 5 so that when the analysis detects a presence of a specific feature of another road user, the output units output a warning signal.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a warning system for warning a vehicle driver and a method for warning a vehicle driver. In particular, when there are other road users / road participants who should be prioritized because they have special rights in a specific traffic situation and are in the vicinity of the vehicle, the present invention relates to warning the vehicle driver.

Background Art

[0002] In the case of a vehicle, the vehicle environment is usually visually monitored for the driver by providing mirrors. However, as a system to replace the mirrors or as an aid to the mirrors, it is increasingly common to provide a camera monitoring system that allows the driver to view the vehicle environment in areas that cannot be directly visually recognized during normal driving.

[0003] Emergency vehicles such as police vehicles and rescue vehicles are equipped with a device that emits a light signal (for example, in Germany, usually a blue flashing light), which lights up and is used when the emergency vehicle is in use. Therefore, it must be prioritized in road traffic. In other words, the vehicle has special rights in a specific traffic situation. The light is a flash with a frequency and sequence that can be seen and recognized from a distance. Furthermore, emergency vehicles are usually equipped with a siren / horn (for example, the so-called Martinshorn in Germany). This siren / horn emits an acoustic signal so that it can surely be heard by the drivers in the vicinity of the emergency vehicle, at least especially in a critical traffic situation, and the emergency vehicle is prioritized. In addition to the visual warning, an acoustic warning is also emitted. By such a device of the emergency vehicle, priority for the emergency vehicle is ensured, and thus the task can be quickly performed without usually affecting traffic safety. In order for the vehicle driver to be able to identify at least the sound, sound sequence, light, light sequence, etc. as clearly as possible as such an emergency vehicle, the use of the corresponding light and sound is often prohibited by road traffic rules for at least other road users. For example, in Germany, the use of blue, yellow, or red flashing lights by road users is limited to emergency vehicles.

[0004] In order for the driver of a vehicle to recognize other road users with special rights at an early stage, it is essential for the driver of the vehicle to constantly and reliably observe the surroundings of the vehicle, for example, by using a rearview mirror or a corresponding camera monitoring system so that the driver can recognize a vehicle approaching from behind at an early stage. Only by doing so can the driver reliably take useful and / or necessary measures according to the traffic situation, such as forming an emergency lane or stopping on the side of the road, in order to give priority to emergency vehicles.

[0005] In a camera monitoring system, it is known to provide additional information to the driver of the vehicle via a monitor. Such additional information may be, for example, vehicle-related additional information as described in DE 10 2023 106 451 (not published), or may be, for example, additional information regarding the surrounding environment as described in DE 10 2015 002 923.

[0006] Road users refer to all vehicles and people participating in road traffic. For example, emergency vehicles, vans, cars, trucks, bicycles, scooters, pedestrians, etc. are road users. A driver refers to a natural person who drives a vehicle. Therefore, a driver is not understood as a road user itself, but rather the vehicle (carrying him as the driver) is the road user. Summary of the Invention Problems to be Solved by the Invention

[0007] It is often difficult for the driver of a vehicle to obtain information indicating that an emergency vehicle is near the own vehicle at an appropriate timing. This is especially the case when the driver is highly concentrated on the current traffic situation.

[0008] Therefore, when an emergency vehicle is in the vicinity of the own vehicle, it is necessary to provide an additional warning to the driver of the vehicle, as well as a corresponding warning system and warning method. Means for Solving the Problems

[0009] The problem is solved by a warning system having the features of claim 1 and a method having the features of claim 15. Preferred embodiments are shown in the dependent claims.

[0010] The present invention is based on the idea of capturing environmental / surrounding information, and in some cases classifying it into specific categories such as other road users or obstacles, and further analyzing the captured surrounding information according to additional criteria related to specific categories of road users, and taking or initiating countermeasures according to the analysis results. The system according to the present invention itself does not necessarily need to assign road users to specific groups or categories, and means such as image recognition or artificial intelligence are possible. If certain criteria can be defined in advance that can lead to a clear conclusion, or substantially a clear conclusion, that a specific road user is involved and that there are specific situations with these special features, then by specifically detecting these special features, classification can be performed implicitly, thereby leading to both the conclusion that it is related to a specific group of road users and the conclusion that the road user has specific features. Examples of such unambiguous criteria can include the presence of a signal emitted from a halogen light housed in a blue housing and having a distinct IR component. Another example can include blue LED light or xenon light sources ("xenon flashers"), which emit very short light pulses. In these examples, it can be concluded that it is an emergency vehicle in operation and the traffic signal is on. In particular, such analysis can be performed as follows. That is, analysis can be performed regarding a road user having specific features, for example, a feature or combination of features that clearly indicates an emergency vehicle with its traffic signal on.

[0011] After that, if the presence of such a feature is detected, a special warning can be issued to the driver, or a corresponding signal can be output to the driver assistance system.

[0012] One possible approach is to first detect and analyze whether there are any other road users in the vehicle's environment. If other road users are detected, this surrounding information is then analyzed, in particular, to determine whether the other road users possess one or more specific characteristics. By having the warning system confirm the presence of other road users and analyze their characteristics in more detail, false warnings caused by, for example, corresponding light sources or sound sources away from road traffic can be minimized.

[0013] In particular, a warning system for warning the vehicle driver has sensors for capturing surrounding information. Such sensors can be any suitable type of sensor, for example, a camera for capturing visual signals, a microphone for receiving sound signals, a radio receiver for capturing radio signals, etc. In principle, any number of sensors of the same or different structures and / or types can be provided, and these can capture surrounding information complementarily or redundantly with each other. The information captured by the sensors is supplied to a processing unit. That is, a processing unit is provided for processing and analyzing the information captured by the sensors. Furthermore, the warning system includes an output unit adapted to output a warning signal to the vehicle driver. Additionally or alternatively, the processing unit can also output a signal to the vehicle driver assistance system, thereby taking appropriate measures and thus, for example, indirectly warning the driver or affecting the vehicle. Furthermore, the driver assistance system can also issue a direct warning. The driver assistance system can also function as a processing unit. In other words, the driver assistance system constitutes the processing unit. Preferably, the signal is transmitted to at least one output unit. There are various types of output units, such as a loudspeaker for outputting sound signals, a display for displaying visual warning information, etc. Output units installed in the vehicle, such as the monitor of a mirror-assisted camera monitoring system, can be used. Furthermore, different types of output units can be used in combination, and these can be provided specifically for the warning system or preferably are components of a mirror replacement system, a vehicle assistance system, or a vehicle entertainment system.

[0014] The sensor is adapted to record information about other road users. That is, this sensor is suitable for capturing ambient information including information about other road users. Environmental or ambient information is any kind of information that can be perceived or captured by human organs and / or devices from the environment / surroundings. The warning system converts the ambient information into processable data, sends the information recorded by the sensor to the processing unit, and the processing unit analyzes and classifies the information or the data obtained therefrom with respect to whether the ambient information contains information regarding specific or particular characteristics of other road users. For example, if information that other road users are present in the vehicle environment is available, the processing unit further analyzes this information with respect to whether one or more specific characteristics of the other road users are present. Another example is to exclusively analyze the ambient information to determine whether there are specific characteristics that are assigned only to specific other road users. In this case, the analysis is successful even if the vehicle itself is not reflected in the sensor.

[0015] In particular, features or properties such that ambient information is analyzed can be aimed at detecting special signals. A special signal, in other words, is a signal that serves to warn other road users of danger by an emergency vehicle and / or to indicate the use of special rights or right of way to other traffic by means of optical signals or sound signals. Such special signals vary by country, but for example, they are flashing lights of blue, red, and / or yellow, sirens (such as short signals (Yelp-signal) and long signals (Wail-signal)), and / or subsequent horns (Martinshorn), etc. Therefore, the features to be determined are, for example, light waves of a certain wavelength (corresponding to the spectral color of light, in other words, the color recognized by human color vision), the IR or UV component of light, in image analysis, for example, the RGB values of the captured image, a predetermined temporal sequence of light waves of different wavelengths, in image analysis, the temporal sequence of the RGB values of light and its intensity, a specific tone (frequency of sound), a predetermined temporal sequence of the frequency of sound, the Doppler effect or shift, or a special radio signal. If a frequency shift of sound due to the Doppler effect is detected and determined, it can also be used to draw conclusions about the relative movement of the vehicle with respect to the sound source. This can also be used, for example, to determine whether the sound source itself is moving. The same can be done by analyzing the volume of the sound signal.

[0016] When the analysis by the processing unit indicates that one or more specific features of one or more other road users are present in the vehicle environment observed by the sensor, the processing unit outputs a signal to the output unit so that the latter can output a warning signal (for example, in the form of a warning tone, reduction of the volume of the entertainment system (radio), a flashing frame in the case of the surroundings of the output unit, an emphasized display of the road user on the output unit (such as a graphic overlay display), etc.). Alternatively, and / or additionally, the processing unit can also send a signal to the driver assistance system of the vehicle, whereby the driver assistance system can take appropriate measures such as low-speed driving, stopping, etc.

[0017] For example, in addition to directly recognizing signals from emergency vehicles, i.e., in addition to the driver seeing the flashing of blue lights or hearing the siren sound, the driver can be warned early and surely, for example, when an emergency vehicle is near the own vehicle, by an additional warning to the driver. As a result, the driver and / or the driver assistance system can take appropriate measures so that the emergency vehicle can move forward smoothly. In addition, since the driver can take well-considered countermeasures at an early stage, traffic safety is also improved.

[0018] When the sensor is an optical sensor, the analysis of whether a road user with specific characteristics is present in the vehicle environment is preferably performed based on a combination of different image parameters without performing object recognition on the image. For example, blue light can be detected as a change over time in the luminance of the color value of the image. In particular, the brightness of individual RGB pixels before or after conversion to the color value of the image can be used, whereby the converted luminance value represents a kind of average value. For example, to identify blue light, the luminance curve / profile is observed over several frames (individual images), and the frequency of luminance changes is determined, and the luminance curve over time is used as a related feature. If this matches the (known) frequency of normal (conventional) blue light, the presence of certain characteristics of other road users is detected according to the embodiment. Additionally, blue light may be recognized as "true (actual) blue" blue light, for example, through a color angle range. This is particularly relevant when the light shines directly onto the camera. When only the light reflected by an object (e.g., the wall of a house) enters the camera, the perception of a color that is significantly different from the actual color of the signal light must be considered (especially when it is not a reflective surface such as the window of another vehicle, as it depends on the color of the object). In this case, the analysis can be largely based on the detected change in luminance. Additionally, or alternatively, the number of pixels in which a periodic change in luminance is perceived can be included in the analysis. This is based on the well-known very large beam angle of blue light, or a design / structure that can be visually recognized in a wide spatial area. Therefore, in relation to the image of the image sensor, blue light is likely to be visually recognized and identified in a large pixel area that spreads over a wide range of the sensor, and is less likely to be discovered in individual pixels, for example, different from the light indicators beside the road.

[0019] Preferably, the warning system is coupled to the vehicle's mirror replacement system, and the output of the warning system is a display or includes a display used to display mirror replacement information to the driver of the vehicle. The mirror replacement system may be a camera monitoring system provided with a device capable of replacing the vehicle's mirror, that is, providing an indirect view to the driver of the vehicle, so that the driver can view areas around the vehicle that are normally impossible or difficult to see. The warning information is visually displayed to the driver on this display. That is, no additional components are required, and since the driver is accustomed to receiving information about the surrounding environment through such a display, they can notice it at an early stage.

[0020] Particularly preferably, the warning signal includes a graphic display (for example, in the form of an overlay display) on the display, which can be, for example, an emphasized display of other road users, a frame around it, etc. This also provides the driver with position information of the observed road users relative to the vehicle. It is particularly preferred that an image of the mirror replacement system, for example, a legal field of view according to UN / ECE-R46, for example, fields of view of groups II and IV, is displayed on the display, and the warning information is displayed in the area of the display where the legal field of view is available, together with the position information regarding other road users.

[0021] In a preferred embodiment, the warning signal is output as long as the presence of one or more specific characteristics of other road users is detected, or until the driver switches the warning signal off. This prevents, for example, the driver from missing the warning signal because they were concentrating on traffic and not looking at the display at the start of the warning signal. Alternatively, whether the driver has perceived the warning signal can also be determined, for example, by the driver monitoring system determining that the driver has recognized the warning signal. In this case, subsequently, the driver monitoring system can switch off the warning signal.

[0022] Alternatively, the warning signal can also be output for a specific predetermined period (duration).

[0023] Preferably, the warning signal is output only after a period (duration) during which the presence of one or more specific features / characteristics is continuously detected has elapsed. That is, the sensor repeatedly and continuously captures ambient information, and the processing unit analyzes it. The warning signal is output only when the presence of a specific feature of other road users is continuously detected for a certain period, for example, 1 to 3 seconds. This can prevent, for example, unnecessary warning signals from being issued when an emergency vehicle crosses behind the vehicle. On the other hand, for example, when an emergency vehicle approaches from behind the vehicle, in this case, since a specific feature continuously exists for a certain period, the warning signal is surely output. "Continuous" does not necessarily mean that it never completely breaks. For example, a pattern of a specific frequency or wavelength, that is, the repeated occurrence of an observed feature in a short period also falls within the term "continuous".

[0024] The time when the warning signal is output may particularly depend on the technical specifications of one or more sensors used to continuously capture ambient information. When using, for example, a camera equipped with an optical sensor as a device for capturing ambient information, the time may depend on, for example, the refresh rate (frame rate) of the camera. When using a plurality of sensors / cameras with different technical specifications, the time basically depends on, for example, the sensor or camera with the slowest sampling rate / frame rate. However, the duration is also affected by other components involved in the processing of the captured ambient information. For example, the time can be influenced by how quickly the processing unit can analyze the information. The propagation times of various signals and the latency that can occur throughout the system can affect the duration.

[0025] Sensors for capturing ambient information can be provided separately or can be part of the mirror replacement system or camera monitoring system described above. For example, in the case of a camera, the warning signal can be output after repeating at least 2 frames to several images (frames), for example, after a period of 3 to 5 frames. Depending on the camera used, this corresponds to 30 ms to 175 ms after the warning signal is output (in the case of a system with a normal 30 - 60 fps). Thus, it is possible to prevent an incorrect warning from being output to the driver.

[0026] According to the present invention, the time duration until a warning signal is actually output can also depend on, for example, a combination of several seconds and a frame rate. This can occur, for example, when using a plurality of different sensors such as optical sensors and acoustic sensors. Optical sensors and acoustic sensors can detect features with different "reliabilities", that is, different levels of "certainty". This function does not need to be detected by all sensors. Rather, it is sufficient if it has already been detected with sufficient accuracy by one or more sensors. For example, if the reliability of an optical sensor is 90%, that is, its feature is detected with a 90% probability, a warning signal can be output without waiting for confirmation from an acoustic sensor. If the reliability is low (for example, less than 90%), it may be necessary to wait for the result of the acoustic sensor before outputting a warning signal.

[0027] Additionally or alternatively, the time until a warning signal is output may depend on the detected feature. In the case of a frequency sequence (e.g., typical features of a siren), if the camera is the sensor, it may take longer to determine than for a continuous feature that needs to be detected only once in order to avoid false warnings by determining whether it still exists in the next frame. In the case of a frequency sequence, the duration must be at least 2 cycles long, preferably 4 - 5 cycles long or more of the frequency sequence in order to ensure that the warning signal is output without error. The duration must also be considered in relation to the available sampling rate (at least the frame rate (fps) of the image sensor of the camera used or the possible exposure time (recording time)). In particular, since the duration of an acoustic frequency sequence (assuming the frequency sequence is a typical feature of a siren) generally must be much longer than that of an optical signal, the duration until a warning signal is output generally depends on the type of signal.

[0028] Finally, the duration of the output of the warning signal, and / or the start of the output, and / or its end may depend on the detection of other information captured by the vehicle, such as the presence of specific traffic situations and / or ambient conditions. Such traffic information and / or ambient information can be determined, for example, by evaluating or analyzing the current characteristics of the vehicle (e.g., information via the CAN bus). For example, depending on the traffic situation, it may be reasonable to issue a warning for as long as a feature is detected, for a longer period, or for a shorter period, i.e., not to issue a warning at all while a specific feature is detected. For example, if the traffic situation and ambient information suggest that a vehicle is driving on a winding road with unclear boundaries (such as trees or hillsides), and a specific feature, such as a blue light, can only be intermittently recognized (e.g., it can only be seen for a short time and then disappears, and this may happen several times), a longer warning would be effective. The driver can constantly recognize that a priority road user is approaching through a warning that lasts longer than when the signal is detected. On the other hand, for example, in a traffic jam situation on a highway (where the vehicle is stopped or moving very slowly on the highway), the driver of the vehicle already anticipates the possibility of a blue light and is therefore already on high alert. In this case, a temporary and rather short warning can be given to the driver of the vehicle, so that the driver of the vehicle is not exposed to an increased stress caused by a continuously lasting warning during subsequent maneuvering actions that require the driver to avoid an emergency vehicle.

[0029] The warning signal is preferably an acoustic signal and / or a visual signal. That is, a device that is usually already installed in the vehicle can surely issue a warning to the driver. That is, no additional equipment is required.

[0030] Examples of visual graphic warning signals include the lighting and flashing of specially provided signal lights, and the lighting and flashing of graphic symbols on displays already present in the vehicle, such as a head-up display. Further examples of visual warning signals include illuminating or flashing the frame around the entire display, and highlighting individual road users on the display of the camera monitoring system of a mirror replacement system.

[0031] Examples of acoustic warning signals include, for example, warning tones output from the vehicle's loudspeakers, and volume reduction of the entertainment system.

[0032] Further warning signals are conceivable, such as haptic signals like the vibration of the steering wheel.

[0033] Preferably, the sensor includes a camera and / or a microphone and / or a wireless receiver. The sensor can be part of a mirror replacement system, especially when a camera is used. When the sensor is a camera, the processing unit can analyze an image, for example, regarding road users / traffic participants and their specific features. For example, the processing unit can analyze the image regarding the presence of a specific color in the image, or a specific color sequence over time, intensity, and / or luminance. In particular, the temporal sequence of the intensity and / or luminance of the signal, or their spatial distribution, can be analyzed.

[0034] The wireless receiver is a particularly reliable and simple means if emergency vehicles or road users whose presence of specific features is observed are equipped with a transmitter that emits a wireless signal unique as a specific feature. The use of wireless signals has the advantage that, like acoustic signals, other road users do not necessarily need to be directly visible compared to visual signals detected by the sensor, and features can always be recognized from the images for detecting the respective signals.

[0035] In a particularly preferred embodiment, at least two independent and different sensors are provided. Independent and different sensors are sensors that are not formed by the same device, for example, the same camera, and can be operated independently of each other. The plurality of sensors may be of the same type or different types. That is, for example, two cameras can be provided as two sensors. When the sensors are of the same type, that is, for example, like a plurality of cameras or a plurality of microphones, the sensors can be provided as redundant sensors for verifying the detection results of any one of the sensors, or as complementary sensors for different regions of the vehicle environment. Furthermore, by using a plurality of independent redundant sensors, the probability of failure can be reduced. Even if the sensors are of the same type, for example, a plurality of cameras, structurally, they can be different cameras or sensors, for example, a plurality of cameras with different resolutions.

[0036] Advantageously, the sensors are of different types, that is, the sensor is a first sensor, and the warning system includes a second sensor that is different from the first sensor and is independent of the first sensor. Different sensors refer to sensors of different types, especially with respect to the type of information detected, for example, visual information on the one hand and acoustic information on the other hand. Both sensors transmit the detected information to the processing unit. For example, the first sensor is a camera and the second sensor is a wireless receiver or a microphone. If the presence of other road users with specific characteristics is detected by the processing unit using the first sensor, this can be independently verified by analyzing the information from the second sensor. In this case, the plurality of sensors are redundant. That is, the information detected and analyzed by one sensor is confirmed and verified by the information detected and analyzed by another sensor. The data from the two sensors do not need to be processed sequentially and can also be processed in parallel. When the sensors are provided as redundant sensors, the warning signal is preferably output only when both sensors independently detect the presence of specific characteristics of other road users. This improves the reliability and redundancy of the system.

[0037] Alternatively, different types of sensors can be provided as complementary sensors. For example, one camera can be provided for the side area of the vehicle, and one camera can be provided for the rear area of the vehicle.

[0038] In principle, the first sensor and the second sensor can be of the same type of sensor. For example, both can be cameras. This is particularly effective when different areas around the vehicle are observed by different sensors. When the same area around the vehicle is observed by two or more sensors, different types of sensors are preferred.

[0039] When a plurality of redundant sensors are provided, an abnormality of one sensor can be detected by comparing the analysis results of both or a plurality of redundant sensors. For example, if one of the sensors has a defect, such a failure will lead to the two redundant sensors repeatedly generating different analysis results. By indicating such a possibility of failure to the driver of the vehicle at an early stage, the driver can start appropriate measures such as repair and can know that the system can only be relied on within a limited range. This is to ensure that the system is functionally safe, for example, in accordance with ISO26262, which means that the system switches to a safe state when a failure occurs. Therefore, it is necessary to inform the driver that a failure has occurred or that the driver cannot rely on the system.

[0040] When using multiple sensors to monitor features, it may make sense to trust or weight the sensors differently and make the output of a warning signal depend on whether there is a sufficient probability (depending on the weighting of the sensors) that other road users with specific features are in the vehicle environment. The weighting may be fixed or, when a warning occurs, the reliability of the individual sensors may be calculated and the multiple sensors weighted accordingly. If a warning is not issued based on the result of one sensor but another sensor clearly senses the features of other road users, for example because the sensors are provided at different mounting positions on the vehicle, a warning should be issued. Additionally or alternatively, it is useful to trust different sensors to different extents in different situations or based on different data. For example, in the case of two optical sensors of different types, one sensor may be known to have a higher quantum efficiency at infrared wavelengths. For example, when detecting halogen blue light, the infrared component detected by such a sensor can be assigned a higher reliability and a warning can be issued even if the other sensor does not detect the infrared component.

[0041] Particularly preferably, the processing unit is adapted to analyze information regarding the presence of a specific pattern over time of the features of other road users. For example, the pattern can be a sequence of frequencies of a predetermined sound over time or a sequence of optical wavelengths over time. For example, when the sensor is a camera, the pattern corresponding to a specific feature can be a specific sequence of detected colors related to blinking pulses and flushing pulses. This can be associated, additionally or alternatively, with a specific color gradient (wavelength gradient of light) over time. In image analysis, in addition to the color itself, an intensity gradient or a luminance gradient over time can also be used. When observing and analyzing over time, the passage of features over time can be analyzed and evaluated from the perspective of how the position of the light source changes with respect to the vehicle.

[0042] A method of warning a driver of a vehicle is A step of capturing surrounding information by a sensor, where the surrounding information may include information regarding specific characteristics of other road users, and the step; A step of processing the captured surrounding information by a processing unit, where the processing includes analyzing the captured surrounding information to determine whether the captured surrounding information includes information regarding other road users, and the step; When the surrounding information includes information regarding other road users, a step of analyzing the surrounding information as to whether one or more specific characteristics of the other road users exist; When one or more specific characteristics of the other road users exist, a step of outputting a warning signal to the driver of the vehicle by an output unit.

[0043] The step of analyzing the surrounding information to determine whether this surrounding information includes information regarding other road users does not necessarily need to be executed as a separate and independent step, and can be executed together with the step of analyzing regarding specific characteristics of the road users. However, this is only limited to the case where this characteristic necessarily leads to the conclusion that it belongs to other road users. In this case, detecting a specific characteristic also means that the road user has that characteristic.

[0044] Preferably, the method further includes capturing surrounding information with a second sensor different from the first sensor, analyzing the surrounding information captured by the second sensor to determine whether the surrounding information includes information regarding other road users and whether one or more specific characteristics of the other road users exist, and verifying the analysis when it is determined that specific characteristics of the other road users exist.

[0045] Preferably, the warning signal is issued to the driver only when it is shown in the verification step that one or more specific characteristics of the other road users exist.

[0046] According to a preferred embodiment, the warning signal is output when one or more specific characteristics exist for a certain period of time as described above.

[0047] Aspects of the Invention 1. A warning system (100) for warning a driver of a vehicle, comprising sensors (3a, 3b) for capturing surrounding information, a processing unit (4) for processing the information captured by the sensors, and an output unit (5, 7, 8, 9) for outputting a warning signal to the driver (1) of the vehicle, wherein the sensors (3a, 3b) are adapted to capture information about other road users, and the processing unit (4) analyzes the information regarding the presence of one or more specific characteristics of the other road users, and is adapted to output a signal to the output unit (5) to output the warning signal when the analysis detects the presence of a specific characteristic of the other road users. Warning system (100). 2. In the warning system (100) according to one of the preceding aspects, the warning system (100) is coupled to a vehicle mirror replacement system having a display for displaying vehicle environment information to the driver of the vehicle, the vehicle environment information includes mirror replacement information, and the output unit (5) includes the display. Warning system (100). 3. In the warning system (100) according to aspect 2, the warning signal includes a graphic display on the display. Warning system (100). 4. In the warning system (100) according to any one of the preceding aspects, the warning signal is output as long as the presence of one or more specific characteristics of the other road users is detected. Warning system (100). 5. In the warning system (100) according to any one of the preceding aspects, the warning signal is output only after a period has elapsed during which the presence of one or more of the specific characteristics is continuously detected. Warning system (100). 6. In the warning system (100) according to any one of the preceding aspects, the warning signal is an acoustic signal and / or a visual signal. Warning system (100). 7. In the warning system (100) according to any one of the preceding aspects, wherein the sensor comprises a camera and / or a microphone and / or a wireless receiver, the warning system (100). 8. In the warning system (100) according to any one of the preceding aspects, wherein the sensor is a first sensor (3a), and the warning system comprises a second sensor (3b) that is different from the first sensor and outputs information to the processing unit, the warning system (100). 9. In the warning system (100) according to aspect 8, wherein the first sensor (3a) and the second sensor (3b) are sensors of different types and / or structures, the warning system (100). 10. In the warning system (100) according to aspect 8 or 9, wherein the processing unit (4) analyzes information from the first sensor (3a) and the second sensor (3b) regarding the presence of the specific characteristics of the other road users, and when it is determined that specific characteristics of other road users are present by analyzing the information from the first sensor (3a) and / or the information from the second sensor (3b), the processing unit is adapted to output a signal for outputting the warning signal to the output units (5, 7, 8, 9), the warning system (100). 11. In the warning system (100) according to any one of the preceding aspects, wherein the processing unit (4) is adapted to analyze the information regarding the presence of a specific pattern over time of one or more of the characteristics of the other road users, the warning system (100). 12. In the warning system (100) according to aspect 11, wherein the pattern is a tone sequence or a frequency sequence of a tone signal, the warning system (100). 13. In the warning system (100) according to aspect 11, wherein the pattern is a specific spatial and / or time-dependent sequence of luminance values and / or wavelengths in a visual signal, the warning system (100). 14. In the warning system (100) according to any one of the preceding aspects, further comprising a driver assistance system (11), the processing unit (4) is adapted to output a signal to the driver assistance system (11) when the analysis detects the presence of specific characteristics of the other road users, warning system (100). 15. A method for warning a driver of a vehicle, comprising the step of capturing ambient information by a sensor, the ambient information may include information regarding specific characteristics of other road users, step; processing the ambient information captured by a processing unit, the processing includes analyzing the captured ambient information to determine whether the captured ambient information includes information regarding other road users, step; analyzing the ambient information to determine whether one or more specific characteristics of the other road users are present; and, when one or more specific characteristics of the other road users are present, outputting a warning signal to the driver of the vehicle by an output unit. 16. In the method according to aspect 15, when it is determined that one or more specific characteristics of the other road users are present, capturing ambient information with a second sensor different from the first sensor, analyzing the ambient information captured by the second sensor to determine whether the ambient information includes information regarding other road users and whether one or more specific characteristics of the other road users are present and further comprising the step of verifying the analysis. 17. In the method according to aspect 16, outputting the warning signal to the driver only when, as a result of the verifying step, one or more specific characteristics of other road users are present. 18. In the method according to any one of aspects 15 to 17, The method wherein the warning signal is output when one or more specific features exist for a certain period of time. 19. In the method according to any one of aspects 15 to 18, The method wherein the warning signal is output until the driver monitoring system determines that the driver has noticed the warning signal.

Brief Description of the Drawings

[0048] Hereinafter, the present invention will be described by way of example only with reference to the accompanying drawings.

[0049]

Figure 1

[0050]

Figure 2a

[0051]

Figure 2b

[0052]

Figure 3a

[0053]

Figure 3b

[0054]

Figure 4

[0055]

Figure 5

[0056]

Figure 6

[0057]

Figure 7

Mode for Carrying Out the Invention

[0058] Hereinafter, preferred embodiments will be described with reference to the accompanying drawings.

[0059] FIG. 1 schematically shows a warning system (100) according to an embodiment of the present invention. The warning system (100) includes a first sensor (3a) and a second sensor (3b) independent of the first sensor (3a). Both sensors (3a, 3b) independently capture / acquire ambient information around the vehicle (2).

[0060] The sensors (3a, 3b) can each be, for example, a camera, a wireless receiver, or a microphone. When there are two sensors (3a, 3b) as in the illustrated embodiment, these sensors are preferably of different types. That is, for example, one of the sensors (3a) is a camera and the other of the sensors (3b) is a microphone. In principle, any number of sensors, that is, two or more sensors, or only one sensor may be provided. For example, different sensors can be directed to different regions around the vehicle. The arrangement of the sensors on the vehicle is arbitrary, and in particular, whether to arrange them at one location or multiple locations on the vehicle is arbitrary.

[0061] The sensors (3a, 3b) are connected to the processing unit (4), and the processing unit (4) transfers the information captured by the sensors (3a, 3b) for further processing, particularly analysis.

[0062] The processing unit (4) is adapted to analyze the ambient information received from the sensors (3a, 3b) to determine whether there is a road user having one or more specific characteristics in the captured vehicle environment.

[0063] To enable the warning system (100) to output a warning signal to the driver (1) of the vehicle, the warning system (100) also includes one or more output units (5, 7, 8, 9). The output units (5, 7, 8, 9) can be, for example, a display, a speaker, a head-up display, a vehicle multimedia system, etc.

[0064] Furthermore, in the illustrated embodiment, the processing unit (4) is coupled to the driver assistance system (11), and the processing unit (4) can also output a signal to the driver assistance system (11) so that the driver assistance system (11) can take or propose appropriate measures such as decelerating the driving.

[0065] Figures 2a and 2b are respectively a side view and a top view of a vehicle (2) equipped with a warning system (100) according to a further embodiment. In this case, the first sensors (3a) are provided on both sides of the vehicle, for example, as cameras in respective cases, whereby different regions around the vehicle are detected by the respective sensors (3a), and each can be analyzed regarding the presence of the same feature, for example, the wavelength of light.

[0066] Figures 3a and 3b are exemplary side views of a vehicle (2) corresponding to the vehicle shown in Figures 2a and 2b, and in its rear region, there is an emergency vehicle, for example, a rescue vehicle, and the emergency vehicle emits a specific signal that can be captured by the sensors (3a) attached to the vehicle (2) during operation. Figures 3a and 3b show that the rescue vehicle emits acoustic ambient information (6c), optical ambient information (6b), and radio waves (6a) as ambient information. Of course, the rescue vehicle may emit all different types of ambient information simultaneously or only a part of it. For example, it may emit only blue light (optical ambient information (6b)) and a siren signal (acoustic ambient information (6c)).

[0067] FIG. 7 schematically shows a flowchart of the embodiment shown in FIG. 1 to explain how the warning system (100) operates, i.e., how the method for warning the driver of the vehicle according to the embodiment is executed. The steps shown in FIG. 7 are continuously repeated.

[0068] When ambient information is captured by the sensors (3a, 3b), the information is analyzed by the processing unit (4) respectively. In this process, it is determined whether the ambient information includes information indicating that there are other road users around the host vehicle. In this case, or simultaneously with this analysis, or in conjunction with this analysis, the information is analyzed to determine whether the ambient information includes one or more specific features indicating, for example, an emergency vehicle or a rescue vehicle in use / operation. When the ambient information is captured by the optical sensors (3a, 3b), for example, it is possible to analyze whether the road user has turned on the warning light (blue light). This can be done, for example, by analyzing the wavelength of the detected light and / or its pattern, or by image analysis regarding color or a continuous color pattern over time. When the ambient information is detected (acoustically) and can be used as acoustic information by, for example, a microphone, it is possible to determine the tone or tone sequence indicating the siren (martins - horn) of the emergency vehicle by frequency analysis.

[0069] When the processing unit (4) determines that one or more specific features exist, it outputs a signal to the output unit (described in relation to FIGS. 4 - 6), and the output unit outputs a warning signal to the driver of the vehicle.

[0070] In the embodiment shown in FIG. 7, the signal is not output immediately when it first occurs, but is output only after it has existed for a certain period of time. Here, these procedures and steps are repeated. By outputting a signal to the driver of the vehicle only when one or several specific features exist for a certain period of time, it is possible to prevent unnecessary warning signals from being output to the driver of the vehicle, such as when an emergency vehicle crosses behind the vehicle.

[0071] In the embodiment shown in FIG. 7, the ambient information is also redundantly captured by two different sensors (3a, 3b) and independently analyzed by the processing unit (4). A warning signal is output only when the ambient information from both sensors (3a, 3b) indicates the presence of specific characteristics of other road users. This also reliably prevents unnecessary warnings to the driver of the vehicle.

[0072] Figures 4 to 6 schematically show various embodiments relating to the output unit and the warning signal.

[0073] FIG. 4 shows a driver's seat equipped with left and right displays (5), on which, for example, mirror replacement information is displayed respectively. When a warning signal is issued, this can be done, for example, by displaying a corresponding superimposed display (10a) as a visual warning signal on the display (5) (FIG. 5), or, for example, by highlighting the frame of the display (5) by flashing, and / or by displaying a symbol as a visual signal (10a) (FIG. 6).

[0074] Alternatively or additionally, as shown in FIG. 4, an acoustic warning can also be given by using the vehicle's multimedia device (8) to output a special sound such as a beep, or by reducing the output volume of the multimedia device (8). An independent dedicated loudspeaker (7) can also be used as the output unit for warning tones, for example, any type of warning tone.

[0075] A further option shown in FIG. 4 is to display visual warning information on the head-up display (9).

[0076] With the warning system (100) and the method of warning the driver of the vehicle described, the driver of the vehicle can be reliably notified at an early stage and at an appropriate timing even when, for example, there is strong attention focused on road traffic and an emergency vehicle is in the vicinity of the own vehicle.

[0077] It is explicitly stated that all features disclosed in the description and / or claims shall be regarded as mutually distinct and independent, regardless of the combination of features in the embodiments and / or claims, both for the purpose of the original disclosure and for the purpose of limiting the invention described in the claims. All range descriptions or descriptions of unit groups are explicitly stated to disclose any possible intermediate values or sub-groups of units, both for the purpose of the original disclosure and for the purpose of limiting the invention described in the claims (especially the limitation of range descriptions).

Explanation of Reference Numerals

[0078] 1 Driver of the vehicle 2 Vehicle 3a, 3b Sensors 4 Processing unit 5 Output unit (display) 6a, 6b, 6c Surrounding information (wireless, visual, acoustic) 7 Output unit (loudspeaker) 8 Output unit (multimedia device) 9 Output unit (head-up display) 10a Warning signal (visual) 11 Driver assistance system 100 Warning system

Claims

1. A warning system (100) for warning a driver of a vehicle, comprising sensors (3a, 3b) for capturing surrounding information, a processing unit (4) for processing the information captured by the sensors, and an output unit (5, 7, 8, 9) for outputting a warning signal to the driver (1) of the vehicle, wherein the sensors (3a, 3b) are adapted to capture information about other road users, and the processing unit (4) is adapted to analyze the information regarding the presence of one or more specific characteristics of the other road users, and to output a signal to the output unit (5) to output the warning signal when the analysis detects the presence of the specific characteristics of the other road users. A warning system (100).

2. In the warning system (100) according to claim 1, the warning system (100) is coupled to a vehicle mirror replacement system of a vehicle comprising a display for displaying vehicle environment information to a driver of the vehicle, the vehicle environment information includes mirror replacement information, and the output unit (5) comprises the display. A warning system (100).

3. In the warning system (100) according to claim 2, the warning signal includes a graphic display on the display. A warning system (100).

4. In the warning system (100) according to claim 1, the warning signal is output as long as the presence of one or more specific characteristics of the other road users is detected. A warning system (100).

5. In the warning system (100) according to claim 1, the warning signal is output only after a period during which the presence of one or more of the specific characteristics is continuously detected has elapsed. A warning system (100).

6. In the warning system (100) according to claim 1, the warning signal is an acoustic signal and / or a visual signal. A warning system (100).

7. In the warning system (100) according to claim 1, the sensor comprises a camera and / or a microphone and / or a wireless receiver. A warning system (100).

8. In the warning system (100) according to claim 1, the sensor is a first sensor (3a), and the warning system comprises a second sensor (3b) that is different from the first sensor and outputs information to the processing unit. A warning system (100).

9. In the warning system (100) according to claim 8, A warning system (100) wherein the first sensor (3a) and the second sensor (3b) are sensors of different types and / or structures.

10. In the warning system (100) according to claim 8 or 9, the processing unit (4) analyzes information from the first sensor (3a) and the second sensor (3b) regarding the presence of the specific features of the other road users, and outputs a signal for outputting the warning signal to the output units (5, 7, 8, 9) when it is determined, by analyzing the information from the first sensor (3a) and / or analyzing the information from the second sensor (3b), that specific features of other road users are present. A warning system (100).

11. In the warning system (100) according to claim 1, the processing unit (4) is adapted to analyze the information regarding the presence of a specific pattern over time of one or more of the features of the other road users. A warning system (100).

12. In the warning system (100) according to claim 11, the pattern is a tone sequence or a frequency sequence of a tone signal. A warning system (100).

13. In the warning system (100) according to claim 11, the pattern is a specific spatial and / or time-dependent sequence of luminance values and / or wavelengths in a visual signal. A warning system (100).

14. In the warning system (100) according to claim 1, further comprising a driver assistance system (11), the processing unit (4) is adapted to output a signal to the driver assistance system (11) when the analysis detects the presence of specific features of the other road users. A warning system (100).

15. A method for warning a driver of a vehicle, comprising: capturing ambient information by a sensor, the ambient information may include information regarding specific features of other road users; processing the ambient information captured by a processing unit, the processing including analyzing the captured ambient information to determine whether the captured ambient information includes information regarding other road users; analyzing the ambient information to determine whether one or more specific features of the other road users are present. A method comprising, when there are one or more specific characteristics of the other road users, outputting a warning signal to the driver of the vehicle by an output unit.

16. In the method according to claim 15, when it is determined that there are one or more specific characteristics of the other road users, capturing ambient information with a second sensor different from the first sensor, analyzing the ambient information captured by the second sensor to determine whether the ambient information includes information about other road users and whether there are one or more specific characteristics of the other road users and further comprising the step of verifying the analysis in this way.

17. In the method according to claim 16, outputting the warning signal to the driver only when, as a result of the verifying step, there are one or more specific characteristics of the other road users.

18. In the method according to claim 15, the warning signal is output when one or more specific characteristics exist for a certain period of time.

19. In the method according to claim 15, the warning signal is output until the driver monitoring system determines that the driver has noticed the warning signal.

Images