System and method for detecting potentially hazardous objects within vehicle

Abstract

Systems and methods for detecting potentially hazardous objects within a vehicle are provided. The present disclosure is directed to an Advanced Driver Assistance System (ADAS) for a vehicle, the system configured to detect potentially hazardous objects inside the vehicle, the system comprising at least one sensor unit, a processing unit, and an output unit, the sensor unit is configured to monitor at least a portion of the interior of the vehicle and to transmit the acquired sensor data to the processing unit, and wherein the processing unit is configured to analyze the transmitted sensor data, detect objects within the vehicle based on the analyzed sensor data, classify the detected objects as potentially hazardous objects, and transmit the potentially hazardous objects to the vehicle. And when the detected object is classified as a potential dangerous object, the output unit is triggered to give an alarm.

Description

Technical Field

[0001] This invention relates to systems and methods for detecting potentially hazardous objects inside a vehicle. Background Technology

[0002] A loose or damaged floor mat, or a loose object rolling in the driver's footwell, can pose a significant hazard. Even a dirty floor mat can be unsafe. If a floor mat gets tangled in the pedals, or a loose object gets stuck under the pedals, a collision may not be avoidable even if the driver reacts in time. Therefore, clean, intact, and properly positioned floor mats, and a footwell free of any objects, are equally important.

[0003] Furthermore, while many countries' laws do not mandate that drivers wear appropriate footwear, unsuitable shoes can affect insurance claims in the event of an accident. It has been proven that clean, lightweight athletic wear is best suited for driving. Conversely, wet, muddy, heavy, or loose shoes are less suitable for driving, as the driver may slip or the shoe may get tangled in the pedals. Therefore, drivers should wear appropriate footwear when driving.

[0004] Therefore, there is a need for systems and methods that can reliably detect objects within a vehicle that pose a potential hazard and may impair the driver's ability to drive safely. Summary of the Invention

[0005] This disclosure provides a system and a computer-implemented method according to the independent claims. Embodiments are given in the dependent claims, the specification, and the drawings.

[0006] In one aspect, this disclosure relates to an advanced driver assistance system (ADAS) for a vehicle, the system being configured to detect potentially hazardous objects within the vehicle, the system comprising: at least one sensor unit, a processing unit, and an output unit, wherein the sensor unit is configured to monitor at least a portion of the interior of the vehicle and transmit acquired sensor data to the processing unit, and wherein the processing unit is configured to: analyze the transmitted sensor data, detect objects within the vehicle based on the analyzed sensor data, classify the detected objects as potentially hazardous objects, and trigger the output unit to issue a warning if the detected objects are classified as potentially hazardous objects.

[0007] The system monitors at least a portion of the vehicle's interior using a sensor unit that includes at least one sensor. Possible locations for the sensor unit monitoring portions of the vehicle's interior include below the steering wheel, inside the seats, inside the doors, in the vehicle's roof when viewed from above, or below the rearview mirror. These locations are not limited to those described and can be changed to make selected portions of the vehicle visible.

[0008] The system is configured to detect anomalies and hazards related to objects detected and identified inside the vehicle based on acquired sensor data. The acquired sensor data is forwarded to a processing unit. The processing unit carries processing algorithms and related software components to analyze the sensor data, detect objects, and classify the detected objects as potentially hazardous.

[0009] The processing unit can be part of a dedicated electronic control unit (ECU). It can also be part of a multi-domain controller, or a component of a sensor unit, such as a smart camera. The processing unit can also be part of a cloud setup external to the vehicle.

[0010] The output unit can trigger warnings, such as visual or audible warnings. For example, a warning can be displayed on the vehicle's screen or output through the vehicle's audio system. The output unit warns or informs the driver about potential hazards posed by objects identified inside the vehicle.

[0011] This system reliably detects and categorizes potentially hazardous objects within a vehicle and warns the driver of such objects or abnormal situations. Therefore, it enhances the driver's ability to safely operate the vehicle and contributes to safer driving practices.

[0012] According to the implementation, the monitored portion of the vehicle's interior includes at least the driver's footwell area. Because anomalies caused by objects in the driver's footwell area (such as loose or broken floor mats, or loose objects rolling within the footwell) can pose significant hazards, this system can specifically monitor the driver's footwell area and immediately issue a warning upon detecting a potentially dangerous object. For example, the system can immediately identify a foot mat tangled in the pedals or a loose object stuck underneath it and notify the driver accordingly. This allows the driver to react promptly to such dangerous situations.

[0013] According to an implementation, the processing unit is configured to classify objects as potentially hazardous objects based on the detected position of the object, and / or the detected movement of the object, and / or the assigned hazard level of the object. A detected object can be classified as a potentially hazardous object based on its detected position (e.g., proximity to a vehicle pedal). The position of the object can also be tracked in sensor data over a certain time period to check if the object is loose, for example, whether it is rolling in a footrest area. Independent of the object's position or movement, objects can also be classified as potentially hazardous objects by assigning predetermined hazard levels to certain categories of objects.

[0014] According to the implementation, potentially hazardous objects are objects in the driver's footwell area, particularly loose objects, and / or damaged flooring or floor mats, and / or footwear unsuitable for driving. Objects in the driver's footwell area can severely impair a driver's ability to safely drive the vehicle, especially their ability to react promptly in situations requiring rapid action. Loose objects are even more dangerous than stationary objects because the driver may not recognize an object rolling into the footwell. The driver can be warned of these objects based on their location and / or state of motion or degree of "looseness." Alternatively, any detected object, regardless of its location or degree of "looseness," can trigger a warning.

[0015] Unidentified damaged floor mats can pose a significant hazard when driving a vehicle. Damaged floor mats may be loose, broken, folded, and / or misplaced. Dirty floor mats in the driver's footwell or on the pedals can also cause problems, as the driver may slip, leading to loss of vehicle control. Therefore, inspecting the footwell and / or pedals for dirt, especially severe dirt, helps ensure vehicle safety.

[0016] Unsuitable footwear worn by the driver can also create dangerous situations during vehicle operation. Unsuitable footwear for driving can include, for example, flip-flops, high heels, platform shoes, or even bare feet. Furthermore, dirt, mud, moisture, snow, etc., can reduce the suitability of footwear for operating the vehicle and may lead to it being classified as unsuitable for driving.

[0017] According to one embodiment, the sensor unit includes at least one camera device configured to provide multiple images of a monitored portion of the vehicle's interior. The camera device may include one or more imaging sensors that monitor that portion of the vehicle's interior at a predetermined frame rate. The imaging sensors can monitor the vehicle's interior efficiently and economically, and when operating at the predetermined frame rate, can accurately track potentially moving or loose objects within the vehicle.

[0018] According to the implementation, the camera device is a 2D near-infrared (NIR) camera and / or a 2D near-infrared RGB camera and / or a 3D time-of-flight (ToF) camera and / or a 3D stereo camera. Cameras operating in the near-infrared (NIR) band can have high sensitivity. RGB-IR cameras can capture RGB colors as well as the infrared spectrum, thus providing more comprehensive spectral coverage of image data. To improve the sensitivity of the camera device, it can also be configured with different color filter arrays, such as RCCC (red filter), Bayer-RGGB, or similar filter arrays. The camera device can be equipped with one or more active illumination sources, such as infrared LEDs or vertical-cavity surface-emitting laser (VCSEL) illuminators operating in the infrared spectrum (e.g., 850 nm or 940 nm). These illuminators can monitor the interior of the vehicle even under low ambient light conditions. The camera device can also include a 3D imaging sensor, such as a ToF camera, a stereo camera, or a structured light camera. 3D information can improve the accuracy of identification and classification of potentially hazardous objects in the image data.

[0019] According to the implementation, the processing unit is configured to analyze sensor data using neural networks and / or support vector machines (SVMs) and / or other machine learning methods and / or image processing tools to detect objects and classify them as potentially hazardous objects. Neural networks or other machine learning methods can also be trained to detect soil, mud, moisture, snow, or similar features in the data. Conventional computer vision methods can also be used to process the image data acquired by the sensor unit. For example, the degree of dampness in a footstool can be detected by searching for overexposed areas in the image data associated with highly reflective components caused by dampness in the footstool. The processing unit can also use other algorithms, such as segmentation, Haar feature detection, scale-invariant feature transform (SIFT), histogram of oriented gradients (HOG), or other computer vision or machine learning algorithms to identify and classify objects in the data. Damaged floors or footpads can also be detected by edge, corner, or blot detection of the data. Anomalies associated with loose objects can be identified by motion detection based on optical flow determined in the data.

[0020] According to one implementation, the processing unit is configured to identify the driver and the area covering the driver's feet in the sensor data. The system can utilize the acquired sensor data to identify persons driving or about to operate and drive the vehicle, particularly when the person is outside the vehicle or entering the vehicle. Therefore, the system can identify the driver not only when the driver is inside the vehicle, but also among persons outside the vehicle and / or entering the vehicle. Thus, the sensor unit is mounted such that its field of view does not include the driver's footrest area, but includes the driver's door. The processing unit can detect and classify the driver's feet, for example, using neural networks, other machine learning methods, or computer vision methods. For example, this can be achieved by first detecting persons outside the vehicle when the driver's door is open. The driver is then identified and their feet detected, for example, by using a landmark detection algorithm or other object detector that focuses only on sensor data including and tracking the driver.

[0021] According to an implementation, the processing unit is configured to mask sensor data covering the area of ​​the driver's feet and ignore the sensor data when an object is detected and classified as a potential hazard. For example, when monitoring the driver's footwell, the system may detect a damaged or folded floor or floor mat when the footwell is empty (i.e., no one has placed their feet or other objects in the footwell area). An algorithm analyzing anomalies in the footwell area can then detect the damage or folding of the floor mat and classify it as a potential hazard accordingly. For such detection, the algorithm can use image processing techniques, neural networks, or other machine learning methods to detect and / or classify the sensor data. If feet are present in the sensor data, additional detection algorithms can be used to detect and locate the feet, and sensor data in these areas can be ignored in subsequent processing steps.

[0022] According to an implementation, the processing unit is configured to identify the driver's footwear in sensor data and classify it as suitable or unsuitable for driving, and potentially hazardous, based on the type and / or degree of soiling of the footwear. For example, the processing unit can locate the driver's foot in the footrest area, identify the driver's footwear within the foot-covered area, and classify it as "safe for driving" or "unsafe for driving." The processing unit can also classify the driver's footwear into more granular categories, describing shoe types such as sneakers, boots, high heels, flip-flops, or barefoot. For example, classification can be achieved using a neural network to classify sensor data of the driver's footrest area or sub-images of detected foot-covered areas. Based on the classification results, the driver can be informed that the footwear may be unsuitable for driving. For example, flip-flops, high heels, platform shoes, or barefoot fall into this category. Furthermore, thick layers of mud, sludge, or snow on footwear, or wet footwear, can all affect vehicle operation and driving. Therefore, cleanliness classification can be implemented to detect the degree of dirt on the driver's footwear and categorize it based on factors such as mud, sludge, moisture, and snow that may reduce the suitability of footwear for vehicle operation. This classification can also be achieved using neural networks or other machine learning methods that categorize sensor data from the footrest area or detected sub-images of the feet.

[0023] According to one embodiment, the sensor unit is configured to monitor at least the driver's door. The sensor unit may be a single sensor unit that also monitors the driver's footwell area. Alternatively, the system may include a sensor unit configured to monitor the driver's door and at least one sensor unit configured to monitor the driver's footwell area. Therefore, the system may include a cabin sensor and a footwell sensor. The cabin sensor may have visibility of the driver's door area, visibility outside the vehicle when the driver's door is open, and visibility of the driver's footwell area. The footwell sensor acquires data for identifying potentially hazardous objects in the driver's footwell area. The cabin sensor acquires data on personnel entering or about to enter the vehicle to operate it. The cabin sensor may be positioned such that the driver's footwear is visible at least during the driver's entry into the vehicle. This ensures that the data from the cabin sensor can be used to identify the driver's footwear before the driver enters the vehicle. If a single sensor unit is used to monitor the driver's door area and the driver's footwell area, the data from the sensor unit can be used to identify potentially hazardous objects within the driver's footwell area as well as data on personnel entering or about to enter the vehicle to operate it.

[0024] According to the implementation, the processing unit is configured to identify the driver entering the vehicle based on data from the sensor unit, identify the area covering the driver's feet, identify the driver's footwear in the sensor data, and classify the driver's footwear as suitable for driving, unsuitable for driving, or potentially hazardous. Personnel detection can be performed by an object detection algorithm trained to detect personnel, a body keypoint detection algorithm, or any other algorithm used to detect the location of personnel in an image. Identifying the driver from the detected personnel can be achieved, for example, by assigning the detected person closest to the driver's door as the driver, or by tracking personnel and assigning the detected person who enters the vehicle at a given moment as the driver. For example, the area covering the driver's feet can be identified by detecting the driver's body keypoints, where the driver's feet are part of these detections. When a camera device with a viewpoint covering the driver's footrest area is used as the sensor unit, sub-images can be cropped from the full image around keypoints indicating the driver's foot position. The driver's footwear in these sub-images can then be classified using algorithms such as neural networks or other machine learning methods. The classification can be based on the type of footwear worn, or on the degree to which the footwear is soiled with mud, slurry, moisture, snow, etc., which reduces the suitability of the footwear for operating the vehicle. Therefore, the system can also warn the driver before they enter the vehicle that their footwear is dirty, as the footwear may soil the driver's footrest area and create a dangerous situation during driving.

[0025] This disclosure also relates to the use of the system for detecting potentially hazardous objects within a vehicle, or to the use of any of the above-described embodiments.

[0026] On the other hand, this disclosure relates to a computer-implemented method for detecting potentially dangerous objects within a vehicle, the method comprising the following steps:

[0027] - At least a portion of the interior of the vehicle is monitored by at least one sensor unit, and the acquired sensor data is sent to the processing unit (16); and the processing unit performs the following steps:

[0028] - Analyze the transmitted sensor data;

[0029] - Detect objects within the vehicle based on the analyzed sensor data;

[0030] - Classify detected objects as potentially hazardous objects; and

[0031] - If a detected object is classified as a potentially dangerous object, the output unit is triggered to issue a warning.

[0032] This method can be performed by the system described above or any of its implementations. The description of the system and its implementations applies accordingly.

[0033] This method monitors parts of the vehicle's interior, identifies potentially hazardous objects, and warns the driver of the potential dangers posed by said objects. Specifically, the method can monitor the driver's footwell area and / or the driver's door, and can further optically categorize the driver's footwear using one or more camera devices installed inside the vehicle cabin. The method can monitor the driver's door, identify the driver before entering the vehicle, detect the driver's feet as they enter, and categorize their footwear as suitable or unsuitable for driving the vehicle. The method can also inform the driver of potential hazards related to floor mats or objects within the driver's footwell area. This method can monitor the driver's footwell area, detect the driver's feet, and categorize their footwear as suitable or unsuitable for driving the vehicle. This categorization can be based on the type of footwear or on conditions such as soiling with mud, moisture, snow, etc., which reduce the driving suitability of the footwear. If unsuitable footwear is detected, the method can remind the driver to wear appropriate footwear, or warn the driver that the current footwear may be unsafe for driving or may pose a danger in the driver's footwell area. Furthermore, the method can also detect hazards associated with objects in the driver's footwell area, including floor mats or floor mats, bottles, or other objects. Attached Figure Description

[0034] This document describes exemplary embodiments and functions of the present disclosure in conjunction with the following accompanying drawings, which are shown in schematic form:

[0035] Figure 1A and Figure 1B : A schematic diagram of a vehicle equipped with the system of the present invention;

[0036] Figure 2A and Figure 2B Two examples of images showing the driver's footwell area;

[0037] Figures 3A to 3F Examples of potentially hazardous objects within the driver's footwell area;

[0038] Figure 4 Further examples of potentially hazardous objects within the driver's footwell area;

[0039] Figure 5 : A flowchart of an implementation method for detecting potentially hazardous objects within a vehicle;

[0040] Figure 6 Example of body keypoint and skeleton detection in an image sequence.

[0041] List of reference numerals

[0042] 10 vehicles

[0043] 12 systems

[0044] 14 Sensor Units / Cameras

[0045] 14A First Camera

[0046] 14B Second Camera

[0047] 16 processing units

[0048] 18 Output Units

[0049] 20. Driver's foot space

[0050] 22 Field of view

[0051] 24 Potentially Dangerous Objects

[0052] 26. Driver's footwear

[0053] 28. Driver's foot area Detailed Implementation

[0054] Figure 1A and Figure 1B A schematic side view and a top view of a vehicle 10 equipped with the system 12 according to an embodiment of the present invention are depicted. The system 12 includes a camera 14, a processing unit 16, and an output unit 18. The camera 14 may be, for example, a 2D NIR or 2D RGB-IR camera, which monitors the driver's footrest area 20 at a given frame rate. The camera may also be a 3D time-of-flight (ToF) camera or a 3D stereo camera. The field of view 22 of the camera 14 monitoring the driver's footrest area 20 is also shown in the figures.

[0055] Camera 14 may be located, for example, under the steering wheel, inside the seat, inside the door, in the vehicle roof (viewing the front row from above), or below the rearview mirror. Figure 2A and Figure 2B The diagrams show the components located on the vehicle roof ( Figure 2A ) and below the steering wheel ( Figure 2B Two example images of the driver's footwell area 20 taken by camera 14.

[0056] Camera 14 is configured to send acquired imaging data to processing unit 16. Processing unit 16 is configured to analyze the sent imaging data, detect objects within vehicle 10 based on the analyzed sensor data, classify the detected objects as potentially hazardous objects, and trigger output unit 18 to issue a warning if the detected objects are classified as potentially hazardous objects. The warning may specifically be a visual warning or an audible warning. For example, the warning may be displayed on a display (not shown) within vehicle 10, or it may be output through an audio system (not shown) of vehicle 10. Output unit 18 warns or informs the driver about the potential danger posed by objects identified inside vehicle 10.

[0057] Figures 3A to 3F Further examples of images captured by camera 14 are shown, including a potentially dangerous object 24 located in the driver's footwell area 20. Figures 3A to 3C An RGB image of the damaged floor or footpad in the driver's footrest area 20 is shown. (Folded) Figure 3A and Figure 3B ) or misplaced ( Figure 3C The floor mat may pose a danger to the safe operation of vehicle 10, and is therefore classified as a potential hazard 24 by system 12. Figure 3D and Figure 3E A 3D ToF image of the damaged floor mats within the driver's footwell area 20 is shown; these mats are also classified as potential hazards 24 by system 12. Therefore, Figure 3D The ToF depth image represents the driver's footwell area 20, and Figure 3E This represents the ToF amplitude image. Figure 3F The image shows RGB images of plastic toys and folded floor mats within the driver's footwell area 20, which would affect the driver's ability to drive safely and are therefore classified as potential hazards 24 by system 12.

[0058] The processing unit 16 may be part of a dedicated electronic control unit (ECU) for the vehicle 10. The processing unit 16 may also be part of a multi-domain controller, or it may be a component of the sensor unit 14, such as a smart camera. In other embodiments, the processing unit 16 may also be part of a cloud setup external to the vehicle 10.

[0059] The processing unit 16 may, for example, use neural networks and / or support vector machines (SVM) and / or image processing tools to analyze the image data acquired by the camera 14, detect objects and classify them as potentially dangerous objects 24.

[0060] Processing unit 16 can classify an object as a potential hazardous object 24 based on its detected location, detected object movement, and / or the assigned hazard level. Detected objects can be classified as potential hazardous objects based on their detected location (e.g., near the footrest of vehicle 10). The position of an object in image data can also be tracked over a certain time period to check for loose objects, such as those rolling around the driver's footrest area 20. An object can also be classified as a potential hazardous object 24 independently of its location or movement by assigning a predetermined hazard level to certain categories of objects.

[0061] Additionally, the processing unit 16 identifies the driver's footwear 26 in the image data, and then classifies it as suitable for driving or unsuitable for driving, and as a potentially dangerous object 24, based on the type of footwear 26 and / or the degree of dirtiness of the footwear 26. The processing unit 16 locates the driver's feet in the driver's footrest area 20, identifies the driver's footwear 26 in the area 28 covered by the driver's feet, and classifies the driver's footwear 26 into two categories: "safe for driving" or "unsafe for driving". Figure 4 It shows in Figure 2B The processing unit 16 identifies an example of an area 28 covered by the driver's feet in an RGB image of the driver's footrest area 20. In this specific case, the processing unit 16 classifies the identified driver's footwear 26 as "safe for driving" and therefore not a potentially dangerous object.

[0062] Figure 5 A flowchart is shown of an embodiment of a computer-implemented method for detecting a potentially dangerous object 24 within a vehicle 10, employing another embodiment of system 12. This embodiment is similar to the previous embodiment, but includes two cameras instead of one. The first camera 14A (in...) Figure 5 The sensor referred to in the text as an "imaging sensor (footrest space)" monitors the driver's footrest space area 20, similar to... Figure 1B Camera 14 in the embodiment. Second camera 14B (in Figure 5 The image sensor (referred to as "the imaging sensor (door)") is visible over the driver's door area and is visible to the outside of the vehicle 10 when the driver's door is open. The second camera 14B may additionally monitor the driver's footwell area 20. Alternatively, the first camera 14A or the second camera 14B may be used alone, or additional cameras may be included to monitor further areas of the interior of the vehicle 10.

[0063] The first camera 14A acquires image data used to identify potentially hazardous objects 24 within the driver's footwell area 20. The second camera 14B acquires additional image data of a person entering or about to enter and operate the vehicle 10. The second camera 14B can be positioned such that the driver's footwear 26 is visible at least during the driver's entry into the vehicle 10. This ensures that the image data from the second camera 14B can be used to identify the driver's footwear 26.

[0064] Reference Figure 5 The flowchart shows that the second camera 14B monitors the interior of the vehicle 10, including the driver's door area. If the door of the vehicle 10 is open, a sensor (not shown) detects the door opening and triggers the processing unit 16 to perform body key point detection on the acquired image. After detecting the body key points, skeleton classification is performed to identify personnel outside the vehicle 10. The body key point detection and skeleton classification steps can also be combined into a single step that performs both tasks simultaneously. Figure 6 A sequence of images from the second camera 14B is depicted, showing a person entering the vehicle 10. At the top row, key body points identified by the processing unit 16 are visualized; at the bottom row, the area covered by the detected person's feet is visualized. If the detected person is near the driver's door, a sub-image of the person's foot area 28 is created, and this sub-image is used to identify and classify the person's footwear 26. The classification results are stored for later use. Image data from the second camera 14B tracks the position of the person outside the vehicle 10 to detect whether the person has entered the driver's seat. As the person is assigned a driver's identity, the system checks the stored classification of their footwear 26 to determine if the footwear 26 corresponds to a dangerous footwear type. If so, the driver's footwear 26 is classified as unsuitable and identified as a potential hazard 24. Therefore, the processing unit 16 triggers the output unit 18 to issue a warning to the driver. Classifying footwear 26 as a potential hazard 24 can be based on the type of footwear 26, but also on its condition of being soiled with mud, sludge, moisture, snow, etc., which reduces its suitability for operating the vehicle 10. In this regard, the output unit 18 can also warn the driver that their footwear 26 is dirty before they enter the vehicle 10, as this could soil the driver's footrest area 20 and create a dangerous situation during driving.

[0065] exist Figure 5 In the right branch of the flowchart, the first camera 14A monitors the driver's footrest space area 20. As a first step in detecting potential hazards 24, it determines in the image data whether the driver's footrest space area 20 is empty, or whether a foot can be identified in the image.

[0066] If no feet are detected, the driver's footrest space 20 is considered empty. Then, the processing unit 16 continues to identify objects in the image of the driver's footrest space area 20 and classifies the identified objects. If the identified object is classified as a potentially dangerous object 24, the processing unit 16 will trigger the output unit 18 to issue a warning to the driver.

[0067] If a foot is present in the image data of the driver's footrest area 20, the processing unit 16 can locate the area 28 covered by the driver's foot and mask that area in the image of the first camera 14A. Then, the masked image ignoring the area 28 covered by the driver's foot is used to identify and classify objects in the driver's footrest area 20.

[0068] To categorize the driver's footwear 26, images of the occupied and therefore non-empty footrest space 20 are used to identify the area 28 covered by the driver's feet, similar to... Figure 4 The process is illustrated. A sub-image of the driver's foot region 28 can then be created and used to identify and classify the driver's footwear 26. If the classification involves a dangerous type of footwear, i.e., footwear 26 unsuitable for driving, the processing unit 16 triggers the output unit 18 to output a warning to the driver. The classification of footwear 26 can be based on the type of footwear 26, but can also be based on conditions such as mud, sludge, moisture, snow, etc., which reduce the suitability of the footwear 26 for operating the vehicle 10.

[0069] System 12 and its method can reliably detect and classify potentially hazardous objects within a vehicle and warn the driver of such objects or abnormal situations. Therefore, this system and method can improve the driver's ability to safely operate the vehicle and contribute to safer vehicle operation.

Claims

1. An advanced driver assistance system (12) for a vehicle, the system being configured to detect potentially hazardous objects within the vehicle (10), the system (12) comprising: At least one sensor unit (14, 14A, 14B), Processing unit (16), and Output unit (18), The sensor units (14, 14A, 14B) are configured to monitor at least a portion of the interior of the vehicle (10) and transmit the acquired sensor data to the processing unit (16). The processing unit (16) is configured to: Analyze the transmitted sensor data. Detecting objects within the vehicle based on the analyzed sensor data. The detected objects were classified as potentially hazardous objects (24), and If the detected object is classified as a potentially dangerous object (24), the output unit (18) is triggered to issue a warning.

2. The system (12) according to claim 1, wherein, The monitored portion of the interior of the vehicle (10) includes at least the driver's footrest area (20).

3. The system (12) according to claim 1 or 2, wherein, The processing unit (16) is configured to classify the object as a potentially hazardous object (24) based on the detected location of the object, and / or the detected movement of the object, and / or the assigned hazard level of the object.

4. The system (12) according to claim 2, wherein, The potential hazardous object (24) is an object in the driver's footrest area (20), and / or footwear (26) unsuitable for driving, and / or a damaged floor or foot mat.

5. The system (12) according to claim 4, wherein, The object in the footrest area (20) of the driver is a detached object.

6. The system (12) according to claim 1, wherein, The sensor units (14, 14A, 14B) include at least one camera device configured to provide multiple images of the monitored portion of the interior of the vehicle (10).

7. The system (12) according to claim 6, wherein, The camera device is a 2D NIR camera and / or a 2DNIR RGB camera and / or a 3D time-of-flight camera or a 3D stereo camera.

8. The system (12) according to claim 1, wherein, The processing unit (16) is configured to use neural networks and / or support vector machines and / or other machine learning methods and / or image processing tools to analyze the sensor data, detect objects, and classify the objects as potentially hazardous objects (24).

9. The system (12) according to claim 1, wherein, The processing unit (16) is configured to identify the driver and the area (28) covering the driver's feet in the sensor data.

10. The system (12) according to claim 9, wherein, The processing unit (16) is configured to shield sensor data covering the area (28) of the driver’s feet and ignore the sensor data when a detected object is identified and the detected object is classified as a potentially dangerous object (24).

11. The system (12) according to any one of claims 4 to 10, wherein, The processing unit (16) is configured to identify the driver's footwear (26) in the sensor data and classify the driver's footwear (26) as suitable for driving or unsuitable for driving and as a potentially dangerous object (24) based on the type of the footwear (26) and / or the degree of dirtiness of the footwear (26).

12. The system (12) according to claim 1, wherein, The sensor units (14, 14A, 14B) are configured to monitor at least the driver's door.

13. The system (12) according to claim 12, wherein, The processing unit (16) is configured to identify a driver entering the vehicle (10) based on data from the sensor units (14, 14A, 14B), identify the area (28) covering the driver's feet, identify the driver's footwear (26) in the sensor data, and classify the driver's footwear (26) as suitable for driving or unsuitable for driving.

14. Use of a system (12) according to any one of claims 1 to 13 for detecting a potentially dangerous object (24) within a vehicle (10).

15. A computer-implemented method for detecting a potentially dangerous object (24) within a vehicle (10), the method comprising the steps of: - At least a portion of the interior of the vehicle (10) is monitored by at least one sensor unit (14, 14A, 14B), and the acquired sensor data is sent to the processing unit (16); and the processing unit (16) performs the following steps: - Analyze the transmitted sensor data; - Detect objects within the vehicle (10) based on the analyzed sensor data; - Classify the detected objects as potentially hazardous objects (24); and - If the detected object is classified as a potentially dangerous object (24), the output unit is triggered to issue a warning.

Images