Method and system for detecting a seatbelt status of a vehicle seat

Abstract

The present invention discloses a method for detecting a seatbelt status of a vehicle seat occupied by an occupant, particularly for detecting improper usage of the seatbelt, having an electronic cont

Description

Description Method and system for detecting a seatbelt status of a vehicle seat The invention relates to a method for detecting a seatbelt status of a vehicle seat occupied by an occupant in a vehicle, particularly for detecting improper usage of the seatbelt. The invention further relates to a system for detecting a seatbelt status of a vehicle seat occupied by an occupant in a vehicle, particularly for detecting improper usage of the seatbelt. The safety of occupants in vehicles assumes an important role in the development of vehicles. Safety approaches frequently include several measures for increasing driving safety, from a structural protection of the vehicle cabin, to occupant restraint devices like airbags and seatbelt assemblies, to an automated transmission of emergency calls with the aim of being able to better protect occupants in the vehicle in case of an accident. Seatbelt assemblies of seats in vehicles are well known and usually comprise a belt strap in a lap-and-diagonal belt arrangement (or “three-point” belt) with one end of the belt strap being anchored to part of the vehicle, an intermediate part of the belt strap being provided with a tongue releasably engageable with affixed buckle, and the other end of the belt strap, which passes through a belt strap guide designed as a pillar-loop or a so-called D-ring, being connected to a belt retractor. So, the belt strap is usually divided into or includes a lap portion intended to be worn across the occupant’s lap and a shoulder portion intended to be worn across the occupant's upper torso. For seatbelt assembly to be effective, seatbelts must be worn as intended. In order to encourage vehicle occupants to wear seatbelts, vehicles normally include warning chimes reminding occupants or the driver if seatbelts are not fastened. For this, there are known methods and systems for detecting a seatbelt status which usually monitor and determine the latched or unlatched status of the seatbelt based on a sensor or switch attached to the buckle. However, it cannot be assumed that a seatbelt is adequately fulfilling its intended use merely because the seatbelt is latched, and furthermore, occupants may be able to circumvent these safety feature. For example, even while a seatbelt is latched, an occupant is still able to remove the shoulder portion of the belt strap and put it behind his or her back. If an occupant wishes to permanently disable the warning feature, the occupant may leave an aftermarket tongue designed to match the shape of the original inserted into the buckle, thus completely circumventing the warning feature. In all of these cases, the driver or occupant will not be warned because the seatbelt is technically "fastened". So, current monitoring methods and systems don't include information on the spatial position of the seatbelt and cannot determine whether an occupant is properly using the seatbelt. In view of the foregoing, it is an object of the present invention to provide an improved method and an improved system for detecting a seatbelt status of a vehicle seat, which ensure a reliable and robust determination of whether an occupant is properly using the seatbelt. According to one aspect of the invention, a method for detecting a seatbelt status of a vehicle seat occupied by an occupant in a vehicle, particularly for detecting improper usage of the seatbelt, is provided, wherein the vehicle comprises an electronic control device, a seatbelt assembly including a belt strap and an interior sensor system comprising an image sensor system and a seatbelt sensor system. The method comprising: - providing to the electronic control device first sensor data from the image sensor system and determining, based on the first sensor data, first status data, - providing to the electronic control device second sensor data from the seatbelt sensor system and determining, based on the second sensor data, second status data, - assigning a first weighting factor to the first status data to obtain a weighted first status data, the first weighting factor indicating a level of reliability of the first status data, - assigning a second weighting factor to the second status data to obtain a weighted second status data, the second weighting factor indicating a level of reliability of the second status data, - comparing the weighted first status data with the weighted second status data, and - determining a classification of the seatbelt status based on the comparison. By such a method with the combination of the sensor data from different sensor systems and the respective weightage according to the reliability of the data of the respective sensor system, a reliable monitoring of the seatbelt in different situations and under different surrounding conditions is enabled, whereby particularly different seatbelt faking scenarios can be detected. Thus, in accordance with the present invention, an improved method for detecting a seatbelt status of a vehicle seat is provided, which ensure a reliable and robust determination of whether an occupant is properly using the seatbelt. The electronic control device can be designed for example as an airbag control unit. Preferably, the first sensor data, the second sensor data, the first status data and / or the second sensor data are recorded over a period of time as a pattern or time series data. According to an advantageous specific embodiment, the step of determining the first status data comprises: - capturing, by the image sensor system, images of at least a portion of the occupant and determining an occupant posture of the occupant based on the captured images, - determining a first seatbelt rollout length based on the occupant posture, an occupant size, a position of the vehicle seat and / or a tilt angle of the backrest of the vehicle seat, and - tracking a change in the first seatbelt rollout length over time. So, the first seatbelt rollout length is determined based on at least one of the occupant posture, an occupant size, a position of the vehicle seat, or a tilt angle of the backrest of the vehicle seat. Preferably, the first seatbelt rollout length is determined based on the occupant posture, an occupant size, a position of the vehicle seat and a tilt angle of the backrest of the vehicle seat. In a preferred embodiment, first, a length of the belt strap is determined based on the occupant posture and the occupant size, wherein the first seatbelt rollout length is then determined based on the determined length of the belt strap, and optionally furthermore on at least one of a position of the vehicle seat, or a tilt angle of the backrest of the vehicle seat. In particular, the tilt angle represents an inclination or angle of the backrest relative to a horizontal line through the seat cushion of the vehicle seat and the position of the vehicle seat comprises a longitudinal position of the vehicle seat on a horizontal seat guide rail and / or a height of seat cushion of the vehicle seat Thereby, according to another specific embodiment, the first status data further comprises a change in occupant posture, wherein the step of determining the first status data further comprises: - tracking a change in the first seatbelt rollout length with a change in occupant posture over time. According to another specific embodiment, the step of determining the second status data comprises: - determining a second seatbelt rollout length based on signals from a belt retractor sensor, wherein the belt retractor sensor is arranged at a belt retractor of the seatbelt assembly and adapted to generate signals representative of the amount of the belt strap rolled out from the belt retractor, and - tracking a change in the second seatbelt rollout length over time. Thereby, according to another specific embodiment, the second status data further comprises a change in an angle of a D-ring of the seatbelt assembly, wherein the step of determining a second status data further comprises: - determining an angle of the D-ring based on signals from a D-ring angle sensor associated with the D-ring, and - tracking a change in the second seatbelt rollout length with a change in the angle of the D-ring over time. In the context of the present invention, the term “D-ring” is intended to mean a deflection and or guiding device of any design for the shoulder portion of the belt strap. According to another specific embodiment, the first weighting factor and / or the second weighting factor are dynamically adjusted based on the light condition in the vehicle interior. Preferably, the first weighting factor differs from the second weighting factor. For example, in a poor light condition in the vehicle interior, the first weighting factor is lowered and the second weighting factor is increased so that the second status data resulting from the second sensor data from the seatbelt sensor system is weighted more heavily than the first status data resulting from the first sensor date from the image sensor system. Thereby, according to another specific embodiment, the light condition is determined based on the signals from ambient light sensor and / or a sun load sensor arranged in the vehicle interior. According to another specific embodiment, the image sensor system comprises a 2D camera and a 3D camera. The 2D camera is preferably designed as an IR camera which uses infrared emitters in order to recognize and detect at least the occupant at all and as well as possible at night or in poor light conditions. The 3D camera preferably operates according to the time of flight method and allows detection of the occupant and / or the seatbelt based on actual movement vectors in three-dimensional space. Preferably the 3D camera and the 2D camera are interior cameras situated in the vehicle, wherein the 3D camera and the 2D camera particularly are integrated in the roof lining between the vehicle front seats and / or in an area close to the rear-view mirror. Particularly, the 3D camera and the 2D camera can form a single structural camera unit. According to another specific embodiment, the step of determining an occupant posture comprises: - detecting of key points of the occupant based on the captured images, and - ascertaining the occupant posture based on the connection of the detected key points to a skeleton-like representation of body parts of the occupant, wherein the skeleton-like representation reflects the relative position and orientation of individual body parts of the occupant. Thereby, according to another specific embodiment, the image sensor system comprises an 2D camera and a 3D camera, wherein the detecting of key points of the occupant is made in using sensor data from the 2D camera and the 3D camera. Thereby, according to another specific embodiment, 2D key points of the occupant detected by the 2D camera are converted into 3D key points of the occupant by fusing the information provided by the 3D camera, wherein preferably the ascertaining of the posture of the occupant is based on the 3D key points of the occupant. According to another specific embodiment, the key points of the occupant are the occupant skeleton joints. Such key points allow a very accurate ascertaining of the occupant posture since they are highly relevant and informative regarding the position, orientation and movement of individual body parts. According to another aspect of the invention, a system for detecting a seatbelt status of a vehicle seat occupied by an occupant in a vehicle, particularly for detecting improper usage of the seatbelt, is provided, comprising a seatbelt assembly including a belt strap, an interior sensor system including an image sensor system and a seatbelt sensor system, and an electronic control device, wherein the system is configured to perform a method according to the invention. The advantages and the specific and preferred embodiments described for the method according to the invention also apply accordingly to the system according to the invention. The drawings described herein are used to provide a further understanding for the present invention, and constitute a part of the present invention. The exemplary embodiments and descriptions of the present invention are used to explain the present invention, and do not constitute an improper limitation to the present invention. In the drawings: FIG. 1 shows in a schematic diagram a vehicle comprising a system for detecting a seatbelt status of a vehicle seat occupied by an occupant in a vehicle according to a preferred embodiment of the invention; and FIG. 2 shows a flow chart of a method for detecting a seatbelt status of a vehicle seat occupied by an occupant in the vehicle according to a preferred embodiment of the invention carried out by the system shown in FIG. 1. The following describes the present invention in detail with reference to the accompanying drawings and in combination with embodiments. It should be noted that, without conflicts, the embodiments in the present invention and features in the embodiments may be combined with each other. Parts corresponding to each other are always provided with the same reference signs in all figures. FIG. 1 shows in a very schematic diagram a vehicle 1 comprising a system 2 for detecting a seatbelt status of a vehicle seat occupied by an occupant in the vehicle 1, particularly for detecting improper usage of the seatbelt. The system 2 comprises seatbelt assembly including a belt strap, a seatbelt buckle, a belt retractor and a D-ring, an interior sensor system 3 for sensing an interior state and an electronic control device 4 connected to the interior sensor system 3. The electronic control device 4 can be designed for example as an airbag control unit. The interior sensing system 3 comprises an image sensor system 5 and a seatbelt sensor system 6. The image sensor system 5 includes a 2D camera 7 and a 3D camera 8. The 2D camera 7 is designed as an IR camera which uses infrared emitters in order to recognize and detect at least the occupant at all and as well as possible at night or in poor light conditions. The 3D camera 8 operates according to the time of flight method and allows detection of the occupant and / or the seatbelt based on actual movement vectors in three-dimensional space. The seatbelt sensor system 6 includes a belt retractor sensor 9 arranged at the belt retractor of the seatbelt assembly and adapted to generate signals representative of the amount of the belt strap rolled out from the belt retractor. The seatbelt sensor systems 6 also includes a D-ring angle sensor 10 associated with the D-ring for determining an angle of the D-ring. The system 2 further comprises a buckle status sensor 11 for detecting the status of the seatbelt buckle and an ambient light sensor 12 arranged in the vehicle interior for determining the light condition in the vehicle interior. The buckle status sensor 11 and the ambient light sensor 12 are also connected to the electronic control device 4. The buckle status sensor 11 and / or the ambient light sensor 12 can also be part of the seatbelt sensor system 6 and / or the image sensor system 5. The sensor data acquired by the image sensor system 5, the seatbelt sensor system 6, the buckle status sensor 11 and the ambient light sensor 12 is forwarded to the electronic control device 4 for further processing. The method 100 carried out by the system 2 is shown and described in in FIG. 2. FIG. 2 shows a flow chart of a method 100 for detecting a seatbelt status of a vehicle seat occupied by an occupant in the vehicle 1 according to a preferred embodiment of the invention carried out by the system 2 shown in FIG 1. In a step 110, when it is detected that the seatbelt is buckled, i.e., that the tongue of the seatbelt is engaged with the seatbelt buckle, first sensor data is provided to the electronic control device 4 from the image sensor system 5, and second sensor data is provided to the electronic control device 4 from the seatbelt sensor system 6. In a step 120, the electronic control device 4 determines a first status data based on the first sensor data. Therefore, first, an occupant posture and an occupant size of the occupant are determined based on the captured images. Thereby, skeleton joints of the occupant are detected as key points of the occupant based on the captured images, wherein 2D key points of the occupant detected based on the sensor data provided by the 2D camera are converted into 3D key points of the occupant 7 by fusing the sensor data provided by the 3D camera. The occupant posture and the occupant size are then ascertained based on the connection of the detected 3D key points to a skeleton-like representation of body parts of the occupant, wherein the skeleton-like representation reflects the relative position and orientation of individual body parts of the occupant. Then, a first seatbelt rollout length is determined based on the occupant posture, the occupant size, a position of the vehicle seat and a tilt angle of the backrest of the vehicle seat on which the occupant is sitting. Alternatively, first, a length of the belt strap is determined based on the occupant posture and the occupant size, wherein the first seatbelt rollout length is then determined based on the determined length of the belt strap, and furthermore on the position of the vehicle seat and the tilt angle of the backrest of the vehicle seat. The tilt angle represents an inclination or angle of the backrest relative to a horizontal line through the seat cushion of the vehicle seat and the position of the vehicle seat comprises a longitudinal position of the vehicle seat on a horizontal seat guide rail and a height of seat cushion of the vehicle seat. The position of the vehicle seat and the tilt angle of the backrest of the vehicle seat may, for example, have been determined in advance based on sensor data of appropriate sensors arranged on the seat or on sensor data of the image sensor system 5 and transmitted to the electronic control device 4. A change in the first seatbelt rollout length is tracked with a change in occupant posture over time, and this is determined to be the first status data. In this step, particularly before the determination of the occupant posture and the occupant size of the occupant, also the light condition in the vehicle interior is determined based on the signals from the ambient light sensor 12. In particular, in case of determined good light conditions where a seatbelt detection directly from the captured images is reliably possible, the first seatbelt rollout length is alternatively determined based on the position of the vehicle seat, the tilt angle of the backrest of the vehicle seat and a belt strap length directly determined from the captured images, so, not based on the occupant posture and the occupant size, whereas in case of determined poor light conditions or no light conditions, where particularly a occupants posture detection is reliable, but a seatbelt detection is improper, the first seatbelt rollout length is determined based on the occupant posture, the occupant size, the position of the vehicle seat and the tilt angle of the backrest of the vehicle seat. In a step 130, the electronic control device 4 determines a second status data based on the second sensor data. Therefore, a second seatbelt rollout length is determined based on signals from the belt retractor sensor 9. Further, an angle of the D-ring is determined based on signals form the D-ring angle sensor 10. A change in the second seatbelt rollout length is tracked with a change in the angle of the D-ring over time, and this is determined to be the second status data. In a step 140, a first weighting factor is assigned to the first status data to obtain a weighted first status data, the first weighting factor indicating a level of reliability of the first status data. The first weighting factor is dynamically determined or adjusted based on the light condition in the vehicle interior which is determined based on the signals from the ambient light sensor 12. In a step 150, a second weighting factor is assigned to the second status data to obtain a weighted second status data, the second weighting factor indicating a level of reliability of the second status data. The second weighting factor is also dynamically determined or adjusted based on the light condition in the vehicle interior which is determined based on the signals from the ambient light sensor 12. Thereby, in a poor light condition in the vehicle interior, the first weighting factor is lowered and the second weighting factor is increased so that the second status data resulting from the second sensor data from the seatbelt sensor system 6 is weighted more heavily than the first status data resulting from the first sensor date from the image sensor system 5. In a step 160, the weighted first status data is compared with the weighted second status data and based on this comparison a classification of the seatbelt status is determined, wherein the comparison is preferably done based on a pattern matching. Thereby, depending on the result of the comparison, the seatbelt status is classified as “waring proper”, “wearing improper” or “not wearing”. In a step 170, depending on the determined classification, a specific action is triggered by the electronic control device 4. In the case that the seatbelt status was classified as “waring proper”, a seatbelt warning is disabled. In the case that the seatbelt status was classified as “wearing improper”, a seatbelt warning is activated and / or an airbag associated to the vehicle seat on which the occupant is sitting is disabled. In the case that the seatbelt status was classified as “not wearing”, a seatbelt warning is activated and / or an airbag associated to the vehicle seat on which the occupant is sitting is disabled. In this way, with the combination of the sensor data from the different sensor systems and the respective weightage according to the reliability of the data, a reliable monitoring of the seatbelt in different situations and under different surrounding conditions is enabled, whereby particularly different seatbelt faking scenarios can be detected. Thus, a reliable and robust determination of whether an occupant is properly using the seatbelt is ensured.

Claims

Patent claims 1. A method (100) for detecting a seatbelt status of a vehicle seat occupied by an occupant in a vehicle (1), particularly for detecting improper usage of the seatbelt, having an electronic control device (4), a seatbelt assembly including a belt strap and an interior sensor system (3) comprising an image sensor system (5) and a seatbelt sensor system (6), the method (100) comprising: - providing to the electronic control device (4) first sensor data from the image sensor system (5) and determining, based on the first sensor data, first status data, - providing to the electronic control device (4) second sensor data from the seatbelt sensor system (6) and determining, based on the second sensor data, second status data, - assigning a first weighting factor to the first status data to obtain a weighted first status data, the first weighting factor indicating a level of reliability of the first status data, - assigning a second weighting factor to the second status data to obtain a weighted second status data, the second weighting factor indicating a level of reliability of the second status data, - comparing the weighted first status data with the weighted second status data, and - determining a classification of the seatbelt status based on the comparison.

2. The method (100) according to claim 1, wherein the step of determining the first status data comprises: - capturing, by the image sensor system (5), images of at least a portion of the occupant and determining an occupant posture of the occupant based on the captured images, - determining a first seatbelt rollout length based on the occupant posture, an occupant size, a position of the vehicle seat and / or a tilt angle of the backrest of the vehicle seat, and - tracking a change in the first seatbelt rollout length over time.

3. The method (100) according to claim 2, wherein the first status data further comprises a change in occupant posture, and wherein the step of determining the first status data further comprises: - tracking a change in the first seatbelt rollout length with a change in occupant posture over time.

4. The method (100) according to one of the preceding claims, wherein the step of determining the second status data comprises: - determining a second seatbelt rollout length based on signals from a belt retractor sensor (9), wherein the belt retractor sensor (9) is arranged at a belt retractor of the seatbelt assembly and adapted to generate signals representative of the amount of the belt strap rolled out from the belt retractor, and - tracking a change in the second seatbelt rollout length over time.

5. The method (100) according to claim 4, wherein the second status data further comprises a change in an angle of a D-ring of the seatbelt assembly, and wherein the step of determining a second status data further comprises: - determining an angle of the D-ring based on signals from a D-ring angle sensor (10) associated with the D-ring, and - tracking a change in the second seatbelt rollout length with a change in the angle of the D-ring over time.

6. The method (100) according to one of the preceding claims, wherein the first weighting factor and / or the second weighting factor are dynamically adjusted based on the light condition in the vehicle interior. / . The method (100) according to claim 6, wherein the light condition is determined based on the signals from ambient light sensor (12) and / or a sun load sensor arranged in the vehicle interior.

8. The method (100) according to one of the preceding claims, wherein the image sensor system (5) comprises a 2D camera (7) and a 3D camera (8).

9. The method (100) according to claim 2, wherein the step of determining an occupant posture comprises: - detecting of key points of the occupant based on the captured images, and - ascertaining the occupant posture based on the connection of the detected key points to a skeleton-like representation of body parts of the occupant, wherein the skeleton-like representation reflects the relative position and orientation of individual body parts of the occupant.

10. The method (100) according to claim 9, wherein the image sensor system (5) comprises an 2D camera (7) and a 3D camera (8), and wherein the detecting of key points of the occupant is made in using sensor data from the 2D camera (7) and the 3D camera (8).

11. The method (100) according to claim 10, wherein 2D key points of the occupant detected by the 2D camera (7) are converted into 3D key points of the occupant by fusing the information provided by the 3D camera (8), and wherein preferably the ascertaining of the posture of the occupant is based on the 3D key points of the occupant.

12. The method (100) according to one of claims 9 to 11, wherein the key points of the occupant are the occupant skeleton joints.

13. A system (2) for detecting a seatbelt status of a vehicle seat occupied by an occupant in a vehicle (1), particularly for detecting improper usage of the seatbelt, comprising a seatbelt assembly including a belt strap, an interior sensor system (3) including an image sensor system (5) and a seatbelt sensor system (6), and an electronic control device (4), wherein the system (2) is configured to perform a method (100) according to one of the preceding claims.

Images