Method and device for determining the position of a vehicle occupant in a vehicle

Abstract

In the inventive method for determining the position of a vehicle occupant (21) in a vehicle, an initial value for the belt extension length of a seat belt (23) fastened by the vehicle occupant is determined (10), wherein the initial value corresponds to the rest position of the fastened seat belt when the vehicle occupant is in a resting position on the seat (22). An increase in the belt extension length of the seat belt (23) is detected from the rest position of the fastened seat belt (11) when the vehicle occupant moves from the resting position. A value for a radial deviation of the vehicle occupant's position from the resting position is determined from the detected increase in belt extension length (12). The vehicle occupant is detected with a 2D camera (25) (13). At least one feature of the vehicle occupant is extracted from the camera images of the 2D camera (25) (14).A two-dimensional position of the vehicle occupant is determined from the extracted features of the vehicle occupant (15). A three-dimensional position of the vehicle occupant (21) in the vehicle coordinate system of the vehicle is determined from the determined two-dimensional position of the vehicle occupant and the value for the radial deviation of the position of the vehicle occupant from the rest position (16).

Description

[0001] The present invention relates to a method for determining the position of a vehicle occupant in a vehicle. The present invention further relates to a device for carrying out the method, a computer program with instructions that cause a device according to the invention to execute the steps of the method, and a vehicle that is configured to carry out such a method or that includes such a device.

[0002] To ensure the passive safety of vehicle occupants, particularly to protect them in the event of an accident, restraint systems are incorporated into the interiors of modern land, air, and sea vehicles. For example, in motor vehicles, seat belts and airbags serve to secure occupants in their seats, thereby preventing or at least reducing injuries during an accident. In modern vehicles, the selection of the restraint strategy for these systems during an accident, such as the deployment of airbags and seat belt pretensioners, relies exclusively on crash sensors. These sensors allow for an assessment of the type and severity of the accident.

[0003] The design of restraint systems in motor vehicles is based on the so-called "50% male," which is derived from the average height, weight, and body measurements of the male population. However, there are already approaches that aim to offer improved protection by adapting the restraint systems to the individual vehicle occupants. For example, adaptive and multi-stage airbags and seatbelt pretensioners, which adjust individually to occupants and accident situations, are already under development. In addition to suitable actuators for the restraint systems, sensors are also required to detect the occupant characteristics.

[0004] Even with a seatbelt force limiter, which in a collision restricts the restraining force exerted by the seatbelt on the occupant to tolerable limits, there are approaches to adaptively adjust the force exerted by the belt, from the so-called 5% female to the 95% male. For this purpose, the seat's longitudinal position is evaluated using sensors located on the seat rails, which determine whether the seat is positioned forward or backward.

[0005] KR 2007 0076202 A discloses a control method for a driver's seat memory system in which a camera in the vehicle is used to capture the driver's face and store the captured image data along with the driver's seat position data. Subsequently, by selecting the displayed driver image data, the stored position data can be retrieved as preset values. The driver's seat can then be automatically adjusted to the preset values ​​without the driver having to remember a specific memory number of a memory function of the driver's seat memory system.

[0006] German patent DE 10 2019 004 864 A1 describes a method for estimating the weight and height of a motor vehicle driver. With the driver seated and wearing a seatbelt, the belt extension length is determined. Furthermore, with the driver's hands on the steering wheel, the driver's seat is adjusted until the driver assumes a posture within a predetermined range. Based on the belt extension length and the driver's seat position, the driver's weight and height are estimated. Additionally, the method may include using an interior camera to capture and measure the driver's posture or face to draw further conclusions about the driver's weight.

[0007] It is an object of the invention to provide an improved method and an improved device for operating a restraint system in a vehicle.

[0008] This problem is solved by the independent claims. Preferred embodiments of the invention are the subject of the dependent claims.

[0009] In the inventive method for determining the position of a vehicle occupant in a vehicle, - an initial value for the belt extension length of a seat belt fitted by the vehicle occupant is determined, wherein the initial value corresponds to a rest position of the fitted seat belt when the vehicle occupant is in a resting position on the seat; - an increase in the belt extension length of the seat belt from the rest position of the fastened seat belt when the vehicle occupant moves from the rest position; - a value for a radial deviation of the vehicle occupant's position from the rest position is determined from the recorded increase in the belt extension length; - the vehicle occupant is captured with a 2D camera; - at least one feature of the vehicle occupant extracted from the camera images of the 2D camera; - a two-dimensional position of the vehicle occupant is determined from the extracted features of the vehicle occupant; and - a three-dimensional position of the vehicle occupant in the vehicle coordinate system of the vehicle is determined from the determined two-dimensional position of the vehicle occupant and the value for the radial deviation of the position of the vehicle occupant from the rest position.

[0010] This enables a robust estimation of the vehicle occupant's body position within the vehicle's three-dimensional coordinate system, which can be used for various applications within the vehicle. In particular, a cost-effective 2D camera can be used to determine the three-dimensional body position; the use of a 3D camera system, which would entail higher costs, lower resolution, or demanding hardware requirements, is unnecessary. Furthermore, it may be possible to utilize components already installed in the vehicle for other functions, which further reduces the costs of implementing the functionality according to the invention.

[0011] In particular, a restraint system can be adapted to the specific position of the vehicle occupant. This allows for optimization of the restraint strategy of the implemented restraint system and thus offers improved protection for the respective vehicle occupant.

[0012] According to one embodiment of the invention, the position of the seat on which the vehicle occupant is located is additionally determined and taken into account when determining the three-dimensional position of the vehicle occupant.

[0013] In particular, the vehicle occupant is in a resting position on the seat when he or she is sitting in the middle of a seat surface and the back of the body is against the backrest.

[0014] Advantageously, a two-dimensional position of the vehicle occupant in a reference plane perpendicular to the optical axis of the 2D camera is determined from the camera images of the vehicle occupant captured by the 2D camera.

[0015] It is particularly advantageous if, in addition to the three-dimensional position of the vehicle occupant, the orientation of the vehicle occupant relative to the interior of the vehicle is also determined.

[0016] Preferably, the three-dimensional position of the vehicle occupant corresponds to the head position of the vehicle occupant, whereby anatomical features on the head of the vehicle occupant are extracted from the camera images of the vehicle occupant captured with the 2D camera using image analysis, in order to then determine the head position.

[0017] Accordingly, a device according to the invention for determining the position of a vehicle occupant in a vehicle comprises - a sensor for detecting the belt extension length of a seat belt worn by the vehicle occupant, which, starting from an initial value for the belt extension length of the seat belt corresponding to a rest position of the worn seat belt when the vehicle occupant is in a rest position on the seat, detects an increase in the belt extension length when the vehicle occupant moves from this rest position; - a 2D camera for detecting the vehicle occupant; and - an evaluation unit to which the sensor data generated by the sensor and camera images generated by the 2D camera are fed, and which - from the recorded increase in the belt extension length, a value for the radial deviation of the vehicle occupant's position from the resting position was determined; - extract at least one feature of the vehicle occupant from the camera images captured by the 2D camera and determine a two-dimensional position of the vehicle occupant from this; and - jointly evaluates the determined two-dimensional position of the vehicle occupant and the value for the radial deviation of the vehicle occupant's position from the rest position in order to determine a three-dimensional position of the vehicle occupant.

[0018] The invention further comprises a computer program with instructions that cause a device according to the invention to carry out the steps of the method according to the invention.

[0019] Finally, the invention also includes a vehicle that is equipped to carry out a method according to the invention or that has a device according to the invention.

[0020] Further features of the present invention will become apparent from the following description and the claims in conjunction with the figures. Fig. Figure 1 shows a flowchart of a method according to the invention; Fig. Figure 2 schematically shows how a camera captures a seat with a seatbelt but without vehicle occupants (a), with vehicle occupants in a resting position and with the seatbelt fastened (b), and with vehicle occupants and the seatbelt fastened outside the resting position (c), and Fig. Figure 3 schematically shows a vehicle occupant in the interior of a motor vehicle with a device according to the invention.

[0021] To better understand the principles of the present invention, embodiments of the invention are explained in more detail below with reference to the figures. It is understood that the invention is not limited to these embodiments and that the described features can also be combined or modified without departing from the scope of protection of the invention as defined in the claims.

[0022] Fig. Figure 1 shows a flowchart of a method according to the invention. The body position of the vehicle occupant determined by the method according to the invention can be used for various applications in the vehicle. For example, infotainment content, such as the spatial output of audio content, and / or the interior climate control can be adjusted based on the current position of the vehicle occupant. In particular, the determined body position can also be used to optimize the restraint strategy of a restraint system implemented in the vehicle, thus providing improved protection for the respective vehicle occupants. The method is described below as an example for the operation of a restraint system in a passenger car, but it can also be applied to other motor vehicles such as vans or buses.

[0023] The process can be started, for example, as soon as the doors of the vehicle are unlocked by a conventional radio key (or a digital car key, which is implemented, for example, by means of a suitable app on a smartphone) or when the radio key is detected approaching the vehicle.

[0024] In process step 10, a baseline value for the webbing extension length of a correctly fastened seat belt is determined for each of the monitored seat belts. This baseline value corresponds to the resting position of the fastened seat belt when the vehicle occupant is in a relaxed position on the seat. This can be done, for example, for the driver's seat when the driver, after unlocking the vehicle and before starting to drive, takes their seat and fastens their seatbelt for the first time.

[0025] By evaluating sensor signals from a seatbelt extension sensor and / or images from an interior camera, it can first be determined whether the seatbelt is fastened. Additionally, the buckle can be checked to ensure the belt is properly engaged. Furthermore, the interior camera can verify that the vehicle occupant is correctly positioned in the seat, i.e., centered and in contact with the backrest. The interior camera can also check whether the vehicle occupant is correctly buckled up, for example, whether the seatbelt strap is positioned over the torso and groin area, and not behind them.

[0026] In a subsequent process step 11, starting from the resting position of the fastened seat belt, an increase in the belt extension length is detected when the vehicle occupant moves from this resting position. For this purpose, a sensor is provided on each seat belt to perform a relative measurement of the belt extension length. However, despite the use of a relative sensor, an absolute measurement of the extension length is also possible if the initial position of the unfastened seat belt is first appropriately detected and evaluated.

[0027] From the recorded increase in the belt extension length, a value for a radial deviation of the vehicle occupant's position from the rest position can then be determined in process step 12, as in connection with Fig. 2 is explained.

[0028] In process step 13, the vehicle occupant is detected by at least one interior camera. The interior camera can be located on the rearview mirror, the vehicle roof, or the upper part of the windshield and can be a 2D camera; a more expensive 3D camera is not required for the method according to the invention. The interior camera can detect the driver's seat and, if present, the front passenger seat of the vehicle, as well as the seat belts attached to each seat. Vehicle occupants are also detected when they are in the driver's or front passenger seat. The vast majority of vehicles have one or more additional rows of seats behind the front row, which can only be partially detected by this first interior camera.If an adaptation of the restraint system is also to be made possible for vehicle occupants located there, an interior camera can be provided for each row of seats, which is then arranged in such a way that the respective row of seats can be recorded as completely as possible.

[0029] The detection of the vehicle occupant can be carried out continuously or only triggered when, in process step 12, a deviation of the vehicle occupant's position from the resting position has been determined from the increase in the belt extension length.

[0030] In process step 14, at least one feature of the vehicle occupant is extracted from the camera images. For this purpose, the camera images are evaluated using suitable image analysis algorithms. This can be done, for example, by a neural network implemented on an analysis unit. To optimize recognition rates, vehicle data such as the position of each seat can be stored in memory and used for the analysis.

[0031] In process step 15, a two-dimensional position of the vehicle occupant can be determined from the extracted characteristics. Through appropriate calibration, the pixels of the camera image are assigned coordinates. This can be done, for example, by the manufacturer before the vehicle is delivered.

[0032] The characteristics of the vehicle occupant can include one or more anatomical features. For example, the occupant's head position can be determined by analyzing the position of extracted anatomical features such as the eyes, nose, mouth, or ears. Alternatively, the entire head or larger parts of the occupant's body can be recorded and analyzed.

[0033] In process step 16, a three-dimensional position of the vehicle occupant within the vehicle's coordinate system can be determined by fusing the calculated two-dimensional position and the value for the radial deviation of the occupant's position from the resting position. Additionally, with suitable analysis, the body pose can also be determined. This allows for adjustments to a vehicle function.

[0034] This allows, in particular, for the adaptation of the vehicle's restraint system, for which knowledge of the position and, if applicable, body pose during the collision phase (crash phase) or the preceding initiation phase (pre-crash phase) is especially helpful. For example, if the vehicle occupant is in an unfavorable position ("out-of-position"), the belt tension of the seat belt or the activation of a multi-stage airbag can be adjusted to prevent serious injuries.

[0035] The method described above can be used to determine the relative position of a vehicle occupant. However, it is also possible to determine the absolute position of a vehicle occupant by using the absolute seat position. For example, the absolute seat position of a so-called memory seat can be determined as follows: Before the vehicle occupant enters the vehicle, the seat is moved to a position that allows for comfortable entry. This starting position is defined by detectable markers (e.g., magnetic or using end-stop sensors). The seat position measurement is then zeroed. As soon as the vehicle occupant has entered, the seat is moved to the position stored for that person. This can be done, for example, using camera-based facial recognition, where the seat automatically moves to the position stored for the recognized person.Facial recognition can be performed using the 2D camera, which is also used for position determination according to the invention. Alternatively, the seat can also be adjusted manually using buttons on the seat. The memory seat determines the relative change in the seat position, which corresponds to the absolute position after prior zeroing. It is also possible to measure the absolute position directly, without prior zeroing or calibration, using a backrest angle sensor and an additional longitudinal adjustment sensor.

[0036] Fig. Figure 2 schematically shows how a three-dimensional position of the vehicle occupant can be determined by fusing the two-dimensional position of the vehicle occupant determined by means of the 2D camera and the value for the radial deviation of the position of the vehicle occupant from the rest position.

[0037] This shows Fig. 2 a) first, an unoccupied vehicle seat 22 with a safety belt 23 is captured in a top view. The vehicle seat 22 and safety belt 23 are captured by a 2D camera 25.

[0038] Fig. Figure 2b) shows the same vehicle seat and seat belt, but now with a vehicle occupant 21 who has fastened the seat belt. The vehicle occupant is positioned centrally on the seat cushion, with their back against the backrest. It is assumed that the occupant's head is centered above the plumb line in their resting position. The fastened seat belt is in its resting position.

[0039] Fig. Figure 2 c) shows, by way of example, that the vehicle occupant leans forward to the left and thus moves out of the resting position. This can occur, for example, when operating vehicle functions where the controls are not located centrally in front of the vehicle occupant and require approach due to poor visibility, or when there is an obstruction of view or other obstructions in the vehicle's surroundings.

[0040] Since the seatbelt remains fastened, it continues to extend, resulting in an increase ΔI in the belt extension length, which is detected by a sensor (not shown). This increase ΔI determines the current belt extension length. From this belt extension length, a resulting boundary for the vehicle occupant's position can be derived, schematically represented as a semicircle in the figure. The belt extension length initially yields the radius of a circle that encompasses all possible occupant positions. However, this circle is reduced to a forward-facing semicircle because the occupant cannot be positioned behind the seatback.

[0041] The vehicle occupant is further captured by the 2D camera 25, whereby a two-dimensional position of the vehicle occupant in the yz-plane of the vehicle coordinate system is determined by the evaluation of the camera images described above. Even though the camera images do not provide depth information for the position of the vehicle occupant due to the use of a 2D camera, the three-dimensional position of the vehicle occupant can be determined by combining the camera-based yz-position with the radial deviation of the vehicle occupant's position from the rest position, which is determined from the seatbelt extension length. In the figure, this corresponds to limiting the position to a point on the previously determined semicircle.

[0042] Fig. Figure 3 schematically shows an embodiment of a device according to the invention for determining the position of a vehicle occupant 21 in the interior 20 of a vehicle.

[0043] The vehicle occupant 21 is seated on a seat 22 of the vehicle and is wearing a seat belt 23, which runs across the chest, typically from one shoulder to the opposite pelvic area of ​​the occupant in the case of a three-point belt. A belt tensioner 24 ensures that the seat belt 23 fits snugly against the vehicle occupant's body. The belt tensioner also has a belt extension sensor 27, which measures the belt extension length. For this purpose, the belt extension sensor can, for example, be equipped with a wheel on the belt shaft of the seat belt, whereby the rotational movement of the wheel, and thus of the belt shaft, is detected, and a signal representing the belt extension length is generated from the rotation angle of the belt shaft. However, the belt extension sensor 27 can also be arranged at another suitable location in the seat belt system.The sensor data from sensor 27 are fed to an evaluation unit 26, for example via a digital data bus in the vehicle.

[0044] The vehicle is equipped with an interior camera 25 that captures the vehicle occupants. The captured image and video data are also fed to the evaluation unit 26, which analyzes the image data using suitable image processing methods. The interior camera can, for example, be located in the vehicle's interior above the central rearview mirror and capture visible light or infrared light.

[0045] Furthermore, the seat belt buckle 28 has a seat belt buckle sensor 29, which detects that the buckle tongue of the seat belt 23 is properly engaged in the seat belt buckle 28. For the sake of simplicity, the seat belt buckle 28 is shown in the figure on the same side of the seat as the seat belt tensioner 24; however, these are usually arranged on opposite sides of the seat.

[0046] Based on the camera images generated by the interior camera 24 and the seatbelt extension length measured by the sensor 27, the evaluation unit 26 determines a three-dimensional position of the vehicle occupant in the vehicle coordinate system. Based on this, the restraint system can be adapted by a control unit 30. For example, the control unit 30 can individually control the seatbelt tensioner 24 and / or an airbag 32 located in the steering wheel 31, adapting it to the position of the vehicle occupant 21. Furthermore, in addition to the adaptive seatbelt tensioner 24 and airbag 32 shown as examples, the control unit 30 can also control other safety components in the vehicle, if necessary.

[0047] The analysis unit 26 and the control unit 30 are shown separately in the figure, but can also be integrated into a single unit. Furthermore, the control unit 30 can be implemented in a control unit for the seat belt tensioner, in an airbag control unit, or in a central control unit of the vehicle.

[0048] In addition to capturing the vehicle interior with an interior camera and measuring the seatbelt extension length with a suitable sensor, it is also possible to use further sensors based on other technologies and to consider their signals as well. For example, a pressure-sensitive sensor integrated into the seat can be used to determine seat occupancy and, if necessary, to estimate the seating position of the occupant. Combining the sensor data can then further improve the quality of the information obtained.

[0049] The method according to the invention can be applied to occupants of any road vehicle, such as cars, trucks or buses, but is not limited to these. It can also be applied to passengers of trains or airplanes. Reference symbol list 10-16 process steps 20 Vehicle interior 21 vehicle occupants 22 vehicle seats 23 Seatbelt 24 seatbelt tensioners 25 Interior camera 26 evaluation units 27 Belt extension length sensor 28 Seatbelt buckle 29 Seatbelt buckle sensor 30 control unit

Claims

[1] Method for determining the position of a vehicle occupant (21) in a vehicle, in which - an initial value for the belt extension length of a safety belt (23) fastened by the vehicle occupant is determined (10), the initial value corresponding to a rest position of the fastened safety belt when the vehicle occupant is in a rest position on the seat (22); - an increase in the belt extension length of the safety belt (23) starting from the rest position of the fastened safety belt is detected (11) when the vehicle occupant moves from the rest position; - a value for a radial deviation of the position of the vehicle occupant from the rest position is determined from the detected increase in the belt extension length (12); - the vehicle occupant is detected (13) by a 2D camera (25); - at least one feature of the vehicle occupant is extracted (14) from the camera images of the 2D camera (25); - a two-dimensional position of the vehicle occupant is determined from the extracted features of the vehicle occupant (15); and - a three-dimensional position of the vehicle occupant (21) in the vehicle coordinate system of the vehicle is determined from the determined two-dimensional position of the vehicle occupant and the value for the radial deviation of the position of the vehicle occupant from the rest position (16). [2] Method according to claim 1, wherein a restraint system in the vehicle is adapted based on the determined position of the vehicle occupant (21). [3] Method according to claim 1 or 2, wherein the position of the seat (22) on which the vehicle occupant is located is additionally determined and taken into account when determining the three-dimensional position of the vehicle occupant (21). [4] Method according to one of the preceding claims, wherein the vehicle occupant is in a rest position on the seat (22) when he sits centrally on a seat surface of the seat (22) and the back of the body rests against the backrest. [5] Method according to one of the preceding claims, wherein a two-dimensional position of the vehicle occupant in a reference plane which is perpendicular to the optical axis of the 2D camera (25) is determined (15) from the camera images of the vehicle occupant captured by the 2D camera (25). [6] Method according to one of the preceding claims, wherein in addition to the three-dimensional position of the vehicle occupant (21), the orientation of the vehicle occupant (21) relative to the interior (20) of the vehicle is determined. [7] Method according to one of the preceding claims, wherein the three-dimensional position of the vehicle occupant corresponds to the head position of the vehicle occupant and anatomical features on the head of the vehicle occupant are extracted from the camera images of the vehicle occupant captured by the 2D camera (25) using an image analysis in order to then subsequently determine the head position. [8] Device for determining the position of a vehicle occupant (21) in a vehicle, comprising - a sensor (29) for detecting a belt webbing extension length of a safety belt (23) fastened by the vehicle occupant, which sensor, starting from an initial value for the belt webbing extension length of the safety belt (23), which corresponds to a rest position of the fastened safety belt when the vehicle occupant is in a rest position on the seat (22), detects an increase in the belt webbing extension length when the vehicle occupant moves from this rest position; - a 2D camera (25) for detecting the vehicle occupant (21); and - an evaluation unit (26) to which sensor data generated by the sensor (27) and camera images generated by the 2D camera (25) are fed and which - a value for the radial deviation of the vehicle occupant's position from the rest position is determined from the recorded increase in the belt extension length; - extracting at least one feature of the vehicle occupant from the camera images of the vehicle occupant captured by the 2D camera (25) and determining a two-dimensional position of the vehicle occupant therefrom; and - the determined two-dimensional position of the vehicle occupant and the value for the radial deviation of the position of the vehicle occupant from the rest position are jointly evaluated in order to determine a three-dimensional position of the vehicle occupant (21). [9] Computer program with instructions which cause a device according to the invention to carry out the steps of the method according to one of claims 1 to 7. [10] Vehicle arranged to carry out a method according to any one of claims 1 to 7 or having a device according to claim 8.

Images