Method for analysing body postures of a vehicle occupant

Abstract

The invention relates to a method (100) for analysing body postures of at least one vehicle occupant (210) in a vehicle interior (200). The described method (100) comprises the following steps. In a step (102) image data is generated by time-resolved optical detection of the vehicle occupant by means of a camera. In another step (104) a body posture of the vehicle occupant is identified for a plurality of time points using at least a part of the image data. In a further step (106) an evaluation method for the identified body postures for the plurality of time points is used for a statistical and / or temporal evaluation of the identified body postures.

Description

[0001] Method for analyzing the posture of a vehicle occupant

[0002] The following disclosure relates to a method for analyzing body postures of at least one vehicle occupant in a vehicle interior and to an arrangement for analyzing body postures of at least one vehicle occupant in a vehicle interior.

[0003] The design of a vehicle interior, for example, the design of seats and the arrangement of control elements such as buttons and touch panels, has a significant influence on how vehicle occupants behave within the vehicle interior. Ergonomic aspects as well as considerations regarding the passive safety of vehicle occupants therefore play an important role in the design of the vehicle interior. However, evaluating a vehicle interior design with regard to these aspects is challenging, as systematically recording the behavior of vehicle occupants is difficult.

[0004] Current technology uses seat mats with embedded pressure sensors for this task, for example. The seating posture of a vehicle occupant can be tracked using detected pressure points. However, not all information about the occupant's behavior can be obtained from pressure points. To find out, for example, which control elements a vehicle occupant uses and how they operate them, video recordings of hand interactions in the vehicle interior are usually made and these are then manually annotated and evaluated. However, such annotations are time-consuming because they require a person to view the video recordings frame by frame and to manually click on and annotate the important features in each frame, e.g. the position of the hands.

[0005] The object underlying the present disclosure is therefore to describe an improved approach for analyzing body postures of a vehicle occupant in a vehicle interior.

[0006] This object is achieved by the method according to claim 1 and by the arrangement according to claim 13.

[0007] The procedure will be discussed first below, followed by a description of the arrangement.

[0008] The method according to the invention serves to analyse body postures of at least one vehicle occupant in a vehicle interior and comprises the following steps:

[0009] Generating image data by time-resolved optical detection of the vehicle occupant using a camera;

[0010] Detecting a posture of the vehicle occupant for a plurality of points in time using at least a portion of the image data; and

[0011] Applying an evaluation method to the recognized body postures for the plurality of points in time for a statistical and / or temporal evaluation of the recognized body postures.

[0012] The basic idea behind this method is that the posture of a vehicle occupant within the vehicle interior can be used to obtain comprehensive information about the ergonomics of the vehicle interior and its passive safety. By applying the evaluation method for statistical and / or temporal analysis of postures, various aspects of the vehicle interior design can be specifically examined for their ergonomics and passive safety. Furthermore, time-consuming annotation is no longer necessary, and the installation of sensors in the seat can be eliminated.

[0013] Further embodiments of the method are now described below.

[0014] In some embodiments, the posture is in particular a posture of the body skeleton of the vehicle occupant.

[0015] Furthermore, in one embodiment of the method, posture detection can be computer-assisted, for example, using machine learning algorithms. Additionally or alternatively, the application of the evaluation method can also be computer-assisted.

[0016] In some embodiments, a 2D camera may be used additionally or alternatively to generate the image data. In an alternative embodiment, a 3D camera may be used to generate the image data. Depending on which camera is used, the image data may comprise a 2D image of the vehicle interior or a 3D image of the vehicle interior.

[0017] In an advantageous embodiment, the method can additionally or alternatively perform activity classification using the image data, with each of the plurality of time points being assigned an activity. Furthermore, in this embodiment, the evaluation method can be applied to the body postures depending on the assigned activity. With this embodiment, the posture can be examined during specific activities. In this context, the evaluation method can be adapted so that it evaluates aspects of the posture relevant to the activity. It is also advantageous if the activity recognition also provides the reason or context why the vehicle occupant has adopted a posture. The method thus automatically provides further important information for the interior design.In a variant of this embodiment, the activity classification can be carried out using a neural network and / or the assigned activity corresponds to one activity from a predefined plurality of activities.

[0018] The recognition of body posture can, for example, be based on the recognition of skeleton point positions. Skeleton point positions are relevant points on the vehicle occupant's body skeleton that allow conclusions to be drawn about the posture of the body skeleton. Examples of relevant points on the body skeleton include the position of the head, shoulder, elbow or wrist, fingers, hip, or knee. In one embodiment of the method, the recognition of body posture can therefore additionally or alternatively include the recognition of a skeleton point position of an upper body, a head, and / or one or more limbs of the vehicle occupant.

[0019] In an advantageous variant of this embodiment, the skeleton point position can be detected relative to a reference point within the vehicle interior. In another variant, additionally or alternatively, at least one mean value and / or at least one variance can be determined within the framework of the evaluation method using skeleton point positions for two or more points in time.

[0020] In addition to the skeletal point positions, a head rotation position of the vehicle occupant's head can also contain important information, for example, about the vehicle occupant's direction of gaze. Therefore, in a further embodiment of the method, the detection of a body posture can additionally or alternatively include the detection of a head rotation position. The head rotation position describes a rotation of the head relative to the upper body, i.e., in a transverse plane of the vehicle occupant's body. In another variant, a tilt of the head in a frontal plane and / or a nodding of the head in the sagittal plane of the body can also be detected.

[0021] For an ergonomic assessment of various aspects of the vehicle interior, it may be useful to evaluate how long a vehicle occupant maintains a given posture. In one embodiment of the method, the evaluation process can therefore additionally or alternatively include determining the length of time the vehicle occupant has assumed a given posture. In a variant of this embodiment, the length of time a vehicle occupant maintains a sitting position can be determined. This can also be used to determine, for example, how long it takes to make certain adjustments to a control element.

[0022] In another embodiment, the evaluation method may additionally or alternatively comprise determining a time period for which a predefined change in posture occurs. In a variant of this embodiment, the time period over which a predefined movement of limbs, in particular an arm, extends can be determined. This can be used, for example, to determine how easily control elements within the vehicle can be reached.

[0023] In another embodiment, a result of applying the evaluation method can additionally or alternatively be visualized graphically, in particular by means of a graph and / or a heat map.

[0024] The application of the evaluation method can take place online or offline. In one embodiment, the application of the evaluation method may therefore not overlap in time with the generation of the image data. This corresponds to the "offline" variant of the evaluation method. Here, the generation of the image data is only fully completed before the evaluation method is applied. Alternatively, in another embodiment, the application of the evaluation method and the generation of the image data may overlap in time. This corresponds to the "online" variant of the method.

[0025] In one embodiment of the method, the posture of two or more vehicle occupants can also be detected, additionally or alternatively. In this embodiment, the evaluation method can be applied separately to the posture of each of the vehicle occupants.

[0026] The invention further relates to an arrangement with which the method described above can be carried out. Like the method, the arrangement also serves to analyze body postures of a vehicle occupant in a vehicle interior. For this purpose, the arrangement comprises a vehicle with a vehicle interior that includes a camera that is designed and arranged to generate image data by time-resolved, optical detection of the vehicle occupant. Furthermore, the arrangement comprises an evaluation unit that receives the image data and is designed to recognize a body posture of the vehicle occupant for a plurality of points in time using at least part of the image data and to apply an evaluation method to the recognized body postures for the plurality of points in time for a statistical and / or temporal evaluation of the recognized body postures.

[0027] In the following, specific embodiments of the method and the arrangement are described using the attached figures. To this end, an overview of the figures is first provided.

[0028] Fig. 1 shows an example of a method for analyzing body postures of a vehicle occupant in a vehicle interior;

[0029] Fig. 2 shows a vehicle interior with a vehicle occupant, wherein a body posture of the vehicle occupant is represented by means of skeleton point positions;

[0030] Fig. 3 shows a further method for analyzing the body postures of a vehicle occupant in a vehicle interior, wherein an activity classification is performed within the scope of this further method; and Fig. 4 shows an arrangement for analyzing the body postures of a vehicle occupant in a vehicle interior of a vehicle.

[0031] The following provides a detailed description of what is shown in the figures. Reference symbols are used to specifically identify specific aspects of what is shown. The same reference symbols are used throughout the figures to identify identical aspects.

[0032] Fig. 1 shows an example of a method 100 for analyzing body postures of a vehicle occupant in a vehicle interior.

[0033] The method comprises three steps. In a first step 102, image data is generated by time-resolved, optical detection of the vehicle occupant using a camera. In a subsequent second step 104, a posture of the vehicle occupant is detected for a plurality of points in time using at least part of the image data. In a third step 106 following this step, an evaluation method is applied to the detected postures for the plurality of points in time for a statistical and / or temporal evaluation of the detected postures.

[0034] To illustrate the method sequence and partial aspects of the method, a vehicle interior to which the method is applied is described below with reference to Fig. 2 as an example.

[0035] Fig. 2 shows a vehicle interior 200 with a vehicle occupant 210, wherein a body posture of the vehicle occupant 210 is represented by means of skeleton point positions 212A-H. The view of the vehicle interior 210 was recorded by the camera in the region of a windshield of the vehicle interior as part of the method.

[0036] Fig. 2 indicates how the posture of the vehicle occupant 210 was detected within the framework of the method. This was done by detecting skeleton point positions of the upper body of the vehicle occupant 210, whereby the skeleton point positions of the head 212A, the shoulders 212B,G, the elbows 212C,F, the hands 212D,E, and the chest 212H were detected. The skeleton point positions 212A-H correspond to coordinates relative to a reference point in the vehicle interior. In addition to the skeleton point position of the head 212A of the vehicle occupant 210, a head rotation position was also detected when the posture was detected. The head rotation position is indicated by an arrow 214 in Fig. 2 and corresponds to the direction in which the vehicle occupant 210 turned their head. Depending on the application and the viewing angle of the camera used, the skeletal point positions of, for example, the fingers, hips and knees can also be detected.From these positions, the vehicle occupant's body skeleton, indicated by the dashed lines, can be reconstructed to a sufficient degree for the application, and the vehicle occupant's posture can be reconstructed even across movements. In the case shown here, the skeleton point positions correspond to coordinates in a two-dimensional space. However, using a 3D camera or multiple 2D cameras, the skeleton point positions can also be captured in a three-dimensional space within the vehicle interior.

[0037] Recognizing the body skeleton allows for fully automated tracking of how occupants move and behave in their seats or within the vehicle interior over extended periods. Both the seating positions of vehicle occupants and interaction areas touched by the hands can be automatically analyzed and evaluated. Furthermore, a body height can be automatically determined from the body skeleton, allowing for the categorization of grips and movements of vehicle occupants according to different body sizes and, if applicable, body volume.

[0038] Within the framework of the evaluation process, for example, mean values ​​or variances of the positions of body parts can be calculated. Such statistical measures can also be determined for individual skeleton point positions. Using the skeleton point positions, for example, the duration of a posture, such as a certain sitting position, can be calculated. The evaluation process can also determine how long a predefined movement takes, such as fastening a seat belt or operating a control element. Using the skeleton point positions, the evaluation process can be used to examine the detected postures at different times using cluster analysis.

[0039] In addition to the process steps shown in Fig. 1, further process steps may be added. This is explained below by way of example with reference to Fig. 3.

[0040] Fig. 3 shows a further method 300 for analyzing body postures of a vehicle occupant in a vehicle interior, wherein an activity classification is carried out within the scope of this further method 300.

[0041] The method 300 is based on the method 100 and also begins with step 102, in which image data are generated by time-resolved, optical detection of the vehicle occupant by means of a camera, and the subsequent step 104, in which a posture of the vehicle occupant is recognized for a plurality of points in time using at least part of the image data.

[0042] Following step 104, in this example, in step 308, an activity classification is carried out using the image data, with an activity being assigned to each of the plurality of points in time. In this example, the activity classification is carried out using a neural network that has been trained in advance using supervised learning to determine an activity of the vehicle occupant from a predetermined plurality of activities based on the detected body postures of the vehicle occupant for each of the plurality of points in time. The predetermined plurality of activities can include, for example, drinking, eating, using the telephone, buckling up, or working on a laptop. The use of a neural network is particularly suitable for this purpose. However, other methods for determining the activity can also be used instead of the neural network.

[0043] In a step 306, analogous to step 106 of method 100, an evaluation method is then applied to the detected body postures for the plurality of points in time for a statistical and / or temporal evaluation of the detected body postures. In step 306, the evaluation method is applied to the body postures depending on the associated activity. Applying the evaluation method depending on the detected activity allows a body posture to be examined in the context of the activity. This makes it possible, for example, to evaluate the body posture within the context of movement sequences that occur while buckling up differently than body postures within the context of movement sequences that occur while operating a control element.

[0044] In step 310, the result of applying the evaluation method is then graphically visualized. Heatmaps can be used, for example, to visualize particularly frequently used or less frequently used interaction areas in the vehicle interior. Heatmaps can also be used to depict particularly frequently or less frequently assumed positions of the limbs.

[0045] Finally, an arrangement for carrying out the method is described below with reference to Fig. 4.

[0046] Fig. 4 shows an arrangement 400 for analyzing body postures of a vehicle occupant 410 in a vehicle interior 404 of a vehicle 402.

[0047] The arrangement 400 comprises the vehicle 402 with the vehicle interior 404. The vehicle 402 is shown here in a cross-sectional view along a section through the vehicle 402 from the vehicle front 406 to the vehicle rear 408. A vehicle occupant 410 is shown in the vehicle interior 404 by way of example. In this example, this is the driver of the vehicle 402, who is sitting at a steering wheel 412 of the vehicle and whose posture is to be analyzed. For this purpose, a camera 420 is also located in the vehicle interior 404, which camera is designed and arranged to generate image data by time-resolved, optical detection of the vehicle occupant 410. In this exemplary embodiment, the camera 420 is a 2D color camera. In other embodiments, however, it may also be an infrared camera or a 3D camera, such as a structured light camera, a time-of-flight camera or even a stereo camera.Furthermore, an evaluation unit 422 is located in the vehicle interior 404, which receives the image data and is designed to recognize a posture of the vehicle occupant 410 for a plurality of points in time using at least part of the image data and to apply an evaluation method to the recognized postures for the plurality of points in time for a statistical and / or temporal evaluation of the recognized postures.

[0048] The evaluation unit 422 is configured to detect the posture and apply the evaluation method in real time (also referred to as an "online" method). This means that the application of the evaluation method and the generation of the image data overlap in time. This makes it possible for a result of the application of the evaluation method to be available while the vehicle is traveling. However, the evaluation unit does not have to be located entirely in the vehicle 402 in all cases. It is also possible for at least parts of the evaluation unit to be located outside the vehicle 402, for example, in a cloud computer. In other embodiments, it is also possible for the application of the evaluation method not to overlap in time with the generation of the image data. In this application, there is usually only one storage device within the vehicle that stores the generated image data.These can then be removed from the vehicle 402 following a journey and analyzed by means of an evaluation unit located outside the vehicle 402.

[0049] In summary, this disclosure describes a method (100) for analyzing body postures of at least one vehicle occupant (210) in a vehicle interior (200). The described method (100) comprises the following steps. In one step (102), image data are generated by time-resolved optical detection of the vehicle occupant using a camera. In another step (104), a body posture of the vehicle occupant is recognized for a plurality of points in time using at least part of the image data. In a further step (106), an evaluation method is used for the recognized body postures for the plurality of points in time for a statistical and / or temporal evaluation of the recognized body postures.

Claims

Patent claims 1. A method (100) for analyzing body postures of at least one vehicle occupant (210) in a vehicle interior (200), comprising the steps: Generating image data by time-resolved optical detection of the vehicle occupant by means of a camera (102); detecting a posture of the vehicle occupant for a plurality of points in time using at least a portion of the image data (104); Applying an evaluation method to the recognized body postures for the plurality of points in time for a statistical and / or temporal evaluation of the recognized body postures (106).

2. The method (300) according to claim 1, comprising the step: Performing an activity classification using the image data, wherein an activity is assigned to each of the plurality of time points (308); and wherein the evaluation method is applied to the body postures depending on the assigned activity (306).

3. The method (300) of claim 2, wherein the activity classification is performed using a neural network and / or the assigned activity corresponds to one of a predefined plurality of activities.

4. The method (100) according to any one of the preceding claims, wherein detecting the posture comprises detecting a skeletal point position (212A-H) of an upper body, a head and / or one or more limbs of the vehicle occupant (210).

5. The method (100) of claim 4, wherein the skeleton point position (212A-H) is detected relative to a reference point within the vehicle interior (210).

6. Method (100) according to claim 4 or 5, wherein in the context of the evaluation method using skeleton point positions (212A-H) for at least one mean and / or at least one variance is determined at two or more points in time.

7. The method (100) according to any one of the preceding claims, wherein detecting a body posture comprises detecting a head rotation position (214).

8. The method (100) according to any one of the preceding claims, wherein the evaluation method comprises determining a period of time during which the vehicle occupant (210) has assumed a given posture.

9. Method (100) according to one of the preceding claims, wherein the evaluation method comprises determining a period of time for which a predefined change in body posture takes place.

10. Method (300) according to one of the preceding claims, wherein a result of applying the evaluation method is graphically visualized, in particular by means of a graph and / or a heat map (310).

11. Method (100) according to one of the preceding claims, wherein the application of the evaluation method (106) does not overlap in time with the generation of the image data (102).

12. The method (100) according to any one of claims 1 to 10, wherein the application of the evaluation method (106) and the generation of the image data (102) overlap in time.

13. Arrangement (400) for analyzing body postures of a vehicle occupant (410) in a vehicle interior (404), comprising: a vehicle (402) with a vehicle interior (404) comprising a camera (420) which is designed and arranged to generate image data by time-resolved, optical detection of the vehicle occupant (410); and an evaluation unit (422) which receives the image data and is designed to recognize a body posture of the vehicle occupant (410) for a plurality of points in time using at least part of the image data and to apply an evaluation method to the recognized body postures for the To apply a multitude of points in time to a statistical and / or temporal evaluation of the recognized body postures.

Images