Method for operating assistance system for person in motor vehicle, computer program product and assistance system

Abstract

The invention relates to a method for operating an assistance system (2) for a person (6) in a motor vehicle (1), comprising the following steps: detecting the person (6) by means of an optical detection device (3) of the assistance system (2); -determining a body posture (8) of the person (6) by means of an electronic computing device (4) of the assistance system (2); -generating a digitized body (9) from the detected person (6) and the determined body posture (8); and displaying the digitized body (9) with the determined body posture (8) on a display device (5) of the assistance system (2). The invention also relates to a computer program product and to an auxiliary system (2).

Description

[0001] The present invention relates to a method for operating an assistance system for a person in a motor vehicle according to the prior patent claim 1. The present invention also relates to a computer program product and an assistance system.

[0002] Sitting statically in a motor vehicle is unhealthy. In addition, many people are currently affected by mental stress. Therefore, there have been many suggestions in motor vehicles to make the sitting posture dynamic or to balance emotions and energy. However, customers rarely use them.

[0003] For example, according to the prior art, the driver's posture is detected in various ways by a camera system. Vital signs are also provided and detected in different ways.

[0004] DE 102018207518A1 relates to a method for animating a 3D avatar of a first user of a communication platform to a second user. The method includes detecting first user attributes of the first user with a 3D scanner and / or a camera device, creating a 3D model of the first user by a first terminal of the first user considering the detected first user attributes, transmitting the 3D model, which is the 3D avatar of the first user, from the first terminal to a second terminal of the second user, detecting second user attributes of the first user with a 3D scanner and / or a camera device, determining joint data indicating the position and / or orientation of joints of the first user based on the second user attributes, transmitting the joint data from the first terminal to the second terminal, and animating the 3D avatar at the second user considering the received joint data. When communicating with a communication partner, the user can see the animated 3D avatar of the communication partner.

[0005] DE 10 2017 114 179 A1 describes a disclosure of a vehicle platform, which includes acquiring information, analyzing information, and controlling the vehicle, and sharing the acquired information and the in-vehicle information entertainment with other vehicles and electronic devices.

[0006] DE 10 2016 004 395 A1 relates to a method for determining the state of at least one vehicle occupant in a vehicle, the method including a detection unit arranged in the vehicle for detecting at least one vehicle occupant. The posture of the vehicle occupant and / or the driving behavior of the vehicle occupant are detected during the entire driving period, wherein the instantaneous state of the vehicle occupant is determined based on the data of the vehicle occupant detected during the entire driving duration. When determining the instantaneous posture of the vehicle occupant or the deviation from a predetermined posture and / or when determining the deviation of the instantaneous driving behavior from a predetermined driving behavior and / or when determining the movement requirement of the vehicle occupant determined by the vehicle, at least one piece of information about the instantaneous state of the vehicle occupant in the vehicle is optically and / or acoustically output.

[0007] DE 10 2021 202 295 A1 discloses a computer-implemented method for assisting a person in locating components of an interior device of an automatically operable driving system including a seat, a steering wheel, and / or a rearview mirror. The method includes the following steps: calculating a body posture of the person in the interior space of the driving system, including the positions of the head, back, thighs, calves, and / or arms, which are achievable for body dimensions within a set range; tracking the body posture adopted by the person based on an image; comparing the currently adopted body posture in each case with a specific set range; and if the deviation of the currently adopted body posture from the body postures achievable within the set range determined for the current driving situation, providing feedback to the person on the currently adopted body posture, including instructions for the optimal positioning of the components and / or the optimal body posture regarding the overview and effective range of the restraint system.

[0008] The object of the present invention is to provide a method, a computer program product, and an assistance system by means of which a person can be assisted in an improved manner during the driving operation of a motor vehicle.

[0009] This object is achieved by a method, a computer program product, and an assistance system according to the independent patent claims. Advantageous embodiments are specified in the dependent claims.

[0010] One aspect of the present invention relates to a method for operating an assistance system for a person in a motor vehicle. The person is detected by means of an optical detection device of the assistance system. The body posture of the person is determined by means of an electronic computing device of the assistance system. A digital avatar is generated based on the detected person and the determined body posture, and the avatar with the determined body posture is displayed on a display device of the assistance system.

[0011] Thus, in particular, a digital avatar is proposed which, in the manner of a companion or a coach, on the one hand adopts the posture of the person, especially the driver or passenger of the motor vehicle for which the avatar is intended, so as to be considered trustworthy, competent, and empathetic, similar to a therapeutic relationship. In addition, similar to the principle of so-called hypnotherapy, also known as NLP (Neuro-Linguistic Programming), a positive, sustainable relationship, also known as pacing, is created between the digital avatar and the person by reflecting the posture. On this basis, this person is more willing to be "guided" towards health-oriented behavior, which is also known by the technical term "leading".

[0012] This pose is derived, for example, from at least one coordinate representing the head position. For example, the head position in the x, y, and z directions. Further coordinates regarding the head pose and body pose can be added. For example, rotation as an indicator of the face direction. Additionally, this pose can also be directly derived from the camera recording by means of image analysis methods. Moreover, if the breathing rate is provided by a camera or a wearable device or other detection sources, it can also be animated on the avatar.

[0013] For example, knowledge about the breathing rate or pose can also be used to guide a person to optimize their activities. For example, if air cessation is detected by an algorithm during exercise, a message is generated via voice output, as a written notification, and / or additional tactile impulses (e.g., in the form of vibration or sitting posture dynamics) to, for example, stop the person from continuing to breathe.

[0014] Furthermore, data regarding the head position exports in the background when motion-oriented advice can or should be provided. This time point is defined by information about whether the person is moving too little or not at all or is in a "bad" posture. Or whether the breathing is becoming shallower and faster, which also indicates a deterioration in posture and the chest is no longer straightened by deep breathing.

[0015] In particular, motion-oriented advice can be provided preventively before a person feels discomfort, tension, or a reduction in the oxygen supply to the brain due to shallow breathing, etc. Here, an intelligent algorithm is defined in the background for the correct time point of behavior recommendation.

[0016] The time point when a rather monotonous driving situation exists is detected in parallel from motor vehicle data and personal data, where, for example, activation advice can be perceived and thus executed. This adaptation to the driving situation is only integrated into the driver's algorithm in the driving mode.

[0017] Visual information on the display device, especially configured as a user interface, is only visible to the driver during driving, especially in highly automated driving phases. For passengers and co-riders, this is possible at any time. Additionally, statistics regarding pose and vital signs, as well as movement behavior, breathing changes, and behavior, and the activity time during driving can be displayed. "Training minutes" during driving and vital signs, etc., can also be displayed as needed.

[0018] The assistance system is actively activated by the person and remains activated until the end of the drive or the customer actively terminates the assistance system. This can be done by touch or voice operation or other means. Ideally, the advice is executed at least once per hour. Upon request, data regarding the driving history for the algorithm can also be displayed and prepared for the customer. Thus, for example, the relationship between pose, head position, breathing rate, and activity can be displayed.

[0019] Accordingly, in particular, a method is provided for detecting a person's body posture by means of an optical detection device, in particular by means of a camera. The intelligent processing of data is carried out by means of algorithms, for example by an application on the main unit of a motor vehicle. Pacing and guidance are also carried out during driving and stopping. In this case, the selection can be a random selection or a selection made by a person. In particular, a series of health promotion systems can be provided in a motor vehicle, for example exercises for muscle activation, relaxation or balance. In addition, sitting dynamics or other activation, mobilization, interactive massage, etc. can be carried out. Furthermore, activities are recommended to the person, where, for example, a digital avatar reflects the posture and guides the person to participate in the activity. Alternatively, a written notice or voice content can also be output. The digital avatar can give tips based on the posture information and vital signs. For example, the breathing rate can also be monitored during an activity, and if the customer forgets to breathe, the avatar gives an indication to continue breathing.

[0020] A method is provided for comparing a determined body posture with at least one predetermined body posture and, in the case of a deviation, indicating, by means of a digital avatar, a movement from the current body posture to the predetermined body posture. In particular, for example, in the case of a currently detected ergonomically poor body posture, the deviation can be correspondingly executed. In this case, for example, the avatar displays the ergonomically incorrect body posture on a display device and then, by means of a digital movement of the avatar into the correct body posture, shows the person again the movement to imitate. Thus, a person in an ergonomically poor body posture can reliably execute the correct movement into an ergonomically correct body posture.

[0021] Furthermore, it has proven to be advantageous to acoustically draw a person's attention to a deviation in the case of a deviation of a determined body posture. For example, this can occur in the form of a warning message or the like, such that when an incorrect, in particular ergonomically incorrect, body posture is detected, the person is correspondingly notified so that no harm is done to the person.

[0022] Furthermore, it has proven to be advantageous to acoustically notify a person of the correction of their current body posture in the case of a deviation. For example, the person can be told that they should move their body to the left or right or forward or backward in order to assume a correct body position or body posture, which in turn can protect them from harm.

[0023] Another advantageous design provides that at least one head position of a person is reflected onto a display device by means of an avatar. In particular, thus, the person can better recognize the digital avatar, whereby improved pacing and thus improved guidance can be achieved.

[0024] Equally advantageously, when a body posture is statically detected within a predetermined time period, a movement request is displayed by means of an avatar. In particular, for example, after 50 minutes or at least one hour later, a corresponding movement request occurs in order to perform so-called micro-muscle movements. This has the advantage that these movements do not cause any damage to the body, and the corresponding micro-movements have a beneficial effect on the health condition of a person during driving.

[0025] Another advantageous design provides that when the digital avatar is displayed, the current driving situation of the motor vehicle is taken into account. For example, only during straight driving or monotonous long-distance driving can a person as a driver be correspondingly informed of a possibly incorrect body posture.

[0026] It is also advantageous to suppress the display of the digital avatar when a person actively intervenes in the driving situation. Active intervention should be particularly understood as driving interventions such as steering, accelerating or braking. For example, in urban traffic or in a traffic light situation, active intervention is expected in the future, so the display of the digital avatar is suppressed to prevent safety-critical situations in road traffic.

[0027] The proposed method is in particular a computer-implemented method. Therefore, another aspect of the present invention relates to a computer program product having program code means which cause an electronic computing device to execute the method according to the foregoing aspects when the program code means are processed by the electronic computing device. The computer program product may also be referred to as a computer program. In addition, the present invention also relates to a computer-readable storage medium having the computer program product.

[0028] Another aspect of the present invention relates to an assistance system for a person in a motor vehicle, comprising at least one optical detection device, an electronic computing device and a display device, wherein the assistance system is configured to execute the method according to the foregoing aspects. In particular, the method is executed by means of the assistance system.

[0029] The electronic computing device includes, for example, a processor, a circuit, in particular an integrated circuit, and other electronic components in order to be able to execute the corresponding method steps.

[0030] The present invention also relates to a motor vehicle having an assistance system according to the foregoing aspects. In this case, the motor vehicle can be configured to be partially autonomous or fully autonomous.

[0031] An advantageous design of the method is considered to be an advantageous design of the computer program product, the assistance system and the motor vehicle. For this purpose, the assistance system and the motor vehicle particularly have target features capable of executing the method.

[0032] Further advantages, features, and details of the present invention are given in the following description of the preferred embodiments and with reference to the drawings. Without departing from the scope of the present invention, the above-mentioned features and combinations of features mentioned in the description, as well as the features and combinations of features mentioned in the description of the drawings and / or shown solely in a single drawing, can be used not only in the respective combinations but also in other combinations or individually.

[0033] In this case, a single figure shows a schematic view of an embodiment of a motor vehicle having an embodiment of an assistance system that can execute the method according to the present invention.

[0034] In the figure, identical or functionally identical elements have the same reference numerals.

[0035] Figure 1 A schematic side view of an embodiment of a motor vehicle 1 having an embodiment of an assistance system 2 is shown. The assistance system 2 particularly has an optical detection device 3 in the form of a camera, for example, and an electronic computing device 4. In addition, the assistance system 2 has at least one display device 5. In this case, the motor vehicle 1 can be operated, for example, at least partially autonomously or fully autonomously.

[0036] In Figure 1 a person 6 is located in the interior space 7 of the motor vehicle 1. In the present embodiment, the person 6 is particularly a passenger in the motor vehicle 1.

[0037] According to the present invention, the person 6 is now detected by means of the optical detection device 3. The body posture 8 of the person 6 is determined by means of the electronic computing device 4. A digital avatar 9 is generated based on the detected person 6 and the determined body posture 8, and the digital avatar 9 having the determined body posture 8 is displayed on the display device 5.

[0038] In particular, it can be provided that the determined body posture 8 is compared with at least one predetermined body posture, and in the case of a deviation, the movement 10 from the current body posture 8 to the predetermined body posture is indicated by means of the digital avatar 9. For example, in the case of a deviation of the determined body posture 8, the person 6 can be acoustically made aware of the deviation. Alternatively or additionally, in the case of a deviation, the person 6 can be acoustically informed of the correction of the current body posture 8.

[0039] Furthermore, in particular, it can be provided that at least one head position 11 of the person 6 is reflected on the display device 5 by means of the avatar 9.

[0040] It can also be provided that a movement request is displayed by means of the avatar 9 when the body posture 8 is statically detected within a predetermined time period.

[0041] In addition, it is possible to provide for taking into account the current driving situation of the motor vehicle 1 when displaying the digital avatar 9. For example, in the case of active intervention of the person 6 in the driving situation, such as in the case of the driver of the motor vehicle 1, the display of the digital avatar is suppressed.

[0042] In particular, therefore, the detection of the head position is provided, for example, by coordinates in space as the point between the eyes. In addition, the detection of the coordinates of the nose relative to the head coordinates can also be used as an indicator of head rotation, which is preferably performed by an internal camera. In addition, the breathing frequency can also be detected accordingly.

[0043] Based on the detected data regarding the head position and rotation, the head of the avatar 9 can be positioned if the avatar 9, for example, wants to or is responding to the person 6.

[0044] In the indication of a task or in the interaction between the person 6 and the avatar 9, the avatar 9 precisely presents the posture and breathing frequency of the person 6, which particularly corresponds to so-called pacing. The avatar 9 now preferably guides the person 6 to adopt the correct posture or behavior, especially towards health-promoting advice. This particularly occurs only in appropriate driving situations. In addition, advice can also be issued via messages or text.

[0045] In particular, vehicle signals and driving signals can be used as indicators of the corresponding driving situation. The corresponding advice can be particularly suitable for the steering and braking behavior of the motor vehicle 1. In addition, the corresponding time can also be observed. For example, when the upright posture sags, remains motionless, or breathing becomes shallower, a corresponding reaction can be initiated.

[0046] When the quality of the posture or body tension deteriorates, the avatar 9 actively speaks to the person 6 in order to initiate a corresponding movement change. In this case, the person 6 can confirm or ignore the movement advice by activating a function or by voice control.

[0047] For example, if the head position has not changed within 30 minutes and breathing becomes increasingly shallow and thus faster, the avatar 9, for example, actively recommends an upright breathing program, especially by performing appropriate breathing exercises by the avatar 9. This can also be confirmed by the customer and can therefore be activated or ignored.

[0048] In this case, the avatar 9 particularly demonstrates the exercise, describes the procedure, and can therefore prevent the person from repeating the corresponding exercise.

[0049] In particular, the method can therefore include, for example, the step of detecting the start of driving accordingly. Then, the detection or sinking of the head occurs, especially moving forward, for example. Then, the avatar 9 assumes the posture of the person 6 and gives advice again. In a further step, the corresponding exercise or movement request is again displayed by the avatar 9. Then, the statistical data can be processed by displaying health-promoting activities and digital data for the person 6 accordingly.

Claims

1. A method for operating an assistance system (2) for a person (6) in a motor vehicle (1), comprising the following steps: - Detecting the person (6) by means of an optical detection device (3) of the assistance system (2); - Determining a body posture (8) of the person (6) by means of an electronic computing device (4) of the assistance system (2); - Generating a digital avatar (9) based on the detected person (6) and the determined body posture (8); and - Displaying the digital avatar (9) with the determined body posture (8) on a display device (5) of the assistance system (2), wherein the determined body posture (8) is compared with at least one predetermined body posture, and in case of a deviation, a movement (10) from the current body posture (8) to the predetermined body posture is indicated by means of the digital avatar (9).

2. The method according to claim 1, characterized in that in case of a deviation of the determined body posture (8), the person (6) is acoustically alerted to the deviation.

3. The method according to claim 1 or 2, characterized in that in case of a deviation, the person (6) is acoustically informed to correct the current body posture (8).

4. The method according to any one of the preceding claims, characterized in that at least one head position (11) of the person (6) is reflected onto the display device (5) by means of the avatar (9).

5. The method according to any one of the preceding claims, characterized in that when the body posture is detected as being static for a predetermined period of time, a movement request is displayed by means of the avatar (9).

6. The method according to any one of the preceding claims, characterized in that when displaying the digital avatar (9), the current driving situation of the motor vehicle (1) is taken into account.

7. The method according to claim 6, characterized in that when the person (6) actively intervenes in the driving situation, the display of the digital avatar (9) is suppressed.

8. A computer program product having program code means which, when processed by the electronic computing device (4), cause the electronic computing device (4) to execute the method according to any one of claims 1 to 7.

9. An assistance system (2) for a person (6) in a motor vehicle (1), comprising at least one optical detection device (3), an electronic computing device (4) and a display device (5), wherein, The assistance system (2) is configured to execute the method according to any one of claims 1 to 7.

Images