Method for operating an assistance system for a person in a motor vehicle, computer program product, and assistance system

Abstract

The invention relates to a method for operating an assistance system (2) for a person (6) in a motor vehicle (1), comprising the steps of: - sensing the person (6) by means of an optical sensing device (3) of the assistance system (2); - determining a posture (8) of the person (6) by means of an electronic computing device (4) of the assistance system (2); - generating a digital avatar (9) according to the sensed person (6) and the determined posture (8); and - displaying the digital avatar (9) with the determined posture (8) on a display device (5) of the assistance system (2). The invention also relates to a computer program product and to an assistance system (2).

Description

[0001] Method for operating a support system for a person in a motor vehicle, computer program product and support system

[0002] The invention relates to a method for operating a support system for a person in a motor vehicle according to the applicable patent claim 1. Furthermore, the invention relates to a computer program product and a support system.

[0003] Static sitting in a vehicle is unhealthy, and many people are currently affected by mental stress. Therefore, there are already many in-vehicle options designed to make sitting more dynamic or to balance mood and energy. Yet customers rarely take advantage of them.

[0004] For example, the driver's posture is detected in a variety of ways using state-of-the-art camera systems. Vital data is also provided and recorded in various ways.

[0005] DE 102018207 518 A1 relates to a method for animating a 3D avatar of a first user of a communication platform for a second user.The method comprises capturing first user properties of the first user with a 3D scanner and / or a camera device, creating a 3D model of the first user taking into account the captured first user properties by a first terminal of the first user, transmitting the 3D model as the 3D avatar of the first user from the first terminal to a second terminal of the second user, capturing second user properties of the first user with the 3D scanner and / or camera device, determining joint data that indicate positions and / or orientations of joints of the first user based on the second user properties, transmitting the joint data from the first to the second terminal, and animating the 3D avatar for a second user taking into account the received joint data.When communicating with a communication partner, a user can see an animated 3D avatar of the communication partner.

[0006] DE 102017 114 179 A1 describes a disclosure for a vehicle platform which comprises obtaining information, analyzing the information and controlling the vehicle as well as sharing the obtained information and infotainment in the vehicle with other vehicles and electronic devices.

[0007] DE 102016 004 395 A1 relates to a method for determining a state of at least one vehicle occupant in a vehicle, comprising a detection unit arranged in the vehicle for detecting the at least one vehicle occupant. A posture of the vehicle occupant and / or a driving behavior of the vehicle occupant is or is detected throughout the entire driving period, wherein a current state of the vehicle occupant is determined depending on the data of the vehicle occupant recorded throughout the entire driving period. Upon detection or deviation of the current posture from a predetermined posture of the vehicle occupant and / or upon detection of a deviation of the current driving behavior from a predetermined driving behavior and / or upon detection of a movement requirement of the vehicle occupant determined by the vehicle, at least one piece of information regarding the current state of the vehicle occupant is output visually and / or acoustically in the vehicle.

[0008] DE 102021 202 295 A1 discloses a computer-implemented method for assisting a person in positioning components of an interior of an automated driving system comprising a seat, steering wheel and / or mirror. The method comprises the steps of calculating body postures of the person in an interior of the driving system, including positions of the head, back, upper legs, lower legs and / or arms that are achievable for the body dimensions within the adjustment ranges; image-based tracking of body postures assumed by the person; comparing the currently assumed body postures with the determined adjustment ranges;Feedback of the currently adopted body posture to the person, including instructions for optimal positioning of the components and / or for an optimal body posture with regard to the overview and the effective range of the restraint systems, if the currently adopted body posture deviates from a body posture that can be achieved within the adjustment ranges determined for the current driving situation;

[0009] The object of the present invention is to provide a method, a computer program product and a support system by means of which a person can be supported in an improved manner during the driving operation of the motor vehicle.

[0010] This object is achieved by a method, a computer program product, and a support system according to the independent patent claims. Advantageous embodiments are specified in the subclaims.

[0011] One aspect of the invention relates to a method for operating a support system for a person in a motor vehicle. The person is detected by means of an optical detection device of the support system. The person's posture is determined by means of an electronic computing device of the support system. A digital avatar is generated depending on the detected person and the determined posture, and the avatar is displayed with the determined posture on a display device of the support system.

[0012] In particular, a digital avatar is proposed which, in the manner of a companion or coach, adopts the posture of the person addressed, who could in particular be a driver or passenger of the vehicle, in order to be perceived as trustworthy, competent, and sympathetic, analogous to a therapeutic relationship. Furthermore, analogous to the principles of so-called hypnotherapy, also known in NLP (Natural Language Processing - neurolinguistic programming), by mirroring the posture, a positive, sustainable relationship is created between the digital avatar and the person, which is also referred to as pacing. On this basis, the person is more likely to be "guided" towards health-oriented behavior, a process also known by the technical term "leading."

[0013] This posture is derived, for example, from at least one coordinate representing the head position. For example, the head position in the x, y, and z directions. Additional coordinates for head and body posture can be added. For example, rotation as an indicator of facial orientation. Furthermore, this posture can also be derived directly from the camera recordings using image analysis methods. Furthermore, the breathing rate, if provided via the camera, wearables, or other recording sources, can also be animated on the avatar.

[0014] Knowledge of breathing rate or posture, for example, can also be used to guide the person's activity optimization. If, for example, the algorithm detects that the person is holding their breath during an exercise, a message is sent via voice output, a written notification, and / or a haptic stimulus, such as a vibration or sitting kinetics, to encourage the person to continue breathing.

[0015] The head position data also serves as a background measure to determine when a movement-oriented suggestion can or should be made. This point is defined by information about whether the person has moved too little or not at all, or is adopting a "poor" posture. Or whether breathing is becoming increasingly shallow and rapid, which also indicates a deterioration in posture, such as the chest no longer being straightened by deep inhalations.

[0016] The movement-oriented approach can be used preventively, especially before discomfort occurs, tension develops, or the oxygen supply to the brain is reduced due to shallow breathing, etc. An intelligent algorithm is defined in the background to determine the right time for a behavioral recommendation.

[0017] Vehicle and passenger data are used in parallel to determine when a more monotonous driving situation exists, in which, for example, activating recommendations can be perceived and thus implemented. This adaptation to the driving situation is integrated into the algorithm only for the driver in driving mode.

[0018] Visual information on the display, which is designed primarily as a user interface, is only visible to the driver during highly automated driving phases. This is possible for front and front passenger passengers at any time.

[0019] Additionally, statistics on posture and vital signs, as well as movement patterns and breathing changes and behavior, as well as the time of activity during the ride, can be displayed. "Training minutes" during the ride, as well as vital signs and the like, can also be displayed as needed.

[0020] The support system is actively activated by the person and remains active until the ride ends or until the support system is actively terminated by the customer. This can be done via touch or voice control, or other methods. The offers are ideally executed at least once per hour. Upon request, the data used throughout the ride can also be presented to the customer and processed for the algorithm. This allows, for example, relationships between posture and head position, breathing rate, and activity to be shown.

[0021] In particular, it is thus provided that the person's posture is recorded by means of the optical detection device, in particular by means of a camera. The data is intelligently processed by an algorithm, for example, an application on a head unit of the motor vehicle. Pacing and leading are carried out while driving and also when stationary. The selection can be random or selected by the person. In particular, a list of health-promoting systems can be available in the motor vehicle, for example, exercises for muscle activation, relaxation, or balance. Furthermore, seat kinetics or other activating, mobilizing, interactive massages or the like can be performed.Furthermore, an activity recommendation is made for the person. For example, the digital avatar reflects the patient's posture and guides the activity, encouraging the patient to participate. Alternatively, written instructions or spoken content can be provided. The digital avatar can provide guidance based on information about posture and vital signs. For example, breathing rate can also be monitored during the activity, and the avatar provides guidance, for example, to continue breathing if the patient forgets to breathe.

[0022] The plan is for the specific posture to be compared with at least one predefined posture, and if there is a deviation, a movement from the current posture to the predefined posture is displayed using the digital avatar. In particular, the deviation can be carried out accordingly, for example, if the currently detected posture is ergonomically poor. In this case, the plan is for the avatar, for example, to display the ergonomically incorrect posture on the display device, and then a digital movement of the avatar into the correct posture will display the movement for the person to imitate. In this way, the person in the ergonomically poor posture can reliably perform the correct movement into the ergonomically correct posture.

[0023] Furthermore, it has proven advantageous to acoustically alert the person to a deviation from a specific posture. For example, this can be done in the form of a warning message or similar, so that the person is alerted accordingly when the incorrect, particularly ergonomically incorrect, posture is detected, preventing any harm to the person.

[0024] Furthermore, it has proven beneficial to provide an acoustic notification of a person's current posture in the event of a deviation. For example, the person can be told to move their body left or right, or forward or backward, in order to assume a correct body position or posture, which in turn can protect against injury.

[0025] A further advantageous embodiment provides that at least one of the person's head positions is reflected on the display device using the avatar. In particular, this allows the person to better identify with the digital avatar, thereby enabling improved pacing and thus improved leading.

[0026] It is also advantageous if, when a static posture is recorded over a specified period of time, a movement request is displayed via the avatar. In particular, for example, after 50 minutes or after at least one hour, a corresponding movement request is made to perform so-called micro-muscle movements. This has the advantage that these movements do not cause any damage to the body and corresponding micro-movements have a beneficial effect on the person's state of health during the journey. Yet another advantageous embodiment provides that the current driving situation of the motor vehicle is taken into account when displaying the digital avatar. For example, the person as the driver can be alerted to potentially incorrect posture only when driving straight ahead or on monotonous long journeys.

[0027] It is also advantageous if the display of the digital avatar is suppressed when the person actively intervenes in the driving situation. Active intervention refers in particular to driving interventions such as steering, accelerating, or braking. For example, in city traffic or at traffic lights, active intervention is expected in the future, and thus the display of the digital avatar is suppressed to prevent safety-critical situations in road traffic.

[0028] The method presented is, in particular, a computer-implemented method. Therefore, a further aspect of the invention relates to a computer program product with program code means that, when the program code means are processed by the electronic computing device, cause an electronic computing device to perform a method according to the preceding aspect. The computer program product can also be referred to as a computer program. Furthermore, the invention also relates to a computer-readable storage medium containing the computer program product.

[0029] A further aspect of the invention relates to a support system for a person in a motor vehicle, comprising at least one optical detection device, an electronic computing device, and a display device, wherein the support system is designed to carry out a method according to the preceding aspect. In particular, the method is carried out by means of the support system.

[0030] The electronic computing device comprises, for example, processors, circuits, in particular integrated circuits, and other electronic components to enable the corresponding method steps to be carried out. Furthermore, the invention also relates to a motor vehicle with a support system according to the preceding aspect. The motor vehicle can be designed to be either partially autonomous or fully autonomous.

[0031] Advantageous embodiments of the method are to be regarded as advantageous embodiments of the computer program product, the support system, and the motor vehicle. The support system and the motor vehicle, in particular, have specific features that enable implementation of the method.

[0032] Further advantages, features, and details of the invention will become apparent from the following description of a preferred embodiment and from the drawings. The features and combinations of features mentioned above in the description, as well as the features and combinations of features mentioned below in the description of the figures and / or shown alone in the single figure, can be used not only in the respective specified combination, but also in other combinations or on their own, without departing from the scope of the invention.

[0033] The only figure shows a schematic view of an embodiment of a motor vehicle with an embodiment of a support system which can carry out a method according to the invention.

[0034] In the figure, identical or functionally identical elements are provided with the same reference numerals.

[0035] Fig. 1 shows a schematic side view of an embodiment of a motor vehicle

[0036] 1 with an embodiment of a support system 2. The support system

[0037] 2 comprises, in particular, an optical detection device 3, for example in the form of a camera, and an electronic computing device 4. Furthermore, the support system 2 comprises at least one display device 5. The motor vehicle 1 can, for example, be operated at least partially autonomously or even fully autonomously. In Fig. 1, a person 6 is located in an interior 7 of the motor vehicle 1. In the present exemplary embodiment, the person 6 is, in particular, a passenger in the motor vehicle 1.

[0038] According to the invention, the person 6 is detected by means of the optical detection device 3. A posture 8 of the person 6 is determined by means of the electronic computing device 4. A digital avatar 9 is generated depending on the detected person 6 and the determined posture 8, and the digital avatar 9 is displayed with the determined posture 8 on the display device 5.

[0039] In particular, it can be provided that the determined posture 8 is compared with at least one predetermined posture, and in the event of a deviation, a movement 10 from the current posture 8 to the predetermined posture is displayed by means of the digital avatar 9. For example, in the event of a deviation from the determined posture 8, the person 6 can be acoustically alerted to the deviation. Alternatively or additionally, in the event of a deviation, a correction to the current posture 8 can be acoustically communicated to the person 6.

[0040] Furthermore, it can be provided in particular that at least one head position 11 of the person 6 is reflected on the display device 5 by means of the avatar 9.

[0041] It can also be provided that if a posture 8 is recorded as static over a predetermined period of time, a movement request is displayed by means of the avatar 9.

[0042] Furthermore, it can be provided that a current driving situation of the motor vehicle 1 is taken into account when displaying the digital avatar 9, for example that if the person 6 actively intervenes in the driving situation, the display of the digital avatar is suppressed, for example in the case of a driver of the motor vehicle 1.

[0043] In particular, it is thus provided that, for example, the head position is recorded using spatial coordinates as a point between the eyes. Furthermore, the detection of the coordinates of the nose relative to the head coordinate can also be used as an indicator of head rotation, with this preferably being performed using an indoor camera. Furthermore, a breathing rate can also be recorded accordingly.

[0044] Based on the recorded data on head position and rotation, the head of avatar 9 can be positioned if, for example, avatar 9 wants to address or addresses person 6.

[0045] During task instructions or during interaction between person 6 and avatar 9, avatar 9 assumes the exact posture and breathing rate of person 6, which corresponds specifically to so-called pacing. Avatar 9 then preferentially guides person 6 toward correct posture or behavior, particularly toward health-promoting options. This occurs only in appropriate driving situations. Furthermore, recommendations can also be issued via messages or texts.

[0046] In particular, vehicle and driving signals can serve as indicators for corresponding driving situations. The corresponding offers can be implemented, in particular, in accordance with the steering and braking behavior of motor vehicle 1. Furthermore, appropriate time intervals can also be observed; for example, if the driver sinks into an upright posture, becomes immobile, or breath becomes shallow, an appropriate reaction can be initiated.

[0047] If the quality of posture or body tension deteriorates, Avatar 9 actively addresses Person 6 to initiate appropriate movement changes. Person 6 can confirm or ignore the movement suggestion by activating a function or using voice commands.

[0048] For example, if the head position hasn't changed for 30 minutes and breathing becomes increasingly shallow and thus faster, Avatar 9 actively suggests the upright breathing program, particularly by performing appropriate breathing exercises. This can also be confirmed by the client and thus activated or ignored.

[0049] Avatar 9 demonstrates the exercises, describes the program, and can thus encourage the person to imitate the corresponding exercises. In particular, the method can thus consist of the steps of, for example, recording the start of the journey. This is followed by detection or sinking of the head, which in particular moves forward, for example. Avatar 9 then assumes the posture of person 6 and in turn makes a recommendation. In a further step, the corresponding exercises or movement requests are then displayed by avatar 9. Statistics can then be prepared by displaying the health-promoting activities and digital data accordingly for person 6.

Claims

Patent claims Method for operating a support system (2) for a person (6) in a motor vehicle (1), comprising the steps: - detecting the person (6) by means of an optical detection device (3) of the support system (2); - determining a posture (8) of the person (6) by means of an electronic computing device (4) of the support system (2); - generating a digital avatar (9) depending on the detected person (6) and the determined posture (8); and - Displaying the digital avatar (9) with the determined posture (8) on a display device (5) of the support system (2), wherein the determined posture (8) is compared with at least one predetermined posture, and in the event of a deviation, a movement (10) from the current posture (8) to the predetermined posture is displayed by means of the digital avatar (9). Method according to claim 1, characterized in that in the event of a deviation from the determined posture (8), the person (6) is acoustically informed of the deviation. Method according to claim 1 or 2, characterized in that in the event of a deviation, a correction of the current posture (8) is acoustically communicated to the person (6). Method according to one of the preceding claims, characterized in that at least one head position (11) of the person (6) is reflected on the display device (5) by means of the avatar (9). Method according to one of the preceding claims, characterized in that, if a body posture is recorded as static over a predetermined period of time, a movement request is displayed by means of the avatar (9). Method according to one of the preceding claims, characterized in that a current driving situation of the motor vehicle (1) is taken into account when displaying the digital avatar (9). Method according to claim 6, characterized in that, if the person (6) actively intervenes in the driving situation, the display of the digital avatar (9) is suppressed. Computer program product with program code means which cause an electronic computing device (4) to carry out a method according to one of claims 1 to 7 when the program code means are processed by the electronic computing device (4).Support system (2) for a person (6) in a motor vehicle (1), with at least one optical detection device (3), an electronic computing device (4) and a display device (5), wherein the support system (2) is designed to carry out a method according to one of claims 1 to 7.

Images