Method for determining the status of a person

Abstract

The invention relates to a method (30) for determining a state of a person, in particular a vehicle occupant (24), comprising the following steps: observing the person by means of an observation device (22, 44), starting a communication with the person by means of a chat robot (46), observing a reaction of the person to the communication with the chat robot (46) by means of the observation device (22, 44), and determining the state of the person by means of the observation device (22, 44). A state of the person is determined from the identified response of the person.

Description

Background Art

[0001] A method and a device for operating a passenger viewing system are known from DE 10 2014 214 352 A1. Summary of the invention

[0002] A method for determining the state of a person, in particular a vehicle occupant, is proposed, the method comprising the following steps:

[0003] Observe people with the help of observation equipment,

[0004] Start communicating with people with the help of chatbots.

[0005] Observe people's reactions to communication with chatbots with the help of observation equipment.

[0006] The state of the person is determined based on the response of the identified person.

[0007] The proposed method has the advantage that the state of a person can be reliably determined. Preferably, the state of the person can be determined by communicating with the chatbot and analyzing the person's reaction to the communication. For example, it can be determined whether the person is awake and / or responsive, whether there is a medical emergency and / or whether the person is in a critical state, what life state and / or fatigue stage the person is in, whether a dangerous situation prevails or other. If the person is in a vehicle as a vehicle occupant, this can improve the safety of the vehicle occupants in the vehicle and thus also the safety in road traffic.

[0008] In an exemplary implementation, the observation device may include one or more cameras and / or one or more radar sensors and / or one or more lidar sensors and / or one or more microphones, thereby safely and / or reliably generating real-time data of monitoring or observing a person, thereby safely and / or reliably determining the state of the person.

[0009] Advantageously, if a change in the state of the person is detected by means of the observation device, communication with the person using the chatbot is initiated. This allows communication with the person, if necessary, in particular in a targeted manner. This allows the state of a person, in particular a vehicle occupant, to be detected, if necessary. By detecting the state, in particular the safety of the person, in particular in the vehicle, can be increased. Furthermore, distraction of the vehicle occupants in the vehicle can be reduced by the targeted initiation of communication with the chatbot.

[0010] In an improved solution, the chatbot can output acoustic signals and / or visual signals for communication. Preferably, the communication with the person using the chatbot can be constructed as a dialogue. Alternatively or additionally, the chatbot can output text on a display for communication. This allows and executes communication with the person on one or more different channels. This ensures that the person perceives and executes the communication. For example, the driver's state can also be determined by acoustic communication by means of a dialogue, because the driver should be fully focused on driving.

[0011] In an advantageous embodiment, the acoustic response of the person can be observed by means of an observation device. For example, the acoustic response can be analyzed to determine the state of the person based on the speech rate, speech errors, grammatical errors, clarity of pronunciation, intonation and / or content of the response. This can advantageously identify the state of the person safely and / or reliably. For example, by analyzing the conversation or the acoustic response, the person in the vehicle can be analyzed and the state can be identified therefrom, wherein the state of the driver who must continue to concentrate on driving can also be identified.

[0012] In a further development, the person can input text as a response by means of an input device, wherein the state of the person is determined based on the input of the text. For example, the input of the text can be analyzed to determine the state of the person based on the speed, grammar, typing errors and / or content of the input or response. This can advantageously identify the state of the person safely and / or reliably. For example, by analyzing the input, the person in the vehicle can be analyzed and the state can be identified thereby.

[0013] Advantageously, the method can also include the further step of triggering a function as a function of the determined state of the person. For example, a warning can be output, a warning can be sent and / or a preset can be output to the vehicle occupants and / or a driving safety function can be triggered and / or an adaptation of the automatic driving mode can be triggered. In particular, suitable actions can be performed as a reaction to the determined state of the person. If the person is in a critical state and / or in danger, a helper can advantageously be warned. In particular, safety for the person can be increased in this way.

[0014] Furthermore, a system for determining the state of a person, in particular a vehicle occupant, is proposed, wherein the system comprises an observation device for observing the person, and wherein the system comprises a chatbot for communicating with the person. The reaction of the person to the communication with the chatbot can be observed by means of the observation device, wherein the state of the person can be determined based on the detected reaction of the person.

[0015] Advantageously, the state of the person can be reliably determined by communicating with the chatbot and analyzing the person's reaction to the communication. This can determine, for example, whether the person is awake and / or responsive, whether there is a medical emergency and / or whether the person is in a critical state, what life state and / or fatigue stage the person is in, whether a dangerous situation is prevalent, or other situations. If the person is in a vehicle as a vehicle occupant, this can improve the safety of the vehicle occupants in the vehicle and thus also the safety in road traffic.

[0016] Advantageously, the observation device may include one or more cameras and / or one or more radar sensors and / or one or more lidar sensors and / or one or more microphones, thereby safely and / or reliably generating real-time data of monitoring or observing a person, thereby safely and / or reliably determining the state of the person.

[0017] In a further development, the system may include an input device for inputting a reaction to an exchange with the chatbot. Data for monitoring a person can thus be generated safely and / or reliably by means of the input device, thereby safely and / or reliably determining the state of the person. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Exemplary embodiments of the invention are shown in the drawings and explained in more detail in the following description. Elements shown in different figures and acting similarly are given the same reference numerals, wherein a repeated description of these elements is not necessary.

[0019] Figure 1 A schematic diagram showing a vehicle having a device for observing a vehicle occupant;

[0020] Figure 2 A schematic diagram showing a method according to an embodiment of the present invention;

[0021] Figure 3 A schematic diagram showing a method according to an embodiment of the present invention;

[0022] Figure 4 A schematic diagram showing a system according to an embodiment of the present invention;

[0023] Figure 5 A schematic diagram of a system according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0024] Figure 1A schematic diagram of a vehicle 20 is shown, which is, for example, a motor vehicle, such as a car, having an observation device 22 for observing one or more vehicle occupants 24 or passengers 24. The vehicle 20 has, in particular, an interior space 26 or a vehicle interior space 26, wherein in the vehicle interior space 26, in particular, one or more seats 27 for one or more vehicle occupants 24 can be arranged. In addition, the vehicle 20 has an observation device 22 for observing the vehicle interior space 26 and / or one or more vehicle occupants 24 (for example, a driver and / or a co-driver and / or one or more other passengers). The observation device 22 can also be referred to as an observation system, a driver monitoring system, a passenger observation system and / or a monitoring system. The observation device 22 can be configured, for example, to detect the direction of sight, the body posture and / or the position of the head or face or eyes of the vehicle occupant 24 or the fatigue state and / or other vital signs of the vehicle occupant 24. In addition, for example, the identity of the vehicle occupant 24 can be detected.

[0025] The observation device 22 can be arranged in particular in the dashboard, in the instrument panel, in the steering wheel, on the roof, on the window glass, on the rearview mirror or on the vehicle pillar, for example on the A-pillar and / or B-pillar of the vehicle 20, or at another location in the vehicle 20.

[0026] In order to observe the vehicle occupant 24, the observation device 22 can have a recording unit. In a further development, the observation device 22 can have an illumination unit for emitting light, in particular infrared light. Alternatively, the observation device 22 can have no separate illumination unit and can illuminate the vehicle interior or the vehicle occupant with the aid of light from the environment.

[0027] The lighting unit can be pointed in particular in the direction of the vehicle occupant 24 and thus in the direction of the vehicle seat 27 in order to illuminate the vehicle occupant 24 with light, in particular with infrared light. In other words, light, in particular infrared light, can be emitted by means of the lighting unit in the direction of the vehicle occupant 24. The lighting unit can be designed, for example, as a lighting unit, a lighting element, a light emitting diode, an LED, an OLED and / or a laser diode and / or has a lighting unit, a lighting element, a light emitting diode, an LED, an OLED and / or a laser diode.

[0028] The recording unit can be designed, for example, as an image recording unit, for example as a sensor or a camera, in particular as an infrared camera module, wherein the recording unit is pointed in the direction of the vehicle occupant 24 and thus in the direction of the vehicle seat 27 in order to visually detect the vehicle occupant 24. By means of the design as an infrared camera module, it is possible to carry out observation even at night without the vehicle occupant 24 being brightly illuminated and thereby in particular being disturbed or blinded.

[0029] In other words, the observation device 22 is configured to observe the vehicle interior of the vehicle. The observation device 22 may include an illumination unit for emitting light, such as infrared light, in the direction of the vehicle interior and a recording unit for recording light, such as infrared light. Preferably, the light emitted by the illumination unit may be reflected at or in the vehicle interior, wherein the reflected light may be deflected in the direction of the recording unit.

[0030] Furthermore, the monitoring device 22 has a control unit 28 or an evaluation unit 28 or a computing unit 28 for controlling the lighting unit and / or the recording unit and / or for processing the data recorded by means of the recording unit.

[0031] Figure 2 A schematic diagram of a method 30 for determining a state of a person, in particular a vehicle occupant, according to an exemplary embodiment of the present invention is shown. Figure 4 and / or Figure 5 A system of systems is used to perform method 30.

[0032] In a first step 32 of the method 30, a person is observed by means of an observation device. Advantageously, one or more persons can be observed. The observation device can include at least one camera and / or at least one radar unit and / or at least one lidar unit and / or at least one microphone, or the observation device can be designed as at least one camera, at least as a radar unit and / or at least as a microphone. In this way, in particular visual and / or acoustic monitoring of the person can take place.

[0033] In a second step 34 of the method 30, communication with the person is started by means of the chatbot. Advantageously, communication with the person can be started by means of the chatbot if a change in the state of the person is detected by means of the observation device. In other words, communication can be started if the state of the person changes and / or if the situation or atmosphere changes. For example, it can be detected that there is no longer any movement in the image, too much movement or the type of movement has changed, it can be detected that the volume of the chattering has changed or that the noise is no longer detected. Alternatively or additionally, communication can be started manually and / or after a certain time.

[0034] The chatbot can output acoustic signals and / or visual signals for communication. Advantageously, communication with a person using the chatbot can be constructed as a dialogue. In other words, the person and the chatbot can communicate with each other by voice, in particular talk to each other. Alternatively or additionally, the chatbot can output text on a display for communication. In other words, communication can be carried out by chatting, wherein the chatbot outputs text, and the person responds by means of text and / or also by means of voice. The types of communication can advantageously be combined with each other. For example, the chatbot can output text and the person can speak by means of an input device and / or respond, or the chatbot can output words and / or sentences by means of a loudspeaker and the person can respond, for example, as text, by means of an input device.

[0035] In a third step 36 of the method 30, the reaction of the person to the communication with the chatbot is observed by means of an observation device. The observation can be performed, for example, as a visual observation by means of an observation camera. For example, how the person behaves when reacting can be observed. Advantageously, the person's body language and / or facial expressions can be observed and analyzed.

[0036] In a refinement, the acoustic response of the person can be observed by means of an observation device. Advantageously, the acoustic response of the person can be recorded by means of a microphone, and the data can be analyzed therefrom. For example, the acoustic response can be analyzed to determine the state of the person based on the speech rate, speech errors, grammatical errors, articulation, intonation and / or content of the response. In other words, the state of the person can be identified based on how the person reacts to the conversation of the chatbot.

[0037] In a further development, the person can input text as a response by means of an input device, wherein the person's state is determined based on the input of the text. Advantageously, the person's response can be performed by means of an input device and thus observed, and data can be analyzed from this. For example, the input of text can be analyzed to determine the person's state based on the speed, grammar, typos and / or content of the input or response. In other words, the person's state can be identified based on how the person performs the input and / or based on the input itself.

[0038] In a fourth step 38 of the method 30, the state of the person is determined based on the person's identified reaction. For example, the vital state of the person can be identified. For example, it can be identified whether the person is awake, tired, sleepy or unconscious, whether there is a critical health state or a medical emergency, whether the person is injured and / or whether the person is in a dangerous situation. Advantageously, further states can be identified. In other words, the vital state of the person can be determined or estimated. For this purpose, a chatbot can be used to communicate with the person. For example, a chatbot can also be called a web chatbot. For example, a person can be located in a vehicle and is therefore a vehicle occupant. The person can be, for example, a driver or other vehicle occupant.

[0039] For example, a chatbot can be used in conjunction with an image of an observation device to ask a person, such as a driver, how the driver feels. For example, the chatbot can be activated by different inputs, such as camera input or other measurement methods. In a refinement, the chatbot can talk to a person. In addition, the camera can observe the person. The evaluation unit can analyze the observation data of the person and analyze the person's reaction to the conversation from this. In an advantageous design, the chatbot can involve the person in a conversation. For example, the person can be asked about the environment, such as the environment of the vehicle. For example, the person should describe the environment. With this information, the state of the person can be estimated. As an input or reference for how the environment looks, an environmental camera can be used. The description of the person can be compared with the image data of the surrounding cameras. In a refinement, a speech to text module can be used to identify and analyze speech errors. For example, the chatbot can ask a person a question. How the person answers can indicate the state of the person. For example, the person may answer slowly and / or tiredly and, for example, there may be characteristic speech errors in the answer. For example, this may indicate a non-conscious state. If the person is in a vehicle, for example, this may indicate an inability to drive state.

[0040] Figure 3 A schematic diagram of a method 30 for determining a state of a person, in particular a vehicle occupant, according to an exemplary embodiment of the present invention is shown. Figure 4 and / or Figure 5 The method 30 is performed by a system of systems. Figure 3 The method 30 may be performed, for example, according to Figure 2 Therefore, the method 30 may have a first step 32, a second step 34, a third step 36 and a fourth step 38.

[0041] according to Figure 3 The method 30 has a further fifth step 40. In the fifth step 40 of the method 30, functions are triggered depending on the determined state of the person. For example, a warning can be output, a warning can be sent and / or a driving safety function can be triggered. The warning can be output, for example, by means of an acoustic signal and / or a visual signal, for example by means of a loudspeaker and / or a display. Alternatively or additionally, the warning can be sent to an external unit, for example to an external server. Thus, for example, an emergency message can be issued. If the person is in the vehicle, for example, a safety function can be triggered. For example, an autonomous driving state can be established and / or the vehicle can be brought to a safe stop.

[0042] The vehicle can also be designed in particular as an autonomous vehicle. In order to be able to estimate whether the driver is able to take over manual control again from the automated state, the driver's attention state should advantageously be estimated, in particular with regard to an upcoming takeover of the driving task. If it is estimated that the attention is too low, the occupants or a brake jolt can be prompted in order to encourage the driver to pay more attention again. An upcoming planned takeover of the driving function can also be prepared by talking to the chatbot.

[0043] Figure 4 A schematic diagram of a system 42 for determining the state of a person, in particular a vehicle occupant, according to an exemplary embodiment of the present invention is shown. With the aid of the system 42, a Figure 2 and / or Figure 3 The system can be, for example, based on Figure 5 Arranged in the vehicle.

[0044] The system 42 has an observation device 44 for observing a person. The observation device 44 may include one or more cameras and / or one or more radar sensors and / or one or more lidar sensors and / or one or more microphones or may be configured as at least one camera and / or at least one radar unit and / or at least one lidar unit and / or at least one microphone. The observation device 44 may be configured, for example, according to Figure 1 The observation equipment is arranged in the vehicle.

[0045] The system 42 further includes a chatbot 46 for communicating with people. The chatbot is in particular a text-based dialogue system that enables communication between people and technical systems. The chatbot 46 includes an input unit and an output unit. For example, data can be processed on a computing unit or server by means of an algorithm. For example, the chatbot 46 can learn using artificial intelligence. The chatbot 46 can be constructed, for example, to output acoustic signals and / or visual signals. For example, the system can have a speaker and / or a display for this purpose. People can communicate with the chatbot 46, in particular, via acoustic communication and / or visual communication. People can, for example, talk to the chatbot 46 and / or input via an input device. For this purpose, the system 42 can, for example, include an input device for inputting a response to the communication with the chatbot 46. In other words, people can react to the output of the chatbot 46 with a specific reaction. This reaction of people to the communication with the chatbot 46 can be observed by means of an observation device. In other words, people's reactions can be observed by means of an observation device. For this purpose, for example, image data of a camera, sound data of a microphone and / or input data can be evaluated or analyzed by means of an input device. The system 42 has a logic circuit or a computing unit 48 for evaluating the data and for determining the state of the person. In other words, the data of the observation device 44 and the chat robot 46 can be evaluated by means of the computing unit 48.

[0046] The system 42 further has an output unit 50 for triggering a function as a function of the determined state of the person. For example, a warning can be output and / or a warning can be sent and / or a driving safety function can be triggered by means of the output unit 50.

[0047] Figure 5 A schematic diagram of a system 42 for determining the state of a person, in particular a vehicle occupant, according to an exemplary embodiment of the present invention is shown. With the aid of the system 42, a Figure 2 and / or Figure 3 The system 42 may be configured, for example, according to Figure 4 Thus, the system 42 has an observation device 44, a chat robot 46, an evaluation unit 48 and an output unit 50. In this advantageous embodiment, the system 42 is arranged in the vehicle 20. The vehicle 20 can be configured, for example, according to Figure 1 of vehicles to construct.

Claims

1. A method (30) for determining a state of a person, in particular a vehicle occupant (24), comprising the following steps: observing the person by means of an observation device (22, 44), starting to communicate with said person by means of a chatbot (46), observing the person's reaction to the communication with the chat robot (46) by means of the observation device (22, 44), The state of the person is determined based on the identified reaction of the person.

2. The method (30) according to claim 1, wherein the observation device (22, 44) comprises one or more cameras and / or one or more radar sensors and / or one or more lidar sensors and / or one or more microphones.

3. The method (30) according to any one of the preceding claims, wherein if a change in the state of the person is detected with the aid of the observation device (22, 44), a communication with the person using the chatbot (46) is started.

4. The method (30) according to any one of the preceding claims, wherein the chatbot (46) outputs acoustic signals and / or visual signals for communication.

5. The method (30) according to any one of the preceding claims, wherein the communication with the person using the chatbot (46) is designed as a dialogue.

6. The method (30) according to any one of the preceding claims, wherein the chatbot (46) outputs text on a display for communication.

7. The method (30) according to any of the preceding claims, wherein an acoustic reaction of the person is observed by means of the observation device (22, 44).

8. The method (30) of claim 7, wherein the acoustic response is analyzed to determine the person's status based on speech rate, phonetic errors, grammatical errors, clarity of pronunciation, tone and / or content of the response.

9. The method (30) according to any one of the preceding claims, wherein the person inputs a text as a reaction by means of an input device, and wherein the state of the person is determined as a function of the input of the text.

10. The method (30) of claim 9, wherein the textual input is analyzed to determine the state of the person based on speed, grammar, typos, and / or content of the input.

11. The method (30) according to any of the preceding claims, comprising the further step of triggering a function depending on the determined state of the person.

12. The method (30) according to claim 11, wherein a warning is output, a warning is sent and / or a preset is output to a vehicle occupant and / or a driving safety function is triggered and / or an adaptation of the automated driving mode is triggered.

13. A system (42) for determining the state of a person, in particular a vehicle occupant (24), characterized in that The system (42) includes an observation device (22, 44) for observing the person, and the system (42) includes a chat robot (46) for communicating with the person. With the help of the observation device (22, 44), the person's reaction to the communication with the chat robot (46) can be observed, and the person's state can be determined based on the recognized reaction of the person.

14. The system (42) according to claim 13, characterized in that The observation device (22, 44) includes one or more cameras and / or one or more radar sensors and / or one or more lidar sensors and / or one or more microphones.

15. System (42) according to any one of the preceding claims 13 to 14, characterized in that The system (42) includes an input device for inputting responses to exchanges with the chatbot (46).

Images