Agent Device

Abstract

To provide an agent device which relieves occupants at a traffic trouble and facilitates a calm response.SOLUTION: An agent device 100 includes: an impact detection sensor 140g which detects an impact of a predetermined value or larger on a vehicle; and an agent control unit 190 which controls a personalized agent. When the impact detection sensor detects an impact of a predetermined value or larger, the agent control unit directly faces an occupant, and performs questioning (talking to relax the occupant in panic) and action (making a big motion to make the occupant calm down) for confirming the safety of the occupant.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present invention relates to an agent device capable of making notifications using personified characters. [Background technology]

[0002] BACKGROUND ART Conventionally, various methods have been proposed as methods for alerting occupants (particularly the driver) in a driving environment of a vehicle such as an automobile. As one such method, an agent device or driving assistance device has been proposed in which an occupant and an anthropomorphized character (a so-called agent) interact with each other to provide information on driving assistance in response to the occupant's requests.

[0003] For example, in conventional driving assistance devices, it is thought that notifications of danger avoidance actions using voice or other means are difficult to convey intuitively, so a proposal has been made to place a small robot on the dashboard, appear as a personified agent, and have this agent simulate the danger avoidance actions that the driver should take (see Patent Document 1). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2018-032204 Summary of the Invention [Problem to be solved by the invention]

[0005] However, conventional agents were impersonal or appeared suddenly to draw attention, making it difficult to build a relationship of trust between the agent and the driver. Also, even if the agent suddenly took the same action as an evasive action, it was difficult for the driver to take that action immediately if they did not understand the cause of the evasive action, and there was a problem that evasive action would not be taken unless a relationship of trust was established. Furthermore, if a traffic accident occurs, such as a collision between a vehicle and another vehicle or an obstacle, anyone can easily fall into a panic. In such a situation, receiving accurate instructions would be incredibly reassuring. However, there is a problem that agents are not adequately prepared to handle such situations. Furthermore, while some vehicles equipped with current emergency notification systems make impersonal announcements, passengers may feel uneasy if the announcement is made in a voice they have never heard before and they do not know where it is coming from, which can lead to a lack of calm response.

[0006] The present invention has been made to solve such conventional problems, and has as its object to provide an agent device that can give passengers a sense of security and encourage them to respond calmly when a traffic trouble occurs. [Means for solving the problem]

[0007] The agent device of the present invention comprises an impact detection unit that detects an impact to a vehicle of a predetermined value or greater, and an agent control unit that controls a personified agent, and when the impact is detected, the agent control unit faces the occupant and asks questions and takes action to check on the occupant's safety.

[0008] According to the present invention, an agent device can be provided that can give passengers a sense of security and encourage them to respond calmly when a traffic accident occurs. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a perspective view showing an interior of a vehicle according to an embodiment of the present invention; [Figure 2] FIG. 2 is a front view showing the interior of the vehicle in front of the driver's seat. [Figure 3] FIG. 10 is a front view showing the placement of agents. [Figure 4] FIG. 2 is a schematic diagram showing a configuration of an agent according to an embodiment of the present invention. [Figure 5]FIG. 2 is a block diagram illustrating an agent device. [Figure 6] 10 is a main flowchart of an agent control process performed by a control unit of the agent device. [Figure 7] 10 is a flowchart showing a process performed by a control unit of the agent device when an impact is detected. [Figure 8] FIG. 10 is a diagram illustrating an example of a change in the direction of an agent. [Figure 9] FIG. 10 is an operational diagram showing an example of an agent's operation for calming down a passenger. [Figure 10] 10A and 10B are diagrams showing an example of an action for having an agent check each part of the body of a passenger. [Figure 11] FIG. 10 is an operational diagram showing an example of an agent's guidance operation in the escape direction. [Figure 12] FIG. 10 is an operational diagram showing an example of an escape guidance operation by an agent. [Figure 13] FIG. 10 is a diagram illustrating an example of a telephone call guidance operation by an agent. [Figure 14] FIG. 10 is a diagram illustrating an example of an agent's listening posture during the accompanying action leading up to rescue. [Figure 15] FIG. 10 is a motion diagram showing an example of a conversation posture in an agent's approaching motion leading up to rescue. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following description, the same reference numerals in different drawings indicate parts with the same functions, and duplicated descriptions in each drawing will be omitted as appropriate.

[0011] (Vehicle interior configuration) As shown in Figures 1 and 2, vehicle 1 has an instrument panel 3 provided in front of a driver's seat 2, and a steering wheel 4 disposed between the driver's seat 2 and the instrument panel 3. The steering wheel 4 is rotatably attached to a steering column (not shown) via a steering shaft (not shown). An airbag 8 is stored inside the steering wheel 4, which is deployed toward the driver H in the event of a collision or the like of the vehicle 1. In the present embodiment, the agent device 100 (described later) and the agent 200 (described later) are described based on dialogue and gestures (movements) with the driver H, but they can also be performed in the same way with other passengers.

[0012] 1, two display panels, a first display panel 30 and a second display panel 40, are provided in the cabin of a vehicle 1. The first display panel 30 is disposed on the instrument panel 3 in front of the driver's seat 2, and the second display panel 40 is disposed on the instrument panel 3 on the left front side of the driver's seat 2.

[0013] (First display panel 30) 1 and 2, the first display panel 30 is configured to include a pointer-type meter configured like an analog clock and a so-called liquid crystal display device in which a liquid crystal panel and a backlight are integrally provided. The driver H can view various information displayed in the first display area 30a of the first display panel 30 through the upper space 4a of the steering wheel 4.

[0014] 2, first display panel 30 is provided with first display area 30a, and on the left and right sides of first display area 30a are two pointer-type meters that display information such as the traveling speed of vehicle 1 (speedometer) and the number of revolutions per unit time of the engine (tachometer), and a small liquid crystal display device that displays an image showing general vehicle information is disposed between the two pointer-type meters and in the center of first display panel 30. Note that first display panel 30 may not be provided with a pointer-type meter and may be configured entirely with a single liquid crystal display device.

[0015] (Second display panel 40) 1, the second display panel 40 is configured as a so-called liquid crystal display device in which a liquid crystal panel and a backlight are integrally provided, for example. Note that the second display panel 40 displays map information and the like, and serves as a so-called car navigation system.

[0016] The liquid crystal display devices configured in the first display panel 30 and the second display panel 40 may be configured as display devices such as a self-luminous display device such as a plasma display or an organic EL display, or a projection type projector.

[0017] (Agent 200) As shown in FIG. 3, an agent 200 is disposed on the upper part of the instrument panel 3 in front of the driver's seat 2. The agent 200 may be located anywhere as long as it is within the peripheral vision of the driver H, and does not necessarily have to be located directly in front of the driver's seat. It is desirable that the agent 200 be located within the peripheral vision of the driver H, but at a position that does not obstruct the driver H's view of the outside of the vehicle. The location of the agent 200 may also be changed using a predetermined operating device or voice instructions, etc.

[0018] Agent 200 is an anthropomorphized character, a three-dimensional object with a humanoid external shape, used for notification and for interaction. In this embodiment, agent 200 is a three-dimensional object that is a real humanoid image, but it is not limited to this and may be a character displayed on a display device such as a screen, a three-dimensional object expressed by a hologram or the like, a virtual three-dimensional object, or the like.

[0019] As shown in FIG. 4, the agent 200 has a head 211, a right eye 212R, a left eye 212L, a neck 213, a chest 214, a waist 215, a buttocks 216, and a right shoulder 2. 21R, left shoulder 221L, right upper arm 222R, left upper arm 222L, right elbow 223R, left elbow 223L, right forearm 224R, left forearm 224L, right wrist 22 5R, left wrist 225L, right hand 226R, left hand 226L, right hip 231R, left hip 231L, right thigh 232R, left thigh 232L, right knee 2 33R, a left knee part 233L, a right lower leg part 234R, a left lower leg part 234L, a right ankle part 235R, a left ankle part 235L, a right foot part 236R, and a left foot part 236L.

[0020] The agent 200 is also provided with a support connector 240, which supports the agent 200 so that it can be placed upright on the instrument panel 3. Incidentally, ears are provided on both sides of the head 211 of the agent 200. This makes it appear as if the agent 200 is listening carefully to what the driver H is saying. Even if these ears are not actually provided, as will be described later, when the agent 200 listens to what the driver H is saying, the agent 200 can make it appear as if he is listening with his hands on his ears by placing his right hand 226R or left hand 226L on the side of the head 211 of the agent 200, thereby deepening the relationship of trust between the agent 200 and the driver H.

[0021] Note that Figure 4(a) is a diagram showing the basic posture of the agent 200 when facing the driver H (the rear side of the vehicle 1), and Figure 4(b) is a diagram showing the basic posture of the agent 200 when facing away from the driver H (facing the front side of the vehicle 1). In addition, in the basic posture, the agent 200 stands with the right palm portion 226Ra of the right hand portion 226R and the left palm portion 226La of the left hand portion 226L facing toward the inside of the agent 200, and the right back portion 226Rb of the right hand portion 226R and the left back portion 226Lb of the left hand portion 226L facing toward the outside of the agent 200. As described above, the agent 200 can freely rotate between a front posture facing the driver H and a rear posture facing away from the driver H. In other words, the agent 200 may be one in which the agent 200 itself rotates around the support connector 240, or the agent 200 may be placed on a rotating body and rotated from the front posture to the rear posture and from the rear posture to the front posture by rotating the rotating body.

[0022] An agent control unit 190, which will be described later, is provided within the agent 200, and controls and operates each joint of the agent 200, allowing the agent 200 to assume a predetermined posture. The agent control unit 190 is wirelessly connected to a control unit 110 (ECU), which will be described later, and operates the agent 200 under the control of the control unit 110. Note that, although the agent control unit 190 and the control unit 110 are connected wirelessly in this embodiment, they may also be connected by wire, for example, by passing a communication line through the support connector 240.

[0023] An agent speaker 191, which will be described later, is provided in the head 211 of the agent 200, and is configured to output audio under the control of the agent control unit 190. Note that the agent speaker 191 is not limited to being provided in the head 211 of the agent 200, but may be provided in another part of the agent 200 or in another location of the vehicle 1. However, providing the agent speaker 191 in the head 211 of the agent 200 allows the driver H to perceive the utterances from the agent 200 more realistically, makes it easier for the driver H to empathize with the agent 200, and can greatly contribute to building trust in the agent 200 by the driver H.

[0024] Next, the configuration of the agent device 100 according to the present invention will be described with reference to the block diagram of FIG.

[0025] As shown in FIG. 5, the agent device 100 includes a control unit 110, a surrounding environment recognition unit 120, an occupant state recognition unit 130, a vehicle state recognition unit 140, a speaker 150, a microphone 160, a memory unit 170, a transceiver 180, an agent control unit 190, and an agent speaker 191.

[0026] 5 is merely an example, and the components of the agent device 100 can be modified as appropriate. For example, the agent device 100 of the present invention can be realized with only either the speaker 150 or the agent speaker 191. Furthermore, an agent microphone may be provided within the agent 200 in addition to the microphone 160, and the agent device 100 of the present invention can also be realized with only an agent microphone provided within the agent 200 instead of the microphone 160. Furthermore, in this embodiment, both the control unit 110 and the agent control unit 190 are provided, but this is not limiting, and it is also possible to have only one of the control unit 110 or the agent control unit 190 and have it perform the functions of both.

[0027] (control unit 110) The control unit 110 is equipped with a CPU, ROM, RAM (e.g., a ring buffer), EEPROM, input / output ports, etc. (not shown), and for example, when information is input through the input port, it controls various devices through the output port based on a control program read from the ROM.

[0028] The ROM of the control unit 110 stores a data table (not shown) of actions and lines to be performed by the agent 200. Furthermore, a learning function such as AI may be provided so that the agent 200 performs actions and lines other than those stored. The CPU of the control unit 110 determines the actions and lines to be performed by the agent 200 from the data table based on information acquired from each recognition unit (described later) and information acquired from the microphone 160. For example, when an utterance from the driver H is acquired from the microphone 160, a "nod" gesture is determined from the data table, and when the driver H finishes speaking, a line such as "I understand" is determined. The data table of actions and lines to be performed by the agent 200 may be stored in the ROM of the agent control unit 190.

[0029] (Surrounding Environment Recognition Division 120) The surrounding environment recognition unit 120 is provided to recognize the surrounding environment of the vehicle 1 (host vehicle). The surrounding environment recognition unit 120 also includes an exterior camera 120a, a radar 120b, a temperature sensor 120c, a weather sensor 120d, and a navigation device 121, and is capable of recognizing the surrounding environment and surrounding conditions of the vehicle 1 using these devices. For example, the surrounding environment recognition unit 120 is capable of detecting the presence of corners or turning points in the traveling direction, the presence or absence of approaching objects, and the like. In this embodiment, the navigation device 121 is configured to be included as part of the surrounding environment recognition unit 120, but this is not limiting, and the navigation device 121 may be included independently, separate from the surrounding environment recognition unit 120.

[0030] (Exterior camera 120a) The exterior camera 120a is attached to, for example, a rearview mirror (not shown) and is capable of capturing images of the area in front of and behind the vehicle 1. The captured image information is input to the control unit 110, which then stores the image information in RAM. This allows the control unit 110 to recognize the situation in front of and behind the vehicle 1 in real time and after the fact.

[0031] (Radar 120b) The radar 120b is, for example, a millimeter-wave radar that detects obstacles and the like by transmitting radio waves. The millimeter-wave radar is attached to the front bumper or rear bumper of the vehicle 1, and is capable of monitoring the area ahead of the vehicle 1, the areas in front of the vehicle 1, and the areas behind the vehicle 1. The monitoring information is input to the control unit 110, which stores the monitoring information in RAM. This allows the control unit 110 to recognize the situation ahead of the vehicle 1, the areas in front of the vehicle 1, and the areas behind the vehicle 1 in real time and after the fact. Note that although a millimeter-wave radar is used in this embodiment, other radars may also be used. For example, infrared radar may be used.

[0032] (Temperature sensor 120c) The air temperature sensor 120c is attached to, for example, the rear of the vehicle 1, and is capable of detecting the air temperature outside the vehicle 1. The detected air temperature is input to the control unit 110, which then stores the air temperature in RAM. This allows the control unit 110 to recognize the air temperature outside the vehicle 1.

[0033] (Weather Sensor 120d) The weather sensor 120d is attached, for example, to the top of the vehicle 1 and is capable of detecting raindrops and the like falling on the vehicle 1. The detected water droplet information is input to the control unit 110, which then stores the water droplet information in RAM. This allows the control unit 110 to recognize the weather conditions outside the vehicle 1.

[0034] (Navigation device 121) The navigation device 121 provides route guidance according to the destination, waypoints, etc. The navigation device 121 also has map information, a GPS, a VICS receiver, etc., and is capable of acquiring map information on the current position of the vehicle 1, surrounding area information, traffic congestion information, road regulation information, parking lot information, etc. Therefore, the control unit 110 can recognize the surrounding situation of the vehicle 1 in real time based on information acquired by the navigation device 121 as well as the outside-vehicle imaging camera 120a, radar 120b, etc. The navigation device 121 can also receive map information at any time, enabling it to acquire the latest map information. The transmitter / receiver of the navigation device 121 may also serve as the transceiver 180, which will be described later.

[0035] In this way, the control unit 110 can recognize the surrounding environment and surrounding situation of the vehicle 1 in real time and after the fact. As a result, the control unit 110 can control the agent 200 and the agent speaker 191 based on the recognition of the surrounding environment and surrounding situation of the vehicle 1 to notify the driver H of information about the surrounding environment and surrounding situation of the vehicle 1. For example, if there is a fallen object in front of the vehicle 1, it can notify the driver H by gesture and voice, such as "There is a fallen object ahead." This can improve safety.

[0036] In this embodiment, the surrounding environment recognition unit 120 includes the exterior camera 120a, the radar 120b, the temperature sensor 120c, the weather sensor 120d, and the navigation device 121, but these are merely examples, and other devices may of course be used.

[0037] (Occupant state recognition unit 130) The occupant state recognition unit 130 is provided to recognize the state of the driver H. In addition, the occupant state recognition unit 130 is equipped with an occupant image capturing camera 130a and a vital sensor 130b, and is able to recognize the state of the driver H using these devices.

[0038] (Occupant photography camera 130a) The occupant photographing camera 130a is attached to, for example, the instrument panel 3, and is capable of photographing the driver H. Then, photographed image information is input to the control unit 110, which stores the image information in RAM. This allows the control unit 110 to recognize the state of the driver H in real time and after the fact. Note that the state of the driver H referred to here specifically includes the state of the driver H's eyelids, the number of blinks, the direction of the driver's gaze, the direction of the face, whether the driver is crouched, or whether the driver is unable to move, etc.

[0039] (Vital Sensor 130b) The vital sensor 130b is attached, for example, to a part of the steering wheel 4 that is gripped by the driver H, and is capable of acquiring vital information such as the heart rate and blood pressure of the driver H. The acquired vital information is then input to the control unit 110, which then stores the vital information in RAM. This allows the control unit 110 to recognize the state of the driver H in real time and after the fact.

[0040] In this way, the control unit 110 can recognize the state of the driver H in real time and after the fact. As a result, the control unit 110 can control the agent 200 and the agent speaker 191 based on the recognition of the state of the driver H to notify the driver H of predetermined information. For example, it is possible to notify the driver H by image and sound such as "Your eyelids are drooping. Would you like to take a break?" or "Your heart rate is faster than usual. Would you like to take a break?" This leads to improved safety.

[0041] The occupant state recognition unit 130 can recognize the thoughts and emotions of the driver H to a certain extent based on information acquired from the occupant image capturing camera 130a and the vital sensor 130b, and information input from the microphone 160. For example, the occupant state recognition unit 130 can acquire the facial expression of the driver H from the occupant image capturing camera 130a, acquire the heart rate and blood pressure of the driver H from the vital sensor 130b, and acquire the voice volume and input content from the microphone 160, and can recognize from this acquired information whether the driver H is thinking and feeling normal thoughts and emotions or thoughts and emotions that are different from normal thoughts and emotions (for example, surprised, angry, etc.).

[0042] In this embodiment, the occupant state recognition unit 130 is configured using the occupant image capturing camera 130a and the vital sign sensor 130b, but this is merely an example, and other devices may of course be used.

[0043] (Vehicle state recognition unit 140) The vehicle state recognition unit 140 is provided to recognize the state of the vehicle 1. The vehicle state recognition unit 140 also includes a vehicle speed sensor 140a, a steering wheel angle sensor 140b, an accelerator pedal sensor 140c, a brake pedal sensor 140d, a G sensor 140e, a shift position sensor 140f, and an impact detection sensor 140g, and is able to recognize the state of the vehicle 1 using these devices.

[0044] (vehicle speed sensor 140a) The vehicle speed sensor 140a is a sensor for detecting the vehicle speed of the vehicle 1. The detected vehicle speed is input as a vehicle speed signal to the control unit 110, and the control unit 110 stores the vehicle speed information in RAM. This allows the control unit 110 to recognize the vehicle speed of the vehicle 1 in real time and after the fact.

[0045] (Steering wheel angle sensor 140b) The steering wheel angle sensor 140b is a sensor for detecting the steering wheel angle (angle of the steering wheel 4) of the vehicle 1, and the detected steering wheel angle is input as an angle signal to the control unit 110, which then stores the angle information in RAM. This allows the control unit 110 to recognize the steering wheel angle (angle of the steering wheel 4) of the vehicle 1 in real time and afterwards.

[0046] (Accelerator pedal sensor 140c) The accelerator pedal sensor 140c is a sensor for detecting the depression amount of an accelerator pedal (not shown), and the detected depression amount is input as a depression amount signal to the control unit 110, which then stores the depression amount information in RAM. This allows the control unit 110 to recognize the depression amount of the accelerator pedal of the vehicle 1 in real time and after the fact.

[0047] (Brake pedal sensor 140d) The brake pedal sensor 140d is a sensor for detecting the depression amount of a brake pedal (not shown), and the detected depression amount is input as a depression amount signal to the control unit 110, which then stores the depression amount information in RAM. This allows the control unit 110 to recognize the depression amount of the brake pedal of the vehicle 1 in real time and afterwards.

[0048] (G sensor 140e) The G sensor 140e is a sensor for detecting the acceleration, deceleration, and inclination of the vehicle 1. When acceleration is detected, the acceleration amount is input as an acceleration amount signal, when deceleration is detected, the deceleration amount is input as a deceleration amount signal, and when inclination is detected, the inclination angle amount is input as an inclination angle signal to the control unit 110, and the control unit 110 stores the acceleration information, deceleration information, and inclination information in RAM. This allows the control unit 110 to recognize the acceleration, deceleration, and inclination of the vehicle 1 in real time and after the fact.

[0049] (Shift position sensor 140f) The shift position sensor 140f is a sensor for detecting the shift range (shift position) of the vehicle 1. In other words, the shift position sensor 140f detects the position to which the shift range is set, such as parking range (stopped position), drive range (driving position), reverse range (backing position), neutral range (neutral position), etc. The shift range detected by the shift position sensor 140f is input to the control unit 110, which then stores the current shift range in RAM. This allows the control unit 110 to recognize the position to which the shift range is set.

[0050] (Impact detection sensor 140g) The impact detection sensor 140g is, for example, a pressure sensor that detects pressure, and the pressure sensor is attached to the front bumper, rear bumper, or the like of the vehicle 1 so as to monitor collisions and impacts from the front, rear, and sides of the vehicle 1. Monitoring information is input to the control unit 110, and the control unit 110 can recognize the situations in front, rear, and on the rear sides of the vehicle 1 in real time. Note that although a pressure sensor is used in this embodiment, other sensors may also be used.

[0051] In this way, the control unit 110 can recognize the state of the vehicle 1 in real time and after the fact. As a result, the control unit 110 can control the agent 200 and the agent speaker 191 based on the recognition of the state of the vehicle 1 to notify the driver H of the state of the vehicle 1. For example, if the vehicle is traveling at an appropriate speed, the control unit 110 can notify the driver H by gestures and voice, such as "You are traveling at an appropriate speed." This leads to improved safety.

[0052] In this embodiment, the vehicle state recognition unit 140 includes the vehicle speed sensor 140a, the steering wheel angle sensor 140b, the accelerator pedal sensor 140c, the brake pedal sensor 140d, the G sensor 140e, the shift position sensor 140f, and the impact detection sensor 140g, but these are merely examples, and other devices may of course be used.

[0053] (Speaker 150) The speaker 150 is attached to, for example, the instrument panel 3, and outputs sounds other than the sound emitted from the agent 200, warning sounds, etc. Note that an audio speaker built into the vehicle 1 may be used instead of providing the speaker 150. Furthermore, as will be described later, the speaker 150 may also serve as the agent speaker 191.

[0054] (Mike 160) The microphone 160 is attached to, for example, the instrument panel 3, and receives input of voices emitted by the driver H and other passengers.

[0055] (Storage unit 170) The storage unit 170 can store information acquired from each of the recognition units described above, dialogues held between the driver H and the control unit 110, and the like. By storing this information in the storage unit 170, the control unit 110 can recognize the driving tendencies of the driver H (for example, what kind of driving style the driver is), the hobbies and preferences of the driver H (for example, what kind of background music the driver H likes), and the like. Furthermore, by recognizing these, it is also possible for the agent device 100 (agent 200) to actively hold a dialogue that is in line with the driving tendencies of the driver H and the hobbies and preferences of the driver H.

[0056] (Transceiver 180) The transceiver 180 can, for example, acquire information using an in-vehicle wireless LAN or acquire location information using a satellite positioning system. Based on the acquired information and the information stored in the storage unit 170, the control unit 110 can also actively conduct a dialogue from the agent device 100 side (agent 200) that is tailored to the driving tendencies and hobbies and preferences of the driver H. The transceiver 180 can also transmit and receive information to and from a call center that handles traffic troubles (accidents, etc.).

[0057] (Agent control unit 190) The agent control unit 190 is equipped with a CPU, ROM, RAM, input / output ports, etc. (not shown), and when information is input from the control unit 110, it can operate each part of the agent 200 and cause the agent speaker 191 to output a predetermined sound.

[0058] (Agent Speaker 191) As described above, the agent speaker 191 is provided inside the head 211 of the agent 200, and outputs sounds and the like emitted from the agent 200. Incidentally, by providing the agent speaker 191 inside the head 211 of the agent 200, particularly in the area corresponding to the mouth, as in this embodiment, it becomes more realistic and contributes to building a relationship of trust, but in cases where the agent 200 is small and it is difficult to provide the agent speaker 191 inside the head 211, the above-mentioned speaker 150 may be used instead, or the agent speaker 191 or a speaker that serves as a substitute may be provided in another part inside the vehicle 1.

[0059] Next, we will explain the operation and dialogue of the agent 200 in the agent device 100. The operation and dialogue of the agent 200 are performed by control processing by the control unit 110, and each unit of the agent 200 is operated by the agent control unit 190, and sound is output from the agent speaker 191. The control flow for performing the control processing is stored in the ROM of the control unit 110, and the CPU of the control unit 110 reads it from the ROM and performs various processes.

[0060] The main processing of the agent control processing performed by the CPU of the control unit 110 will be described below with reference to Fig. 6. Note that the main processing of the agent control processing starts when the ignition is turned on, for example, and ends when the ignition is turned off.

[0061] (Step S10) In step S10, the CPU of the control unit 110 performs processing to determine whether or not an impact equal to or greater than a predetermined value has been detected. Specifically, the CPU of the control unit 110 determines whether or not the value of the impact input from the impact detection sensor 140g is equal to or greater than a predetermined value. In other words, it determines whether or not a collision or the like has occurred with respect to the vehicle 1. If the CPU of the control unit 110 detects an impact equal to or greater than the predetermined value, it proceeds to step S600, and if it determines that an impact equal to or greater than the predetermined value has not been detected, it proceeds to step S20.

[0062] (Step S20) In step S20, the CPU of the control unit 110 performs normal agent control processing. In normal agent control processing, normal agent control processing is performed when no impact has been applied to the vehicle 1. For example, the CPU of the control unit 110 supports safety while driving by notifying the driver of avoidance actions against approaching objects and guidance when turning right or left by voice or the action of the agent 200. Details of the normal agent control processing will not be explained here. Then, after performing the normal agent control processing, the processing proceeds to step S10.

[0063] (Step S600) In step S600, the CPU of the control unit 110 performs impact detection processing. In the impact detection processing, processing is performed when the vehicle 1 receives an impact. In other words, processing is performed when there is a possibility that the vehicle 1 has collided with another vehicle or an obstacle. For example, the CPU of the control unit 110 performs safety checks for the driver H and dialogue with him. Details of the impact detection processing will be described later. Then, after the impact detection processing has been performed, the processing proceeds to step S30.

[0064] (Step S30) In step S30, the CPU of control unit 110 performs processing to determine whether the normal state has been restored. That is, the CPU of control unit 110 detects an impact of a predetermined value or more, and determines whether the normal state has been restored as a result of performing impact detection processing. For example, if the CPU of control unit 110 detects an impact of a predetermined value or more but does not interfere with normal driving, it determines that the normal state has been restored. If the CPU of control unit 110 determines that the normal state has been restored, it proceeds to step S10, and if it determines that the normal state has not been restored, it proceeds to step S600.

[0065] Next, the shock detection process performed by the CPU of the control unit 110 will be described with reference to Fig. 7. Fig. 7 shows a subroutine of step S600 (shock detection process) in Fig. 6.

[0066] (Step S601) In the impact detection process, first, in step S601, the CPU of the control unit 110 performs a process of making the agent 200 face the direction of the occupant. Specifically, while the vehicle 1 is traveling, the CPU of the control unit 110 makes the agent 200, which faces the traveling direction of the vehicle 1 as shown in Fig. 8(a), face the direction of the driver H as shown in Fig. 8(b). Then, once the agent 200 has been made to face the direction of the occupant, the process proceeds to step S602.

[0067] (Step S602) In step S602, the CPU of the control unit 110 inquires about the passenger's safety and performs an action. Specifically, the CPU of the control unit 110 speaks to the passenger through the agent speaker 191, such as "Are you OK?" or "Is there a problem?", to calm the passenger's state of panic, and causes the agent 200 to make large movements to calm the passenger. For example, as shown in FIG. 9, the CPU of the control unit 110 causes the agent 200 to turn its palms toward the passenger, stretch out both arms, and shake them up and down. Because palms attract people's attention as much as facial expressions, such large movements by the agent 200 can calm the passenger. After the inquiry about the passenger's safety and the action have been performed, the process proceeds to step S603.

[0068] (Step S603) In step S603, the CPU of the control unit 110 inquires about the presence or absence of any physical abnormalities and performs an action. That is, the CPU of the control unit 110 confirms, by voice and action, whether the occupant has sustained any injuries due to the collision. Specifically, the CPU of the control unit 110 asks the occupant via the agent speaker 191, for example, "Are you injured?", and simultaneously touches each part of the agent 200's body to check the condition of each part of the occupant's body for any injuries. For example, as shown in FIG. 10 , the CPU of the control unit 110 has the agent 200 touch the head, chest, abdomen, thighs, hips, knees, etc., in turn with both hands to check the presence or absence of any abnormalities in each part of the occupant's body. Here, since the occupant is likely to be in an excited state, the CPU firmly points to each part of the body and has the agent 200 check with its tentacles. After the inquiry about the presence or absence of any physical abnormalities and the action have been performed, the process proceeds to step S604.

[0069] (Step S604) In step S604, the CPU of the control unit 110 performs processing to determine whether the impact is small and causes no problems. That is, the CPU of the control unit 110 determines whether the degree of the impact is small and the safety of the occupants has been confirmed. If the CPU of the control unit 110 determines that the impact is small and causes no problems, it ends the impact detection processing, but if it determines that there is a problem with the occupants or driving, it proceeds to step S605.

[0070] (Step S605) In step S605, the CPU of control unit 110 performs processing to determine whether or not an audio signal has been received from the call center. For example, when the CPU of control unit 110 detects an impact equal to or greater than a predetermined value, it makes a call to the call center via transceiver 180 and determines whether or not an audio signal has been received from the call center. If the CPU of control unit 110 determines that an audio signal has been received from the call center, it proceeds to step S606, and if it determines that an audio signal has not been received from the call center, it proceeds to step S607.

[0071] (Step S606) In step S606, the CPU of the control unit 110 performs processing to have the agent utter a voice signal. That is, the CPU of the control unit 110 causes the occupant to utter the voice signal received from the call center via the agent speaker 191 as the words of the agent 200. This makes the occupant feel as if they are being handled by a familiar agent 200, giving the occupant a sense of security. Then, after having caused the agent to utter the voice signal, the process proceeds to step S607.

[0072] (Step S607) In step S607, the CPU of control unit 110 performs processing to determine whether an emergency has occurred. For example, if the CPU of control unit 110 detects a sudden rise in temperature or a change in current or resistance, it determines that a critical situation requiring escape has occurred and determines that an emergency has occurred. If the CPU of control unit 110 determines that an emergency has occurred, it proceeds to step S608, and if it determines that an emergency has not occurred, it proceeds to step S610.

[0073] (Step S608) In step S608, the CPU of the control unit 110 issues an escape alarm, notifies the emergency situation, and performs processing to take action. Specifically, the CPU of the control unit 110 issues an escape alarm sound via the speaker 150, and notifies the agent 200 of the occurrence of the emergency situation by voice and action. For example, the CPU of the control unit 110 notifies the occupant via the agent speaker 191, such as "An emergency situation has occurred. Please escape," and causes the agent 200 to perform the following action. Note that the call may be individually memorized by the user (driver H, occupant, etc.).

[0074] First, the agent 200 spreads both arms out to the left and right and raises and lowers them to indicate an emergency. Then, when the agent 200 detects that the vehicle 1 is in a critical situation, it rotates from a front-facing position to a rear-facing position, facing away from the occupants. Next, as shown in FIG. 11 , it raises and lowers one arm to express a sense of urgency, while pointing with the other arm in the direction of escape. Note that, by changing the agent 200's orientation from a front-facing position to a rear-facing position as described above, it is possible to induce a habitual reflex action in the occupants, encouraging them to escape quickly. Next, as shown in FIG. 12 , it waves both arms as if running toward the door (escape direction), assuming a sprinting pose. This allows the occupants to visualize sprinting to escape, making them more likely to take action. Furthermore, if the nearest door does not open, it assumes a similar escape pose toward another door. After issuing an escape alarm, announcing the emergency, and taking action, the process proceeds to step S608.

[0075] (Step S609) In step S609, the CPU of the control unit 110 performs processing to determine whether or not it has confirmed that the occupant has escaped. If it has confirmed that the occupant has escaped, the CPU of the control unit 110 ends the impact detection processing, and if it has not confirmed that the occupant has escaped, the CPU proceeds to step S610.

[0076] (Step S610) In step S610, the CPU of control unit 110 performs processing to determine whether or not the occupant has made a vocalization, groan, or the like. That is, the CPU of control unit 110 determines whether or not the occupant has made any vocalization. For example, an occupant may be injured and make a sound (such as a groan) that cannot be recognized as speech, and such unrecognizable sounds from the occupant may be input. If the CPU of control unit 110 determines that the occupant has made a vocalization, groan, or the like, it proceeds to step S611, and if it determines that the occupant has not made a vocalization, groan, or the like, it proceeds to step S614.

[0077] (Step S611) In step S611, the CPU of the control unit 110 performs a process of determining whether the utterance is a responseable utterance. That is, the CPU of the control unit 110 determines whether the voice uttered by the occupant can be heard as words, whether the content is conducive to conversation, or whether the voice is a moan-like utterance (such as an unrecognizable voice). Note that if the voice is not a simple moan-like utterance but is something that could be responded to but is difficult to hear, the CPU of the control unit 110 may ask the occupant to repeat the utterance. If the CPU of the control unit 110 determines that the utterance is a responseable utterance, it proceeds to step S612, and if it determines that the utterance is not a responseable utterance, it proceeds to step S613.

[0078] (Step S612) In step S612, the CPU of the control unit 110 takes action in response to the occupant's speech. Furthermore, once it is confirmed that Bluetooth (registered trademark) is connected to the mobile phone, the CPU makes a gesture as if making a phone call, as shown in FIG. 13. It may also ask aloud who to call to receive instructions. After taking action in response to the occupant's speech, the CPU of the control unit 110 proceeds to step S614.

[0079] (Step S613) In step S613, the CPU of control unit 110 outputs encouraging voice and performs an action. The encouraging gesture may be individually memorized by the user. In this case, the CPU may also perform a gesture such as making a phone call. After outputting encouraging voice and performing an action, the CPU of control unit 110 proceeds to step S614.

[0080] (Step S614) In step S614, the CPU of the control unit 110 performs processing to determine whether or not rescue has arrived. If the CPU of the control unit 110 determines that rescue has arrived, it ends the impact detection processing, and if it determines that rescue has not arrived, it proceeds to step S615.

[0081] (Step S615) In step S615, the CPU of the control unit 110 performs processing to get close to the occupant and have them assume a standby position. For example, as shown in FIG. 14, the CPU of the control unit 110 causes the agent 200 to get close to the occupant in a bent and stretched position, with its head tilted to the right and its arms on its knees, and to listen to what the occupant is saying until rescue arrives. Furthermore, as shown in FIG. 15, the CPU of the control unit 110 continues to encourage the occupant by moving its arms and nodding while speaking encouraging words. This allows the occupant to feel as if a familiar person is always accompanying them, thereby alleviating their anxiety. After performing processing to get close to the occupant and have them assume a standby position, the CPU of the control unit 110 proceeds to step S613.

[0082] As described above, when the agent device 100 of this embodiment detects an impact of a predetermined value or more, the agent 200 faces the occupant and asks questions and takes actions to confirm the occupant's safety, thereby giving the occupant a sense of security and encouraging them to respond calmly in the event of a traffic accident.

[0083] In the agent control process of this embodiment, a program stored in the ROM, RAM, EEPROM, etc. of the control unit 110 is loaded into the RAM, etc. of the control unit 110 and executed by the CPU, etc. of the control unit 110. Furthermore, in this embodiment, the impact detection sensor 140g constitutes the impact detection unit of the present invention, and the transceiver 180 constitutes the call center communication unit of the present invention.

[0084] Furthermore, in this embodiment, the agent control unit of the present application is configured by both the control unit 110 and the agent control unit 190, but as mentioned above, the agent control unit 190 alone may have the functions of both the control unit 110 and the agent control unit 190 of this embodiment. [Explanation of symbols]

[0085] 1: vehicle, 100: agent device, 110: control unit, 120: surrounding environment recognition unit, 130: occupant state recognition unit, 140: vehicle state recognition unit, 140f: shift position sensor, 140g: impact detection sensor, 150: speaker, 160: microphone, 170: memory unit, 180: transceiver, 190: agent control unit, 191: agent speaker, 200: agent

Claims

1. an impact detection unit that detects an impact to the vehicle that is equal to or greater than a predetermined value; an agent control unit that controls a personified agent; Equipped with The agent control unit If the impact is detected, Facing the crew, he asks questions and makes actions to confirm the crew's safety. At the same time as asking the occupant to check for any abnormalities in their body, the agent touches each part of the occupant's body to check the condition of each part of the occupant's body.

1. An agent device comprising:

2. The agent control unit While moving both arms up and down, he asks if the passengers are safe.

2. The agent device according to claim 1.

3. a call center communication unit that communicates with the call center; Equipped with The agent control unit When receiving a voice signal from the call center, the voice signal is spoken by the agent as the agent's words; 3. The agent device according to claim 1 or 2.

4. The agent control unit When the impact is detected and the safety confirmation and the physical abnormality confirmation are made, if an unrecognizable voice is input by the occupant, an encouraging voice is given.

4. The agent device according to claim 1, wherein the agent device is a device for receiving an information from a user.

Images