Agent device

Abstract

To provide an agent device that can increase the safety by preventing a driver from keep looking at a specific target and paying no attention to regions around the driver.SOLUTION: An agent device 100 includes: a shift position sensor 140f for detecting a shift position; an ambient surrounding recognition unit 120 for acquiring a surrounding situation; and an agent controller 190 for controlling a personalized agent. The agent controller 190 expresses the curvature of a corner by using both arms and stretch when the presence of a corner is determined in a travelling route by the acquisition of the surrounding situation.SELECTED DRAWING: Figure 9

Description

Technical Field

[0001] The present invention relates to an agent device capable of notification by anthropomorphic characters.

Background Art

[0002] Conventionally, in a driving environment in a vehicle such as an automobile, various methods have been proposed as a method for prompting a passenger (especially a driver) to pay attention. As one of such methods, an agent device or a driving support device has been proposed in which a passenger and an anthropomorphic character (so-called agent) provide information related to driving support according to the passenger's request while having a dialogue.

[0003] For example, in a conventional driving support device, it is considered that it is difficult to intuitively convey a notification of a danger avoidance action by voice or the like, and a small robot is arranged on a dashboard and made to appear as an anthropomorphic agent, and this agent is made to simulate the danger avoidance action that a driver should perform. (See Patent Document 1).

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, conventional agents were inorganic or suddenly appeared to prompt attention, making it difficult to build a trust relationship between the agent and the driver. Also, even if the agent suddenly performs the same action as the avoidance action, if the driver does not understand the reason why the avoidance action must be taken, it is difficult to immediately take that action, and in particular, without a trust relationship, it does not lead to an avoidance action. In particular, when a predetermined response measure is required for the driver, such as a danger avoidance action during driving, if the agent suddenly mimics the avoidance action, it may be impossible to understand why the agent took that action, and there is a risk that the predetermined avoidance action will not be in time while thinking about it. For example, when a corner appears during driving, if the driver's attention becomes concentrated on the agent, or if the line of sight goes to oncoming vehicles, the scenery outside the corner, guardrails, etc., it may induce understeer and there is a risk of running off into the oncoming lane.

[0006] The present invention has been made to solve such conventional problems, and an object thereof is to provide an agent device that can prevent the driver's line of sight from concentrating on a specific object and neglecting attention to the surroundings, thereby improving safety.

Means for Solving the Problems

[0007] The agent device according to the present invention includes a shift position detection unit that detects a shift position, a surrounding situation acquisition unit that acquires a surrounding situation, and an agent control unit that controls an anthropomorphic agent. When the agent control unit determines that there is a corner on the traveling route based on the acquisition of the surrounding situation, it is characterized in that it expresses the curvature of the corner using both arms and flexion and extension.

[0008] According to the present invention, it is possible to provide an agent device that can prevent the driver's line of sight from concentrating on a specific object and neglecting attention to the surroundings, thereby improving safety.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Embodiments for Carrying Out the Invention

[0010] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Also, in the following description, the same reference numerals in different figures indicate parts having the same function, and duplicate descriptions in each figure are omitted as appropriate.

[0011] (Each Configuration in the Vehicle Interior) As shown in FIGS. 1 and 2, the vehicle 1 has an instrument panel 3 provided on the front side of the vehicle of the driver's seat 2 and a steering wheel 4 disposed between the driver's seat 2 and the instrument panel 3. The steering wheel 4 is rotatably attached to a steering column (not shown) via a steering shaft (not shown). Further, inside the steering wheel 4, an airbag 8 that deploys toward the driver (hereinafter referred to as the driver) H when the vehicle 1 collides or the like is stored. In the agent device 100 (described later) and the agent 200 (described later) according to the present embodiment, although the description is based on the interaction with the driver H and gestures (operations), the same can be performed for other passengers as well.

[0012] As shown in FIG. 1, two display panels, a first display panel 30 and a second display panel 40, are provided in the passenger compartment of the vehicle 1. The first display panel 30 is disposed on the instrument panel 3 on the front side of the driver's seat 2, and the second display panel 40 is disposed on the instrument panel 3 on the left front side of the driver's seat 2.

[0013] (First display panel 30) As shown in FIGS. 1 and 2, the first display panel 30 includes a pointer-type meter configured like an analog clock, and a so-called liquid crystal display device in which a liquid crystal panel, a backlight, and the like are integrally provided. The driver H can view various information displayed in the first display area 30a of the first display panel 30 through the upper space portion 4a of the steering wheel 4.

[0014] As shown in FIG. 2, the first display panel 30 is provided with a first display area 30a. On the left and right of the first display area 30a, two pointer-type meters for displaying information such as the traveling speed (speedometer) of the vehicle 1 and the number of revolutions per unit time of the engine (tachometer), and between the two pointer-type meters and at the center of the first display panel 30, a small liquid crystal display device for displaying an image indicating general vehicle information is arranged. Note that the first display panel 30 may be configured by a single liquid crystal display device without including the pointer-type meter.

[0015] (Second display panel 40) As shown in FIG. 1, the second display panel 40 is configured by a so-called liquid crystal display device in which a liquid crystal panel and a backlight are integrally provided. Note that map information and the like are displayed on the second display panel 40, which serves as a so-called car navigation system.

[0016] Note that the liquid crystal display device constituted by the first display panel 30 and the second display panel 40 may be constituted by a self-luminous display device such as a plasma display or an organic EL, or a display device such as a projection type projector.

[0017] (Agent 200) Also, as shown in FIG. 3, an agent 200 is disposed above the instrument panel 3 on the front side of the driver's seat 2. Note that the agent 200 may be located anywhere as long as it enters the peripheral vision of the driver H, and does not necessarily have to be disposed directly in front of the driver's seat. Note that it is desirable that the agent 200 is in the peripheral vision of the driver H and is located at a position that does not block the driver H's view outside the vehicle. Further, the arrangement position of the agent 200 may be changed by a predetermined operation device or voice instruction or the like.

[0018] The agent 200 is an anthropomorphic character and is a notification and interactive three-dimensional object having a humanoid outer shape. Note that in the present embodiment, the agent 200 is a three-dimensional object that forms a humanoid real image, but is not limited thereto, and may be a character displayed on a display device such as a display, a three-dimensional display represented by a hologram or the like, or a virtual three-dimensional object.

[0019] As shown in FIG. 4, the agent 200 has a head 211, a right eye part 212R, a left eye part 212L, a neck part 213, a chest part 214, a waist part 215, a hip part 216, a right shoulder part 221R, a left shoulder part 221L, a right upper arm part 222R, a left upper arm part 222L, a right elbow part 223R, a left elbow part 223L, a right forearm part 224R, a left forearm part 224L, a right wrist part 225R, a left wrist part 225L, a right hand part 226R, a left hand part 226L, a right hip joint part 231R, a left hip joint part 231L, a right thigh part 232R, a left thigh part 232L, a right knee part 233R, a left knee part 233L, a right lower leg part 234R, a left lower leg part 234L, a right ankle part 235R, a left ankle part 235L, a right foot part 236R, and a left foot part 236L.

[0020] In addition, the agent 200 is provided with a support connector 240, and the agent 200 is supported so as to be able to stand upright on the instrument panel 3. Note that ears are provided on both sides of the head 211 of the agent 200. Thereby, it can be made to seem that the agent 200 is listening well to what the driver H is saying. Also, regarding these ears, even if they are not actually provided, as will be described later, when the agent 200 listens to what the driver H is saying, by placing the right hand part 226R or the left hand part 226L on the side part of the head 211 of the agent 200, it can be made to seem as if it is listening with a hand on the ear, and the trust relationship with the driver H can be deepened.

[0021] Note that Fig. 4(a) is a diagram showing the basic posture when the agent 200 is facing the driver H side (the rear side of the vehicle 1), and Fig. 4(b) is a diagram showing the basic posture when the agent 200 has its back to the driver H (is facing the front side of the vehicle 1). Also, in the basic posture, the agent 200 stands up with the right palm part 226Ra of the right hand part 226R and the left palm part 226La of the left hand part 226L facing the inside of the agent 200, and the back of the right hand part 226Rb of the right hand part 226R and the back of the left hand part 226Lb of the left hand part 226L facing the outside of the agent 200. As described above, the agent 200 can freely rotate between the front posture facing the driver H side and the back posture with its back to the driver H. That is, the agent 200 may be such that the agent 200 itself rotates around the support connector 240, or the agent 200 may be placed on a rotating body and the rotating body may be rotated to rotate from the front posture to the back posture and from the back posture to the front posture.

[0022] In addition, an agent control unit 190, which will be described later, is provided within the agent 200. The agent control unit 190 controls and operates each joint of the agent 200 so that the agent 200 can assume a predetermined posture. The agent control unit 190 is wirelessly connected to a control unit 110 (ECU), which will be described later, and operates the agent 200 under the control of the control unit 110. In this embodiment, the agent control unit 190 and the control unit 110 are wirelessly connected. However, for example, they may be wired-connected through a communication line within a support connector 240.

[0023] Also, an agent speaker 191, which will be described later, is provided within the head 211 of the agent 200, and voice output is performed under the control of the agent control unit 190. Note that the agent speaker 191 is not limited to being within the head 211 of the agent 200 and may be provided at other parts of the agent 200 or at other locations in the vehicle 1. However, if the agent speaker 191 is provided within the head 211 of the agent 200, it is possible to make the driver H more realistically perceive the speech from the agent 200, which is also more likely to evoke empathy and can greatly contribute to building trust between the driver H and the agent 200.

[0024] Next, the configuration of the agent device 100 according to the present invention will be described with reference to the block diagram of FIG. 5.

[0025] As shown in FIG. 5, the agent device 100 includes a control unit 110, a surrounding environment recognition unit 120, an occupant state recognition unit 130, a vehicle state recognition unit 140, a speaker 150, a microphone 160, a storage unit 170, a transceiver 180, an agent control unit 190, and an agent speaker 191.

[0026] Note that the agent device 100 shown in FIG. 5 is only an example, and the components of the agent device 100 can be changed as appropriate. For example, the agent device 100 of the present invention can be realized by using only one of the speaker 150 and the agent speaker 191. In addition to the microphone 160, an agent microphone may be provided in the agent 200, or the agent device 100 of the present invention can be realized by using only the agent microphone provided in the agent 200 instead of the microphone 160. Furthermore, in the present embodiment, both the control unit 110 and the agent control unit 190 are provided, but the present invention is not limited to this, and only one of the control unit 110 or the agent control unit 190 may be provided to perform the functions of both.

[0027] (Control Unit 110) The control unit 110 includes a CPU, a ROM, a RAM (e.g., a ring buffer), an EEPROM, input / output ports, etc. (not shown). For example, when information is input from an input port, the control unit 110 controls various devices via an output port based on a control program read from the ROM.

[0028] Note that the ROM of the control unit 110 stores the operations performed by the agent 200 and a dialogue data table (not shown). In addition, a learning function such as AI may be provided to perform operations other than the stored operations and dialogues. The CPU of the control unit 110 determines the operations and dialogues to be performed by the agent 200 from the data table based on the information acquired from each recognition unit described later and the information acquired from the microphone 160. For example, when the speech from the driver H is acquired from the microphone 160, the gesture of "nodding" is determined from the data table, and when the speech of the driver H ends, a dialogue such as "Understood" is determined. Note that the operations performed by the agent 200 and the dialogue data table may be stored in the ROM of the agent control unit 190.

[0029] (Peripheral Environment Recognition Unit 120) The surrounding environment recognition unit 120 is provided to recognize the surrounding environment of the vehicle 1 (own vehicle). The surrounding environment recognition unit 120 includes an in-vehicle camera 120a, a radar 120b, an air temperature sensor 120c, a weather sensor 120d, and a navigation device 121, and the surrounding environment and surrounding situation of the vehicle 1 can be recognized by these devices. For example, the surrounding environment recognition unit 120 can detect the presence of corners or turning angles in the traveling direction, the presence of approaching objects, and the like. In this embodiment, the navigation device 121 is configured as a part of the surrounding environment recognition unit 120. However, the present invention is not limited to this, and the navigation device 121 may be separate from the surrounding environment recognition unit 120 and have it independently.

[0030] (In-vehicle camera 120a) The in-vehicle camera 120a is attached to, for example, an interior rearview mirror (not shown) and can photograph the front and rear of the vehicle 1. The photographed image information is input to the control unit 110, and the control unit 110 stores the image information in the RAM. Thereby, the control unit 110 can recognize the situations in front of and behind the vehicle 1 in real time and retrospectively.

[0031] (Radar 120b) For the radar 120b, for example, a millimeter-wave radar that emits radio waves to detect obstacles or the like is used. The millimeter-wave radar is attached to the front bumper or rear bumper of the vehicle 1, and can monitor the front of the vehicle 1, the front side of the vehicle 1, and the rear side of the vehicle 1. The monitoring information is input to the control unit 110, and the control unit 110 stores the monitoring information in the RAM. Thereby, the control unit 110 can recognize the situations in front of, on the front side of, and on the rear side of the vehicle 1 in real time and retrospectively. In this embodiment, a millimeter-wave radar is used, but other radars may be used. For example, an infrared radar may be used.

[0032] (Air temperature sensor 120c) The temperature sensor 120c is attached, for example, to the rear of the vehicle 1 and is capable of detecting the outside temperature of the vehicle 1. Then, the detected temperature is input to the control unit 110, and the control unit 110 stores the temperature in the RAM. Thereby, the control unit 110 can recognize the outside temperature of the vehicle 1.

[0033] (Weather sensor 120d) The weather sensor 120d is attached, for example, to the upper part of the vehicle 1 and is capable of detecting raindrops or the like falling on the vehicle 1. Then, the detected water droplet information is input to the control unit 110, and the control unit 110 stores the water droplet information in the RAM. Thereby, the control unit 110 can recognize the weather conditions outside the vehicle 1.

[0034] (Navigation device 121) The navigation device 121 provides route guidance according to the destination, waypoints, etc. Also, the navigation device 121 has map information, GPS, a VICS receiver, etc., and is capable of acquiring the current position of the vehicle 1, surrounding map information, traffic jam information, road regulation information, parking lot information, etc. Therefore, in addition to the navigation device 121, the control unit 110 can recognize the surrounding situation of the vehicle 1 in real time based on the information acquired by the external camera 120a for vehicle exterior photography, the radar 120b, etc. Note that the navigation device 121 can also receive map information at any time and acquire the latest map information. Also, the transceiver unit of the navigation device 121 may be shared with the transceiver 180 described later.

[0035] As described above, the control unit 110 can recognize the surrounding environment and situation of the vehicle 1 in real time and retrospectively. Thereby, based on the recognition of the surrounding environment and situation of the vehicle 1, the control unit 110 can control the agent 200 and the agent speaker 191 to notify the driver H of the information on the surrounding environment and situation of the vehicle 1. For example, if there is a falling object in front of the vehicle 1, it is possible to notify the driver by gestures and voice such as "There is a falling object in front". Thereby, safety can be improved.

[0036] In addition, in the present embodiment, as the surrounding environment recognition unit 120, an in-vehicle camera 120a for external shooting, a radar 120b, an air temperature sensor 120c, a weather sensor 120d, and a navigation device 121 are exemplified, but this is only an example, and of course, other devices may be used.

[0037] (Occupant state recognition unit 130) The occupant state recognition unit 130 is provided to recognize the state of the driver H. Further, the occupant state recognition unit 130 includes an in-vehicle camera 130a for shooting the occupant and a vital sensor 130b, and can recognize the state of the driver H by these devices.

[0038] (In-vehicle camera 130a for shooting the occupant) The in-vehicle camera 130a for shooting the occupant is attached to the instrument panel 3, for example, and can shoot the driver H. Then, the captured image information is input to the control unit 110, and the control unit 110 stores the image information in the RAM. Thereby, the control unit 110 can recognize the state of the driver H in real time and retrospectively. Here, the state of the driver H specifically includes, for example, the state of the driver H's eyelids, the number of blinks, the direction of the line of sight, the direction of the face, and whether the driver is in a hunched state or an immobile state.

[0039] (Vital sensor 130b) The vital sensor 130b is attached to a part gripped by the driver H of the steering wheel 4, for example, and can acquire vital information such as the heart rate and blood pressure of the driver H. Then, the acquired vital information is input to the control unit 110, and the control unit 110 stores the vital information in the RAM. Thereby, the control unit 110 can recognize the state of the driver H in real time and retrospectively.

[0040] As described above, the control unit 110 can recognize the state of the driver H in real time and retrospectively. Thereby, based on the recognition of the state of the driver H, the control unit 110 can control the agent 200 and the agent speaker 191 to notify the driver H of predetermined information. For example, it is possible to notify by image and voice such as "Your eyelids are drooping. Won't you take a break?" or "Your heart rate is faster than usual. Won't you take a break?" This leads to an improvement in safety.

[0041] The occupant state recognition unit 130 can recognize the thinking and emotions of the driver H within a certain range based on the information acquired from the in-vehicle camera 130a for photographing the occupant, the vital sensor 130b, and the information input from the microphone 160. For example, the facial expression of the driver H is acquired from the in-vehicle camera 130a for photographing the occupant, the heart rate and blood pressure of the driver H are acquired from the vital sensor 130b, the volume and input content are acquired from the microphone 160, and it is possible to recognize whether the driver H is in normal thinking and emotions or in thinking and emotions different from normal (for example, surprised, angry, etc.) from the acquired information.

[0042] In this embodiment, the in-vehicle camera 130a for photographing the occupant and the vital sensor 130b are cited as the occupant state recognition unit 130, but this is only an example, and of course, other devices may be used.

[0043] (Vehicle state recognition unit 140) The vehicle state recognition unit 140 is provided to recognize the state of the vehicle 1. The vehicle state recognition unit 140 includes a vehicle speed sensor 140a, a steering angle sensor 140b, an accelerator pedal sensor 140c, a brake pedal sensor 140d, a G sensor 140e, and a shift position sensor 140f, and the state of the vehicle 1 can be recognized by these devices.

[0044] (Vehicle speed sensor 140a) The vehicle speed sensor 140a is a sensor for detecting the vehicle speed of the vehicle 1, and the detected vehicle speed is input to the control unit 110 as a vehicle speed signal, and the control unit 110 stores the vehicle speed information in the RAM. Thereby, the control unit 110 can recognize the vehicle speed of the vehicle 1 in real time and retrospectively.

[0045] (Steering angle sensor 140b) The steering angle sensor 140b is a sensor for detecting the steering angle (the angle of the steering wheel 4) of the vehicle 1, and the detected steering angle is input to the control unit 110 as an angle signal, and the control unit 110 stores the angle information in the RAM. Thereby, the control unit 110 can recognize the steering angle (the angle of the steering wheel 4) of the vehicle 1 in real time and retrospectively.

[0046] (Accelerator pedal sensor 140c) The accelerator pedal sensor 140c is a sensor for detecting the depression amount of an accelerator pedal (not shown), and the detected depression amount is input to the control unit 110 as a depression amount signal, and the control unit 110 stores the depression amount information in the RAM. Thereby, the control unit 110 can recognize the depression amount of the accelerator pedal of the vehicle 1 in real time and retrospectively.

[0047] (Brake pedal sensor 140d) The brake pedal sensor 140d is a sensor for detecting the depression amount of a brake pedal (not shown). The detected depression amount is input as a depression amount signal to the control unit 110, and the control unit 110 stores the depression amount information in the RAM. Thereby, the control unit 110 can recognize the depression amount of the brake pedal of the vehicle 1 in real time and retrospectively.

[0048] (G sensor 140e) The G sensor 140e is a sensor for detecting the acceleration, deceleration, and inclination of the vehicle 1. When acceleration is detected, the acceleration amount, when deceleration is detected, the deceleration amount, and when inclination is detected, the inclination angle amount are input as an acceleration amount signal, a deceleration amount signal, and an inclination angle signal, respectively, to the control unit 110, and the control unit 110 stores the acceleration information, deceleration information, and inclination information in the RAM. Thereby, the control unit 110 can recognize the acceleration, deceleration, and inclination of the vehicle 1 in real time and retrospectively.

[0049] (Shift position sensor 140f) The shift position sensor 140f is a sensor for detecting the shift range (shift position) of the vehicle 1. That is, the shift position sensor 140f detects in which position the shift range is set, such as the parking range (parking position), drive range (driving position), reverse range (reverse position), neutral range (neutral position), etc. The shift range detected by the shift position sensor 140f is input to the control unit 110, and the control unit 110 stores the current shift range in the RAM. Thereby, the control unit 110 can recognize in which position the shift range is set.

[0050] In the above manner, the control unit 110 can recognize the state of the vehicle 1 in real time and retrospectively. Thereby, based on the recognition of the state of the vehicle 1, the control unit 110 can control the agent 200 and the agent speaker 191 to notify the driver H of the information on the state of the vehicle 1. For example, if the vehicle is traveling at an appropriate speed, notifications can be made by gestures and voices such as "You are driving at an appropriate speed." This leads to an improvement in safety.

[0051] In the present embodiment, as the vehicle state recognition unit 140, a vehicle speed sensor 140a, a steering angle sensor 140b, an accelerator pedal sensor 140c, a brake pedal sensor 140d, a G sensor 140e, and a shift position sensor 140f are listed, but this is only an example, and of course, other devices may be used.

[0052] (Speaker 150) The speaker 150 is attached to, for example, the instrument panel 3, and outputs voices other than the voice emitted from the agent 200, warning sounds, etc. Note that instead of providing the speaker 150, an audio speaker built into the vehicle 1 may be used. Also, as will be described later, the speaker 150 may also serve as the agent speaker 191.

[0053] (Microphone 160) The microphone 160 is attached to, for example, the instrument panel 3, and inputs voices emitted from the driver H and other passengers.

[0054] (Memory unit 170) The memory unit 170 can store the information acquired from each of the above-described recognition units, as well as the dialogues and the like conducted between the driver H and the control unit 110. By accumulating this information in the memory unit 170, the control unit 110 can recognize the driving tendency of the driver H (for example, what kind of driver the driver is) and the hobbies and preferences of the driver H (for example, what kind of BGM the driver likes). Also, by recognizing these, the agent device 100 side (agent 200) can actively conduct a dialogue in accordance with the driving tendency of the driver H and the hobbies and preferences of the driver H.

[0055] (Transceiver 180) The transceiver 180 can, for example, acquire information using an in-vehicle wireless LAN or acquire position information using a satellite positioning system. Based on the information thus acquired and the information stored in the above-described memory unit 170, the control unit 110 can also actively conduct a dialogue in accordance with the driving tendency of the driver H and the hobbies and preferences of the driver H from the agent device 100 side (agent 200).

[0056] (Agent control unit 190) The agent control unit 190 includes a CPU, a ROM, a RAM, input / output ports, etc. (not shown). For example, when information is input from the control unit 110, it can operate each part of the agent 200 or cause a predetermined voice output to be performed on the agent speaker 191.

[0057] (Agent speaker 191) As described above, the agent speaker 191 is provided inside the head 211 of the agent 200, and voices and the like emitted from the agent 200 are output. Note that, as in the present embodiment, by providing the agent speaker 191 inside the head 211 of the agent 200, particularly in the part corresponding to the mouth, it becomes more realistic and contributes to the construction of a trust relationship. However, if the agent 200 is small and it is difficult to provide it inside the head 211, the above-described speaker 150 may be substituted, or an agent speaker 191 or a speaker serving as a substitute therefor may be provided in other parts of the vehicle 1.

[0058] Next, the operations and interactions of the agent 200 in the agent device 100 will be described. The operations and interactions of the agent 200 are performed by control processing by the control unit 110. Each part of the agent 200 is operated by the agent control unit 190, and voices are output from the agent speaker 191. Note that the control flow for performing the control processing is stored in the ROM of the control unit 110, and the CPU of the control unit 110 reads it from the ROM and performs various processes.

[0059] Hereinafter, with reference to FIG. 6, the main processing of the agent control processing performed by the CPU of the control unit 110 will be described.

[0060] (Step S10) In step S10, when the ignition is turned on, the CPU of the control unit 110 starts the main processing of the agent control processing. Specifically, the CPU of the control unit 110 reads the main processing of the agent control processing from the ROM and starts the processing, and transfers the processing to step S100.

[0061] (Step S100) In step S100, the CPU of the control unit 110 performs boarding processing. In the boarding processing, the initial processing when driver H boards vehicle 1 is performed. For example, in the boarding processing, the CPU of the control unit 110 performs processing to face agent 200 towards driver H. Also, the CPU of the control unit 110 performs processing such as having a conversation with driver H and causing agent 200 to speak and act out matters to be explained during boarding, such as weather, remaining fuel information, and notable items for vehicle maintenance. Then, when the boarding processing is completed, the processing proceeds to step S200.

[0062] (Step S200) In step S200, the CPU of the control unit 110 performs pre - driving start processing. In the pre - driving start processing, the processing before driver H starts driving is performed. For example, in the pre - driving start processing, explanations are given about precautions for the driving environment to the destination and rest points. In the explanation of the pre - driving start processing, agent 200 faces driver H, moves both hands for each item to explain, and also gives a signal of completion such as nodding the head up and down for each item completion. Then, when the pre - driving start processing is completed, the processing proceeds to step S20.

[0063] (Step S20) In step S20, the CPU of the control unit 110 performs processing to determine whether the shift range is "P" (parking range). Specifically, the CPU of the control unit 110 determines whether the shift range input from the shift position sensor 140f is the parking range. If the CPU of the control unit 110 determines that the shift range is not the parking range (in this case, the shift position has been changed), the processing proceeds to step S300. If it determines that the shift range is the parking range (in this case, the shift position has not been changed), the processing proceeds to step S400.

[0064] (Step S300) In step S300, the CPU of the control unit 110 performs running processing. In the running processing, control processing of the agent 200 during running is performed. For example, when the shift range input from the shift position sensor 140f is "D" (drive range), the CPU of the control unit 110 performs control processing of the agent 200 during forward running, such as making the agent 200 face forward (the traveling direction of the vehicle 1) (assuming a predetermined posture). When the shift range is "R" (reverse range), control processing of the agent 200 during reverse running is performed. Details of the running processing will be described later. After finishing the running processing, the process proceeds to step S30.

[0065] (Step S30) In step S30, the CPU of the control unit 110 performs a process of determining whether the shift range is "P" (parking range). As described above, the CPU of the control unit 110 determines whether the shift range input from the shift position sensor 140f is the parking range. If the CPU of the control unit 110 determines that the shift range is not the parking range, the process proceeds to step S300. If it determines that the shift range is the parking range, the process proceeds to step S400.

[0066] (Step S400) In step S400, the CPU of the control unit 110 performs end-of-driving processing. In the end-of-driving processing, processing at the end when a predetermined driving is completed is performed. For example, in the end-of-driving processing, the CPU of the control unit 110 makes the agent 200 face the driver H again and assume the basic posture. Also, the CPU of the control unit 110 performs voice output and operations for rewarding at the end of driving, and if there are abnormalities or notification information in the vehicle, it performs notification of abnormalities, sounding of warning sounds, and notification of information by voice or operations. After finishing the end-of-driving processing, the process proceeds to step S40.

[0067] (Step S40) In step S40, the CPU of the control unit 110 performs a process of determining whether the shift range is "P" (parking range). As described above, the CPU of the control unit 110 determines whether the shift range input from the shift position sensor 140f is the parking range. If the CPU of the control unit 110 determines that the shift range is not the parking range (in this case, the shift position has been changed), the process proceeds to step S300. If it determines that the shift range is the parking range (in this case, the shift position has not been changed), the process proceeds to step S50.

[0068] (Step S50) In step S50, the CPU of the control unit 110 performs a driving restart determination process. In the driving restart determination process, a process of determining whether the driver H intends to restart driving is performed. For example, when the CPU of the control unit 110 receives a voice input such as "Restart driving" from the driver H, or when a new destination is input to the navigation system by voice input or manual input, etc., it is determined that there is an intention to restart driving. If the CPU of the control unit 110 determines that there is an intention to restart driving, the process proceeds to step S200. If it determines that there is no intention to restart driving, the process proceeds to step S60.

[0069] (Step S60) In step S60, the CPU of the control unit 110 determines whether the ignition has been turned off. If the CPU of the control unit 110 determines that the ignition has been turned off, the main process of the agent control process ends. If the ignition has not been turned off, the process proceeds to step S40.

[0070] Next, with reference to FIG. 7, the running process performed by the CPU of the control unit 110 will be described. FIG. 7 is a subroutine of step S300 (running process) in FIG. 6.

[0071] (Step S301) During driving, first, in step S301, the CPU of the control unit 110 performs startup processing. In the startup processing, a warning is issued at startup. That is, at startup, it is easy for the thoughts and actions (including past operations) of driver H to not match. Specifically, for example, the elderly or others may mistake the shift position (such as "D" (Drive Range) and "R" (Reverse Range) in the shift range), and collisions or the like may occur. Here, in order to eliminate the assumption of safety, startup processing is performed. For example, the CPU of the control unit 110 performs action confirmation by reading out the shift range during startup processing. In addition, the CPU of the control unit 110 prompts visual confirmation of the traveling direction. Thereby, even if the vehicle 1 is equipped with a mis-start function, the misoperation by driver H itself can be suppressed, and safety can be improved.

[0072] (Step S302) In step S302, the CPU of the control unit 110 performs a process of determining whether the shift range is "D" (Drive Range). Specifically, the CPU of the control unit 110 determines whether the shift range input from the shift position sensor 140f is the drive range. If the CPU of the control unit 110 determines that the shift range is not the drive range, the process proceeds to step S303. If the CPU of the control unit 110 determines that the shift range is the drive range, the process proceeds to step S310.

[0073] (Step S303) In step S303, the CPU of the control unit 110 performs non-drive range processing. In the non-drive range processing, processing during driving other than when the shift range is "D" (Drive Range) is performed. For example, if the shift range is "R" (Reverse Range), the CPU of the control unit 110 performs reverse range processing and the like. In the reverse range processing, processing assuming parking and the like is performed. Details of the non-drive range processing are omitted. Then, when the non-drive range processing is completed, the driving processing is terminated.

[0074] (Step S310) In step S310, the CPU of the control unit 110 performs in - driving processing. In the in - driving processing, control processing of the agent 200 during driving is performed according to the surrounding situation. For example, the CPU of the control unit 110 performs processing such as transmitting a deceleration instruction before entering a corner or notifying the radius of a curve through the operation of the agent 200. Details of the in - driving processing will be described later. Then, when the in - driving processing ends, the driving - time processing ends.

[0075] Next, with reference to FIG. 8, the in - driving processing performed by the CPU of the control unit 110 will be described. FIG. 8 is a sub - routine of step S310 (in - driving processing) in FIG. 7.

[0076] (Step S311) In the in - driving processing, first, in step S311, the CPU of the control unit 110 performs processing to determine whether there is an approaching object. Specifically, the CPU of the control unit 110 determines whether there is something (such as a vehicle or a person) approaching the vehicle 1 within a predetermined range and at a speed equal to or higher than a predetermined speed from the information on the surrounding situation input from the in - vehicle camera 120a, the radar 120b, etc. That is, the CPU of the control unit 110 determines whether it is necessary to avoid an obstacle based on the acquired surrounding situation. If the CPU of the control unit 110 determines that there is an approaching object, the processing proceeds to step S312, and if it determines that there is no approaching object, the processing proceeds to step S314.

[0077] (Step S312) In step S312, the CPU of the control unit 110 performs an obstacle avoidance process. The obstacle avoidance process includes an emergency avoidance process and a normal avoidance process. That is, when there is an approaching object to the vehicle 1, there is a risk of collision, and there are two cases: a case where immediate avoidance action must be taken, and a case where it is only necessary to notify the driver H and let the driver take appropriate measures. The CPU of the control unit 110 performs an emergency avoidance process or a normal avoidance process according to the acquired surrounding situation. In the emergency avoidance process, depending on the situation, the emergency brake may be activated or the airbag 8 may be deployed without depending on the operation of the driver H. Then, after the obstacle avoidance process, the CPU of the control unit 110 transfers the process to step S313.

[0078] (Step S313) In step S313, the CPU of the control unit 110 performs a process of determining whether or not the vehicle has stopped. This vehicle stop includes both the case where the control unit 110's CPU stops the vehicle 1 without depending on the operation of the driver H and the case where the driver H stops the vehicle 1. If the CPU of the control unit 110 determines that the vehicle has not stopped, it transfers the process to step S311. If it determines that the vehicle has stopped, it ends the in-driving process.

[0079] (Step S314) In step S314, the CPU of the control unit 110 performs a process of determining whether or not the vehicle speed is "0". Specifically, the CPU of the control unit 110 determines whether or not the vehicle 1 has stopped based on the vehicle speed input from the vehicle speed sensor 140a. If the CPU of the control unit 110 determines that the vehicle speed is not "0", it transfers the process to step S315. If it determines that the vehicle speed is "0", it ends the in-driving process.

[0080] (Step S315) In step S315, the CPU of the control unit 110 performs a process of determining whether it has approached a corner. For example, when there is a corner on the traveling route of the vehicle 1 and the distance to the corner reaches a predetermined distance or the estimated time until reaching the corner is within a predetermined time, the CPU of the control unit 110 determines that it has approached the corner. Note that it is desirable to change the distance, time, etc. for this determination according to the speed of the vehicle 1, the condition of the traveling road, etc. When the CPU of the control unit 110 determines that it has approached the corner, the process proceeds to step S320, and when it determines that it has not approached the corner, the process proceeds to step S316.

[0081] (Step S316) In step S316, the CPU of the control unit 110 performs a process during traveling outside the corner. In the process during traveling outside the corner, the process during traveling is performed when the traveling route is outside the corner and outside the immediate vicinity of the corner. For example, if the traveling road is a straight road, the CPU of the control unit 110 performs a process during straight traveling, etc. The details of the process during traveling outside the corner are omitted. After performing the process during traveling outside the corner, the process proceeds to step S311.

[0082] (Step S320) In step S320, the CPU of the control unit 110 performs a process for corner traveling. In the process for corner traveling, control processes of the agent 200 during traveling from before the corner to during corner traveling are performed. For example, the CPU of the control unit 110 performs a process for preventing the vehicle 1 from protruding into the oncoming lane due to understeer during corner traveling. The details of the process for corner traveling will be described later. After performing the process for corner traveling, the process proceeds to step S311.

[0083] Next, with reference to FIG. 9, the process for corner traveling performed by the CPU of the control unit 110 will be described. FIG. 9 is a subroutine of step S320 (process for corner traveling) in FIG. 8.

[0084] (Step S321) In the corner driving process, first, in step S321, the CPU of the control unit 110 performs a process of determining whether it is before entering the corner. For example, the CPU of the control unit 110 determines whether the driving route of the vehicle 1 is a straight road or has already turned. If the CPU of the control unit 110 determines that it is before entering the corner, the process proceeds to step S322. If it determines that it is not before entering the corner, that is, it has already entered the corner, the process proceeds to step S326.

[0085] (Step S322) In step S322, the CPU of the control unit 110 performs a process of making the driver H recognize the corner. Specifically, the CPU of the control unit 110 notifies the agent 200 of approaching the corner by voice, an action such as pointing, or both, to make the driver H recognize the corner. After executing the process of making the driver recognize the corner, the process proceeds to step S323.

[0086] (Step S323) In step S323, the CPU of the control unit 110 gives an instruction to decelerate. Specifically, as shown in FIG. 10, the CPU of the control unit 110 makes the agent 200 face the palm downward and spread both arms horizontally. This makes the driver H aware that in addition to concentrating on the central vision, peripheral attention is also necessary. Then, with both arms of the agent 200 spread, the CPU of the control unit 110 bends and extends both legs to squat down. This can give an instruction to decelerate and cause the driver H to perform a deceleration action. Note that by changing the amount of flexion and extension (the depth of flexion and extension) according to the degree of deceleration required, the amount of deceleration can be visualized. Also, road conditions and the like can be collected from map information and the like. Then, the required amount of deceleration is obtained including road conditions and, in some cases, information such as oncoming vehicles, and the amount of flexion and extension is set according to the deceleration rate.

[0087] For example, as shown in FIG. 11, the view in front of the corner may be blocked. In such a case, the situation ahead cannot be visually recognized, and there is a possibility that an oncoming vehicle may cross the center line and enter the lane on which the host vehicle is traveling. Therefore, in such a case, it is necessary to increase the deceleration amount more than when the visibility is good. For this reason, it is desirable to set the deceleration amount according to the road conditions and the like. Further, when it is determined that it is necessary to prompt the driver H to stop the vehicle 1, the palm of the agent 200 is directed toward the driver H. Thereby, safety can be improved. Note that not only in this step, but when it is determined that it is necessary to prompt the driver H to stop the vehicle 1, the palm of the agent 200 is directed toward the driver H at any time. Then, after giving the deceleration instruction, the process proceeds to step S324.

[0088] (Step S324) In step S324, the CPU of the control unit 110 notifies the corner exit. Here, the CPU of the control unit 110 shows the corner exit while having the driver H capture oncoming vehicles and the like in the peripheral vision so that the driver H's line of sight does not concentrate on the corner exit. Specifically, the CPU of the control unit 110 performs line-of-sight guidance on the agent 200 while pointing at the corner exit with one arm and keeping the other arm extended horizontally so that the deceleration and the driver H's line of sight do not concentrate on the corner exit. Further, by making the pointing at the corner exit for a short time, it is possible to further prevent the driver H's line of sight from concentrating on the corner exit. Then, after notifying the corner exit, the process proceeds to step S325.

[0089] (Step S325) In step S325, the CPU of the control unit 110 performs a process of determining whether or not it has reached immediately before the corner. That is, the CPU of the control unit 110 determines that it has approached the corner when the distance to the corner is a predetermined distance or the estimated time until reaching the corner is within a predetermined time, and then continues to travel. It is determined whether or not the distance, time, etc. are shorter than the above distance, time, etc. and are the distance, time, etc. set as immediately before the corner.

[0090] In addition, the distance, time, etc. set immediately before this corner can also be set as appropriate. That is, the determination immediately before the corner may be made just before the driving route starts to turn, or the determination may be made as immediately before the corner at a stage with a certain margin. Further, this determination value immediately before the corner may be changed according to the characteristics of the driver H in addition to the speed of the vehicle 1 and the situation of the driving route. When the CPU of the control unit 110 determines that it has reached immediately before the corner, the process proceeds to step S326, and processes during the progress of the corner are performed. When it is determined that it has not reached immediately before the corner, the process proceeds to step S322.

[0091] (Step S326) In step S326, the CPU of the control unit 110 causes the agent 200 to perform the expression of the curvature (R: radius of rotation) of the corner. Further, when performing the expression of the curvature of this corner, the CPU of the control unit 110 performs the expression of the curvature of the corner while imaging deceleration, and continues to perform the expression of the exit of the corner.

[0092] Specifically, as shown in FIG. 12, the CPU of the control unit 110 causes the agent 200 to imagine deceleration by stretching one arm horizontally while pointing at the corner exit with the other arm. Then, the upper body is tilted and twisted to express the tightness of the corner by the degree of tilt and twist of the upper body. Also, the depth of the corner is reproduced by changing the amount of flexion and extension on the left and right and by the twist of the upper body and the degree of flexion and extension of one knee of the agent 200. Furthermore, while continuing to point at the corner exit with one arm, the horizontal position of the other arm is maintained to imagine deceleration. Also, the amount of deceleration is expressed by the magnitude of the amount of flexion and extension. In this way, the agent 200 expresses the depth of the curve by twisting the upper body and the degree of flexion and extension of one knee, and by using not only the movement of the arms but also the flexion and extension with the shoulders to twist the body. As a result, just by the driver H catching the agent 200 in the peripheral vision, the driver H can grasp the situation such as the size of the radius R of the corner and can drive safely without looking around. After expressing the curvature of the corner, the process proceeds to step S327.

[0093] (Step S327) In step S327, the CPU of the control unit 110 performs a process of determining whether the corner has ended. If the CPU of the control unit 110 determines that the corner has ended, it ends the corner driving process. If it determines that the corner has not ended, the process proceeds to step S328.

[0094] (Step S328) In step S328, the CPU of the control unit 110 performs a process of determining whether the curvature has changed. That is, even at one corner, the curvature may change midway. For example, even at one corner, there may be a plurality of curves. Specifically, at one corner, the curvature of the first half curve, that is, the turning radius, may be different from the curvature of the second half curve. Also, the direction of the curve may reverse midway. Therefore, it is determined whether the curvature has changed, and if the curvature has changed, corresponding actions are taken according to the change. When the CPU of the control unit 110 determines that the curvature has changed, the process proceeds to step S329, and when it determines that the curvature has not changed, the process proceeds to step S326.

[0095] (Step S329) In step S329, the CPU of the control unit 110 changes the expression of the agent 200 according to the changed curvature. Specifically, when the curvature of the curve changes, or before it changes, the CPU of the control unit 110 changes the inclination and the degree of torsion of the upper body of the agent 200. Also, in order to indicate that the curvature changes, an operation such as bending one end arm may be inserted. Then, after changing to the expression according to the curvature, the process proceeds to step S326.

[0096] As described above, the agent device 100 of the present embodiment simultaneously gives the depth (R) of the corner and the line-of-sight guidance to the agent 200, so that the driver H can be made to pay attention to the surroundings at the corner while also performing a predetermined driving operation, and it is possible to prevent the vehicle from protruding into the oncoming lane due to understeer. That is, it is possible to prevent the driver H's line of sight from concentrating on a specific object and overlooking the surroundings, thereby improving safety.

[0097] Note that the agent control process of the present embodiment is executed by the CPU etc. of the control unit 110 after a program stored in the ROM, RAM, EEPROM, etc. of the control unit 110 is expanded to the RAM etc. of the control unit 110. Furthermore, in the present embodiment, the shift position sensor 140f constitutes the shift position detection unit of the present application. Also, in the present embodiment, the surrounding environment recognition unit 120 constitutes the surrounding situation acquisition unit of the present application.

[0098] Also, in the present embodiment, the agent control unit of the present application is configured by both the control unit 110 and the agent control unit 190. However, as described above, only the agent control unit 190 may have the functions of both the control unit 110 and the agent control unit 190 in the present embodiment.

Description of Reference Numerals

[0099] 1: Vehicle, 100: Agent device, 110: Control unit, 120: Surrounding environment recognition unit, 130: Occupant state recognition unit, 140: Vehicle state recognition unit, 140f: Shift position sensor, 150: Speaker, 160: Microphone, 170: Memory unit, 180: Transceiver, 190: Agent control unit, 191: Agent speaker, 200: Agent

Claims

1. a memory unit that stores conversations with the driver; a surrounding situation acquisition unit that acquires the surrounding situation; an agent control unit that controls an anthropomorphic agent; comprising the agent is arranged at a position that enters the driver's peripheral vision, the agent control unit is capable of executing a conversation according to the driver's hobbies and preferences based on at least the information stored in the memory unit; when it is determined that there is a corner on the driving route by acquiring the surrounding situation, after executing a process of making the agent recognize the corner, the agent spreads both arms horizontally and uses the flexion and extension of both arms to represent the curvature of the corner; An agent device characterized by the above.

2. the agent control unit tilts the upper body and applies torsion, and represents the curvature of the corner by the degree of tilt and torsion of the upper body; The agent device according to claim 1, characterized by the above.

3. the agent control unit During the progress of the corner, maintains one arm raised and points to the exit of the corner with the other arm; The agent device according to claim 1 or claim 2, characterized by the above.

4. the agent control unit when it is determined that it is necessary to prompt the driver to stop, turns the palm towards the driver; The agent device according to any one of claims 1 to 3, characterized by the above.

Images