Vehicle operation support system

Abstract

To provide a vehicle operation support system 10 capable of evaluating driver's receptiveness to driving assistance.SOLUTION: A vehicle operation support system 10, which supports driver's operation through a vehicle agent V performing communication with a driver of a vehicle 1, comprises: history storing means (driver data 29) which stores history information (behavior history information 291 and operation history information 292) with history of driver's operation registered therein; driver state detection means (an in-vehicle camera 24, an activation sensor 25, and a microphone 26) which detects a driver's state; intimacy calculation means (ECU 27) which calculates driver's intimacy to a vehicle 1 based on the history information; and dependency calculation means (ECU 27) which calculates driver's dependency on the vehicle 1 based on a driver's response state with respect to words and acts of the vehicle agent V identified from the driver's state.SELECTED DRAWING: Figure 7

Description

[Technical Field]

[0001] The present invention relates to a vehicle driving assistance system that assists a driver in driving a vehicle by communicating with the driver through the words and actions of a vehicle agent, which is an avatar of the vehicle. [Background technology]

[0002] In automobiles and other vehicles, known vehicle driving assistance systems that assist the driver in driving operations include, for example, advanced driving assistance systems that partially intervene in the driver's driving operations, and autonomous driving systems that take over driving operations from the driver under certain conditions and perform autonomous driving.

[0003] Furthermore, in vehicle driving assistance systems, there are also known technologies that provide driving assistance by providing guidance on driving operations according to the conditions around the vehicle, and guidance on handing over driving operations between the driver and the vehicle, not only through voice and display but also through a vehicle agent, which is an avatar of the vehicle.

[0004] However, if the driver does not trust the driving assistance provided via the vehicle agent, for example, the driving assistance provided by the vehicle driving assistance system may not function effectively to ensure the safety of the driver while driving. Therefore, a technology has been proposed that simulates emotions in a vehicle and builds a good psychological relationship between the driver and the vehicle via a vehicle agent.

[0005] For example, in Patent Document 1, when the vehicle driving assistance system is not intervening and the shift position is in the parking range, a vehicle agent that personifies the vehicle faces the driver and behaves in accordance with voice guidance, thereby building a good psychological relationship between the driver and the vehicle.

[0006] However, in Patent Document 1, the vehicle agent builds a one-sided psychological relationship with the driver, regardless of whether the driver accepts the vehicle agent's words and actions or the intervention of the vehicle's driving assistance. For this reason, Patent Document 1 does not necessarily build a good psychological relationship between the driver and the vehicle via the vehicle agent, and there is room for improvement.

[0007] Therefore, it is possible to consider building a good psychological relationship between the driver and the vehicle via a vehicle agent, taking into account the driver's receptivity to driving assistance, but there is a problem in that it is not easy to evaluate the driver's receptivity to driving assistance. [Prior art documents] [Patent documents]

[0008] [Patent Document 1] Japanese Patent Publication No. 2023-19243 Summary of the Invention [Problem to be solved by the invention]

[0009] In view of the above-mentioned problems, an object of the present invention is to provide a vehicle driving assistance system that can evaluate a driver's acceptability for driving assistance. [Means for solving the problem]

[0010] This invention is a vehicle driving assistance system that assists the driver's driving operations via a vehicle agent that communicates with the driver of the vehicle, and is characterized by comprising a history storage means that stores history information in which the history of the driver's driving operations is registered, a driver state detection means that detects the state of the driver, an intimacy calculation means that calculates the driver's intimacy with the vehicle based on the history information, and a dependency calculation means that calculates the driver's dependency on the vehicle based on the driver's response status to the behavior and words of the vehicle agent identified from the driver's state.

[0011] The vehicle agent refers to, for example, a three-dimensional model displayed on a display means inside the vehicle, or a robot or the like placed inside the vehicle. Assisting the driver's driving operation means partially intervening in the driver's driving operation to control the operation of the vehicle, or taking over the driving operation from the driver to control the operation of the vehicle.

[0012] The history of the driver's driving operations refers to a history of the driver's driving dates and times, a history of the driving behavior of the driver based on the driver's driving operations, and a history of compliance with traffic rules. Detecting the driver's state means detecting the driver's vocalizations, detecting the driver's movements, detecting the driver's heart rate, detecting the driver's amount of sweat, and detecting the driver's amount of exercise.

[0013] The driver's response state refers to whether the driver responded to the vehicle agent's words and actions, whether the driver nodded, whether the driver's activity level such as heart rate increased, whether the driver performed driving operations, and so on.

[0014] According to the present invention, it is possible to evaluate the driver's receptivity to driving assistance based on the driver's familiarity with the vehicle and the driver's degree of dependence on the vehicle. Specifically, by calculating the degree of intimacy of the driver with the vehicle based on the history information, it is possible to know the degree of intimacy and attachment the driver feels with the vehicle.

[0015] Furthermore, by calculating the driver's dependence on the vehicle based on the driver's response to the vehicle agent's words and actions, the vehicle driving assistance system can determine whether the driver's psychological state tends to be submissive to the vehicle and accept the vehicle agent's words and actions, or whether the driver tends to be driver-led and rely on his or her own judgment, ignoring the vehicle agent's words and actions.

[0016] As a result, if both the intimacy and the dependence are low, the vehicle driving assistance system can evaluate that the driver not only has no attachment to the vehicle or the vehicle agent, but also does not readily accept driving assistance via the vehicle agent.

[0017] On the other hand, when both the intimacy and the dependence are high, the vehicle driving assistance system can be evaluated as indicating that the driver has a favorable impression and accepts not only the vehicle and the vehicle agent, but also the driving assistance via the vehicle agent.

[0018] Therefore, the vehicle driving assistance system can evaluate the driver's receptivity to driving assistance based on the driver's familiarity with the vehicle and the driver's dependency on the vehicle, and thus can build a good psychological relationship between the driver and the vehicle, for example, based on the driver's receptivity to driving assistance.

[0019] As an aspect of the present invention, a mapping means may be provided that associates the degree of dependence calculated by the dependence calculation means with the degree of intimacy calculated by the intimacy calculation means on a virtual two-dimensional coordinate plane formed by a first axis indicating the degree of dependence and a second axis indicating the degree of intimacy, and maps them as the psychological state of the driver.

[0020] According to this configuration, for example, by outputting the driver's psychological state regarding the vehicle, mapped on a virtual two-dimensional coordinate plane, to a display means, the driver and vehicle developers can be objectively informed of the driver's receptivity to driving assistance.

[0021] As another aspect of the present invention, the driver state detection means may be configured to detect the driver's state based on speech and behavior and biological information of the driver.

[0022] The biological information includes the driver's heart rate, amount of sweating, amount of exercise, and the like. According to this configuration, the driver's response state to the vehicle agent's words and actions can be accurately identified, and the driver's dependency on the vehicle can be accurately calculated.

[0023] As another aspect of the present invention, the intimacy calculation means may calculate the intimacy based on an index value indicating roughness of driving operation, an index value indicating driving frequency, and an index value indicating driving intervals. According to this configuration, the degree of familiarity of the driver with the vehicle can be easily calculated based on the roughness of the driver's driving operations and the frequency of the driver's driving opportunities.

[0024] Another aspect of the present invention may include a behavioral information storage means for storing a plurality of behavioral information indicating the content of the behavior to be performed by the vehicle agent, a behavioral information selection means for selecting from the behavioral information storage means the behavioral information that makes the intimacy calculated by the intimacy calculation means higher than the median value of intimacy and brings the dependency calculated by the dependency calculation means closer to the median value of dependency, and an agent control means for controlling the behavior of the vehicle agent based on the behavioral information selected by the behavioral information selection means.

[0025] Controlling the behavior of the vehicle agent includes not only controlling the voice output and the operation based on the behavior information, but also controlling the timing of the control based on the behavior information.

[0026] According to this configuration, the vehicle agent can encourage the driver to accept the intervention of driving assistance via the vehicle agent while preventing the driver from becoming too dependent on the vehicle's driving assistance. Therefore, the vehicle driving assistance system can build a psychological relationship between the driver and the vehicle via the vehicle agent, for example. [Effects of the Invention]

[0027] The present invention provides a vehicle driving assistance system that can evaluate a driver's acceptability for driving assistance. [Brief explanation of the drawings]

[0028] [Figure 1] FIG. 2 is a front view showing the exterior of the vehicle driving assistance system as seen from inside the vehicle cabin. [Figure 2] FIG. 1 is a block diagram showing the internal configuration of a vehicle driving assistance system. [Figure 3] FIG. 2 is a schematic explanatory diagram illustrating an outline of speech and behavior information. [Figure 4] FIG. 1 is an explanatory diagram illustrating the psychological state of a driver. [Figure 5] 10 is a flowchart showing a processing operation relating to the control of a vehicle agent. [Figure 6] 10 is a flowchart showing the processing operations of psychological information update processing. [Figure 7] 5 is a flowchart showing the processing operation of a driver's psychology estimation process. [Figure 8] FIG. 10 is an explanatory diagram illustrating an outline of a closeness calculation process. [Figure 9] FIG. 10 is a schematic explanatory diagram of a driver's psychology estimation means according to another embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0029] An embodiment of the present invention will be described below with reference to the drawings. In this embodiment, a vehicle driving assistance system 10 that assists the driver's driving operation via a vehicle agent V that communicates with the driver of a vehicle 1 will be described with reference to FIGS. 1 to 4. FIG.

[0030] Note that Figure 1 shows a front view of the vehicle driving assistance system 10 as seen from inside the cabin of the vehicle 1, Figure 2 shows a block diagram of the vehicle driving assistance system 10, Figure 3 shows a schematic diagram outlining the speech and behavior data 28, and Figure 4 shows an explanatory diagram explaining the psychological state of the driver.

[0031] First, as shown in FIG. 1, the interior of a vehicle 1 is provided with an instrument panel 2 extending in the vehicle width direction (reference numerals omitted), and a center console 3 extending from approximately the center of the instrument panel 2 in the vehicle width direction toward the rear of the vehicle. Furthermore, the vehicle 1 is provided with a steering wheel 4 in the cabin, which is disposed in front of a driver's seat (not shown) and receives steering operations from a passenger seated in the driver's seat.

[0032] As shown in FIG. 2, the vehicle driving assistance system 10 in such a vehicle 1 includes an engine 11, a brake device 12, and a steering device 13 arranged outside the vehicle cabin, a display unit 14 provided at approximately the center of the instrument panel 2 in the vehicle width direction, and a speaker 15 arranged inside the vehicle cabin.

[0033] Furthermore, as shown in FIG. 2, the vehicle driving assistance system 10 is equipped with an accelerator opening sensor 16, a brake pedal stroke sensor 17, a vehicle speed sensor 18, a steering angle sensor 19, an acceleration sensor 20, and a yaw rate sensor 21 for detecting the driving behavior of the vehicle.

[0034] In addition, as shown in FIG. 2, the vehicle driving assistance system 10 includes an exterior camera 22 and a millimeter wave radar 23 for acquiring information about the area in front of the vehicle, an interior camera 24 for detecting the driver's condition, an activity sensor 25, and a microphone 26, and an electronic control unit (ECU) for controlling the operation of each of the above-mentioned components. The engine is equipped with an ECU (Electric Control Unit) 27.

[0035] More specifically, the engine 11 is made up of an engine body (not shown in detail) and a fuel injection device that supplies fuel to the engine body, etc. The engine output and engine speed of the engine 11 are controlled based on an output signal from the ECU 27.

[0036] The braking device 12 is configured with a brake caliper, a brake actuator, etc. (details of which are omitted from the drawing). The braking device 12 is controlled based on an output signal from the ECU 27 so as to generate a braking force.

[0037] The steering device 13 is composed of the above-mentioned steering wheel 4, an electric motor (not shown) that assists the driver in steering, etc. The steering device 13 is controlled to change the traveling direction of the vehicle based on an output signal from the ECU 27.

[0038] The display unit 14 is configured, for example, with a liquid crystal display, and has a function of displaying various information to a passenger, such as a driver, seated in the driver's seat, based on control signals from the ECU 27. For example, as shown in Fig. 1, the display unit 14 displays a vehicle agent V that communicates with the driver as an entity of the vehicle 1.

[0039] The speaker 15 is disposed inside the vehicle cabin and has the function of outputting a warning sound or voice based on a notification signal from the ECU 27 to notify the occupants. For example, the speaker 15 outputs a voice corresponding to the operation of the vehicle agent V displayed on the display unit 14.

[0040] The accelerator position sensor 16 is a sensor that is integrally provided on an accelerator pedal operated by the driver. The accelerator position sensor 16 has a function of detecting the displacement angle of the accelerator pedal caused by the driver's operation and a function of outputting the detected displacement angle to the ECU 27 as an accelerator position signal.

[0041] The brake pedal stroke sensor 17 is a sensor that is integrally provided on the brake pedal operated by the driver. The brake pedal stroke sensor 17 has a function of detecting the displacement angle of the brake pedal operated by the driver and a function of outputting the detected displacement angle to the ECU 27 as a brake signal.

[0042] The vehicle speed sensor 18 also has a function of detecting the rotation speed of a drive shaft connected to the wheels, for example, and a function of outputting the detected rotation speed to the ECU 27 as a rotation speed signal. The steering angle sensor 19 has a function of detecting the steering angle of the steering wheel 4 operated by the driver, and a function of outputting the detected steering angle to the ECU 27 as a steering angle signal.

[0043] In addition, the acceleration sensor 20 is arranged, for example, inside the center console 3, and has the function of detecting the longitudinal acceleration along the fore-and-aft direction of the vehicle 1 and the lateral acceleration along the width direction of the vehicle 1, and the function of outputting the detected longitudinal acceleration and lateral acceleration to the ECU 27 as acceleration signals.

[0044] The yaw rate sensor 21 is disposed, for example, inside the center console 3, and has a function of detecting the yaw rate of the vehicle 1 and a function of outputting the detected yaw rate to the ECU 27 as a yaw rate signal. The vehicle exterior camera 22 is disposed, for example, at an upper portion of the windshield on the passenger compartment side, and has a function of capturing an image in front of the vehicle and a function of outputting the captured video image to the ECU 27 .

[0045] The millimeter wave radar 23 is disposed, for example, in the front bumper, and has the functions of emitting millimeter waves as irradiated waves in front of the vehicle, receiving reflected waves reflected by preceding vehicles, parked vehicles, pedestrians, etc., and outputting information obtained from the irradiated waves and reflected waves to the ECU 27 as a radar signal.

[0046] 1, the in-vehicle camera 24 is provided on the instrument panel 2 or a front pillar (reference numeral omitted) facing the driver seated in the driver's seat. The in-vehicle camera 24 has a function of capturing an image of the driver seated in the driver's seat, focusing on the head, and a function of outputting the captured video to the ECU 27.

[0047] In addition, the activity sensor 25 is built into, for example, the steering wheel 4, and has the function of detecting the heart rate, amount of sweating, and amount of exercise of the driver holding the steering wheel 4, and the function of outputting an activity signal indicating the detected heart rate, amount of sweating, and amount of exercise to the ECU 27.

[0048] In addition, the microphone 26 is arranged on the top of the column cover (reference numeral omitted), for example, as shown in FIG. 1, and has the function of detecting the voice emitted by the driver seated in the driver's seat and the function of outputting the detected voice to the ECU 27 as an audio signal.

[0049] The ECU 27 is made up of hardware components such as a CPU, a memory, and a storage unit for storing various information, and software components such as programs and data. This ECU 27 has the function of acquiring various signals output by the above-mentioned sensors, the exterior camera 22, the millimeter-wave radar 23, the interior camera 24, the activity sensor 25, and the microphone 26, and the function of performing various information processing based on the acquired signals. Furthermore, the ECU 27 has a function of controlling the operation of the display unit 14 and the speaker 15 , and a function of controlling the operation of the engine 11 , the brake device 12 and the steering device 13 .

[0050] More specifically, the ECU 27 realizes the functions of calculating the accelerator opening in cooperation with the accelerator opening sensor 16, calculating the amount of brake pedal depression in cooperation with the brake pedal stroke sensor 17, and calculating the vehicle speed of the vehicle 1 in cooperation with the vehicle speed sensor 18.

[0051] Furthermore, the ECU 27 realizes a function of calculating the steering angle of the steering wheel 4 in cooperation with the steering angle sensor 19, and a function of detecting the driving behavior of the vehicle 1 in cooperation with the acceleration sensor 20 and the yaw rate sensor 21.

[0052] In addition, the ECU 27 has the function of identifying the situation in front of the vehicle in cooperation with the exterior camera 22 and millimeter wave radar 23, and the function of identifying the driver seated in the driver's seat and identifying the driver's condition in cooperation with the interior camera 24, activity sensor 25 and microphone 26.

[0053] Such an ECU 27 is configured to alert the driver and notify them of the need for driving assistance intervention through the words and actions of the vehicle agent V depending on the driving behavior and the situation ahead of the vehicle, and to control the operation of the engine 11, braking device 12, and steering device 13 to provide driving assistance that assists the driver in driving operations.

[0054] The vehicle agent V is an entity of the vehicle 1 that is set with pseudo-emotions, and its speech and behavior are controlled by the ECU 27 so as to act and speak in accordance with the driving behavior and the situation ahead of the vehicle.

[0055] In order to realize driving assistance via this vehicle agent V, the ECU 27 stores speech and behavior data 28 in which the lines and actions of the vehicle agent V are registered, and driver data 29 in which various information for each driver is registered, as shown in Figure 2.

[0056] As shown in FIG. 3, the speech and behavior data 28 includes a plurality of pieces of speech and behavior information that associate the lines to be output to the speaker 15 as the voice of the vehicle agent V with the actions that the vehicle agent V takes in response to the lines. In the speech and behavior data 28, even if the content indicates a greeting at the start, for example, a plurality of pieces of speech and behavior information are registered, which are expressed differently depending on the degree of dependence and the degree of intimacy, which will be described later.

[0057] Furthermore, although detailed illustration is omitted, the speech and behavior information is assigned a flag indicating whether the expression increases or decreases intimacy, as described below, and a flag indicating whether the expression increases or decreases dependence.

[0058] Specifically, the behavioral data 28 includes the behavioral and verbal information of the vehicle agent V at the time of starting or during normal times when driving assistance is not being provided, the behavioral and verbal information of the vehicle agent V regarding driving assistance, and the behavioral and verbal information of the vehicle agent V in response to the driver's driving operations.

[0059] For example, as shown in Figure 3, the speech and behavior data 28 includes registered speech and behaviors such as "Good morning..." spoken by the vehicle agent V to a driver with a low level of intimacy and "Stand still" as speech and behaviors at the time of starting, etc., and speech and behaviors such as "Good morning! Thank you for today" and "Expression of joy" spoken by the vehicle agent V to a driver with a high level of intimacy.

[0060] Furthermore, for example, the speech and behavior data 28 stores, as speech and behavior when sudden braking is performed under the control of the ECU 27, lines such as "You're driving carelessly" and an action of "expressing anger" for drivers with low intimacy, and lines such as "I'm surprised, I'm nervous" and an action of "expressing surprise" for drivers with high intimacy.

[0061] Furthermore, for example, speech and behavior data 28 includes registered speech and behaviors to request driving operations by the driver, such as "There is construction ahead, so I need help changing lanes," and an action called "expression of distress."

[0062] In addition, for example, in the speech and behavior data 28, when the driver's response to the speech and behavior of the vehicle agent V is detected, the line "Thank you for operating it" and the action "Stay still" are registered as speech and behaviors for a driver with a low level of intimacy, and the line "Thank you for helping me" and the action "Expression of joy" are registered as speech and behaviors for a driver with a high level of intimacy.

[0063] On the other hand, the driver data 29 stores behavior history information 291 in which the driving behavior during driving and the like are registered, driving history information 292 in which the date and time when the driver drove and the like are registered, and driver psychological information 293 indicative of the driver's psychological state regarding the vehicle 1.

[0064] Specifically, the behavior history information 291 chronologically registers the angular velocity of the accelerator pedal, the angular velocity of the brake pedal, the angular velocity of the steering wheel 4, the acceleration change, the yaw rate, etc. Furthermore, the behavior history information 291 also registers the number of sudden operations (for example, sudden acceleration, sudden braking, or sudden steering) calculated by the ECU 27 based on the various registered information.

[0065] In addition, the driving history information 292 stores the date and time when the driver drove and the distance traveled, the number of times the driver drove in the most recent month calculated by the ECU 27 based on the date and time when the driver drove and the distance traveled, and the number of days that have passed since the last time the driver drove, in association with each other.

[0066] In addition, the driver psychology information 293 registers the degree of familiarity of the driver with the vehicle 1 calculated by the ECU 27 and the degree of dependency of the driver on the vehicle 1 calculated by the ECU 27 in association with each other. As shown in FIG. 4, this driver psychological information 293 can be mapped as the driver's psychological state on a virtual two-dimensional coordinate plane with the horizontal axis representing the degree of dependence and the vertical axis representing the degree of intimacy, and output to the display unit 14.

[0067] In addition, if the intersection P of the horizontal axis indicating the degree of dependence and the vertical axis indicating the degree of intimacy in Figure 4 is taken as the median of the degree of dependence and the median of the degree of intimacy, the initial value of the driver psychological information 293 is set so that the degree of intimacy of the driver psychological information 293 is approximately the same as the median of the intimacy, and the degree of dependence of the driver psychological information 293 is lower than the median of the degree of dependence.

[0068] Here, the degree of familiarity of the driver with the vehicle 1 and the degree of dependency of the driver on the vehicle 1 will be described in more detail. First, the driver's intimacy with vehicle 1 is an indicator of the driver's psychological state estimated by ECU 27 based on behavior history information 291 and driving history information 292, and indicates the degree of intimacy and attachment the driver feels toward vehicle 1.

[0069] The intimacy of this driver with vehicle 1 is determined by the psychological state of the driver who feels a sense of familiarity or attachment to vehicle 1 being considered to be an intimate state, with a high intimacy level, and conversely, the psychological state of the driver who does not feel a sense of familiarity or attachment to vehicle 1 being considered to be an estranged state, with a low intimacy level (see Figure 4).

[0070] In this embodiment, it is assumed that the more familiar and attached the driver is to the vehicle 1, the less abrupt the driver's driving will be and the more opportunities to drive it will be, and the more abrupt the driver's driving will be, the more intimate the state will be, and the more abrupt the driver's driving will be and the fewer opportunities to drive it will be, the more distant the state will be.

[0071] In addition, the driver's dependence on vehicle 1 is an indicator of the driver's psychological state estimated by ECU 27 based on the driver's words and actions in response to the words and actions of vehicle agent V, and indicates the extent to which the driver depends on driving assistance via vehicle agent V.

[0072] The degree of dependency of this driver on the vehicle 1 is high when the driver's psychological state is such that he or she is easily reliant on driving assistance via the vehicle agent V, as this state is considered to be subordinate to the vehicle 1, while conversely, the degree of dependency is low when the driver's psychological state is such that he or she does not rely on driving assistance via the vehicle agent V and prioritizes his or her own judgment, as this state is considered to be dominant (see Figure 4).

[0073] In this embodiment, if the driver responds to the words and actions of the vehicle agent V by replying or nodding, the vehicle is considered to be subordinate to the vehicle 1, and if the driver does not respond to the words and actions of the vehicle agent V, the driver is considered to be dominant.

[0074] Next, the processing operation of the ECU 27 in the vehicle driving assistance system 10 having the above-described configuration will be described with reference to FIGS. Note that Figure 5 shows a flowchart regarding the control of the vehicle agent V, Figure 6 shows a flowchart of the psychological information update process, Figure 7 shows a flowchart of the driver psychological estimation process, and Figure 8 shows an explanatory diagram outlining the intimacy calculation process.

[0075] First, the processing operation of the ECU 27 for controlling the behavior of the vehicle agent V will be described. When power is supplied to the ECU 27 from the battery, the ECU 27 executes a predetermined program stored therein and starts a processing operation for controlling the behavior of the vehicle agent V.

[0076] The ECU 27, which has started processing operations to control the behavior of the vehicle agent V, identifies the driver sitting in the driver's seat based on video images from the in-vehicle camera 24 and audio signals from the microphone 26 (step S101), as shown in Figure 5, and reads the behavior data 28 and driver data 29 corresponding to the identified driver (step S102).

[0077] Furthermore, after displaying the vehicle agent V on the display unit 14 (step S103), the ECU 27 determines whether it is time to have the vehicle agent V perform normal speech and behavior, such as greetings at start-up (step S104).

[0078] If it is not the timing to make the vehicle agent V speak or act normally (step S104: No), the ECU 27 determines whether it is the timing to make the vehicle agent V speak or act regarding driving assistance (step S105).

[0079] For example, the ECU 27 determines whether it is the right time to warn the driver when the distance between the vehicle and the vehicle ahead becomes narrow, the right time to provide guidance to the driver regarding his driving operation, or the right time to forcibly intervene in the driver's driving operation and provide guidance after controlling the operation of the vehicle 1.

[0080] If it is not the time to have the vehicle agent V make any statements or actions related to driving assistance (step 105: No), the ECU 27 identifies the driver sitting in the driver's seat based on video images from the in-vehicle camera 24, and determines whether the identified driver is the same as the driver identified in step S101 (step S106).

[0081] If the driver is not the same (step S106: No), the ECU 27 returns the process to step S101 and repeats the processes from step S101 to step S108, which will be described later, until the supply of electric power is interrupted.

[0082] On the other hand, if the driver is the same (step S106: Yes), the ECU 27 returns the process to step S104 and repeats the processes from step S104 to step S108, which will be described later, until the supply of electric power is interrupted.

[0083] Also, in step S104 of Figure 5, if it is time to have the vehicle agent V behave in a normal manner (step S104: Yes), or if it is time to have the vehicle agent V behave in a manner related to driving assistance (step S105: Yes), the ECU 27 determines the behavior to be performed by the vehicle agent V based on the behavior data 28 (step S107).

[0084] At this time, the ECU 27 determines the words and actions of the vehicle agent V based on the driver's psychological information 293 so as to bring the current psychological state of the driver indicated by the driver's psychological information 293 closer to the target state.

[0085] In addition, the intersection P of the horizontal axis indicating the degree of dependence and the vertical axis indicating the degree of intimacy in Figure 4 is taken as the median of the degree of dependence and the median of the degree of intimacy, and the target state in this embodiment is a state in which the degree of dependence is at the median and the degree of intimacy is higher than the median, as shown in Figure 4.

[0086] As an example, in step S107, it is assumed that the current driver's intimacy with the vehicle 1 is lower than the intimacy indicating the target state, and the current driver's dependency on the vehicle 1 is lower than the median dependency (see FIG. 4).

[0087] In this case, based on the flags assigned to the behavioral information (lines and actions) of the behavioral data 28, the ECU 27 extracts and determines from the behavioral data 28 lines and actions of the vehicle agent V that are suitable for increasing both the driver's intimacy with the vehicle 1 and the driver's dependence on the vehicle 1.

[0088] Thereafter, the ECU 27 outputs a voice based on the lines determined in step S107 from the speaker 15, and operates the vehicle agent V displayed on the display unit 14 based on the action determined in step S107 (step S108). Then, the ECU 27 proceeds to step S106 and repeats the processes from step S101 to step S108 until the supply of electric power is interrupted.

[0089] In this way, the vehicle driving assistance system 10 assists the driver in driving by having the vehicle agent V act in a manner determined based on the driver's psychological information 293. At this time, the vehicle driving assistance system 10 successively updates the driver psychological information 293 at appropriate timing in order to bring the driver's psychological state closer to a target state.

[0090] Next, a psychological information updating process for updating the driver psychological information 293 that determines the behavior of the vehicle agent V will be described. When power is supplied to the ECU 27 from the battery, the ECU 27 executes a predetermined program stored therein to start a psychological information update process for updating the driver psychological information 293.

[0091] As shown in FIG. 6, the ECU 27, which has started the psychological information update process, identifies the driver sitting in the driver's seat based on the video image from the in-vehicle camera 24 and the audio signal from the microphone 26 (step S111), and reads the speech and behavior data 28 and the driver data 29 corresponding to the identified driver (step S112). Note that the data in steps S111 and S112 may be obtained in steps S101 and S102 in FIG. 5 described above.

[0092] Furthermore, after starting to record the driving behavior of the vehicle 1 in the behavior history information 291 (step S113), the ECU 27 performs a driver's psychology estimation process (step S114) to estimate the driver's psychological state.

[0093] Specifically, the ECU 27, which has started the driver's psychology estimation process, determines whether or not the intimacy has been calculated at least once since the engine 11 was started, for example, as shown in FIG. 7 (step S121). If the degree of intimacy has been calculated once (step S121: Yes), the ECU 27 proceeds to step S123, which will be described later.

[0094] On the other hand, if the intimacy has not been calculated (step S121: No), the ECU 27 performs an intimacy calculation process to calculate the intimacy of the driver with the vehicle 1 based on the behavior history information 291 and the driving history information 292 (step S122).

[0095] Here, the driver's familiarity with vehicle 1 is calculated based on a driving roughness index value indicating the roughness of the driver's driving operation, a driving frequency index value indicating the number of times the driver has driven per past month, and a driving interval index value indicating the number of days elapsed since the last time the driver drove, as shown in Figure 8.

[0096] The driving roughness index value is calculated by dividing the actual value obtained by dividing the number of sudden operations (such as sudden braking or sudden steering) during one drive by the distance traveled during one drive, and then dividing the actual value from the previous drive by the average actual value from the previous drive to the drive before that. Specifically, the degree of familiarity of the driver with the vehicle 1 is calculated by subtracting the driving interval index value from the sum of the roughness index value and the driving frequency index value, as shown in FIG.

[0097] After calculating the degree of familiarity of the driver with the vehicle 1, the ECU 27 determines whether the current state of the vehicle 1 is during driving assistance via the vehicle agent V, as shown in FIG. 7 (step S123). If driving assistance via the vehicle agent V is not being performed (step S123: No), the ECU 27 ends the driver's psychology estimation process and proceeds to step S115 in FIG.

[0098] On the other hand, if driving assistance is being provided via the vehicle agent V (step S123: Yes), the ECU 27 determines whether or not a response other than the driver's driving operation has been detected when the vehicle agent V makes a statement or action related to driving assistance (step S124).

[0099] For example, the ECU 27 identifies whether the driver has responded favorably to the voice of the vehicle agent V based on the audio signal from the microphone 26, and if the driver has responded favorably, determines that a response other than the driver's driving operation has been detected.

[0100] Alternatively, the ECU 27 determines whether the driver nods in response to the words and actions of the vehicle agent V based on video images from the in-vehicle camera 24, and if the driver nods, determines that a response other than the driver's driving operation has been detected.

[0101] Alternatively, the ECU 27 detects a change in the driver's activity level based on the activity level signal from the activity level sensor 25, and when the driver's activity level increases, determines that a response other than the driver's driving operation has been detected.

[0102] If the driver's response to the vehicle agent V's words and actions cannot be detected (step S124: No), the ECU 27 determines whether the driver performed appropriate driving operations in response to the vehicle agent V's words and actions regarding driving assistance (step S125).

[0103] If the driver is not performing an appropriate driving operation (step S125: No), the ECU 27 determines that the driver has little interest in the behavior of the vehicle agent V and is ignoring the driving assistance provided through the vehicle agent V.

[0104] Therefore, the ECU 27 reduces the current degree of dependency of the driver on the vehicle 1 and temporarily stores it as a new degree of dependency (step S126), then ends the driver psychology estimation process and proceeds to step S115 in FIG.

[0105] On the other hand, if the ECU 27 detects in step S124 a response from the driver to the behavior of the vehicle agent V (step S124: Yes), or if the driver performs appropriate driving operations in step S125 (step S125: Yes), the ECU 27 increases the driver's current degree of dependence on the vehicle 1 and temporarily stores it as a new degree of dependence (step S127) because the driver is dependent on driving assistance via the vehicle agent V. Thereafter, the ECU 27 ends the driver's psychology estimation process and proceeds to step S115 in FIG.

[0106] Returning to FIG. 6, when the driver psychology estimation process in step S114 is completed, the ECU 27 updates the driver psychology information 293 (step S115). Specifically, the ECU 27 updates the driver psychology information 293 using the degree of familiarity of the driver with the vehicle 1 and / or the degree of dependency of the driver on the vehicle 1 that are temporarily stored in the driver psychology estimation process in step S114.

[0107] Thereafter, the ECU 27 identifies the driver sitting in the driver's seat based on the video image from the in-vehicle camera 24, and determines whether the identified driver is the same as the driver identified in step S101 (step S116).

[0108] If the driver is not the same (step S116: No), the ECU 27 returns the process to step S111 and repeats the processes from step S111 to step S116 until the supply of electric power is interrupted.

[0109] On the other hand, if the driver is the same (step S116: Yes), the ECU 27 returns the process to step S114 and repeats the processes from step S114 to step S116 until the supply of electric power is interrupted.

[0110] In this way, the vehicle driving assistance system 10 successively updates the driver psychological information 293 that determines the behavior of the vehicle agent V. As a result, the vehicle driving assistance system 10 can change the behavior of the vehicle agent V over time, thereby bringing the driver's psychological state closer to a target state.

[0111] As described above, the vehicle driving assistance system 10 of this embodiment is a system that assists the driver in driving the vehicle 1 via the vehicle agent V that communicates with the driver. This vehicle driving assistance system 10 includes a history storage means (driver data 29) that stores history information (behavior history information 291, driving history information 292) that registers the history of driving operations by the driver, and a driver state detection means (in-vehicle camera 24, activity sensor 25, microphone 26) that detects the state of the driver.

[0112] Furthermore, the vehicle driving assistance system 10 is equipped with an intimacy calculation means (ECU 27) that calculates the driver's intimacy with the vehicle 1 based on historical information, and a dependency calculation means (ECU 27) that calculates the driver's dependency on the vehicle 1 based on the driver's response to the behavior of the vehicle agent V identified from the driver's state.

[0113] According to this configuration, the driver's receptivity to driving assistance can be evaluated based on the driver's familiarity with the vehicle 1 and the driver's dependency on the vehicle 1. Specifically, by calculating the degree of intimacy of the driver with the vehicle 1 based on the history information (behavior history information 291, driving history information 292), it is possible to know the degree of intimacy and attachment the driver feels toward the vehicle 1.

[0114] Furthermore, by calculating the driver's degree of dependence on the vehicle 1 based on the driver's response to the vehicle agent V's words and actions, the vehicle driving assistance system 10 can determine whether the driver's psychological state tends to be submissive to the vehicle 1 and accept the vehicle agent V's words and actions, or whether the driver tends to be driver-led and rely on his or her own judgment, ignoring the vehicle agent V's words and actions.

[0115] As a result, if both the intimacy and the dependence are low, the vehicle driving assistance system 10 can evaluate that the driver not only has no attachment to the vehicle 1 or the vehicle agent V, but also does not willingly accept driving assistance via the vehicle agent V.

[0116] On the other hand, if both the intimacy and the dependence are high, the vehicle driving assistance system 10 can evaluate that the driver has a favorable impression and accepts not only the vehicle 1 and the vehicle agent V, but also the driving assistance via the vehicle agent V.

[0117] Therefore, the vehicle driving assistance system 10 can evaluate the driver's receptivity to driving assistance based on the driver's familiarity with the vehicle 1 and the driver's dependency on the vehicle 1. Therefore, the vehicle driving assistance system 10 can build a good psychological relationship between the driver and the vehicle 1, for example, based on the driver's receptivity to driving assistance.

[0118] The vehicle driving assistance system 10 also includes a mapping means (ECU 27) that associates the degree of dependency calculated by the dependency calculation means with the degree of intimacy calculated by the intimacy calculation means on a virtual two-dimensional coordinate plane formed by a first axis indicating the degree of dependency and a second axis indicating the degree of intimacy, and maps this as the driver's psychological state.

[0119] According to this configuration, for example, by outputting the driver's psychological state regarding the vehicle 1, which is mapped on a virtual two-dimensional coordinate plane, to a display means, the driver and vehicle 1 developers can be objectively informed of the driver's receptivity to driving assistance.

[0120] The driver state detection means (ECU 27) is configured to detect the driver's behavior and biological information (heart rate, amount of sweating, amount of exercise) as the driver's state. According to this configuration, the driver's response state to the behavior of the vehicle agent V can be accurately identified, and therefore the degree of dependency of the driver on the vehicle 1 can be calculated accurately.

[0121] The intimacy calculation means (ECU 27) may calculate the intimacy based on the rough driving index value, the driving frequency index value, and the driving interval index value. According to this configuration, the degree of familiarity of the driver with the vehicle 1 can be easily calculated based on the roughness of the driver's driving operation and the frequency of the driver's driving opportunities.

[0122] The vehicle driving assistance system 10 also includes speech and behavior data 28 that stores a plurality of pieces of speech and behavior information (lines and actions) that indicate the content of speech and behavior that the vehicle agent V is to perform. Furthermore, the vehicle driving assistance system 10 is equipped with a behavior information selection means (ECU 27) that selects behavior information (lines and actions) from the behavior data 28 that makes the intimacy calculated by the intimacy calculation means higher than the median value of the intimacy and brings the dependency calculated by the dependency calculation means closer to the median value of the dependency.

[0123] The vehicle driving assistance system 10 is provided with an agent control means (ECU 27) that controls the behavior of the vehicle agent V based on the behavior information (words and actions) selected by the behavior information selection means.

[0124] According to this configuration, the vehicle agent V can encourage the driver to accept the intervention of driving assistance via the vehicle agent V, while preventing the driver from becoming too dependent on the driving assistance of the vehicle 1. Therefore, the vehicle driving assistance system 10 can build a psychological relationship between the driver and the vehicle 1 via the vehicle agent V, for example.

[0125] In correspondence between the configuration of this invention and the above-mentioned embodiment, The history information of this invention corresponds to the behavior history information 291 and the driving history information 292 of the embodiment. Similarly, The history storage means corresponds to the driver data 29, The driver state detection means corresponds to an in-vehicle camera 24, an activity sensor 25, and a microphone 26. The intimacy calculation means, the dependency calculation means, the mapping means, the speech and behavior information selection means, and the agent control means correspond to the ECU 27; The biological information corresponds to heart rate, sweat rate, and exercise amount, The index value indicating the roughness of the driving operation corresponds to the roughness index value, The index value indicating the driving frequency corresponds to the driving frequency index value, The index value indicating the headway corresponds to the headway index value, The speech and behavior information corresponds to the lines and actions of the speech and behavior data 28, The speech and behavior information storage means corresponds to speech and behavior data 28, The present invention is not limited to the configurations of the above-described embodiments, and many other embodiments can be obtained.

[0126] For example, in the above embodiment, the vehicle agent V is a character displayed on the display unit 14, but the present invention is not limited to this and may be, for example, a robot placed on the instrument panel 2. Furthermore, the vehicle driving assistance system 10 is not limited to the configuration described above, and may include, for example, a camera, millimeter wave radar, or a sensor that detects the situation behind or to the sides of the vehicle. Furthermore, the vehicle driving assistance system 10 may include, for example, a navigation system, which assists the driver in driving operations based on the vehicle position and map data.

[0127] In addition, as one of the means for detecting the driver's state, an activity sensor 25 for detecting the driver's activity level is provided on the steering wheel 4, but this is not limited to this, and the activity sensor 25 may also be provided on a wearable terminal worn by the driver. Furthermore, the speech and behavior data 28 shown in FIG. 3 is an example, and is not limited to this, and may have a configuration in which appropriate speech and behavior information (lines and actions) is registered.

[0128] Furthermore, the processing operation of the driver psychology estimation process in Figure 7 is an example, and is not limited to the above-mentioned embodiment, and the driver's intimacy with the vehicle 1 and the driver's dependence on the vehicle 1 may be calculated based on appropriate conditions.

[0129] For example, the degree of intimacy of the driver with the vehicle 1 may be calculated by determining whether the driver behaves in a manner that is favorable to the vehicle 1. Alternatively, the calculation of the degree of intimacy based on the driving history information 292 and the behavior history information 291 may be combined with the calculation of the degree of intimacy based on whether the driver behaves in a manner that is favorable to the vehicle 1.

[0130] More specifically, for example, ECU 27 identifies whether the driver is operating in compliance with traffic rules based on the driving behavior of the vehicle and the situation ahead of the vehicle, and if the driver is operating in compliance with traffic rules, determines that the driver is behaving in a manner that is favorable for vehicle 1.

[0131] Alternatively, the ECU 27 determines whether the driver has responded favorably to the voice of the vehicle agent V based on the audio signal from the microphone 26, and if the driver has responded favorably, determines that the driver has behaved in a manner that is favorable to the vehicle 1.

[0132] Alternatively, the ECU 27 identifies whether the driver has nodded in response to the words and actions of the vehicle agent V based on video images from the in-vehicle camera 24, and if the driver has nodded, determines that the driver has behaved in a manner that is favorable to the vehicle 1; or identifies whether the driver has spoken to the vehicle agent V based on audio signals from the microphone 26, and if the driver has spoken, determines that the driver has behaved in a manner that is favorable to the vehicle 1.

[0133] Furthermore, for example, when a driving assistance system forcibly intervenes without any prior warning while the driver is driving, the degree of dependency of the driver on the vehicle 1 may be calculated based on the driver's behavior in response to the forcible intervention of the driving assistance system. Alternatively, in the driver psychology estimation process of FIG. 7, the driver psychology calculation means for calculating the driver psychology information 293 may be constructed using a machine learning model.

[0134] In this case, the machine learning model is constructed so that, as shown in Figure 9, which shows a schematic explanatory diagram of a driver psychology estimation means in another embodiment, behavior history information 291, driving history information 292, video images from the exterior camera 22 and the interior camera 24, and various signals from the millimeter-wave radar 23, activity sensor 25, and microphone 26 are used as input information, and the driver's degree of intimacy with the vehicle 1 and the driver's degree of dependence on the vehicle 1 are obtained as output information in response to the input information.

[0135] The machine learning model is constructed by repeatedly learning, as training data for input information, information indicating the driver's driving behavior that is appropriate for the situation in front of the vehicle, information indicating the driver's preferred driving behavior, and information indicating the driver's behavior that is appropriate for the vehicle agent V's behavior.

[0136] Also, although not mentioned in the above-described embodiment, in the driver psychology estimation process, the driver's dependence on the vehicle 1 may be calculated each time driving assistance via the vehicle agent V intervenes, or may be calculated at an appropriate timing when driving assistance is not intervening.

[0137] For example, while the driver is driving the vehicle 1, information indicating the behavior of the vehicle agent V and information indicating the behavior of the driver may be sequentially stored in the ECU 27, and the next time the driver drives the vehicle 1, the driver's dependence on the vehicle 1 may be calculated based on the stored information. Even with this configuration, the same effects as those of the above-described embodiment can be achieved.

[0138] Although not mentioned in the above embodiment, the driver psychological information 293 may be displayed on the display unit 14 by mapping it on a virtual two-dimensional coordinate plane with the horizontal axis representing the dependency level and the vertical axis representing the intimacy level.

[0139] Furthermore, although the speech and behavior information (words and actions) is selected according to the degree of intimacy and dependency of the driver on the vehicle 1 indicated by the driver psychological information 293, this is not limitative. For example, the ECU 27 may control the operation timing of the vehicle agent V according to the degree of intimacy and dependency of the driver on the vehicle 1 indicated by the driver psychological information 293. [Explanation of symbols]

[0140] 1...Vehicle 10...Vehicle driving assistance system 24...In-car camera 25...Activity sensor 26...Mike 27…ECU 28... Behavioral data 29...Driver data 291...Behavior history information 292...Driving history information V…Vehicle Agent

Claims

1. A vehicle driving assistance system that assists a driver of a vehicle with a driving operation via a vehicle agent that communicates with the driver, a history storage means for storing history information in which a history of driving operations by the driver is registered; a driver state detection means for detecting the state of the driver; an intimacy calculation means for calculating an intimacy of the driver with respect to the vehicle based on the history information; a dependency calculation means for calculating a dependency of the driver on the vehicle based on a response state of the driver to the behavior of the vehicle agent identified from the state of the driver. Vehicle driving assistance system.

2. a mapping means for mapping the degree of dependence calculated by the dependence calculation means and the degree of intimacy calculated by the intimacy calculation means as a psychological state of the driver on a virtual two-dimensional coordinate plane formed by a first axis indicating the degree of dependence and a second axis indicating the degree of intimacy; The vehicle driving assistance system according to claim 1 .

3. The driver condition detection means The driver's behavior and biological information are detected as the driver's state. The vehicle driving assistance system according to claim 1 .

4. The intimacy calculation means The degree of familiarity is calculated based on an index value indicating roughness of driving operation, an index value indicating driving frequency, and an index value indicating driving intervals. The vehicle driving assistance system according to claim 1 .

5. a speech and behavior information storage means for storing a plurality of speech and behavior information items indicating the content of speech and behavior to be performed by the vehicle agent; a speech and behavior information selection means for selecting, from the speech and behavior information storage means, speech and behavior information that makes the intimacy calculated by the intimacy calculation means higher than a median of the intimacy and makes the dependency calculated by the dependency calculation means closer to the median of the dependency; and an agent control means for controlling the behavior of the vehicle agent based on the behavior information selected by the behavior information selection means. The vehicle driving assistance system according to claim 1 .

Images