Vehicle operation support system

Abstract

To provide a vehicle operation support system 10 in which a driver can obtain an effect as if a passenger existed even when no man rides on a vehicle except the driver.SOLUTION: A vehicle operation support system 10 comprises: own vehicle condition detection means (ECU26) which detects an own vehicle condition based on traveling behavior of a vehicle 1 and a situation around the own vehicle; and voice guide means (speaker 15 and ECU 26) which notifies voice guide based on the own vehicle condition detected by the own vehicle condition detection means to a driver of the vehicle 1. The vehicle operation support system includes: a display part 14 which displays a character V disposed in a virtual three-dimensional space S; and character control means (ECU26) which controls behavior of the character V in the virtual three-dimensional space S. The character control means controls the behavior of the character V waiting in the virtual three-dimensional space S in a second behavior pattern that is a behavior mixing a movement where size and shape of the character change by 1 / f fluctuation with a movement where the character moves by a prescribed movement characteristic.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present invention relates to a vehicle driving assistance system that provides a driver with audio guidance based on, for example, the running behavior of a vehicle and the situation around the vehicle, thereby assisting the driver in driving operations. [Background technology]

[0002] In automobiles and other vehicles, known vehicle driving assistance systems that assist the driver in driving operations include, for example, advanced driving assistance systems that partially intervene in the driver's driving operations, and autonomous driving systems that take over driving operations from the driver under certain conditions and perform autonomous driving.

[0003] In such vehicle driving assistance systems, various types of guidance based on the situation around the vehicle, and guidance for transferring driving operations between the driver and the vehicle, are communicated not only through voice and text but also through a character that is an avatar of the vehicle.

[0004] For example, in Patent Document 1, a character that personifies a vehicle is displayed on a display unit inside the vehicle cabin, and the vehicle's status, identified based on the driver's driving operations and the vehicle's driving behavior, is communicated to the driver by changing the character's facial expression.

[0005] Incidentally, in vehicles such as automobiles, when there are passengers other than the driver, the driver's mentalizing, which involves paying attention to and understanding the psychological states of the driver and others, can lead to safer driving, resulting in a lower accident rate than when there are no passengers, a phenomenon known as the passenger effect.

[0006] However, since there is not necessarily always a passenger other than the driver in the vehicle, there is a problem in that it is difficult to obtain the passenger effect when there is no passenger present. Therefore, it is conceivable to have the character displayed on the display unit function as a passenger, but for example, in a character intended to convey the vehicle's status identified based on the driver's driving operations and the vehicle's driving behavior, as in Patent Document 1, there is a risk that the driver will not fully recognize the character as a passenger, and the passenger effect cannot be expected. [Prior art documents] [Patent documents]

[0007] [Patent Document 1] Japanese Patent Application Laid-Open No. 2003-72488 Summary of the Invention [Problem to be solved by the invention]

[0008] In view of the above-mentioned problems, an object of the present invention is to provide a vehicle driving assistance system that can provide a passenger effect even when no passenger other than the driver is in the vehicle. [Means for solving the problem]

[0009] The present invention is a vehicle driving assistance system comprising a host vehicle status detection means for detecting the host vehicle status based on the vehicle's driving behavior and the status around the host vehicle, and a voice guidance means for providing voice guidance to the driver of the vehicle based on the host vehicle status detected by the host vehicle status detection means, and is characterized in that the system comprises a display unit for displaying a character arranged in a three-dimensional virtual space, and a character control means for controlling the movement of the character in the three-dimensional virtual space, and the character control means is configured to control the movement of the character waiting in the three-dimensional virtual space in a first movement mode that combines movement in which the size and shape change with 1 / f fluctuations with movement with predetermined movement characteristics.

[0010] The character refers to an object of a three-dimensional shape or a two-dimensional image displayed on the display unit, an avatar that is an incarnation of the vehicle, or an anthropomorphic character. The 1 / f fluctuation is a fluctuation in which the spectral density is inversely proportional to the frequency f, where f is a finite range of f>0.

[0011] The predetermined movement characteristics include, for example, one or more of the following: goal directionality, which means moving in a straight line towards a target point; finality, which means moving along a different path each time when moving towards a target point; self-propulsion, which means moving without the influence of an external force; and violation of physical laws, which means moving in a way that ignores the laws of physics.

[0012] According to this invention, a first operating mode that combines movement in which size and shape change with 1 / f fluctuation with movement with predetermined movement characteristics allows a character waiting in a three-dimensional virtual space to move with initiative.

[0013] Therefore, the vehicle driving assistance system can stimulate the driver's animacy perception with lifelike character movements compared to characters that stand still and wait in a three-dimensional virtual space.

[0014] Furthermore, since the character's movements are different from those associated with voice guidance regarding driving operations, the vehicle driving assistance system can allow the driver to recognize the character's movements as the character's own actions.

[0015] This allows the vehicle driving assistance system to improve the driver's mentalizing, as the character waiting in the three-dimensional virtual space becomes more easily recognized by the driver as a passenger in the vehicle. Therefore, the vehicle driving assistance system can obtain the passenger effect even when there is no passenger other than the driver on board.

[0016] As an aspect of this invention, the character control means may be configured to include a gaze detection means for detecting gaze movement of the driver, and when the gaze detection means detects gaze movement of the driver, to control the movement of the character in a second movement mode which is a combination of a movement in which size and shape change with 1 / f fluctuation and a movement in which the character moves in accordance with the gaze movement of the driver.

[0017] With this configuration, the character moves in accordance with the driver's line of sight, which enhances the lifelikeness of the character displayed on the display unit and allows the driver to recognize the character's subjectivity.

[0018] This allows the vehicle driving assistance system to make the driver recognize the character in the three-dimensional virtual space as a passenger, thereby improving the driver's mentalizing and improving the passenger effect.

[0019] In another aspect of the present invention, the voice guidance means may be configured to issue a voice guidance warning to the driver when the vehicle situation requires a warning in a direction other than the vehicle's direction of travel, and the character control means may be configured to control the movement of the character in a third movement mode, which is a combination of a movement in which the size and shape change with 1 / f fluctuations and a movement in which the character moves in a direction other than the direction of travel that requires attention when a warning to the driver is issued by voice guidance.

[0020] According to this configuration, when a warning to the driver is given by voice guidance, the character moves in the third operating mode, so that the character displayed on the display unit can move in a lifelike manner.

[0021] Therefore, the vehicle driving assistance system can make the character behave in a way that tells the driver in a direction where passengers need to be careful, by making the character move like a life form when operating in the third operating mode and by providing audio guidance to alert the driver.

[0022] This allows the vehicle driving assistance system to allow the driver to recognize the character in the three-dimensional virtual space as a passenger, improving the driver's mentalizing and assisting the driver's driving operation to ensure the driver's safety.

[0023] In another aspect of the present invention, the character control means may be configured to control the movement of the character in a fourth movement mode, which is a combination of a movement in which the size and shape change with 1 / f fluctuation and a movement in which the character faces the direction of travel of the vehicle, when the vehicle situation detected by the vehicle situation detection means is that the vehicle is moving, and to control the movement of the character in a fifth movement mode, which is a combination of a movement in which the size and shape change with 1 / f fluctuation and a movement in which the character faces the driver directly, when the voice guidance is announced by the voice guidance means.

[0024] According to this configuration, when audio guidance is announced, the character operating in the fourth operating mode moves in the fifth operating mode, so that the character displayed on the display unit can move in a lifelike manner.

[0025] Therefore, the vehicle driving assistance system can make the character behave as if a passenger is conveying the contents of the voice guidance to the driver by the character's lifelike movements as it transitions from the fourth operating mode to the fifth operating mode and by providing voice guidance to the driver.

[0026] This allows the vehicle driving assistance system to allow the driver to recognize the character in the three-dimensional virtual space as a passenger, improving the driver's mentalizing and assisting the driver's driving operation to ensure the driver's safety.

[0027] In another aspect of the present invention, the vehicle may include a driver state detection means for detecting the state of the driver while driving, and the character control means may be configured to suppress the movement of the character in the second operating mode when the state of the driver detected by the driver state detection means is busy.

[0028] The driver being busy means that the driver's operation amount, operation frequency, and eye movement amount are greater than when driving straight ahead, and the driver is busy with driving operations. To suppress the movement of the character means, for example, reducing the scaling rate and / or the amount of deformation, or slowing down the movement speed.

[0029] With this configuration, in situations where the driver is required to perform driving operations according to the situation around the vehicle or to check the area around the vehicle, the driver's attention is directed to the movement of the character, preventing a decrease in the driver's attention to the area around the vehicle. This allows the vehicle driving assistance system to improve the driver's mentalizing without compromising driver safety. [Effects of the Invention]

[0030] The present invention can provide a vehicle driving assistance system that can provide the passenger effect even when there is no passenger other than the driver on board. [Brief explanation of the drawings]

[0031] [Figure 1] FIG. 2 is a front view showing the exterior of the vehicle driving assistance system as seen from inside the vehicle cabin. [Figure 2] FIG. 1 is a block diagram showing the internal configuration of a vehicle driving assistance system. [Figure 3] FIG. 3 is an explanatory diagram illustrating an outline of a character displayed on a display unit. [Figure 4] FIG. 1 is an explanatory diagram illustrating an outline of the passenger effect in the present invention. [Figure 5] 10 is a flowchart showing the processing operation of an ECU related to the movement of a character. [Figure 6] 10 is a flowchart showing the processing operation of detailed operation processing. [Figure 7] FIG. 10 is an explanatory diagram illustrating the movement of a character. [Figure 8]FIG. 10 is an explanatory diagram illustrating the movement of a character. [Figure 9] FIG. 10 is an explanatory diagram illustrating the movement of a character. [Figure 10] FIG. 10 is an explanatory diagram illustrating the movement of a character. [Figure 11] FIG. 10 is an explanatory diagram illustrating the movement of a character. DETAILED DESCRIPTION OF THE INVENTION

[0032] An embodiment of the present invention will be described below with reference to the drawings. In this embodiment, a vehicle driving assistance system 10 that provides a driver with voice guidance based on the driving behavior of a vehicle 1 and the situation around the vehicle, thereby assisting the driver in driving operations, will be described with reference to FIGS. 1 to 4. FIG.

[0033] FIG. 1 shows a front view of the vehicle driving assistance system 10 as seen from inside the cabin of the vehicle 1, FIG. 2 shows a block diagram of the vehicle driving assistance system 10, FIG. 3 shows an explanatory diagram outlining the character V displayed on the display unit 14, and FIG. 4 shows an explanatory diagram outlining the passenger effect in the present invention.

[0034] First, as shown in FIG. 1, the interior of a vehicle 1 is provided with an instrument panel 2 extending in the vehicle width direction (reference numerals omitted), and a center console 3 extending from approximately the center of the instrument panel 2 in the vehicle width direction toward the rear of the vehicle. Furthermore, the vehicle 1 is provided with a steering wheel 4 in the cabin, which is disposed in front of a driver's seat (not shown) and receives steering operations from a passenger seated in the driver's seat.

[0035] As shown in FIG. 2, the vehicle driving assistance system 10 in such a vehicle 1 includes an engine 11, a brake device 12, and a steering device 13 arranged outside the vehicle cabin, a display unit 14 provided at approximately the center of the instrument panel 2 in the vehicle width direction, and a speaker 15 arranged inside the vehicle cabin.

[0036] Furthermore, as shown in FIG. 2, the vehicle driving assistance system 10 is equipped with an accelerator opening sensor 16, a brake pedal stroke sensor 17, a vehicle speed sensor 18, a steering angle sensor 19, an acceleration sensor 20, and a yaw rate sensor 21 for detecting the driving behavior of the vehicle.

[0037] In addition, as shown in FIG. 2, the vehicle driving assistance system 10 includes an exterior camera 22 and a millimeter wave radar 23 for acquiring information about the area in front of the vehicle, an interior camera 24 and a microphone 25 for detecting the driver's condition, and an electronic control unit (ECU) for controlling the operation of each of the above-mentioned components. The engine is equipped with an ECU (Electric Control Unit) 26.

[0038] More specifically, the engine 11 is made up of an engine body (details of which are omitted from the drawings) and a fuel injection device that supplies fuel to the engine body. The engine output and engine speed of the engine 11 are controlled based on an output signal from the ECU 26.

[0039] The braking device 12 is composed of a brake caliper, a brake actuator, etc., which are not shown in detail. The braking device 12 is controlled to generate a braking force based on an output signal from the ECU 26.

[0040] The steering device 13 is composed of the above-mentioned steering wheel 4, an electric motor (not shown) that assists the driver in steering, etc. The steering device 13 is controlled to change the traveling direction of the vehicle based on an output signal from the ECU 26.

[0041] The display unit 14 is configured, for example, with a liquid crystal display, and has a function of displaying various information to a passenger, such as a driver, seated in the driver's seat, based on control signals from the ECU 26. For example, as shown in Fig. 1, the display unit 14 displays a character V that communicates with the driver as an incarnation of the vehicle 1.

[0042] The speaker 15 is disposed in the vehicle cabin and has a function of outputting a warning sound or voice based on a notification signal from the ECU 26 to notify the occupants. For example, the speaker 15 outputs voice information associated with the action of the character V.

[0043] The accelerator position sensor 16 is a sensor that is integrally provided on an accelerator pedal operated by the driver. The accelerator position sensor 16 has a function of detecting the displacement angle of the accelerator pedal caused by the driver's operation and a function of outputting the detected displacement angle to the ECU 26 as an accelerator position signal.

[0044] The brake pedal stroke sensor 17 is a sensor that is integrally provided on the brake pedal operated by the driver. The brake pedal stroke sensor 17 has a function of detecting the displacement angle of the brake pedal operated by the driver and a function of outputting the detected displacement angle to the ECU 26 as a brake signal.

[0045] The vehicle speed sensor 18 also has a function of detecting the rotation speed of a drive shaft connected to the wheels, for example, and a function of outputting the detected rotation speed to the ECU 26 as a rotation speed signal. The steering angle sensor 19 has a function of detecting the steering angle of the steering wheel 4 operated by the driver, and a function of outputting the detected steering angle to the ECU 26 as a steering angle signal.

[0046] In addition, the acceleration sensor 20 is arranged, for example, inside the center console 3, and has the function of detecting the longitudinal acceleration along the fore-and-aft direction of the vehicle 1 and the lateral acceleration along the width direction of the vehicle 1, and the function of outputting the detected longitudinal acceleration and lateral acceleration to the ECU 26 as acceleration signals.

[0047] The yaw rate sensor 21 is disposed, for example, inside the center console 3, and has a function of detecting the yaw rate of the vehicle 1 and a function of outputting the detected yaw rate to the ECU 26 as a yaw rate signal. The exterior camera 22 is disposed, for example, above the front windshield on the passenger compartment side, and has the function of capturing images in front of the vehicle and the function of outputting the captured video images to the ECU 26 .

[0048] The millimeter wave radar 23 is disposed, for example, in the front bumper, and has the functions of emitting millimeter waves as irradiated waves in front of the vehicle, receiving reflected waves reflected by preceding vehicles, parked vehicles, pedestrians, etc., and outputting information obtained from the irradiated waves and reflected waves to the ECU 26 as a radar signal.

[0049] 1, the in-vehicle camera 24 is mounted on the instrument panel 2 or a front pillar (reference numeral omitted) facing the driver seated in the driver's seat. The in-vehicle camera 24 has a function of capturing an image of the driver seated in the driver's seat, focusing on the head, and a function of outputting the captured video to the ECU 26.

[0050] In addition, the microphone 25 is arranged on the top of the column cover (reference numeral omitted), for example, as shown in FIG. 1, and has the function of detecting the voice emitted by the driver seated in the driver's seat and the function of outputting the detected voice to the ECU 26 as an audio signal.

[0051] The ECU 26 is made up of hardware components such as a CPU, a memory, and a storage unit for storing various types of information, and software components such as programs and data. This ECU 26 has the function of acquiring various signals output by the above-mentioned sensors, the exterior camera 22, the millimeter-wave radar 23, the interior camera 24, and the microphone 25, and the function of performing various information processing based on the acquired signals. Furthermore, the ECU 26 has a function of controlling the operation of the display unit 14 and the speaker 15 , and a function of controlling the operation of the engine 11 , the brake device 12 and the steering device 13 .

[0052] More specifically, the ECU 26 realizes the function of detecting the driving behavior of the vehicle 1 in cooperation with the accelerator opening sensor 16, the brake pedal stroke sensor 17, the vehicle speed sensor 18, the steering angle sensor 19, the acceleration sensor 20, and the yaw rate sensor 21.

[0053] Furthermore, the ECU 26 cooperates with the accelerator opening sensor 16, the brake pedal stroke sensor 17, and the steering angle sensor 19 to realize a function of detecting driving operations by the driver.

[0054] In addition, the ECU 26 has the function of identifying the situation in front of the vehicle in cooperation with the exterior camera 22 and millimeter-wave radar 23, and the function of identifying the driver seated in the driver's seat and identifying the driver's condition in cooperation with the interior camera 24 and microphone 25.

[0055] Such an ECU 26 is configured to warn the driver and notify them of driving assistance interventions based on the driving behavior and the situation ahead of the vehicle through the words and actions of a character V displayed on the display unit 14 and voice guidance via a speaker 15, and to control the operation of the engine 11, braking device 12, and steering device 13 to provide driving assistance that assists the driver in driving operations.

[0056] For this reason, the ECU 26 stores speech and behavior data 27 in which the speech and behavior of the character V and the contents of the voice guidance to be notified to the driver are registered in association with flags indicating the driving behavior and the situation ahead of the vehicle.

[0057] The character V is an alter ego of the vehicle 1 with simulated emotions set thereon, and is controlled by the ECU 26 to behave differently depending on the running behavior and the situation ahead of the vehicle. Specifically, as shown in FIG. 3, the character V is a three-dimensional model placed in a three-dimensional virtual space S, and is formed in the shape of a teardrop lying on its side.

[0058] In other words, the character V is formed in a shape in which a substantially spherical body and a tapered tail extending from the substantially spherical body are integrally formed. The roughly spherical part of the teardrop-shaped character V lying on its side is the front side of the character V, and the tail part extending from the roughly spherical part is the rear side of the character V.

[0059] Furthermore, the character V is controlled by the ECU 26 to embody lifelike movements, combining movements in which the size and shape change with 1 / f fluctuations, with movements based on the driving behavior, the situation in front of the vehicle, or the driver's state.

[0060] By making the character V move like a living thing, the vehicle driving assistance system 10 of this embodiment makes the driver recognize the character V as a passenger, thereby achieving the passenger effect even when there are no other passengers on board other than the driver.

[0061] In more detail, the applicant has conducted various experiments and verifications and found that when there are passengers other than the driver in the vehicle, the driver recognizes the passengers as "passengers" by perceiving the passengers' agency and ability to express emotions.

[0062] Furthermore, according to the applicant, the passenger's recognition activates the driver's mPFC (medial prefrontal cortex), which is thought to improve the driver's mentalizing and result in the passenger effect.

[0063] Therefore, the present invention invented by the applicant allows the driver to recognize character V as a passenger by controlling the movement of character V displayed on display unit 14 so that the driver perceives character V as having agency and the ability to express emotions, as shown in Figure 4.

[0064] For example, when a traffic jam ahead of the vehicle clears and the vehicle speed begins to increase, the present invention controls the movement of character V to make it appear as if the character is happy that the traffic jam has cleared, as input information to the driver, thereby making the driver recognize character V as a passenger.

[0065] As a result, the present invention activates the driver's mPFC (medial prefrontal cortex) in the same way as when there is a passenger other than the driver in the vehicle, thereby improving the driver's mentalization. In this way, the present invention indirectly inputs changes in the surrounding situation of the vehicle 1 to the driver through the movement of the character V, and also encourages the driver to drive safely and independently, thereby achieving the passenger effect.

[0066] Next, the processing operation of the ECU 26 in the vehicle driving assistance system 10 configured as described above will be described with reference to FIGS. 5 shows a flowchart of the processing operation of the ECU 26 regarding the movement of the character V, FIG. 6 shows a flowchart of the detailed movement processing, and FIGS. 7 to 11 show explanatory diagrams for explaining the movement of the character V. FIG. 7 to 11, the arrow RH indicates the right direction in the three-dimensional virtual space S as seen by the driver, and the arrow LH indicates the left direction in the three-dimensional virtual space S as seen by the driver.

[0067] First, when power is supplied to the ECU 26 from the battery, the ECU 26 executes a predetermined program stored therein and starts a processing operation for controlling the behavior and speech of the character V. The ECU 26, which has started the processing operation for controlling the behavior of the character V, identifies the driver seated in the driver's seat based on the video image from the in-vehicle camera 24 and the audio signal from the microphone 25, as shown in FIG. 5 (step S101).

[0068] After identifying the driver, the ECU 26 displays the character V placed in the three-dimensional virtual space S on the display unit 14 inside the vehicle cabin (step S102), and then causes the character V in the three-dimensional virtual space S to move in a first movement pattern for a predetermined period of time (step S103).

[0069] Specifically, the ECU 26 controls the movement of the character V in a first movement pattern in which the substantially spherical portion of the character V faces the driver and the character V repeatedly moves up and down in the three-dimensional virtual space S in a hopping manner.

[0070] Furthermore, the ECU 26 displays a message welcoming the driver seated in the driver's seat on the display unit 14, and outputs the message welcoming the driver to the speaker 15 as a line of the character V associated with the action of the character V.

[0071] For example, as shown in FIG. 7, the ECU 26 outputs to the display unit 14 a character V moving in a first movement pattern such as hopping around in the three-dimensional virtual space S, and a message saying "Hello! Let's go for a drive together!"

[0072] Thereafter, as shown in FIG. 5, the ECU 26 determines whether or not a predetermined time has elapsed since the character V started moving in the first movement pattern (step S104). If the time elapsed since the start of operation in the first operation pattern has not yet reached the predetermined time (step S104: No), the ECU 26 puts the process on hold until the predetermined time has elapsed since the start of operation in the first operation pattern.

[0073] On the other hand, if the elapsed time since the start of movement in the first movement pattern has exceeded a predetermined time (step S104: Yes), ECU 26 transitions the movement of character V in the three-dimensional virtual space S to a second movement pattern, which is a combination of movement in which the size and shape change with 1 / f fluctuation and movement with predetermined movement characteristics (step S105).

[0074] As a result, ECU 26 realizes the movement of character V floating autonomously and independently in the three-dimensional virtual space S, just like a dog or cat moving freely within a specified space, as a standby behavior of character V, which waits until it detects a change in the vehicle's situation or the driver's actions.

[0075] More specifically, ECU26 enlarges or reduces the size of character V at a predetermined enlargement or reduction ratio with 1 / f fluctuation, and moves character V with predetermined movement characteristics while deforming the shape of character V by a predetermined deformation amount with 1 / f fluctuation.

[0076] 1 / f fluctuation is an unpredictable rhythm that can be seen, for example, in the interval between human heartbeats or the flickering of a flame, and when felt by humans through the five senses, it has the effect of making us feel comfortable, relaxed, or energized. This 1 / f fluctuation is defined as the spectral density being inversely proportional to the frequency f, where f>0 has a finite range, for example, 2 Hz.

[0077] The predetermined movement characteristics may include one or more of goal-directed, equifinal, self-propelled, and physics-violating. Specifically, as shown in Figure 8(a), when the position of character V shown by the dashed double-dashed line is the target point, the target directionality is a characteristic in which character V moves in a straight line toward the target point as indicated by the arrow in the figure.

[0078] Equal finality is a characteristic in which, as shown in Figure 8(b), when character V moves toward a destination point, for example, the position of character V shown by a dotted line, character V moves along a different path each time it moves, as indicated by the arrows in the figure.

[0079] Self-propulsion is a property that allows the character V to move without any external force acting on it, as shown in FIG. Violation of physical laws refers to the characteristic of moving in a manner that ignores the laws of physics, such as moving against gravity, as shown in FIG.

[0080] In step S105 of FIG. 5, when the movement of character V is shifted to the second movement pattern, ECU 26 determines whether the current vehicle speed is greater than 0 km / h based on the rotation speed signal from vehicle speed sensor 18, as shown in FIG. 5 (step S106).

[0081] If the vehicle 1 is stopped and the current vehicle speed is 0 km / h (step S106: No), the ECU 26 causes the character V to move in the second movement pattern and waits for the process to finish until the current vehicle speed exceeds 0 km / h.

[0082] On the other hand, if the current vehicle speed is greater than 0 km / h (step S106: Yes), ECU 26 begins to transition the movement of character V operating in the second movement pattern to a third movement pattern, which is a movement that combines movement in which the size and shape change with 1 / f fluctuation and movement toward the direction of travel of vehicle 1 (step S107).

[0083] As a result, the ECU 26 causes the character V, which moves autonomously in the second movement pattern, to turn toward the traveling direction of the vehicle 1, and realizes a movement in which the character V faces the traveling direction of the vehicle 1 and floats.

[0084] Specifically, ECU26 changes the orientation of character V, which moves autonomously in the second movement pattern, so that the approximately spherical portion of character V faces the back in the depth direction of three-dimensional virtual space S and the tail portion of character V faces the front in the depth direction of three-dimensional virtual space S.

[0085] At this time, the ECU 26 enlarges or reduces the size of the character V at a predetermined enlargement or reduction rate with 1 / f fluctuation, and changes the orientation of the character V while deforming the shape of the character V by a predetermined deformation amount with 1 / f fluctuation.

[0086] Furthermore, the ECU 26 enlarges or reduces the size of the character V whose orientation has been changed with 1 / f fluctuation, and causes the character V to float in the three-dimensional virtual space S while deforming the shape of the character V with 1 / f fluctuation.

[0087] Then, while transitioning the character V to the third movement pattern, the ECU 26 determines whether the driver is currently busy with driving operations and has no time to pay attention to the character V on the display unit 14 (step S108).

[0088] At this time, the ECU 26 detects the amount and frequency of operation of the accelerator, brake, steering, etc. based on various signals from the accelerator opening sensor 16, brake pedal stroke sensor 17, and steering angle sensor 19.

[0089] Furthermore, the ECU 26 detects the frequency of the driver's line of sight movement based on the video image from the in-vehicle camera 24, and if the detected amount of operation, operation frequency and line of sight movement frequency are greater than those during straight-ahead driving, it determines that the driver is busy with driving operations.

[0090] If the driver is not currently busy with driving operations (step S108: No), ECU 26 controls the movement of character V in the third movement pattern facing the traveling direction of vehicle 1, and proceeds to step S1110 described later.

[0091] On the other hand, if the driver is currently busy with driving operations (step S108: Yes), the ECU 26 starts a movement suppression process to moderate the movement of the character V in the third movement pattern (step S109).

[0092] Specifically, the ECU 26 reduces the scaling rate and the deformation amount of the character V compared to when the character V is not busy operating the vehicle, thereby reducing the movement of the character V in the third movement pattern. Thereafter, the ECU 26 starts a detailed action process for controlling the movement of the character V in more detail while the vehicle 1 is traveling (step S110). When the detailed operation process starts, the ECU 26 determines whether or not to notify the driver of voice guidance based on the vehicle situation, as shown in FIG. 6 (step S121).

[0093] At this time, ECU26 determines that if the current vehicle situation requires avoidance by driver operation, if the driver needs to be alerted, or if driving assistance that partially intervenes in the driver's driving operation is to be started or ended, it will provide audio guidance to the driver based on the vehicle situation.

[0094] If it is not necessary to notify the driver of voice guidance (step S121: No), the ECU 26 determines whether the driver's line of sight has moved in a direction other than the traveling direction of the vehicle 1 based on the moving image from the in-vehicle camera 24 (step S122).

[0095] If the driver's line of sight moves in a direction other than the direction of travel of the vehicle 1 (step S122: Yes), the ECU 26 begins to transition the movement of the character V operating in the third movement pattern to a fourth movement pattern, which is a combination of movement in which the size and shape change with 1 / f fluctuation and movement that follows the movement of the driver's line of sight (step S123).

[0096] Specifically, ECU 26 regards the direction in which the driver's line of sight moves as the line of sight movement direction, and moves character V, which is facing in the direction of travel of vehicle 1, toward the direction in which the driver's line of sight moves, while enlarging or reducing the size of character V at a predetermined enlargement / reduction rate and with 1 / f fluctuation, and deforming the shape of character V by a predetermined deformation amount and with 1 / f fluctuation. At this time, the ECU 26 moves the character V while changing the orientation of the character V so that the substantially spherical portion of the character V faces the direction in which the driver's line of sight is moving.

[0097] For example, if the driver's line of sight in the video image 24a acquired from the in-vehicle camera 24 moves to the right of the vehicle, the ECU 26 moves the character V to the right in the three-dimensional virtual space S as seen by the driver, as shown in Figure 9(a). On the other hand, if the driver's line of sight in the video image 24a acquired from the in-vehicle camera 24 moves to the left of the vehicle, the ECU 26 moves the character V to the left in the three-dimensional virtual space S as seen by the driver, as shown in Figure 9(b).

[0098] Furthermore, as shown in FIG. 6, while transitioning the character V to the fourth movement pattern, the ECU 26 determines whether the current state of the driver is such that the driver is busy with driving operations and has no time to pay attention to the character V (step S124).

[0099] If the driver is not currently busy with driving (step S124: No), the ECU 26 controls the movement of the character V in the fourth movement pattern of facing the direction of the driver's line of sight for a predetermined time, while terminating the detailed movement process, and proceeds to step S106 in Fig. 5. Thereafter, the ECU 26 repeats the processes from step S106 to step S110 until the supply of power is interrupted.

[0100] On the other hand, if the driver is currently busy with driving operations (step S124: Yes), the ECU 26 starts a movement suppression process to moderate the movement of the character V in the fourth movement pattern and move it for a predetermined time (step S125).

[0101] Specifically, the ECU 26 reduces the scaling rate and the deformation amount of the character V compared to when the character V is not busy operating the vehicle, thereby reducing the movement of the character V in the fourth movement pattern. After moving the character V in the fourth movement pattern for a predetermined time, the ECU 26 ends the detailed movement process, proceeds to step S106 in FIG. 5, and then repeats the processes from step S106 to step S110 until the power supply is cut off.

[0102] Also, in step S122, if the driver's line of sight is directed in the direction of travel of the vehicle 1 (step S122: No), as shown in FIG. 6, the ECU 26 ends the detailed operation processing, advances the processing to step S106 in FIG. 5, and then repeats the processing from step S106 to step S110 until the power supply is interrupted.

[0103] In addition, in step S121, when audio guidance based on the vehicle situation is to be notified to the driver (step S121: Yes), as shown in FIG. 6, ECU 26 determines whether the audio guidance is a warning to the driver in a direction other than the traveling direction (step S126).

[0104] If the voice guidance is a warning in a direction other than the direction of travel (step S126: Yes), ECU26 transitions the movement of character V operating in the third movement pattern to a fifth movement pattern, which is a combination of movement in which the size and shape change with 1 / f fluctuation and movement in the direction warned about by the voice guidance (step S127). As a result, the ECU 26 turns the character V, which is facing the traveling direction of the vehicle 1, toward the direction of attention being called by the voice guidance, thereby realizing a movement of the character V that makes the driver aware of the direction of attention being called by the voice guidance.

[0105] Specifically, ECU 26 enlarges or reduces the size of character V at a predetermined enlargement or reduction rate with 1 / f fluctuation, and deforms the shape of character V by a predetermined deformation amount with 1 / f fluctuation, thereby moving character V, which is facing in the direction of travel of vehicle 1, in the direction alerted by the voice guidance. At this time, the ECU 26 moves the character V while changing the orientation of the character V so that the substantially spherical portion of the character V faces the direction in which the attention is called by the voice guidance.

[0106] Furthermore, the ECU 26 displays a message corresponding to the voice guidance on the display unit 14, and outputs the voice guidance to the speaker 15 as the lines of the character V associated with the character V's actions.

[0107] For example, if the direction warned by the voice guidance is the blind spot on the left side of the vehicle, the ECU 26 moves the character V, which is facing in the direction of travel of the vehicle 1, to the left in the three-dimensional virtual space S as seen by the driver, as shown in Figure 10(a), and outputs a warning message to the display unit 14 saying, "Watch out for the blind spot on the left side!"

[0108] Alternatively, if the direction warned by the voice guidance is the blind spot on the right side of the vehicle, the ECU 26 moves the character V, which is facing in the direction of travel of the vehicle 1, to the right in the three-dimensional virtual space S as seen by the driver, as shown in Figure 10(b), and outputs a warning message to the display unit 14 saying, "Watch out for the blind spot on the right side!"

[0109] After moving the character V in the fifth movement pattern for a predetermined time, the ECU 26 ends the detailed movement process, proceeds to step S106 in FIG. 5, and then repeats the processes from step S106 to step S110 until the power supply is cut off.

[0110] On the other hand, in step S126, if the voice guidance is not a warning in a direction other than the direction of travel (step S126: No), ECU 26 transitions the movement of character V operating in the third movement pattern to a sixth movement pattern, which is a movement that combines movement in which the size and shape change with 1 / f fluctuation and movement that faces the driver directly (step S128). As a result, the ECU 26 makes the character V, which faces in the traveling direction of the vehicle 1, turn towards the driver, thereby realizing a movement of the character V that appeals to the driver.

[0111] Specifically, the ECU 26 enlarges or reduces the size of the character V at a predetermined enlargement or reduction ratio with 1 / f fluctuation, and deforms the shape of the character V by a predetermined deformation amount with 1 / f fluctuation, thereby turning the character V, which is facing in the direction of travel of the vehicle 1, toward the driver.

[0112] At this time, the ECU 26 changes the orientation of the character V so that the approximately spherical part of the character V faces the front side in the depth direction in the three-dimensional virtual space S, and moves the character V toward the front side in the depth direction in the three-dimensional virtual space S.

[0113] Furthermore, the ECU 26 displays a message corresponding to the voice guidance on the display unit 14, and outputs the voice guidance to the speaker 15 as the lines of the character V associated with the character V's actions.

[0114] For example, if there is not enough distance between the vehicle and the vehicle in front, the ECU 26 causes the character V, which is facing in the direction of travel of the vehicle 1 as shown in Figure 11(a), to face the driver directly as shown in Figure 11(b), and outputs a message to the display unit 14 saying, "Please keep a safe distance between your vehicles."

[0115] After moving the character V in the sixth movement pattern for a predetermined time, the ECU 26 ends the detailed movement process, proceeds to step S106 in FIG. 5, and then repeats the processes from step S106 to step S110 until the power supply is cut off.

[0116] In this way, the character V moves in the three-dimensional virtual space S in a movement pattern based on the vehicle situation and the driver's actions, so that the vehicle driving assistance system 10 gives the character V the appearance of an autonomously moving living thing, making it easier for the driver to recognize the character V as a passenger.

[0117] As described above, the vehicle driving assistance system 10 of this embodiment includes a vehicle status detection means (ECU 26) that detects the vehicle status based on the driving behavior of the vehicle 1 and the status around the vehicle, and a voice guidance means (speaker 15 and ECU 26) that provides voice guidance to the driver of the vehicle 1 based on the vehicle status detected by the vehicle status detection means.

[0118] Furthermore, the vehicle driving assistance system 10 includes a display unit 14 that displays a character V placed in the three-dimensional virtual space S, and a character control means (ECU 26) that controls the behavior of the character V in the three-dimensional virtual space S.

[0119] The character control means is configured to control the movement of the character V waiting in the three-dimensional virtual space S using a second movement pattern, which is a movement that combines movement in which the size and shape change with 1 / f fluctuation with movement with predetermined movement characteristics.

[0120] According to this configuration, the second movement pattern, which combines movement in which the size and shape change with 1 / f fluctuation with movement with predetermined movement characteristics, allows the character V waiting in the three-dimensional virtual space S to move with initiative.

[0121] Therefore, the vehicle driving assistance system 10 can stimulate the driver's animacy perception with the movement of the character V that is more lifelike than the character V that stands still and waits in the three-dimensional virtual space S.

[0122] Furthermore, since the actions are different from the actions of character V associated with voice guidance regarding driving operations, the vehicle driving assistance system 10 can make the driver recognize the actions of character V as the actor of character V.

[0123] As a result, the vehicle driving assistance system 10 makes it easier for the driver to recognize the character V waiting in the three-dimensional virtual space S as a passenger in the vehicle 1, thereby improving the driver's mentalizing. Therefore, the vehicle driving assistance system 10 can obtain the passenger effect even when there is no passenger other than the driver on board.

[0124] The vehicle driving assistance system 10 also includes a line-of-sight detection means (in-vehicle camera 24 and ECU 26) for detecting the driver's line-of-sight movement. When the gaze detection means detects the driver's gaze movement, the character control means (ECU 26) controls the movement of character V using a fourth movement pattern, which is a combination of a movement in which the size and shape change with 1 / f fluctuation and a movement in which the character moves in accordance with the driver's gaze movement.

[0125] According to this configuration, the character V moves in accordance with the driver's line of sight, which enhances the lifelikeness of the character V displayed on the display unit 14 and allows the driver to recognize the subjectivity of the character V.

[0126] As a result, the vehicle driving assistance system 10 allows the driver to recognize the character V in the three-dimensional virtual space S as a passenger, thereby improving the driver's mentalizing and improving the passenger effect.

[0127] Furthermore, the voice guidance means (speaker 15 and ECU 26) is configured to issue a voice guidance to call the driver's attention when the situation of the vehicle 1 requires attention in a direction other than the traveling direction of the vehicle 1.

[0128] The character control means (ECU 26) is configured to control the movement of character V using a fifth movement pattern, which is a combination of a movement in which the size and shape change with 1 / f fluctuations and a movement in a direction other than the direction of travel that requires caution when a warning to the driver is issued by voice guidance.

[0129] According to this configuration, when the driver is alerted by voice guidance, the character V moves in the fifth movement pattern, so that the character V displayed on the display unit 14 can move in a lifelike manner.

[0130] Therefore, the vehicle driving assistance system 10 can make the character V behave in a way that tells the driver which direction the passenger needs to be careful in by making the character V move like a living thing in the fifth movement pattern and by providing audio guidance to alert the driver.

[0131] As a result, the vehicle driving assistance system 10 allows the driver to recognize the character V in the three-dimensional virtual space S as a passenger, thereby improving the driver's mentalization and assisting the driver in his driving operations to ensure the driver's safety.

[0132] In addition, when the vehicle status detected by the vehicle status detection means is that the vehicle is moving, the character control means (ECU 26) is configured to control the movement of character V using a third movement pattern, which is a combination of a movement in which the size and shape change with 1 / f fluctuations and a movement facing the direction of travel of vehicle 1.

[0133] Furthermore, when voice guidance is announced by the voice guidance means, the character control means (ECU 26) is configured to control the movement of character V using a sixth movement pattern, which is a combination of movement in which the size and shape change with 1 / f fluctuation and movement facing the driver directly.

[0134] According to this configuration, when audio guidance is announced, the character V, which moves in the third movement pattern, moves in the sixth movement pattern, so that the character V displayed on the display unit 14 can move in a lifelike manner.

[0135] Therefore, the vehicle driving assistance system 10 can make the character V behave as if a passenger is conveying the contents of the voice guidance to the driver by the lifelike movements of the character V that transition from the third movement pattern to the sixth movement pattern and the voice guidance to the driver.

[0136] As a result, the vehicle driving assistance system 10 allows the driver to recognize the character V in the three-dimensional virtual space S as a passenger, thereby improving the driver's mentalization and assisting the driver in his driving operations to ensure the driver's safety.

[0137] The vehicle driving assistance system 10 also includes driver state detection means (accelerator opening sensor 16, brake pedal stroke sensor 17, steering angle sensor 19, in-vehicle camera 24, and ECU 26) for detecting the state of the driver while driving. The character control means (ECU 26) is configured to suppress the movement of the character V in the fourth movement pattern when the driver's state detected by the driver state detection means is busy.

[0138] With this configuration, in situations where the driver is required to perform driving operations according to the situation around the vehicle or to check the area around the vehicle, the driver's attention is directed to the movements of character V, preventing the driver's attention to the area around the vehicle from decreasing. As a result, the vehicle driving assistance system 10 can improve the driver's mentalizing without compromising the safety of the driver.

[0139] In correspondence between the configuration of this invention and the above-mentioned embodiment, The vehicle status detection means and the character control means of the present invention correspond to the ECU 26 of the embodiment, Similarly, The voice guidance means corresponds to the speaker 15 and the ECU 26. The first operation mode corresponds to the second operation pattern, The gaze detection means corresponds to the in-vehicle camera 24 and the ECU 26, The second operation mode corresponds to the fourth operation pattern, The third operation mode corresponds to the fifth operation pattern, The fourth operation mode corresponds to the third operation pattern, The fifth operation mode corresponds to the sixth operation pattern, The driver state detection means corresponds to the accelerator opening sensor 16, the brake pedal stroke sensor 17, the steering angle sensor 19, the in-vehicle camera 24, and the ECU 26. The present invention is not limited to the configurations of the above-described embodiments, and many other embodiments can be obtained.

[0140] For example, in the above-described embodiment, the character V displayed on the display unit 14 is a three-dimensional model in the shape of a teardrop lying on its side, but this is not limited to this, and the shape of the character V may be any suitable shape or design. Furthermore, instead of the three-dimensional model character V, a two-dimensional image character, an avatar or a personified character that personifies the vehicle 1, or the like may be displayed on the display unit 14.

[0141] Furthermore, the vehicle driving assistance system 10 is not limited to the configuration described above, and may include, for example, a camera, millimeter wave radar, or a sensor that detects the situation behind or to the sides of the vehicle. Furthermore, the vehicle driving assistance system 10 may include, for example, a navigation system, which assists the driver in driving operations based on the vehicle position and map data.

[0142] 5, the first movement pattern is a movement of repeatedly moving up and down like jumping as a movement to welcome the driver seated in the driver's seat, but is not limited to this. For example, the first movement pattern may be a movement in which the size and shape change with 1 / f fluctuation and a movement of repeatedly moving up and down are performed simultaneously.

[0143] Furthermore, in the second movement pattern, character V is moved with predetermined movement characteristics including one or more of goal-directedness, equifinality, self-propelledness, and violation of physical laws, but this is not limited to these, and any other appropriate movement characteristics may be used.

[0144] 5 and step S125 of Fig. 6, the scaling rate and deformation amount of the character V are suppressed, but this is not limited to this. For example, either the scaling rate of the character V or the deformation amount of the character V may be suppressed, or the movement speed of the character V may be slowed.

[0145] Furthermore, the processing flow in the processing operation for controlling the behavior of character V in FIG. 5 and the processing flow in the detailed operation processing in FIG. 6 are merely examples, and are not limited to these, and any appropriate processing flow may be used. [Explanation of symbols]

[0146] 1...Vehicle 10...Vehicle driving assistance system 14...Display section 15...Speaker 16...Accelerator opening sensor 17...Brake pedal stroke sensor 19...Steering angle sensor 24...In-car camera 26...ECU S...3D virtual space V...Character

Claims

1. A vehicle driving assistance system including: a vehicle status detection means for detecting a vehicle status based on a driving behavior of the vehicle and a status around the vehicle; and a voice guidance means for providing a voice guidance based on the vehicle status detected by the vehicle status detection means to a driver of the vehicle, a display unit that displays a character placed in a three-dimensional virtual space; character control means for controlling the movement of the character in the three-dimensional virtual space; The character control means The first movement mode is a combination of a movement in which the size and shape change with 1 / f fluctuation and a movement with a predetermined movement characteristic, and controls the movement of the character waiting in the three-dimensional virtual space. Vehicle driving assistance system.

2. a gaze detection means for detecting a gaze movement of the driver, The character control means When the gaze detection means detects the gaze movement of the driver, the character's movement is controlled in a second movement mode that combines a movement in which the size and shape change with 1 / f fluctuation and a movement in which the character moves in accordance with the gaze movement of the driver. The vehicle driving assistance system according to claim 1 .

3. The voice guidance means When the situation of the vehicle requires a driver to pay attention to a direction other than the traveling direction of the vehicle, the driver is notified of the attention by voice guidance, The character control means When the driver is notified of the warning by voice guidance, the character's movement is controlled in a third movement mode, which is a movement in which the size and shape change with 1 / f fluctuation is combined with a movement in which the character moves in a direction other than the traveling direction, which requires caution. The vehicle driving assistance system according to claim 1 .

4. The character control means When the host vehicle status detected by the host vehicle status detection means is that the host vehicle is moving, the action of the character is controlled in a fourth action mode, which is a combination of a movement in which the size and shape change with 1 / f fluctuation and a movement in which the character faces the traveling direction of the vehicle, and when the voice guidance is announced by the voice guidance means, the action of the character is controlled in a fifth action mode, which is a combination of a movement in which the size and shape change with 1 / f fluctuation and a movement in which the character faces the driver directly. The vehicle driving assistance system according to claim 1 .

5. a driver state detection means for detecting the state of the driver while driving; The character control means When the driver's state detected by the driver state detection means is busy, the movement of the character in the second operating mode is suppressed. The vehicle driving assistance system according to claim 2 .

Images