Character display method and driving support device

Abstract

To reduce the dissatisfaction of an occupant with travel support control.SOLUTION: A travel support device 10 comprises: a sensor 13 which detects an object around an own vehicle 1; a display device 17 which is arranged in the own vehicle 1; and a controller 18 which executes processing of detecting an ambient environment of the own vehicle 1 on the basis of the output signal of the sensor 13, travel support control of automatically controlling at least one of the acceleration, deceleration or steering angle of the own vehicle 1 on the basis of the detected ambient environment, processing of estimating an expectation value of an occupant of the own vehicle 1 about the travel support control, processing of generating a character image expressing a character which changes at least one of the action or facial expression according to a difference between the estimated expectation value and the actually executed travel support control, and processing of displaying the character image on the display device.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a character display method and a driving assistance device. [Background technology]

[0002] There are known cruise assist devices that use sensors to detect the surrounding environment of a vehicle and automatically control at least one of the vehicle's acceleration, deceleration, or steering angle based on the detected surrounding environment. For example, Patent Document 1 proposes a vehicle control device with a follow-up cruise control function that adjusts the vehicle speed so that the distance between the vehicle and a leading vehicle approaches a target distance. This vehicle control device disables the follow-up cruise control function before the vehicle reaches a brake fade state. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-19455 Summary of the Invention [Problem to be solved by the invention]

[0004] If the driving assistance function is cancelled without the occupant's consent, the occupant may feel uncomfortable. The same applies if the results of the driving assistance function do not meet the occupant's expectations. The present invention aims to reduce discomfort felt by occupants in response to driving support control that automatically controls at least one of the acceleration, deceleration, and steering angle of the vehicle. [Means for solving the problem]

[0005] A character display method according to one aspect of the present invention has a controller perform the following processes: detecting the surrounding environment of the vehicle; driving assistance control that automatically controls at least one of the acceleration, deceleration, or steering angle of the vehicle based on the detected surrounding environment; estimating the expectations of the vehicle's occupants regarding the driving assistance control; and generating and displaying on a display device within the vehicle a character image representing a character whose behavior or facial expression changes depending on the difference between the estimated expectations and the driving assistance control actually executed. [Effects of the Invention]

[0006] According to the present invention, it is possible to reduce discomfort felt by occupants in response to driving support control that automatically controls at least one of the acceleration, deceleration, and steering angle of the vehicle. [Brief explanation of the drawings]

[0007] [Figure 1] 1 is a diagram illustrating an example of a schematic configuration of a vehicle equipped with a driving assistance device according to an embodiment. [Figure 2] 10(a) and 10(b) are diagrams showing examples of character images displayed according to the difference between the passenger's expectations regarding driving support control and the actual driving support control. [Figure 3] 2 is a block diagram illustrating an example of a functional configuration of a controller in FIG. 1. FIG. [Figure 4] 10A and 10B are diagrams showing examples of character images displayed during driving support control. [Figure 5] (a) is a diagram showing an example of a character image displayed when driving assistance control is successfully completed, and (b) is a diagram showing an example of a character image displayed during manual driving control. [Figure 6] 10(a) and 10(b) are diagrams showing examples of character images that are displayed when there is a large difference between the temperature setting of the air conditioner and the temperature inside the vehicle cabin. [Figure 7] (a) is a diagram showing an example of a character image that is displayed when a sensor detects an object around the vehicle, and (b) is a diagram showing an example of a character image that is displayed when the automatic brake is activated. [Figure 8] 1 is a flowchart of a first example of a character display method according to an embodiment. [Figure 9] 10 is a flowchart of a second example of a character display method according to an embodiment. [Figure 10] 10 is a flowchart of a third example of a character display method according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the description of the drawings, identical or similar parts are designated by identical or similar reference numerals, and redundant explanations will be omitted. Each drawing is a schematic diagram and may differ from the actual product. The embodiments shown below exemplify devices and methods for embodying the technical concept of the present invention, and the technical concept of the present invention is not limited to the devices and methods exemplified in the following embodiments. The technical concept of the present invention can be modified in various ways within the technical scope described in the claims.

[0009] (composition) 1 is a diagram showing an example of a schematic configuration of a vehicle equipped with a driving assistance device according to an embodiment. The vehicle 1 is equipped with a driving assistance device 10 that assists the driving of the vehicle 1. The driving assistance device 10 detects the driving environment around the vehicle 1 and automatically controls the driving of the vehicle 1 based on the detected driving environment, thereby assisting a passenger (e.g., a driver) of the vehicle 1 in driving the vehicle 1. For example, the driving assistance provided by the driving assistance device 10 to the host vehicle 1 may include steering assistance control that automatically controls at least the steering angle. For example, the driving assistance provided by the driving assistance device 10 may be assistance to prevent deviation from a lane.

[0010] Furthermore, for example, the driving assistance of the vehicle 1 by the driving assistance device 10 may include speed control that automatically controls at least one of the acceleration and deceleration of the vehicle. For example, the driving assistance by the driving assistance device 10 may be a following control that causes the vehicle to travel so as to maintain a predetermined distance from a preceding vehicle, or a constant speed driving control. Further, for example, driving assistance by the driving assistance device 10 may include autonomous driving control in which the vehicle 1 is driven automatically without the involvement of a passenger. Furthermore, driving assistance by the driving assistance device 10 may be automatic parking assistance in which the vehicle is automatically parked at a target parking position by controlling the acceleration, deceleration, and steering angle of the vehicle.

[0011] The driving assistance device 10 includes a positioning device 11, a map database 12, an object sensor 13, a vehicle sensor 14, a mode selector switch 15, a human-machine interface 17, a controller 18, and an actuator 19. In the drawings, the map database is referred to as a "map DB," the mode selector switch is referred to as a "mode selector SW," and the human-machine interface is referred to as an "HMI."

[0012] The positioning device 11 measures the current position of the vehicle 1. The positioning device 11 may include, for example, a Global Navigation System (GNSS) receiver. The GNSS receiver is, for example, a Global Positioning System (GPS) receiver, and receives radio waves from multiple navigation satellites to measure the current position of the vehicle 1. The positioning device 11 may also be an inertial navigation system. The map database 12 stores road map data. For example, the map database 12 may store high-precision map data (hereinafter simply referred to as "high-precision map") suitable as map information for autonomous driving. The road map data stored in the map database 12 may also be map data for navigation (hereinafter simply referred to as "navigation map"). The high-precision map is map data with higher precision than the navigation map.

[0013] The object sensor 13 acquires various information (surrounding environment information) about the driving environment around the host vehicle 1. For example, the object sensor 13 detects objects around the host vehicle 1. The object sensor 13 detects the surrounding environment of the host vehicle 1, such as objects present around the host vehicle 1, the relative position between the host vehicle 1 and the objects, the distance between the host vehicle 1 and the objects, and the direction in which the objects exist. The object sensor 13 outputs the detected surrounding environment information to the controller 18. For example, the object sensor 13 may include a camera, a LiDAR (Light Detection and Ranging), a radar, a millimeter wave radar, a laser range finder, a sonar, or the like.

[0014] The vehicle sensors 14 detect various information (vehicle information) obtained from the host vehicle 1. The vehicle sensors 14 include, for example, a vehicle speed sensor that detects the traveling speed (vehicle speed) V of the host vehicle 1, a wheel speed sensor that detects the rotational speed of each tire equipped on the host vehicle 1, a three-axis acceleration sensor (G sensor) that detects the acceleration (including deceleration) of the host vehicle 1 in three axial directions, a steering angle sensor that detects the steering angle of the steering wheel, a turning angle sensor that detects the turning angle of the steered wheels, a gyro sensor that detects the angular velocity generated in the host vehicle 1, a yaw rate sensor that detects the yaw rate, an accelerator sensor that detects operation of the accelerator pedal of the host vehicle 1, a brake sensor that detects operation of the brake pedal by the driver, and a temperature sensor that detects the temperature inside the cabin of the host vehicle 1. The vehicle sensors 14 output the vehicle information to the controller 18.

[0015] The controller 18 is an electronic control unit (ECU) that performs driving assistance control of the host vehicle 1. The controller 18 includes a processor 18a and peripheral components such as a storage device 18b. The processor 18a may be, for example, a CPU (Central Processing Unit) or an MPU (Micro-Processing Unit). The storage device 18b may include a semiconductor storage device, a magnetic storage device, an optical storage device, etc. The storage device 18b may include a register, a cache memory, a memory such as a ROM (Read Only Memory) and a RAM (Random Access Memory) used as a main memory device. The functions of the controller 18 described below are realized by, for example, the processor 18a executing a computer program stored in the storage device 18b. The controller 18 may be formed of dedicated hardware for executing each of the information processes described below. For example, the controller 18 may include a functional logic circuit configured in a general-purpose semiconductor integrated circuit. For example, the controller 18 may include a programmable logic device (PLD) such as a field-programmable gate array (FPGA).

[0016] The actuator 19 operates the steering wheel, accelerator opening, and brake device of the host vehicle 1 in response to control signals from the controller 18 to generate vehicle behavior of the host vehicle 1. The actuator 19 includes a steering actuator, an accelerator opening actuator, and a brake control actuator. The steering actuator controls the steering direction and steering amount of the steering of the host vehicle 1. The accelerator opening actuator controls the accelerator opening of the host vehicle 1. The brake control actuator controls the braking operation of the brake device of the host vehicle 1.

[0017] The mode changeover switch 15 is a switch that accepts operation by the occupant to switch the driving mode of the host vehicle 1. The occupant can switch the driving mode of the host vehicle 1 by operating the mode changeover switch 15. For example, the mode changeover switch 15 may be a switch that switches the driving mode of the vehicle 1 between an automatic driving mode and a manual driving mode. The autonomous driving mode is a mode in which the controller 18 performs autonomous driving control to automatically drive the vehicle 1 without the involvement of an occupant by driving the actuator 19 based on the surrounding environment information output from the object sensor 13 and the vehicle information output from the vehicle sensor 14.

[0018] On the other hand, the manual driving mode is a mode in which the driver drives the vehicle 1 by manually operating the steering wheel, accelerator pedal, and brake pedal. In the manual driving mode, the controller 18 may execute driving assistance control that automatically controls at least one of the acceleration, deceleration, or steering angle of the host vehicle 1. When executing driving assistance control, the controller 18 controls at least one of the acceleration, deceleration, or steering angle by driving the actuator 19. For example, the controller 18 may execute lane departure prevention control, following control, or constant speed cruise control. The mode selector switch 15 may be a switch for switching between activation and deactivation of the lane departure prevention control, following control, or constant speed cruise control. Furthermore, for example, the mode selector switch 15 may be a switch for switching between activation and deactivation of automatic parking control.

[0019] The human-machine interface 17 is an interface device that exchanges information between the driving assistance device 10 and the occupant. The human-machine interface 17 includes an operator that accepts an operation input from the occupant to the driving assistance device 10. The operator may be a mechanical interface device such as a button, switch, lever, dial, or keyboard, or may be a button, switch, lever, dial, keyboard, or the like displayed on a touch panel. The human-machine interface 17 may also include a speaker or buzzer for outputting alarm sounds, notification sounds, and audio information. Furthermore, the human-machine interface 17 includes a display device that can be viewed by a passenger in the vehicle 1 (for example, a display screen of a navigation system or a display device provided near the meter in front of the driver's seat). In the following description, the display device of the human-machine interface 17 will be simply referred to as the "display device."

[0020] This display device displays character images that characterize the driving support functions of the driving support device 10. Figures 2(a) and 2(b) are diagrams showing examples of character images 20 and 21 displayed on the display device. As described above, the occupant may feel uncomfortable with the driving assistance control executed by the vehicle 1. For example, the occupant may feel uncomfortable if the driving assistance function is deactivated without the occupant's intention or if the results of the driving assistance function do not meet the occupant's expectations.

[0021] Therefore, the controller 18 estimates the passenger's expectations regarding the driving support control by the driving support device 10. Then, the controller 18 changes the behavior and facial expression of the character 2 displayed on the display device according to the difference between the estimated expectations and the driving support control that is actually executed. For example, if the actually executed driving support control is likely to deviate from the passenger's expectations, a character image 20 of a character 2 driving hard is displayed on the display device as shown in FIG. 2(a).

[0022] Also, for example, if the driving support control deviates from the passenger's expectations, a character image 21 of the character 2 with his head down as shown in Fig. 2(b) may be displayed on the display device, followed by a character image 20 of the character 2 driving hard as shown in Fig. 2(a). Also, for example, when the driving support function is deactivated unintentionally by the occupant, a character image 21 of the character 2 with its head down may be displayed as shown in FIG. 2(b). In this way, by displaying character 2 according to the difference between the estimated expected value and the driving support control that was actually executed, the discomfort felt by the driver regarding the results of the driving support control or the discomfort felt by the driver regarding the cancellation of the driving support control can be alleviated by the behavior and facial expression of character 2.

[0023] Note that the controller 18 may display, on the display device, a character image of the character 2 adjusting the temperature when the temperature difference between the air temperature inside the vehicle 1 and the set temperature of the air conditioner 16 of the vehicle 1 is equal to or greater than a threshold. Information regarding the set temperature may be acquired from the air conditioner 16. Furthermore, the controller 18 may display a character image on the display device in accordance with an object around the vehicle detected by the object sensor 13.

[0024] The function of the controller 18 to generate and display a character image will be described in more detail below. Figure 3 is a block diagram showing an example of the functional configuration of the controller 18. The controller 18 includes a mode switching unit 30, a situation recognition unit 31, an operation input recognition unit 32, a room temperature recognition unit 33, an automatic driving control unit 34, an expected value estimation unit 35, a sensor display determination unit 36, a room temperature comparison unit 37, a comparison unit 38, and a character image generation unit 39. The situation recognition unit 31 recognizes the current situation of the vehicle 1. The situation recognition unit 31 includes a traveling state recognition unit 31a, a surrounding environment recognition unit 31b, a position recognition unit 31c, and a vehicle state recognition unit 31d.

[0025] The running state recognition unit 31a recognizes the running state of the host vehicle 1 based on vehicle information from the vehicle sensor 14. For example, the running state recognition unit 31a may recognize the speed, acceleration, deceleration, angular velocity, and yaw rate of the host vehicle 1 as the running state of the host vehicle 1, and detect the magnitude of each as a state quantity of the running state. Further, for example, when automatic parking control is being executed as driving support control by the driving support device 10, the time elapsed since the start time of the automatic parking control or the number of turns may be detected as the state quantity of the driving state.

[0026] The position recognition unit 31c measures the absolute position of the vehicle 1, i.e., the position, attitude, and speed of the vehicle 1 relative to a predetermined reference point, based on odometry using the measurement results from the positioning device 11 and the detection results from the vehicle sensor 14. The position recognition unit 31c acquires map information about the area around the vehicle 1 from the map database 12. The map information may also be acquired from an external map data server via a communication device. The position recognition unit 31c estimates the position and attitude of the vehicle 1 on the map from the absolute position of the vehicle 1 and the map information. The position recognition unit 31c identifies the road on which the vehicle 1 is traveling and further the lane on that road in which the vehicle 1 is traveling, and detects the lateral position of the vehicle 1 within the traveling lane (vehicle width position, lateral position within the lane) and the amount of variation thereof as state quantities of the vehicle's traveling state.

[0027] The surrounding environment recognition unit 31b recognizes the surrounding environment of the vehicle 1 based on the detection result of the object sensor 13. For example, based on the detection result of the object sensor 13, it detects the position, posture, size, speed, etc. of objects around the vehicle 1, such as vehicles (cars and motorcycles), pedestrians, obstacles, etc. The surrounding environment recognition unit 31b then integrates the multiple detection results obtained from the multiple object detection sensors and outputs a single two-dimensional position, orientation, size, speed, etc. for each object. Specifically, from the object behavior obtained from each object detection sensor, the most reasonable object behavior that minimizes error is calculated, taking into account the error characteristics of each object detection sensor. Specifically, by using known sensor fusion technology, the detection results obtained from multiple types of sensors are comprehensively evaluated to obtain more accurate detection results.

[0028] Furthermore, the surrounding environment recognition unit 31b verifies (corresponds) the identity of objects between different times based on the behavior of the objects output at different times based on the integrated detection results, and predicts the behavior of the object, such as its speed, based on the correspondence. When the surrounding environment recognition unit 31b detects a vehicle ahead of the host vehicle 1, the surrounding environment recognition unit 31b may detect the inter-vehicle distance between the host vehicle 1 and the preceding vehicle as a state quantity of the traveling state. The surrounding environment recognition unit 31b also recognizes the driving scene of the vehicle 1 based on the image captured by the camera of the object sensor 13 and map information about the area around the vehicle 1. For example, the driving scene may be recognized as whether the vehicle 1 is driving on a curved road. Alternatively, the driving scene may be recognized as whether the vehicle 1 is driving on a narrow road. For example, the driving scene may be recognized as whether the vehicle 1 is traveling in a merging lane. For example, the driving scene may be recognized as whether there is a stopping target such as a stop line or an obstacle ahead of the vehicle 1.

[0029] The vehicle state recognition unit 31d recognizes other vehicle states of the host vehicle 1. For example, the vehicle state recognition unit 31d recognizes the soundness and detection accuracy of the object sensor 13 and the vehicle sensor 14, and any abnormalities that have occurred in the object sensor 13 and the vehicle sensor 14. Furthermore, for example, the vehicle state recognition unit 31d may recognize whether map information necessary for driving assistance control by the driving assistance device 10 is available. For example, the vehicle state recognition unit 31d may recognize whether a high-precision map necessary for autonomous driving control of the vehicle 1 is available. For example, the vehicle state recognition unit 31d may recognize whether map information for the section in which the vehicle 1 is traveling is stored in the map database 12.

[0030] The operation input recognition unit 32 recognizes the steering angle of the steering wheel, the operation amount of the accelerator pedal, and the operation amount of the brake pedal based on the detection results of the steering angle sensor, accelerator sensor, and brake sensor of the vehicle sensor 14. The room temperature recognition unit 33 recognizes the temperature inside the vehicle compartment of the vehicle 1 based on the detection result of the temperature sensor of the vehicle sensor 14.

[0031] The mode switching unit 30 switches the driving mode of the vehicle 1 between an automatic driving mode and a manual driving mode based on the operation of the mode switching switch 15 by the occupant. Furthermore, when the operation input recognition unit 32 recognizes an override operation by the occupant, the mode switching unit 30 switches the driving mode of the host vehicle 1 to the manual driving mode and cancels the driving support control by the driving support device 10. An override operation refers to an occupant operating the steering wheel, accelerator pedal, or brake pedal in autonomous driving mode to intervene in driving assistance control. Switching from the automatic driving mode to the manual driving mode by operating the mode selector switch 15 or by an override operation is an example of cancellation of the driving assistance control at the driver's discretion.

[0032] Furthermore, the mode switching unit 30 switches the driving mode of the host vehicle 1 from the automatic driving mode to the manual driving mode based on the surrounding environment recognized by the surrounding environment recognition unit 31b or the vehicle state of the host vehicle 1 recognized by the vehicle state recognition unit 31d. In other words, the driving assist control is cancelled. Cancellation of driving assist control based on the surrounding environment or the vehicle state is an example of cancellation of driving assist control unintentionally by the occupant (or cancellation of driving assist control unintentionally by the occupant). For example, when the surrounding environment recognition unit 31b recognizes that the vehicle 1 has reached a point a predetermined distance before a section where autonomous driving is difficult (hereinafter, may be referred to as a "section where autonomous driving is difficult"), the mode switching unit 30 may switch the driving mode of the vehicle 1 from the autonomous driving mode to the manual driving mode. The section where autonomous driving is difficult may be, for example, a merging section where the vehicle merges into a congested lane.

[0033] Furthermore, for example, the driving mode of the host vehicle 1 may be switched from the autonomous driving mode to the manual driving mode when the soundness or detection accuracy of the object sensor 13 or the vehicle sensor 14 falls below an allowable limit value or when the vehicle state recognition unit 31d recognizes that an abnormality has occurred in the object sensor 13 or the vehicle sensor 14. Furthermore, for example, when the vehicle state recognition unit 31d recognizes that map information required for driving assistance control by the driving assistance device 10 is unavailable, the driving mode of the host vehicle 1 may be switched from the autonomous driving mode to the manual driving mode. Similarly, the mode switching unit 30 may cancel lane departure prevention control, following control, constant speed driving control, or automatic parking control based on the surrounding environment recognized by the surrounding environment recognition unit 31b or the vehicle state of the host vehicle 1 recognized by the vehicle state recognition unit 31d.

[0034] Furthermore, before switching the driving mode of the host vehicle 1 from the autonomous driving mode to the manual driving mode, the mode switching unit 30 may determine whether or not the driving mode of the host vehicle 1 is about to be switched from the autonomous driving mode to the manual driving mode, based on the surrounding environment recognized by the surrounding environment recognition unit 31b or the vehicle state of the host vehicle 1 recognized by the vehicle state recognition unit 31d. In other words, the mode switching unit 30 may determine whether or not the driving assist control is about to be released. For example, when the vehicle 1 is approaching an area where autonomous driving is difficult or an area where map information is unavailable, it may be determined that the driving mode of the vehicle 1 is likely to be switched from the autonomous driving mode to the manual driving mode. Also, when the soundness or detection accuracy of the object sensor 13 or the vehicle sensor 14 approaches an allowable limit value, it may be determined that the driving mode of the vehicle 1 is likely to be switched from the autonomous driving mode to the manual driving mode. Similarly, the mode switching unit 30 may determine whether or not lane departure prevention control, following control, constant speed driving control, or automatic parking control is about to be released based on the surrounding environment recognized by the surrounding environment recognition unit 31b or the vehicle state of the vehicle 1 recognized by the vehicle state recognition unit 31d.

[0035] When the driving mode of the host vehicle 1 is set to the autonomous driving mode, the autonomous driving control unit 34 executes autonomous driving control of the host vehicle 1 based on the driving state of the host vehicle 1 recognized by the driving state recognition unit 31a, the current position and attitude of the host vehicle 1 recognized by the position recognition unit 31c, and the surrounding environment of the host vehicle 1 recognized by the surrounding environment recognition unit 31b. For example, the automatic driving control unit 34 calculates a target driving trajectory for the vehicle 1 to travel based on the current position and attitude of the vehicle 1, a target route to the destination set by a navigation system or the like (not shown), and the surrounding environment of the vehicle 1.

[0036] For example, a route space map that shows the route around the vehicle 1 and the presence or absence of objects, and a risk map that quantifies the danger level of the driving area, are generated, and a target driving trajectory for the vehicle 1 to travel is generated based on the movement characteristics of the vehicle 1, the route space map, and the risk map. The automatic driving control unit 34 drives the actuator 19 so that the host vehicle 1 travels along the generated target travel trajectory.

[0037] In addition, when the driving mode of the host vehicle 1 is set to manual driving mode and lane departure prevention control is activated, the automatic driving control unit 34 may automatically control the steering angle of the host vehicle 1 so as to maintain the host vehicle 1's lateral position within the lane (for example, so as to drive in the center of the lane in accordance with the lane shape) based on the driving state of the host vehicle 1 and the host vehicle 1's current lateral position and attitude within the lane. For example, when follow-up control or constant speed driving control is activated, the automatic driving control unit 34 may automatically control the acceleration and deceleration of the vehicle based on the driving state of the vehicle 1 and the surrounding environment so as to maintain the distance between the vehicle and the preceding vehicle, or so as to drive at a preset speed.

[0038] For example, when automatic parking control is activated, the automatic driving control unit 34 calculates the relative position of the target parking position with respect to the current position of the vehicle 1 based on the surrounding environment of the vehicle 1, generates a target driving trajectory from the current position of the vehicle 1 to the target parking position, and drives the actuator 19 so that the vehicle 1 drives along the generated target driving trajectory. For example, when the surrounding environment recognition unit 31b detects an obstacle located ahead of the vehicle 1 in the direction of travel, the automatic driving control unit 34 performs automatic braking control (e.g., emergency braking control) to stop the vehicle 1 in front of the obstacle.

[0039] The expected value estimation unit 35 estimates the expected values of the occupants of the vehicle 1 regarding the driving support control by the driving support device 10. For example, it is considered that the passenger has the following expectations (A) to (G) regarding the driving support control by the driving support device 10. (A) The driving assistance control is not released unintentionally by the occupant. (B) Driving within the lane (i.e., without fluctuations in lateral position within the lane). (C) Avoid sudden acceleration or deceleration, and do not accelerate or decelerate too slowly. (D) Do not drive too fast or too slow. (E) The distance between your vehicle and the vehicle ahead is not too short. (F) The time required for automatic parking should not be too long. (G) The number of maneuvers during automatic parking should not be too many.

[0040] For this reason, the expected value estimation unit 35 estimates the occupant's expectations regarding the vehicle width position of the vehicle 1 within the driving lane, the amount of variation in the vehicle width position, the speed, acceleration, and deceleration of the vehicle 1, the distance between the vehicle 1 and the preceding vehicle, the time required for parking when automatic parking control is activated, and the number of turns required for parking. For example, the expected value estimation unit 35 may estimate upper limits acceptable to the occupant as expected values for the vehicle width direction position, the amount of change in the vehicle width direction position, the time required for parking, and the number of turns required for parking. For example, the expected value estimation unit 35 may estimate lower limits acceptable to the occupant as expected values for the inter-vehicle distance. For example, the expected value estimation unit 35 may estimate upper and lower limit values acceptable to the occupant as expected values for the speed, acceleration, and deceleration of the vehicle 1. In other words, it may estimate the ranges of acceleration, deceleration, and speed acceptable to the occupant. These upper and lower limit values are examples of the "limit values" described in the claims.

[0041] For example, the expected value estimation unit 35 may calculate the expected value for the lateral position of the host vehicle 1 in the lane it is traveling in and the inter-vehicle distance from the preceding vehicle based on the traveling conditions such as the width of the lane the host vehicle 1 is traveling in, the lane shape (curvature, etc.), and the host vehicle's speed. Alternatively, a lookup table of typical driver's expected values for the lateral position and inter-vehicle distance according to the width, lane shape, and traveling conditions may be stored in advance, and the expected value may be read from this lookup table. The lookup table may be created based on a history of the lateral position and inter-vehicle distance when the occupant of the host vehicle 1 was driving in the past.

[0042] Furthermore, for example, the expected value estimation unit 35 may estimate expected values for the acceleration and deceleration of the host vehicle 1 based on the traveling scene recognized by the surrounding environment recognition unit 31b. For example, when the surrounding environment recognition unit 31b recognizes that the host vehicle 1 is traveling in a merging lane, the expected value for acceleration in the merging lane may be estimated. Also, when the surrounding environment recognition unit 31b recognizes a stopping target ahead of the host vehicle 1, the expected value estimation unit 35 may estimate the expected value for deceleration for each driving scene based on the history of acceleration and deceleration when the occupant of the host vehicle 1 previously drove in a similar driving scene.

[0043] Furthermore, for example, the expected value estimation unit 35 may estimate an expected value for the vehicle speed of the host vehicle 1 based on the driving scene recognized by the surrounding environment recognition unit 31b. For example, when the surrounding environment recognition unit 31b recognizes that the vehicle 1 is traveling on a curved road or a narrow road, the expected value estimation unit 35 may estimate an expected value for the vehicle speed of the vehicle 1 traveling on these sections. For example, the expected value estimation unit 35 may estimate an expected value for the vehicle speed for each traveling scene based on a history of vehicle speeds when an occupant of the vehicle 1 has driven in similar traveling scenes in the past.

[0044] For example, when automatic parking control is activated, the expected value estimation unit 35 may estimate the expected value for the time required for parking and the number of turns required for parking based on obstacles around the vehicle 1 recognized by the surrounding environment recognition unit 31b and the relative distance between the current position of the vehicle 1 and the target parking position calculated by the automatic driving control unit 34. For example, it is possible to pre-store and read from a lookup table of typical driver's expectations for the time required and the number of turns required depending on the arrangement pattern and relative distance of obstacles around the vehicle 1. The lookup table may be created based on the history of the time required and the number of turns required when the occupant of the vehicle 1 parked in the past.

[0045] The comparison unit 38 compares the driving states of the host vehicle 1 recognized by the driving state recognition unit 31a, the surrounding environment recognition unit 31b, and the position recognition unit 31c with the expected values estimated by the expected value estimation unit 35, and determines whether the driving states of the host vehicle caused by the driving support control of the driving support device 10 deviate from the expected values estimated by the expected value estimation unit 35. The comparison unit 38 also determines whether the driving states of the host vehicle caused by the driving support control are likely to deviate from the expected values. For example, the comparison unit 38 may determine that the driving state of the vehicle 1 deviates from the expected value when the state quantities of the driving state of the vehicle 1 detected by the driving state recognition unit 31a, the surrounding environment recognition unit 31b, and the position recognition unit 31c exceed the limit values (i.e., upper and lower limit values) estimated as expected values by the expected value estimation unit 35. In the following description, the state quantities of the traveling state of the host vehicle 1 detected by the traveling state recognition unit 31a, the surrounding environment recognition unit 31b, and the position recognition unit 31c may be referred to as "detected state quantities."

[0046] Also, for example, when the detected state quantity approaches an estimated limit value, it may be determined that the running state of the host vehicle 1 is likely to deviate from the expected value. For example, it may be determined whether the difference between the detected state quantity and the estimated limit value is equal to or less than a predetermined threshold, and if the difference is equal to or less than the predetermined value, it may be determined that the detected state quantity has approached the limit value. Furthermore, for example, when an upper limit value is estimated as an expected value, it may be determined whether or not the ratio of the detected state quantity to the upper limit value estimated as an expected value (detected state quantity / upper limit value) is equal to or greater than a predetermined value. When the ratio (detected state quantity / upper limit value) is equal to or greater than the predetermined value, it may be determined that the detected state quantity has approached the limit value. Furthermore, for example, when a lower limit value is estimated as an expected value, it may be determined whether or not the ratio of the detected state quantity to the estimated lower limit value (detected state quantity / lower limit value) is equal to or less than a predetermined value. When the ratio (detected state quantity / lower limit value) is equal to or less than the predetermined value, it may be determined that the detected state quantity has approached the limit value.

[0047] The sensor display determination unit 36 determines whether or not an object around the vehicle has been recognized by the surrounding environment recognition unit 31b in order to generate a character image according to an object around the vehicle detected by the object sensor 13. Furthermore, the sensor display determination unit 36 acquires information on the direction of the recognized object from the surrounding environment recognition unit 31b. For example, the sensor display determination unit 36 determines whether or not an object around the vehicle has been detected by the sonar of the object sensor 13, and acquires information on the direction of the detected object from the surrounding environment recognition unit 31b. The room temperature comparison unit 37 detects the temperature difference between the air temperature inside the vehicle 1 recognized by the room temperature recognition unit 33 and the set temperature of the air conditioner 16. The room temperature comparison unit 37 determines whether the detected temperature difference is equal to or greater than a predetermined threshold value. In other words, it determines whether the air temperature inside the vehicle is too high compared to the set temperature, and whether the air temperature inside the vehicle is too low compared to the set temperature.

[0048] The character image generating unit 39 generates a character image to be displayed on the display device based on the driving mode of the vehicle 1, the comparison results of the comparing unit 38 and the room temperature comparing unit 37, and the determination result of the sensor display determining unit . For example, when the driving mode of the host vehicle 1 is the automatic driving mode (i.e., when driving assistance control of the host vehicle 1 is being executed), the character image generation unit 39 generates a character image of a character performing an action related to driving the vehicle. For example, the character image generation unit 39 may generate a character image 22 in which the character 2 is driving the host vehicle 1 as shown in FIG. 4(a). As indicated by arrow 22a, the character image generation unit 39 may tilt the steering wheel 3 held by the character 2 or the body of the character 2 in accordance with the actual steering angle of the host vehicle 1. Similarly, in the character image 20 of Fig. 2(a) and the character image 23 of Fig. 4(b), the steering wheel 3 held by the character 2 or the body of the character 2 may be tilted in accordance with the actual steering angle of the host vehicle 1, as indicated by arrows 20a and 23a. Also, for example, when the automatic parking control is operating and the vehicle is moving backward, a character image 22 is generated in which the character 2 is driving the vehicle 1 while facing backward, as shown in FIG. 4(b).

[0049] The character image generation unit 39 generates a character image of the character 2 whose behavior and facial expression change depending on the difference between the expected value estimated by the expected value estimation unit 35 and the driving state of the vehicle 1 recognized by the driving state recognition unit 31a, the surrounding environment recognition unit 31b, and the position recognition unit 31c (i.e., the driving state of the vehicle 1 generated by the driving assistance control of the driving assistance device 10). Specifically, when the comparison unit 38 determines that the driving state of the vehicle 1 is likely to deviate from the expected value, the character image generation unit 39 generates a character image 20 in which the character 2 is driving hard, as shown in Fig. 2(a). Alternatively, a character image in which the character 2 is impatient or sweating may be generated.

[0050] Furthermore, when the running state of the vehicle 1 deviates from the expected value, a character image 21 is generated in which the character 2 hangs its head and expresses regret, as shown in Fig. 2(b). Alternatively, a character image of the character 2 apologizing, expressing apologetic feelings, feeling regretful, or with a pale face may be generated. Furthermore, even when the mode switching unit 30 cancels the driving support function unintentionally by the occupant, the character image generating unit 39 generates a character image 21 in which the character 2 hangs its head and expresses regret, as shown in Fig. 2(b). Alternatively, a character image in which the character 2 apologizes, expresses apologetic intent, feels regretful, or looks pale may be used. Furthermore, even when the mode switching unit 30 determines that the driving support function is likely to be deactivated unintentionally by the occupant, the character image generating unit 39 generates a character image 20 in which the character 2 is driving hard, as shown in Fig. 2(a). Alternatively, a character image in which the character 2 is impatient or sweating may be generated.

[0051] On the other hand, the character image generation unit 39 also determines whether the driving assistance control of the driving assistance device 10 has been successfully completed. For example, if the mode switching unit 30 cancels the driving assistance function at the driver's discretion, it may determine that the driving assistance control has been successfully completed. Also, for example, if automatic parking control is being executed, it may determine that the driving assistance control has been successfully completed even if parking is completed. Furthermore, when the driving support control is completed, it may be determined whether the driving support control has been successfully completed depending on whether the occupant feels uncomfortable. For example, the facial expression of the occupant may be recognized from an image captured by an in-vehicle camera, and it may be determined that the occupant feels uncomfortable if the facial expression is classified as a "surprise" or "anger" expression. When the driving support control is successfully completed, the character image generating unit 39 may generate a character image 24 of the character 2 looking happy, as shown in FIG. 5(a).

[0052] On the other hand, when the driving mode of the vehicle 1 is a manual driving mode (for example, when driving assistance control of the vehicle 1 is not being executed), the character image generation unit 39 generates a character image of the character 2 performing an action that shows the immaturity or unreliability of the character 2. As a character image of character 2 performing an action that shows immaturity or unreliability, for example, a character image of character 2 performing an action unrelated to driving a vehicle may be generated. For example, a character image of character 2 wandering around without driving a vehicle, or a character image in which images of other objects such as small animals, toys, or balls are drawn simultaneously with character 2 and character 2 is distracted by other objects may be generated. Furthermore, a character image of a character 2 failing to perform an action unrelated to driving a vehicle may be generated as a character 2 performing an action that shows immaturity or unreliability. For example, as shown in Fig. 5(b), a character image 24 of a character 2 wandering around aimlessly and falling, a character image 24 of a character 2 failing to perform an action unrelated to driving a vehicle and therefore taking a long time to perform an action related to driving a vehicle, or a character image 24 of a character 2 becoming distracted or preoccupied by the surroundings while performing an action unrelated to driving a vehicle may be generated. In this way, by presenting the occupant with the character image 24 that represents the immaturity or unreliability of the character 2, it is possible to lower the occupant's expectations regarding the driving support control of the driving support device 10. This reduces the discomfort felt by the occupant when the driving support function is cancelled without the occupant's intention or when the results of the driving support function do not meet the occupant's expectations.

[0053] In addition, when the room temperature recognition unit 33 determines that the temperature difference between the air temperature inside the vehicle cabin and the set temperature of the air conditioner 16 is equal to or greater than a threshold value, the character image generation unit 39 generates a character image in which the character 2 is adjusting the temperature inside the vehicle cabin. For example, if the temperature inside the vehicle is higher than the set temperature, a character image of the character 2 desperately trying to cool the vehicle interior may be generated. For example, as shown in Fig. 6(a), a character image 26 of the character 2 desperately fanning himself with a fan 26a may be generated. For example, if the temperature inside the vehicle is too low compared to the set temperature, a character image of the character 2 frantically trying to warm up the vehicle may be generated. For example, as shown in FIG. 6(b), a character image 27 of the character 2 frantically starting a bonfire 27a may be generated.

[0054] Furthermore, when the sensor display determination unit 36 determines that an object around the vehicle has been recognized by the surrounding environment recognition unit 31b (i.e., that an object around the vehicle has been detected by the object sensor 13), the character image generation unit 39 generates a character image 28 in which the character 2 faces in the direction in which the object was detected with an anxious expression, as shown in Fig. 7(a). For example, the character image generation unit 39 generates a character image 28 in which the character faces in the direction in which the object detected by the sonar of the object sensor 13 is detected. In addition, when the automatic driving control unit 34 performs automatic braking control (e.g., emergency braking control) to stop the vehicle 1 in front of an obstacle, the character image generation unit 39 generates a character image 29 in which the character 2 is scared, as shown in Figure 7(b). The character image generating unit 39 displays these character images on a display device.

[0055] (operation) FIG. 8 is a flowchart of a first example of a character display method according to an embodiment. In step S1, the character image generation unit 39 determines whether the driving mode of the host vehicle 1 is the automatic driving mode. If the driving mode of the host vehicle 1 is the automatic driving mode (step S: Y1), the process proceeds to step S2. If the driving mode of the host vehicle 1 is not the automatic driving mode (step S1: N), the process proceeds to step S8. In step S2, the character image generating unit 39 generates a character image 22 of the character 2 driving the vehicle 1 as shown in FIG. 4(a), and displays it on the display device.

[0056] In step S3, the comparison unit 38 determines whether the driving support control (driving control) of the driving support device 10 deviates from the expected value estimated by the expected value estimation unit 35. If the driving support control deviates from the expected value (step S3: Y), the process proceeds to step S4. If the driving support control does not deviate from the expected value (step S3: N), the process proceeds to step S6. In step S4, the character image generating unit 39 generates a character image 21 in which the character 2 expresses regret as shown in FIG. 2(b), and displays it on the display device. Thereafter, in step S5, the character image generating unit 39 generates a character image 20 of the character 2 driving hard as shown in Fig. 2(a) and displays it on the display device, after which the process ends.

[0057] In step S6, the comparison unit 38 determines whether the driving support control (driving control) of the driving support device 10 is likely to deviate from the expected value estimated by the expected value estimation unit 35. If the driving support control is likely to deviate from the expected value (step S6: Y), the process proceeds to step S7. If the driving support control is unlikely to deviate from the expected value (step S6: N), the process ends. In step S7, the character image generating unit 39 generates a character image 20 of the character 2 driving hard as shown in Fig. 2(a) and displays it on the display device, after which the process ends.

[0058] In step S8, the character image generation unit 39 determines whether the driving mode of the host vehicle 1 was the automatic driving mode in the previous control cycle. If the driving mode of the host vehicle 1 was not the automatic driving mode (step S8: N), that is, if the driving mode of the host vehicle 1 was the manual driving mode in the previous control cycle, the process proceeds to step S9. If the driving mode of the vehicle 1 was the automatic driving mode in the previous control cycle (step S8: Y), that is, if the driving mode has switched from the automatic driving mode to the manual driving mode in the current control cycle, the process proceeds to step S10. In step S9, the character image generating unit 39 generates an unreliable character image 24 as shown in Fig. 5(b) and displays it on the display device, after which the process ends.

[0059] In step S10, the character image generation unit 39 determines whether the driving mode of the vehicle 1 has switched from automatic driving mode to manual driving mode without the intention of the occupant. If the driving mode of the vehicle 1 has switched to manual driving mode without the intention of the occupant (step S10: N), the process proceeds to step S11. If the driving mode has switched to manual driving mode without the intention of the occupant (step S10: Y), the process proceeds to step S12. In step S11, the character image generating unit 39 generates a character image 24 of the character 2 looking happy, as shown in Fig. 5(a), and displays it on the display device, after which the process ends. In step S12, the character image generating unit 39 generates a character image 21 in which the character 2 expresses regret as shown in Fig. 2(b) and displays it on the display device, after which the process ends.

[0060] FIG. 9 is a flowchart of a second example of the character display method according to the embodiment. In step S20, the room temperature recognition unit 33 determines whether the temperature inside the vehicle cabin is significantly higher than the set temperature of the air conditioner 16. If the temperature inside the vehicle cabin is significantly higher than the set temperature (step S20: Y), the process proceeds to step S21. If the temperature inside the vehicle cabin is not significantly higher than the set temperature (step S20: N), the process proceeds to step S22. In step S21, the character image generating unit 39 generates a character image 26 of the character 2 desperately trying to cool down the interior of the vehicle, as shown in Fig. 6(a), and displays it on the display device, after which the process ends.

[0061] In step S22, the room temperature recognition unit 33 determines whether the temperature inside the vehicle cabin is significantly lower than the set temperature of the air conditioner 16. If the temperature inside the vehicle cabin is significantly lower than the set temperature (step S22: Y), the process proceeds to step S23. If the temperature inside the vehicle cabin is not significantly lower than the set temperature (step S22: N), the process ends. In step S23, the character image generating unit 39 generates a character image 27 of the character 2 desperately trying to warm the interior of the vehicle, as shown in Fig. 6(b), and displays it on the display device, after which the process ends.

[0062] FIG. 10 is a flowchart of a third example of the character display method according to the embodiment. In step S30, the sensor display determination unit 36 determines whether or not an object has been detected around the vehicle by the object sensor 13 (e.g., sonar). If an object has been detected (step S30: Y), the process proceeds to step S31. If an object has not been detected (step S30: N), the process proceeds to step S32. In step S31, the character image generating unit 39 generates a character image 28 in which the character 2 looks in the direction of the detected object with an anxious expression, as shown in Fig. 7(a), and displays it on the display device, after which the process ends.

[0063] In step S32, the character image generation unit 39 determines whether the automatic driving control unit 34 has performed automatic braking control (e.g., emergency braking control). If automatic braking control has been performed (step S32: Y), the process proceeds to step S33. If automatic braking control has not been performed (step S32: N), the process ends. In step S33, the character image generating unit 39 generates a character image 29 of the character 2 that is scared, as shown in Fig. 7(b), and displays it on the display device, after which the process ends.

[0064] (Effects of the embodiment) (1) The controller 18 performs the following processes: detecting the surrounding environment of the vehicle 1; driving assistance control that automatically controls at least one of the acceleration, deceleration, or steering angle of the vehicle 1 based on the detected surrounding environment; estimating the expectations of the occupants of the vehicle 1 regarding the driving assistance control; generating a character image representing a character whose behavior or facial expression changes depending on the difference between the estimated expectations and the driving assistance control actually executed; and displaying the character image on a display device within the vehicle 1. This allows the character's facial expressions and movements to mitigate the discomfort felt by the occupant due to the difference between the actually executed driving support control and the occupant's expectations.

[0065] (2) The controller 18 may execute a process for detecting a state quantity indicating the driving state of the vehicle 1 generated by the driving assistance control, and may generate a character image in which at least one of the behavior or facial expression changes depending on the difference between the estimated expected value and the detected state quantity. This makes it possible to compare the actually executed driving support control with the passenger's expectations, and generate a character image according to the comparison result. (3) When the detected state quantity exceeds a limit value estimated as an expected value, the controller 18 may generate a character image in which the character apologizes, expresses an apologetic attitude, hangs its head, looks disappointed, or turns pale. This reduces the discomfort felt by the occupants when the driving support control deviates from the expected value.

[0066] (4) When the detected state quantity approaches a limit value estimated as an expected value, the controller 18 may generate a character image in which the character appears to be in a hurry, driving hard, or sweating. This reduces the discomfort felt by the occupants when the driving support control is likely to deviate from the expected value. (5) The detected state quantity may be the widthwise position of the host vehicle 1 within the driving lane, the amount of change in the widthwise position, the speed, acceleration, deceleration of the host vehicle 1, the distance between the host vehicle 1 and the preceding vehicle, the elapsed time from the start time of automatic parking control when the driving assistance control is automatic parking control, or the number of turns that occurred during automatic parking control. This allows the results of driving assistance control to be quantified and compared with expected values.

[0067] (6) The controller 18 may determine whether the driving assistance control has been released based on the surrounding environment or the state of the vehicle 1, and if the driving assistance control has been released based on the surrounding environment or the state of the vehicle 1, may generate a character image in which the character apologizes, expresses an apologetic attitude, hangs its head, looks disappointed, or looks pale. This reduces the discomfort felt by the occupants when the driving support control is cancelled. (7) The controller 18 It may be determined whether or not the driving support control is likely to be released based on the surrounding environment or the state of the vehicle 1, and if it is determined that the driving support control is likely to be released based on the surrounding environment or the state of the vehicle 1, a character image may be generated in which the character is in a hurry, driving hard, or sweating. This makes it possible to notify the occupants that the driving support control is about to be released, and also to reduce the discomfort felt by the occupants when the driving support control is subsequently released.

[0068] (8) When driving assistance control is not executed, the controller 18 may generate a character image of a character that performs an action related to driving the vehicle when driving assistance control is executed, and may generate a character image of a character that performs an action unrelated to driving the vehicle when driving assistance control is not executed. For example, when the controller 18 does not execute driving assistance control, the controller 18 may generate a character image of a character that performs an action that is unrelated to driving the vehicle and fails. This can lower the occupant's expectations regarding the driving support control of the driving support device 10. This can reduce the discomfort felt by the occupant when the driving support function is cancelled without the occupant's intention or when the results of the driving support function do not meet the occupant's expectations. (9) When the driving support control is successfully completed, the controller 18 may generate a character image in which the character looks happy. In this way, by expressing the joy of the passengers when the results are as intended, the passengers can empathize with the vehicle, and conversely, the passengers can be more receptive when the results are not as intended. (10) The controller 18 may execute a process of detecting objects present around the vehicle 1 using a sensor, and when an object is detected, may generate a character image in which the character looks anxious and faces in the direction in which the object is detected. This allows the driver to be informed of the direction in which an object that could become an obstacle exists around the vehicle 1. (11) The controller 18 may execute a process to detect the temperature difference between the set temperature of the air conditioner 16 and the temperature inside the vehicle 1, and if the temperature difference is equal to or greater than a threshold, generate a character image of a character adjusting the temperature. This can reduce the discomfort felt by the occupants when the temperature inside the vehicle 1 differs from the set temperature of the air conditioner 16. [Explanation of symbols]

[0069] 1...Own vehicle, 10...Driving assistance device, 11...Positioning device, 12...Map database, 13...Object sensor, 14...Vehicle sensor, 15...Mode changeover switch, 16...Air conditioning device, 17...Human-machine interface, 18...Controller, 18a...Processor, 18b...Storage device, 19...Actuator, 20...Character image, 30...Mode changeover unit, 31...Situation recognition unit, 31a...Driving state recognition unit, 31b...Surrounding environment recognition unit, 31c...Position recognition unit, 31d...Vehicle state recognition unit, 32...Operation input recognition unit, 33...Room temperature recognition unit, 34...Autonomous driving control unit, 35...Expected value estimation unit, 36...Sensor display determination unit, 37...Room temperature comparison unit, 38...Comparison unit

Claims

1. A process of detecting the surrounding environment of the vehicle; a driving support control that automatically controls at least one of an acceleration, a deceleration, or a steering angle of the host vehicle based on the detected surrounding environment; a process of estimating an expectation value of an occupant of the host vehicle regarding the driving assist control based on a width or a shape of a lane in which the host vehicle is traveling, a driving state of the host vehicle, or a result of detecting the surrounding environment; a process of generating a character image representing a character whose behavior or facial expression changes according to a difference between the estimated expected value and the actually executed driving support control; a process of displaying the character image on a display device in the host vehicle; A character display method characterized by causing a controller to execute the above.

2. The controller executes a process of detecting a state quantity that indicates a traveling state of the host vehicle generated by the traveling assist control; generating the character image in which at least one of the behavior or facial expression of the character changes according to the difference between the estimated expected value and the detected state quantity; 2. The character display method according to claim 1.

3. The character display method according to claim 2, wherein the controller generates the character image in which the character apologizes, shows an apologetic attitude, hangs its head, looks disappointed, or becomes pale when the detected state quantity exceeds a limit value estimated as the expected value.

4. The character display method according to claim 3, wherein the controller generates the character image in which the character appears impatient, is driving hard, or is sweating when the detected state quantity approaches a limit value estimated as the expected value.

5. The character display method according to any one of claims 2 to 4, characterized in that the detected state quantity is the vehicle width direction position of the host vehicle within the driving lane, the amount of variation in the vehicle width direction position, the speed, acceleration, deceleration of the host vehicle, the inter-vehicle distance between the host vehicle and a preceding vehicle, the elapsed time from the start time of automatic parking control when the driving assistance control is automatic parking control, or the number of turns occurring during the automatic parking control.

6. The character display method according to any one of claims 1 to 5, wherein the controller determines whether the driving assist control has been released based on the surrounding environment or the state of the host vehicle, and when the driving assist control has been released based on the surrounding environment or the state of the host vehicle, generates the character image in which the character apologizes, expresses an apologetic attitude, hangs its head, looks disappointed, or becomes pale.

7. The character display method according to any one of claims 1 to 6, characterized in that the controller determines whether the driving assistance control is likely to be released based on the surrounding environment or the state of the vehicle, and if the driving assistance control is likely to be released based on the surrounding environment or the state of the vehicle, generates the character image in which the character appears impatient, is driving hard, or is sweating.

8. The character display method according to any one of claims 1 to 7, characterized in that the controller generates the character image of the character that performs an action related to vehicle driving when the driving assistance control is executed, and generates the character image of the character that performs an action unrelated to vehicle driving when the driving assistance control is not executed.

9. The character display method according to claim 8, wherein the controller generates the character image of the character that performs an action that fails to perform the action unrelated to vehicle driving when the driving assistance control is not executed.

10. The character display method according to any one of claims 1 to 9, characterized in that the controller generates the character image in which the character is happy when the driving assistance control is successfully completed.

11. The controller Execute a process of detecting an object present around the vehicle by a sensor; When the object is detected, the character image is generated in which the character looks in the direction in which the object is detected with an anxious expression.

11. The character display method according to claim 1.

12. The controller Execute a process of detecting a temperature difference between a set temperature of an air conditioning device of the host vehicle and a temperature inside the vehicle interior; If the temperature difference is equal to or greater than a threshold, a character image is generated in which the character is adjusting the temperature.

12. The character display method according to claim 1, wherein the character display method is a character display method for displaying a character.

13. a sensor that detects objects around the vehicle; a display device disposed in the vehicle; a controller that executes the following processes: detecting the surrounding environment of the host vehicle based on an output signal of the sensor; driving support control that automatically controls at least one of the acceleration, deceleration, or steering angle of the host vehicle based on the detected surrounding environment; estimating an expectation value of an occupant of the host vehicle regarding the driving support control based on the detection result of the width or shape of the lane in which the host vehicle is traveling, the driving state of the host vehicle, or the surrounding environment; generating a character image representing a character whose behavior or facial expression changes depending on the difference between the estimated expectation value and the driving support control actually performed; and displaying the character image on the display device; A driving assistance device comprising:

Images