Driving assistance device, driving assistance processing method, and recording medium

Abstract

This driving assistance device determines an overconfident state of a driver with respect to execution of a prescribed driving assistance function, determines a function limit state indicating the degree at which safety achieved by the execution of the prescribed driving assistance function is not guaranteed, and, when the driver is in an overconfident state, performs virtual image display for causing the driver to recognize that the function limit state is greater than a prescribed limit reference according to the overconfident state.

Description

Driving assistance device, driving assistance processing method, and recording medium

[0001] The present disclosure relates to a driving assistance device, a driving assistance processing method, and a recording medium.

[0002] In recent years, practical application of driving assistance devices capable of performing autonomous driving of vehicles has been progressing. When autonomous driving is performed by a driving assistance device, a driver is required to perform driving operations in an emergency that exceeds the limits of the execution conditions. For example, during execution of tracking control that automatically drives a vehicle to follow a preceding vehicle, if environmental conditions occur that reduce the detection accuracy of a sensor such as a camera that detects the preceding vehicle, it becomes difficult to cause the vehicle to follow the preceding vehicle, and the driver must switch to driving operations. In such cases, if the driver places too much trust in the driving assistance device, the driver's reaction may be delayed, putting the vehicle in danger.

[0003] In response to this, Patent Document 1 discloses a technology that recognizes at least one of the driver's reaction delay time to changes in the vehicle's external environment, the driver's reaction delay time to an alarm, and the driver's non-driving activity time, and determines whether the driver is in a state of system overconfidence based on the recognition result.If it is determined that the driver is in a state of system overconfidence, an alert is output, thereby eliminating the driver's state of system overconfidence during automated driving.

[0004] Furthermore, Patent Document 2 discloses a technology that determines the degree of dependency based on the proportion of times the service brake is actuated by the brake actuator during multiple actuations of the brake, sets the braking start time and braking force of the brake depending on the vehicle driving conditions such as vehicle speed and the degree of dependency, and drives the brake actuator so that the brake operates with the set braking force after the braking start time, thereby preventing the driver from becoming overly dependent on a driving assistance device.

[0005] JP 2020-91790 A JP 2004-090789 A

[0006] However, the technology described in Patent Document 1 is configured to output a visual, audio, or vibration alert when the driver is in a state of system overconfidence. The technology described in Patent Document 1 simply outputs an alert to make the driver aware of the state of system overconfidence, and is not configured to prompt the driver to improve preparatory actions for switching from automated driving to manual driving depending on the degree of the driver's system overconfidence.

[0007] Furthermore, the technology described in Patent Document 2 sets control parameters for a driving assistance function according to the driver's degree of dependency on the driving assistance device. Therefore, although the technology described in Patent Document 2 is configured to ensure vehicle safety using the driving assistance function, it is not configured to encourage the driver to improve preparatory actions for switching from automated driving to manual driving according to the driver's degree of dependency on the driving assistance device.

[0008] The present disclosure has been made in consideration of the above-mentioned problems, and an object of the present disclosure is to provide a driving assistance device, a driving assistance processing method, and a recording medium that are capable of encouraging an improvement in preparatory actions for switching from automated driving to manual driving, depending on the degree of overconfidence of the driver, when the driver is overconfident in the driving assistance function.

[0009] In order to solve the above problem, according to one aspect of the present disclosure, there is provided a driving assistance device that assists a driver in driving a vehicle, the driving assistance device comprising one or more processors and one or more memories configured to be able to communicate with the one or more processors, wherein the one or more processors perform the following: a confidence state determination process that determines a driver's overconfidence state regarding the execution of a predetermined driving assistance function; a function limit determination process that determines a function limit state that indicates the degree to which safety is not guaranteed by the execution of the predetermined driving assistance function; and a display process that, when the driver is in the overconfidence state, displays a virtual image in accordance with the overconfidence state to make the driver aware that the function limit state is in a state greater than a predetermined limit standard.

[0010] Furthermore, in order to solve the above-mentioned problems, according to another aspect of the present disclosure, there is provided a driving assistance processing method for assisting a driver in driving a vehicle, in which a computer determines a driver's overconfidence state regarding the execution of a predetermined driving assistance function, determines a functional limit state indicating the degree to which safety is not guaranteed by the execution of the predetermined driving assistance function, and, when the driver is in the overconfidence state, displays a virtual image in accordance with the overconfidence state to make the driver recognize that the functional limit state is in a state greater than a predetermined limit standard.

[0011] In addition, according to another aspect of the present disclosure, in order to solve the above problem, there is provided a non-transitory tangible recording medium having recorded thereon a computer program that executes the following: determining, by a computer, a driver's state of overconfidence regarding the execution of a predetermined driving assistance function; determining a functional limit state that indicates the degree to which safety is not guaranteed by the execution of the predetermined driving assistance function; and, when the driver is in the overconfident state, displaying a virtual image in accordance with the overconfidence state to make the driver recognize that the functional limit state is in a state greater than a predetermined limit standard.

[0012] As described above, according to the present disclosure, when a driver is overconfident in driving assistance functions, the driver can be encouraged to improve preparatory actions for switching from automated driving to manual driving depending on the degree of overconfidence.

[0013] FIG. 1 is a schematic diagram showing an example of the configuration of a vehicle equipped with a driving assistance device according to an embodiment of the present disclosure. FIG. 2 is a block diagram showing an example of the configuration of the driving assistance device according to the same embodiment. FIG. 3 is a flowchart showing a routine of overconfidence state determination processing by the driving assistance device according to the same embodiment. FIG. 4 is an explanatory diagram showing examples of evaluation items for determining a functional limit state by the driving assistance device according to the same embodiment. FIG. 5 is an explanatory diagram showing an example of a method for calculating a functional limit occurrence probability by the driving assistance device according to the same embodiment. FIG. 6 is an explanatory diagram showing an example of a method for determining a driver's overconfidence state by the driving assistance device according to the same embodiment. FIG. 7 is a flowchart showing a routine of processing operations by the driving assistance device according to the same embodiment. FIG. 8 is a flowchart showing a routine of display processing by the driving assistance device according to the same embodiment. FIG. 9 is an explanatory diagram showing a pattern of a safety requirement degree calculated by the driving assistance device according to the same embodiment. FIG. 10 shows an example of a virtual image display displayed by the driving assistance device according to the same embodiment. FIG. 11 shows another example of a virtual image display displayed by the driving assistance device according to the same embodiment. FIG. 12 shows an example of a virtual image display displayed by a driving assistance device according to a second embodiment of the present disclosure.

[0014] Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In this specification and drawings, components having substantially the same functional configurations are designated by the same reference numerals, and redundant description will be omitted.

[0015] In the embodiments described below, "autonomous driving" includes not only fully autonomous driving in which a computer performs all of the vehicle's driving operations, but also driving assistance functions in which a computer performs some of the vehicle's driving operations and driving assistance functions in which a computer temporarily intervenes in the vehicle's driving operations.

[0016] <<1. First embodiment>> <1-1. Overall configuration of vehicle> First, an example of the overall configuration of a vehicle equipped with a driving assistance device according to an embodiment of the present disclosure will be described.

[0017] 1 is a schematic diagram showing an example configuration of a vehicle 1. The vehicle 1 is configured as a two-wheel-drive four-wheel vehicle in which a driving torque output from a driving force source 9 that generates driving torque is transmitted to a left front wheel and a right front wheel. The driving force source 9 may be an internal combustion engine such as a gasoline engine or a diesel engine, a driving motor, or both an internal combustion engine and a driving motor.

[0018] Vehicle 1 may be a four-wheel drive vehicle that transmits drive torque to the front and rear wheels. Vehicle 1 may also be an electric vehicle equipped with two drive motors, for example, a front-wheel drive motor and a rear-wheel drive motor, or an electric vehicle equipped with drive motors corresponding to the respective wheels. If vehicle 1 is an electric vehicle or hybrid electric vehicle, vehicle 1 is equipped with a secondary battery that stores power supplied to the drive motors, and a motor or a generator such as a fuel cell that generates power to charge the battery.

[0019] The vehicle 1 is equipped with a driving force source 9, an electric steering device 15, and brake devices 17LF, 17RF, 17LR, and 17RR (hereinafter collectively referred to as "brake devices 17" unless a distinction is required) as devices used to control the operation of the vehicle 1. The driving force source 9 outputs driving torque that is transmitted to the front-wheel drive shaft 5F via a transmission and a differential mechanism 7 (not shown). The operation of the driving force source 9 and the transmission is controlled by a vehicle control unit 41 that includes one or more electronic control units (ECUs: Electronic Control Units).

[0020] The front-wheel drive shaft 5F is provided with an electric steering device 15. The electric steering device 15 includes an electric motor and a gear mechanism (not shown), and is controlled by a vehicle control unit 41 to adjust the steering angle of the front wheels. During manual driving, the vehicle control unit 41 controls the electric steering device 15 based on the steering angle of the steering wheel 13 operated by the driver. During automatic driving, the vehicle control unit 41 controls the electric steering device 15 based on a set steering angle or steering angular velocity.

[0021] Brake devices 17LF, 17RF, 17LR, and 17RR apply braking force to the respective wheels. Brake devices 17 are configured as, for example, hydraulic brake devices, and vehicle control unit 41 adjusts the hydraulic pressure supplied to each brake device 17 by controlling the drive of hydraulic unit 16. If vehicle 1 is an electric vehicle or a hybrid electric vehicle, brake devices 17 are used in combination with regenerative braking using a drive motor.

[0022] The vehicle control unit 41 includes one or more electronic control devices that control the driving of the driving force source 9, the electric steering device 15, and the hydraulic unit 16. If the vehicle 1 is equipped with a transmission that changes the speed of the output from the driving force source 9 and transmits it to the wheels 3, the vehicle control unit 41 has a function of controlling the driving of the transmission. The vehicle control unit 41 is configured to be able to acquire information transmitted from the driving assistance device 50, and is configured to be able to execute automatic driving control of the vehicle 1.

[0023] The vehicle 1 also includes front-facing cameras 31LF and 31RF, a vehicle state sensor 33, a GNSS (Global Navigation Satellite System) sensor 35, an interior camera 37, and a notification device 43.

[0024] The front imaging cameras 31LF, 31RF constitute ambient environment sensors for acquiring information about the ambient environment of the vehicle 1. The front imaging cameras 31LF, 31RF capture images of the area ahead of the vehicle 1 and generate image data. The front imaging cameras 31LF, 31RF are equipped with imaging elements such as CCDs (Charged Coupled Devices) or CMOS (Complementary Metal Oxide Semiconductors), and transmit the generated image data to the driving assistance device 50. In the vehicle 1 shown in FIG. 1 , the front imaging cameras 31LF, 31RF are configured as stereo cameras including a pair of left and right cameras, but the front imaging cameras may also be monocular cameras.

[0025] In addition to the front-facing cameras 31LF, 31RF, the surrounding environment sensor may include, for example, a camera that captures an image behind the vehicle 1, or a camera that is provided on a side mirror and captures an image behind the left or right of the vehicle 1. In addition, the surrounding environment sensor may include one or more sensors selected from the group consisting of a radar sensor such as a light detection and ranging (LiDAR) or a millimeter-wave radar, and an ultrasonic sensor.

[0026] The vehicle state sensor 33 is composed of at least one sensor that detects the operating state and behavior of the vehicle 1. The vehicle state sensor 33 includes, for example, at least one of a steering angle sensor, an accelerator position sensor, a brake stroke sensor, a brake pressure sensor, or an engine rotation speed sensor. The vehicle state sensor 33 also includes, for example, at least one of a vehicle speed sensor, an acceleration sensor, or an angular velocity sensor. The vehicle state sensor 33 transmits a sensor signal indicating the detected information to the driving assistance device 50.

[0027] The GNSS sensor 35 receives satellite signals from positioning satellites such as GPS (Global Positioning System) satellites. The GNSS sensor 35 transmits position information of the vehicle 1 contained in the received satellite signals to the driving assistance device 50. Note that the GNSS sensor 35 may be provided with an antenna, in addition to the GPS sensor, that receives satellite signals from other satellite systems that identify the position of the vehicle 1.

[0028] The interior camera 37 captures images of the interior of the vehicle and generates image data. The interior camera 37 is equipped with an imaging element such as a CCD or CMOS, and transmits the generated image data to the driving assistance device 50. The interior camera 37 is installed in a position where it can capture an image of at least the face of the driver.

[0029] The notification device 43 is driven by the driving assistance device 50 and notifies the driver of various information by means of image display, audio output, etc. The notification device 43 includes, for example, a display device provided in the instrument panel and a speaker provided in the vehicle 1. The display device may be a display device that displays information from a navigation system. The notification device 43 may also include a HUD (head-up display) that displays information on the front window superimposed on the scenery around the vehicle 1.

[0030] <1-2. Driving Assistance Device> Next, the driving assistance device 50 according to this embodiment will be described in detail.

[0031] (1-2-1. Configuration Example) The driving assistance device 50 functions as a device that assists in driving a vehicle by having one or more processors, such as CPUs (Central Processing Units), execute a computer program. The computer program is a computer program that causes the processor to execute the operations, described below, that should be performed by the driving assistance device 50. The computer program executed by the processor may be recorded on a recording medium that functions as a storage unit (memory) 53 provided in the driving assistance device 50, or may be recorded on a recording medium built into the driving assistance device 50 or any recording medium that can be externally attached to the driving assistance device 50.

[0032] The recording medium for recording a computer program may be a magnetic medium such as a hard disk, a floppy disk, or a magnetic tape; an optical recording medium such as a CD-ROM (Compact Disc Read Only Memory), a DVD (Digital Versatile Disc), or a Blu-ray (registered trademark); a magneto-optical medium such as a floptical disk; a memory element such as a RAM or a ROM; a flash memory such as a USB (Universal Serial Bus) memory or an SSD (Solid State Drive); or any other medium capable of storing a program.

[0033] 2 is a block diagram showing an example of the configuration of a driving assistance device 50 according to this embodiment. A surrounding environment sensor 31, a vehicle state sensor 33, and a GNSS sensor 35 are connected to the driving assistance device 50 via a dedicated line or communication means such as a controller area network (CAN) or local internet (LIN). A vehicle control unit 41 and a notification device 43 are also connected to the driving assistance device 50.

[0034] The driving assistance device 50 is not limited to an electronic control device mounted on the vehicle 1, but may also be a terminal device such as a smartphone or a wearable device.

[0035] The driving assistance device 50 includes a processing unit 51, a storage unit 53, a map data storage unit 55, and a driver information storage unit 57. The processing unit 51 is configured to include one or more processors such as a CPU and various peripheral components. Part or all of the processing unit 51 may be configured with updatable firmware or the like, or may be a program module or the like that is executed in response to a command from the CPU or the like.

[0036] (Storage Unit) The storage unit 53 is composed of one or more storage elements such as RAM or ROM connected to the processing unit 51 so as to be able to communicate with it. However, the type and number of storage units 53 are not particularly limited. The storage unit 53 stores information such as computer programs executed by the processing unit 51, various parameters used in arithmetic processing, detection data, and calculation results. A part of the storage unit 53 is used as a work area for the processing unit 51.

[0037] (Map Data Storage Unit) The map data storage unit 55 is configured by a storage element such as RAM or ROM communicably connected to the processing unit 51, or a storage medium such as an HDD, CD, DVD, SSD, USB flash, or storage device. The map data stored in the map data storage unit 55 is configured to be able to identify the position of the vehicle 1 based on the position information detected by the GNSS sensor 35. For example, the map data is associated with latitude and longitude information, and the processing unit 51 can identify the position of the vehicle 1 on the map data based on the latitude and longitude information of the vehicle 1 detected by the GNSS sensor 35.

[0038] (Driver Information Storage Unit) The driver information storage unit 57 is configured by a storage element such as a RAM or a ROM that is communicatively connected to the processing unit 51, or a storage medium such as an HDD, CD, DVD, SSD, USB flash, or storage device. The driver information storage unit 57 stores identification information of the driver of the vehicle 1 as well as information on the driver's reliability with respect to the driving assistance function of the driving assistance device 50.

[0039] (1-2-2. Functional Configuration of Processing Unit) Next, the functional configuration of the processing unit 51 of the driving assistance device 50 will be described. The processing unit 51 includes an acquisition unit 61, a surrounding environment detection unit 63, a driving assistance processing unit 65, a trust state determination unit 67, a function limit determination unit 69, and a display processing unit 71. Each of these units is a function realized by execution of a computer program by one or more processors such as a CPU. However, part of the acquisition unit 61, the surrounding environment detection unit 63, the driving assistance processing unit 65, the trust state determination unit 67, the function limit determination unit 69, and the display processing unit 71 may be configured using analog circuits.

[0040] (Acquisition Unit) The acquisition unit 61 acquires information or messages transmitted from the surrounding environment sensor 31, the vehicle state sensor 33, and the GNSS sensor 35.

[0041] (Surrounding environment detection unit) The surrounding environment detection unit 63 detects the surrounding environment of the vehicle 1. The surrounding environment detection unit 63 detects the surrounding environment, such as other vehicles and obstacles present around the vehicle 1, and road boundaries, based on image data or detection information transmitted from the surrounding environment sensor 31. Furthermore, the surrounding environment detection unit 63 detects the surrounding environment, such as the shape of the road around the vehicle 1, based on the position information and map data of the vehicle 1 transmitted from the GNSS sensor 35.

[0042] (Driving Assistance Processing Unit) The driving assistance processing unit 65 executes a driving assistance function that temporarily and automatically controls part or all of the driving operation of the vehicle 1, or the driving operation of the vehicle 1. Examples of the driving assistance function include automatic driving (fully automatic driving) that controls the acceleration / deceleration and steering angle of the vehicle 1 to automatically drive the vehicle 1 to a destination, following control that controls the acceleration / deceleration of the vehicle 1 to cause the vehicle 1 to follow a preceding vehicle, lane keeping control that controls the steering angle of the vehicle 1 to prevent the vehicle 1 from going out of a boundary line, emergency braking control that generates braking force for the vehicle 1 in an emergency to avoid a collision between the vehicle 1 and an obstacle or to mitigate the impact of a collision, and automatic steering control that controls the steering angle of the vehicle 1 in an emergency to avoid a collision between the vehicle 1 and an obstacle or to mitigate the impact of a collision, but other driving assistance functions may also be executed.

[0043] The driving assistance processing unit 65 controls the acceleration / deceleration, braking force, or steering angle of the vehicle 1 based on information about the surrounding environment of the vehicle 1 detected by the surrounding environment detection unit 63, and executes the driving assistance function described above. However, if the safety of the vehicle 1 due to the driving assistance function is not guaranteed, such as when the detection accuracy of the surrounding environment sensor 31 is reduced or when predetermined surrounding environment conditions are not met, execution of some or all of the driving assistance function by the driving assistance processing unit 65 is stopped. In this case, some or all of the driving operation is switched to driving operation by the driver of the vehicle 1.

[0044] (Trust State Determination Unit) The trust state determination unit 67 determines the driver's overconfidence state regarding the execution of the driving assistance processing functions executed by the driving assistance processing unit 65. The trust state determination unit 67 determines the driver's overconfidence state regarding the driving assistance functions based on the driver's behavior when the driving assistance functions are executed. The trust state determination unit 67 may determine the driver's overconfidence state for each of the driving assistance functions exemplified above. In this embodiment, the trust state determination unit 67 determines the driver's overconfidence state regarding the driving assistance functions in three stages: "high," "medium," and "low." However, the index representing the overconfidence state is not limited to the above example.

[0045] (Function Limit Determination Unit) The function limit determination unit 69 determines a function limit state indicating the degree to which safety is not ensured by the execution of the driving assistance processing function executed by the driving assistance processing unit 65. The functional limits of the driving assistance processing include, for example, functional limits caused by sensors for detecting the surrounding environment. For example, during tracking control in which the vehicle 1 follows a preceding vehicle, if the inter-vehicle distance from the preceding vehicle is too short, the front-facing imaging cameras 31LF, 31RF may not be able to recognize the preceding vehicle. Furthermore, if the preceding vehicle deviates from the recognition range of the front-facing imaging cameras 31LF, 31RF due to a sharp curve during tracking control, the vehicle 1 may suddenly accelerate to a set target vehicle speed and deviate from the road. Furthermore, if the vehicle 1 changes lanes during tracking control and there is no preceding vehicle in the new driving lane, the vehicle 1 may suddenly accelerate to the set target vehicle speed.

[0046] In addition, there are functional limits due to a decrease in the detection accuracy of the sensors due to weather, road conditions, or traffic conditions. The functional limit determination unit 69 determines the functional limit state based on the status and detection conditions of the surrounding environment sensor 31, the vehicle state sensor 33, and the GNSS sensor 35, as well as information on the running state of the vehicle 1 and the traffic environment around the vehicle 1. In this embodiment, the functional limit determination unit 69 determines the functional limit state in three stages: "high," "medium," and "low." However, the index representing the functional limit state is not limited to the above example.

[0047] (Display Processing Unit) When the driver is in an overconfident state, the display processing unit 71 displays a virtual image in accordance with the driver's overconfidence state to make the driver recognize that the functional limit state of the driving assistance function is in a state greater than a predetermined limit standard. The virtual image displayed by the display processing unit 71 is not a notification by text, voice, or sound, but rather displays an image relating to the situation around the vehicle 1. The display processing unit 71 does not directly reflect the actual situation around the vehicle in the display image, but rather varies the display content depending on the driver's overconfidence state so that the driver can intuitively recognize that the vehicle 1 is in a dangerous state. The display processing unit 71 displays information such that the driver recognizes that the vehicle 1 is in a more dangerous state the more overconfident the driver is.

[0048] (1-2-3. Operation of Driving Assistance Device) Next, an example of the processing operation by the processing unit 51 of the driving assistance device 50 according to this embodiment will be described with reference to a flowchart. Below, a flowchart of the overconfidence state determination process by the confidence state determination unit 67 will be described, followed by a flowchart of the display process.

[0049] In this embodiment, an example will be described in which the driving assistance function executes automatic driving control to drive the vehicle 1 to a destination by automatic driving. In the automatic driving control, the driving assistance processing unit 65 sets the acceleration / deceleration rate and steering angular velocity of the vehicle 1, and causes the vehicle 1 to travel at a target speed along a set driving route while avoiding collisions with obstacles. Furthermore, when there is a leading vehicle, the driving assistance processing unit 65 causes the vehicle 1 to follow the leading vehicle while maintaining a distance according to the speed of the vehicle 1.

[0050] 3 shows a flowchart of the overconfident state determination process by the processing unit 51. The overconfident state determination process may be executed continuously after the system of the vehicle 1 is started, or may be executed at any timing.

[0051] First, the trust state determination unit 67 records identification information of the driver of the vehicle 1 (step S1). Specifically, the trust state determination unit 67 extracts facial feature points of the driver based on image data transmitted from the in-vehicle camera 37 and identifies the driver. The trust state determination unit 67 also assigns identification information to each identified driver and records the assigned identification information. The identification of the driver is not limited to the example performed based on image data. For example, the trust state determination unit 67 may identify the driver based on the driver's name, nickname, etc. input by the driver himself / herself via a touch panel or microphone.

[0052] Next, the driving assistance processing unit 65 activates (turns on) the driving assistance function (step S3). The driving assistance processing unit 65 may activate the driving assistance function in response to an input operation by the driver, or may activate the driving assistance function together with the startup of the system of the vehicle 1. The driving assistance processing unit 65 may activate each driving assistance function under appropriate conditions. In the example of this embodiment, the driving assistance processing unit 65 activates the automatic driving control when the driver performs an input operation to start the execution of the automatic driving control.

[0053] Next, the function limit determination unit 69 determines the function limit state of the driving assistance function (step S5). In this embodiment, the function limit determination unit 69 evaluates each of the preset evaluation items and determines the function limit state based on the evaluation results.

[0054] 4 is an explanatory diagram showing examples of evaluation items for determining the functional limit state when automatic driving control is executed as a driving assistance function. The functional limit determination unit 69 determines the functional limit state based on seven evaluation items.

[0055] First, the function limit determination unit 69 determines whether the radius of curvature of the curve on the road on which the vehicle 1 is to travel exceeds a preset threshold. If the radius of curvature of the road curve is too large, the vehicle 1 may not be able to make the turn and may deviate from the road. The radius of curvature of the curve may be estimated based on information about the surrounding environment detected by the surrounding environment sensor 31, or may be obtained from the map data storage unit 55, which stores data on the radius of curvature of the road based on the position information of the vehicle 1.

[0056] The threshold value (m) is set in advance as the value of the radius of curvature at which it becomes difficult for the vehicle 1 to turn the curve. The threshold value (m) may be set according to a set target vehicle speed or the current vehicle speed of the vehicle 1. If the radius of curvature of the curve on the road on which the vehicle is to travel exceeds the preset threshold value, the function limit determination unit 69 further calculates the degree of deviation Fa (%). The degree of deviation Fa indicates the ratio of the difference between the actual radius of curvature of the road and the threshold value, divided by the threshold value.

[0057] Second, the function limit determination unit 69 determines whether the number of boundary line recognitions is less than two. That is, the function limit determination unit 69 determines whether the surrounding environment sensor 31 has recognized boundary lines on both the left and right sides of the driving lane in which the vehicle 1 is traveling. If the boundary lines are not recognized, there is a risk that the vehicle 1 will deviate from the driving lane. For example, whether the front-viewing cameras 31LF, RF can correctly recognize boundary lines may vary depending on weather and sunlight conditions, the clarity of boundary lines painted on the road, poor recognition due to obstacles such as fallen leaves, etc.

[0058] If the number of recognized boundary lines is less than 2, the functional limit determination unit 69 further calculates a deviation degree Fb (%). The deviation degree Fb indicates the ratio obtained by subtracting the number of recognized boundary lines from 2 and dividing the difference by 2. In other words, the deviation degree Fb indicates 50% or 100%.

[0059] Third, the function limit determination unit 69 determines whether the visibility rate is below a preset threshold as the object recognition status. That is, the function limit determination unit 69 determines the recognition status of surrounding vehicles, obstacles, etc. by the surrounding environment sensor 31. If other vehicles, including a preceding vehicle, and obstacles, etc. are not correctly recognized, there is a risk of colliding with other vehicles or other obstacles, or deviating from the driving lane. For example, the recognition status of the surrounding environment recognized by the front-viewing cameras 31LF, RF may change due to poor visibility caused by weather such as heavy rainfall or snowfall, backlighting caused by the setting sun, etc. The visibility rate may be calculated using known technology.

[0060] The threshold value (%) of the visibility assurance rate is set in advance as an allowable range of the accuracy of object recognition. If the visibility assurance rate is less than the preset threshold value, the function limit determination unit 69 further calculates a deviation degree Fc (%). The deviation degree Fc indicates the ratio obtained by subtracting the calculated visibility assurance rate from the threshold value and dividing the difference by the threshold value.

[0061] Fourth, the function limit determination unit 69 determines whether the number of satellites with which communication is possible is less than two, as the recognition status of the GNSS sensor 35. In other words, the function limit determination unit 69 determines the accuracy of the position information of the vehicle 1 in the map data detected by the GNSS sensor 35. If the accuracy of the position information of the vehicle 1 in the map data is low, there is a risk that the vehicle 1 will not be able to travel along the travel route. For example, whether the front imaging cameras 31LF, RF can correctly recognize boundary lines may vary depending on weather and sunlight conditions, the clarity of boundary lines painted on the road, poor recognition due to obstacles such as fallen leaves, etc.

[0062] If the number of recognized boundary lines is less than 2, the functional limit determination unit 69 further calculates the deviation degree Fd (%). The deviation degree Fd indicates the ratio obtained by subtracting the number of recognized boundary lines from 2 and dividing the difference by 2. In other words, the deviation degree Fd indicates 50% or 100%.

[0063] Fifth, the functional limit determination unit 69 determines whether the probability of a functional limit state (functional limit event occurrence probability) caused by another vehicle exceeds a threshold value as the status of the surrounding vehicles. In other words, the functional limit determination unit 69 determines whether a situation exists in which a collision is highly likely to be unavoidable even if the vehicle 1 attempts to avoid the collision by steering or applies the emergency brake, such as when another vehicle cuts in front of the vehicle 1 within a short distance. The functional limit determination unit 69 calculates the functional limit event occurrence probability based on information such as the presence, relative positions, and behavior of the preceding vehicle, surrounding vehicles, bicycles, pedestrians, etc., detected by the surrounding environment sensor 31. In this case, the more surrounding moving objects there are and the more moving objects moving in the direction of travel of the vehicle 1, the higher the functional limit event occurrence probability. The calculation method for the functional limit event occurrence probability may be set in advance using any method.

[0064] The threshold value (%) of the probability of occurrence of a functional limit event is set in advance. If the probability of occurrence of a functional limit event exceeds the preset threshold value, the functional limit determination unit 69 further calculates a deviation degree Fe (%). The deviation degree Fe indicates the ratio obtained by dividing the difference between the calculated probability of occurrence of a functional limit event and the threshold value by the threshold value.

[0065] Sixth, the function limit determination unit 69 determines whether the speed of the vehicle 1 exceeds a limit speed as the status of the vehicle itself. For example, if the driver depresses the accelerator while autonomous driving control is being executed and the vehicle speed increases to exceed a predetermined threshold, a collision may not be avoided even if the driver operates the steering wheel to avoid a collision or applies the emergency brake. The limit speed may be a preset value or may be a value set according to the width of the road and the status of surrounding moving objects.

[0066] If the speed of the vehicle 1 exceeds the limit speed, the function limit determination unit 69 further calculates the degree of deviation Ff (%). The degree of deviation Ff indicates the ratio of the difference between the vehicle speed and the limit speed divided by the limit speed.

[0067] Seventh, the functional limit determination unit 69 determines whether or not there is a road condition such as an intersection or lane merging point where the vehicle 1 and another vehicle merge or intersect (a merging event). In a situation where the vehicle 1 and another vehicle merge or intersect, the vehicle 1 must be driven while taking into consideration the driving conditions of the other vehicles, and a situation that cannot be avoided by autonomous driving control may occur. The functional limit determination unit 69 may determine whether or not there is a merging event based on information about the surrounding environment detected by the surrounding environment sensor 31, or may determine whether or not there is a merging event based on position information about the vehicle 1 and road structure data stored in the map data storage unit 55.

[0068] The functional limit determination unit 69 performs an evaluation and calculates the degree of deviation for each evaluation item, and then calculates the probability of occurrence of a functional limit based on the evaluation result and the degree of deviation for each evaluation item.

[0069] 5 is an explanatory diagram showing an example of a method for calculating the functional limit occurrence probability. Based on the evaluation results and deviation degrees of each evaluation item, the functional limit determination unit 69 determines the functional limit occurrence probability to be "high" if two or more evaluation items are met or if there is at least one item whose deviation degree is equal to or greater than a first threshold value (e.g., 70%). In this embodiment, if the functional limit occurrence probability is "high," the safety requirement level is assigned "10 points."

[0070] Furthermore, based on the evaluation results and the degree of deviation for each evaluation item, if there is at least one evaluation item that corresponds to the evaluation item or if there is at least one item whose degree of deviation is equal to or greater than the second threshold value (e.g., 30%) and less than the first threshold value (e.g., 70%), the functional limit determination unit 69 determines the probability of functional limit occurrence to be "medium." In this embodiment, if the probability of functional limit occurrence is "medium," the safety requirement level is assigned "5 points."

[0071] Furthermore, based on the evaluation results and the degree of deviation of each evaluation item, if there is at least one item whose degree of deviation is equal to or greater than a third threshold value (e.g., 1%) and less than a second threshold value (e.g., 30%), the functional limit determination unit 69 determines the probability of functional limit occurrence to be "low." In this embodiment, if the probability of functional limit occurrence is "low," the point for the safety requirement degree is set to "1 point."

[0072] The methods for determining the functional limit state shown in Figures 4 and 5 are merely examples. For example, the functional limit occurrence probability may be calculated by weighting the evaluation items. The evaluation items may be set or omitted as appropriate depending on the driving assistance function and the type of sensor used. Furthermore, the functional limit occurrence probability may be divided into more levels than the three levels of "high," "medium," and "low."

[0073] Next, the function limit determination unit 69 determines whether or not a function limit state of the driving support function has occurred (step S7). If the result of the function limit state determination process in step S5 indicates that the function limit occurrence probability is at least "small" or higher, the function limit determination unit 69 determines that a function limit state of the driving support function has occurred.

[0074] If the function limit determination unit 69 does not determine that the function limit state of the driving assistance function has occurred (S7 / No), the process returns to step S5 and the determination of the function limit state is repeated. On the other hand, if the function limit determination unit 69 determines that the function limit state of the driving assistance function has occurred (S7 / Yes), the trust state determination unit 67 determines the driver's overconfidence in the driving assistance function based on the driver's preparatory behavior when the function limit state of the driving assistance function has occurred (step S9).

[0075] 6 is an explanatory diagram showing an example of a method for determining the driver's overconfidence state. In the example shown in Fig. 6, the trust state determination unit 67 determines the driver's overconfidence state based on the driver's brake preparatory behavior when a functional limit state of the driving assistance function occurs. Specifically, after it is determined in step S7 that a functional limit state of the driving assistance function has occurred, the trust state determination unit 67 classifies the level of the driver's preparatory behavior based on the timing until the brake pedal is depressed, and determines the overconfidence state.

[0076] More specifically, if the time between when it is determined that a functional limit state of the driving assistance function has occurred and when the driver presses the brake pedal is, for example, 1 to 2 seconds, the trust state determination unit 67 determines that the brake preparation timing is somewhat late and classifies the driver's overconfidence as "low." Furthermore, if the time between when it is determined that a functional limit state of the driving assistance function has occurred and when the driver presses the brake pedal is, for example, 2 to 3 seconds, the trust state determination unit 67 determines that the brake preparation timing is late and classifies the driver's overconfidence as "medium." Furthermore, if the time between when it is determined that a functional limit state of the driving assistance function has occurred and when the driver presses the brake pedal exceeds, for example, 3 seconds, the trust state determination unit 67 determines that the brake preparation was not made and classifies the driver's overconfidence as "high."

[0077] In this embodiment, the points for the safety requirement level when the driver's overconfidence in the driving assistance functions is "high", "medium", or "low" are set to "10 points", "5 points", and "1 point", respectively.

[0078] Next, the trust state determination unit 67 associates information about the determined overconfidence state with the driver's identification information and stores it in the driver information storage unit 57 (step S11). The trust state determination unit 67 may always overwrite and update the information about the overconfidence state, or may store the determination result with the greatest overconfidence state among the most recent multiple determination results, or may store the determination result with the greatest frequency among the most recent multiple determination results.

[0079] The method of determining the overconfidence state shown in FIG. 6 is merely an example. The content of the preparatory action to be determined may be set appropriately depending on the driving assistance function. Moreover, the overconfidence state may be classified into more levels than the three levels of "high," "medium," and "low."

[0080] The trust state determination unit 67 repeats the above-described processing from step S5 to step S11 to determine and update the state of overconfidence in the driving assistance functions for each driver.

[0081] (Display Processing) FIG. 7 is a flowchart showing the operation of the display processing by the driving assistance device 50. First, the display processing unit 71 identifies the driver of the vehicle 1 (step S21). Specifically, the display processing unit 71 extracts feature points of the driver's face based on image data transmitted from the in-vehicle camera 37 and identifies the driver. The display processing unit 71 also specifies identification information assigned to the identified driver. The identification of the driver is not limited to the example in which it is performed based on image data. For example, the display processing unit 71 may identify the driver based on the driver's name, nickname, etc., input by the driver himself / herself via a touch panel or microphone.

[0082] Next, the driving assistance processing unit 65 activates (turns on) the driving assistance function (step S23), similarly to step S3 described above.

[0083] Next, the function limit determination unit 69 determines the function limit state of the driving support function in the same procedure as in step S5 described above (step S25). The function limit state is determined to be "high," "medium," or "low."

[0084] Next, the function limit determination unit 69 determines whether or not a function limit state of the driving assistance function has occurred, similarly to step S7 described above (step S27). If the result of the function limit state determination process in step S5 indicates that the function limit occurrence probability is at least "small" or greater, the function limit determination unit 69 determines that the function limit state of the driving assistance function has exceeded the limit criterion.

[0085] If the function limit determination unit 69 does not determine that the function limit state of the driving assistance function has occurred (S27 / No), the process returns to step S25 and repeats the determination of the function limit state. On the other hand, if the function limit determination unit 69 determines that the function limit state of the driving assistance function has occurred (S27 / Yes), the display processing unit 71 acquires information on the driver's overconfidence state with respect to the driving assistance function (step S29). Specifically, the display processing unit 71 acquires overconfidence state information stored in association with the identification information of the driver of the vehicle 1 from the overconfidence state information for each driver stored in the driver information storage unit 57. The driver's overconfidence state is identified as one of "high," "medium," "low," or "no overconfidence."

[0086] Next, the display processing unit 71 executes a display process to make the driver aware that the functional limit state of the driving assistance function has exceeded the limit standard, depending on the driver's level of overconfidence in the driving assistance function (step S31).

[0087] 8 is a flowchart showing the display process performed by the display processing unit 71 of this embodiment. The display processing unit 71 calculates the safety requirement degree based on the information on the functional limit state and the information on the driver's overconfidence state (step S41). As shown in FIGS. 5 and 6, in this embodiment, points for the safety requirement degree are set for each of the functional limit occurrence probability and the overconfidence state. The larger each point, the higher the safety requirement degree. The display processing unit 71 calculates the safety requirement degree by multiplying the point for the functional limit occurrence probability determined in step S25 by the point for the driver's overconfidence state identified in step S29.

[0088] 9 is an explanatory diagram showing patterns of calculated safety requirement degrees. In this embodiment, the display processing unit 71 determines that the safety requirement degree is "high" when the safety requirement degree obtained by multiplying the points for the overconfident state by the points for the probability of functional limit occurrence is 100. Furthermore, the display processing unit 71 determines that the safety requirement degree is "medium" when the safety requirement degree obtained by multiplying the points for the overconfident state by the points for the probability of functional limit occurrence is 50, 25, or 10. Furthermore, the display processing unit 71 determines that the safety requirement degree is "low" when the safety requirement degree obtained by multiplying the points for the overconfident state by the points for the probability of functional limit occurrence is 5 or 1.

[0089] Next, the display processing unit 71 drives the notification device 43 to display a virtual image according to the calculated safety requirement level (step S43).

[0090] Fig. 10 shows an example of a virtual image displayed by the display processing unit 71. The image displayed in Fig. 10 is an example of a display in a functional limit state where the radius of curvature of the curve exceeds a threshold value and the vehicle 1 may stray from the driving lane into the oncoming lane.

[0091] 10 , when the safety requirement level is “low,” the display processing unit 71 displays a virtual driving trajectory showing how a virtual vehicle 81 representing vehicle 1 deviates from the driving lane. When the safety requirement level is “medium,” the display processing unit 71 displays one virtual oncoming vehicle 91 in the oncoming lane along with the virtual driving trajectory showing how the virtual vehicle 81 deviates from the driving lane. When the safety requirement level is “high,” the display processing unit 71 displays two virtual oncoming vehicles 93 and 95 in the oncoming lane along with the virtual driving trajectory showing how the virtual vehicle 81 deviates from the driving lane.

[0092] In this way, the display processing unit 71 displays a virtual image in which the vehicle 1 is in a more dangerous state as the driver's state of overconfidence increases. In the example shown in Fig. 10, the display processing unit 71 displays an image of a virtual driving environment in which the vehicle 1 is in a more dangerous state as the driver's state of overconfidence increases. Specifically, the display processing unit 71 changes the display state of a virtual collision target object (oncoming vehicle) rather than an actual vehicle, depending on the driver's state of overconfidence. In the example shown in Fig. 10, the number of oncoming vehicles displayed is changed depending on the driver's state of overconfidence, but the display positions may be changed without changing the number of vehicles displayed to indicate that the vehicle 1 is in a more dangerous state.

[0093] In addition, in this embodiment, the display processing unit 71 displays an image of a virtual driving environment that indicates that the vehicle 1 is in a more dangerous state not only depending on the driver's overconfidence but also on the functional limit state of the driving assistance function. In the example shown in Fig. 10, the higher the functional limit state, the more oncoming vehicles are displayed. Therefore, the driver's overconfidence or the functional limit state increases, and the driver perceives the vehicle 1 as being in a dangerous state, which increases the sense of crisis or distrust regarding the current situation, and can encourage the driver to improve their driving behavior in preparation for dangerous situations.

[0094] Fig. 11 shows another example of a virtual image displayed by the display processing unit 71. The image displayed in Fig. 11 is an example of a display in a functional limit state where the radius of curvature of the curve exceeds a threshold value and the vehicle 1 may stray from the driving lane into the oncoming lane.

[0095] In the example shown in FIG. 11 , whether the safety requirement level is "high," "medium," or "low," the display processing unit 71 displays a virtual driving trajectory showing a virtual vehicle 81 representing the vehicle 1 deviating from the driving lane. However, the display processing unit 71 varies the deviation amount of the virtual driving trajectory of the virtual vehicle 81 from the driving lane depending on the driver's overconfidence. Specifically, the display processing unit 71 displays a larger deviation amount of the virtual driving trajectory from the driving lane as the safety requirement level increases. Therefore, the greater the driver's overconfidence or the greater the functional limit state, the more the driver perceives the vehicle 1 as being in a dangerous state, which increases the sense of crisis or distrust regarding the current situation, and this can encourage the driver to improve their driving behavior in preparation for dangerous situations.

[0096] 7 , after the display processing unit 71 executes the display process, the function limit determination unit 69 determines whether the driving support function has been turned off (step S33). If the function limit determination unit 69 does not determine that the driving support function has been turned off (S33 / No), the process returns to step S25, determines the functional limit state of the driving support function, and repeats the processing of each of the steps described above. On the other hand, if the function limit determination unit 69 determines that the driving support function has been turned off (S33 / Yes), the process ends.

[0097] As described above, the driving assistance device 50 according to this embodiment is configured to determine the driver's overconfidence regarding the execution of the driving assistance function, determine the functional limit state indicating the degree to which safety is not ensured by the execution of the driving assistance function, and, if the driver is in an overconfident state, display a virtual image in accordance with the overconfidence state to make the driver aware that the functional limit state is greater than a predetermined limit criterion. Therefore, when the functional limit state of the driving assistance function exceeds the limit criterion, the greater the driver's overconfidence, the more dangerous the vehicle 1 becomes, increasing the driver's sense of crisis or distrust regarding the current situation. As a result, the driver's overconfidence can be alleviated, leading to an improvement in the driver's ability to take preparatory action in preparation for dangerous situations.

[0098] Furthermore, the driving assistance device 50 according to this embodiment notifies the driver that the driving assistance function is in a functional limit state by displaying a virtual image indicating that the vehicle 1 is in a dangerous state, rather than by displaying a text or sound or voice warning. This allows the driver to intuitively recognize the current situation of the vehicle 1 and to quickly take preparatory action for a dangerous situation.

[0099] <<2. Second Embodiment>> Next, a driving assistance device according to a second embodiment of the present disclosure will be described. The driving assistance device according to the first embodiment is configured to display an image of a virtual driving environment that indicates that the vehicle 1 is in a more dangerous state as the driver's state of overconfidence increases. In the second embodiment, the display state of the virtual display of objects recognized by the surrounding environment sensor 31 is changed depending on the driver's state of overconfidence.

[0100] The driving assistance device according to this embodiment has a configuration similar to that of the driving assistance device 50 according to the first embodiment, except that the display content by the display processing unit 71 is different from that of the driving assistance device according to the first embodiment. Below, the differences between the driving assistance device according to this embodiment and the driving assistance device according to the first embodiment will be described.

[0101] Fig. 12 shows an example of a virtual image displayed by the display processing unit 71 in step S43 of the flowchart shown in Fig. 8. The image displayed in Fig. 12 is an example of a display in a functional limit state in which the detection accuracy of the surrounding environment sensor 31 for the preceding vehicle has decreased while the vehicle 1 is traveling following the preceding vehicle under automatic driving control of the vehicle 1.

[0102] 12 , the display processing unit 71 varies the ambiguity (resolution) of the outline of the virtual display 97 of the preceding vehicle depending on the driver's overconfidence. Specifically, the greater the degree of safety demand, the greater the ambiguity of the outline of the virtual display 97 of the preceding vehicle displayed by the display processing unit 71. Therefore, the greater the driver's overconfidence, the lower the accuracy of detection of the preceding vehicle by the surrounding environment sensor 31, which increases the driver's sense of crisis or distrust about the current situation, which can encourage the driver to improve their driving behavior in preparation for dangerous situations.

[0103] In the example shown in Fig. 12, the degree of vagueness of the contour of the virtual display of the recognized object to be displayed is varied depending on the driver's overconfidence level, but the method of varying the display depending on the driver's overconfidence level is not limited to the above example. For example, the display processing unit 71 may vary the density of the virtual display of the recognized object or the blinking interval of the display of the recognized object depending on the driver's overconfidence level. Even when displayed in this manner, the driver will perceive that the detection accuracy of the surrounding environment sensor 31 for the preceding vehicle is lower as the driver's overconfidence level increases, which increases the driver's sense of crisis or distrust about the current situation, and can encourage the driver to improve their driving behavior in preparation for dangerous situations.

[0104] As described above, the driving assistance device 50 according to this embodiment can also make the driver perceive the vehicle 1 as being in a more dangerous state when the functional limit state of the driving assistance function exceeds the limit standard, thereby increasing the driver's sense of crisis or distrust regarding the current situation. As a result, the driver's overconfidence can be alleviated, and the driver can be encouraged to take preparatory action for a dangerous situation. Furthermore, the driving assistance device 50 according to this embodiment also notifies the driver that the driving assistance function is in a functional limit state by using a virtual image display method that indicates that the vehicle 1 is in a dangerous state, so the driver can intuitively recognize the current situation of the vehicle 1 and quickly take preparatory action for a dangerous situation.

[0105] Although the preferred embodiments of the present disclosure have been described in detail above with reference to the accompanying drawings, the present disclosure is not limited to such examples. It is clear that a person skilled in the art to which the present disclosure pertains can conceive of various modifications or alterations within the scope of the technical ideas set forth in the claims, and it is understood that these also naturally fall within the technical scope of the present disclosure.

[0106] For example, in the above embodiment, when a virtual image is displayed indicating that the vehicle 1 is in a dangerous situation, the result of the determination of the driver's overconfidence state and the probability of the functional limit occurring of the driving assistance function may also be displayed. In the above embodiment, the result of the determination of the driver's overconfidence state and the probability of the functional limit occurring of the driving assistance function are determined using the same index (three levels), and the driver can recognize which element of the overconfidence state or the probability of the functional limit occurring indicates that the vehicle 1 is in a dangerous situation.

[0107] Furthermore, in the above embodiment, the process of assisting the driving of the vehicle is performed by a driving assistance device mounted on the vehicle 1, but some or all of the functions may be provided by an external management server.

[0108] In addition, the technology of the present disclosure can also be realized as a vehicle equipped with the driving assistance device described in the above embodiment, a driving assistance processing method using the driving assistance device, a computer program that causes a computer to function as the above driving assistance system, and a non-transitory tangible recording medium on which the computer program is recorded.

[0109] 1: Vehicle 50: Driving assistance device 51: Processing unit 53: Storage unit 55: Map data storage unit 57: Driver information storage unit 61: Acquisition unit 63: Surrounding environment detection unit 65: Driving assistance processing unit 67: Trust state determination unit 69: Functional limit determination unit 71: Display processing unit 81: Virtual vehicle 91: Oncoming vehicle 93: Oncoming vehicle 95: Oncoming vehicle 97: Virtual display

Claims

1. A driving assistance device that assists a driver in driving a vehicle, comprising one or more processors and one or more memories configured to communicate with the one or more processors, wherein the one or more processors execute a confidence state determination process that determines a driver's overconfidence state regarding the execution of a specified driving assistance function; a functional limit determination process that determines a functional limit state indicating the degree to which safety is not guaranteed by the execution of the specified driving assistance function; and a display process that, when the driver is in the overconfidence state, displays a virtual image in accordance with the overconfidence state to make the driver recognize that the functional limit state is greater than a specified limit standard.

2. The driving assistance device according to claim 1, wherein the one or more processors display a virtual image in which the vehicle is in a more dangerous state as the driver's state of overconfidence increases.

3. The driving assistance device according to claim 2, wherein the one or more processors display a virtual image in which the vehicle is in a more dangerous state as the driver's overconfidence increases and as the functional limit state of the driving assistance function increases.

4. The driving assistance device according to claim 1, wherein the one or more processors display an image of a virtual driving environment that indicates that the vehicle is in a more dangerous state as the driver's state of overconfidence increases.

5. The driving assistance device according to claim 1, wherein the one or more processors change a display state of a virtual display of an object recognized by an ambient environment sensor that detects the ambient environment of the vehicle depending on the driver's overconfidence state.

6. A driving assistance processing method for assisting a driver in driving a vehicle, the driving assistance processing method comprising: a computer determining a driver's overconfidence state regarding the execution of a specified driving assistance function; determining a functional limit state indicating the degree to which safety is not guaranteed by the execution of the specified driving assistance function; and, when the driver is in the overconfidence state, displaying a virtual image in accordance with the overconfidence state to make the driver recognize that the functional limit state is greater than a specified limit standard.

7. A non-transitory tangible recording medium having recorded thereon a computer program that executes the following: determining a driver's overconfidence state regarding the execution of a specified driving assistance function; determining a functional limit state indicating the degree to which safety is not guaranteed by the execution of the specified driving assistance function; and, when the driver is in the overconfidence state, displaying a virtual image in accordance with the overconfidence state to make the driver recognize that the functional limit state is greater than a specified limit standard.

Images