Vehicle information providing apparatus, vehicle information providing method, and storage medium

Abstract

The present application provides a vehicle information providing device, a vehicle information providing method, and a storage medium, which can improve the safety of traffic by seeking preventive safety by allowing a driver to recognize a specific object early. The vehicle information providing device includes a display device that allows a driver to visually recognize an image item, a detection unit that detects a specific object present in the vicinity of the host vehicle, a calculation unit that calculates a collision risk with respect to the detected specific object, and a display control unit that controls the display device to make the image item less observable as the calculated collision risk increases.

Description

TECHNICAL FIELD

[0001] The present application relates to a vehicle information providing apparatus, a vehicle information providing method, and a storage medium. BACKGROUND

[0002] In the field of traffic, countermeasures against environmental problems are becoming urgent. For example, when a driver drives a vehicle, by performing various kinds of driving support, the traffic of the vehicle is maintained smooth, and the amount of travel of the vehicle is reduced, whereby the reduction of the amount of discharge of CO2 and the like that lighten the load on the environment can be achieved. In addition, by providing the vehicle that performs the driving support to public transportation, it is easy to operate the public transportation under a stable schedule, and thus the convenience of the public transportation can be improved. By improving the convenience of the public transportation, the public transportation is enriched, and on the other hand, in order to make the convenience of the public transportation higher, for example, a traffic environment with high safety is sought to be constructed.

[0003] In order to construct a traffic environment with high safety, for example, in a case where there is a specific object that can collide with a pedestrian, another vehicle, or the like, the risk of collision with the vehicle becomes larger. In the past, in order to avoid the collision of the specific object with the vehicle, there is a technology that displays a warning according to the degree of risk, and the larger the risk, the more the warning is phasedly more noticeable (for example, refer to Japanese Patent Application Laid-Open No. 2015-187806). SUMMARY

[0004] PROBLEMS TO BE SOLVED BY THE INVENTION

[0005] However, in a case where the display of the warning is phasedly made noticeable, sometimes the characteristics of change blindness that is directed to the existence of continuity change act. Therefore, it is possible to have an influence on the early recognition of the specific object existing in the surroundings of the vehicle by the driver, and there is room for research in further improvement of the safety of traffic by seeking safety in prevention.

[0006] The present application is completed in consideration of such a situation, and one of the objects thereof is to provide a vehicle information providing apparatus, a vehicle information providing method, and a storage medium that can improve the safety of traffic by seeking safety in prevention by early recognition of a specific object by a driver.

[0007] MEANS FOR SOLVING THE PROBLEMS

[0008] The vehicle information providing apparatus, the vehicle information providing method, and the storage medium of the present application adopt the following structure.

[0009] (1): The vehicle information providing apparatus of one aspect of the present application includes: a display device that causes a driver to visually recognize an image item; an identification unit that identifies a specific object present in the vicinity of the host vehicle; a calculation unit that calculates a collision risk with respect to the specific object identified; and a display control unit that controls the display device to cause the observation ease of the image item to decrease as the collision risk calculated increases.

[0010] (2): In the aspect of (1) above, causing the observation ease of the image item to decrease includes at least any one of causing the brightness, lightness, chroma, resolution, sharpness, or contrast of the image item to decrease, reducing the display area of the image item, and simplifying the display of the image item.

[0011] (3): In the aspect of (1) or (2) above, the display control unit causes the observation ease of the image item to decrease while guiding the driver to visually recognize a visual recognition position of the image item toward a point on the line of sight at the time the driver visually recognized the specific object.

[0012] (4): In the aspect of (3) above, as the display device, there is included a first display device that causes an image to overlap with a landscape in front of the host vehicle for the driver to visually recognize, and a second display device that is disposed around the first display device and displays the image item, and the display control unit guides the visual recognition position of the image item displayed on the second display device toward the first display device.

[0013] (5): In any one of the aspects of (1) to (4) above, the vehicle information providing apparatus further includes a line-of-sight monitoring unit that monitors the orientation of the line of sight of the driver, and in a case where it is determined that the line of sight of the driver is directed toward the image item, the display control unit causes the observation ease of the image item to decrease.

[0014] (6): In the aspect of (5) above, in a case where it is determined that the line of sight of the driver is directed toward the specific object, the display control unit performs processing of suspending the control to cause the observation ease of the image item to decrease, and causes the driver to visually recognize the image item in a state before the control to cause the observation ease of the image item to decrease was performed.

[0015] (7): In the aspect of (6) above, the vehicle information providing apparatus further includes a load monitoring unit that monitors the load borne by the driver, and in a case where it is determined that the load borne by the driver is large, the display control unit easily starts the control to cause the observation ease of the image item to decrease.

[0016] (8): The vehicle information providing method of one aspect of the present application causes a computer to perform the following processing: recognizing a specific object present in the vicinity of the host vehicle; calculating a collision risk with respect to the recognized specific object; the greater the calculated collision risk, the lower the ease of observation of an image item visually recognized by the driver is controlled to be.

[0017] (9): The storage medium of one aspect of the present application stores a program, wherein the program causes a computer to perform the following processing: recognizing a specific object present in the vicinity of the host vehicle; calculating a collision risk with respect to the recognized specific object; the greater the calculated collision risk, the lower the ease of observation of an image item visually recognized by the driver is controlled to be.

[0018] Effects of Invention

[0019] According to the aspects of (1) to (9), it is possible to seek safety in prevention by causing the driver to recognize the specific object early, thereby improving the safety of traffic. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 is a diagram showing an example of the structure of the vehicle information providing apparatus 100.

[0021] Figure 2 is a diagram illustrating the structure of the vehicle room of the host vehicle M equipped with the vehicle information providing apparatus 100.

[0022] Figure 3 is a partial structural diagram of the first display device 110.

[0023] Figure 4 is a flowchart showing an example of the processing in the control section 180.

[0024] Figure 5 is a flowchart showing an example of the processing in the control section 180.

[0025] Figure 6 is a line graph showing an example of the change in the display position of the image and the ease of observation of the image with time.

[0026] Figure 7 is a diagram showing an example of the change in the image in the first display device 110 and the central display 141.

[0027] Figure 8 is a diagram showing an example of the change in the image in the first display device 110.

[0028] Figure 9 is a diagram showing an example of the change in the image in the first display device 110 and the electronic rearview mirror 142. DETAILED DESCRIPTION

[0029] Hereinafter, embodiments of the vehicle information providing device, vehicle information providing method, and storage medium of the present invention will be described with reference to the accompanying drawings. The vehicle information providing device of the embodiment is, for example, mounted on a vehicle. The vehicle information providing device includes, for example, a display device for displaying image entries. The vehicle information providing device reduces the ease of viewing image entries when the risk of collision with pedestrians around the vehicle M, a preceding vehicle, etc. is high. In the following description, the XYZ coordinate system is appropriately used to illustrate positional relationships, etc. In addition, in the following description, a "virtual image" is sometimes referred to as an "image."

[0030] Figure 1 This figure shows an example of the configuration of a vehicle information providing device 100. The vehicle information providing device 100 includes, for example, a first display device 110, a second display device 140, an object recognition system 150, an in-cabin camera 160, a vehicle sensor 170, and a control unit 180. The first display device 110 superimposes an image on the scenery in front of the vehicle M for visual recognition by the driver. The first display device 110 can be referred to as a HUD (Head Up Display) device.

[0031] Figure 2 This figure illustrates the interior structure of a vehicle M equipped with a vehicle information providing device 100. Hereinafter, the vehicle equipped with the vehicle information providing device 100 is referred to as the vehicle M. As an example, the vehicle information providing device 100 projects light containing an image onto the windshield 20 of the vehicle M, allowing a viewer to visually recognize a virtual image VI. The vehicle information providing device 100 allows the driver to visually recognize, for example, the vehicle speed, legal speed, and lane information of the vehicle M as the virtual image VI. A virtual image is an example of an image item.

[0032] The viewer is, for example, the driver, but may also be a passenger other than the driver. Furthermore, the display device may be implemented by projecting a real image onto a light-transmitting display device (e.g., a liquid crystal display, organic EL (Electroluminescence), or a transparent member (a mask, glasses, etc.) of a device worn on the body) mounted on the windshield 20 of the vehicle M. The display device may also be a device in which a light-transmitting display device is mounted on a device. In the following description, the display device is assumed to be a device mounted on the vehicle M that projects light containing an image onto the windshield 20.

[0033] In the host vehicle M, for example, a steering wheel 10 that controls steering of the host vehicle M, a front windshield 20 that separates an outside of the vehicle from an inside of the vehicle, and an instrument panel 30 are provided. The front windshield 20 is a member having light transmittance. A first display device 110 visually recognizes a virtual image VI for a driver seated on a seat in front of the steering wheel 10 by projecting light containing an image (performing light projection) to a displayable area provided to a part or all of the front windshield 20. In the following description, the virtual image is sometimes referred to as an "image".

[0034] In the host vehicle M, a display portion 50 can be provided in addition to the vehicle information providing apparatus 100. The display portion 50 is provided, for example, in the vicinity of the front of the seat of the driver in the instrument panel 30, and is a display device that the driver can visually recognize from a gap of the steering wheel 10 or over the steering wheel 10. In the following description, the display portion 50 is sometimes referred to as a "display device". Figure 2 In the example shown, in front of the host vehicle M, a pedestrian W and an oncoming vehicle MF are present to cross in front of the host vehicle M, and the driver can visually recognize the pedestrian W and the oncoming vehicle MF through the front windshield 20.

[0035] Figure 3 A partial structure of the first display device 110 is shown. The first display device 110, for example, houses a light projecting device 120, an optical mechanism 112, a plane mirror 113, a concave mirror 114, a light-transmissive cover 115, a lens position sensor 116, a concave mirror angle sensor 117, and an environmental sensor 118 in a housing 111.

[0036] The light projecting device 120, for example, has a light source 120A and a display element 120B. The light source 120A is, for example, a cold cathode tube, and outputs visible light corresponding to the virtual image VI that the driver visually recognizes. The display element 120B controls transmission of the visible light from the light source 120A. The display element 120B is, for example, a liquid crystal display device (LCD) of a thin film transistor (TFT) type. In addition, the display element 120B controls a degree of transmission of each color element of the visible light from the light source 120A by individually controlling a plurality of pixels, and thereby causes the virtual image VI to contain an image element, and determines a form (appearance) of the virtual image VI. Hereinafter, the visible light that is transmitted through the display element 120B and contains an image is referred to as image light IL. Note that the display element 120B can be an organic EL display, in which case the light source 120A can be omitted.

[0037] The optical mechanism 112 includes, for example, one or more lenses. The position of each lens is adjustable in, for example, the optical axis direction. The optical mechanism 112 is provided, for example, on the path of the image light IL output from the light projecting device 120, passes the image light IL incident from the light projecting device 120, and emits it toward the front windshield glass 20. The optical mechanism 112 is able to adjust the distance (hereinafter, referred to as the virtual image visual recognition distance DT) from the driver's line-of-sight position PI to the formation position P2 at which the virtual image VI based on the image light IL is formed, for example, by changing the position of the lens.

[0038] The driver's line-of-sight position PI is the position at which the image light IL reflected and converged by the concave mirror 114 and the front windshield glass 20, and is the position at which the driver's eye is assumed to exist. The virtual image visual recognition distance DT is strictly the distance of a line segment having a tilt in the up-and-down direction, but in the following description, in the case of "the virtual image visual recognition distance DT is 7 [m]" and the like, the distance can also refer to the distance in the horizontal direction.

[0039] In the following description, the depression angle θ is defined as the angle formed by the horizontal plane passing through the driver's line-of-sight position PI and the line segment from the driver's line-of-sight position PI to the formation position P2. The greater the depression angle θ, the more the virtual image VI is formed downward, that is, the more the driver's line-of-sight direction in which the virtual image VI is viewed is downward. The depression angle θ is determined based on the reflection angle θ of the concave mirror 114 and the display position of the original image in the display element 120B as described later. The reflection angle θ is the angle formed by the incident direction of the image light IL incident to the concave mirror 114 after being reflected by the plane mirror 113 and the emission direction of the image light IL emitted by the concave mirror 114.

[0040] In the case where the formation position P2 of the virtual image VI is, for example, on the road in front of the vehicle, the greater the distance of the virtual image VI from the host vehicle M, the higher the position of the virtual image VI on the front windshield glass 20. Therefore, for example, in the case where the virtual image VI moves in a manner approaching the host vehicle M, the position of the virtual image VI on the front windshield glass 20 moves in a manner lowering.

[0041] The plane mirror 113 reflects the visible light (that is, the image light IL) emitted from the light source 120A and passed through the display element 120B toward the direction of the concave mirror 114.

[0042] The concave mirror 114 reflects the image light IL incident from the plane mirror 113 and emits it toward the front windshield glass 20. The concave mirror 114 is supported so as to be rotatable (pivotable) about the axis in the width direction of the host vehicle M, that is, the Y axis.

[0043] The light-transmissive cover 115 transmits the image light IL from the concave mirror 114 to the front windshield glass 20, and suppresses entry of foreign matter such as dust, dirt, water droplets, and the like into the housing 111. The light-transmissive cover 115 is provided to an opening portion formed in the upper side member of the housing 111. In addition, an opening portion or a light-transmissive member is also provided in the instrument panel 30, and the image light IL transmits through the light-transmissive cover 115 and the opening portion or the light-transmissive member of the instrument panel 30 to reach the front windshield glass 20.

[0044] The image light IL incident to the front windshield glass 20 is reflected by the front windshield glass 20, and converges at the driver's line-of-sight position PI. At this time, the driver feels that the image reflected by the image light IL is displayed in front of the host vehicle M.

[0045] The lens position sensor 116 detects the position of one or more lenses included in the optical mechanism 112. The lens position sensor 116 generates a lens position signal indicating the detected position of the lens, and outputs it to the control portion 180. The concave mirror angle sensor 117 detects the rotation angle of the concave mirror 114 about the Y axis. The concave mirror angle sensor 117 generates a rotation angle signal indicating the detected rotation angle, and outputs it to the control portion 180.

[0046] The environment sensor 118 detects, for example, the temperature of the light projecting device 120, the optical mechanism 112, as the surrounding environment of the first display device 110. The environment sensor 118 generates an environment signal indicating the detected surrounding environment, and outputs it to the control portion 180. The environment sensor 118 can also detect the illuminance of the surroundings of the host vehicle M, the speed of the host vehicle M, the steering angle, and the presence of an object (for example, another vehicle, a pedestrian, or the like) in the surroundings, as the surrounding environment.

[0047] Returning to Figure 1 , the other parts of the vehicle information providing device 100 will be described. The first display device 110 has, in addition to the light projecting device 120 and the like shown in Figure 3 , for example, an optical system controller 121, a display controller 122, a lens actuator 123, and a concave mirror actuator 124.

[0048] The optical system controller 121 drives and controls the lens actuator 123 and the concave mirror actuator 124 in accordance with the control of the control portion 180. The display controller 122 controls the content and the display mode of the image displayed on the display element 120B of the light projecting device 120, and causes the light projecting device 120 to project the prescribed image light IL in accordance with the control of the control portion 180. The display mode of the image refers to, for example, the brightness of the image, the formation position P2 of the image, the size of the image, and the like.

[0049] The lens actuator 123 includes a motor or the like coupled to the optical mechanism 112, moves the position of one or more lenses in the optical mechanism 112, and adjusts the virtual image visual recognition distance DT. The concave mirror actuator 124 includes a motor or the like coupled to the rotation shaft of the concave mirror 114, and adjusts the reflection angle of the concave mirror 114. The concave mirror actuator 124 acquires a drive signal from the optical system controller 121, and based on the acquired drive signal, drives the motor or the like to rotate the concave mirror actuator 124 around the Y axis, thereby adjusting the reflection angle of the concave mirror 114 Thus, the depression angle θ is adjusted.

[0050] The operation switch 130 is, for example, a switch for accepting an instruction to switch the on / off of the display by the first display device 110, and an instruction to move the position of the virtual image VI. The operation switch 130 is provided, as shown in FIG. 1, for example, at a position that the driver seated on the seat can operate without greatly changing the body posture, for example, the upper portion of the back side of the steering wheel 10. The operation switch 130 can also be provided, for example, at the hub portion of the steering wheel 10, or at the spoke that links the steering wheel 10 and the instrument panel 30. The operation switch 130 accepts an instruction to switch the on / off of the display by the first display device 110, and an instruction to adjust the position of the virtual image VI. Figure 2

[0051] The second display device 140 includes, for example, a center display 141, an electronic rearview mirror 142, a first electronic side mirror 143, and a second electronic side mirror 144. The second display device 140 is disposed, as shown in FIG. 1, around the first display device 110, and displays image items. The center display 141 displays, for example, an image corresponding to a navigation process performed by a navigation device (not shown) mounted on the host vehicle M, or an image of the opposite party in a video phone, and the like. In addition, the center display 141 can also display a television program, play a DVD, display a downloaded movie, and the like. Figure 2

[0052] The electronic rearview mirror 142 is, for example, a display device provided at the central upper portion in the front windshield glass 20. The electronic rearview mirror 142 can also be installed on the ceiling of the vehicle. The electronic rearview mirror 142 displays, for example, an image of the rear portion of the host vehicle M captured by a rear camera (not shown). In the image of the electronic rearview mirror 142, the first frame image GA42F is included as an outer frame.

[0053] The first electronic side mirror 143 is, for example, a display device provided at the left end portion of the instrument panel 30. The first electronic side mirror 143 displays, for example, an image of the left side of the host vehicle captured by a left side camera (not shown). In the image of the first electronic side mirror 143, the second frame image GA43F is included as an outer frame.

[0054] ​​The second electronic side mirror 144 is, for example, a display device provided at the right end portion of the instrument panel 30. The second electronic side mirror 144 displays, for example, an image of the left side of the host vehicle captured by a right side camera (not shown). In the image of the second electronic side mirror 144, the third frame image GA44F is included as an outer frame.

[0055] The object recognition system 150 includes, for example, an outside camera 151, a radar device 152, a LIDAR 153, and an object recognition device 154. The outside camera 151 is, for example, a digital camera that uses a solid-state image pickup element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The outside camera 151 is mounted at an arbitrary position of the host vehicle M. In the case of capturing the front, the outside camera 151 is mounted at the upper portion of the front windshield glass 20, the back surface of the electronic rearview mirror 142, or the like. The outside camera 151 repeatedly captures the periphery of the host vehicle M periodically, for example. The outside camera 151 can also be a stereo camera.

[0056] The radar device 152 radiates millimeter waves or the like to the periphery of the host vehicle M and detects a wave reflected by an object (reflected wave) to detect at least the position (distance and direction) of the object. The radar device 152 is mounted at an arbitrary position of the host vehicle M. The radar device 152 can also detect the position and speed of an object by an FM-CW (Frequency Modulated Continuous Wave) method.

[0057] The LIDAR 153 irradiates light (or an electromagnetic wave having a wavelength close to light) to the periphery of the host vehicle M and measures scattered light. The LIDAR 153 detects the distance to an object based on the time from light emission to light reception. The irradiated light is, for example, pulsed laser light.

[0058] The object recognition device 154 performs sensor fusion processing on the detection results detected by some or all of the outside camera 151, the radar device 152, and the LIDAR 153 to recognize the position, type, speed, and the like of an object. The object recognition device 154, for example, implements an AI (Artificial Intelligence)-based function and a function based on a model given in advance in parallel. The object recognition device 154 recognizes an oncoming vehicle facing the host vehicle based on the reflection intensity or the like of a wave reflected by an oncoming vehicle facing the host vehicle M after being radiated by the radar device 152. The object recognition device 154 identifies an object existing in the periphery including the front of the host vehicle, such as a pedestrian W or a preceding vehicle MF, as a specific object based on the reflection intensity or the like of a wave reflected by the object after being radiated by the radar device 152.

[0059] The object recognition device 154, for example, concurrently executes recognition of a specific object based on deep learning or the like and recognition of a specific object based on pattern matching, and comprehensively evaluates the recognition results of both, thereby recognizing the specific object. In addition, the object recognition device 154, for example, concurrently executes recognition of a specific object based on deep learning or the like and recognition of a specific object based on a condition given in advance (presence of a signal capable of pattern matching, a road sign, or the like). Recognition of a specific object can be achieved by comprehensively evaluating the recognition of a specific object of both by scoring.

[0060] The object recognition system 150 outputs the recognition result recognized by the object recognition device 154 to the control portion 180. The object recognition device 154 can also be a software function portion included in the control portion 180. In this case, the object recognition device 154 can be omitted from the object recognition system 150, and the vehicle exterior camera 151, the radar device 152, and the LIDAR 153 directly output the detection results to the control portion 180.

[0061] The vehicle interior camera 160 is a driver monitoring camera (DMC) that captures the face and the movement of the driver. The vehicle interior camera 160 is provided, for example, in the lower portion of the electronic rearview mirror 142 toward the inside of the vehicle cabin. The vehicle interior camera 160 can also be provided at the front windshield glass 20, the instrument panel 30, the ceiling portion of the vehicle cabin, or the like.

[0062] The vehicle sensor 170 is a sensor that detects information related to the running state of the host vehicle M. The vehicle sensor 170 includes a vehicle speed sensor that detects the speed of the host vehicle M, an acceleration sensor that detects acceleration, a yaw rate sensor that detects the angular velocity about the vertical axis, a direction sensor that detects the orientation of the host vehicle M, an illuminance sensor that detects the brightness of the surroundings of the host vehicle M, and the like. In the vehicle sensor 170, a position sensor that acquires the position of the host vehicle M can be included. The position sensor is, for example, a sensor that acquires position information (longitude, latitude information) from a GPS (Global Positioning System) device. In addition, the position sensor can also be a sensor that acquires position information using a GNSS (Global Navigation Satellite System) receiver of a navigation device.

[0063] The control section 180 includes, for example, an identification section 181, a line-of-sight monitoring section 182, a calculation section 183, an alarm control section 184, a load monitoring section 185, a first display device control section 186, and a second display device control section 187. The first display device control section 186 includes a drive control section 188 and a display control section 189. These constituent elements are realized by, for example, a hardware processor such as a CPU (Central Processing Unit) executing a program (software). Alternatively, part or all of these constituent elements can be realized by hardware (including circuitry) such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), a GPU (Graphics Processing Unit), and the like, and can be realized by a combination of software and hardware. The program can be stored in advance in a storage device (not shown) such as an HDD or a flash memory of the control section 180, or can be stored in a removable storage medium such as a DVD or a CD-ROM, and installed in the control section 180 by mounting the storage medium in a drive device.

[0064] The identification section 181 identifies a specific object such as a pedestrian W or a preceding vehicle MF present around the host vehicle M, and identifies the position and speed of the specific object, based on the identification result output by the object identification system 150. In the case where the object identification device 154 is a software function included in the control section 180, the identification section 181 can be the software function.

[0065] The line-of-sight monitoring section 182 monitors the direction of the driver's line of sight (line-of-sight direction) by performing image processing on an image including the driver's face captured by the in-vehicle camera 160, and detects the line-of-sight direction of the driver. The line-of-sight monitoring section 182 can also detect the driver's line of sight by performing processing other than image processing on the image captured by the in-vehicle camera 160.

[0066] The calculation section 183 calculates the possibility of collision between the specific object identified by the identification section 181 and the host vehicle M. For example, in the case where the identification section 181 identifies a crossing object that is considered to be a pedestrian crossing in front of the host vehicle M as the specific object, the calculation section 183 calculates the possibility of collision between the specific object and the host vehicle M based on the distance between the specific object and the host vehicle M.

[0067] For example, in a case where the oncoming vehicle MF is recognized as a specific object, the calculation section 183 calculates a collision possibility based on a reflection intensity of a wave reflected by the specific object after the wave is radiated by the radar device 152, and whether or not a micro-Doppler signal is detected by the radar device 152. The micro-Doppler signal is a signal in which a frequency of a Doppler signal proportional to a moving speed of a cross object as a measured object is higher than a prescribed value. The collision possibility is an example of a collision risk. The higher the collision possibility, the greater the collision risk.

[0068] The alarm control section 184 causes the first display device control section 186 and the second display device control section 187 to perform alarm display control in a case where the collision possibility calculated by the calculation section 183 is equal to or higher than a threshold value. The alarm control section 184 causes the first display device control section 186 and the second display device control section 187 to perform alarm display control, for example, in a case where the cross object exists in front of the host vehicle M and there is a possibility of collision with the cross object. The alarm control section 184 causes the storage section to store the number of times alarm display control is performed. Furthermore, the alarm control section 184 calculates a frequency of the number of times alarm display control is performed and stores the frequency in the storage section.

[0069] The load monitoring section 185 calculates and monitors a driving load borne by the driver. The load monitoring section 185, for example, calculates a change in an expression of the driver by performing image processing on an image including a face of the driver captured by the in-vehicle camera 160. The load monitoring section 185 calculates a load borne by the driver based on the calculated change in the expression of the driver, such as stability of a gaze, and the like. The load monitoring section 185 can also calculate a driving load by other factors. The load monitoring section 185, for example, can also detect biological information such as a heart rate, a breathing rate, and the like of the driver, and calculate a load of the driver based on the detected biological information.

[0070] The drive control section 188 in the first display device control section 186 causes the optical system controller 121 to drive the lens actuator 123 and the concave mirror actuator 124. The display control section 189 in the first display device control section 186 controls the light projecting device 120. The first display device control section 186, for example, performs alarm display control based on a gaze direction of the driver monitored by the gaze monitoring section 182 and a collision possibility calculated by the calculation section 183.

[0071] The first display device control section 186 reduces the ease of observation of the virtual image VI that is visually recognized by the driver, moves the visual recognition position of the virtual image VI that is visually recognized by the driver, and guides the visual recognition position to a point on the line of sight when the driver visually recognizes a specific object, as the alarm display control. When the visual recognition position is moved, for example, in the case where the visual recognition position is moved upward, the angle of depression θ from the direction in which the virtual image VI is visually recognized from the line of sight position PI is reduced. The point on the line of sight when the driver visually recognizes a specific object is, for example, a point on the front windshield glass 20.

[0072] The control that reduces the ease of observation of the virtual image VI can include, for example, at least any one of a control that reduces the brightness, the lightness, the chroma, the resolution, the sharpness, or the contrast of the virtual image VI, a control that reduces the display area of the virtual image VI, and a control that simplifies the display of the virtual image VI. The control section 180 reduces the ease of observation of the virtual image VI by making the virtual image VI obscure, making the content not easy to understand at a glance, and the like, through these controls.

[0073] The drive control section 188 in the first display device control section 186 drives the lens actuator 123 and the concave mirror actuator 124 by the optical system controller 121 in order to implement the above-described alarm display control. The display control section 189 in the first display device control section 186 controls the light projecting device 120 in order to implement the above-described alarm display control. The first display device control section 186 is an example of the display control section.

[0074] The second display device control section 187 controls the images displayed on the center display 141, the electronic rearview mirror 142, the first electronic side rearview mirror 143, and the second electronic side rearview mirror 144 in the second display device 140. The second display device control section 187 causes the center display 141 to display a map image used for a navigation device, for example, based on an operation by the driver or the like. The images displayed on the center display 141, the electronic rearview mirror 142, the first electronic side rearview mirror 143, and the second electronic side rearview mirror 144 are examples of the image items.

[0075] The second display device control section 187 performs alarm display control based on, for example, the line of sight direction of the driver monitored by the line of sight monitoring section 182 and the collision possibility calculated by the calculation section 183. The second display device control section 187 reduces the ease of observation of the virtual image VI that is visually recognized by the driver, and guides the visual recognition position of the virtual image VI that is visually recognized by the driver to a point on the line of sight when the driver visually recognizes a specific object, as the alarm display control. The second display device control section 187 is an example of the display control section.

[0076] Next, the processing in the control section 180 will be described. The control section 180 determines whether or not to perform the alarm display control after adjusting the threshold value for determining whether or not to perform the alarm display control. Therefore, first, the step of adjusting the threshold value will be described, and then the control performed by the control section 180 after that will be described. Figure 4 is a flowchart showing an example of the processing in the control section 180. In the processing shown in Figure 4 , the control section 180 adjusts the threshold value for determining whether or not to perform the alarm display control. Figure 4 The processing shown in

[0077] In the control section 180, the alarm control section 184 adjusts the threshold value for causing the alarm display control to be performed. At the time of adjustment of the threshold value, the alarm control section 184 determines whether or not the frequency of the number of times the alarm display control is caused to be performed is greater than the value obtained by adding the prescribed value a to the average value (step S101). The average value and the prescribed value are values set in advance, but can also be updated at an appropriate timing. The average value and the prescribed value can also be provided to the vehicle information providing device 100 via, for example, a communication device or the like.

[0078] In the case where it is determined that the frequency of the number of times the alarm display control is caused to be performed is greater than the value obtained by adding the prescribed value a to the average value, the alarm control section 184 lowers the threshold value (step S103). In the case where it is determined that the frequency of the number of times the alarm display control is caused to be performed is not greater than the value obtained by adding the prescribed value a to the average value, the alarm control section 184 determines whether or not the frequency of the number of times the alarm display control is caused to be performed is less than the value obtained by subtracting the prescribed value a from the average value (step S105).

[0079] In the case where it is determined that the frequency of the number of times the alarm display control is caused to be performed is less than the value obtained by subtracting the prescribed value a from the average value, the alarm control section 184 raises the threshold value (step S107). The prescribed value a added to the average value and the prescribed value a subtracted from the average value are the same value, but can also be different values.

[0080] Next, the load monitoring section 185 calculates the load borne by the driver, and the alarm control section 184 determines whether or not the load borne by the driver is the reference value or more (step S109). In the case where it is determined that the load borne by the driver is the reference value or more, the alarm control section 184 lowers the threshold value (step S111). In this way, the control section 180 ends the processing shown in Figure 4 In the case where it is determined that the load borne by the driver is not the reference value or more, the control section 180 directly ends the processing shown in Figure 4 .

[0081] Next, the step after the threshold value is adjusted will be described. Figure 5is a flowchart showing an example of the processing in the control section 180. With regard to the processing shown in Figure 5 the threshold value for determining whether to perform the alarm display control, the control section 180 is executed in the travel of the host vehicle M.

[0082] In the control section 180, the recognition section 181 determines whether the specific object is recognized based on the recognition result output by the object recognition system 150 (step S201). In a case where it is determined that the specific object is not recognized, the control section 180 repeatedly performs the processing of step S201.

[0083] In a case where it is determined by the recognition section 181 that the specific object is recognized, the calculation section 183 calculates the collision possibility of the specific object with the host vehicle M (step S203). Next, the alarm control section 184 determines whether the collision possibility calculated by the calculation section 183 exceeds the threshold value adjusted in accordance with the flow shown in Figure 4 In a case where it is determined that the collision possibility calculated by the calculation section 183 does not exceed the threshold value, the alarm control section 184 returns the processing to step S201.

[0084] In a case where it is determined by the alarm control section 184 that the collision possibility calculated by the calculation section 183 exceeds the threshold value, the line-of-sight monitoring section 182 detects the line of sight of the driver and determines whether the driver gazes at the image (virtual image VI) of the first display device or the image of the second display device (step S207). In a case where it is determined that the driver does not gaze at the image (virtual image VI) of the first display device or the image of the second display device, the alarm control section 184 causes the first display device control section 186 to perform, as the alarm display control, the control that gradually reduces the ease of observation of the virtual image visually recognized by the driver by the first display device 110 (step S209).

[0085] In a case where it is determined by the alarm control section 184 that the driver gazes at the image (virtual image VI) of the first display device or the image of the second display device, the line-of-sight monitoring section 182 determines the display device at which the driver gazes (step S211). Next, the alarm control section 184 causes the first display device control section 186 or the second display device control section 187 to perform, as the alarm display control, the control that gradually reduces the ease of observation of the image displayed by the display device determined by the line-of-sight monitoring section 182 (step S213).

[0086] The alarm control section 184 moves the image displayed by the determined display device in the direction of the specific object at substantially the same time as the control that starts to gradually reduce the ease of observation of the image displayed by the display device (step S215). In the case where the determined display device is any of the display devices included in the second display device 140, a part or all of the image that has been displayed by the second display device 140 is changed to the image (virtual image VI) that the first display device 110 causes the driver to visually recognize.

[0087] The image (virtual image VI) that the first display device causes the driver to visually recognize is reduced in ease of observation compared to the image that has been displayed by the second display device 140. Even in the case where the image that has been displayed by the second display device 140 is any of the first to third frame images GA42F to GA44F in the case where the image that becomes the moving object is any of the first to third frame images GA42F to GA44F, the ease of observation can not be reduced compared to the image that has been displayed by the second display device 140.

[0088] Next, the line-of-sight monitoring section 182 determines whether or not the driver's line of sight is in the direction of the specific object (step S217). In the case where it is determined that the driver's line of sight is not in the direction of the specific object, the alarm control section 184 returns the processing to step S213. The alarm control section 184 omits the processing of step S213 and step S215 after the display of the display device is eliminated, after the display of the display device is moved to the position where it cannot be moved.

[0089] In the case where it is determined by the line-of-sight monitoring section 182 that the driver's line of sight is in the direction of the specific object, the alarm control section 184 stops the alarm display control and eliminates the image in the display device that is moving, and restores the image to the original display position (step S219). Next, the calculating section 183 calculates the collision possibility of the specific object with the host vehicle M again, and determines whether or not the calculated collision possibility is less than the threshold value (step S221).

[0090] In the case where it is determined that the collision possibility is not less than the threshold value, the alarm control section 184 returns the processing to step S207. In the case where it is determined that the collision possibility is less than the threshold value, the alarm control section 184 restores the images of the first display device 110 and the second display device 140 to the state before the start of the alarm display control (step S223). At the time of restoring each image, for example, instead of moving the trajectory after the movement in the reverse direction or the like, the state immediately before the start of the alarm display control is directly restored. After that, the control section 180 ends the processing shown in the flowchart. Figure 5

[0091] ​The start time of the movement of the images of the first display device 110 and the second display device 140 (hereinafter referred to as the movement start time), the time of the end of the movement (hereinafter referred to as the movement end time), the start time of the reduction of the ease of observation of the images of the first display device 110 and the second display device 140 (hereinafter referred to as the reduction start time), and the time of the end of the reduction (hereinafter referred to as the reduction end time) can be arbitrary. Here, the movement start time and the reduction start time, and the movement end time and the reduction end time coincide with each other, respectively.

[0092] In addition, the time change of the display position of the image from the movement start time to the movement end time and the time change of the ease of observation of the image from the reduction start time to the reduction end time can also be arbitrary. Hereinafter, the time change of the display position of the image from the movement start time to the movement end time and the time change of the ease of observation of the image from the reduction start time to the reduction end time will be described.

[0093] Figure 6 is a line graph indicating an example of the time change of the display position of the image and the ease of observation of the image. Figure 6 In the line graph shown, the vertical axis indicates the display position of the image and the ease of observation, and the horizontal axis indicates time. The display position of the image can also move in proportion to the passage of time as indicated by the first line graph LI. The ease of observation of the image can also reduce in proportion to the passage of time as indicated by the second line graph L2.

[0094] In addition, the display position of the image can also move with the amount of reduction gradually increasing with the passage of time as indicated by the third line graph L3. In this case, the more time passes, the faster the display position of the image moves. The ease of observation of the image can also reduce with the amount of reduction gradually decreasing with the passage of time as indicated by the fourth line graph L4. In this case, the more time passes, the smaller the amount of reduction of the image. By displaying the image with the display position and the ease of observation changed according to the passage of time as described above, the image can be overlapped with the specific object without being easily visually recognized by the driver.

[0095] The reduction start time of the image can coincide with the movement start time, but can also be different, for example, the reduction start time can be earlier than the movement start time, or later than the movement start time, but it is appropriate to avoid a case where the reduction start time is later than the movement start time and the image whose ease of observation is not reduced at the movement destination is displayed.

[0096] Next, the change in the display of the first display device 110 and the second display device 140 when the alarm display control is performed will be described with reference to Figures 7 to 9 First, a case where the specific object is the preceding vehicle MF and the driver is looking at the center display 141 when the collision possibility becomes equal to or higher than the threshold value will be described.Figure 7 is an example of a change in the image displayed in the first display device 110 and the central display 141.

[0097] As shown in the upper drawing of Figure 7 , the preceding vehicle MF is traveling in front of the host vehicle M, but when the collision possibility with the preceding vehicle MF is less than a threshold value, the control section 180 does not perform the alarm display control. Therefore, the first display device 110 causes the driver to visually recognize the virtual image VI in normal times, and the central display 141, the electronic rearview mirror 142, the first electronic side mirror 143, and the second electronic side mirror 144 of the second display device 140 each display a normal-time image (hereinafter referred to as an original image) GA10 to GA40. In the original image GA10 displayed in the central display 141, a map image GA11 based on a navigation device, and an accompanying image GA12 including a background image other than the map image, a GUI (Graphical User Interface) image, and the like are included.

[0098] Here, when the collision possibility becomes equal to or greater than the threshold value, the control section 180 performs control so that the map image GA11 in the original image GA10 already displayed in the central display 141 is moved in the direction of the preceding vehicle MF as shown in the middle drawing of Figure 7 . Therefore, the central display 141 displays the accompanying image GA12 other than the map image GA11 as it is. In addition, the central display 141 lowers the brightness, the lightness, the chroma of the display of the accompanying image GA12 lower than the original image GA10, lowers the contrast, and lowers the ease of observation. The portion in which the map image GA11 has been displayed is extinguished, but other display such as display of an image without a pattern can be used.

[0099] In addition, the control section 180 controls the first display device 110 so that the driver visually recognizes a first map virtual image VI11 equivalent to the map image GA11 already displayed in the central display 141. The first map virtual image VI11 is an image in which the ease of observation is lowered compared to the map image GA11 already displayed in the central display 141, for example, a text is omitted, an image of a map is displayed by a simple quadrangle, and a color image is a black-and-white image.

[0100] Moreover, when the driver does not move the line of sight to the preceding vehicle MF and time elapses, the control section 180 performs control so that the map image GA11 in the original image GA10 already displayed in the central display 141 is moved in the direction of the preceding vehicle MF as shown in the lower drawing of Figure 7As shown in the lower drawing of FIG. 18, when there is no specific object in front of the host vehicle M and the collision possibility of the host vehicle M is less than the threshold value, the control section 180 does not perform the warning display control.

[0101] Next, with reference to Figure 8 , a case where the first display device 110 visually recognizes the first virtual image VI25 to the driver when the specific object is the pedestrian W and the collision possibility becomes equal to or more than the threshold value will be described. Figure 8 is a drawing showing an example of a change in the image in the first display device 110. As shown in the upper drawing of FIG. 19, when there is no specific object in front of the host vehicle M and the collision possibility of the host vehicle M is less than the threshold value, the control section 180 does not perform the warning display control. Figure 8

[0102] Here, as shown in the middle drawing of FIG. 19, when the pedestrian W is about to cross in front of the host vehicle M and the collision possibility of the pedestrian W and the host vehicle M starts to become equal to or more than the threshold value, the control section 180 moves the second virtual image VI26 toward the pedestrian W and guides the driver's line of sight toward a point on the line of sight when the driver visually recognizes the pedestrian W. In addition, the control section 180 lowers the brightness, the lightness, and the color saturation of the second virtual image VI26 lower than those of the first virtual image VI25 and lowers the contrast to lower the ease of observation. Figure 8

[0103] Moreover, when the driver does not move the line of sight toward the pedestrian W and time elapses, as shown in the lower drawing of FIG. 19, the control section 180 controls the first display device 110 to display the third virtual image VI27 on the left side of the screen. Immediately before the third virtual image VI27 reaches the position of the pedestrian W, the control section 180 controls the first display device 110 to eliminate the virtual image VI. In this way, the control section 180 guides the driver's line of sight toward the pedestrian W. Figure 8

[0104] Next, with reference to Figure 9 ​​​, indicating that the driver is looking at the electronic rearview mirror 142 when the specific object is the preceding vehicle MF and the collision possibility becomes above the threshold value. Figure 9 is a diagram showing an example of changes in the images in the first display device 110 and the electronic rearview mirror 142. As shown in the upper diagram of Figure 9 , the preceding vehicle MF is traveling in front of the host vehicle M, but when the collision possibility of the preceding vehicle MF with the host vehicle M is less than the threshold value, the control section 180 does not perform the warning display control.

[0105] Here, when the collision possibility becomes above the threshold value, the control section 180 controls the electronic rearview mirror 142 so that the first frame image GA42F included in the original image GA20 that has been displayed in the electronic rearview mirror 142 moves in the direction of the preceding vehicle MF, as shown in the center diagram of Figure 9 . The center display 141 displays the first upper left image GA21 corresponding to the upper left portion of the first frame image GA42F.

[0106] At this time, the control section 180 reduces the brightness, the lightness, the chroma, and the contrast of the images displayed in the electronic rearview mirror 142 other than the frame image to the extent that the visual recognition of the driver can be ensured, compared with the original images, and displays them. In addition, the control section 180 controls the first display device 110 so that the driver visually recognizes the first lower right virtual image VI21 corresponding to the lower right portion of the image that has been displayed in the center display 141.

[0107] Furthermore, when the driver does not move the line of sight to the preceding vehicle MF and time elapses, the control section 180 controls the display of the center display 141 so that the second upper left image GA22 corresponding to the lower left portion of the first upper left image GA21 is displayed in the lower right of the screen, and controls the first display device 110 so that the driver visually recognizes the lower right portion of the second upper left image GA22 as the second lower right virtual image VI22, as shown in the lower diagram of Figure 9 . The control section 180 further reduces the brightness, the lightness, the chroma, and the contrast of the images displayed in the electronic rearview mirror 142 other than the frame image to the extent that the visual recognition of the driver can be ensured, and displays them.

[0108] Immediately before the second lower right virtual image VI22 reaches the position of the preceding vehicle MF, the control section 180 eliminates the virtual image VI that the driver visually recognizes by the first display device 110, and restores the images displayed in the electronic rearview mirror 142 other than the frame image to the images before the start of the warning display control. In this way, the control section 180 guides the line of sight of the driver in the direction of the preceding vehicle MF.

[0109] In the case where the specific object is the preceding vehicle MF and the collision possibility becomes equal to or greater than the threshold value and the driver gazes at the first electronic side mirror 143 or the second electronic side mirror 144, the second frame image GA43F and the third frame image GA44F are moved in the direction of the specific object to guide the driver's line of sight in the direction of the specific object, as in the case where the driver gazes at the electronic mirror 142. As for the images other than the frame images displayed on the first electronic side mirror 143 and the second electronic side mirror 144, the ease of observation is reduced and recovered as in the case of the electronic mirror 142.

[0110] The vehicle information providing apparatus 100 of the embodiment calculates the collision possibility of the specific object, such as the preceding vehicle MF or the pedestrian W, with the host vehicle M and performs alarm display control when the collision possibility becomes equal to or greater than the threshold value. In the alarm display control, the ease of observation of the display based on the display device is reduced and the display of the display device on which the driver gazes is moved in the direction of the specific object in order to direct the driver's attention to the specific object. Thus, it is possible to suppress the situation where the driver fails to notice the change in the display due to the change in the blind spot and the like. Thus, by causing the driver to recognize the specific object early, it is possible to improve the safety of traffic by seeking safety in prevention.

[0111] Further, it is possible to urge the driver to prepare for the situation that is approaching in the future without easily causing the driver to feel bored due to a large number of alarms. Further, the driver is hardly stimulated before the start of the alarm display control, and thus it is possible to make the mental load of the driver very small and to cause the driver to allocate more attention to the situation of the outside, the surroundings, and to easily notice the change in the situation.

[0112] In the above-described embodiment, the alarm display control in which the greater the collision possibility, the lower the ease of observation of the image of the display device is performed when the collision possibility becomes equal to or greater than the threshold value. At this time, the threshold value is one, but a plurality of threshold values can be set and different alarm display controls can be performed each time the collision possibility becomes equal to or greater than each threshold value. In the above-described embodiment, the specific object is the preceding vehicle or the pedestrian, but the specific object can be other, such as an oncoming vehicle or a stationary object on the road.

[0113] In the above-described embodiments, the vehicle information providing apparatus 100 can also provide the driver with an alarm other than the alarm display control. The vehicle information providing apparatus 100 can also provide an alarm by giving a stimulus to the driver's sense of hearing, sight, or the like, for example, in conjunction with the alarm display control. The stimulus given to the sense of hearing can be, for example, a sound simulating an engine start, road noise, music, or the like. The stimulus given to the sense of touch can be, for example, a stimulus of vibration imparted to the seat or the steering wheel 10, or a stimulus of temperature by the seat heater, air conditioner, or the like. In the above-described embodiments, the HUD is used as the first display device, but a display device capable of projecting a prescribed alarm signal, such as a HUW (Head Up Warning), can also be used as the first display device.

[0114] The above-described embodiments can be expressed as follows.

[0115] A vehicle information providing apparatus including:

[0116] a storage device storing a program; and

[0117] a hardware processor,

[0118] the program stored in the storage device by the hardware processor to perform the following processing:

[0119] recognize a specific object present in the vicinity of the host vehicle;

[0120] calculate a collision risk with respect to the recognized specific object;

[0121] the greater the calculated collision risk, the lower the ease of observation of an image item visually recognized by the driver is controlled.

[0122] The above-described embodiments have explained specific embodiments of the present application, but the present application is by no means limited to such embodiments, and various modifications and substitutions can be made within the scope of the gist of the present application.

Claims

1. An information providing apparatus for a vehicle, wherein the information providing apparatus for a vehicle is provided with: a display device that causes a driver to visually recognize an image item; an identification unit that identifies a specific object present in the periphery of a host vehicle; a calculation unit that calculates a collision risk with respect to the specific object identified; a display control unit that controls the display device to make the image item less easily observable as the calculated collision risk is greater; and a load monitoring unit that monitors a load borne by the driver, the display control unit easily starts the control to make the image item less easily observable in a case where it is determined that the load borne by the driver is large.

2. The information providing apparatus for a vehicle according to claim 1, wherein making the image item less easily observable includes at least any one of making the brightness, the lightness, the chroma, the resolution, the sharpness, or the contrast of the image item lower, reducing the display area of the image item, and simplifying the display of the image item.

3. The information providing apparatus for a vehicle according to claim 1 or 2, wherein the display control unit makes the image item less easily observable while guiding the driver to visually recognize a visual recognition position of the image item in the direction of a point on the line of sight at the time when the driver visually recognizes the specific object.

4. The information providing apparatus for a vehicle according to claim 3, wherein as the display device, there is provided a first display device that overlaps an image with a landscape in front of the host vehicle to be visually recognized by the driver, and a second display device that is disposed around the first display device and displays the image item, the display control unit guides the visual recognition position of the image item displayed on the second display device to the first display device.

5. The information providing apparatus for a vehicle according to claim 1 or 2, wherein the information providing apparatus for a vehicle is further provided with a line-of-sight monitoring unit that monitors the orientation of the line of sight of the driver, the display control unit makes the image item less easily observable in a case where it is determined that the line of sight of the driver is directed toward the image item.

6. The information providing apparatus for a vehicle according to claim 5, wherein in a case where it is determined that the line of sight of the driver is directed toward the specific object, the display control unit performs the following processing: suspend the control to make the image item less easily observable, and causes the driver to visually recognize the image item in the state before the control to make the image item less easily observable is performed.

7. An information providing method for a vehicle, wherein the information providing method causes a computer to perform the following processing: identify a specific object present in the periphery of a host vehicle; calculate a collision risk with respect to the specific object identified; control a display device to make an image item visually recognized by a driver less easily observable as the calculated collision risk is greater; monitor a load borne by the driver; ​ In a case where it is determined that the load on the driver is large, control to easily start reducing the ease of observation of the image item is started.

8. A storage medium storing a program, wherein The program causes a computer to perform the following processing: identifying a specific object present in the periphery of the host vehicle; calculating a collision risk with respect to the identified specific object; the greater the calculated collision risk, the lower the ease of observation of an image item that is visually recognized by the driver is controlled to be; monitoring a load on the driver; In a case where it is determined that the load on the driver is large, control to easily start reducing the ease of observation of the image item is started.

Images