Display control device, display device, and display control method

Abstract

To prevent an occupant from viewing (gazing at) the travel route or the like by an image displayed on a display device when the vehicle is turning on a road. [Solution] The images include first and second images with different perceived distances, which are the distances perceived by the occupant ahead, and a display control device (140) detects that the vehicle is turning on a road based on at least one of the following information: the vehicle's steering angle, driving position, translational motion speed, and the occupant's line of sight, and acquires information on viewpoint positions (Q1-Q4) estimated to be the location where occupant (3) is looking. When the vehicle is turning on a road, the visibility of the image with the longer perceived distance is set to be equal to or lower than the visibility of the image with the shorter perceived distance, and when the vehicle has finished turning on a road, the visibility is increased, and visibility change control is performed to change the manner in which visibility is increased or decreased based on the acquired information on viewpoint positions (Q1-Q4).

Description

[Technical Field]

[0001] The present invention relates to a display control device mounted on a vehicle such as an automobile, a display device such as a head-up display (HUD) device, a display control method, and the like. [Background technology]

[0002] Patent Document 1 discloses a technology in which, in a head-up display (HUD) device, when an occupant temporarily loses sight of the road ahead or loses their viewpoint, a first control is executed, which includes hiding images (virtual images) for at least some of the displayed content, reducing visibility, or reducing visibility until the content is hidden, or reducing the number of displayed content, and then, when the vehicle returns to a state of forward visibility or a state of viewpoint detection, a second control is executed, which includes gradually increasing the visibility or the number of displayed content, or setting a period in which the state of hidden or reduced visibility is maintained, and then gradually or quickly increasing the visibility or the number of displayed content over time.

[0003] Patent Document 2 discloses a technology that includes a first projection means for making a first virtual image visible and a second projection means for making a second virtual image visible closer to the occupant than the first virtual image, and when the first virtual image deviates from the actual scene in the longitudinal direction in front of the occupant, controls the visibility of the first virtual image to be lower than the visibility of the second virtual image, and displays an auxiliary virtual image related to the first virtual image at a position where it is visible closer to the occupant than the first virtual image. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2022-44136 [Patent Document 2] Patent No. 6176478 Summary of the Invention [Problem to be solved by the invention]

[0005] The present inventors have studied display control particularly when a vehicle is turning on a road, and have come to the following findings. (1) As a principle of safe driving, it is important for passengers to pay attention to positions in front of the vehicle, relatively close to the vehicle. However, when a vehicle turns on a road, the occupant needs to check the conditions of the road around the bend (for example, the degree of curvature and shape of the road, and whether there are any obstacles or people on the road beyond the bend), so the occupant's line of sight deviates from the normal line of sight that looks straight ahead. In other words, the occupant moves their line of sight to a position quite far away on the road around the bend in order to quickly and accurately check the road conditions, etc. (2) If it is not possible to quickly and accurately check road conditions, the occupants will be inattentive to the road ahead for a long period of time if they are in a position relatively close to the vehicle, which is undesirable from the viewpoint of safe driving. (3) On the other hand, images (virtual images, etc.) displayed in front of the occupants by display devices such as HUD devices can sometimes hinder the occupants from quickly and accurately checking road conditions when the vehicle is turning on a road. In other words, if an image from a display device such as a HUD device comes into the field of view of a passenger who is moving his or her gaze to a position quite far away on a curved road to check road conditions, etc., and the passenger's gaze is momentarily drawn to the image, it is possible that the passenger will feel annoyed or be delayed in checking the road conditions. (4) In the case of (3) above, the further away the occupant's gaze is, the more difficult it becomes to quickly and accurately check the road conditions. In addition, it takes longer to return the gaze to its original position after changing direction, which is likely to result in a longer period of inattention to the road ahead. In other words, it can be estimated that the risk is higher when the occupant's forward viewpoint is farther away than when it is closer. Therefore, it is necessary to more effectively suppress delays in checking road conditions, etc., caused by the above-mentioned displayed image. (5) Therefore, it is preferable to estimate the forward viewpoint position of the occupant and, based on that viewpoint position, control the display mode of the image on a display device such as a HUD device in a more detailed manner. (6) Furthermore, when images displayed on a display device such as a HUD device include images with long and short perceptual distances (the distance perceived by the occupant ahead), it is believed that the image with the longer perceptual distance is more likely to be in the field of view of an occupant looking at the far side of a curved road. Therefore, when controlling the visibility of images, it is preferable to first consider the image with the longer perceptual distance so as not to interfere with the occupant's confirmation of the road conditions. (7) The above Patent Documents 1 and 2 do not mention anything about the problems described in (4) to (6) above, nor do they mention any solutions to these problems. Therefore, it is currently difficult to implement more precise display control, such as that described in (5) above.

[0006] Such a problem has been identified by the present inventors.

[0007] The present invention aims to prevent an image displayed on a display device from interfering with a passenger's view (gazing) at the driving route, etc., when the vehicle is turning on a road.

[0008] Other objects of the present invention will become apparent to those skilled in the art by referring to the following exemplary aspects and best modes, as well as the accompanying drawings. [Means for solving the problem]

[0009] In order to facilitate an understanding of the outline of the present invention, the following examples are given of embodiments according to the present invention.

[0010] In a first aspect, a display control device is a display control device that is mounted on a vehicle and controls display of an image in a display device that can allow an occupant of the vehicle to view an image as a virtual image, the image including first and second images having different perceived distances that are distances perceived ahead by the occupant, the display control device has a visibility control unit that can change the visibility of each of the first and second images, and the visibility control unit detects that the vehicle is turning on a road based on at least one of information of a steering angle of the vehicle, a traveling position of the vehicle, a translational motion speed of the vehicle, and a line of sight direction of the occupant of the vehicle, When the vehicle is turning on the road, a visibility reduction control is performed to reduce the visibility of the image with the longer perceptual distance between the first and second images to the visibility of the image with the shorter perceptual distance or lower, and when the vehicle has finished turning on the road, a visibility increase control is performed to increase the visibility of the image with reduced visibility, and a visibility change control is performed to change the mode of reducing visibility and the mode of increasing visibility based on the obtained information on the viewpoint position during the visibility reduction control and the visibility increase control.

[0011] In the first mode, a state in which a vehicle is turning on a road (for example, turning left or right) (including a state immediately before the turn and a state immediately after the turn) is assumed. It is also assumed that the images displayed by the display device include images with long perception distances (distance perceived by the occupant ahead) and images with short perception distances. When turning on a road, it is assumed that the occupant will often be looking at a point quite far into the road where the road is turning. At this time, if the displayed image comes into the occupant's field of vision, the occupant may feel annoyed, or the image may distract the occupant, delaying their confirmation of the road conditions in the distance. Furthermore, images with a long perceptual distance are more likely to be in the field of view of a passenger looking into the distance. Taking these points into consideration, in this embodiment, greater importance is placed on images with a long perceptual distance, and visibility control is performed so that they do not enter the field of view of the occupant and interfere with the occupant's visibility of road conditions, etc. Specifically, when the vehicle is turning on the road, visibility reduction control is implemented to make the visibility of the image with a longer perceptual distance equal to or lower than the visibility of the image with a shorter perceptual distance (in other words, visibility control that prioritizes the reduction in visibility of the image with a longer perceptual distance over the reduction in visibility of the image with a shorter perceptual distance), and on the other hand, when the vehicle has finished turning on the road (including the states immediately before and after finishing the turn), visibility increase control is implemented to increase the visibility of the image with reduced visibility. The visibility reduction control can prevent the image with the longer perceptual distance from entering the occupant's field of view, causing annoyance to the occupant or interfering with distant visibility. Furthermore, once the vehicle has finished turning, the image display is resumed at the original visibility, for example, so that the occupant can immediately view the image showing useful information at a visibility that is easy to see. However, there are various shapes of "curved roads," for example, there are short curves that can be turned relatively easily, there are also curves that continue for a relatively long time ahead of the vehicle (long vertical curves), and there are also cases where there are a series of S-shaped curves, for example. For example, on long vertical curves or S-shaped curves, occupants need to quickly grasp the situation on the curved road at a considerable distance and perform appropriate driving operations, which tends to take a long time to confirm the road conditions, etc. In this case, the period of time when the driver is not paying attention to what is ahead is longer, and the risk of danger while driving increases. Therefore, when the occupant is looking at something further away, it is important to further suppress factors that impede the occupant's visibility and enable the occupant to quickly check road conditions, etc. Considering these points, when carrying out the above-mentioned "visibility reduction control" and "visibility increase control," it is preferable to take into consideration the viewpoint position of the occupant (in other words, taking into consideration how far away they are looking) and to more finely control the visibility control aspects, such as the lower limit value when reducing visibility, the timing at which visibility reduction begins, the time required for visibility to reach the lower limit value, and the aspect of change when visibility is changed (rate of change, etc.). For example, if the occupant is looking at something further away, it is preferable to set the lower limit of image visibility lower or to start reducing visibility earlier, taking sufficient care not to obstruct distant visibility. Therefore, in this embodiment, for example, the viewpoint position from which the occupant is looking is estimated based on the steering angle of the vehicle, etc., and visibility change control is implemented to more finely change the manner in which visibility is reduced and the manner in which visibility is increased depending on that viewpoint position. This makes it possible to further reduce the possibility that the displayed image will enter the field of view of the occupant and obstruct distant visibility, thereby improving the convenience of the display device. This also contributes to improved safety when the vehicle turns on the road.

[0012] In a second aspect dependent on the first aspect, the visibility control unit may adjust the degree of visibility change by taking into account the importance of each of the first and second images when controlling the visibility change.

[0013] In the first mode described above, the mode of changing visibility (changing visibility) was adjusted taking into account the viewpoint position of the occupant, but in the second mode, the mode of changing visibility is adjusted in more detail, taking into account the importance of the image being displayed (in other words, the extent to which each type of image has an impact on the safe operation of the vehicle, for example). This enables more precise (more detailed) display control that takes safety into consideration, such as not significantly reducing the visibility of images that are important to the occupants, while still allowing them to obtain information when needed.

[0014] In a third aspect dependent on the first or second aspect, at least one of the following may be set for the first image or for each of the first and second images: a lower limit value when reducing visibility, a first threshold value for when to start reducing visibility, a lower limit arrival time required for visibility to reach the lower limit value, a change in visibility over time when reducing visibility, an upper limit value when increasing reduced visibility, a second threshold value for when to start increasing visibility, a lower limit arrival time required for visibility to reach the upper limit value, and a change in visibility over time when increasing visibility, and the visibility change control may be performed based on the settings.

[0015] In the third aspect, specific control items (control parameters) are exemplified when visibility change control is performed based on the viewpoint position of the occupant (driver, etc.). For example, when reducing visibility, the lower limit value of visibility, the timing at which the reduction in visibility begins (first threshold value), the time required for visibility to reach the lower limit value (time to reach lower limit), and the manner of change over time when changing visibility (specifically, the shape of the characteristic line indicating the rate of change, etc.) are individually set to implement more detailed display control. For example, when reducing visibility, if the occupant is looking at something further away, it is possible to more effectively prevent the displayed image from interfering with the occupant's ability to see things at a distance by setting the lower limit of image visibility lower, by earliering the timing at which visibility begins to decrease, by earliering the time it takes to reach the lower limit, or by making the change in visibility more abrupt in the period immediately after visibility begins to decrease.

[0016] The same applies when increasing visibility; if the occupant is looking further away, the upper limit value of the image visibility can be set higher than a predetermined reference value (for example, a preset reference value when increasing visibility, or the original visibility value before the visibility change), the timing at which the visibility increase begins can be made earlier, the time to reach the upper limit can be made earlier, or the change can be made more abrupt in the period immediately after the visibility increase begins, so that the occupant who returns their gaze to the front can read the necessary information more quickly.

[0017] In a fourth aspect dependent on any one of the first to third aspects, the visibility control unit is configured to: determine whether or not one of the first and second images, which has a longer perceptual distance, is a superimposed image to be superimposed on a real scene, and the image, which has a shorter perceptual distance, is a non-superimposed display that is not intended to be superimposed on the real scene; or whether or not one of the first and second images, which has a longer perceptual distance, is a three-dimensional image based on a parallax image or a first two-dimensional image in which a sense of depth is emphasized by perspective drawing, and is a superimposed image to be superimposed on a real scene, and the image, which has a shorter perceptual distance, has a reduced sense of depth compared to the first two-dimensional image. When the image is a second two-dimensional image and is a non-superimposed display that is not intended to be superimposed on the actual scene, during the visibility control, if the superimposed image is an image of low importance that does not include a warning to the occupant, the superimposed image is not displayed, and if the image is an image of high importance that includes the warning, the superimposed image may be made visible at normal times or its visibility may be reduced below the visibility under normal times when it is visible, and the non-superimposed display may be made visible at normal times or its visibility may be reduced below the visibility under normal times when it is visible, regardless of the importance of the superimposed image.

[0018] In the fourth aspect, an example of a suitable driving situation (a suitable display example) to which the present invention can be applied is shown. A suitable display example to which the present invention can be applied is a case in which a superimposed image (e.g., an image of an arrow indicating a turn around a curve) that is superimposed on the actual scene is displayed as the image with a longer perceptual distance, and a non-superimposed image (e.g., a vehicle speed display indicating the vehicle's traveling speed) that is not intended to be superimposed on the actual scene is displayed as the image with a shorter perceptual distance. Superimposed images have a longer perceptual distance than non-superimposed images, making them more likely to catch the occupant's eye. Furthermore, non-superimposed images are important information indicating the vehicle's status, so it is considered preferable to make them as visible as possible. When such a display is in place, it is useful for the safe driving of the vehicle to take care not to reduce the visibility of the superimposed image when the vehicle is turning on the road, thereby preventing the occupants from seeing things at a distance, while at the same time keeping the non-superimposed image visible so that the occupants can read the information.

[0019] Furthermore, when a superimposed image with a long perceptual distance is a more conspicuous and eye-catching display than a non-superimposed image, specifically when the superimposed image is a 3D display using a stereoscopic display device, or when the superimposed image is a 2D display but the sense of depth is more greatly emphasized by perspective drawing, the conspicuousness of the superimposed image is increased, and the display control of the present invention that reduces visibility and reduces annoyance to occupants is effective. However, depending on the driving situation, there are cases where control that uniformly reduces the visibility of the superimposed image based only on the viewpoint (gazing point) of the occupant, for example, is not desirable. In particular, in this embodiment, when the superimposed image is a display including a warning to the occupant (for example, a display notifying the risk of contact or collision with a preceding vehicle), the superimposed image is controlled to maintain its original visibility without reducing its visibility, or to maintain a high level of visibility to maintain a visible state, thereby ensuring the safety of vehicle travel. On the other hand, for non-superimposed images, it is preferable to similarly control the display to maintain normal visibility or to slightly reduce visibility while maintaining visibility, taking into consideration the safe operation of the vehicle. In this manner, according to this aspect, for example, it is possible to take into consideration the importance of each type of superimposed image and to perform more appropriate display control, thereby ensuring the safety of vehicle operation.

[0020] In a fifth aspect dependent on the fourth aspect, the superimposed image of low importance may be a navigation display that assists in driving the vehicle, the superimposed image of high importance may be a warning display that notifies the occupant that the vehicle is in a dangerous state, and the non-superimposed display may be an instrument display that shows the driving status of the vehicle.

[0021] The fifth aspect shows an example of a superimposed image of low importance (e.g., a navigation display), an example of a superimposed image of low importance (e.g., a warning display notifying of danger), and an example of a non-superimposed image (e.g., an instrument display) in the fourth aspect described above. According to this aspect, when these images are displayed, it is possible to individually and appropriately control the visibility of each image.

[0022] In a sixth aspect dependent on any one of the first to fifth aspects, the visibility control unit may control the visibility of at least one of the first and second images based on a first characteristic line in which visibility changes linearly over time, or a second characteristic line in which visibility changes curvedly over time.

[0023] In the sixth aspect, it is possible to freely set the rate of change with respect to the time axis when changing (varying) the visibility of the image. For example, the visibility can be changed based on a linear characteristic line, or more complex visibility changes can be achieved using a curved characteristic line.

[0024] In a seventh aspect, the display device has an image generation unit that generates the first and second images, an optical system that emits display light for the generated superimposed depth image, and a display control device described in any one of claims 1 to 6.

[0025] According to the seventh aspect, it is possible to realize a display device (on-board display device) with improved convenience that can effectively prevent the image displayed by the display device from interfering with the occupants' visibility (gazing) at the driving route, etc. when the vehicle is turning on a road.

[0026] In an eighth aspect dependent on the seventh aspect, the display device may be a head-up display device that displays the first and second images as virtual images by projecting the display light onto a projection member of the vehicle.

[0027] According to the eighth aspect, a head-up display (HUD) device with improved convenience can be realized, which can effectively prevent the image displayed by the display device from interfering with the occupants' visibility (gazing) at the driving route, etc., when the vehicle is turning on a road.

[0028] In a ninth aspect, a display control method is a display control method for controlling display of an image in a display device that is mounted on a vehicle and that allows an occupant of the vehicle to view an image as a virtual image, wherein the image includes a first image and a second image having different perceived distances that are distances perceived ahead by the occupant, and the display control method includes the steps of detecting that the vehicle is turning on a road based on at least one of information on a steering angle of the vehicle, a traveling position of the vehicle, a translational motion speed of the vehicle, and a line of sight direction of the occupant of the vehicle, and acquiring information on a viewpoint position that is estimated to be a location where the occupant is looking. and a step of performing visibility reduction control to reduce the visibility of one of the first and second images, whichever has the longer perceptual distance, to equal or exceed the visibility of the image, which has the shorter perceptual distance, when the vehicle is turning on the road, and performing visibility increase control to increase the visibility of the image with reduced visibility when the vehicle has finished turning on the road; and a step of performing visibility change control to change the mode of reducing visibility and the mode of increasing visibility based on the acquired information on the viewpoint position during the visibility reduction control and the visibility increase control.

[0029] According to the ninth aspect, when the vehicle is turning on a road, it is possible to effectively reduce the possibility that the displayed image will enter the field of view of the occupant and obstruct distant visibility, thereby improving the convenience of the display device, which also contributes to improving safety when the vehicle is turning on a road.

[0030] Those skilled in the art will easily understand that the exemplified embodiments according to the present invention can be further modified without departing from the spirit of the present invention. [Brief explanation of the drawings]

[0031] [Figure 1] FIG. 1 is a diagram showing an example of the configuration of an in-vehicle display system including a display control device. [Figure 2] FIG. 2 is a diagram illustrating an example of the configuration of a display control device. [Figure 3]3A and 3B are diagrams showing examples of display forms when a plurality of images with different perceptual distances are displayed. [Figure 4] FIG. 4 is a diagram showing an example of a case where visibility control is performed based on the viewpoint (point of gaze) of the occupant (driver) when the vehicle is turning on a road, for a navigation display as a superimposed image and an instrument display (vehicle speed display) as a non-superimposed image. [Figure 5] FIG. 5 is a diagram showing another example of a case where visibility control is performed based on the viewpoint (point of gaze) of the occupant (driver) when the vehicle is turning on a road, for a navigation display as a superimposed image and an instrument display (vehicle speed display) as a non-superimposed image. [Figure 6] FIG. 6 is a diagram showing an example of a manner of change (transformation) on the time axis when the visibility of an image is changed (transformed). [Figure 7] FIG. 7 is a flowchart showing an example of control in the display control method of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0032] The best mode described below is used to facilitate understanding of the present invention, and therefore, those skilled in the art should be aware that the present invention is not unduly limited by the embodiments described below.

[0033] (First embodiment) Please refer to Fig. 1. Fig. 1 is a diagram showing an example of the configuration of an in-vehicle display system including a display control device. Fig. 1 shows an in-vehicle display system including a parallax 3D HUD device as an in-vehicle display system 10 capable of displaying a stereoscopic image with a sense of depth. However, the present invention is not limited to this.

[0034] In FIG. 1, the width direction of the vehicle 1 is defined as the left-right direction (or lateral direction: X direction), the direction along a line segment that is perpendicular to the left-right direction and perpendicular to the ground or a surface equivalent to the ground (here, road surface 6) is defined as the up-down direction (or height direction: Y direction), and the direction along a line segment that is perpendicular to each of the left-right direction and the up-down direction (directions indicating the forward and backward directions of the vehicle 1) is defined as the fore-aft direction (Z direction). The positive Z direction is defined as the front, and the negative Z direction is defined as the rear. The front can also be referred to as the "depth direction." This is the same in the other drawings.

[0035] The term "viewpoint" means both the position of the occupant's eyes and the location where the occupant is looking. In this specification, the term is primarily used to mean the location where the occupant is looking, but to avoid any ambiguity, the term "viewpoint (point of gaze) indicating the location where the occupant is looking" will be used instead.

[0036] In FIG. 1, an in-vehicle display system 10 provided in a vehicle (own vehicle) 1 includes an pupil detection camera 43 for pupil (or face) detection that detects the line of sight and position of the left eye EL and right eye ER of an occupant 3 (a viewer such as a driver), a viewpoint position detection unit 44 that detects the viewpoint as the eye position, a viewpoint (gazing point) estimation unit 49 that estimates the position of the viewpoint (gazing point) as the location where the occupant 3 is looking based on the steering angle of the vehicle 1, a front (broadly speaking, surroundings) imaging camera (for example, a stereo camera) 45, an image processing unit 46 (a distance measurement unit 47, an object type / size The vehicle 1 includes a navigation system (navigation ECU) 400, a communication unit (capable of communicating with the driving assistance system 600 and having functions such as inter-vehicle communication), a GPS receiving unit 502, various sensors 505 (including a translational speed sensor 507 and a steering angle sensor 509), and an ECU 700 capable of collecting various information related to the vehicle 1 (for example, lighting on / off information, vehicle speed information, vehicle information related to the engine, etc.).

[0037] The ECU 700 is capable of mutual communication with the navigation device (navigation ECU) 400 via a system bus BUS.

[0038] The distance measurement unit 47 included in the image processing unit 46 refers to a pair of left and right original images captured by a stereo camera as the imaging camera 45, for example, and detects the parallax to the same object (the target object) by, for example, stereo matching that searches for corresponding points in each image, and measures the distance to the target object using the principle of triangulation based on this parallax.

[0039] Furthermore, the radar unit 120 emits radio waves toward an object (a target object in the distance) and measures the reflected waves to measure the distance and direction to the object (a target object in the distance).

[0040] The navigation device (navigation ECU) 400 has a depth mapping unit 402, a navigation information (road guidance information, road sign information, etc.) generation unit 404, a driving route information acquisition unit 406 that acquires driving route information based on destination information etc. set by the occupant 3, a vehicle position information acquisition unit 408, a map information acquisition unit 410, and a memory unit (database that stores maps, road guidance information, road signs, etc.) 412.

[0041] The HUD device 100 is installed, for example, in a dashboard (not shown) of the vehicle 1. The HUD device 100 includes a stereoscopic display device 111, an optical system 116, a light exit window 118, and a display control device 140.

[0042] Here, the stereoscopic display device 111 is a parallax 3D display device. The stereoscopic display device (parallax 3D display device) 111 includes an image generation unit 112, an image display unit (such as a liquid crystal display device having an image display surface for displaying an image) 113, a light beam separation unit 114 having a lenticular lens, a parallax barrier (parallax barrier), or the like, which separates light emitted from the image display surface into light beams for the left and right eyes, and a display control unit 130.

[0043] The display control device 140 is configured by, for example, at least one processor that operates based on a display control program.

[0044] The display control device 140 has a control unit 141 that controls image display. This control unit 141 has a visibility control unit 142 that controls the visibility (brightness, etc.) of the display image, a lower limit value / upper limit value 144 of the visibility when changing the visibility, a decrease / increase start threshold 146 that indicates the timing to start decreasing / increasing the visibility, a time to reach the lower limit value / upper limit value (lower limit arrival time / upper limit arrival time) 148, and an importance 149 that is set for each type of display.

[0045] The optical system 116 has a curved mirror (concave mirror or the like) 117 that reflects the light from the light beam separation unit 114 and projects the image display lights K1 and K2 onto the windshield (projection target member) 2. However, the optical system 116 may further have other optical members (lenses, auxiliary reflecting mirrors, etc.).

[0046] In the in-vehicle display system 10 of FIG. 1, the stereoscopic display device 111 of the HUD device 100 displays an image with parallax (parallax image) for each of the left and right eyes, and an image 26 without parallax. Each parallax image is displayed as a virtual image 25L for the left eye and a virtual image 25R for the right eye, which are formed on an adjustment surface (imaging surface) PS, as shown in the upper left of Fig. 1. The focus of each eye of the occupant (passenger) 3 is adjusted to match the position of the adjustment surface PS.

[0047] The position of the adjustment surface PS is referred to as the "adjustment position," and the distance from a predetermined reference position set on the vehicle 1 side (for example, the viewpoint position OP shown in Figure 3(C), or a reference position set at a predetermined location on the vehicle 1, or a reference position set in the space near the vehicle 1) to the adjustment surface PS is referred to as the "adjustment distance."

[0048] However, in reality, the human brain fuses the images (virtual images) 25L and 25R, and so the human perceives a three-dimensional virtual image (here, a "superimposed depth image" 27, which is superimposed on a three-dimensional navigation arrow figure indicating that the vehicle 1 is turning left on the road (in other words, the left turn road, or the real space or background near the left turn road, etc.), with an emphasis on the sense of depth) 27 being displayed on the convergence plane VS at a position further back than the adjustment position (a position determined by the convergence angle, which is called the "convergence position").

[0049] The superimposed depth image 27 can be referred to as a "stereoscopic virtual image" or a "stereoscopic image," and can also be called a "stereoscopic image," "3D display," etc. In the drawing, the superimposed depth image is indicated as 27(3D), and 3D indicates that it is a three-dimensional display.

[0050] In addition, a vehicle speed display (50 km / h), which is a non-overlapping two-dimensional image (2D display) that does not have parallax and is not intended to be superimposed on the actual scene, is also displayed on the convergence plane VS in parallel (simultaneously) with the above-mentioned superimposed depth image 27 (3D). In the figure, the vehicle speed display is indicated as 28 (2D), and 2D indicates that it is a two-dimensional display.

[0051] The "adjustment distance" is a physical distance. In the example of Fig. 1, the depth-superimposed image 27 (3D) and the non-superimposed two-dimensional image, the vehicle speed display 28 (2D), have the same convergence distance, which is a physical distance. However, the depth superimposed image 27 (3D) appears to the occupant 3 with an enhanced sense of depth, and is therefore perceived as being located farther away. Therefore, the perceived distance of the depth superimposed image 27 (3D) is longer than the perceived distance of the vehicle speed display 28 (2D), which is a non-superimposed two-dimensional image.

[0052] The visibility control unit 142 detects that the vehicle 1 is turning on the road based on at least one of the following pieces of information: the steering angle of the vehicle 1 detected by the steering angle sensor 509, the running position of the vehicle 1 acquired by the vehicle position information acquisition unit 408, the translational movement speed of the vehicle 1 detected by the translational speed sensor 507, and the gaze direction of the occupant 3 detected by the gaze detection unit 44.

[0053] In addition, the visibility control unit 142 acquires information on the viewpoint position estimated by the viewpoint (gazing point) estimation unit 49 as the point where the occupant 3 is looking, and when the vehicle 1 is turning on a road, it performs "visibility reduction control" to reduce the visibility of the image with the longer perceptual distance (here, 27(3D)) of the displayed images (27(3D), 28(2D)) to less than the visibility of the image with the shorter perceptual distance (here, 28(2D)).

[0054] In addition, when the vehicle 1 is about to finish turning on a road (including the state immediately before and after the turn), the visibility control unit 142 performs "visibility increase control" to increase the visibility of images with reduced visibility.

[0055] Furthermore, when performing the "visibility reduction control" and "visibility increase control," the "visibility change control" is implemented, which appropriately changes the manner in which visibility is reduced and the manner in which visibility is increased (in other words, the manner in which visibility is changed (altered)) based on the information on the acquired viewpoint position.

[0056] For example, assume that the vehicle 1 is turning on a road (for example, turning left or right) (including a state immediately before the turn and a state immediately after the turn). As described above, assume that the images displayed by the display device (HUD device 100) include an image (here, 27(3D)) with a long perceived distance (the distance perceived by the occupant 3 ahead) and an image (here, 28(2D)) with a short perceived distance.

[0057] When turning on a road, it is assumed that occupant 3 will often be looking at a point quite far into the road where the road is turning. At this time, if the displayed image comes into occupant 3's field of vision, occupant 3 may feel annoyed, or the image may distract occupant 3, delaying his or her confirmation of distant road conditions, etc.

[0058] Furthermore, images with a long perceived distance are more likely to be in the field of view of the occupant 3 who is looking at something far away.

[0059] Taking these points into consideration, in the example of Figure 1, greater importance is placed on the image (27(3D)) with a long perceptual distance, and visibility control is performed so that it does not enter the field of view of occupant 3 and interfere with occupant 3's visibility of road conditions, etc.

[0060] Specifically, when vehicle 1 is turning on the road, visibility reduction control is implemented to make the visibility of the image with the longer perceptual distance (27(3D)) equal to or lower than the visibility of the image with the shorter perceptual distance (27(2D)) (in other words, visibility control that prioritizes the reduction in visibility of the image with the longer perceptual distance over the reduction in visibility of the image with the shorter perceptual distance), and on the other hand, when vehicle 1 has finished turning on the road (including the states immediately before and after finishing the turn), visibility increase control is implemented to increase the visibility of the image with reduced visibility.

[0061] The visibility reduction control can prevent the image with the longer perceived distance (27(3D)) from entering the field of view of the occupant 3, causing annoyance to the occupant 3 or interfering with distant visibility.

[0062] Furthermore, once the vehicle 1 has finished turning the road, the image display resumes, for example, at the original visibility, so that the occupant 3 can immediately view the image showing useful information with easy-to-read visibility once the vehicle 1 has finished turning the road.

[0063] However, there are various shapes of "curved roads," for example, there are short curves that can be turned relatively easily, there are also curves that continue for a relatively long time ahead of the vehicle (long vertical curves), and there are also cases where there are a series of S-shaped curves, for example.

[0064] For example, on long vertical curves or S-shaped curves, occupants need to quickly grasp the situation on the curved road at a considerable distance and perform appropriate driving operations, which tends to take a long time to confirm the road conditions, etc. In this case, the period of time when the driver is not paying attention to what is ahead is longer, and the risk of danger while driving increases.

[0065] Therefore, when the occupant is looking further into the distance, it is important to further suppress factors that impede the occupant's visibility (in other words, to finely adjust the degree of visibility change according to the level of danger) so that the occupant can more quickly check road conditions, etc.

[0066] Considering these points, when carrying out the above-mentioned "visibility reduction control" and "visibility increase control," it is preferable to take into consideration the viewpoint position from which the occupant 3 is looking (in other words, taking into consideration how far away they are looking) and to more finely control the manner of visibility control, such as the lower limit value when reducing visibility, the timing at which visibility reduction begins, the time required for visibility to reach the lower limit value, and the manner of change when visibility is changed (rate of change, etc.).

[0067] For example, if occupant 1 is looking at something further away, it is preferable to set the lower limit of image visibility lower or to start reducing visibility earlier, taking sufficient care not to obstruct distant visibility.

[0068] Therefore, in the example of Figure 1, for example, the viewpoint position from which the occupant 1 is looking is estimated using a predetermined estimation algorithm or the like based on the steering angle of the vehicle 1, or further taking into consideration the importance of the displayed image, and visibility change control (adaptive visibility change control) is implemented to more finely change the manner in which visibility is reduced and the manner in which visibility is increased depending on the viewpoint position.

[0069] This makes it possible to further reduce the possibility that the displayed image will enter the field of view of the occupant 1 and obstruct distant visibility, improving the convenience of the display device, which also contributes to improving safety when the vehicle 1 turns on a road.

[0070] Furthermore, when controlling the visibility change, the visibility control unit 142 may also take into consideration the importance of each image to be displayed and adjust the degree of change in visibility.

[0071] In other words, in the above explanation, the manner in which visibility is changed (the visibility is varied) is adjusted taking into consideration the viewpoint position of the occupant 3, but the manner in which visibility is varied may also be adjusted in more detail taking into consideration the importance of the image being displayed (in other words, for example, the extent to which each type of image has an impact on the safe operation of the vehicle).

[0072] This enables more detailed (more precise) display control that takes into consideration safe driving, such as not significantly reducing the visibility of images that are important to the occupant 3, and keeping the information available when needed.

[0073] A specific example of display control will be described later (see FIGS. 4 and 5).

[0074] Next, reference will be made to Fig. 2. Fig. 2 is a diagram showing an example of the configuration of a display control device. In Fig. 2, parts that are common to Fig. 1 are given the same reference numerals. This also applies to the subsequent figures.

[0075] The visibility control unit 142 provided in the control unit 141 of the display control device 140 has a visibility control mode determination unit 143 and a control parameter memory 151 that stores various control parameters used to control visibility.

[0076] The visibility control mode determination unit 143 has a viewpoint position estimation unit 145 that estimates the viewpoint position indicating the position where the occupant (driver, viewer) 3 is looking, a distance (near / far) determination unit 147 to the viewpoint, and a visibility change mode determination unit 153 that appropriately determines (sets) the rate of change on the time axis when changing visibility (for example, whether to use a straight line or a curved line as the characteristic line indicating the change in visibility) based on driving situation information.

[0077] If the distance (near / far) from the occupant 3 to the viewpoint can be determined uniquely based on the estimated viewpoint, there is no need to provide the distance (near / far) to viewpoint determination unit 147. In other words, the distance (near / far) to viewpoint determination unit 147 can be referred to as a relative positional relationship determination unit 147 between the viewpoint and the occupant.

[0078] The viewpoint position estimation unit 145 estimates the viewpoint position of the occupant 3 based on input driving situation information (e.g., steering angle information of the vehicle 1, translational speed (translational motion speed) information of the vehicle 1, information on the line of sight indicating the direction in which the occupant 3 is looking, etc.).

[0079] As a method for estimating the viewpoint position of the occupant 3, the following methods (1) to (4) can be mentioned, for example. (1) Steering angle method This method involves collecting a large amount of data on the steering angle and viewpoint position of the vehicle 1 in advance, and statistically determining that if the steering angle is this value, the viewpoint of the occupant 3 should be in this location. Alternatively, the relationship between the steering angle of the vehicle 1 and the viewpoint position is learned by machine learning using, for example, AI (artificial intelligence), and the viewpoint is estimated from the steering angle using the obtained model. In the above method, for example, if the steering angle increases, it can be determined that the viewpoint position is far away. For example, an increase in the steering angle means that the road is turning and the viewpoint is at the end of the curve, i.e., the viewpoint is far away. In this case, the brightness of the image can be reduced to make it difficult for the image to enter the field of view of the occupant 3, allowing the occupant 3 to concentrate on visually checking the road conditions, etc., at the end of the curve. (2) Method based on vehicle position This method detects the driving position of vehicle 1 and, for example, determines from a map and vehicle position that the vehicle is on a curve, and determines that the viewpoint position is far away when, for example, the curvature of the curve becomes large (the radius of curvature becomes small). (3) Method based on the vehicle's translational velocity (translational movement velocity) This method estimates the curve curvature from the translational movement speed of the vehicle 1 in the forward / backward and left / right directions, and estimates the viewpoint position based on this estimate. For example, if the forward / backward speed is high and the left / right speed is low, it can be estimated (determined) that the curve curvature is small, and if the forward / backward speed is low and the left / right speed is high, it can be estimated (determined) that the curve curvature is large. (4) Method based on the occupant's line of sight This method detects the line of sight of the occupant 3, and estimates that the position of the viewpoint has become farther away if the angle of the occupant's gaze increases, for example.

[0080] In the configuration of FIG. 2, a distance (near / far) determination unit 147 to a viewpoint determines the distance (near / far) from the occupant 3 to the estimated viewpoint position.

[0081] As explained above, it is considered that the further away the occupant 3 looks, the longer it takes to check the road conditions, etc., and the longer the period of inattention to the road ahead, which increases the degree of danger. Therefore, it is considered that the longer the distance to the viewpoint determined by the distance (near / far) determination unit 147, the higher the degree of danger, and it is preferable to finely control the mode of visibility change in accordance with the degree of increase in the degree of danger.

[0082] In order to realize such detailed visibility change control, the control parameter memory 151 stores various control parameters.

[0083] Specifically, the control parameter memory 151 stores lower / upper limit values ​​144 for lowering or increasing visibility, thresholds (first and second thresholds) 146 for the timing to start lowering / increasing visibility, lower / upper limit reaching times 148 required for visibility to reach the lower / upper limit values, and importance 149 for each type of display (image).

[0084] The visibility control mode determination unit 143 can read and set the above control parameters as appropriate, and perform visibility change control based on the settings to generate a visibility control signal.

[0085] For example, when reducing visibility, if occupant 3 is looking at something further away, it is preferable to more effectively prevent the displayed image from interfering with occupant 3's ability to see things at a distance by setting the lower limit of the visibility of the image (display) lower, accelerating the timing at which visibility begins to decrease, shortening the time it takes to reach the lower limit, or making the change in the period immediately after visibility begins to decrease more abruptly.

[0086] The same applies when increasing visibility; if the occupant 3 is looking further away, it is preferable to set the upper limit of the visibility of the image (display) higher than a predetermined reference value (for example, a preset reference value when increasing visibility, or the original visibility value before the visibility change), or to start increasing the visibility earlier, or to speed up the time to reach the upper limit, or to make the change more abrupt in the period immediately after the visibility increase begins, so that the occupant who returns their gaze to the front can read the necessary information more quickly.

[0087] The visibility control mode determination unit 143 may be configured as a look-up table (LUT).

[0088] Next, reference is made to Fig. 3. Fig. 3(A) and (B) are diagrams showing examples of display forms when a plurality of images with different perceptual distances are displayed.

[0089] In the example of FIG. 1, an example has been described in which a 3D image and a 2D image are displayed as images with different perceptual distances, but images with different perceptual distances are not limited to this.

[0090] 3(A), two images 31 and 33 are displayed on the virtual image display surface PS. The virtual image display distances of the images (the distances from the eyes of the occupant 3 to the virtual image display surface PS) are the same.

[0091] However, in Fig. 3(A), the virtual image (image) 31 is drawn using perspective, which emphasizes the sense of depth. On the other hand, the virtual image (image) 33 is not drawn using perspective (or is drawn using perspective, but the degree of emphasis on the sense of depth is less than that of the virtual image 31). Therefore, in the example of Fig. 3(A), the perceived distance of the virtual image 31 is longer than that of the virtual image 33.

[0092] In the example of Figure 3(B), the two virtual images (images) 35 and 37 are not drawn in perspective (or, although each virtual image is drawn in perspective, there is no difference in the degree of emphasis on the sense of depth).

[0093] 3(B), the virtual image display surface PS' is inclined with respect to the road surface 6 on which the vehicle 1 is traveling, and the virtual image 35 is displayed higher on the virtual image display surface PS', so the virtual image display distance is longer than that of the virtual image 37. Therefore, the perceived distance of the virtual image 35 is also longer.

[0094] Next, reference is made to Fig. 4. Fig. 4 is a diagram showing an example of a case where visibility control is performed based on the viewpoint (point of gaze) of the occupant (driver) regarding the navigation display as a superimposed image and the instrument display (vehicle speed display) as a non-superimposed image when the vehicle is turning on a road.

[0095] Fig. 4 shows an example of a suitable driving situation (a suitable display example) to which the present invention can be applied. In A-1 of Fig. 4, an S-shaped curve is visible ahead of the vehicle 1. The images displayed include an image DR with a long perceptual distance and an image SP with a short perceptual distance.

[0096] 4, the area VS indicated by the dashed line is the virtual image display area of ​​the HUD device 100. Also, reference numeral 2 denotes a windshield, reference numeral 4 denotes a steering wheel, and reference numeral 40 denotes a road on which the vehicle 1 is traveling.

[0097] As shown in A-1 of Figure 4, the image DR with a long perceptual distance is a superimposed image (specifically, a 3D arrow image (navigation arrow image) indicating that a curve is to be turned) that is superimposed on the actual scene. This navigation arrow image DR is not a warning (attention-grabbing) image for the occupant 3, and therefore can be assumed to be an image of relatively low importance (however, this is just one example, and even if it is not a warning image, it may be determined to be of high importance in terms of road conditions and driving conditions, and it is preferable to determine the importance flexibly).

[0098] On the other hand, an image with a short perceptual distance is a non-superimposed image that is not intended to be superimposed on the actual scene (here, an instrument display showing the status of vehicle 1, specifically, a vehicle speed display SP (a text image of "50 km / h") showing the traveling speed of vehicle 1).

[0099] The arrow image DR as a superimposed image has a longer perceptual distance than the vehicle speed display SP as a non-superimposed image, and is therefore more likely to catch the eye of the occupant 3. Furthermore, the vehicle speed display SP as a non-superimposed image is important information that indicates the state of the vehicle 1, and is often required by law to be displayed at all times, so it is considered preferable to make it as visible as possible.

[0100] When such a display is made, care should be taken to reduce the visibility of the arrow image DR as a superimposed image when the vehicle 1 is turning on the road so as not to hinder the occupant 3 from seeing at a distance, while at the same time, it is preferable from the viewpoint of ensuring the safe driving of the vehicle 1 to ensure that the vehicle speed display SP as a non-superimposed image has at least a certain degree of brightness to maintain it in a visible state so that the occupant 3 can read the information.

[0101] At A-2 in FIG. 4, vehicle 1 is approaching the first curve.

[0102] Further, the viewpoint position Q1 from which the occupant 3 looks is estimated using one of the methods (1) to (4) described above.

[0103] The display of the navigation arrow image DR, which has a long perception distance, is temporarily erased so as not to interfere with the visibility of the road conditions ahead after a turn. However, this is just an example and is not limited to this. For example, the visibility of the navigation arrow image DR may be the same as the visibility of the vehicle speed display SP. However, the visibility of the navigation arrow DR is controlled so as not to be lower than the visibility of the vehicle speed display SP.

[0104] On the other hand, in A-2 of Fig. 3, the vehicle speed is decelerated to "20 km / h", and the visibility (brightness) of the vehicle speed display SP, which has a short perception distance, is set lower than the standard (normal) visibility (brightness) in A-1 of Fig. 4. This ensures that the vehicle speed display SP does not interfere with the occupant 3's visibility at a distance.

[0105] In A-3 of FIG. 4, the second curve is visible ahead of the occupant 3. The viewpoint (gaze position) from which the occupant 3 is looking is indicated by the symbol Q2. In A-3 of FIG. 4, the image DR of the navigation arrow continues to be temporarily hidden. Also, the vehicle speed display SP continues to have reduced visibility, similar to A-2 of FIG. 4.

[0106] At A-4 in Fig. 4, the vehicle 1 is approaching the second curve. The viewpoint (gaze position) of the occupant 3 is indicated by the symbol Q3. In A-4 of FIG. 4, the image DR of the navigation arrow continues to be temporarily hidden. Furthermore, the vehicle speed display SP shows an increase from 20 km / h to 25 km / h. Such a change in vehicle speed is considered to be important in terms of safe operation, and therefore, in A-4 of FIG. 4, even though the vehicle 1 is in the middle of turning a curve, the visibility of the vehicle speed display SP is quickly restored to its original visibility, in other words, the standard (normal) visibility in A-1 of FIG. 4.

[0107] At A-5 in FIG. 4, the vehicle 1 has just finished turning the curve. The viewpoint (gaze position) of the occupant 3 is indicated by the symbol Q4. The viewpoint position (gazing point) Q4 is in a relatively close position directly in front of the driver 3, and since the curve has been turned, the possibility of the driver not paying attention to what is ahead is reduced.

[0108] In A-5 of Figure 4, a sign indicating the speed limit (a sign indicating that the upper speed limit is "60 km / h") LM is displayed as a superimposed image, and the visibility of this sign LM has returned to the standard (normal) visibility in A-1 of Figure 4.

[0109] Furthermore, the vehicle speed display SP shows that the vehicle speed has increased from "25 km / h (A-4 in FIG. 4)" to "30 km / h." Since the vehicle 1 has finished turning the curve, there is no particular problem in restoring the visibility of the vehicle speed display SP to its original state. Therefore, in A-5 of Fig. 4, the visibility of the vehicle speed display SP is restored to the standard (normal) visibility in A-1 of Fig. 4.

[0110] Next, reference is made to Fig. 5. Fig. 5 is a diagram showing another example of a case where visibility control is performed based on the viewpoint (point of gaze) of the occupant (driver) for the navigation display as a superimposed image and the instrument display (vehicle speed display) as a non-superimposed image when the vehicle is turning on a road. In Fig. 5, parts that are common to Fig. 4 are assigned the same reference numerals.

[0111] In the example of FIG. 5, an example of image visibility control will be described in which a warning image with high importance is displayed as a superimposed image with a long perceptual distance.

[0112] A-1 in FIG. 5 is the same as A-1 in FIG. 4 described above, except that a leading vehicle CA is traveling ahead of the vehicle 1.

[0113] A-2 in FIG. 5 is the same as A-2 in FIG. 4 described above, except that a leading vehicle CA is traveling ahead of the vehicle 1.

[0114] A-3 in FIG. 5 is the same as A-3 in FIG. 4 described above, except that a leading vehicle CA is traveling ahead of the vehicle 1.

[0115] In A-4 of Figure 5, the operation of vehicle 1 is the same as that of A-4 of Figure 4 described above. However, in A-4 of Figure 5, leading vehicle CA is extremely close to vehicle 1, increasing the risk of contact or collision.

[0116] In such a situation, it is important to give priority to informing the driver of the danger rather than reducing the visibility of the superimposed image and temporarily hiding it. Therefore, in A-4 of Fig. 5, the warning display WN is superimposed on the preceding vehicle CA and displayed with normal visibility to notify the occupant 3 of the risk of contact or collision. This allows the occupant 3 to quickly decelerate by, for example, applying the brakes appropriately.

[0117] At A-5 in Figure 5, vehicle 1 has finished turning the curve, and the preceding vehicle CA has moved far away, so the dangerous situation has been resolved. Therefore, at this timing (in other words, at a timing later than the timing of A-5 in Figure 4 described above), the speed limit sign display LM and the vehicle speed display SP are restored to their original visibility (standard (normal) visibility at A-1 in Figure 5).

[0118] Next, reference is made to Fig. 6. Fig. 6 is a diagram showing an example of how the visibility of an image is changed (varied) on the time axis.

[0119] 2 can also control the visibility of at least one of two images with different perceptual distances based on a first characteristic line in which visibility changes linearly over time, or a second characteristic line in which visibility changes curvedly over time. In other words, it is possible to freely set the rate of change with respect to the time axis when changing (varying) the visibility of an image.

[0120] In the example of Figure 6, the characteristic line showing the change in visibility (luminance) over time for an image with a long perceptual distance is denoted as J1, and the characteristic line showing the change in visibility (luminance) over time for an image with a short perceptual distance is denoted as J2. Characteristic line J1 is shown as a solid line, and characteristic line J2 is shown as a dashed line.

[0121] The two images have a visibility (brightness) of P1 at time t0 (the timing before the vehicle 1 turns).

[0122] For images with a long perceptual distance, the occupant 3 will be inattentive to the road ahead for a long period of time, or it will take a long time to return their gaze to the original position after shifting their gaze, and therefore the image is determined to be highly dangerous, and therefore is temporarily hidden, for example, at an early timing, time t1, as shown by characteristic line J1. The non-display period lasts from time t1 to t4, and in the period from time t4 to t5, visibility increases in proportion to the passage of time according to the linear characteristic line J1, and returns to the original visibility at time t5.

[0123] On the other hand, for images with a short perceptual distance, the impact on the distant visibility of the occupant 3 is thought to be small, and as shown by characteristic line J2, the visibility (brightness) changes from P1 to P2 (P1>P2) at a slightly later timing t2. During the period from time t2 to t3, the visibility is maintained at P2.

[0124] During the period from time t3 to t5, the visibility returns from P2 to P1. During this period from time t3 to t5, the visibility can be increased in proportion to the passage of time in accordance with the linear characteristic line J2, or can be increased in accordance with the upwardly convex curved characteristic line J2' or the downwardly convex curved characteristic line J2''.

[0125] In this way, in the example of FIG. 6, visibility can be changed based on a linear characteristic line, and more complex changes in visibility can also be achieved using a curved characteristic line.

[0126] Next, reference is made to Fig. 7. Fig. 7 is a flowchart showing an example of control in the display control method of the present invention.

[0127] In step S1, the position of the viewpoint (point of gaze) is estimated.

[0128] In step S2, a plurality of images with different perceptual distances are generated.

[0129] In step S3, display is started and information on the viewpoint (gazing point) is acquired.

[0130] In step S4, based on the relative positional relationship between the perceived distance and the viewpoint (point of gaze), or the relative positional relationship between the perceived distance and the viewpoint (point of gaze) and the importance of the image, the lower limit value for when visibility (brightness) is reduced, the first threshold (first steering angle threshold) for the timing to start reducing the visibility (brightness), the time required for the visibility (brightness) to reach the lower limit value (time to reach the lower limit), the upper limit value for when visibility (brightness) is increased, the second threshold (second steering angle threshold) for the timing to start increasing or decreasing the visibility (brightness), and the time required for the visibility (brightness) to reach the upper limit value (time to reach the upper limit).

[0131] In step S5, various information is acquired, for example, the steering angle (actual measurement value) detected by the steering angle sensor.

[0132] In step S6, it is determined whether or not the steering angle (actual measurement value)≧first steering angle threshold is satisfied. If N, step S6 is repeated, and if Y, the process proceeds to step S7.

[0133] In step S7, the visibility (luminance) decrease flag is turned on, and the visibility (luminance) is decreased to the lower limit value in the lower limit arrival time.

[0134] In step S8, it is determined whether or not the steering angle (actual measurement value)=first steering angle threshold value is satisfied. If N, the process returns to step S7, and if Y, the process proceeds to step S9.

[0135] In step S9, the visibility (brightness) decrease flag is turned off, and the visibility is maintained.

[0136] In step S10, it is determined whether or not the steering angle (actual measurement value)<first steering angle threshold is satisfied. If N, the process returns to step S9, and if Y, the process proceeds to step S11.

[0137] In step S11, the visibility (luminance) increase flag is turned on, and the visibility (luminance) is increased to the upper limit value in the upper limit reaching time.

[0138] In step S12, it is determined whether or not the visibility has reached the upper limit value. If the answer is No, the process returns to step S11, and if the answer is Yes, the process proceeds to step S13.

[0139] In step S13, it is determined whether or not to end the display. If N, the process returns to step S1, and if Y, the display ends.

[0140] As described above, according to an embodiment of the present invention, when a vehicle is turning on a road, it is possible to prevent the image displayed on the display device from interfering with the occupants' visibility (gazing) at the driving route, etc.

[0141] In this specification, the term "vehicle" may be broadly interpreted as a vehicle. Furthermore, navigation-related terms (such as signs) are also broadly interpreted, taking into account the perspective of navigation information in the broad sense that is useful for driving a vehicle.

[0142] Furthermore, the terms road and road surface are to be interpreted broadly to include, for example, the deck of a large ship or the floor of an indoor parking lot.

[0143] Furthermore, display devices such as HUD devices also include those used as simulators (for example, aircraft simulators, simulators as game devices, etc.).

[0144] The present invention is not limited to the above-described exemplary embodiments, and those skilled in the art will be able to easily modify the above-described exemplary embodiments to the extent that they fall within the scope of the claims. [Explanation of symbols]

[0145] 1···Vehicle (host vehicle), 2···Windshield (projected object), 3···Occupant (driver, passenger), 10···In-vehicle display system, 25L, 25R···Parallax image of a navigation arrow displayed on a convergence surface, 26···Image showing vehicle speed displayed on a convergence surface, 27 (3D)···3D image of a navigation arrow as a superimposed image superimposed on a real scene (superimposed depth image), 28 (2D)···2D image showing vehicle speed (vehicle speed display) as a non-superimposed image (instrument display) not intended to be superimposed on a real scene, 43···Pupil detection for pupil (or face) detection Camera, 44... line of sight detection unit, 45... front (broadly speaking, surroundings) imaging camera (e.g., stereo camera), 46... image processing unit, 47... distance measurement unit, 48... object type / size detection unit, 49... viewpoint (gazing point) estimation unit, 100... HUD device, 111... stereoscopic display device (parallax type 3D display device, etc.), 112... image generation unit, 113... image display unit, 114... light beam separation unit, 116... optical system, 117... curved mirror (concave mirror, etc.), 118... light exit window, 120... radar unit as distance measurement means, 130... display control unit, 14 0···Display control device, 141···Control unit, 142···Visibility control unit, 143···Visibility control mode determination unit, 144···Lower limit value / upper limit value when changing visibility, 145···Viewpoint position estimation unit, 146···Decrease / increase start threshold when starting to decrease / increase visibility, 147···Distance to viewpoint (near / far) determination unit (unit for determining relative positional relationship between viewpoint and occupant), 148···Lower limit / upper limit arrival time, 149···Importance of each type of display, 151···Control parameter memory, 153···Visibility change mode (change rate over time, etc.) determination unit, 400· Navigation device (navigation ECU), 402, depth mapping unit, 404, navigation information (road guidance information, road sign information, etc.) generation unit, 406, driving route information acquisition unit, 408, vehicle position information acquisition unit, 410, map information acquisition unit, 412, memory unit (database for storing maps, road guidance information, road signs, etc.), 500, communication unit, 502, GPS receiving unit, 505, various sensors, 507, translational speed sensor, 509, steering angle sensor, 600, driving assistance system, 700, ECU (on-vehicle ECU),PS: Adjustment plane (imaging plane), OP: Viewpoint position as reference position (reference point), VS (VS1, VS2): Convergence plane, ER: Right eye, EL: Left eye, K1, K2: Image display light, BUS: Bus (system bus).

Claims

1. A display control device that controls display of an image in a display device that is mounted in a vehicle and allows an occupant of the vehicle to view an image as a virtual image, the images include first and second images having different perceived distances, which are distances perceived by the occupant ahead, the display control device has a visibility control unit capable of changing visibility of each of the first and second images, The visibility control unit detecting that the vehicle is turning on a road based on at least one of the information on the steering angle of the vehicle, the traveling position of the vehicle, the translational motion speed of the vehicle, and the line of sight direction of an occupant of the vehicle, and acquiring information on a viewpoint position estimated to be a point where the occupant is looking; When the vehicle is turning on a road, a visibility reduction control is performed to reduce the visibility of the image having a longer perceptual distance out of the first and second images to be equal to or lower than the visibility of the image having a shorter perceptual distance, When the vehicle has finished turning on the road, a visibility increase control is performed to increase the visibility of the image whose visibility has been reduced; and, When performing the visibility reduction control and the visibility increase control, a visibility change control is performed in which a mode in which the visibility is reduced and a mode in which the visibility is increased are changed based on the acquired information on the viewpoint position. Display control device.

2. The visibility control unit During the visibility change control, the degree of change in visibility is adjusted taking into consideration the importance of each of the first and second images. The display control device according to claim 1 .

3. For the first image, or for each of the first and second images, The lower limit for reducing visibility, a first threshold for when to start reducing visibility; a time required for visibility to reach the lower limit value; A change in visibility over time when visibility is reduced; The upper limit for increasing the reduced visibility, a second threshold for when to start increasing or decreasing visibility; The time required for visibility to reach the upper limit value, A change in visibility over time when visibility is increased; Set at least one of and performing the visibility change control based on the setting. The display control device according to claim 1 .

4. The visibility control unit When the image having the longer perceptual distance out of the first and second images is a superimposed image to be superimposed on a real scene, and the image having the shorter perceptual distance is a non-superimposed image that is not intended to be superimposed on the real scene, Or, the image having the longer perceptual distance of the first and second images is a three-dimensional image based on a parallax image or a first two-dimensional image in which the sense of depth is emphasized by drawing using perspective, and is a superimposed image to be superimposed on a real scene; When the image with the shorter perceptual distance is a second two-dimensional image in which the sense of depth is suppressed more than that of the first two-dimensional image, and the image is displayed in a non-overlapping manner without the intention of being overlaid on a real scene, During the visibility control, When the superimposed image is an image of low importance that does not include a warning to the occupant, the superimposed image is not displayed, and when the superimposed image is an image of high importance that includes the warning, the superimposed image is made visible at normal times, or when visible, the visibility is reduced below the visibility at normal times. Regarding the non-superimposed display, regardless of the importance of the superimposed image, the visibility is set to the visibility in normal times, or the visibility is lowered below the visibility in normal times when the superimposed image is visible. The display control device according to claim 1 .

5. the superimposed image of low importance is a navigation display that assists in driving the vehicle, the superimposed image with a high degree of importance is a warning display that notifies the occupant that the vehicle is in a dangerous state, the non-superimposed display is an instrument display showing the running state of the vehicle; The display control device according to claim 4 .

6. The visibility control unit The visibility of at least one of the first and second images is controlled based on a first characteristic line in which the visibility changes linearly over time, or a second characteristic line in which the visibility changes curvedly over time. The display control device according to claim 1 .

7. an image generating unit that generates the first and second images; an optical system that emits display light for the generated superimposed depth image; A display control device according to any one of claims 1 to 6; A display device having:

8. The display device includes: a head-up display device that displays the first and second images as virtual images by projecting the display light onto a projection target member of the vehicle; The display device according to claim 7 .

9. A display control method for controlling display of an image in a display device that is mounted on a vehicle and allows an occupant of the vehicle to view an image as a virtual image, the method comprising: the images include first and second images having different perceived distances, which are distances perceived by the occupant ahead, detecting that the vehicle is turning on a road based on at least one of the following information: a steering angle of the vehicle, a traveling position of the vehicle, a translational speed of the vehicle, and a line of sight direction of an occupant of the vehicle; and acquiring information on a viewpoint position estimated to be a point where the occupant is looking; a step of performing visibility reduction control to reduce the visibility of the image having the longer perceptual distance out of the first and second images to the visibility of the image having the shorter perceptual distance or lower when the vehicle is turning on the road, and performing visibility increase control to increase the visibility of the image having the reduced visibility when the vehicle has finished turning on the road; a step of performing visibility change control in which, during the visibility decrease control and the visibility increase control, a mode in which the visibility is decreased and a mode in which the visibility is increased are changed based on the acquired information on the viewpoint position; A display control method including:

Images