Display control device, display device, and display control method

Abstract

To clearly present a route to be traveled by an own vehicle by a display image for guiding the traveling route, and to allow an occupant to easily pay proper attention to a caution-needed object (caution-needed position).SOLUTION: A display control device including a controller capable of causing a viewer to perceive a sense of depth and displaying a superimposed depth image superimposed on a first object, the display control device comprising: The controller includes precarious object detector 142, and superimposed depth image display changing processor 146 that performs a first display changing process of temporarily hiding the entire superimposed depth image when it is determined that at least a part of the superimposed depth image is superimposed on the second object which is the precarious object, or performs a second display changing process of displaying the superimposed depth image such that a display position of the superimposed depth image is closer than a position of the second object and the superimposed depth image is not superimposed on the second object when viewed from the viewer.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The present invention relates to a display control device, a display device, a display control method, etc., that are mounted on a vehicle such as an automobile. [Background technology]

[0002] Examples of display devices that can display images (virtual images) with a sense of depth include a parallax 3D head-up display (HUD) device, a 3D head-up display (HUD) device with variable virtual image display distance that variably controls the distance to the virtual image display surface (virtual image display distance), and a 3D head-up display (HUD) device that uses an inclined surface to create a sense of depth by displaying a virtual image at any position on a virtual image display surface that is inclined relative to the road surface (a 3D head-up display position variable virtual image display device).

[0003] Patent Document 1 shows an in-vehicle stereo image display device that utilizes the principle of stereoscopic vision due to binocular parallax to display information about the outside of the vehicle, guidance information for vehicle travel, etc. as a three-dimensional stereo image (stereoscopic image) (

[0029] ,

[0031] , Figure 10, etc.). For example, in

[0031] of Patent Document 1, the effect of the invention is described as follows: "The image can be recognized with a sense of depth and distance that matches the actual scene seen by the occupants through the windshield, and the advantage is that outside information and guidance information can be provided to the occupants accurately."

[0004] Patent Document 2 shows an example of a three-dimensional navigation display using a 3D HUD device with a variable virtual image display distance.

[0005] Furthermore, Patent Document 3 shows an example of a 3DHUD device that uses an inclined surface. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Application Publication No. 7-144578 [Patent Document 2] Japanese Patent Application Laid-Open No. 2015-69656 [Patent Document 3] Japanese Patent Application Publication No. 2018-120141 Summary of the Invention [Problem to be solved by the invention]

[0007] The present inventors have studied display control that allows a viewer to perceive a sense of depth, and have obtained the following novel findings.

[0008] In the following explanation, we will mainly use as an example a head-up display device (herein referred to as a parallax 3D HUD device) that allows the driver to view a 3D image by providing an arbitrary convergence angle to the image. However, the issues described below can be applied not only to parallax 3D HUD devices but also to other types of 3D HUD devices (display devices in a broad sense).

[0009] (1) For example, when the display control device described in Patent Document 1 is used to superimpose navigation content to the destination on the road surface, the navigation content is displayed just as it would on a normal road at points along the driving route where passengers (drivers, etc.) should pay attention (which can also be called "objects requiring caution" or "locations requiring caution"), such as railroad crossings, railroad tracks, stop signs, narrow roads, and sharp curves. Images of navigation content superimposed on the road surface are considered to be highly inductive. Furthermore, since the image of the navigation content has a sense of depth, if it is superimposed on part of an object requiring attention, it may obscure that part, making it difficult for the viewer to recognize the existence of the object requiring attention. As a result, the occupants' attention is focused solely on checking the driving route, and they may neglect to pay attention to the points where they should be careful (objects requiring caution, locations requiring caution), and it is not impossible that the occupants may end up driving without paying attention. For example, it is conceivable that a failure to carefully check one's surroundings could be one of the causes of an accident. (2) This problem is not mentioned in any of the above Patent Documents 1 to 3, and no solution to the problem is mentioned.

[0010] The present invention aims to make it easier for occupants and other observers to pay appropriate attention to objects requiring caution (locations requiring caution), and to enable vehicles to safely travel through areas where objects requiring caution are present.

[0011] Other objects of the present invention will become apparent to those skilled in the art by referring to the following exemplary aspects and best modes, as well as the accompanying drawings. [Means for solving the problem]

[0012] In order to facilitate an understanding of the outline of the present invention, the following examples are given of embodiments according to the present invention.

[0013] In a first aspect, the display control device is a display device mounted on a vehicle and capable of allowing a viewer to view an image as a virtual image, and includes a control unit that controls display of the image in the display device, wherein the image is a superimposed depth image that allows the viewer to perceive a sense of depth and is displayed superimposed on a first object, and the control unit has a superimposed depth image display change processing unit that acquires information indicating the position of an object that the viewer should pay attention to that is present on a driving path of the vehicle, or information that allows the position of the object to be detected, and when it is determined that at least a part of the superimposed depth image is superimposed on a second object that is a cautionary object other than the first object while the vehicle is driving, performs a first display change process to temporarily hide the entire superimposed depth image, or performs a second display change process to temporarily change the display mode of the superimposed display image so that the display position of the superimposed depth image is closer to the viewer than the position of the second object and so that the superimposed depth image is not superimposed on the second object.

[0014] According to the first aspect, when it is determined (predicted) that a superimposed depth image (for example, an image of a navigation arrow superimposed on the road surface, which is the first object) will also be superimposed on an object requiring caution (for example, a railroad track without a railroad crossing that crosses the road), a display change process is carried out to change the display mode of the superimposed depth image. The display change process may be categorized into two types: a first display change process in which the superimposed depth image is temporarily hidden to temporarily suspend the display; There is a "second display change process" in which the display of the superimposed depth image is maintained without interruption, but the size of the superimposed depth image is changed (including, for example, a change in the distance between the near end and the far end of the image as seen by the viewer), and the position or shape is changed so that the image is displayed closer to the viewer than the position of the object of interest, but not superimposed on the object of interest. In each of the first and second display change processes, the display mode of the superimposed depth image is temporarily changed, which allows the occupant (driver, etc.) to notice that there is some kind of unusual change near the point where the display mode has changed, and encourages them to drive with caution. Furthermore, by changing the display mode of the superimposed depth image, the superimposed depth image is no longer superimposed on the attention-requiring object, making it easier for the viewer to notice the presence of the attention-requiring object. As a result, the vehicle can safely travel through areas where there are objects requiring special attention.

[0015] In a second aspect dependent on the first aspect, the superimposed depth image may be a route guidance navigation image that guides the route that the vehicle should travel, and the route guidance navigation image may be displayed so as to be superimposed on the road surface of the road on which the vehicle, as the first object, is traveling.

[0016] As explained in the first aspect, the superimposed depth image for navigation superimposed on the road surface is highly instructive and has a sense of depth. Therefore, when it is superimposed on a part of an object requiring attention, it may obscure that part, making it difficult for the viewer to recognize the existence of the object requiring attention. According to the second aspect, a display change process is performed on the superimposed depth image for navigation. Therefore, for example, the route on which the vehicle should travel can be clearly presented by a display image (superimposed depth image for navigation) that guides the travel route, while at the same time, it is made easy for the occupant to pay appropriate attention to objects and locations that require caution, thereby achieving both.

[0017] In a third aspect dependent on the first or second aspect, the control unit may have a non-superimposed image display processing unit that performs a non-superimposed image display process to display a non-superimposed image that is not superimposed on the first object, farther away from the viewer than the position of the object to be watched and so that the non-superimposed image is not superimposed on the object to be watched.

[0018] In the third aspect, it is possible to display a non-superimposed image that is not superimposed on the first object (such as the road surface). In the first and second aspects, display control of the superimposed depth image makes it easier for the viewer to pay attention to objects requiring attention, but it is conceivable that this display control alone may not be sufficient to draw attention to objects requiring attention. According to this aspect, it is possible to display a non-overlapping image separately from the superimposed depth image, and by making this non-overlapping image, for example, an image of various traffic signs (for example, warning signs such as "Stop" or "Width decreasing"), it is possible to further increase the ability to draw attention to objects requiring caution. This allows the vehicle to travel more safely through areas where there are objects requiring caution.

[0019] In a fourth aspect dependent on the third aspect, the display processing unit of the non-superimposed image may also perform display processing of the non-superimposed image when display change processing of the first or second superimposed depth image is performed.

[0020] In the fourth aspect, the display processing of the non-superimposed image described in the third aspect is carried out in parallel with the "first display change processing (processing to temporarily hide the entire superimposed depth image)" or "second display change processing (display processing to continue displaying the superimposed depth image but change its size, position, etc. to avoid overlapping with objects requiring attention)" for the superimposed depth image described in the first aspect. When the first display change process is performed on the superimposed depth image, the image is temporarily hidden, and the presentation of information to the viewer by the image display is interrupted. According to this aspect, even in this case, it is possible to display a display (warning display) that calls attention to the object requiring attention, thereby enabling the vehicle to travel more safely through areas where the object requiring attention is present. Furthermore, by using the display process of the non-superimposed image together with the second display change process of the superimposed depth image, it is possible to reliably call attention to an object requiring caution while providing easy-to-understand guidance on the driving route, for example. This allows the viewer to safely pass through a point where an object requiring caution exists while checking the vehicle's driving route, etc.

[0021] In a fifth aspect dependent on the third or fourth aspect, the non-superimposed image may be a navigation image that has less guidance for the vehicle's driving route than the superimposed depth image, or may be an attention-warning image that alerts the viewer to the object requiring attention.

[0022] In the fifth aspect, a navigation image with low guidance for the vehicle's travel route (for example, map information that displays a bird's-eye view of the surrounding road conditions including the road currently being traveled) may be used as the non-superimposed image. Also, an attention-calling image (such as a traffic sign such as "Stop" or "Width narrowing") may be used to alert the viewer to an object requiring caution. For example, by displaying different types of non-overlapping images such as those described above together with a navigation image as a superimposed depth image, the viewer can more easily notice objects requiring caution while checking the driving route, etc., and can also pay sufficient attention to the objects requiring caution.

[0023] In a sixth aspect, the display device has an image generation unit that generates the superimposed depth image, an optical system that emits display light for the generated superimposed depth image, and a display control device of any one of the first to fifth aspects.

[0024] According to the sixth aspect, it is possible to realize a display device (on-board display device) that is more convenient than ever, which can clearly show the route that the vehicle should take, for example, by displaying an image that guides the driving route, while at the same time making it easier for the occupants to pay appropriate attention to objects requiring caution (locations requiring caution), thereby achieving both.

[0025] In a seventh aspect dependent on the sixth aspect, the display device may be a head-up display device that displays the superimposed depth image as a virtual image by projecting the display light onto a projection target of the vehicle.

[0026] According to the seventh aspect, it is possible to realize a head-up display (HUD) device (in-vehicle HUD device) with improved convenience that can clearly present the route that the vehicle should travel by, for example, using a display image that guides the travel route, while at the same time making it easier for the occupants to pay appropriate attention to objects requiring caution (locations requiring caution), thereby achieving both.

[0027] In an eighth aspect, a display control method is a display control method for controlling display of an image on a display device mounted on a vehicle and capable of allowing a viewer to view an image as a virtual image, the image being a superimposed depth image that allows the viewer to perceive a sense of depth and is displayed superimposed on a first object, the display control method including: an attention object detection step for detecting an object that exists on a driving path of the vehicle and that the viewer should pay attention to; and a superimposed depth image display change processing step for performing a display change process of the first superimposed depth image to temporarily hide the entire superimposed depth image when it is determined that at least a part of the superimposed depth image is superimposed on a second object that is different from the first object and is an attention object while the vehicle is driving, or performing a display change process of the second superimposed depth image to display the superimposed depth image so that the display position of the superimposed depth image is closer to the viewer than the position of the second object and so that the superimposed depth image is not superimposed on the second object.

[0028] According to the eighth aspect, it becomes possible for the vehicle to safely travel through a location where an object requiring caution exists.

[0029] Those skilled in the art will easily understand that the exemplified embodiments according to the present invention can be further modified without departing from the spirit of the present invention. [Brief explanation of the drawings]

[0030] [Figure 1] FIG. 1 is a diagram showing an example of the configuration of an in-vehicle display system including a display control device. [Figure 2] Figure 2(A) shows an example of a superimposed depth image displayed on a parallax-type 3D display device, which is superimposed on the road surface and provides a sense of depth. Figure 2(B) shows an example of a superimposed depth image displayed on an inclined surface HUD device, which uses an inclined surface as a virtual image display surface. [Figure 3] 3A and 3B are diagrams showing a display example in a conventional example, and FIGS. 3C and 3D are diagrams showing a display example in the first embodiment of the present invention (an example in which the entire superimposed depth image is temporarily hidden). [Figure 4] 4A and 4B are diagrams showing a display example in the second embodiment of the present invention (an example in which the size and arrangement of the superimposed depth image are changed and it is displayed in front of the object requiring attention as seen from the viewer). [Figure 5] 5(A) and (B) are diagrams showing a display example in the third embodiment of the present invention (an example in which a non-superimposed image (attention-calling image) that is not superimposed on the road surface is displayed instead of a superimposed depth image). [Figure 6] 6A and 6B are diagrams showing a display example (an example in which the display mode of FIG. 4 and the display mode of FIG. 5 are used together) in the fourth embodiment of the present invention. [Figure 7] FIG. 7 is a diagram showing a display example (yet another example in which the display mode of FIG. 4 and the display mode of FIG. 5 are used together) in the fifth embodiment of the present invention. [Figure 8] FIG. 8(A) is a diagram showing a conventional display example when the road width decreases, and FIG. 8(B) is a diagram showing a display example in the sixth embodiment of the present invention (another example when the display mode of FIG. 4 and the display mode of FIG. 5 are used together). [Figure 9] FIG. 9(A) is a diagram showing a conventional display example on a curved road, and FIG. 9(B) is a diagram showing a display example in the seventh embodiment of the present invention (another example in which the display mode of FIG. 4 and the display mode of FIG. 5 are used together). [Figure 10] 3 is a flowchart showing a main control example of the display control method of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0031] The best mode described below is used to facilitate understanding of the present invention, and therefore, those skilled in the art should be aware that the present invention is not unduly limited by the embodiments described below.

[0032] (First embodiment) Please refer to Fig. 1. Fig. 1 is a diagram showing an example of the configuration of an in-vehicle display system including a display control device. Fig. 1 shows an in-vehicle display system including a parallax 3D HUD device as an in-vehicle display system 10 capable of displaying a stereoscopic image with a sense of depth. However, the present invention is not limited to this.

[0033] In Fig. 1, the width direction of the vehicle 1 is defined as the left-right direction (or lateral direction: X direction), the direction along a line segment that is perpendicular to the left-right direction and perpendicular to the ground or a surface equivalent to the ground (here, road surface 6) is defined as the up-down direction (or height direction: Y direction), and the direction along a line segment that is perpendicular to both the left-right direction and the up-down direction (directions that indicate the forward and backward directions of the vehicle 1) is defined as the fore-aft direction (Z direction). The positive Z direction is defined as the forward direction, and the negative Z direction is defined as the backward direction. This is the same in the other drawings.

[0034] In the following description, the reference numeral 6 (or the reference numeral 6' or 6'') may refer to the road surface or the road itself.

[0035] In Figure 1, the in-vehicle display system 10 provided in a vehicle (own vehicle) 1 has an pupil detection camera 43 for pupil (or face) detection that detects the gaze direction and position of the left eye EL and right eye ER of the viewer (occupant such as the driver) 3, a viewpoint position detection unit 44, a front (broadly speaking, surroundings) imaging camera (e.g., a stereo camera) 45, an image processing unit 46 (including a distance measurement unit 47 and an object type / size detection unit 48), an HUD device 100, a radar unit 120 as a distance measurement means, a navigation device (navigation ECU) 400, a communication unit 500 (capable of communicating with a driving assistance system 600 and having functions such as GPS communication and vehicle-to-vehicle communication), a GPS receiving unit 502, various sensors 505, and an ECU 700 that can collect various information related to the vehicle 1 (e.g., lighting on / off information, vehicle speed information, engine information, etc.). The ECU 700 is capable of mutual communication with the navigation device (navigation ECU) 400 via a system bus BUS.

[0036] The distance measurement unit 47 included in the image processing unit 46 refers to a pair of left and right original images captured by a stereo camera as the imaging camera 45, for example, and detects the parallax to the same object (the target object) by, for example, stereo matching that searches for corresponding points in each image, and measures the distance to the target object using the principle of triangulation based on this parallax.

[0037] Furthermore, the radar unit 120 emits radio waves toward an object (a target object in the distance) and measures the reflected waves to measure the distance and direction to the object (a target object in the distance).

[0038] The navigation device (navigation ECU) has a depth mapping unit 402, a navigation information (road guidance information, road sign information, etc.) generation unit 404, a driving route information acquisition unit 406 that acquires driving route information based on destination information etc. set by the occupant 3, a vehicle position information acquisition unit 408, a map information acquisition unit 410, and a memory unit (database that stores maps, road guidance information, road signs, etc.) 412.

[0039] The HUD device 100 is installed, for example, in a dashboard (not shown) of the vehicle 1. The HUD device 100 includes a stereoscopic display device 111, an optical system 116, a light exit window 118, and a display control device 140.

[0040] Here, the stereoscopic display device 111 is a parallax 3D display device. The stereoscopic display device (parallax 3D display device) 111 includes an image generation unit 112, an image display unit (such as a liquid crystal display device having an image display surface for displaying an image) 113, a light beam separation unit 114 having a lenticular lens, a parallax barrier (parallax barrier), or the like, which separates light emitted from the image display surface into light beams for the left and right eyes, and a display control unit 130.

[0041] The display control device 140 is configured by at least one processor that operates based on a display control program, for example. The display control device 140 has a caution object detection unit 142, a caution position detection unit 144, a display change processing unit 146 for superimposed depth images, and a display processing unit 148 for non-superimposed images.

[0042] The optical system 116 has a curved mirror (concave mirror or the like) 117 that reflects the light from the light beam separation unit 114 and projects the image display lights K1 and K2 onto the windshield (projection target member) 2. However, the optical system 116 may further have other optical members (lenses, auxiliary reflecting mirrors, etc.).

[0043] In the in-vehicle display system 10 of Fig. 1, a stereoscopic display device 111 of the HUD device 100 displays images with parallax for each of the left and right eyes (parallax images). As shown in Fig. 1, each parallax image is displayed as a virtual image 25L for the left eye and a virtual image 25R for the right eye, which are formed on an adjustment surface (imaging surface) PS. The focus of each eye of the viewer (occupant) 3 is adjusted to match the position of the adjustment surface PS. The position of the adjustment surface PS is referred to as the "adjustment position," and the distance from a predetermined reference position set on the vehicle 1 side (for example, the viewpoint position OP shown in Figure 3(C), or a reference position set at a predetermined location on the vehicle 1, or a reference position set in the space near the vehicle 1) to the adjustment surface PS (see, for example, distance D1 in Figure 5(A)) is referred to as the "adjustment distance."

[0044] However, in reality, the human brain fuses the images (virtual images) 25L and 25R, so the human perceives a three-dimensional virtual image (here, a three-dimensional navigation arrow figure (in other words, a "superimposed depth image")) 27 superimposed on the road surface and with a sense of depth) on the convergence plane VS at a position further back than the adjustment position (a position determined by the convergence angle, which is called the "convergence position").

[0045] The superimposed depth image 27 can be called a "stereoscopic virtual image" or a "stereoscopic image", and can also be called a "stereoscopic image", "3D display", or the like.

[0046] Next, let us refer to Figure 2. Figure 2(A) is a diagram showing an example of a superimposed depth image displayed in a parallax-type 3D display device, which is superimposed on the road surface and provides a sense of depth, and Figure 2(B) is a diagram showing an example of a superimposed depth image displayed in an inclined surface HUD device that uses an inclined surface as a virtual image display surface. In Figure 2, parts that are common to Figure 1 are assigned the same reference numerals.

[0047] In Figure 2(A), in an area 21 (area indicated by a sand pattern) where parallax 3D display is possible, an adjustment surface (imaging surface) PS is set on the near side as seen by the viewer (passenger) 3, and parallax images GL and GR are displayed on PS.

[0048] A stereoscopic image (virtual image) is displayed (visually recognized) on the rear side of the adjustment plane (imaging plane) PS. Three examples of the display (visual recognition) mode are shown.

[0049] The virtual image Ga on the convergence plane VS1 is displayed in a manner that is inclined with respect to the road surface 6, which is the first object.

[0050] The virtual image Gb on the convergence plane VS2 is displayed in a manner that it is superimposed on the road surface 6, which is the first object. This virtual image Gb is superimposed on the road surface 6 and is a three-dimensional display that is perceived by the viewer 3 with a sense of depth. This virtual image GB can be said to be a typical example of a "superimposed depth image" in this embodiment.

[0051] The virtual image Gc on the convergence plane VS3 is displayed in a manner that it stands upright and is approximately perpendicular to the road surface 6. Any of the above-mentioned manners can be selected as appropriate. There are no particular restrictions on the height at which the virtual images Ga, Gb, and Gc are displayed (visually recognized), and the height position can be freely selected within the range 21 where 3D display is possible. In Figure 2(B), the virtual image display surface is an inclined surface, and an image (virtual image) is displayed on that inclined surface, so that an arrow AW indicating straight ahead is displayed in front of the viewer (passenger) 3, superimposed on the road surface 6 as the first object, and perceived with a sense of depth.

[0052] In this embodiment, either of the display modes shown in FIG. 2(A) and (B) can be used. Furthermore, even in a two-dimensional display, it is possible to pseudo-display a three-dimensional image with a sense of depth by using perspective, shading, etc. Such a display mode can also be used in this embodiment.

[0053] Next, reference will be made to Fig. 3. Fig. 3(A) and (B) are diagrams showing a display example in a conventional example, and Fig. 3(C) and (D) are diagrams showing a display example (an example in which the entire superimposed depth image is temporarily hidden) in the first embodiment of the present invention. In Fig. 3, parts common to Fig. 1 are assigned the same reference numerals.

[0054] In FIG. 3(A), the superimposed depth image is an image (virtual image) 27 of a navigation arrow superimposed on the road surface 6 of the road, which is the first object.

[0055] This superimposed depth image 27 is also superimposed on a second object (in other words, an object requiring caution) different from the first object (road surface 6), namely, a railroad track 50 without a railroad crossing, which crosses (in other words, crosses in the left-right direction) the road surface 6. In the figure, the symbol OP indicates the viewpoint (viewpoint position) of the viewer 3, which is the basis for the adjustment distance.

[0056] As shown in Figure 3(B), to the viewer (passenger) 3, the image (virtual image) 27 of the navigation arrow as a superimposed depth image is superimposed on the railroad tracks 50, which are an object requiring attention, and this superimposition appears to cover up part of the railroad tracks 50.

[0057] The image of the navigation arrow is highly inductive, and the viewer 3's attention may be focused on checking the route, and the viewer 3 may not pay attention to the tracks 50. Furthermore, as described above, the image of the navigation arrow obscures part of the tracks 50, which also prevents the viewer 3 from recognizing the tracks 50 and paying the necessary attention.

[0058] Therefore, in Figure 3 (C), when it is determined that at least a part of the superimposed depth image (navigation arrow) 27 is superimposed on a second object (railroad 50) that is a cautionary object other than the first object (road surface 6) while the vehicle 1 is traveling, the superimposed depth image display change processing unit 140 shown previously in Figure 1 performs a first display change process that temporarily hides the entire superimposed depth image 27.

[0059] In this specification, such display change control is referred to as a “first display change process” for the superimposed depth image. Note that the detection of a cautionary object is performed by the cautionary object detection unit 142 shown in FIG.

[0060] In this case, as shown in FIG. 3(D), the superimposed depth image (navigation arrow) 27 that was displayed before the first display change process is temporarily hidden (in other words, the display is temporarily interrupted), which causes a visual change to the viewer (passenger) 3.

[0061] In other words, by temporarily changing the display mode of the superimposed depth image 27, the viewer (passenger) 3 can notice that there is some kind of change that is out of the ordinary near the point where the display mode has changed (near the tracks 50), and therefore will drive while paying attention to the tracks 50, which are an object requiring caution.

[0062] Furthermore, the first display change process causes the superimposed depth image 27 to no longer be superimposed on the object requiring attention (railroad tracks 50). In other words, as shown in Fig. 3(B), the superimposed depth image 27 does not obscure a portion of the railroad tracks 50, and therefore the viewer (passenger) 3 can clearly see the railroad tracks 50, which are the object requiring attention, and is more likely to notice their presence.

[0063] As a result, the vehicle 1 can travel more safely than ever before through areas where the railroad tracks 50, which are objects requiring caution, are present.

[0064] (Second embodiment) Next, reference will be made to Fig. 4. Fig. 4(A) and (B) are diagrams showing a display example (an example in which the size and arrangement of the superimposed depth image are changed and the image is displayed in front of the attention-requiring object as seen from the viewer) in the second embodiment of the present invention.

[0065] In the example of FIG. 4, the superimposed depth image display change processing unit 146 shown in FIG. 1 performs a "second display change process" on the superimposed depth image (navigation arrow) 27.

[0066] The second display change process maintains the display of the superimposed depth image 27 without interrupting it, but changes the size, position, shape, etc. of the superimposed depth image 27 so that it is displayed closer to the viewer (passenger) 3 than the position of the object of caution 50 and does not overlap with the object of caution 50.

[0067] 4A, a parallax image 25′ (including a virtual image 25L′ for the left eye and a virtual image 25R′ for the right eye) is displayed on the adjustment surface PS. The sizes and display positions of the virtual image 25L′ for the left eye and the virtual image 25R′ for the right eye are adjusted as appropriate.

[0068] As a result, as shown in Figure 4(B), the superimposed depth image (navigation arrow) 27 is displayed closer to the viewer (passenger) 3 than the position of the object of caution 50, and so as not to overlap with the object of caution 50.

[0069] By carrying out this second display change process, the viewer (passenger) 3 is given a visual change, and is able to notice that there is some kind of unusual change near the point where the display mode has changed (near the tracks 50), and therefore will be inclined to drive while paying attention to the tracks 50, which are an object requiring caution.

[0070] Furthermore, the second display change process causes the superimposed depth image 27 to no longer be superimposed on the object requiring attention (railroad tracks 50). In other words, as shown in Fig. 4(B), the superimposed depth image 27 does not obscure a portion of the railroad tracks 50, and therefore the viewer (passenger) 3 can clearly see the railroad tracks 50, which are the object requiring attention, and is more likely to notice their presence.

[0071] Furthermore, unlike Figure 3(B) described above, in Figure 4(B), an image 27 of a navigation arrow is displayed, so the viewer (passenger) 3 can still check the driving route as before, which has the effect of maintaining a sense of security.

[0072] As a result, the vehicle 1 can travel more safely than ever before through areas where the railroad tracks 50, which are objects requiring caution, are present.

[0073] Also, in Figure 4, D0 indicates the distance from the reference position (viewpoint) OP to the near end position R5 of the object to be watched as seen by the viewer 3, D1 indicates the adjustment distance, D2 indicates the distance from the convergence plane PS to the near end NP of the superimposed depth image 27' as seen by the viewer 3, D3 indicates the distance from the reference position (viewpoint) OP to the near end NP of the superimposed depth image 27' as seen by the viewer 3, D4 indicates the extension distance of the superimposed depth image 27' in the direction of travel of the vehicle 1 (the distance from the near end NP to the far end FP), R1 indicates the position of the convergence plane PS, R2 indicates the position of the near end NP of the superimposed depth image 27', R4 indicates the position of the far end FP of the superimposed depth image 27', and R5 indicates the position of the near end of the object to be watched 50.

[0074] The display control device 140 can appropriately change and adjust the position and size of the superimposed depth image 27', for example, by changing the various distances and positions described above, or by changing the size and placement of the virtual images for the left eye and right eye at the convergence plane PS.

[0075] (Third embodiment) Next, reference will be made to Fig. 5. Fig. 5(A) and (B) are diagrams showing a display example (an example in which a non-superimposed image (e.g., a warning image) that is not superimposed on the road surface is displayed instead of a superimposed depth image) in a third embodiment of the present invention. In Fig. 5, parts that are common to Fig. 4 are given the same reference numerals.

[0076] The example of FIG. 5 is common to the examples of FIG. 3(C) and FIG. 3(D) shown above in that the superimposed depth image (image of the navigation arrow) 27 is temporarily hidden.

[0077] 4, however, in parallel with the display control for temporarily hiding the superimposed depth image (image of a navigation arrow) 27 superimposed on the road surface 6, which is the first object, a non-superimposed image display process is carried out for displaying a non-superimposed image that is not superimposed on the road surface 6, which is the first object, farther away from the position of the attention-requiring object 50 as seen by the viewer (passenger) 3 and so that the non-superimposed image is not superimposed on the attention-requiring object 50. This non-superimposed image display process is carried out by the non-superimposed image display processing unit 148 previously shown in FIG.

[0078] In the first display change process shown in Figures 3(C) and (D) or the second display change process shown in Figures 4(A) and (B), the display control of the superimposed depth image 27 makes it easier for the viewer (occupant) 3 to pay attention to the object requiring attention 50, but it is possible that this display control alone may not be sufficient to draw attention to the object requiring attention.

[0079] 5(A), a first adjustment plane PS1 and a second adjustment plane PS2 are provided, and a parallax image of the superimposed depth image is not displayed on the first adjustment plane PS1, and a parallax image 26 (including a virtual image 26L for the left eye and a virtual image 26R for the right eye) of a traffic sign saying "Stop" is displayed as a non-superimposed image on the convergence plane VS2. As a result, an image 28 of the traffic sign saying "Stop" is displayed as a non-superimposed image that is not superimposed on the road surface 6.

[0080] As shown in FIG. 5(B), the non-superimposed image 28 is displayed farther away from the position of the object to be watched 50 when viewed by the viewer (passenger) 3, and so that the non-superimposed image does not overlap with the object to be watched 50.

[0081] According to the example of FIG. 5, a non-overlapping image 28 can be displayed separately from the overlaid depth image 27. For example, by making this non-superimposed image 28 an image of various traffic signs (for example, an attention-grabbing display such as "Stop" or "Width narrowing"), it is possible to further increase the attention-grabbing capability of the caution-requiring object 50. Therefore, it becomes possible for the vehicle 1 to travel more safely through a point where the caution-requiring object 50 exists.

[0082] 5, D5 indicates the extension distance of the attention object 50 in the traveling direction of the vehicle 1 (the distance from the near end position R5 to the far end position R6), D6 indicates the distance from the edge position R6 of the attention object 50 to the non-superimposed image (attention calling image) 28, D7 indicates the distance from the reference position (viewpoint) OP to the non-superimposed image (attention calling image), R6 indicates the position of the far end of the attention object 50, and R7 indicates the position of the non-superimposed image (image of a traffic sign saying "Stop"). The other symbols are as explained above in the example of FIG. 4.

[0083] The display control device 140 can appropriately change and adjust the position and size of the superimposed depth image 27' and the non-superimposed image by changing the various distances and positions described above, or by changing the size and arrangement of the virtual images for the left eye and right eye on the convergence plane PS (PS1, PS2).

[0084] (Fourth embodiment) Please refer to Fig. 6. Fig. 6(A) and (B) are diagrams showing a display example in the fourth embodiment of the present invention (an example in which the display mode of Fig. 4 and the display mode of Fig. 5 are used together).

[0085] In the example of FIG. 6, the display mode (display control) of FIG. 4 and the display mode (display control) of FIG. 5 are used together.

[0086] In Figure 6(A), a first adjustment surface PS1 and a second adjustment surface PS2 are provided, and the first adjustment surface PS1 displays a parallax image 25' of the superimposed depth image described in Figure 4, and the second adjustment surface PS2 displays a parallax image 26 (including a virtual image 26L for the left eye and a virtual image 26R for the right eye) of a traffic sign saying "Stop" as a non-superimposed image.

[0087] As a result, an image 28 of a traffic sign saying "Stop" is displayed on the convergence surface VS2 as a non-overlapping image that is not superimposed on the road surface 6 of the road.

[0088] 6(B), a navigation arrow image 27' is displayed as a superimposed depth image on the near side of the attention-requiring object 50 as seen by the viewer 3, while a traffic sign image 28 saying "Stop" is displayed as a non-superimposed image on the far side. This non-superimposed image 28 is displayed farther away from the position of the attention-requiring object 50 as seen by the viewer (passenger) 3, and in such a way that the non-superimposed image is not superimposed on the attention-requiring object 50.

[0089] According to the example of Figure 6, the route that the vehicle 1 should travel is clearly presented using a display image (superimposed depth image 27' for navigation) that guides the travel route, while at the same time making it easier for the viewer (occupant) 3 to pay appropriate attention to the object 50 requiring attention (or the position requiring attention: not shown in Figure 6, see symbols R10 and R11 in Figures 8 and 9), thereby achieving both.

[0090] (Fifth embodiment) Next, reference will be made to Fig. 7. Fig. 7 is a diagram showing a display example (still another example in which the display mode of Fig. 4 and the display mode of Fig. 5 are used together) in the fifth embodiment of the present invention.

[0091] In the example of FIG. 7, similarly to the example of FIG. 6 described above, the display mode (display control) of FIG. 4 and the display mode (display control) of FIG. 5 are used in combination.

[0092] However, in Figure 7, the image of the "Stop" traffic sign (image of a navigation arrow as a superimposed depth image) 27' in Figure 6 has been replaced with an image 31 of an overhead map of the area around the road on which vehicle 1 is traveling. This bird's-eye view map image 31 can be said to be a navigation image that has low guidance for the travel route of the vehicle 1.

[0093] The route that the vehicle 1 should take can also be presented using an image 31 of an overhead map of the area around the road on which the vehicle 1 is traveling.

[0094] Therefore, in the example of Figure 7, as in the example of Figure 6 described above, the route that the vehicle 1 should travel is clearly presented, while at the same time making it easier for the viewer (occupant) 3 to pay appropriate attention to the object 50 requiring attention (or the position requiring attention: not shown in Figure 6, see symbols R10 and R11 in Figures 8 and 9), thereby achieving both.

[0095] (Sixth embodiment) Next, reference will be made to Fig. 8. Fig. 8(A) is a diagram showing a conventional display example when the road width is decreasing, and Fig. 8(B) is a diagram showing a display example in the sixth embodiment of the present invention (yet another example when the display mode of Fig. 4 and the display mode of Fig. 5 are used together).

[0096] In Figure 8(A), the width of road 6' suddenly changes at the caution point (caution point) R10. It is decreasing.

[0097] In FIG. 8(A), a point 51 where the width of the road 6' is reduced can be regarded as an object requiring caution. 8(A), a navigation arrow image 27 as a superimposed depth image is superimposed on the road surface 6' and also extends to and is superimposed on a portion 51 where the width of the road 6' is narrowing as an object requiring special attention. Therefore, the viewer (passenger) 3 may be distracted from the object requiring special attention 51.

[0098] Therefore, in FIG. 8(B), the display mode (display control) of FIG. 4 and the display mode (display control) of FIG. 5 described above are used together.

[0099] As shown in FIG. 8(B), a navigation arrow image 27'' is displayed as a superimposed depth image on the near side of the attention-requiring object 51 as viewed from the viewer 3, while an image 33 of a traffic sign indicating "narrowing road width" is displayed as a non-superimposed image on the far side. This non-superimposed image 33 is displayed farther away from the position of the attention-requiring object 51 as viewed from the viewer (passenger) 3, and in such a way that the non-superimposed image is not superimposed on the attention-requiring object 50.

[0100] According to the example of Figure 8, the route that the vehicle 1 should travel is clearly presented using a display image (superimposed depth image 27'' for navigation) that guides the travel route, while at the same time making it easier for the viewer (passenger) 3 to pay appropriate attention to the object 51 requiring attention (or the position R10 requiring attention), thereby achieving both.

[0101] (Seventh embodiment) Next, reference will be made to Fig. 9. Fig. 9(A) is a diagram showing a conventional display example on a curved road, and Fig. 9(B) is a diagram showing a display example in the seventh embodiment of the present invention (yet another example in which the display mode of Fig. 4 and the display mode of Fig. 5 are used together).

[0102] In FIG. 9(A), the road 6'' curves to the left at the caution position (point requiring caution) R11. In FIG. 9(A), the point 53 where the road 6'' curves to the left can be regarded as an object requiring caution.

[0103] In FIG. 9(A), a navigation arrow image 27 as a superimposed depth image is superimposed on the road surface 6'' and also extends to and is superimposed on a curved left portion 53 of the road 6'' as an object requiring attention. Therefore, the viewer (passenger) 3 may be distracted from the object requiring attention 53.

[0104] Therefore, in FIG. 9B, the display mode (display control) of FIG. 4 and the display mode (display control) of FIG. 5 described above are used together.

[0105] As shown in Figure 9(B), on the front side of the object of interest 53 as seen by the viewer 3, an image 27'' of a navigation arrow is displayed as a superimposed depth image, while on the back side, an image 35 of a traffic sign indicating "turn left" is displayed as a non-superimposed image. This non-superimposed image 35 is displayed farther away than the position of the attention-requiring object 53 when viewed by the viewer (passenger) 3, and in such a way that the non-superimposed image is not superimposed on the attention-requiring object 53.

[0106] According to the example of Figure 9, the route that the vehicle 1 should travel is clearly presented using a display image (superimposed depth image 27'' for navigation) that guides the travel route, while at the same time making it easier for the viewer (passenger) 3 to pay appropriate attention to the object 53 requiring attention (or the position R11 requiring attention), thereby achieving both.

[0107] (Eighth embodiment) Next, reference is made to Fig. 10. Fig. 10 is a flowchart showing a main control example of the display control method of the present invention.

[0108] In step S1, for example, the display control device 140 shown in FIG. 1 acquires route information to the destination from the navigation device 400.

[0109] In step S2, for example, the display control device 140 generates a superimposed depth image based on the image data mapped by the depth mapping unit 402 of the navigation device 400. Generate images (such as original images for navigation).

[0110] In step S3, the display control device 140 acquires information on the vehicle position and the cautionary position. For example, the cautionary object detection unit 142 detects the cautionary object included in the depth superimposed image, and the cautionary position detection unit 144 detects the cautionary position.

[0111] Subsequently, steps S4 and S5 are performed to determine whether or not the superimposed depth image is also superimposed on a cautionary object (or cautionary position).

[0112] In step S4, for example, the superimposed depth image display change processing unit 146 shown in FIG. 1 detects the far end position (or the leading end position) of the superimposed depth image. For example, referring to the example shown in FIG. 4 (an example in which a navigation arrow image 27' is displayed as a superimposed depth image on the near side of a railroad crossing), the "far end position of the superimposed depth image" is the position of the apex (top) FP of the arrow in the navigation arrow graphic. Note that the near end position (rear end position) is the position NP of the base end of the arrow graphic 27' closest to the viewer 3 in FIG. 4. These positions are determined, for example, by the distance from the viewpoint position OP of the viewer 3 (which is the reference position) in FIGS. 4 to 6.

[0113] In step S5, it is determined whether the caution position is farther away from the vehicle position and closer to the far end position of the superimposed depth image.

[0114] 4, the caution position is a near end (rear end) position R5 of the attention object 50. By determining the relative positional relationship between this caution position R5 and the edge position FP of the superimposed depth image (navigation arrow image) 27′, it is possible to determine whether at least a part of the superimposed snow image 27′ is superimposed on the attention object 50.

[0115] For example, in the example of Figure 4, if the sum of the adjustment distance D1, the distance D2 from the convergence plane PS to the near end position (rear end position) NP of the superimposed depth image (navigation arrow image) 27', and the distance D4 between the near end position and the edge position FP (extension distance of the superimposed depth image in the direction of travel of the vehicle 1) (this is the distance from the reference position OP to the far end FP of the superimposed depth image 27') is smaller than the distance D0 from the reference position OP to the attention position R5, it is determined that the superimposed depth image 27' is not superimposed on the attention object 50, and if it is larger, it is determined that the superimposed depth image 27' is superimposed on the attention object 50.

[0116] If the answer is Y in step S5 (if there is overlap), the process proceeds to step S6, and if the answer is N (if there is no overlap), the process proceeds to step S7.

[0117] In step S6, a display change process for the superimposed depth image (for example, the first display change process in FIG. 4 and the second display change process in FIG. 5) is performed, and if necessary, a display process for the non-superimposed image is also performed (for example, the display control in FIGS. 6 to 9).

[0118] In step S7, an image is displayed on a display device such as a HUD device.

[0119] In step S8, it is determined whether the vehicle has arrived at the destination. If the result is Y, the display control ends, and if the result is N, the process returns to step S1.

[0120] As described above, according to an embodiment of the present invention, it becomes easier for occupants and other observers to pay appropriate attention to objects requiring attention (locations requiring attention), and it becomes possible for a vehicle to travel safely through areas where objects requiring attention are present.

[0121] In this specification, the term "vehicle" may be broadly interpreted as a vehicle. Furthermore, navigation-related terms (such as signs) are also broadly interpreted, taking into account the perspective of navigation information in the broad sense that is useful for driving a vehicle. Furthermore, the terms road and road surface are to be interpreted broadly to include, for example, the deck of a large ship or the floor of an indoor parking lot. Furthermore, HUD devices and display devices (and display devices in a broad sense) also include those used as simulators (for example, aircraft simulators, simulators as game devices, etc.).

[0122] Furthermore, the term "object requiring attention" ("target requiring attention") is to be interpreted broadly and can broadly include, for example, an object that is at a height position similar to that of a road or the like on which the superimposed depth image is to be superimposed, and on which the superimposed depth image can be superimposed. For example, it may include obstacles on the road (whether stationary or moving).

[0123] The present invention is not limited to the above-described exemplary embodiments, and those skilled in the art will be able to easily modify the above-described exemplary embodiments to the extent that they fall within the scope of the claims.

[0124] The display control device 140 may not have all or part of the function of the caution object detection unit 142 that acquires data from a camera or a sensor and identifies a caution object, and the function of the caution position detection unit 144 that identifies the position of a caution object. In other words, the display control device 140 may acquire information indicating the position of a caution object detected by the caution object detection unit 142 and caution position detection unit 144 that are provided outside the display control device 140. [Explanation of symbols]

[0125] 1 Vehicle (own vehicle), 2 Windshield (projected object), 23 Viewer (driver or other passenger), 10 In-vehicle display system, 27 Superimposed depth image (image of a navigation arrow superimposed on the road surface, etc.), 43 Eye detection camera for detecting eyes (or faces), 44 Viewpoint position detection unit, 45 Front (broadly speaking, surroundings) imaging camera (e.g., stereo camera), 46 Image processing unit, 47 Distance measurement unit, 48 Object type / Size detection unit, 100...HUD device, 111...3D display device (parallax type 3D display device, etc.), 112...image generation unit, 113...image display unit, 114...light beam separation unit, 116...optical system, 117...curved mirror (concave mirror, etc.), 118...light exit window, 120...radar unit as distance measurement means, 130...display control unit, 140...display control device, 142...cautionary object detection unit, 144...cautionary position detection unit, 146...superimposition Depth image display change processing unit, 148...non-superimposed image display processing unit, 400...navigation device (navigation ECU), 402...depth mapping unit, 400...navigation information (road guidance information, road sign information, etc.) generation unit, 406...driving route information acquisition unit, 408...own vehicle position information acquisition unit, 410...map information acquisition unit, 412...storage unit (database for storing maps, road guidance information, road signs, etc.), 500...communication unit, 502... ··GPS receiver, 505··Various sensors, 600··Driver assistance system, 700··ECU, PS (PS1, PS2)··Adjustment surface (imaging surface), 25L··Virtual image (image) for the left eye of superimposed depth image, 25R···Virtual image (image) for the right eye of superimposed depth image, 26L···Virtual image (image) for the left eye of non-superimposed image, 26R···Virtual image (image) for the right eye of non-superimposed image, OP···Viewpoint position as reference position (reference point), VS (VS1, VS2)···Convergence surface.

Claims

1. A display control device is mounted on a vehicle and includes a control unit that controls display of an image in a display device that allows a viewer to view an image as a virtual image, the image is a superimposed depth image that allows the viewer to perceive a sense of depth and is displayed superimposed on a first object, The control unit acquiring information indicating the position of an object that the viewer should pay attention to and that is present on a travel route of the vehicle, or information that enables the position of the object to be detected; When it is determined that at least a part of the superimposed depth image is superimposed on a second object that is a caution object and is different from the first object while the vehicle is traveling, performing a first display change process of temporarily hiding the entire superimposed depth image; Or, a superimposed depth image display change processing unit that performs a second display change process by temporarily changing the display mode of the superimposed display image so that the display position of the superimposed depth image is closer to the viewer than the position of the second object and so that the superimposed depth image is not superimposed on the second object; A display control device having:

2. the superimposed depth image is a route guidance navigation image that guides the vehicle along a route to be traveled, and, the route guidance navigation image is displayed so as to be superimposed on a road surface on which the vehicle as the first object is traveling; The display control device according to claim 1 .

3. The control unit a non-superimposed image display processing unit that performs a non-superimposed image display process to display a non-superimposed image that is not superimposed on the first object at a position farther away from the viewer than the position of the attention-requiring object and so that the non-superimposed image is not superimposed on the attention-requiring object; The display control device according to claim 1 .

4. The non-superimposed image display processing unit When the display change process of the first or second superimposed depth image is performed, A display process of the non-superimposed image is also performed. The display control device according to claim 3 .

5. The non-overlapping image is a navigation image that has lower guidance for the vehicle's travel route than the superimposed depth image; Or, an attention-calling image that calls the viewer's attention to the attention-requiring object; The display control device according to claim 3 .

6. an image generation unit that generates the superimposed depth image; an optical system that emits display light for the generated superimposed depth image; A display control device according to any one of claims 1 to 5; A display device having:

7. The display device includes: a head-up display device that projects the display light onto a projection target of the vehicle to display the superimposed depth image as a virtual image; The display device according to claim 6.

8. A display control method for controlling display of an image in a display device that is mounted on a vehicle and allows a viewer to view an image as a virtual image, the method comprising: the image is a superimposed depth image that allows the viewer to perceive a sense of depth and is displayed superimposed on a first object, a warning object detection step of detecting an object that the viewer should pay attention to and that is present on a travel path of the vehicle; When it is determined that at least a part of the superimposed depth image is superimposed on a second object that is a caution object and is different from the first object while the vehicle is traveling, performing a display change process of the first superimposed depth image to temporarily hide the entire superimposed depth image; Or, a superimposed depth image display change processing step of performing a display change processing of the second superimposed depth image so that the display position of the superimposed depth image is closer to the viewer than the position of the second object and so that the superimposed depth image is not superimposed on the second object; A display control method comprising:

Images