Display control device, head-up display device, display control program, and display system for vehicle

Abstract

To prevent frame-out of a route guide at a course change point to allow display that does not cause confusion and anxiety to a driver who is a viewer.SOLUTION: A display control device 50 controls a display device for a vehicle 60 (60a) that displays a route guidance image as if looked down from a virtual viewpoint generated on the basis of route information included in map information, superimposed on a road included in a front view of a vehicle, on an image forming surface that is virtually set in front of the vehicle. The display control unit has a control unit 51 that, when the vehicle performs course change, displays the route guidance image RG so that the virtual viewpoint moves on a track RT passing on the inside of the route guidance image RG on the basis of position information of the vehicle.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to a display control device that controls a vehicle display device that displays a route guidance image generated based on route information included in map information along and superimposed on the road surface on which the vehicle travels, and the like.

Background Art

[0002] For example, in Patent Document 1, when a route line represented by route information included in map information for navigation is corrected (offset) in the vehicle width direction according to the distance along the vehicle width direction between the position of the vehicle and the route line, a content (route guide) for guiding the driving route of the vehicle is displayed at a position corresponding to the offset route line. A head-up display device (hereinafter, referred to as a HUD device unless otherwise specified) is described. According to this HUD device, it is possible to suppress the route guide from being displayed as if it is displaced in the vehicle width direction with respect to the driving lane, and thus, the appearance of the route guide can be improved.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] By the way, a viewing angle (the range that a virtual viewpoint camera can capture) is set for the HUD device. If, for example, in the HUD device, a virtual range in the space where a virtual image such as the above-described route guide can be imaged is defined as a virtual image formation surface (imaging region), the viewing angle is a viewing angle defined based on a virtual line connecting the viewpoint of the viewer, which is the driver, and the outer edge of the imaging region. In other words, the viewing angle is the angular range within which the viewer can view the virtual image.

[0005] Therefore, when the HUD device displays along the actual route guide at a route change point such as an intersection, the HUD device moves the virtual viewpoint camera along the route guide. As a result, when the vehicle enters a route change point such as an intersection, the route guide may extend beyond the viewing angle of the HUD device near the bending point of the intersection with the maximum curvature, and frame out from the field of view of the driver, who is the viewer, and disappear from the screen. In this case, it may cause confusion and anxiety to the driver, who is the viewer.

[0006] The present invention has been made to solve the above-described problems, and an object thereof is to provide a display control device and the like that can suppress the occurrence of frame-out of the route guide at a route change point and enable a display that does not cause confusion and anxiety to the driver, who is the viewer.

[0007] Other objects of the present invention will become apparent to those skilled in the art by referring to the embodiments and the best mode illustrated below and the accompanying drawings.

Means for Solving the Problems

[0008] Hereinafter, embodiments according to the present invention will be exemplified to facilitate understanding of the outline of the present invention.

[0009] A first aspect is a display control device that controls a vehicle display device that displays a route guidance image as seen from a virtual viewpoint generated based on route information included in map information on an imaging surface that is virtually set in front of a vehicle, wherein when the vehicle makes a route change, based on the position information of the vehicle, a control unit that displays the route guidance image such that the virtual viewpoint moves along a trajectory passing inside the route guidance image.

[0010] In the first aspect, when the vehicle changes its course, the control unit performs control to move a virtual viewpoint, which is a position serving as a reference for viewing the route guidance image, along a trajectory passing inside the route guidance image. Therefore, according to the first aspect, when the vehicle changes its course, for example, when turning right or left at an intersection, the virtual viewpoint moves along a trajectory passing inside the bending point with the maximum curvature of the route guidance image, so that when the vehicle passes through the actual route, the route guidance image deviates from the viewing angle of the vehicle display device, suppressing the phenomenon that the route guidance image that may occur near the bending point frames out from the field of view of the driver who is the viewer. Therefore, it is possible to reduce the chance of causing confusion and anxiety to the driver who is the viewer due to the route guidance image disappearing from the screen.

[0011] In a second aspect subordinate to the first aspect, the control unit sets a rotation start point at which the virtual viewpoint starts to pass inside the route guidance image and a rotation end point at which the virtual viewpoint finishes passing inside the route guidance image in accordance with the course change, and the route guidance image may be displayed such that the trajectory of the virtual viewpoint camera curves between the rotation start point and the rotation end point.

[0012] In the second aspect, by displaying the route guidance image such that the virtual viewpoint moves along a trajectory that curves between the point where the rotation starts (begins to curve) and the point where the rotation ends (ends curving) as the course changes, in addition to suppressing the frame-out phenomenon of the route guidance image, it is possible to suppress the sense of discomfort given to the driver who is the viewer.

[0013] In a third aspect subordinate to the second aspect, the control unit may display the route guidance image such that the trajectory of the virtual viewpoint curves within a triangular region connecting the rotation start point, the bending point with the maximum curvature in the route guidance image, and the rotation end point in accordance with the course change.

[0014] In the third aspect, the control unit displays the route guidance image such that the trajectory of the virtual viewpoint curves within the triangular region connecting the rotation start point, the bending point, and the rotation end point, thereby realizing a natural turning manner and suppressing the discomfort given to the driver, who is the viewer, with respect to the behavior of the vehicle, such as the vehicle speed. Here, the "route change point" refers to an intersection, a T-junction, a right or left turn point such as a curve, etc. that the vehicle passes through when traveling toward the destination.

[0015] In a fourth aspect subordinate to the third aspect, when the vehicle turns right to make the route change at an intersection, the control unit sets the trajectory of the virtual viewpoint such that the rotation start point is closer to the bending point than when the vehicle turns left, and the rotation end point is farther from the bending point than when the vehicle turns left, and may display the route guidance image accordingly.

[0016] In the fourth aspect, when turning right at an intersection, the control unit displays the route guidance image such that the trajectory of the virtual viewpoint has the point where the virtual viewpoint starts rotating closer to the bending point than when the vehicle turns left, and the point where the virtual viewpoint ends rotating farther from the bending point than when the vehicle turns left. Thus, a route change can be realized with a trajectory that takes into account the oncoming lane in the direction of travel (a turning manner that does not cross the oncoming lane in the direction of travel) when turning right. According to the fourth aspect, in the vehicle display device, when controlling the trajectory of the virtual viewpoint such that the point where the virtual viewpoint starts rotating is closer to the bending point than a trajectory passing inside the route guidance image, and the point where the vehicle 1 ends rotating is farther from the bending point than a trajectory passing inside the route guidance image, when the interval between the point where rotation starts and the point where rotation ends is short, it is assumed that the curvature of the trajectory approaches a right angle. Therefore, when the vehicle attempts to turn right at an intersection, the display will shortcut through the intersection and cross the oncoming lane, which may cause the route guidance image to frame out. Thus, it is useful in order to avoid this.

[0017] In a fifth aspect that depends on the first to fourth aspects, the control unit may display the route guidance image such that the orbit of the virtual viewpoint starts rotating when the vehicle approaches the route change point by a predetermined distance, and ends rotating when the vehicle moves away from the route change point by the predetermined distance.

[0018] In the fifth aspect, when the control unit moves the virtual viewpoint along an orbit that passes inside the actual route of the vehicle, the rotation starts when the vehicle approaches the route change point by a predetermined distance (for example, 6 [m]), and ends when the vehicle moves away from the route change point by a predetermined distance (for example, 6 [m]). This enables the driver, who is the viewer, to achieve an appropriate turning manner without feeling a sense of discomfort in the display of the route guidance image. Here, the 6 [m] used as the predetermined distance is based on the fact that the shortest distance that can be projected in 3D display in the HUD device used as the vehicle display device is 6 [m], and as a result of verification by the inventors, within 0 - 15 [m], a person can absolutely perceive an object (absolute perception distance), and within 15 - 30 [m], a person can relatively perceive an object (relative perception distance).

[0019] In a sixth aspect that depends on the first to fourth aspects, the control unit may display the route guidance image such that the orbit of the virtual viewpoint starts rotating when the vehicle approaches the route change point by a second predetermined distance that is longer than the predetermined distance, and ends the rotation when the vehicle moves away from the route change point by the predetermined distance.

[0020] In the sixth aspect, when the orbit of the virtual viewpoint moves along an orbit that passes inside the route guidance image, the rotation of the virtual viewpoint starts when the vehicle approaches the route change point by a second predetermined distance that is longer than the predetermined distance, and ends when the vehicle moves away from the route change point by the predetermined distance. This enables the driver, who is the viewer, to achieve an appropriate curve without feeling a sense of discomfort in the display of the route guidance image.

[0021] In a seventh aspect subordinate to the second aspect, the control unit may display the route guidance image such that the trajectory of the virtual viewpoint gradually bends at a second rotation start point extended from the rotation start point according to the magnitude of the curvature of the route change point, bends greatly at the rotation start point, and gradually returns between the rotation end point and a second rotation end point extended from the rotation end point.

[0022] In the seventh aspect, when the control unit displays the route guidance image, the trajectory of the virtual viewpoint gradually bends at a second rotation start point extended from the rotation start point according to the magnitude of the curvature of the route change point, bends greatly at the rotation start point, and gradually returns between the rotation end point and a second rotation end point extended from the rotation end point. Therefore, according to the seventh aspect, when the virtual viewpoint moves along a trajectory passing inside the route guidance image, by performing control to gradually bend between the extended second rotation start point and the second rotation end point extended from the point where the rotation ends, it is possible to suppress a sudden change in display at the original rotation start point and present a gentle bending manner, thus eliminating the sense of discomfort given to the driver who is the viewer.

[0023] In an eighth aspect subordinate to the seventh aspect, the control unit may display the route guidance image such that the greater the curvature of the route change point, the farther the second rotation start point is.

[0024] In the eighth aspect, when the control unit moves the trajectory of the virtual viewpoint along a trajectory passing inside the route guidance image, it performs display control to make the second rotation start point farther as the curvature of the route change point is greater. Thus, according to the eighth aspect, by performing control to gradually bend at the second rotation start point, it is possible to suppress a sudden change in display at the original rotation start point and present a gentle bending manner, thus eliminating the sense of discomfort given to the driver who is the viewer.

[0025] In a ninth aspect that depends on the fifth aspect, when the vehicle approaches the route change point by a predetermined distance, the control unit may control the display speed in accordance with the speed of the vehicle approaching the route change point.

[0026] In the ninth aspect, when the control unit moves the virtual viewpoint along a trajectory that passes inside the actual route of the vehicle, if the vehicle approaches the route change point by a predetermined distance, the control unit controls the display speed in accordance with the speed of the vehicle approaching the route change point. As a result, it is possible to perform a display effect that simulates the actual vehicle speed of the vehicle by controlling the display speed, and as a result, the driver as the viewer can visually recognize it with a sense of reality. Note that the control of the display speed can be realized by changing the frame rate (the number of images displayed per second) or the refresh rate (the screen update cycle) according to the detected vehicle speed.

[0027] In a tenth aspect that depends on the fifth aspect, when the trajectory of the virtual viewpoint is along the gradient or curvature of the road on which the vehicle travels toward the route change point, the control unit may display the route guidance image so as to match the traveling speed of the vehicle.

[0028] In the tenth aspect, when the control unit moves the virtual viewpoint along a trajectory that passes inside the route guidance image, the control unit controls the display speed in accordance with the gradient or curvature of the road on which the vehicle travels toward the route change point and the traveling speed of the vehicle. As a result, it is possible to realize an effect of simulating the actual traveling environment of the vehicle by controlling the display speed, and as a result, the driver as the viewer can visually recognize it with a sense of reality.

[0029] An eleventh aspect includes an image display unit that displays a route guidance image as seen from a virtual viewpoint generated based on route information included in map information, and a control unit that, when the vehicle makes a route change, displays the route guidance image so that the virtual viewpoint moves along a trajectory that passes inside the route guidance image based on the position information of the vehicle.

[0030] In the eleventh aspect, when the vehicle changes its course, the control unit displays the route guidance image so that a virtual viewpoint for imaging a virtual visual field range of the vehicle's forward visual field based on the viewpoint position of the viewer moves along a trajectory passing inside the route guidance image. Therefore, according to the eleventh aspect, when the vehicle changes its course, for example, when turning right or left at an intersection, the virtual viewpoint moves along a trajectory passing inside the bending point with the maximum curvature of the route guidance image, thereby suppressing the frame-out phenomenon of the route guidance image that may occur in the vicinity of the bending point due to the route guidance image deviating from the viewing angle of the image display unit. Thus, it is possible to reduce the chance of causing confusion and anxiety to the driver, who is the viewer, due to the route guidance image disappearing from the screen.

[0031] A twelfth aspect is a display control program for controlling a vehicle display device that displays a route guidance image as seen from a virtual viewpoint generated based on route information included in map information, and causes a processor included in the display control device to display the route guidance image such that when the vehicle changes its course, the virtual viewpoint moves along a trajectory passing inside the route guidance image based on the position information of the vehicle.

[0032] In the twelfth aspect, when the vehicle changes its course, the processor of the display control device reads and executes a program recorded in an externally attached or built-in memory, and performs control to display the route guidance image such that the virtual viewpoint moves along a trajectory passing inside the route guidance image. Therefore, according to the twelfth aspect, when the vehicle changes its course, for example, when turning right or left at an intersection, by displaying the route guidance image such that the virtual viewpoint moves along a trajectory passing inside the bending point with the maximum curvature of the route guidance image, it is possible to suppress the frame-out phenomenon of the route guidance image that may occur in the vicinity of the bending point due to the route guidance image deviating from the viewing angle of the vehicle display device when the vehicle passes along the actual route. Thus, it is possible to reduce the chance of causing confusion and anxiety to the driver, who is the viewer, due to the route guidance image disappearing from the screen.

[0033] Aspect 13 is a vehicle display system including a head-up display device that superimposes and displays a route guidance image generated based on route information included in map information on a road included in the forward field of view of a vehicle, and a display control device that, when the vehicle changes its course, performs control to display the route guidance image such that a virtual viewpoint moves along a trajectory passing inside the route guidance image based on the position information of the vehicle.

[0034] In Aspect 13, the display control device performs control to display the route guidance image such that when the vehicle changes its course, the virtual viewpoint moves along a trajectory passing inside the route guidance image based on the position information of the vehicle. Therefore, according to Aspect 13, when the vehicle changes its course, for example, when turning right or left at an intersection, the virtual viewpoint moves along a trajectory passing inside the bending point with the maximum curvature of the route guidance image, thereby suppressing the frame-out phenomenon of the route guidance image that may occur in the vicinity of the bending point due to the route guidance image deviating from the viewing angle of the vehicle display device when the vehicle passes through the actual route. Thus, it is possible to reduce the chance of causing confusion and anxiety to the driver, who is the viewer, due to the route guidance image disappearing from the screen. BRIEF DESCRIPTION OF THE DRAWINGS

[0035]

Figure 1A

Figure 1B

Figure 1C

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

MODE FOR CARRYING OUT THE INVENTION

[0036] The best mode for carrying out the invention described below is used for easily understanding the present invention. Therefore, those skilled in the art should note that the present invention is not unduly limited by the embodiments described below (hereinafter referred to as the present embodiment).

[0037] (Configuration of Embodiment) FIG. 1A is a diagram showing an application example of the HUD device 60a used as the vehicle display device 60 in the display control device 50 according to the present embodiment to the vehicle 1. In the following description, when the driver 4, who is the viewer of the HUD device 60a sitting in the driver's seat of the vehicle 1, faces the front of the vehicle 1, the left - right direction is the X - axis (the left direction is the positive X - axis direction), the up - down direction is the Y - axis (the up direction is the positive Y - axis direction), and the front - rear direction is the Z - axis (the front direction is the positive Z - axis direction).

[0038] Referring to FIG. 1A, the HUD device 60a is provided in the dashboard 5 of the vehicle 1 and includes a control unit 61 and an image display unit 62 (within the broken - line frame). The HUD device 60a emits display light L toward the front windshield 2 (an example of a projection - target member), and causes an image (a virtual image, which is a route - guidance image described later) to be visually recognized in an imaging region VA which is an imaging surface virtually set on the front - side (the positive Z - axis direction) of the front windshield 2. Thereby, the driver 4, who is the viewer, can visually recognize the route - guidance image R (see FIGS. 1C and 3(a)) superimposed on the foreground (front visual field FV (here, the road)), which is the real space visually recognized through the front windshield 2. G (See FIGS. 1C and 3(a)).

[0039] FIG. 1B is a view showing a scene that a driver 4 sees when sitting in the driver's seat of the vehicle 1 and facing forward. The driver 4 visually recognizes a foreground passing through the front windshield 2, and a route guidance image RG (or a map image IM11 including the route guidance image RG and the road image IM11) shown in FIG. 1C. The route guidance image RG or the map image IM10 including the route guidance image RG and the road image IM11 is displayed in an imaging region VA disposed at a position overlapping the foreground passing through the front windshield 2. Since the imaging region VA itself is not visually recognized by the driver 4 or is hardly visually recognized, the route guidance image RG or the map image IM10 including the route guidance image RG and the road image IM11 is visually recognized as floating in the air.

[0040] The imaging region VA is a region of a plane, a curved surface, or a partially curved surface on which an image generated inside the HUD device 60a is formed as a virtual image, and is also called an imaging surface. The imaging region VA itself is not actually visually recognized by the driver 4 or has low visibility to the extent that it is hardly visually recognized. In the imaging region VA, an angle (tilt angle θt) formed with a horizontal direction (XZ plane) about the left-right direction (X-axis direction) of the vehicle 1 as an axis, a line segment connecting the center of the instrument panel (the eye position of the driver 4) and the upper end of the imaging region VA, and a line segment connecting the center of the instrument panel (the eye position of the driver 4) and the lower end of the imaging region VA are used as a vertical painting angle, and an angle (vertical arrangement θv) formed by the bisector of this vertical painting angle and the horizontal direction (XZ plane) is set.

[0041] The image display unit 62 includes, for example, a liquid crystal display 621 (see FIG. 2) having a display surface for displaying an image, and an image projection unit 622 (see FIG. 2) including a relay optical system. The liquid crystal display 621 may be a transmissive display that transmits light from a backlight (not shown), or may be a projection display that projects an image (virtual image) onto a screen. In these cases, the display surface is the display surface in the transmissive display and the screen of the projection display. The liquid crystal display 621 may be attached with an actuator (not shown) such as a motor controlled by the control unit 61 and may be rotatable.

[0042] The image projection unit 622 is disposed on the optical path of the light (display light L) of the image (virtual image) from the liquid crystal display 621 between the liquid crystal display 621 and the front windshield 2, and projects the display light L of the image (virtual image) from the liquid crystal display 621 onto the front windshield 2 outside the image display unit 62. The image projection unit 622 is composed of one or more optical members. The image projection unit 622 includes at least one concave mirror, and in addition to this, for example, it may include refractive optical members such as one or more lenses, diffractive optical members such as holograms, reflective optical members, or combinations thereof.

[0043] The image projection unit 622 may be rotatable with an actuator (not shown) such as a motor controlled by the control unit 61. Specifically, while the vehicle display system 100 (see FIG. 2) is stopped, the actuator changes the position and angle of the image projection unit 622 (the relay optical system described later) so that external sunlight does not pass through the image projection unit 622 and reach the liquid crystal display 621. While the vehicle display system 100 is in operation, the position and angle of the relay optical system can be changed so that the display light L of the image from the liquid crystal display 621 is projected onto a predetermined position of the front windshield 2.

[0044] FIG. 2 is a block diagram showing a configuration example of a vehicle display system 100 including the display control device 50 according to the present embodiment. As shown in FIG. 2, in addition to the display control device 50, the vehicle display system 100 includes a vehicle ECU 10, a navigation device 20, and an in-vehicle monitoring device 30, which are connected via an I / O interface 40 so as to be able to communicate bidirectionally with each other.

[0045] The vehicle ECU 10 is an electronic control unit that controls drive systems such as engines and motors, braking systems such as brakes, or steering systems such as steering. For example, based on various detection information obtained from the in-vehicle monitoring device 30 described later, in addition to controlling the drive system, braking system, or steering system of the vehicle 1, it can provide various display information indicating the state of the vehicle 1 to the display control device 50 of the present embodiment via the I / O interface 40. Note that the vehicle ECU 10 includes electronic control units for driving assistance such as ACC (Adaptive Cruse Control) or for autonomous driving.

[0046] The navigation device 20 uses a global satellite navigation system (GNSS) such as a built-in GPS (Global Positioning System) and a gyro sensor to obtain map information from map information or by wireless communication with the outside of the vehicle, and performs guidance on surrounding facilities and route guidance for the vehicle 1. The navigation device 20 can transfer a signal that prompts a display output based on this guidance information to the display control device 50 of the present embodiment via the I / O interface 40 at an appropriate timing.

[0047] The navigation device 20 has a map information database (map information DB200). The map information DB200 can acquire and store the latest map information, for example, by communicating with an external center (not shown) via a V2X (Vehicle to X) type communication system (not shown). Here, the map information stored in the map information DB200 is mapping data that is digitized to represent the driving environment of the vehicle 1. The mapping data is preferably digital data of a particularly high-precision dynamic map. Here, the "dynamic map" refers to a digital map that combines a vast amount of dynamic information that changes moment by moment, such as traffic regulations, construction information, accidents and traffic jams, pedestrian and signal information, and static information such as high-precision three-dimensional position information (road surface information, diagonal line information, three-dimensional structures).

[0048] The in-vehicle monitoring device 30 is sensors necessary for recognizing the surrounding driving environment including the front of the vehicle 1, and includes a camera 31, a LiDAR 32 (Light Detection And Ranging), a behavior sensor 33, and the like. Here, the camera 31 captures at least the front view FV (real scene) of the vehicle 1 and the eyes (pupil images) of the passengers. The LiDAR 32 uses near-infrared light, visible light, or ultraviolet light to irradiate light, for example, on obstacles existing in front of the vehicle 1 photographed by the camera 31, captures the reflected light with a light sensor, and can determine the distance to the obstacle based on the time difference. The behavior sensor 33 includes an IMU (Inertial Measurement Unit), a steering angle sensor, a vehicle speed sensor, and the like. The IMU can measure (detect translational motion in three axial directions from acceleration [m / s 2 , and rotational motion from angular velocity [deg / s]) the driving situation and posture of the vehicle 1 using a three-axis acceleration sensor and a three-axis angular velocity sensor (gyro sensor). The steering angle sensor detects the steering angle of the steering wheel, and the vehicle speed sensor detects the speed of the vehicle 1, and can transfer them to the vehicle ECU 10, the navigation device 20, and the display control device 50 via the I / O interface 40, respectively.

[0049] The I / O interface 40 performs communication (also referred to as CAN communication) with the vehicle ECU 10 and the navigation device 20, in addition to the display control device 50 of the present embodiment, according to, for example, the CAN (Controller Area Network) standard. Note that the communication standard adopted by the I / O interface 40 is not limited to CAN, and includes, for example, wired communication interfaces such as CANFD (CAN with Flexible Data Rate), LIN (Local Interconnect Network), Ethernet (registered trademark), MOST (Media Oriented Systems Transport: MOST is a registered trademark), UART (Universal Asynchronous Receiver Transmitter), or USB (Universal Serial Bus), or in-vehicle communication (internal communication) interfaces such as personal area networks (PAN) such as Bluetooth (registered trademark) networks and short-range wireless communication interfaces within several tens of meters such as 802.11x Wi-Fi (registered trademark) local area networks (LAN).

[0050] Further, the I / O interface 40 may include an out-of-vehicle communication (external communication) interface such as a wide-area communication network (for example, an Internet communication network) according to cellular communication standards such as wireless wide-area network (WWAN0, IEEE802.16-2004 (WiMAX: Worldwide Interoperability for Microwave Access)), IEEE802.16e-based (Mobile WiMAX), 4G, 4G-LTE, LTE Advanced, 5G, etc.

[0051] The display control device 50 controls a vehicle display device 60 that superimposes and displays a route guidance image generated based on route information included in map information on the forward field of view FV of the vehicle 1. For this reason, the display control device 50, for example, as shown in FIG. 6, when the vehicle 1 changes its course, a virtual viewpoint camera V for displaying the route guidance image P(See Fig. 4(a) and (b) described later) is displayed on the route guidance image R G Orbit R that passes inside T and a storage unit 52 that stores programs and data necessary for the control unit 51 to perform the above-mentioned control. In Fig. 6, Rd denotes a road, and Cc denotes a lane change point Cc including an intersection where two or more roads Rd intersect.

[0052] FIG. 4(a) shows a virtual viewpoint camera V P However, the route guidance image R G FIG. 4(b) shows a conventional example in which, when an image is captured along the trajectory of the intersection Cc, the object goes out of the field of view of the HUD device 60a (the area that can be captured by the virtual viewpoint camera) and goes out of the frame near the bend point of the intersection Cc. P Route guidance image R G Orbit R passing through the inner side T Move the image to the route guide R G 1 shows an example of the present invention in which the chance of frame-out occurring is reduced by capturing an image of the subject.

[0053] Figure 4(a) and (b) show the V P indicates a virtual viewpoint camera that captures a route guidance image RG arranged in a virtual space based on map information. The virtual viewpoint camera VP has a predetermined virtual viewing range VA, and a route guidance image R G (Or, a road image IM11 arranged in a virtual space based on map information may be added to this) is displayed on the HUD device 60a. G The route guidance image route is R T is the route guidance image R along which the virtual viewpoint camera VP passes. G VA0 to VA3 are the virtual viewpoint cameras V P The field of view of the HUD device 60a (virtual viewpoint camera V PThe viewing angles VA0 to VA3 of the HUD device 60a, which indicate the range that can be imaged, are represented in a trapezoidal shape where the horizontal range is wide in the distance (upper side) and narrow in the vicinity (lower side).

[0054] In the example shown in FIG. 4(a), along the path indicated by the route guidance image R G the virtual viewpoint camera V P is being moved. In this case, when the virtual viewpoint camera V P reaches near the bending point of the intersection (route change point Cc) of the route guidance image R G superimposed on the road Rd arranged in the virtual space based on the map information, the route guidance image R G goes out of any of the viewing angles VA1, VA2, VA3 of the HUD device 60a and is not displayed on the HUD device 60a, that is, a so-called frame out occurs. On the other hand, in the example shown in FIG. 4(b), since the virtual viewpoint camera V P moves on an inner track near the bending point of the intersection of the route guidance image R G superimposed on the road Rd arranged in the virtual space based on the map information, it is difficult to have a frame out. That is, according to FIG. 4(b), since the orbit of the virtual viewpoint camera V P changing with the movement of the vehicle 1 is included in any of the virtual visual field ranges VA0, VA1, VA2, VA3, it is difficult to have a frame out.

[0055] Also, for example, as shown in FIG. 7, the control unit 51 can control the orbit of the virtual viewpoint camera V P to curve between the point S P where the virtual viewpoint camera V rp starts to rotate and the point E P where the virtual viewpoint camera V rp ends the rotation as the route of the vehicle 1 changes.

[0056] Also, for example, as shown in FIG. 7, the control unit 51 can control the orbit of the virtual viewpoint camera V P to curve between the point S P where the virtual viewpoint camera V rpand the bending point B that has the maximum curvature at the route change point Cc, and the virtual viewpoint camera V P and the point E where the rotation of rp ends, it is also possible to perform control so as to curve smoothly within the triangular curve region TA connecting them.

[0057] Further, for example, as shown in FIG. 8, when the vehicle 1 turns right to change its route at the intersection C C , the control unit 51 sets the position S P where the orbit of the virtual viewpoint camera V P starts to rotate to be closer to the bending point B than the orbit R G of the virtual viewpoint camera V when a vehicle 1 passing inside the route guidance image R turns left, and sets the position E P where the rotation of the virtual viewpoint camera V T ends to be farther from the bending point B than the orbit R rp of the virtual viewpoint camera V when the vehicle 1 turns left. This is a measure to avoid frame out by setting the position where the vehicle 1 starts to rotate to be close to the bending point B to obtain an orbit considering the lane, and further, by setting the position where the rotation ends to be far from the bending point B to make the curvature of the orbit gentle. In FIG. 8, L indicates shading. P P T rp

[0058]

[0058] Further, for example, as shown in FIG. 9(a), the control unit 51 starts the rotation of the orbit of the virtual viewpoint camera V P when the vehicle 1 approaches a predetermined distance (for example, 6 [m], which is a distance that the driver 4 can definitely perceive) to the route change point Cc (here, the bending point B at the center of the intersection), and passes inside the route guidance image R G , and ends the rotation when it is a predetermined distance (for example, 6 [m]) away from the route change point, and passes inside the route guidance image R T to perform control to end the rotation (rotation end point E rp ) with respect to the orbit R G passing inside. rp )

[0059] Also, for example, as shown in FIG. 9(b), the control unit 51 causes the virtual viewpoint camera V P to start rotating the orbit when the vehicle 1 approaches a second predetermined distance (for example, 12 [m], which is a distance that the driver 4 can relatively perceive) longer than a predetermined distance (for example, 6 [m]) from the route change point Cc (here, the bending point B at the center of the intersection) to the inside of the route guidance image R G along the orbit R T (rotation start point S rp ), and when the vehicle is separated from the route change point C c by a predetermined distance, the rotation with respect to the orbit is terminated (rotation end point E rp ). Control can also be performed

[0060] Also, for example, as shown in FIG. 10(a), the control unit 51 causes the virtual viewpoint camera V P to gradually bend the orbit R rp1 from the second rotation start point S rp2 extended from the rotation start point S T corresponding to the curvature of the route change point (assuming a curve in the case of a U-turn here) according to the magnitude of the curvature, start rotating at the rotation start point S rp1 bend greatly, and gradually return between the rotation end point E rp1 and the second rotation end point E rp2 extended from the rotation end point E. Control can also be performed

[0061] Also, for example, as shown in FIG. 10(b), the greater the curvature of the route change point C c , the control unit 51 moves the second rotation start point S rp1 (orbit R T without expansion control) extended from the rotation start point S rp2 farther from the bending point B (orbit R T´ with expansion control). Control can also be performed

[0062] Further, when the vehicle 1 approaches a route change point Cc by a predetermined distance (for example, 6 [m] or 12 [m]), the control unit 51 can also perform control to adjust the display speed according to the speed of the vehicle 1 approaching the route change point.

[0063] Further, the control unit 51 can also perform control to adjust the display speed according to the traveling speed of the vehicle 1 according to the gradient or curvature of the road on which the vehicle 1 travels toward the route change point Cc for the trajectory of the virtual viewpoint camera V P Note that, in order to perform the above-described control, the control unit 51 is implemented with, for example, a processor having a built-in memory (ROM / RAM) or an externally attached memory, and the processor reads and sequentially executes a program recorded in the memory (which may be the storage unit 52). When the vehicle 1 makes a route change, the virtual viewpoint camera that captures the route guidance image R

[0064] is moved along a trajectory R that passes inside the route guidance image R G Note that, in order to perform the above-described control, the control unit 51 is implemented with, for example, a processor having a built-in memory (ROM / RAM) or an externally attached memory, and the processor reads and sequentially executes a program recorded in the memory (which may be the storage unit 52). When the vehicle 1 makes a route change, the virtual viewpoint camera that captures the route guidance image R G is moved along a trajectory R that passes inside the route guidance image R T Thus, a function for performing control to move the virtual viewpoint camera along a trajectory R that passes inside the route guidance image R can be realized. Further, at least a part of the above-described functions can also be realized by hardware such as an FPGA (Field Programmable Gate Array) or a logic circuit, without relying on a processor.

[0065] The storage unit 52 is a memory in which, for example, a static RAM, a dynamic RAM, or a flash memory, etc., in which a program area and a work area are allocated, is implemented. Here, in the program area, a processor included in the display control device 50, when the vehicle 1 makes a route change, the virtual viewpoint camera V that captures the route guidance image R G is moved along a trajectory R that passes inside the route guidance image R based on the position information of the vehicle for the route guidance image R P Note that, in order to perform the above-described control, the control unit 51 is implemented with, for example, a processor having a built-in memory (ROM / RAM) or an externally attached memory, and the processor reads and sequentially executes a program recorded in the memory (which may be the storage unit 52). When the vehicle 1 makes a route change, the virtual viewpoint camera that captures the route guidance image R G is moved along a trajectory R that passes inside the route guidance image R TA program for executing a process of moving is written, and in the work area, for example, a route guidance image RG that sequentially changes as the vehicle 1 moves, or a map image IM10 including the route guidance image RG and the road image IM11, etc., which are generated during the execution of the above-described program, are assigned and updated.

[0066] In addition, in the above embodiment, the route guidance image R is arranged in the virtual space, and as the vehicle 1 moves, the position of the virtual viewpoint camera V G is changed, and the range captured by the virtual viewpoint camera V P is generated (rendered) as an image. However, it is not limited to this as long as the viewpoint for displaying the route guidance image R P can be displayed as passing through the inside of the bending point of the route guidance image R G . In some embodiments, the image data may be stored in the storage unit 52 in advance, and the image stored in the storage unit 52 in advance may be appropriately switched (sequentially displayed) according to the distance to the target (intersection). That is, the display control device (processor) may execute a process of displaying the route guidance image R G such that the virtual viewpoint moves along a trajectory passing through the inside of the route guidance image R G when the vehicle 1 changes its course, based on the position information of the vehicle 1. G

[0067] In addition, the vehicle display device 60 is not limited to the above-described HUD60a, and may be replaced by, for example, a center information display (CID) provided in the central part of the vehicle 1 or a head-mounted display device (HMD device) mounted on the head of the occupant.

[0068] (Operation of the embodiment) FIG. 5 is a flowchart showing the operation of the display control device 50 of the present embodiment, FIGS. 3(a) and (b) are diagrams showing screen display examples of a map image including a route guidance image, FIG. 3(a) is the route guidance image R C (route guide) generated by the display control device 50 of the present embodiment when entering the intersection C G , and FIG. 3(b) is the intersection CC A route guidance image R that is framed out from the viewing angle for displaying when entering G is shown by the prior art route guidance image R that frames out from the viewing angle for displaying when entering G (Route guide).

[0069] Further, FIG. 6 is a diagram showing both the route guidance image R generated by the display control device 50 according to the present embodiment G and the route R of the virtual viewpoint camera V P . FIG. 7 is a diagram showing the range (triangular curved region TA) in which the virtual viewpoint camera V T rotates at the route change point C C (intersection C C ). FIG. 8 is a diagram cited to explain the trajectory of the virtual viewpoint camera V P when turning right at the intersection C C (a turning method that does not cross the oncoming lane ahead). P FIG. 9(a) is an example of a trajectory of turning near the bending point B at the intersection C

[0070] and FIG. 9(b) is a diagram showing another example of the trajectory of turning near the bending point B at the intersection C C . FIG. 10(a) shows, in addition to the rotation start point S C and the end point E rp1 of the vehicle 1, an example of the extended rotation start point S rp1 (the second rotation start point) and the rotation end point E rp2 (the second rotation end point), and FIG. 10(b) is a diagram showing another example of the extended rotation start point S rp2 (the second rotation start point) and the rotation end point S rp2 (the second rotation end point). rp2 Hereinafter, the operation of the display control device 50 according to the present embodiment shown in FIG. 2 will be described in detail with reference to FIGS. 3 to 10(a)(b).

[0071]

[0072] ​5, the display control device 50 first determines whether or not it is time to display route guidance (step ST101). For example, when the driver 4, who is the viewer, requests route guidance by setting a destination by operating the navigation device 20, the control unit 51 acquires route information included in the map information (map information DB 200) based on the current location information of the vehicle 1 measured by the built-in GPS or the like (step ST103). Note that when it is determined that route guidance display by navigation is not requested or the vehicle ECU 10 for driving assistance such as ACC is traveling following a preceding vehicle, etc., it is not time to display route guidance (step ST101 "NO"), the control unit 51 executes display processing of other contents (for example, a following distance setting screen for ACC, etc.) (step ST102).

[0073] After acquiring the above-mentioned route information, the control unit 51 displays the route guidance image R G (route guide) is displayed on the vehicle display device 60 (HUD device 60a) (step ST104). G For example, the arrow mark may be an arrow mark having a width indicating the traveling direction of the vehicle 1 as shown in FIG.

[0074] Next, the control unit 51 judges whether or not the vehicle 1 has approached the lane change point Cc (here, an intersection) by a predetermined distance (step ST105). Here, the predetermined distance is, for example, 6 [m]. For example, as shown in FIG. 9(a), C When the robot approaches the central bending point B by 6 [m] (step ST105 “YES”), the virtual viewpoint (virtual viewpoint camera V P ) is the route guidance image R G Inside (trajectory R T ) as the route guidance image R starts to rotate through G (step ST106). If the predetermined distance is not reached (step ST105 "NO"), the control unit 51 repeats the process of step ST104.

[0075] Virtual viewpoint camera V P After the virtual viewpoint camera V starts rotating, for example, as shown in FIG. 6, the control unit 51 controls to display the route guidance image R P such that the virtual viewpoint (virtual viewpoint camera V G ), which captures the virtual field of view, moves along a trajectory R T inside the trajectory of the route guidance image R G (see FIG. 6 for example) (step ST107). Supplementing this control with FIGS. 4(a) and 4(b) described above, in the example shown in FIG. 4(a), the virtual viewpoint camera V G is moved along the route indicated by the route guidance image R P . In this case, when the virtual viewpoint camera V P approaches the vicinity of the bending point of the intersection Cc of the route guidance image R G , the route guidance image R G goes out of the viewing angles VA1, VA2, VA3 of the HUD device 60a and is not displayed on the HUD device 60a, that is, a so-called frame out occurs. On the other hand, in the example shown in FIG. 4(b), since the virtual viewpoint camera V P moves along the inner trajectory near the bending point of the intersection Cc of the route guidance image R G , it is difficult to have a frame out. That is, since the trajectory of the virtual viewpoint camera V P , which changes as the vehicle 1 moves, is included in any of the virtual fields of view VA0, VA1, VA2, VA3, it is difficult to have a frame out.

[0076] At this time, for example, as shown in FIG. 7, the control unit 51 controls to move the virtual viewpoint camera V P along a curved trajectory within a curved triangular region TA connecting the point S rp where the virtual viewpoint camera V starts rotating, the bending point B with the maximum curvature at the course change point, and the point E P where the virtual viewpoint camera V ends rotating rp ( P ).

[0077] Note that at this time, when the vehicle 1 approaches the course change point by a predetermined distance (for example, 6 [m]), the control unit 51 adjusts the course change point C according to the remaining distance.​​C By controlling the display speed according to the actual speed of the vehicle 1 approaching (obtained from the behavior sensor 33 of the in-vehicle monitoring device 30), the route guidance image R G is displayed on the vehicle display device 60 (HUD device 60a) (step ST108). Note that the control unit 51 controls the display speed according to the traveling speed of the vehicle 1 according to the gradient or curvature of the road on which the vehicle 1 travels toward the route change point, and the route guidance image R G can also be displayed on the vehicle display device 60. In this case, a display effect simulating the actual vehicle speed of the vehicle 1 can be achieved by controlling the display speed, and as a result, the driver 4 who is the viewer can be visually recognized with a sense of reality. Note that the control of the display speed can be realized by changing the frame rate (the number of images displayed per second) or the refresh rate (the screen update cycle) according to the detected vehicle speed.

[0078] Subsequently, the control unit 51 determines whether the vehicle 1 has moved a predetermined distance (for example, 6 [m]) away from the route change point C C (step ST109). Here, when the control unit 51 detects that the vehicle 1 has moved 6 [m] (predetermined distance) away from the route change point C C , it performs control to end the rotation (rotation end point E rp ) (step ST110). If the predetermined distance is not reached (step ST109 “NO”), the control unit 51 repeatedly executes the processes after step ST107.

[0079] As described above, when the vehicle 1 changes its route, the control unit 51 moves the trajectory of the virtual viewpoint camera V P along a trajectory R G that passes inside the route guidance image R T . Therefore, when the vehicle 1 changes its route, for example, when turning right or left at the intersection C C , by setting the trajectory of the virtual viewpoint camera V P to pass inside the bending point B at the center of the intersection with the maximum curvature, when the vehicle 1 passes through the actual route, the route guidance image R GThe route guidance image R that may be generated near the bending point B due to (see Fig. 3(b)) being out of the viewing angle of the vehicle display device 60 G can be prevented from being out of the field of view of the driver 4 who is the viewer. Fig. 3(a) shows, for comparison purposes, an intersection C C when entering, a display example in which the route guidance image R G is out of the field of view of the driver 4 who is the viewer.

[0080] Also, the control unit 51, along with a change in the driving route, curves the trajectory of the virtual viewpoint camera V P in a curved manner within a triangular region TA (see Fig. 7) that connects the point S P where the virtual viewpoint camera V starts to rotate, the bending point B with the maximum curvature at the driving route change point Cc, and the point E rp where the virtual viewpoint camera V ends its rotation. By doing so, a natural turning manner can be realized, and the discomfort given to the driver 4 who is the viewer with respect to the behavior of the vehicle 1, such as the vehicle speed, can be suppressed. P rp T In the flowchart shown in Fig. 3, the predetermined distance approaching the driving route change point Cc at the start of the rotation of the vehicle 1 and the predetermined distance leaving the driving route change point Cc at the end of the rotation were both described as 6 [m]. However, for example, as shown in Fig. 9(b), when the vehicle 1 approaches the driving route change point Cc (here, the bending point B at the center of the intersection) by a second predetermined distance (for example, 12 [m]) longer than 6 [m], a rotation is started with respect to the track R

[0081] that passes inside the actual route track (rotation start point S T rp ), and when it is separated from the driving route change point by a predetermined distance, a rotation is ended with respect to the track R T rp G rp (rotation end point E). In this case, an appropriate turning manner can be realized in which the driver 4 who is the viewer does not feel discomfort with the display of the route guidance image R. G

[0082] Also, although not shown in the flowchart of FIG. 5, for example, as shown in FIG. 8, when the vehicle 1 turns right to change its course at the intersection C C , when turning right to change the course at the intersection C, the virtual viewpoint camera V P 's orbit, the point where the virtual viewpoint camera V P starts to rotate is a position S G closer to the bending point B than the orbit passing inside the route guidance image R T (the orbit R rp when turning right without control). The point where the virtual viewpoint camera V P ends its rotation is a position E G farther from the bending point B than the orbit R T passing inside the route guidance image R rp . By performing control to set the orbit R T´ (the orbit when turning right with control) in this way, it is possible to realize a course change along an orbit that takes into account the oncoming lane in front when turning right (a turning method that does not cross the oncoming lane). This is because in the vehicle display device 60 (HUD device 60a) that superimposes and displays the route guidance image on the forward field of view FV, it is assumed that when the interval between the rotation start point and the rotation end point is short, the curvature of the orbit approaches a right angle. Therefore, when the vehicle 1 attempts to turn right at an intersection, a display that shortcuts through the intersection and crosses the oncoming lane may occur, and the route guidance image R G becomes a factor causing frame out. This is useful for avoiding this situation.

[0083] Also, for example, as shown in FIG. 10(a), the control unit 51 gradually bends the orbit of the virtual viewpoint camera V T according to the magnitude of the curvature at the course change point (here, assuming a curve when making a U-turn), starting from the second rotation start point S rp1 which is an extension from the rotation start point S rp2 on the orbit R P inside the orbit that the vehicle 1 actually passes through. The orbit is bent greatly at the point S rp1 where the rotation originally starts, and at the point (the rotation end point E G on the orbit R T inside the route guidance image R rp1 where the rotation originally ends, the orbit is bent greatly to the second rotation end point E rp2By performing control to gradually return the trajectory until [a certain point], it is possible to suppress a sudden change in display at the original rotation start point, present a gentle bending manner, and thus eliminate the sense of discomfort given to the driver who is the viewer.

[0084] Also, for example, as shown in FIG. 10(b), the control unit 51 makes the trajectory of the extended viewpoint camera V P such that the greater the curvature at the route change point, the second rotation start point S rp1 extended from the point S where the rotation originally starts rp2 is a trajectory that moves farther from the bending point B (trajectory R T´ ) in the case of having extended control. In this way, when the virtual viewpoint camera V P moves along a trajectory (trajectory R G ) that passes inside the route guidance image R T ) in the case of no extended control, by performing control to gradually bend between the extended second rotation start point S rp2 and the second rotation end point E rp2 extended from the original point where the rotation ends (trajectory R T´ ) in the case of having extended control, it is possible to suppress a sudden change in display at the original rotation start point S rp1 and present a gentle bending manner, thus eliminating the sense of discomfort in the display given to the driver 4 who is the viewer. Note that by performing control to move the second rotation start point S rp2 farther as the curvature of the route change point Cc increases, it is also possible to perform control to gradually bend at the second rotation start point S rp2 .

[0085] (Modification example) As described above, in the display control device 50 according to the present embodiment, when the vehicle 1 makes a route change, the control unit 51 moves the virtual viewpoint camera V P based on the position information of the vehicle 1 and the route guidance image R GAlthough described as performing control to move along a trajectory that passes inside the trajectory of , by incorporating this function into the HUD device 60a, the HUD device 60a can realize the above-described function alone. In this case, the load on the display control device 50 can be reduced.

[0086] In this case, the head-up display device (HUD device 60a) according to the present embodiment includes, for example, as shown in FIG. 2, a control unit 61 and an image display unit 62. When the vehicle 1 changes its course, the control unit 61 moves the virtual viewpoint camera V P along a trajectory that passes inside the trajectory of the route guidance image R G based on the position information of the vehicle 1. At this time, the image display unit 62 superimposes and displays the route guidance image R G generated based on the route information included in the map information on the front view FV of the vehicle 1.

[0087] The image display unit 62 includes a liquid crystal display 621 and an image projection unit 622. The liquid crystal display 621 is, for example, a transmissive display that transmits light from a backlight, and an actuator such as a motor controlled by the control unit 61 is attached to the display surface and is rotatable. The image projection unit 622 is disposed on the optical path of the image light (display light L) from the liquid crystal display 621 between the liquid crystal display 621 and the front windshield 2, and is composed of one or more optical members that project the display light L of the image from the liquid crystal display 621 onto the front windshield 2 outside the image display unit 62.

[0088] The internal configuration of the image projection unit 622 is shown in FIG. 11. The image projection unit 622 is composed of a stereoscopic image display unit 641 and a relay optical unit 642. The stereoscopic image display unit 641 forms a 3D real image 643, and the relay optical unit 642 enlarges the 3D real image 643 to form a second 3D real image 644, and projects the display light L of this second 3D real image 644 toward the front windshield 2 of the vehicle 1.

[0089] The stereoscopic image display unit 641 in FIG. 2 generates a 3D real image 643 formed in three dimensions, and includes a projection unit 645 and a vibrating screen 646. The projection unit 645 is a projector that emits video light (not shown) representing the image included in the display data based on the display data input from the control unit 61, adjusts the timing of displaying the image according to the display distance data included in the display data, and quickly switches the images projected in synchronization with the vibration position of the vibrating screen 646. In other words, the projection unit 645 projects an image suitable for the vibration position of the vibrating screen 646 onto the vibrating screen 646 based on the display distance data.

[0090] The vibrating screen 646 is, for example, a polycarbonate diffusion film that diffuses the video of the projection unit 645 within a certain angular range, forms a real image by receiving the display light L from the projection unit 645, and reciprocates along the optical axis of the display light L emitted from the projection unit 645. The vibrating screen 646 can transmit a signal indicating the vibration position to the projection unit 645 continuously or intermittently, and the projection unit 645 may adjust the timing of displaying the image so that the image is visually recognized at a position corresponding to the display distance data included in the display data based on the signal indicating the vibration position.

[0091] The HUD device 60a according to this embodiment adjusts the length in the depth direction Z of the image while keeping the amplitude of the vibrating screen 646 in the screen vibration direction constant under the control of the control unit 61. That is, the display light L is not emitted during the period when the vibrating screen 646 is located in the first range of its amplitude, and the display light L is emitted during the period when the vibrating screen 646 is located in the second range of its amplitude. By adjusting the positions and ratios of the first range and the second range, the position and length in the depth direction Z of the image are adjusted. Specifically, the vibrating screen 646 vibrates at a frequency of 60 [Hz] or more, and within this period of 1 / 60 [sec], the projection unit 645 projects different images of a plurality of frames, so that different images (real images) of a plurality of frames are formed at each vibration position. That is, the HUD device 60a according to this embodiment can generate a 3D real image 643 by overlapping the real images of a plurality of frames in the vibration direction of the vibrating screen 646. Note that the HUD device 60a may adjust the length in the depth direction Z of the image by changing the amplitude of the vibrating screen 646.

[0092] The relay optical unit 642 in FIG. 2 receives the light of the 3D real image 643 generated by the stereoscopic image display unit 641, forms a 3D real image obtained by enlarging this 3D real image in the middle, and then projects the display light L, which is the light of this 3D real image, toward the front windshield 2. The relay optical unit 642 includes, for example, a first relay optical unit 647 composed of a lens group that receives the light of the 3D real image 643 generated by the stereoscopic image display unit 641, a second relay optical unit 648 that reflects the light that has passed through the first relay optical unit 647 and forms a 3D real image 644 obtained by enlarging the 3D real image 643 in cooperation with the optical power of the first relay optical unit 647, and a third relay optical unit 649 that reflects the display light L, which is the light of the 3D real image 644, toward the front windshield 2.

[0093] The first relay optical unit 647 has a function of magnifying each image formed at each vibration position of the vibration screen 646 in the 3D real image 643 at different magnifications, and is schematically illustrated as a single lens in FIG. 10, but is actually composed of a synthetic lens formed by synthesizing a plurality of thin film lenses (not shown).

[0094] The second relay optical unit 648 is composed of, for example, a mirror having a concave reflecting surface with positive optical power, receives the light of the 3D real image 43 from the first relay optical unit 647, reflects the incident light toward the third relay optical unit 649, and forms a 3D real image 644 obtained by magnifying the 3D real image 643 in cooperation with the optical power of the first relay optical unit 647 between the second relay optical unit 648 and the third relay optical unit 649. Note that the second relay optical unit 648 may be omitted by giving the first relay optical unit 647 the optical action that the second relay optical unit 648 has.

[0095] The third relay optical unit 649 is a concave mirror that reflects the display light L of the 3D real image 644 toward the front windshield 2, and has a function of correcting image distortion due to the curved surface shape of the front windshield 2 and a function of magnifying the 3D real image 644.

[0096] Note that the image projection unit 622 may adopt a known 3D display method such as a parallax division method including a parallax barrier method or a lenticular lens method, a spatial reproduction method including a light field method or a hologram method, for example, a transmittance adjustment screen method in which a plurality of screens having a light control layer with adjustable transmittance are arranged by stacking them in the thickness direction as disclosed in JP-A-2016-212318, and a projector projects an image onto the plurality of screens while quickly switching the projected image, and each of the plurality of screens appropriately adjusts the light transmittance in accordance with the quick switching of the projected image to display a 3D real image inside, for example, a method in which a plurality of liquid crystal display elements are stacked in the thickness direction to display a 3D real image inside as disclosed in JP-A-2004-168230, or a light field display method.

[0097] According to the HUD device 60a according to this embodiment, when the vehicle 1 changes its course, the control unit 61 controls the virtual viewpoint camera V P to move along a trajectory that passes inside the trajectory of the route guidance image R G based on the position information of the vehicle 1. Therefore, when the vehicle 1 changes its course, for example, when turning right or left at an intersection, the virtual viewpoint camera V P is moved along a trajectory that passes inside the bending point at the center of the intersection with the maximum curvature, so that when the vehicle 1 passes through the actual route, the route guidance image R G does not deviate from the viewing angle of the image display unit 62, thereby suppressing the frame out of the route guidance image R G that may occur in the vicinity of the bending point B. Therefore, it is possible to reduce the chance of causing confusion and anxiety to the driver 4, who is the viewer, due to the route guidance image disappearing from the screen.

[0098] (Effect of the embodiment) As described above, the display control device 50 according to this embodiment controls, for example, as shown in FIG. 2, a vehicle display device 60 that superimposes and displays a route guidance image as seen from a virtual viewpoint generated based on route information included in map information on a virtual imaging surface (imaging region VA) virtually set in front of the vehicle 1 on the front visual field FV of the vehicle 1. And the display control device 50, for example, as shown in FIGS. 3(a) and 6, when the vehicle 1 changes its course (for example, turns left at intersection C C ), for example, as shown in FIG. 6, the virtual viewpoint (virtual viewpoint camera V P )(see FIG. 4(b)) is moved along a trajectory R G that passes inside the trajectory of the route guidance image R T based on the position information of the vehicle 1, and controls the display of the route guidance image R G as if it has moved along the trajectory.

[0099] When the vehicle 1 changes its course, the virtual viewpoint camera V P is moved along a trajectory R G that passes inside the trajectory of the route guidance image R Tto move the route guidance image R as if it has moved G By performing control to display, for example, when turning right or left at an intersection, the virtual viewpoint camera V P moves along a trajectory R that passes inside the bending point B at the center of the intersection where the maximum curvature is obtained T By moving in R, when the vehicle 1 passes through the actual route, the route guidance image R G moves out of the viewing angle of the vehicle display device 60, so that the route guidance image R that may occur near the bending point G can be prevented from being framed out of the field of view of the driver 4 who is the viewer. Therefore, it is possible to reduce the chance of causing confusion and anxiety to the driver 4 who is the viewer due to the route guidance image R G disappearing from the screen

[0100] Also, according to the display control device 50 of the present embodiment, the control unit 51, for example, as shown in FIG. 7, along with a change in the driving route, the virtual viewpoint camera V P is set so that the trajectory of the virtual viewpoint camera V P curves between the point S where the rotation starts (the point where it starts to bend) and the point E where the rotation ends (the point where it finishes bending). By displaying the route guidance image R rp in this way, in addition to suppressing the frame-out phenomenon of the route guidance image, it is possible to suppress the sense of discomfort given to the driver 4 who is the viewer with respect to the display of the route guidance image rp G

[0101] Also, according to the display control device 50 of the present embodiment, the control unit 51, for example, as shown in FIG. 7, sets the trajectory of the virtual viewpoint camera V P as a trajectory that curves in a triangular region TA connecting the rotation start point S rp the bending point B, and the rotation end point E rp By displaying the route guidance image R G in this way, a natural bend can be realized, and it is possible to suppress the sense of discomfort given to the driver 4 who is the viewer with respect to the behavior of the vehicle 1, for example, the vehicle speed. Note that the "driving route change point" is the intersection C C ​​In addition to this, when traveling towards the destination, it is assumed to include right and left turning points such as a T-junction or a curve through which the vehicle 1 passes.

[0102] Further, according to the display control device 50 of the present embodiment, for example, as shown in FIG. 8, the control unit 51 causes the virtual viewpoint camera V P V P to follow a trajectory such that when the vehicle 1 makes a right turn to change its course at the intersection C C , the starting point S P where the virtual viewpoint camera V rp starts to rotate is closer to the bending point B than the trajectory passing inside the route guidance image R G (the trajectory R T when turning right without control), and the ending point E P where the virtual viewpoint camera V rp ends its rotation is farther from the bending point B than the trajectory passing inside the route guidance image R G (the trajectory R T when turning right with control). By displaying the route guidance image R T as described above, it is possible to realize a course change along a trajectory that takes into account the oncoming lane in the direction of travel (a turning method that does not cross the oncoming lane). Therefore, when, for example, the HUD device 60a is used as the vehicle display device 60 to display the route guidance image in the front field of view FV of the vehicle 1, the starting point of the trajectory of the virtual viewpoint camera V G is set closer to the bending point than the trajectory passing inside the route guidance image R P , and the ending point of the trajectory of the virtual viewpoint camera V P is set farther from the bending point than the trajectory passing inside the route guidance image R G . When the route guidance image R P is displayed as such, when the interval between the starting point S G and the ending point E G is short, it is assumed that the curvature of the trajectory approaches a right angle. Therefore, when the vehicle 1 attempts to turn right at the intersection C rp , it will display a shortcut through the intersection C rp to cross the oncoming lane, and the route guidance image R C will be displayed. C Inside the intersection C GSince it causes frame out, it is useful to avoid this.

[0103] Also, according to the display control device 50 of the present embodiment, for example, as shown in FIG. 9(a), the control unit 51 causes the virtual viewpoint camera V P to move along a trajectory R that passes inside the route guidance image R G When moving along the trajectory R T and approaching a predetermined distance (for example, 6 [m]) from the route change point Cc, rotation is started (rotation start point S rp ), and when moving away from the route change point Cc by a predetermined distance (for example, 6 [m]), the rotation is ended (rotation end point E rp ). By displaying the route guidance image R G in this way, the driver 4, who is the viewer, can realize an appropriate turning method without feeling discomfort.

[0104] Also, according to the display control device 50 of the present embodiment, for example, as shown in FIG. 9(b), when the control unit 51 moves the trajectory of the virtual viewpoint camera V P along a trajectory R that passes inside the route guidance image R G When approaching a second predetermined distance (for example, 12 [m]) longer than a predetermined distance (for example, 6 [m]) from the route change point, rotation with respect to the trajectory is started (rotation start point S T ), and when moving away from the route change point by a predetermined distance (for example, 6 [m]), the rotation with respect to the trajectory is ended (rotation end point E rp ). By displaying the route guidance image R rp in this way, the driver 4, who is the viewer, can realize an appropriate turn without feeling discomfort. G

[0105] Also, according to the display control device 50 of the present embodiment, for example, as shown in FIG. 10(a), the control unit 51 determines the trajectory of the virtual viewpoint camera V P from the point (rotation start point S on a trajectory inside the trajectory actually passed by the vehicle 1) where rotation is originally started according to the curvature of the route change point (assuming a curve when making a U-turn here) rp1 to an extended second rotation start point S​rp2 The orbit is gradually bent at the point S where the rotation originally started. rp1 The vehicle then makes a large turn at the point where it should end its rotation (the point E on the inner side of the track where vehicle 1 actually passes). rp1 ) to the second rotation end point E rp2 In addition, for example, as shown in FIG. 10(b), the control unit 51 can control the virtual viewpoint camera V P For the orbit of, the greater the curvature of the course change point, the closer the point S rp1 The second rotation starting point S extended from rp2 A trajectory that moves away from the bending point B (trajectory R with extended control) T´ In this case, a sudden change in the display at the original turning start point can be suppressed, and a gentle turning direction can be presented, so that the sense of incongruity felt by the driver 4 who is the viewer can be eliminated.

[0106] Furthermore, according to the display control device 50 of this embodiment, the route guidance image R G Route guidance image R on a track that runs closer to the center G When displaying the above, the display speed is controlled to match the speed of the vehicle 1 approaching the lane change point when the vehicle 1 approaches the lane change point by a predetermined distance, or the display speed is controlled to match the traveling speed of the vehicle 1 according to the gradient or curvature of the road on which the vehicle 1 travels toward the lane change point, thereby making it possible to produce a display that simulates the actual vehicle speed of the vehicle by controlling the display speed, and as a result, the driver 4 who is the viewer can view it with a sense of reality. The display speed can be controlled by changing the frame rate (the number of images displayed per second) or the refresh rate (the screen update cycle) according to the detected vehicle speed.

[0107] In addition, the head-up display device (HUD device 60a) of the present embodiment displays a route guidance image R generated based on route information included in map information on an imaging surface (imaging area VA) that is virtually set in front of the vehicle 1, as shown in FIG. GAn image display unit 61 that superimposes and displays on a road Rd included in the forward field of view FV of the vehicle 1, and a virtual viewpoint camera V that captures a virtual field of view range of the forward field of view FV based on the viewpoint position of the viewer when the vehicle 1 changes its course P (see Fig. 4(b)) is moved along a trajectory that passes inside the trajectory of the route guidance image R G by a control unit 61 that performs control. According to the HUD device 60a of the present embodiment, when the vehicle 1 changes its course, the control unit 61 causes the virtual viewpoint camera V P to move along a trajectory that passes inside the trajectory of the route guidance image R G based on the position information of the vehicle 1. Therefore, when turning right or left at an intersection, for example, the virtual viewpoint camera V P moves along a trajectory that passes inside the bending point B at the center of the intersection with the maximum curvature, so that when the vehicle 1 passes through the actual route, the route guidance image may deviate from the viewing angle of the image display unit 62 and may occur near the bending point B. It is possible to provide a HUD device 60a that suppresses the frame-out phenomenon of the route guidance image R G . Therefore, it is possible to reduce the chance of causing confusion and anxiety to the driver 4, who is the viewer, due to the disappearance of the route guidance image R G from the screen.

[0108] Also, as shown in Fig. 2, for example, the display control program of the present embodiment superimposes and displays a route guidance image R G generated based on the route information included in the map information on an imaging surface (imaging region VA) virtually set in front of the vehicle 1 on a road included in the forward field of view FV of the vehicle 1. And. The display control program causes the processor included in the display control device 50 to, when the vehicle 1 changes its course, for example, as shown in Fig. 6, a virtual viewpoint camera V based on the viewpoint position of the viewer P (see Fig. 4(b)) is moved along a trajectory R that passes inside the trajectory of the route guidance image R G (route guidance image R G ) based on the position information of the vehicle 1 TExecute the process of moving it. In this way, when the vehicle 1 changes its course, the virtual viewpoint camera V is caused by the processor of the display control device 50 reading and executing a program recorded in a memory that is externally attached or built-in. P (See Fig. 4(b)) is moved along a trajectory R (guide path image R G ) that passes inside the trajectory of (guide path image R G ). For this reason, when the vehicle 1 passes through the actual route, the guide path image R T may deviate from the viewing angle of the in-vehicle display device 60, and the frame-out phenomenon of the guide path image R G that may occur near the bending point can be suppressed. Therefore, the driver 4, who is the viewer, can be prevented from being confused and feeling uneasy due to the guide path image R G disappearing from the screen. G

[0109] Also, the in-vehicle display system 100 of the present embodiment, for example, as shown in Fig. 2, superimposes and displays a guide path image R G generated based on the route information included in the map information on the road included in the forward field of view FV of the vehicle 1, and when the vehicle 1 changes its course, the virtual viewpoint camera is moved along a trajectory that passes inside the trajectory of the guide path image R G based on the position information of the vehicle 1. The display control device 50 controls the movement. According to the in-vehicle display system 100 of the present embodiment, when the vehicle 1 changes its course, for example, when turning right or left at an intersection, the virtual viewpoint camera V P moves along a trajectory that passes inside the bending point B at the center of the intersection with the maximum curvature. When the vehicle 1 passes through the actual route, the guide path image R G may deviate from the viewing angle of the in-vehicle display device, and the frame-out phenomenon of the guide path image R G that may occur near the bending point B can be suppressed. Therefore, the driver 4, who is the viewer, can be prevented from being confused and feeling uneasy due to the guide path image R GIt is possible to reduce the chances of causing confusion and anxiety due to disappearance from the screen.

[0110] The present invention is not limited to the above-described exemplary embodiments, and those skilled in the art will be able to easily modify the above-described exemplary embodiments to the extent included in the claims.

Explanation of Reference Numerals

[0111] 1 ··· Vehicle, 2 ··· Front windshield, 4 ··· Viewer (driver), 5 ··· Dashboard, 10 ··· Vehicle ECU, 20 ··· Navigation device, 30 ··· In-vehicle monitoring device, 40 ··· I / O interface, 50 ··· Display control device, 51 ··· Control unit, 52 ··· Storage unit, 60 ··· In-vehicle display device, 60a ··· HUD device, 61 ··· Control unit (HUD device), 62 ··· Image display unit, 621 ··· Liquid crystal display, 622 ··· Image projection unit, 100 ··· In-vehicle display system, 200 ··· Map information DB, VA ··· Imaging area (image plane), FV ··· Foreground (forward field of view), R G ··· Route guidance image, V P ··· Virtual viewpoint camera, VA0~VA3 ··· Field of view angle of HUD device, C C ··· Route change point (intersection), R T ··· Orbit passing inside the actual route of the vehicle, Rd ··· Road, S rp ,S rp1 ,S rp2 ··· Rotation start point, B ··· Bending point, E rp ,E rp1 ,E rp2 ··· Rotation end point, TA ··· Triangular curve area

Claims

1. A display control device for controlling a vehicle display device that displays a route guidance image as if viewed from a virtual viewpoint generated based on route information included in map information on an imaging surface virtually set in front of the vehicle, the display control device including a control unit configured to, when the vehicle changes its course, display the route guidance image such that the virtual viewpoint moves along a trajectory passing inside the route guidance image based on the position information of the vehicle.

2. The control unit, sets a rotation start point at which the virtual viewpoint starts to pass inside the route guidance image and a rotation end point at which the virtual viewpoint finishes passing inside the route guidance image in association with the course change, and displays the route guidance image such that the trajectory of the virtual viewpoint curves between the rotation start point and the rotation end point. The display control device according to claim 1.

3. The control unit, displays the route guidance image such that the trajectory of the virtual viewpoint curves within a triangular region connecting the rotation start point, an inflection point having the maximum curvature in the route guidance image, and the rotation end point in association with the course change. The display control device according to claim 2.

4. The control unit, when the vehicle turns right to change its course at an intersection, displays the route guidance image such that the trajectory of the virtual viewpoint positions the rotation start point closer to the inflection point and the rotation end point farther from the inflection point compared to when the vehicle turns left. The display control device according to claim 3.

5. The control unit, displays the route guidance image such that the trajectory of the virtual viewpoint starts rotating when the vehicle approaches the course change point by a predetermined distance and ends rotating when the vehicle moves away from the course change point by the predetermined distance. The display control device according to any one of claims 1 to 4.

6. The control unit, displays the route guidance image such that the trajectory of the virtual viewpoint starts rotating when the vehicle approaches the course change point by a second predetermined distance longer than the predetermined distance and ends rotating when the vehicle moves away from the course change point by the predetermined distance. The display control device according to any one of claims 1 to 4.

7. The control unit, The display control device according to claim 2, wherein the path guidance image is displayed such that the trajectory of the virtual viewpoint is gradually bent at a second rotation start point extended from the start point according to the magnitude of the curvature of the route change point, the trajectory is greatly bent at the start point, and gradually returned between the start point and a second rotation end point extended from the end point.

8. The control unit The display control device according to claim 7, wherein the path guidance image is displayed such that the greater the curvature of the route change point, the farther the second rotation start point is.

9. The control unit The display control device according to claim 5, wherein when the vehicle approaches the route change point by a predetermined distance, the display speed is controlled in accordance with the speed of the vehicle approaching the route change point.

10. The control unit The display control device according to claim 5, wherein the path guidance image is displayed such that the trajectory of the virtual viewpoint matches the traveling speed of the vehicle according to the gradient or curvature of the road on which the vehicle travels toward the route change point.

11. An image display unit that displays a path guidance image as if viewed from a virtual viewpoint generated based on path information included in map information on an imaging surface virtually set in front of the vehicle; A head-up display device comprising: a control unit that, when the vehicle changes its route, displays the path guidance image such that the virtual viewpoint moves along a trajectory passing inside the path guidance image based on the position information of the vehicle.

12. A display control program for controlling a vehicle display device that displays a path guidance image as if viewed from a virtual viewpoint generated based on path information included in map information on an imaging surface virtually set in front of the vehicle, wherein the processor included in the display control device executes a process of displaying the path guidance image such that the virtual viewpoint moves along a trajectory passing inside the path guidance image based on the position information of the vehicle when the vehicle changes its route.

13. A head-up display device that displays a path guidance image as if viewed from a virtual viewpoint generated based on path information included in map information on an imaging surface virtually set in front of the vehicle, A vehicle display system comprising: a display control device that, when the vehicle changes its course, displays the route guidance image so that the virtual viewpoint appears to move along a trajectory passing inside the route guidance image based on the position information of the vehicle.

Images