Display control device, head-up display device, display control program, and display system for vehicle

Abstract

To display information presented by a content related to an object to be superposed in an easily recognizable manner, even when the object to be superposed and the content are too far away from each other in a depth direction.SOLUTION: A display control device 60 for controlling a display device 10 for a vehicle for generating an image associated with a specific object in a field of front vision of a vehicle 1, and imaging an image in an imaged region virtually set in front of the vehicle 1 includes a control part 61 for acquiring relative positional relation information to a vehicle for the specific object, imaging a first mode image superposed on the specific object in the imaged region at a position to the front of the vehicle by a first distance, and imaging a second mode image non-superposed on the specific object out of the imaged region at a position to the front of the vehicle by the first distance or a second distance shorter than the first distance.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a display control device that generates an image associated with a specific object in the forward field of view of a vehicle and controls a vehicle display device that forms the image in an imaging region virtually set in front of the vehicle, and the like.

Background Art

[0002] Conventionally, a technique for displaying information related to a specific object in the forward field of view of a vehicle in a superimposed manner on the object has been known. According to this technique, a driver as a viewer can visually recognize the information without significantly shifting the line of sight from the forward field of view. Therefore, the driver can reduce the load of line-of-sight movement and drive the vehicle.

[0003] For example, in Patent Document 1, when it is not possible to superimpose and display information related to a specific object in the forward field of view of a vehicle based on the relationship between the driver's viewpoint and the imaging region, instead, a different type of content (for example, paragraph

[0059] and 80b in FIG. 11) indicating the direction to the object to be superimposed is displayed to ensure understandability for the driver.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, according to the technology described in Patent Document 1, regarding the relationship between the viewing point of the driver as the viewer and the imaging area, the vertical and horizontal directions are considered, but the depth direction is not considered (the display distance, which is the distance to the content viewed by the viewer, is fixed). Therefore, when the object in the front view to be superimposed and the content related to the superimposed object are too far apart in the depth direction, the sense of immersion of the content into the front view is poor, the driver as the viewer feels uncomfortable, and there is a problem that the information presented by the content cannot be accurately grasped.

[0006] The present invention has been made to solve the above-described problems, and an object thereof is to provide a display control device or the like that can easily recognize and display the information presented by the content even when the superimposed object and the content related to the superimposed object are too far apart in the depth direction.

[0007] Other objects of the present invention will become apparent to those skilled in the art by referring to the embodiments and best modes exemplified below and the accompanying drawings.

Means for Solving the Problems

[0008] Hereinafter, embodiments according to the present invention will be exemplified in order to easily understand the outline of the present invention.

[0009] A first aspect is a display control device that controls a vehicle display device that generates an image associated with a specific object in the front view of a vehicle and forms the image in an imaging area virtually set in front of the vehicle, the display control device obtaining relative position relationship information between the vehicle and the specific object, forming a first aspect image superimposed on the specific object within the imaging area at a position in front of the vehicle by a first distance, and forming a second aspect image not superimposed on the specific object outside the imaging area at a position in front of the vehicle by the first distance or a second distance shorter than the first distance, and having a control unit.

[0010] In the first aspect, the control unit acquires relative position relationship information between the vehicle and a specific object in the forward field of view of the vehicle, forms an image (first-aspect image) expressed in the first aspect and superimposed on the specific object within the imaging region at a position forward of the vehicle by a first distance, and performs control to form an image (second-aspect image) expressed in the second aspect and not superimposed on the specific object outside the imaging region at a position forward of the vehicle by the first distance or a second distance shorter than the first distance. In this way, by changing the display mode of the image related to the specific object according to whether the specific object in the forward field of view of the vehicle is within the imaging region, the recognizability of the information presented by the image associated with the specific object can be enhanced (it becomes easier to recognize). Also, imaging control considering the depth direction can be performed by controlling the display distance indicated by the distance between the driver as the viewer and the generated image. Therefore, even when the object in the forward field of view to be superimposed and the image related to the superimposed object are too far apart in the depth direction, the immersion of the image into the forward field of view can be enhanced, and the driver as the viewer can accurately grasp the information presented by the image without a sense of discomfort.

[0011] Here, the "specific object in the forward field of view of the vehicle" refers to the scenery (foreground) visible in front of the vehicle through the front window as seen by the driver as the viewer, specifically, objects such as a preceding vehicle, a pedestrian, and a building. Here, for example, it refers to the pedestrian Ob shown in (a) of FIG. 3A, (b) of FIG. 3A, (a) of FIG. 3C, etc. Also, the "image associated with the specific object" is content whose display position changes in accordance with the movement of the specific object. For example, it refers to the first-aspect image VI1 shown in (a) of FIG. 3A, the second-aspect image VI2 shown in (a) of FIG. 3B, and the second-aspect image VI 2´ shown in (a) of FIG. 3C. Also, the "relative position relationship information" refers to the relative position of the specific object based on the current position of the vehicle. For example, it is the position information of a pedestrian measured by a LiDAR 303 of an in-vehicle monitoring device 30 shown in FIG. 1, GPS information of a building or other ground features, etc., and is calculated based on the position information of the pedestrian or ground feature and the position information of the vehicle.

[0012] Also, the "first-mode image" is, for example, VI1 shown in (a) of FIG. 3A, which is expressed in a form superimposed on a specific object within the imaging region, and is superimposed and displayed on a specific object within the imaging region, for example, a pedestrian. Also, the "second-mode image" is an object Ob related to a specific object outside the imaging region, for example, VA2 shown in (b) of FIG. 3A, VI shown in (a) of FIG. 3C 2´ and is displayed non-superimposed on a pedestrian that is an object Ob related to a specific object located outside the imaging region. Also, the "first distance" refers to the display distance, which is the distance from the viewpoint position of the driver, who is the viewer, to the first-mode image formed in the imaging region. For example, it is the "display distance d1 (= distance to OB)" shown in (b) of FIG. 3A. Also, the "second distance" refers to the display distance, which is shorter than the first distance and is the distance from the viewpoint position of the driver, who is the viewer, to the second-mode image formed in the imaging region. For example, it is the "display distance d3 < distance to Ob" shown in (b) of FIG. 3C.

[0013] In a second aspect subordinate to the first aspect, the control unit may perform control to form the second-mode image in the imaging region by setting the second distance to be shorter as the degree of deviation from a predetermined visual field range related to the acquired relative position relationship information with the vehicle is greater.

[0014] In the second aspect, the control unit performs control to form the second-mode image in the imaging region by setting the second distance to be shorter as the degree of deviation from a predetermined visual field range related to the acquired relative position relationship information with the vehicle is greater. In this way, by controlling the display distance, imaging control considering the depth direction can be performed, the sense of immersion in a specific object in the front view of the generated image can be enhanced, and the discomfort of the imaged image can be eliminated.

[0015] In addition, the display position when away from the visual field range of 5° in the vertical and horizontal directions is, for example, as shown in (a) of FIG. 3C, a straight line extending from the vehicle speed display (40 km / h) to the center of a specific target in the forward visual field to be targeted and serving as the end of the imaging region VA. For the depth position, for example, in the side view shown in (b) of FIG. 3C, it is set as the position where the straight line drawn from the viewpoint position EP of the driver, who is the viewer, to a specific target in the forward visual field to be targeted is closest to the display space of the vehicle display device (HUD device).

[0016] In a third aspect dependent on the first or second aspect, when it is determined that the imaging position of the second aspect image is within a predetermined visual field range with respect to the specific target, the control unit performs control to image the image at a position advanced by the first distance in front of the vehicle. When it is determined that the imaging position of the second aspect image is not within the predetermined visual field range with respect to the specific target, the control unit may perform control to image the image at a position advanced by the second distance in front of the vehicle.

[0017] In the third aspect, when it is determined that the imaging position of the second aspect image is within a predetermined visual field range with respect to the specific target, the control unit performs control to image the image at a position advanced by the first distance in front of the vehicle. When it is determined that the imaging position is not within the predetermined visual field range, the control unit performs control to image the image at a position advanced by the second distance in front of the vehicle. For this reason, when line-of-sight movement is necessary (when the specific target is not within the predetermined visual recognition region and the distance between the second aspect image and the specific target is not in a position relationship that can be viewed singly), it is assumed that the driver, who is the viewer, will have a behavioral pattern of looking at the specific target in the forward visual field after looking at the image formed in the imaging region. Therefore, the focus movement between the second aspect image and the specific target becomes smooth and the load associated with line-of-sight movement is reduced. Also, since a second aspect image that can be visually recognized as the same as the specific target is provided within the range where line-of-sight movement is unnecessary and visual recognition is possible, it becomes easier to grasp the information.

[0018] In a fourth aspect that depends on the first or second aspect, when it is determined that the imaging position of the second aspect image is not within a predetermined visual field range with respect to the specific object, the control unit forms an image on the front side of the imaging region having an oblique image plane that is installed tilted forward with respect to the longitudinal direction of the vehicle, and when it is determined that the imaging position of the second aspect image is within the predetermined visual field range with respect to the specific object, the control unit may perform control to form an image on the back side of the imaging region.

[0019] In the fourth aspect, when it is determined that the imaging position of the second aspect image is not within a predetermined visual field range with respect to a specific object, the control unit forms an image on the front side of the imaging region having an oblique image plane that is installed tilted forward with respect to the longitudinal direction of the vehicle, and when it is determined that it is within the predetermined visual field range, the control unit performs control to form an image on the back side of the imaging region. For this reason, also in a vehicle display device (head-up display device) having an oblique image plane, similar to a head-up display device having an erect image plane, when line-of-sight movement is necessary, the driver who is the viewer views the image formed in the imaging region and then views the object in the forward field of view, so that the focus shift between the image and the object becomes smooth and the load associated with line-of-sight movement is reduced. When line-of-sight movement is not necessary and within the visible range, an image that can be viewed as the same as the object is provided, making it easier to grasp information.

[0020] A fifth aspect is a head-up display device that generates an image associated with a specific object in the forward field of view of a vehicle and forms the image in an imaging region that is virtually set in front of the vehicle, the head-up display device including an image display unit that projects the generated image toward a projection member, and a control unit that acquires relative position relationship information between the vehicle and the specific object and forms a first aspect image that is superimposed on the specific object within the imaging region at a position that is a first distance forward of the vehicle, and forms a second aspect image that is not superimposed on the specific object outside the imaging region at a position that is the first distance or a second distance shorter than the first distance forward of the vehicle.

[0021] In a fifth aspect, a control unit acquires at least relative position relationship information with respect to a vehicle regarding a specific object, forms a first aspect image that overlaps the specific object within an imaging region at a position ahead of the vehicle by a first distance, and forms a second aspect image that does not overlap the specific object outside the imaging region at a position ahead of the vehicle by the first distance or a second distance shorter than the first distance, and performs control to display the images on an image display unit. In this way, by changing the display mode of an image related to a specific object according to whether the specific object in the forward field of view of the vehicle is within the imaging region, it is possible to provide a head-up display device that enhances the recognizability (makes it easier to recognize) of the information presented by the image associated with the specific object. Further, by controlling the display distance indicated by the distance between the driver as the viewer and the generated image, imaging control considering the depth direction can be performed, and even when the object in the forward field of view to be superimposed and the image related to the superimposed object are too far apart in the depth direction, the sense of immersion of the image into the forward field of view can be enhanced. Therefore, it is possible to provide a head-up display device that allows the driver as the viewer to accurately grasp the information presented by the image without discomfort.

[0022] A sixth aspect is a display control program for a display control device that controls a vehicle display device that generates an image associated with a specific object in the forward field of view of the vehicle and projects the image onto an imaging region virtually set in front of the vehicle, the program causing a processor included in the display control device to execute a process of acquiring relative position relationship information with respect to the vehicle regarding the specific object, a process of forming the image expressed in a first aspect that overlaps the specific object within the imaging region at a position ahead of the vehicle by a first distance, and a process of forming the image expressed in a second aspect that does not overlap the specific object outside the imaging region at a position ahead of the vehicle by the first distance or a second distance shorter than the first distance.

[0023] In a sixth aspect, a processor included in a display control device reads and executes a program recorded in a memory to obtain relative position relationship information between the vehicle and a specific object in the forward field of view of the vehicle, and forms an image expressed in a first mode that is superimposed on the specific object within an imaging region at a position forward of the vehicle by a first distance, and performs control to form an image expressed in a second mode that is not superimposed on the specific object outside the imaging region at a position forward of the vehicle by the first distance or a second distance shorter than the first distance. In this way, by changing the display mode of the image related to the specific object depending on whether the specific object in the forward field of view of the vehicle is within the imaging region, the recognizability of the information presented by the image associated with the specific object can be enhanced (it becomes easier to recognize). Further, by controlling the display distance indicated by the distance between the driver, who is the viewer, and the generated image, imaging control considering the depth direction can be performed. Therefore, even when the object in the forward field of view that is the superimposed object and the image related to the superimposed object are too far apart in the depth direction, the immersion of the image into the forward field of view can be enhanced, and the driver, who is the viewer, can accurately grasp the information presented by the image without discomfort.

[0024] A seventh aspect is a vehicle display system including a head-up display device that generates a virtual image associated with a specific object in the forward field of view of the vehicle and forms the virtual image in an imaging region that is virtually set in front of the vehicle, and a display control device that performs display control of the head-up display device. The display control device obtains relative position relationship information between the vehicle and the specific object, forms the image expressed in a first mode that is superimposed on the specific object within the region of the imaging region at a position forward of the vehicle by a first distance, and performs control to form the image expressed in a second mode that is not superimposed on the specific object outside the imaging region at a position forward of the vehicle by the first distance or a second distance shorter than the first distance. The head-up display device projects the first-mode image or the second-mode image toward a projection member.

[0025] In a seventh aspect, a display control device forms an image expressed in a first mode that is superimposed on a specific target within an imaging region at a position ahead of the vehicle by a first distance based on relative position relationship information between the vehicle and the specific target, and forms an image expressed in a second mode that is not superimposed on the specific target outside the imaging region at a position ahead of the vehicle by the first distance or a second distance shorter than the first distance. A head-up display device projects the first-mode image or the second-mode image toward a projection member. In this way, by changing the display mode of an image related to a specific target according to whether the specific target in the forward field of view of the vehicle is within the imaging region, it is possible to provide a vehicle display system that enhances the recognizability of the information presented by the image associated with the specific target (it becomes easier to recognize). Further, by controlling the display distance indicated by the distance between the driver as the viewer and the generated image, imaging control considering the depth direction can be performed. Therefore, even when the target in the forward field of view to be superimposed and the image related to the superimposed target are too far apart in the depth direction, it is possible to enhance the sense of immersion of the image into the forward field of view, and it is possible to provide a vehicle display system that allows the driver as the viewer to accurately grasp the information presented by the image without discomfort.

Brief Description of Drawings

[0026]

Figure 1

Figure 2

Figure 3A

Figure 3B

Figure 3C

Figure 4

Figure 5A

Figure 5B

Figure 5C

Figure 6

Figure 7

Mode for Carrying Out the Invention

[0027] The best mode described below is used to easily understand the present invention. Therefore, those skilled in the art should note that the present invention is not unduly limited by the embodiments described below (hereinafter referred to as the present embodiments).

[0028] (Configuration of the Embodiment) FIG. 1 shows the configuration of the vehicle display system 300 of the present embodiment and images (first-mode image VI1, second-mode image VI2, VI described later) displayed by, for example, a head-up display device (hereinafter simply referred to as HUD device 10a) used as the vehicle display device 10. 2´) is a diagram cited to explain an example. In FIG. 1, the longitudinal direction of the vehicle 1 is defined as the Z direction (the forward direction is the positive Z direction), the direction along the left - right direction (the width direction of the vehicle 1) of the vehicle 1 is defined as the X direction (the left direction is the positive X direction), and the vertical direction is defined as the Y direction (the upward direction is the positive Y direction).

[0029] In FIG. 1, the vehicle display system 300 of the present embodiment includes a vehicle ECU 20, an in - vehicle monitoring device 30, a navigation device 40, an operation input device 50, and a display control device 60, which are connected via an I / O interface 70 so that they can communicate bidirectionally with each other.

[0030] The vehicle ECU 20 is, for example, a driving support ECU (Electronic Control Unit) that supports the driver's driving, and has at least one of a driving support function for assisting the driver's driving operation and an automatic driving function capable of substituting the driver's driving operation. Based on the detection information acquired from the in - vehicle monitoring device 30, it recognizes the driving environment in front of the vehicle 1, controls the vehicle 1 according to the analysis result of the recognized detection information, and can also provide various display information indicating the state of the vehicle 1 to the display control device 60 of the present embodiment via the I / O interface 70. In addition, as the vehicle ECU 20, an electronic control unit that controls a drive system such as an engine or a motor, a braking system such as a brake, and a steering system such as a steering is also connected.

[0031] The in - vehicle monitoring device 30 is sensors necessary for recognizing the surrounding driving environment including the front of the vehicle 1, and includes a camera 301, a GPS (Global Positioning System) 302, a LiDAR (Light Detection And Ranging) 303, a behavior sensor 304, etc. The information recognized or detected by the in - vehicle monitoring device 30 is transferred to the vehicle ECU 20, the navigation device 40, and the display control device 60 via the I / O interface 70.

[0032] Note that the camera 301 captures at least the forward view (real scene) of the vehicle 1 and the eyes of the driver who is the viewer, the GPS 302 measures the current location of the vehicle 1, and the LiDAR 303 uses near-infrared light, visible light, and ultraviolet light to irradiate light, for example, on obstacles existing in front of the vehicle 1 captured by the camera 301, captures the reflected light with a light sensor, and determines the distance to the obstacle based on the time difference. Further, the behavior sensor 304 includes an IMU (Inertial Measurement Unit) that detects the behavior of the vehicle 1, a vehicle speed sensor, etc. The IMU uses a three-axis acceleration sensor and a three-axis angular velocity sensor (gyro sensor) to measure (detect translational motion in three-axis directions from acceleration [m / s 2 , and rotational motion from angular velocity [deg / s]).

[0033] The navigation device 40 uses a global navigation satellite system (GNSS) such as the GPS 302 and a gyro sensor to obtain map information from the map information DB 400 or through wireless communication with the outside of the vehicle, and performs guidance on surrounding facilities and route guidance of the vehicle 1. The navigation device 40 can transfer a signal prompting a display output based on this guidance information to the display control device 60 via the I / O interface 70 at an appropriate timing. Note that the map information (map information DB 400) can obtain and store the latest map information, for example, through communication with an external center (not shown) via a V2X (Vehicle to X) type communication system. Here, the map information is mapping data digitized to represent the driving environment of the vehicle 1. As the mapping data, it is preferably digital data of a particularly high-precision dynamic map. Note that a dynamic map is a digital map that combines a vast amount of dynamic information that changes every moment, such as traffic regulations, construction information, accidents and traffic jams, pedestrian and signal information, and static information such as high-precision three-dimensional position information (three-dimensional structures such as road surface information, lane information, and road signs).

[0034] The operation input device 50 is, for example, a group of switches provided near the steering wheel. By operating these switches, the driver, who is the viewer, can perform operations such as ON / OFF switching of the HUD 10a, selection of the execution functions of the driving support device (vehicle ECU 20), air volume setting of the air conditioner, etc. Further, setting information (equivalent to the position of the glove box 4) for instructing the rotation amount (rotation position information) of the concave mirror (see the second relay optical unit 148 in FIG. 7) of the HUD device 10a, which will be described later, can be input.

[0035] The I / O interface 70 performs communication (also referred to as CAN communication) with the vehicle ECU 20 and the navigation device 40, in addition to the display control device 60 of the present embodiment, according to, for example, the CAN (Controller Area Network) standard. Note that the communication standard adopted by the I / O interface 70 is not limited to CAN. For example, it includes wired communication interfaces such as CANFD (CAN with Flexible Data Rate), LIN (Local Interconnect Network), Ethernet (registered trademark), MOST (Media Oriented Systems Transport: MOST is a registered trademark), UART (Universal Asynchronous Receiver Transmitter), or USB (Universal Serial Bus), or in-vehicle communication (internal communication) interfaces such as personal area networks (PAN) like Bluetooth (registered trademark) networks or short-range wireless communication interfaces within several tens of meters such as 802.11x Wi-Fi (registered trademark) local area networks (LAN). Further, the I / O interface 70 may include an out-of-vehicle communication (external communication) interface such as a wide-area communication network (e.g., an Internet communication network) according to cellular communication standards such as wireless wide area network (WWAN0, IEEE802.16-2004 (WiMAX: Worldwide Interoperability for Microwave Access)), IEEE802.16e-based (Mobile WiMAX), 4G, 4G-LTE, LTE Advanced, 5G, etc.

[0036] The display control device 60 of the present embodiment is associated with an image related to a specific object in the forward field of view of the vehicle 1 (the first aspect image VI1, the second aspect image VI2, VI to be described later). 2´) is generated, and a vehicle display device 10 that forms an image in an imaging area virtually set in front of the vehicle 1 is controlled. Here, the HUD device 10a is used as the vehicle display device 10 for the sake of explanation. However, the description is not limited to the HUD device 10a, and it may be replaced by a center information display (CID) provided at the center of the console of the vehicle 1 or a head-mounted display worn on the driver's head.

[0037] The HUD device 10a used as the vehicle display device 10 has a projection unit (see 121 shown in FIGS. 6 and 7) that projects display light L. The HUD device 10a projects the display light L toward a projection member 2 that transmits part of the light and reflects part of the light. This projection member 2 is constituted by a part of the front windshield of the vehicle 1. An eyebox 4 can be formed by the display light L reflected by the projection member 2 in an area where the eyes 3 of the driver, who is the viewer, are assumed to be placed. The driver, who is the viewer, can view the entire image 70 displayed by the HUD device 10a by placing the eyes 3 inside this eyebox 4, and when the eyes 3 move out of the eyebox 4, a part of the image 70 becomes unviewable (difficult to view). The HUD device 10a can adjust the positions where the images 71, 72, and 73 are displayed in the X direction, Y direction, and Z direction. That is, the HUD device 10a can particularly adjust the display distance 100, which is the distance in the depth direction Z until the images 71, 72, and 73 are formed from the eyebox 4, and can arrange (display) the images 71, 72, and 73 in the imaging area VA, which is a three-dimensional space.

[0038] Next, the image 70 displayed by the HUD device 10a will be described. The image 70 has, for example, 2D images 71 and 72 represented two-dimensionally, and a 3D image 73 represented three-dimensionally. The 2D image 71 is imaged two-dimensionally on the XY plane at the display distance 101 and is an image that does not give the driver, who is the viewer, a sense of perspective. According to such a 2D image 71, the entire display can be clearly recognized generally without adjusting the focus of the eyes 3. On the other hand, the 2D image 72 has one end on the XY plane at the display distance 103 and the other end on the XY plane at the display distance 104, which is farther from the driver than the display distance 103. It is imaged two-dimensionally and is an image that gives the driver, who is the viewer, a sense of perspective. Such a 2D image 72 can give a three-dimensional impression because the entire display is clearly recognized by adjusting the focus of the eyes 3 of the driver, who is the viewer. An example of the 3D image 73 has one end on the XY plane at the display distance 102 and the other end on the XY plane at the display distance 103, which is farther from the driver than the display distance 102. It is imaged as a three-dimensional object having a volume and is an image that gives the driver, who is the viewer, a sense of three-dimensionality.

[0039] Note that increasing the display distance 100 means, in other words, imaging the image 70 at a position farther from the eyebox 4. For example, it means bringing closer to the display distance 104, which is on the far side, the first 2D image 71 displayed at the display distance 101 and the 3D image 73 displayed between the display distances 102 and 103. Also, decreasing the display distance 100 means, in other words, imaging the image 70 at a position closer to the eyebox 4. For example, it means bringing closer to the display distance 101, which is on the near side, the first 2D image 71 displayed at the display distance 104 and the 3D image 73 displayed between the display distances 102 and 103.

[0040] The display control device 60 of this embodiment is composed of a control unit 61 and a storage unit 62. The control unit 61 acquires relative position relationship information between the vehicle 1 and a specific object in the forward field of view of the vehicle 1, and forms a first mode image superimposed on the specific object within the imaging region VA at a position in front of the vehicle 1 by a first distance, and forms a second mode image not superimposed on the specific object outside the imaging region VA at a position in front of the vehicle by a first distance or a second distance shorter than the first distance.

[0041] Here, the "specific object in the forward field of view of the vehicle" refers to a landscape (foreground) visible in front of the vehicle 1 through the front window as seen from the eyes 3 of the driver who is the viewer, specifically, an object such as a preceding vehicle, a pedestrian, or a building. Here, it refers to a pedestrian shown in, for example, (a) of FIG. 3A, (b) of FIG. 3A, (a) of FIG. 3C, etc., which will be described later. Also, the "image associated with a specific object" is content whose display position changes according to the movement of the specific object. For example, the first mode image VI1 shown in (a) of FIG. 3A, the second mode image VI2 shown in (a) of FIG. 3B, and the second mode image VI shown in (a) of FIG. 3C. 2´ Also, the "relative position relationship information" refers to the relative position with an object related to a specific object in the forward field of view of the vehicle 1 based on the current position of the vehicle 1. For example, it is the position information of a pedestrian measured by a LiDAR 303 of the in-vehicle monitoring device 30 shown in FIG. 1, the GPS information of a ground object such as a building, etc. Actually, the relative position is calculated based on the position information of the pedestrian ground object and the position information of the vehicle 1.

[0042] Also, the "first mode image" is expressed in a form superimposed on a specific object located within the imaging region VA. For example, it is VI1 shown in (a) of FIG. 3A and is displayed superimposed on a pedestrian who is a specific object within the imaging region VA. Also, the "second mode image" is expressed non-superimposed on a pedestrian who is a specific object located outside the imaging region VA. For example, VI2 shown in (b) of FIG. 3A and VI shown in (a) of FIG. 3C. 2´That is, the "first distance" refers to the display distance, which is the distance from the viewpoint position EP of the driver, who is the viewer, to the image (the first-mode image VI1, the second-mode image VI2) formed in the imaging region VA. For example, it is the "display distance d1 (= the distance to Ob)" shown in (b) of FIG. 3A and the "display distance d2 (= the distance to Ob)" shown in (b) of FIG. 3B. The "second distance" refers to the display distance, which is shorter than the first distance and is the distance from the viewpoint position EP of the driver, who is the viewer, to the image (the second-mode image VI 2´ ) formed in the imaging region VA. For example, it refers to the "display distance d3 < the distance to Ob" shown in (b) of FIG. 3C.

[0043] In addition, the control unit 61 can perform control to form an image in the imaging region VA by setting the second distance shorter as the degree of deviation from a predetermined visual field range regarding the acquired relative position relationship information with the vehicle 1 is larger for the second-mode image VI 2´ . Here, the "predetermined visual field range" refers to, for example, a visual field range of a viewing angle of 5 degrees defined as the range of the peripheral visual field. The visual field of the human eye is divided into the central visual field and the peripheral visual field. The central visual field is about a viewing angle of 2 degrees (a range of a radius of 1 degree from the fovea) from the viewpoint of resolution visual acuity, a viewing angle range of 5 degrees from the viewpoint of the retinal structure, and a range within a viewing angle of 7 degrees in the neuroanatomical classification regarding the visual cortex, respectively. The central visual field is a part with many cone cells and high resolution ability, with the best color discrimination ability and visual acuity, and can clearly see things. On the other hand, in the peripheral visual field, there are few cone cells, colors are not well visible, and the visual acuity is lower than that of the central visual field, but the density of rod cells is high and it is extremely sensitive to motion stimuli. The peripheral visual field plays an important role in grasping the spatial position relationship or motion perception. That is, since the human eye has a function of capturing moving objects in the peripheral visual field and grasping the details in the central visual field, here, a visual field range of a viewing angle of 5 degrees is used as the "predetermined threshold".

[0044] In addition, when the display position is outside the visual field range of 5 degrees in the vertical and horizontal directions, for example, as shown in (a) of FIG. 3C, it is a straight line extending from the vehicle speed display (40 km / h) to the center of a specific target in the forward visual field and is the end of the imaging region VA. For the depth position, for example, in the side view of (b) of FIG. 3C, it is the position where the display space of the HUD device 10a is closest to the straight line drawn from the viewpoint position EP of the driver, who is the viewer, to a specific target in the forward visual field of the target vehicle 1.

[0045] In addition, when it is determined that the imaging position of the second mode image VI2 is within a predetermined visual field range with respect to a specific target in the forward visual field of the vehicle 1, the control unit 61 performs control to image the second mode image VI2 at a position advanced by a first distance in front of the vehicle 1. On the other hand, when the imaging position of the second mode image VI 2´ is not within the predetermined visual field range with respect to the specific target and it is determined that they are in a positional relationship where they cannot be viewed in a single view, the second mode image VI 2´ can be controlled to be imaged at a position advanced by a second distance in front of the vehicle 1.

[0046] In addition, when it is determined that the imaging position of the second mode image VI2´ is not within the predetermined visual field range with respect to a specific target in the forward visual field of the vehicle 1 and they are in a positional relationship where they cannot be viewed in a single view, it is imaged on the front side of the imaging region VA having an oblique image plane (for example, VAE in FIG. 5(a)) that is tilted forward with respect to the front-rear direction of the vehicle 1. When it is determined that the imaging position of the second mode image VI2 is within the predetermined visual field range with respect to the specific target, control can be performed to image it on the back side of the imaging region VA.

[0047] To perform the above-described control, the control unit 61 includes a viewpoint position detection unit 611, a photographed image acquisition unit 612, an image generation unit 613, a display position calculation unit 614, a display distance determination unit 615, and a display control unit 616.

[0048] The viewpoint position detection unit 611 has a function of detecting the viewpoint position of the driver who is the viewer. The viewpoint position detection unit 611 can detect the driver's viewpoint position EP (refer to EP in (b) of FIG. 3B described later) with high precision, for example, by performing image recognition on the eyes 4 (pupil images) of the driver captured by the camera 301 of the in-vehicle monitoring device 30. Note that the driver's viewpoint position EP can also be detected by taking in the rotation position information (position of the eye box 4) of a concave mirror (refer to the second relay optical unit 148 in FIG. 7) described later, which is set by the driver who is the viewer operating the operation input device 50.

[0049] The captured image acquisition unit 612 has a function of recognizing an object Ob related to a specific target such as a pedestrian from the captured image of the front view of the vehicle 1 captured by the camera 301 of the in-vehicle monitoring device 30, and delivering the relative position related information with the vehicle 1 to the image generation unit 613, the display position calculation unit 614, the display distance determination unit 615, and the display control unit 616.

[0050] The image generation unit 613 has a function of generating a first aspect image VI1 and a second aspect image VI2, VI 2´ related to the specific target acquired by the image acquisition unit 612. The first aspect image is content assumed to be displayed superimposed on the object Ob related to the specific target when the object Ob (here, a pedestrian) related to the specific target is captured within the viewing angle of the HUD device 10a (within the imaging region VA). For example, as shown in FIG. 3A(a), it is an image indicated by VI1 in which the display position changes according to the movement of the pedestrian who is the object Ob related to the specific target. The second aspect image is content prepared for non-superimposed display to draw attention to the object Ob related to the specific target when the object Ob related to the specific target exists outside the imaging region VA. The display position changes according to the movement of the pedestrian who is the object Ob related to the specific target, and it is an image VI2 (refer to FIG. 3B(a)) shown by the head portion of an arrow having a width indicating the direction in which the pedestrian is located, and VI 2´ (refer to FIG. 3C(a)).

[0051] The first-mode image VI1, the second-mode images VI2, VI generated by the image generation unit 613 2´ can drive the HUD device 10a (see the projection unit 121 shown in FIGS. 6 and 7). The first-mode image VI1, the second-mode images VI2, VI 2´ generated here shall include display distance data for driving the projection unit 121 to display the image 70 at the display distance 100 (see FIG. 1) determined by the display distance determination unit 615.

[0052] The display position calculation unit 614 has a function of calculating the display position for allocating and displaying the first-mode image VI1 or the second-mode images VI2, VI 2´ at an arbitrary position within the imaging region VA. The display position calculation unit 614 calculates the display position of the image based on a preset correction value in order to display the image in accordance with the position of the captured image (front view of the vehicle 1) captured and acquired by the camera 301 of the in-vehicle monitoring device 30. Here, the correction value is a value set based on the deviation amount between the position of the instrument box 4 where the viewpoint EP of the viewer, i.e., the driver, is assumed to be located and the mounting position of the camera 301 on the vehicle 1. That is, the display position calculation unit 614 calculates the display position when displaying the image on the image plane (imaging region VA) based on the camera coordinate system with the mounting position of the camera 301 as a reference and the viewpoint coordinate system with the position of the instrument box 4 as a reference, that is, the display position based on the viewpoint position EP of the driver. Based on the above-described viewpoint position of the camera 301 and the viewpoint position EP of the viewer, i.e., the driver, the display position when projecting the first-mode image VI1 or the second-mode images VI2, VI 2´ onto the virtual upright image plane VAE or the diagonal image plane VAD set in front of the vehicle 1 is specified. The technique for specifying the display position when displaying the content based on the viewpoint position EP of the viewer, i.e., the driver, is described in detail, for example, in Japanese Patent Application Laid-Open No. 2021-142770.

[0053] Incidentally, since the viewpoint position of the camera 301 is different from that of the driver, a technique is also known for converting an image generated at the viewpoint position of the camera 301 into a display image adjusted to the driver's viewpoint position using three-dimensional view conversion (see, for example, Japanese Patent Application Laid-Open No. 2017-185988). View conversion is a drafting method for expressing an image generated in a state seen from a certain viewpoint position in a state seen from a different viewpoint position in a three-dimensional space. Specifically, using a parallel transformation matrix, the position of the image in the reference coordinate system in which the image was generated is translated so as to match the coordinate system based on the viewpoint position of the person viewing the image, and using a rotation transformation matrix, the translated image is rotated toward the viewpoint position. The image is converted by combining the coordinate systems. By using this view conversion generally used in the field of three-dimensional graphics, it is possible to generate, convert, and display an image in real time so as to overlap the driver's field of view following the change in the foreground accompanying the movement of the vehicle 1 and the change in the viewpoint position of the driver who is the viewer.

[0054] The display distance determination unit 615 determines the display distance 100 (see FIG. 1) of the first mode images VI1, second mode images VI2, VI that are assigned and displayed in a superimposed or non-superimposed manner on the imaging region VA with respect to an object Ob (for example, a pedestrian shown in FIG. 3A(a)) related to a specific target in the forward field of view of the vehicle 1. 2´ It has a function of determining at what display distance 100 (the distance from the viewpoint position EP of the driver who is the viewer or the distance from the reference point set in the vehicle 1 to the image assigned and displayed in the imaging region VA (also referred to as the imaging distance)) and in what spatial arrangement to display these. Note that the control of the display distance 100 means controlling, from the perspective of the driver who is the viewer, how far forward the first mode images VI1, second mode images VI2, VI are displayed from the vehicle 1. Here, "increasing the display distance" means, in other words, imaging the first mode images VI1, second mode images VI2, VI at a position farther from the instrument box 4. 2´ For example, the first mode images VI1, second mode images VI2, VI 2´ 2´is arranged on the upper side (rear side). Also, "shortening the display distance" means, in other words, the first aspect images VI1, the second aspect images VI2, VI are imaged at positions closer to the eyepiece box 4. 2´ is imaged, for example, the first aspect images VI1, the second aspect images VI2, VI 2´ is arranged on the lower side (front side).

[0055] Regarding the control of the display distance 100, the first aspect images VI1, the second aspect images VI2, VI within the imaging region VA are realized by performing display control in a manner with depth. By displaying these first aspect images VI1, the second aspect images VI2, VI in a manner with depth, for example, based on the distance (relative position relationship information) to a specific object Ob such as a pedestrian in the forward field of view of the vehicle 1, the first aspect images VI1, the second aspect images VI2, VI generated using a perspective method (for example, the one-point perspective method) are realized by performing drawing control. Also, as shown in, for example, (b) of FIG. 5A, (b) of FIG. 5B, and (b) of FIG. 5C described later, the first aspect images VI1, the second aspect images VI2, VI may have depth by using the HUD device 10a that uses the obliquely imaged surface VAD (virtual surface) virtually set in front of the vehicle 1. In this case, considering the projection of the first aspect images VI1, the second aspect images VI2, VI that the viewer, the driver, wants to view from the viewpoint position EP to the obliquely imaged surface VAD, the image generation unit 613 may perform drawing control of the imaging region VA in a manner with depth for the first aspect images VI1, the second aspect images VI2, VI. 2´ are realized by performing display control in a manner with depth. To display these first aspect images VI1, the second aspect images VI2, VI in a manner with depth, for example, based on the distance (relative position relationship information) to a specific object Ob such as a pedestrian in the forward field of view of the vehicle 1, the first aspect images VI1, the second aspect images VI2, VI generated using a perspective method (for example, the one-point perspective method) are realized by performing drawing control. 2´ To display these first aspect images VI1, the second aspect images VI2, VI, for example, based on the distance (relative position relationship information) to a specific object Ob such as a pedestrian in the forward field of view of the vehicle 1, the first aspect images VI1, the second aspect images VI2, VI generated using a perspective method (for example, the one-point perspective method) are realized by performing drawing control. Also, as shown in, for example, (b) of FIG. 5A, (b) of FIG. 5B, and (b) of FIG. 5C described later, the first aspect images VI1, the second aspect images VI2, VI may have depth by using the HUD device 10a that uses the obliquely imaged surface VAD (virtual surface) virtually set in front of the vehicle 1. 2´ In this case, considering the projection of the first aspect images VI1, the second aspect images VI2, VI that the viewer, the driver, wants to view from the viewpoint position EP to the obliquely imaged surface VAD, the first aspect images VI1, the second aspect images VI2, VI are realized by performing drawing control of the imaging region VA in a manner with depth using the HUD device 10a that uses the obliquely imaged surface VAD (virtual surface) virtually set in front of the vehicle 1 as shown in, for example, (b) of FIG. 5A, (b) of FIG. 5B, and (b) of FIG. 5C described later. 2´ In this case, considering the projection of the first aspect images VI1, the second aspect images VI2, VI that the viewer, the driver, wants to view from the viewpoint position EP to the obliquely imaged surface VAD, the image generation unit 613 may perform drawing control of the imaging region VA in a manner with depth for the first aspect images VI1, the second aspect images VI2, VI. 2´ Considering the projection of the first aspect images VI1, the second aspect images VI2, VI that the viewer, the driver, wants to view from the viewpoint position EP to the obliquely imaged surface VAD, the image generation unit 613 may perform drawing control of the imaging region VA in a manner with depth for the first aspect images VI1, the second aspect images VI2, VI. 2´ by performing drawing control of the imaging region VA in a manner with depth for the first aspect images VI1, the second aspect images VI2, VI.

[0056] The display control unit 616 reads the first aspect images VI1, the second aspect images VI2, VI drawn (written) in synchronization with the update timing by the image generation unit 613 from the VRAM (Video RAM) area assigned to a partial area of the storage unit 62. 2´It is read in synchronization with the display timing and control is performed to display it on the vehicle display device 10 (here, the HUD device 10a).

[0057] The control unit 61 including the above-described viewpoint position detection unit 611, imaging image acquisition unit 612, image generation unit 613, display position calculation unit 614, display distance determination unit 615, and display control unit 61 has, for example, a processor with a built-in memory (ROM / RAM) or an externally attached processor mounted thereon, and the processor is associated with an image (first aspect image VI1, second aspect image VI2, VI 2´ ) related to a specific object in the forward field of view of the vehicle 1 and performs control to form the image in an imaging region VA virtually set in front of the vehicle 1. In this way, the processor executes a program recorded in the memory and operates in cooperation with the graphics controller to execute each of the above-described functions. Further, at least a part of the above-described functions can also be realized by hardware such as an FPGA (Field Programmable Gate Array) or a logic circuit, regardless of the processor or the graphics controller.

[0058] The storage unit 62 is a memory in which, for example, a static RAM, a dynamic RAM, or a flash memory or the like in which a program area and a work area are allocated is mounted. Here, in the program area, the display control device 60 (control unit 61) generates an image (first aspect image VI1, second aspect image VI2, VI 2´ ) related to a specific object in the forward field of view of the vehicle 1 and stores a program for executing a process of performing control to form the image in an imaging region VA virtually set in front of the vehicle 1. Further, in the work area, display information including images (first aspect image VI1, second aspect image VI2, VI 2´ ) generated during the execution of the above-described program is also stored, and this display information is developed in a VRAM area allocated to a part of the work area and the display content is sequentially updated in synchronization with the update timing.

[0059] (Operation of the Embodiment) Hereinafter, the operation of the display control device 60 of the present embodiment will be described. As a case of Example 1, the HUD device 10a is used as the vehicle display device 10, a vertical image plane VAE (virtual plane) perpendicular to the road surface is set in front of the vehicle 1, and a 3D image 73 (see FIG. 1) is projected onto this vertical image plane VAE. As a case of Example 2, an oblique image plane VAD (virtual plane) is set with respect to the road surface, and a 2D image 72 (see FIG. 1) that can give a sense of perspective is projected onto this oblique image plane VAD. Each case will be described in detail below.

[0060] (Example 1) FIG. 2 is a flowchart showing the operation of Example 1. (a) of FIG. 3A shows an example of the positional relationship between a first image VI1 arranged and displayed in an imaging region VA when a pedestrian to be superimposed is captured within the viewing angle of the HUD device 10a having a vertical image plane VAE and the pedestrian (object Ob) to be superimposed. (b) of FIG. 3B shows an example of the display distance 100 from the viewpoint position EP of the driver, who is the viewer in the case of a side view of the vehicle 1, to the vertical image plane VAE. (a) of FIG. 3B shows an example of the positional relationship between a second aspect image VI2 arranged and displayed in the imaging region VA when the pedestrian (object Ob) to be superimposed is outside the viewing angle of the HUD device 10a and the pedestrian (off-object Ob) to be superimposed. (b) of FIG. 3B shows an example of the display distance 100 from the viewpoint position EP of the driver in the case of a side view of the vehicle 1 to the vertical image plane VAE. (a) of FIG. 3C shows another example of the positional relationship between a second aspect image VI 2´ and the pedestrian to be superimposed, and (c) of FIG. 3C shows an example of the distance from the viewpoint position EP of the driver in the case of a side view of the vehicle 1 to the vertical image plane VAE.

[0061] First, with reference to FIGS. 2, 3A (a) and (b) to 3C (a) and (b), the operation of Example 1 of the display control device 60 according to the present embodiment shown in FIG. 1 will be described in detail.

[0062] The display control device 60 of this embodiment first has the control unit 61 acquire an object Ob related to a specific target in the forward field of view of the vehicle 1 (for example, the pedestrian shown in FIG. 3A(a)) and its relative position relationship information (step ST101). The relative position information here refers to the relative position of the object Ob related to a specific target (here, the pedestrian) based on the current position of the vehicle 1. For example, it is the position information of the pedestrian measured by the camera 301 of the in-vehicle monitoring device 30 or the LiDAR 303 shown in FIG. 1. In the control unit 61, the captured image acquisition unit 612 recognizes a specific target such as a pedestrian from the captured image of the forward field of view of the vehicle 1 captured by the camera 301, and delivers the relative position information obtained by calculation to the image generation unit 613, the display position calculation unit 614, the display distance determination unit 615, and the display control unit 616.

[0063] Next, the control unit 61 generates a first mode image VI1 (for example, refer to FIG. 3A(a)), a VI2 which is a second mode image (for example, refer to FIG. 3B(a)), and VI 2´ (refer to FIG. 3C(a)) in a mode that overlaps the object Ob related to the specific target within the standing image plane VAE (imaging region VA) (step ST102). The first mode image VI1 and the second mode images VI2, VI 2´ are both generated by the image generation unit 613, and the HUD device 10a (refer to the projection unit 121 in FIG. 7) can be driven by these images. The images (VI1, VI2, VI 2´ ) generated here include display distance data for driving the projection unit 121 so as to display the image 70 at the display distance 100 (refer to FIG. 1) determined by the display distance determination unit 615. Note that the images (VI1, VI2, VI 2´ ) generated here are images associated with the object Ob related to the specific target, and for example, as shown in FIG. 3A(a) as an example of the form of its expression, the display position changes in synchronization with the movement of the object Ob related to the specific target (here, the pedestrian).

[0064] Next, the control unit 61 determines whether or not an object Ob related to a specific target is within the imaging region VA of the virtual image plane VAE (within the angular field of view of the HUD device 10a) (step ST103). In other words, the control unit 61 determines whether or not an object Ob related to a specific target in front of the vehicle 1 can be captured within a predetermined visual field range that can be visually recognized by the driver who is the viewer. Here, the predetermined "visual field range" refers to, for example, a visual field range with a viewing angle of 5 degrees defined as the range of peripheral vision. Peripheral vision plays an important role in grasping spatial positional relationships or motion perception. That is, since the human eye has a function of capturing moving objects in the peripheral vision, a visual field range with a viewing angle of 5 degrees is used as a threshold value.

[0065] The first-mode image VI1 is an image assuming that when an object Ob related to a specific target is captured within the angular field of view of the HUD device 10a (within the imaging region VA), it is superimposed on this specific target. Here, as the optimal method for performing the superimposed display, it is desirable that when the driver who is the viewer visually recognizes an object Ob related to a specific target in the forward field of view of the vehicle 1, the driver can also grasp the information about the object Ob. For this reason, the control unit 61 presents the information by arranging and displaying the first-mode image VI1 at a distance where it can be viewed with a single glance on the object Ob related to a specific target in the forward field of view. Here, "single glance" refers to a state in which the object Ob related to a specific target and the virtual image VI1 of the first-mode image can be visually recognized without moving the line of sight. If it is not possible to view with a single glance, it becomes a state of "double vision", and when one is viewed, the other does not fuse. As this condition, it is known that the value obtained by dividing the parallax angle between the object Ob related to a specific target and the first-mode image VI1, that is, the so-called angle between the objects, by the divergence angle (a parameter indicating the degree of divergence) is 1 or more (for example, refer to "Stereoscopic depth perception in video see-through augmented reality within action space" in Journal of Electronic Imaging Jan-Feb Vol.23(1)).

[0066] That is, in step ST103, when it is determined that the object Ob related to a specific target has entered a predetermined visual field range (area within a viewing angle of 5 degrees) that can be visually recognized by the driver who is the viewer (step ST103 “YES”), the control unit 61 superimposes and displays the first-mode image VI1 on the object Ob related to the specific target at the same distance d1 as the distance measured by the camera 301 or the LiDAR 303 of the in-vehicle monitoring device 30 to the object Ob related to the specific target (Figure 3A (b): the first distance, where the display distance d1 = the distance to Ob) (step ST104). Here, the “same distance” means a distance that allows a single-viewable display distance difference. For example, the display distance 100 may be set to be farther than the distance to the object Ob related to the specific target. Specifically, even if the display distance 100 is different, it is within the range that the driver who is the viewer can visually recognize as being at the same distance. For example, a criterion such as within a viewing angle of 19 seconds (= 1 / 60°) is known.

[0067] In addition, in order to superimpose and display the first-mode image VI1 at the same distance as the object Ob related to the specific target, the control unit 61, according to the display distance 100 (the first distance) determined by the display distance determination unit 615, controls the display position calculation unit 614 to convert the relative position information (camera coordinates) between the object Ob related to the specific target acquired by the captured image acquisition unit 612 and the vehicle 1 into coordinates based on the viewpoint position EP of the driver who is the viewer detected by the viewpoint position detection unit 611 and calculate the display position within the imaging region VA. Then, the display control unit 616 outputs the first-mode image VI1 generated by the image generation unit 613 to the HUD device 10a according to the display timing, thereby performing control to superimpose and display on the object Ob related to the specific target in the front visual field of the vehicle 1.

[0068] On the other hand, in step ST103, when it is determined that the object Ob related to a specific target is not within a predetermined visual field range, in other words, the object Ob related to the specific target is outside the imaging region VA (step ST103 "NO"), the control unit 61 further determines whether the object Ob related to the specific target and the second mode image VI2 (see FIG. 3B(a)) are in a position relationship where they can be viewed singly (step ST105). Here, the second mode image VI2 is an image that is displayed non-overlapping with the object Ob related to the specific target in order to draw attention to the specific target when the object Ob related to the specific target exists outside the imaging region VA. Here, it is shown by the head portion of an arrow having a width indicating the direction in which the pedestrian, which is the object Ob related to the specific target, is located (see FIG. 3B(a)). However, although it is displayed non-overlapping, since the distance between the second mode image VI2 and the pedestrian, which is the object Ob related to the specific target, is relatively close (see FIG. 3B(a)) and it is a situation where they should be viewed singly, the control unit 61 displays the second mode image VI1 non-overlapping with the object Ob related to the specific target at the same distance d2 (FIG. 3B(b): the first distance, where the display distance d2 = the distance to Ob) (step ST106).

[0069] Also, in step ST105, when it is determined that the object Ob (here, the pedestrian), which is the specific target, and the second mode image VI 2´ are in a relatively separated position (see FIG. 3C(a)) and it is determined that they are in a position relationship where single viewing is difficult (step ST105 "NO"), since it is assumed that the viewing behavior of the driver, who is the viewer, will be in the order of looking at the second mode image VI 2´ and then looking at the pedestrian, the control unit 61 2´Perform control to arrange and display it at a position (second distance) in front of the object Ob related to a specific target (see "Display distance d3 < distance to Ob" in (b) of FIG. 3C) (step ST107). By presenting the distance closer (in other words, performing control to shorten the display distance 100) in this way, the driver as the viewer can be made to feel the spatial connection to the object Ob related to the specific target. Therefore, the recognition ability is enhanced and the situation grasp becomes easier. Also, when eye movement is necessary, since it is assumed that the flow will be to look at the object Ob related to the specific target after looking at the second mode image VI 2´ the focus shift between the second mode image VI 2´ and the object Ob (here, a pedestrian) related to the specific target becomes smooth, and the load on the driver as the viewer is reduced.

[0070] Note that the control to shorten the display distance 100 means bringing the second mode image VI 2´ closer (second distance) to the driver (instrument box 4) as the viewer and imaging it in the imaging region VA. For example, the second mode image VI 2´ is arranged on the front side (lower side), and is controlled by the display distance determination unit 615. Also, at this time, the control unit 61 controls the second mode image VI 2´ to be imaged in the imaging region VA by setting the second distance shorter as the degree of deviation from the predetermined visual field range regarding the relative position relationship information with the acquired vehicle 1 is larger. Thus, imaging control considering the depth direction can be performed, the sense of immersion of the generated second mode image VI 2´ into the forward field of view can be further enhanced, and the discomfort with respect to the imaged second mode image VI 2´ can be eliminated.

[0071] (Example 2) FIG. 4 is a flowchart showing the operation of Example 2. Further, (a) of FIG. 5A shows an example of the positional relationship between a first aspect image VI1 arranged and displayed in an imaging region VA when an object Ob (here, a pedestrian) related to a specific object to be superimposed is captured within the angular field of view of the HUD device 10a having an oblique image plane VAD (within the imaging region VA), and the pedestrian as the superimposed object. (b) of FIG. 5A shows an example of the display distance 100 from the viewpoint position EP of the driver, who is the viewer when the vehicle 1 is viewed from the side in that case, to the oblique image plane VAD. (a) of FIG. 5B shows an example of the positional relationship between a second aspect image VI2 arranged and displayed in the imaging region VA when the pedestrian as the superimposed object exists outside the angular field of view of the HUD device 10a (outside the imaging region VA), and the pedestrian. (b) of FIG. 5B shows an example of the display distance from the viewpoint position EP of the driver, who is the viewer when the vehicle 1 is viewed from the side in that case, to the oblique image plane VAD. (a) of FIG. 5C shows a second aspect image VI arranged and displayed in the imaging region VA when the pedestrian as the superimposed object exists outside the angular field of view of the HUD device 10a (outside the imaging region VA) and they are in a positional relationship where they cannot be seen in a single view 2´ and the pedestrian, and (b) of FIG. 5C shows an example of the distance from the viewpoint position EP of the driver, who is the viewer when the vehicle 1 is viewed from the side in that case, to the oblique image plane VAD.

[0072] Hereinafter, with reference to FIGS. 4, 5A to 5C, the operation of Example 2 of the display control device 60 according to the present embodiment shown in FIG. 1 will be described in detail.

[0073] In FIG. 4, first, in the display control device 60 of the present embodiment, the control unit 61 acquires an object Ob (for example, the pedestrian in FIG. 5A(a)) related to a specific target in the forward field of view of the vehicle 1 and relative position relationship information thereof (step ST201). The relative position information referred to here means the relative position of the object Ob related to a specific target based on the current position of the vehicle 1, and is, for example, the position information of the pedestrian measured by the camera 301 or the LiDAR 303 of the in-vehicle monitoring device 30 shown in FIG. 1. In the control unit 61, the imaging image acquisition unit 612 recognizes a specific target such as a pedestrian from the imaging image of the forward field of view of the vehicle 1 captured by the camera 301, and delivers the calculated relative position information to the image generation unit 613, the display position calculation unit 614, the display distance determination unit 615, and the display control unit 616.

[0074] Next, the control unit 61 generates a first aspect image VI1 (for example, refer to FIG. 5A(a)), a second aspect image VI2 (for example, refer to FIG. 5B(a)), and VI 2´ (refer to FIG. 5C(a)) which are expressed in a manner of overlapping the object Ob related to the specific target within the oblique image plane VAD (imaging region VA) (step ST202). The first aspect image VI1 and the second aspect images VI2, VI 2´ are both generated by the image generation unit 613, and the HUD device 10a (refer to the projection unit 121 in FIGS. 6 and 7) can be driven by these images. The first aspect image VI1, the second aspect images VI2, VI 2´ generated here include display distance data for driving the projection unit 121 so as to display the image 70 at the display distance 100 determined by the display distance determination unit 615. The image generated here is an image associated with the object Ob related to a specific target, and is, for example, an image whose display position changes according to the movement of the object Ob (here, the pedestrian) related to the specific target as shown in an example of the form of its expression in FIG. 5A(a).

[0075] Subsequently, the control unit 61 determines whether an object Ob related to a specific target is within the imaging region VA of the oblique image plane VAD (within the angular field of view of the HUD device 10a) (step ST203). In other words, the control unit 61 determines whether an object Ob related to a specific target in front of the vehicle 1 can be captured within a predetermined visual field range visible to the driver who is the viewer. Here, the predetermined "visual field range" refers to, for example, a visual field range with a viewing angle of 5 degrees defined as the range of peripheral vision. Peripheral vision plays an important role in grasping spatial positional relationships or motion perception. That is, since the human eye has a function to capture moving objects in the peripheral vision, here, a visual field range with a viewing angle of 5 degrees is used as a threshold value.

[0076] The first-mode image VI1 is an image assuming that when an object Ob related to a specific target is captured within the angular field of view of the HUD device 10a (within the imaging region VA), it is superimposed on this specific target. Here, as the optimal method for performing the superimposed display, it is desirable that when the driver who is the viewer visually recognizes an object Ob related to a specific target in the front view of the vehicle 1, the driver can also grasp information about the object Ob. For this reason, the control unit 61 presents the information by arranging and displaying the first-mode image VI1 at a distance where it can be viewed in a single view on an object Ob related to a specific target in the front view of the vehicle 1. Here, "single view" refers to a state where the object Ob related to a specific target and the virtual image VI1 of the first-mode image can be visually recognized without eye movement. When a single view is not possible, it is a state of "double vision", and when one is viewed, the other does not fuse. As this condition, it is known that the ratio of the parallax angle between the object Ob related to a specific target and the first-mode image VI1, that is, the so-called angle between the objects, divided by the divergence angle (a parameter indicating the degree of divergence) is 1 or more.

[0077] That is, in step ST203, when it is determined that the object Ob related to a specific target has entered a predetermined visual field range (an area within a viewing angle of 5 degrees) that can be visually recognized by the driver who is the viewer (step ST203 “YES”), the control unit 61 arranges the first mode image VI1 as far as possible on the back side (the first distance) within the diagonal image plane VAD (imaging region VA) (refer to “display distance d1 → arrange as far as possible in the distance within VAD” in (b) of FIG. 5A), and performs control to display it superimposed on the object Ob related to the specific target (step ST204). When the first mode image VI1 is arranged on the diagonal image plane VAD and displayed superimposed on the object Ob (here, a pedestrian) related to the specific target, a sense of depth can be felt by arranging it as far as possible on the back side within the imaging region VA. In this case, considering the projection of the first mode image VI1 to be visually recognized from the viewpoint position EP of the driver who is the viewer onto the diagonal image plane VAD, the image generation unit 613 performs control to draw the first mode image VI1 in the imaging region VA in a mode with depth.

[0078] In the HUD device 10a having the diagonal image plane VAD (virtual plane) that is virtually set in front of the vehicle 1 as the imaging region VA, when the vertical direction of the imaging region VA is the vertical viewing angle and the horizontal direction is the horizontal viewing angle, in the imaging region VA, the region between the upper end and the lower end of the viewing angle set for the HUD device 10a is bisected by a boundary line, and among the regions partitioned thereby, the lower side (the side that feels closer when viewed from the driver who is the viewer) is defined as the “front side”, and the upper side (the side that feels farther when viewed from the driver who is the viewer) can be defined as the “back side”. Note that the boundary line is an example and is not limited thereto.

[0079] On the other hand, in step ST203, when it is determined that the object Ob related to a specific target is not within a predetermined visual field range, in other words, when it is determined that the object Ob related to a specific target is outside the imaging region VA (step ST203 “NO”), the control unit 61 further determines whether or not the object Ob related to a specific target and the second-mode image VI2 (see (a) of FIG. 5B) are in a position relationship where they can be viewed singly (step ST205). Here, the second-mode image VI2 is an image that is displayed non-overlapping with the object Ob related to a specific target, which is prepared to draw attention to the specific target when the object Ob related to the specific target exists outside the imaging region VA. Here, it is indicated by the head portion of an arrow having a width indicating the direction in which the pedestrian, who is the object Ob related to the specific target, is located (see FIG. 5(a)). However, although it is displayed non-overlapping, since the distance between the second-mode image VI2 and the pedestrian, who is the object Ob related to the specific target, is relatively close (see (a) of FIG. 5B) and it is a situation where they should be viewed singly, the control unit 61 arranges the second-mode image VI2 as far as possible on the back side (first distance) within the oblique image plane VAD (within the imaging region VA) (see “Display distance d2 → Arrange as far as possible distantly within VAD” in (b) of FIG. 5B) and performs control to display it in a non-overlapping form with the object Ob (here, the pedestrian) related to the specific target (step ST206).

[0080] Also, in step ST205, when it is determined that the object Ob (here, the pedestrian), which is a specific target, and the second-mode image VI 2´ are in a relatively distant position (see FIG. 5C(a)) and it is determined that they are in a position relationship where single viewing is difficult (step ST205 “NO”), since it is assumed that the viewing behavior of the driver, who is the viewer, will be in the order of looking at the second-mode image VI 2´ and then looking at the pedestrian, the control unit 61 arranges the second-mode image VI 2´ as far as possible on the front side (second distance) within the oblique image plane VAD and performs control to display it in a non-overlapping form with the object Ob related to a specific target in front of the vehicle 1 (see “Display distance d3 → Arrange as close as possible to VAD” in (b) of FIG. 5C) (step ST207). Thus, when line-of-sight movement is necessary, the second-mode image VI 2´Since a flow of viewing the object Ob related to a specific target after viewing is assumed, the second aspect image VI 2´ The focus shift between 2´ and the object Ob (here, a pedestrian) related to a specific target becomes smooth, and the burden on the driver who is the viewer is reduced.

[0081] (Modification example) In the display control device 60 of the present embodiment, the control unit 61 acquires relative position relationship information between the vehicle 1 with respect to a specific target, and forms an image of the first aspect image VI1 superimposed on the specific target within the imaging region VA at a position advanced by a first distance in front of the vehicle 1, and forms an image of the second aspect image VI2 (VI 2´ ) that is not superimposed on the specific target outside the imaging region VA at a position advanced by the first distance or a second distance shorter than the first distance in front of the vehicle 1. However, by incorporating these functions into the HUD device 10a, that is, by incorporating the function of acquiring relative position relationship information between the vehicle 1 with respect to a specific target, the function of forming an image of the first aspect image superimposed on the specific target within the imaging region VA at a position advanced by a first distance in front of the vehicle 1, and the function of forming an image of the second aspect image that is not superimposed on the specific target outside the imaging region VA at a position advanced by the first distance or a second distance shorter than the first distance in front of the vehicle 1 into the HUD device 10a, the functions of the display control device 60 can be realized by the HUD 10a alone. In this case, the processing load of the display control device 60 can be reduced.

[0082] In this case, the HUD device 10a of the present embodiment is composed of, for example, a control unit 11, an image display unit 12, and a storage unit 13 as shown in FIG. 6. The control unit 11 acquires relative position relationship information between the vehicle 1 with respect to a specific target (object Ob), forms an image of the first aspect image superimposed on the specific target within the imaging region VA at a position advanced by a first distance in front of the vehicle 1, forms an image of the second aspect image that is not superimposed on the specific target outside the imaging region VA at a position advanced by the first distance or a second distance shorter than the first distance in front of the vehicle 1, and performs control to display on the image display unit 12.

[0083] The image display unit 12 mainly includes a light source composed of light-emitting diodes mounted on a wiring board and a projection unit 121 including a relay optical system, and a TFT (Thin Film Transistor Liquid Crystal) type liquid crystal display element 122 located on the emission side (directly above) of the light source so as to transmit the illumination light from the light source and form display light. The liquid crystal display element 122 can output display light by transmitting the light emitted from the light source. The liquid crystal display element 122 forms a desired image based on display image data (drive signal) generated under the control of the control unit 11, and superimposes or non-superimposes the first mode image or the second mode image formed in the imaging region VA virtually set in front of the vehicle 1 on the forward field of view to make it visible to the viewer.

[0084] The control unit 11 includes a viewpoint position detection unit 111, a captured image acquisition unit 112, an image generation unit 113, a display position calculation unit 114, a display distance determination unit 115, and a display control unit 116. The viewpoint position detection unit 111 has a function of detecting the viewpoint position of the driver who is the viewer. The captured image acquisition unit 112 has a function of recognizing an object related to a specific target in the forward field of view of the vehicle 1 and capturing its position information. The image generation unit 113 has a function of generating the first mode image VI1, the second mode images VI2, VI 2´ to be displayed on the image display unit 12 and allocating them to the imaging region VA. The display position calculation unit 114 has a function of calculating the display position for allocating the first mode image VI1 or the second mode images VI2, VI 2´ within the imaging region VA and superimposing or non-superimposing it on the object related to the specific target in the forward field of view. The display distance determination unit 115 determines the display distance 100 of the first mode image or the second mode image allocated and displayed in the imaging region VA, and the first mode image VI1 or the second mode images VI2, VI 2´It has a function of displaying in a mode with depth, and the display control unit 116 reads out the first-mode image or the second-mode image (written) drawn by the image generation unit 113 in synchronization with the update timing from the VRAM area assigned to a partial area of the storage unit 13, and displays it on the image display unit 12 in synchronization with the display timing. Since these have the same functions as those of the viewpoint position detection unit 611, the photographed image acquisition unit 612, the image generation unit 613, the display position calculation unit 614, the display distance determination unit 615, and the display control unit 616 of the display control device 60 shown in FIG. 1, the details here are omitted for the purpose of avoiding duplication.

[0085] In the control unit 61 including the above-described viewpoint position detection unit 111, the photographed image acquisition unit 112, the image generation unit 113, the display position calculation unit 114, the display distance determination unit 115, and the display control unit 116, for example, a processor with a built-in memory (ROM / RAM) or an externally attached processor is mounted, and the processor acquires relative position relationship information with the vehicle 1 regarding a specific target, and forms an image of the first-mode image VI1 superimposed on the specific target within the imaging region VA at a position ahead of the vehicle 1 by a first distance, and forms an image of the second-mode images VA2, VA that are not superimposed on the specific target outside the imaging region VA at a position ahead of the vehicle 1 by a first distance or a second distance shorter than the first distance. 2´ Control is performed to form an image at a position. In this way, it is assumed that the processor executes a program recorded in the memory and operates in cooperation with a graphics controller or the like to execute the above-described various functions. Also, at least a part of the above-described functions can be realized by hardware such as an FPGA or a logic circuit, regardless of the processor or the graphics controller.

[0086] Note that the memory unit 13 is a memory in which a program area and a work area are allocated, and for example, a static RAM, a dynamic RAM, or a flash memory, etc. is implemented. Here, in the program area, the control unit 11 acquires relative position relationship information with respect to a specific target and the vehicle 1, and forms a first-mode image VI1 superimposed on the specific target within the imaging area VA at a position advanced by a first distance in front of the vehicle 1, and a second-mode image VA2, VA that is not superimposed on the specific target outside the imaging area VA 2´ is stored with a program for executing a series of processes for performing control to form an image at a position advanced by a first distance or a second distance shorter than the first distance in front of the vehicle 1. Also, in the work area, various information including the first-mode image or the second-mode image generated during the execution of the above-described program is stored, and these images or information are developed in a VRAM (Video RAM) area allocated to a part of the work area, and the display content is sequentially updated in synchronization with the update timing.

[0087] The image display unit 12 includes a projection unit 121 and a liquid crystal display 122. The liquid crystal display 122 is, for example, a transmissive display that transmits light from a backlight, and an actuator such as a motor controlled by the control unit 11 is attached to the display surface and is rotatable. The projection unit 121 is arranged on the optical path of the display light L of the image from the liquid crystal display 122 between it and the windshield 2 (see FIG. 1), and is composed of one or more optical members that project the display light L of the image from the liquid crystal display 122 onto the windshield 2 outside the image display unit 12.

[0088] The internal structure of the projection unit 121 is shown in FIG. 7. The projection unit 121 is composed of a stereoscopic image display unit 141 and a relay optical unit 142. The stereoscopic image display unit 141 forms a first 3D real image 143, and the relay optical unit 142 enlarges the first 3D real image 143 and forms a second 3D real image 144, and projects the display light L of this second 3D real image 144 toward the windshield 2 which is an external projection member.

[0089] The stereoscopic image display unit 141 in FIG. 2 generates a first 3D real image 143 formed in three dimensions, and includes an image projection unit 145 and a vibrating screen 146. The image projection unit 145 is a projector that emits video light (not shown) representing the images (the first mode image and the second mode image) included in the display data based on the display data input from the control unit 11, adjusts the timing of displaying these images according to the display distance data included in the display data, and rapidly switches the images projected in synchronization with the vibration position of the vibrating screen 146. In other words, the image projection unit 145 projects an image suitable for the vibration position of the vibrating screen 146 onto the vibrating screen 146 based on the display distance data.

[0090] The vibrating screen 146 is, for example, a polycarbonate diffusing film that diffuses the video of the image projection unit 145 within a certain angular range, forms a real image by receiving the display light L from the image projection unit 145, and reciprocates along the optical axis of the display light L emitted by the image projection unit 145. The vibrating screen 146 can transmit a signal indicating the vibration position to the image projection unit 145 continuously or intermittently, and the image projection unit 145 may adjust the timing of displaying the image 70 (see FIG. 1) so that the image 70 is visually recognized at a position corresponding to the display distance data included in the display data based on the signal indicating the vibration position.

[0091] The HUD device 10a adjusts the length of the image 70 in the depth direction Z while keeping the amplitude of the vibrating screen 146 in the screen vibration direction constant under the control of the control unit 11. That is, the display light L is not emitted during the period when the vibrating screen 146 is located in the first range of its amplitude, and the display light L is emitted during the period when the vibrating screen 146 is located in the second range of its amplitude. By adjusting the positions and ratios of the first range and the second range, the position and length of the image 70 in the depth direction Z are adjusted. Specifically, the vibrating screen 146 vibrates at a frequency of 60 [Hz] or more, and within this period of 1 / 60 [sec], the image projection unit 145 projects different videos of a plurality of frames to form different videos (real images) of a plurality of frames at each vibration position. That is, the HUD device 10a can generate the first 3D real image 143 when the real images of a plurality of frames overlap in the vibration direction of the vibrating screen 146. Note that the HUD device 10a may adjust the length of the image 70 in the depth direction Z by changing the amplitude of the vibrating screen 146.

[0092] The relay optical unit 142 in FIG. 2 receives the light of the first 3D real image 143 generated by the stereoscopic image display unit 141, forms an intermediate second 3D real image 144 obtained by enlarging the first 3D real image 143, and then projects the display light L, which is the light of the second 3D real image 144, toward the windshield 2. The relay optical unit 142 includes, for example, a first relay optical unit 147 composed of a lens group that receives the light of the first 3D real image 143 generated by the stereoscopic image display unit 141, a second relay optical unit 148 that reflects the light passing through the first relay optical unit 147 and forms a second 3D real image 144 obtained by enlarging the first 3D real image 143 in cooperation with the optical power of the first relay optical unit 147, and a third relay optical unit 149 that reflects the display light L, which is the light of the second 3D real image 144, toward the windshield 2.

[0093] The first relay optical unit 147 has a function of magnifying each image formed at each vibration position of the vibration screen 146 in the first 3D real image 143 at different magnifications. Although it is schematically shown as a single lens in FIG. 7, it is actually composed of a synthetic lens formed by synthesizing a plurality of thin film lenses (not shown).

[0094] The second relay optical unit 148 is composed of, for example, a mirror having a concave reflecting surface with positive optical power. It receives the light of the first 3D real image 143 from the first relay optical unit 147, reflects this incident light toward the third relay optical unit 149, and forms a second 3D real image 144 that magnifies the first 3D real image 143 in cooperation with the optical power of the first relay optical unit 147 between the second relay optical unit 148 and the third relay optical unit 149. Note that the second relay optical unit 148 may be omitted by giving the first relay optical unit 147 the optical action that the second relay optical unit 148 is responsible for. The second relay optical unit 148 is a concave mirror that reflects the display light L of the second 3D real image 144 toward the projection member 2, and has a function of correcting image distortion due to the curved surface shape of the projection member 2 and a function of magnifying the second 3D real image 144.

[0095] Note that the projection unit 140 may adopt a known 3D display method such as a parallax division method including a parallax barrier method or a lenticular lens method, a spatial reproduction method including a light field method or a hologram method, for example, a transmissivity adjustment screen method in which a plurality of screens having a light control layer with adjustable transmissivity are stacked in the thickness direction as disclosed in JP-A-2016-212318, and a projector projects an image while quickly switching the projection image at high speed toward the plurality of screens, and the plurality of screens appropriately adjust their light control rates according to the high-speed switching of the projection image to display a 3D real image inside, a method in which a plurality of liquid crystal display elements are stacked in the thickness direction to display a 3D real image inside as disclosed in JP-A-2004-168230, or a light field display method.

[0096] According to the head-up display device (HUD device 10a) of the present embodiment, the control unit 11 acquires relative position relationship information with the vehicle 1 regarding a specific target, and forms an image of a first-mode image VI1 superimposed on the specific target within the imaging region VA at a position advanced by a first distance in front of the vehicle 1, and forms an image of a second-mode image VA2, VA that is not superimposed on the specific target outside the imaging region VA at a position advanced by a first distance or a second distance shorter than the first distance in front of the vehicle 1, and controls the image display unit 12 to display it. In this way, by changing the display mode of the image related to the specific target according to whether the specific target (object Ob) in the forward field of view of the vehicle 1 is within the imaging region VA, the recognizability (ease of recognition) of the information presented by the image associated with the specific target can be enhanced, and the head-up display device (HUD device 10a) can be provided. Also, by controlling the display distance indicated by the distance between the driver, who is the viewer, and the generated images (the first-mode image VI1 and the second-mode image VI2, VI 2´ ), imaging control considering the depth direction can be performed, and even when the object in the forward field of view to be superimposed and the image related to the superimposed object are too far apart in the depth direction, the sense of immersion of the image into the forward field of view can be enhanced. Therefore, a head-up display device (HUD device 10a) can be provided that allows the driver, who is the viewer, to accurately grasp the information presented by the image without discomfort. 2´ )

[0097] (Effect of the embodiment) As described above, the display control device of the present embodiment, for example, as shown in FIG. 1, images (the first-mode image VI1 and the second-mode image VI2, VI 2´A display control device 60 that generates ) and controls a vehicle display device 10 that virtually forms an image in an imaging region VA set in front of the vehicle 1. The display control device 60 acquires relative position relationship information between the vehicle 1 with respect to a specific target (for example, referring to a pedestrian who is the object Ob in FIG. 3(a)), and forms a first mode image VI1 (for example, refer to (a) in FIG. 3A) that is superimposed on the specific target in the imaging region VA at a position ahead of the vehicle 1 by a first distance, and second mode images VI2, VI 2´ (for example, refer to (a) in FIG. 3B(a) and FIG. 3C) at a position ahead of the vehicle 1 by a first distance or a second distance shorter than the first distance. The display control device 60 has a control unit 61.

[0098] According to the display control device 60 of the present embodiment, by changing the display mode of the image related to the specific target Ob according to whether the specific target Ob in the forward view of the vehicle 1 is within the imaging region VA, the image (the first mode image VI1, and the second mode images VI2, VI 2´ ) can enhance the recognizability (become easier to recognize) of the information presented. Also, by controlling the display distance 100 indicated by the distance between the driver, who is the viewer, and the generated image (the first mode image VI1, and the second mode images VI2, VI 2´ ), imaging control considering the depth direction can be performed. Therefore, even when the object Ob in the forward view that is the superimposed target and the image related to the superimposed target (the first mode image VI1, and the second mode images VI2, VI 2´ ) are too far apart in the depth direction, the immersion of the image into the forward view Ob can be enhanced, and the driver, who is the viewer, can accurately grasp the information presented by the image without a sense of discomfort.

[0099] Here, the "first distance" refers to the display distance 100, which is the distance from the viewpoint position of the driver, who is the viewer, to the first aspect image VI1 imaged in the imaging region VA. For example, it is the "display distance d1 (distance to OB)" shown in Fig. 3A(b). Also, the "second distance" refers to the display distance 100, which is shorter than the first distance and is the distance from the viewpoint position of the driver, who is the viewer, to the second aspect image imaged in the imaging region VA. For example, it is the "display distance d3 < distance to Ob" shown in Fig. 3C(b).

[0100] Also, in the display control device 60 of the present embodiment, the control unit 61 2´ obtains the second aspect image VI and performs control to set the second distance shorter as the degree of deviation from a predetermined visual field range regarding the relative position relationship information with the vehicle 1 increases, and then images it in the imaging region VA. By doing so, the display distance 100 can be controlled to perform imaging control considering the depth direction, and the sense of immersion in a specific object Ob in the front visual field of the generated image (second aspect image VI 2´ ) can be enhanced, and the discomfort of the imaged image can be eliminated. Note that the "predetermined visual field range" refers to, for example, a visual field range with a visual angle of 5 degrees defined as the range of the peripheral visual field. The human eye's visual field is divided into the central visual field and the peripheral visual field. The central visual field is about 2 degrees of visual angle (range with a radius of 1 degree from the fovea) from the perspective of resolution visual acuity, 5 degrees of visual angle from the perspective of the retinal structure, and within 7 degrees of visual angle in the neuroanatomical classification regarding the visual cortex. The central visual field is a part with many cone cells and high resolution ability, with the best color discrimination ability and visual acuity, and can clearly see things. On the other hand, in the peripheral visual field, there are few cone cells, colors cannot be seen well, and the visual acuity is lower than that of the central visual field, but the density of rod cells is high and it is extremely sensitive to motion stimuli. The peripheral visual field plays an important role in grasping spatial position relationships or motion perception. That is, since the human eye has a function of capturing moving objects in the peripheral visual field and grasping the details in the central visual field, here, a visual field range with a visual angle of 5 degrees is used as the "predetermined threshold".

[0101] Also, according to the display control device 60 of the present embodiment, the control unit 61 is the second aspect image (VI2, VI2´ ) If it is determined that the imaging position of ) is within a predetermined visual field range with respect to a specific target Ob, control is performed to image the image (second-mode image VI2) at a position forward of the vehicle 1 by a first distance. If it is determined that the position is not within the predetermined visual field range and the two are in a positional relationship where they cannot be viewed in a single view, control is performed to image the image (second-mode image VI 2´ ) at a position forward of the vehicle 1 by a second distance. Therefore, when line-of-sight movement is necessary (when the specific target Ob is not within the predetermined visual recognition region and the distance between the second-mode image VI 2´ and the specific target Ob is not in a positional relationship where single viewing is possible), since it is assumed that the driver, who is the viewer, will have a behavior pattern of looking at the specific target Ob in the forward field of view after looking at the image VI 2´ imaged in the imaging region VA, the focus movement between the second-mode image VI 2´ and the specific target Ob becomes smooth and the load on line-of-sight movement is reduced. Also, in the range where line-of-sight movement is not required and viewing is possible, a second-mode image VI 2´ that can be regarded as the same as the specific target is provided, making it easier to grasp information.

[0102] Further, according to the display control device 60 of the present embodiment, when the control unit 61 determines that the imaging position of the second-mode image VI2 is not within the predetermined visual field range with respect to a specific target, it images on the front side of the imaging region VA having an oblique image plane VAD that is tilted forward with respect to the front-rear direction of the vehicle 1. When it is determined that the position is within the predetermined visual field range, control is performed to image on the back side of the imaging region VA. Therefore, also in the HUD device 10a having the oblique image plane VAD, similar to the HUD device 10a having the upright image plane VAE, when line-of-sight movement is necessary, the driver, who is the viewer, will view the object Ob in the forward field of view after viewing the image imaged in the imaging region VA. Therefore, the focus movement between the image (second-mode image VI 2´ ) and the target Ob becomes smooth and the load on line-of-sight movement is reduced. In the range where line-of-sight movement is not required and viewing is possible, an image (second-mode image VI2) that can be regarded as the same as the target Ob is provided, making it easier to grasp information.

[0103] In addition, as shown in FIG. 1 for example, the head-up display device of the present embodiment is an HUD device 10a that generates images (a first-mode image VI1 and second-mode images VI2, VI 2´ ) associated with a specific object Ob in the forward field of view of the vehicle 1 and forms the images in an imaging region VA virtually set in front of the vehicle 1. The HUD device 10a includes an image display unit 12 that projects the generated images toward a projection member 2 (windshield), acquires relative position relationship information between the vehicle 1 and a specific object, and forms the first-mode image VI1 superimposed on the specific object within the imaging region VA at a position forward of the vehicle 1 by a first distance, and forms the second-mode images VA2, VA 2´ that are not superimposed on the specific object outside the imaging region VA at a position forward of the vehicle 1 by a first distance or a second distance shorter than the first distance.

[0104] According to the HUD device 10a of the present embodiment, by changing the display mode of the image associated with the specific object Ob depending on whether the specific object Ob in the forward field of view of the vehicle 1 is within the imaging region VA, the recognizability of the information presented by the images (the first-mode image VI1 and the second-mode images VI2, VI 2´ ) can be enhanced (it becomes easier to recognize). In addition, imaging control considering the depth direction can be performed by controlling the display distance 100 indicated by the distance between the driver as the viewer and the generated images (the first-mode image VI1 and the second-mode images VI2, VI 2´ ). Therefore, even when the object Ob in the forward field of view to be superimposed and the images (the first-mode image VI1 and the second-mode images VI2, VI 2´ ) associated with the superimposed object are too far apart in the depth direction, the immersiveness of the images into the forward field of view Ob can be enhanced, and the HUD device 10a can be provided such that the driver as the viewer can accurately grasp the information presented by the images without a sense of discomfort.

[0105] In addition, as shown in FIG. 1 for example, the display control program of the present embodiment generates images (a first-mode image VI1 and second-mode images VI2, VI 2´ ) associated with a specific object in the front view of the vehicle 1, and is a display control program for controlling the vehicle display device 10 that forms an image of the images in an imaging region VA virtually set in front of the vehicle 1. And, by the display control program, a processor included in the display control device reads and executes a program recorded in a memory, and for example, as shown in FIG. 2, a process of acquiring relative position relationship information between the vehicle 1 and a specific object Ob (step ST101), a process of forming an image (a first-mode image VI1) expressed in a first mode and superimposed on the specific object Ob within the imaging region VA at a position advanced by a first distance in front of the vehicle 1 (steps ST102 to ST104), and a process of forming an image (second-mode images VI2, VI 2´ ) expressed in a second mode and not superimposed on the specific object Ob outside the imaging region VA at a position advanced by the first distance or a second distance shorter than the first distance in front of the vehicle 1 (steps ST105 to ST107) are executed.

[0106] According to the display control program of the present embodiment, by a processor included in the display control device reading and executing a program recorded in a memory, depending on whether a specific object Ob in the front view of the vehicle 1 is within the imaging region VA, the display mode of an image associated with the specific object Ob is changed, so that the recognizability of information presented by the images (a first-mode image VI1 and second-mode images VI2, VI 2´ ) can be enhanced (it becomes easier to recognize). Also, imaging control considering the depth direction can be performed by controlling the display distance 100 indicated by the distance between the driver, who is the viewer, and the generated images (a first-mode image VI1 and second-mode images VI2, VI 2´ ). Therefore, with respect to the object Ob in the front view that is the superimposition target and the images (a first-mode image VI1 and second-mode images VI2, VI 2´Even when they are too far apart in the depth direction, the sense of immersion of the image in the forward field of view Ob can be enhanced, and the driver, who is the viewer, can accurately grasp the information presented by the image without discomfort.

[0107] Also, as shown in FIG. 1, for example, the vehicle display system according to the present embodiment generates an image (first aspect image VI1, second aspect images VI2, VI 2´ ) associated with a specific object Ob in the forward field of view of the vehicle 1, and a head-up display device (HUD device 10a) that forms the image in an imaging region VA virtually set in front of the vehicle 1, and a display control device 60 that controls the display of the head-up display device (HUD device 10a). The vehicle display system 300 is composed of these. And in the vehicle display system 300, the display control device 60 acquires relative position relationship information between the vehicle 1 and a specific object (for example, refer to the pedestrian which is the object Ob in FIG. 3(a)), and a first aspect image VI1 (for example, refer to (a) in FIG. 3A) that is superimposed on the specific object within the imaging region VA is formed at a position in front of the vehicle 1 by a first distance, and a second aspect image VI2, VI 2´ (for example, refer to (a) in FIG. 3B, (a) in FIG. 3C) that is not superimposed on the specific object outside the imaging region VA is formed at a position in front of the vehicle 1 by a first distance or a second distance shorter than the first distance, and the HUD device 10a projects the first aspect image VI1 or the second aspect images VI2, VI 2´ toward the projection member 2 (windshield).

[0108] According to the vehicle display system 300 of the present embodiment, the display control device 60 performs control to change the display mode of the image associated with the specific object Ob according to whether the specific object Ob in the forward field of view of the vehicle 1 is within the imaging region VA, and the HUD device 10a projects the first aspect image VI1 or the second aspect images VI2, VI 2´ toward the projection member 2, so that the image (first aspect image VI1 and second aspect images VI2, VI 2´) can enhance the recognizability (make it easier to recognize) of the information presented. Also, by controlling the display distance 100 indicated by the distance between the driver as the viewer and the generated images (the first-mode image VI1 and the second-mode images VI2, VI 2´ ) imaging control considering the depth direction can be performed. Therefore, even when the object Ob in the front view to be superimposed and the images (the first-mode image VI1 and the second-mode images VI2, VI 2´ ) are too far apart in the depth direction, the sense of immersion of the images into the front view Ob can be enhanced, and the vehicle display system 300 can be provided such that the driver as the viewer can accurately grasp the information presented by the images without discomfort.

[0109] The present invention is not limited to the above-described exemplary embodiments, and those skilled in the art will be able to easily modify the above-described exemplary embodiments to the extent included in the scope of the claims.

Explanation of Reference Numerals

[0110] 1 ··· Vehicle, 2 ··· Projection member (windshield), 3 ··· Eye, 4 ··· Eyebox, 10 Vehicle display device, 10a ··· HUD device, 11 ··· Control unit (HUD side), 12 ··· Image display unit, 13 ··· Storage unit, 20 ··· Vehicle ECU, 30 ··· In-vehicle monitoring device, 40 ··· Navigation device, 50 ··· Operation input device, 60 ··· Display control device, 70 ··· I / O interface, 100 (d1, d2, d3) ··· Display distance (first distance, second distance), 121 ··· Projection unit, 122 ··· Liquid crystal display, 300 ··· Vehicle display system, 301 ··· Camera, 302 ··· GPS, 303 ··· LiDAR, 304 ··· Behavior sensor, 400 ··· Map information DB, 611 ··· Viewpoint position detection unit, 612 ··· Captured image acquisition unit, 613 ··· Image generation unit, 614 ··· Position calculation unit, 615 ··· Display distance determination unit, 616 ··· Display control unit, VA ··· Imaging region, VAE ··· Erect image plane, VAD ··· Oblique image plane, Ob ··· Object of a specific object in the front view, VI1 ··· First-mode image, VI2, VI 2´···Second Aspect Image

Claims

1. A display control device for controlling a vehicle display device that generates an image associated with a specific object in the vehicle's forward field of view and forms the image in an imaging region virtually set in front of the vehicle, acquiring relative position relationship information between the vehicle and the specific object, forming a first aspect image superimposed on the specific object within the imaging region at a position forward of the vehicle by a first distance, and a control unit that forms a second aspect image not superimposed on the specific object outside the imaging region at a position forward of the vehicle by the first distance or a second distance shorter than the first distance. A display control device having the above.

2. The control unit, performs control to form the second aspect image in the imaging region by setting the second distance shorter as the degree of deviation from a predetermined field of view range regarding the acquired relative position relationship information between the vehicle and the specific object is greater. The display control device according to Claim 1.

3. The control unit, when it is determined that the imaging position of the second aspect image is within a predetermined field of view range with respect to the specific object, performs control to form the image at a position forward of the vehicle by the first distance, and when it is determined that the imaging position of the second aspect image is not within a predetermined field of view range with respect to the specific object, performs control to form the image at a position forward of the vehicle by the second distance. The display control device according to Claim 1 or 2.

4. The control unit, when it is determined that the imaging position of the second aspect image is not within a predetermined field of view range with respect to the specific object, forms the image on the front side of the imaging region having an inclined image plane installed inclined forward with respect to the longitudinal direction of the vehicle, and when it is determined that the imaging position of the second aspect image is within a predetermined field of view range with respect to the specific object, performs control to form the image on the back side of the imaging region. The display control device according to Claim 1 or 2.

5. A head-up display device that generates an image associated with a specific object in the vehicle's forward field of view and forms the image in an imaging region virtually set in front of the vehicle, an image display unit that projects the generated image toward a projection member, acquiring relative position relationship information between the vehicle and the specific object, A head-up display device having a control unit that forms a first aspect image superimposed on the specific object within the imaging region at a position in front of the vehicle by a first distance, and forms a second aspect image not superimposed on the specific object outside the imaging region at a position in front of the vehicle by the first distance or a second distance shorter than the first distance.

6. A display control program for a display control device that controls a vehicle display device that generates an image associated with a specific object in the forward field of view of the vehicle and projects the image onto an imaging region virtually set in front of the vehicle, wherein the processor included in the display control device, performs a process of acquiring relative position relationship information between the vehicle and the specific object, performs a process of forming a first aspect image superimposed on the specific object within the imaging region at a position in front of the vehicle by a first distance, and performs a process of forming a second aspect image not superimposed on the specific object outside the imaging region at a position in front of the vehicle by the first distance or a second distance shorter than the first distance.

7. A vehicle display system including a head-up display device that generates an image associated with a specific object in the forward field of view of the vehicle and forms the image in an imaging region virtually set in front of the vehicle, and a display control device that performs display control of the head-up display device, wherein the display control device, acquires relative position relationship information between the vehicle and the specific object, performs control to form a first aspect image superimposed on the specific object within the region of the imaging region at a position in front of the vehicle by a first distance, performs control to form a second aspect image not superimposed on the specific object outside the imaging region at a position in front of the vehicle by the first distance or a second distance shorter than the first distance, and the head-up display device, projects the first aspect image or the second aspect image toward a projection member.

Images