Head-up display device, display control device, and display control system

Abstract

To facilitate grasping a contextual relationship between images without weakening the power of depth expression even when the eye position in the height direction of a viewer changes.SOLUTION: Provided is a head-up display device (HUD device 100) for projecting and displaying one or more virtual images V to a display area VA virtually set forward of a vehicle 10 by superimposition on the foreground, the HUD device comprising a display unit 120 for displaying the virtual image V and a control unit 160 for exercising control to display the virtual image V on the display unit 120. When the angle (depression angle) formed by a line parallel to a road surface, with the eye position in the height direction of a viewer 14 (eye point EP) taken as a point of reference, and a line linking the eye position in the height direction of the viewer 14 and the center of the display area VA is smaller than or equal to a prescribed threshold, the control unit 160 exercises control to change the display mode of the virtual image V before being displayed on the display unit 120.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a head-up display device or the like that is used in a moving body such as a vehicle and that allows a virtual display object to be visually superimposed on the foreground of the moving body. [Background technology]

[0002] For example, Patent Document 1 describes a head-up display device (hereinafter, simply referred to as a HUD device) that uses a virtual display object (virtual image) to enhance expressiveness in the depth direction (front-rear direction of the vehicle) and enables intuitive information presentation to a viewer such as a driver. This HUD device arranges virtual images at predetermined positions in the depth, top-bottom-left-right directions within the space of the foreground (the actual view in the forward direction as seen by the viewer of the vehicle), and forms an augmented reality (AR) in which the virtual images are added to the real scenery (foreground) such as the road surface and displayed.

[0003] Such HUD devices can give the viewer a sense of depth by using an inclined imaging surface (display area), which makes the vertically arranged images appear to be at different distances, making it easier to distinguish between different pieces of information. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication No. 2021-051231 Summary of the Invention

[0005] In the above-mentioned HUD device, there is known a device that sets a vanishing point in linear perspective inside or outside the display area (imaging surface), arranges the speed limit sign of the currently traveling road and the speed limit sign image (virtual image) of the nearest road along an image (virtual image) that imitates the driving lane passing through this vanishing point, and controls the display of these images (virtual images) so that they are aligned with the road surface as seen by the viewer. This allows the viewer to grasp the front-back relationship between the images, with these images (virtual images) arranged along the road surface.

[0006] Incidentally, an image (virtual image) that is arranged and projected on a display area (imaging surface) that is virtually set in front of the vehicle is displayed assuming that the viewer's eyes are positioned at a reference position in the height direction. Therefore, if the viewer's eyes change in the height direction, the way the image (virtual image) appears to the viewer also changes.

[0007] Here, for example, assume that the distance difference between the top and bottom ends of the display area (imaging surface) is large. As described above, when images (virtual images) are arranged vertically, they are perceived as being at different distances. For example, when an image simulating a traffic lane is displayed based on the viewer's eye position in the height direction, and multiple images having the same meaning arranged at different distances along the image are displayed, if the viewer's eye position in the height direction is lower than the reference position, the drawable area is limited (the display area becomes narrower), and the vertical component of the image (virtual image) arranged in the display area (imaging surface) becomes shorter, so that the expressive power of depth is weakened (it becomes difficult for the viewer to perceive a sense of depth). As a result, each image (virtual image) may not be perceived as being at a different distance by the viewer, and in this case, the anteroposterior relationship between the images is not correctly grasped, and as a result, the display content may not be correctly conveyed to the viewer.

[0008] The present invention has been made to solve such problems, and aims to provide a head-up display device, a display control device, and a display control system that make it easy to grasp the anteroposterior relationship between images without weakening the expressive power of depth even when the viewer's eye position in the vertical direction changes.

[0009] Other objects of the present invention will become apparent to those skilled in the art by referring to the following exemplary aspects and best modes, as well as the accompanying drawings. [Means for solving the problem]

[0010] In order to facilitate an understanding of the outline of the present invention, embodiments according to the present invention will be exemplified below.

[0011] A first aspect is a head-up display device that projects and displays one or more virtual images superimposed on the foreground in a display area virtually set in front of a vehicle, and has a display unit that displays the virtual image, and a control unit that controls the display of the virtual image on the display unit, and when an angle between a straight line parallel to the road surface based on the viewer's eye position in the height direction and a straight line connecting the viewer's eye position in the height direction and the center of the display area is equal to or less than a predetermined threshold value, the control unit controls to change the display mode of the virtual image and display it on the display unit.

[0012] Here, the "position of the viewer's eyes in the height direction" refers to, for example, the eye point EP of the viewer 14 as shown in FIG. 1(a), and can be detected by an eye position detection unit. In addition, the virtual image displayed by the head-up display device is correctly viewed in a predetermined limited area (hereinafter, also referred to as the eye box). Therefore, since the eye point EP of the viewer 14 can be estimated to be within the eye box, information on the "position of the eye box" can be substituted for information on the "position of the eye box". Since the "position of the eye box" and the position of the display area are in a corresponding relationship (when the eye box moves, the display area also moves accordingly), in some embodiments, it can also be determined by the "position of the eye box" whether or not the angle formed by the straight line connecting the viewer's eye position in the height direction and the center of the display area is equal to or smaller than a predetermined threshold value.

[0013] Furthermore, since the "position of the eye box" is determined by the direction of the display light L emitted from the head-up display device (the position projected onto the projection target), the position of the eye box changes depending on the angle and / or position of a relay optical system (e.g., a concave mirror) in the head-up display device that determines the direction of the display light L emitted from the head-up display device. Therefore, information on the "position of the eye box" (information on the position of the viewer's eyes in the height direction) can be replaced with information on the angle and / or position of the relay optical system (e.g., a concave mirror) or the like. In some embodiments, the determination of whether the angle formed by the straight line connecting the viewer's eye position in the height direction and the center of the display area is equal to or smaller than a predetermined threshold value can also be determined by the "angle and / or position of the relay optical system (e.g., a concave mirror) or the like." In the head-up display device, the concave mirror is connected to a drive unit, and the drive unit, under the control of the control unit, can rotate the concave mirror (an example of a relay optical system) within a predetermined angle range based on the viewer's eye position in the height direction (eye point EP) set by the viewer or detected by an eye position detection unit, thereby changing the arrival point of the display light in the windshield. In other words, the position of the display area where the image (virtual image) is placed can be changed with the change in the position of the eye box.

[0014] According to the first aspect, the control unit controls the display unit to change the display mode of the virtual image when the angle between a line parallel to the road surface based on the viewer's eye position in the height direction (eye point EP) and a line connecting the viewer's eye position in the height direction (eye point EP) and the center of the display area is equal to or smaller than a predetermined threshold. In this way, by changing the display mode of the virtual image and displaying it even if the viewer's eye position in the height direction (eye point EP) is lower than the reference position, it is possible to provide a head-up display device that allows the viewer to easily grasp the front-to-back relationship of each image without losing the sense of perspective (without weakening the expressive power of depth) even if the drawable area is limited (the vertical component of the virtual image placed in the display area becomes shorter).

[0015] In a second aspect dependent on the first aspect, the control unit may change the display mode by reducing the total number of the virtual images that can be arranged in a direction that includes at least a vertical component of the display area, and control the display on the display unit.

[0016] Here, the "vertical component of the display area" refers to the vertical angle of view having a vertical component of the angle of view AV within the angle range in which the viewer can view the images (virtual images V1, V2) as seen from the viewer's eye position in the height direction (eye point EP), as shown in the relationship between the display area VA and the angle of view AV in Fig. 1(b). Also, the "total number of virtual images that can be arranged" refers to the maximum number of virtual images that can be arranged in a direction that includes at least the vertical component of the display area, and is a value determined by, for example, the size of the virtual images to be arranged, the interval between the virtual images, and further the size of the vertical component of the display area (drawable area).

[0017] According to the second aspect, when the control unit changes the display mode, it controls the display unit to reduce the total number of virtual images that can be arranged in a direction that includes at least the vertical component of the display area. This makes it easier to grasp the relationship between the front and back of each image without compromising the sense of perspective (without weakening the expressive power of depth) even if the viewer's eye position in the vertical direction (eye point EP) is lower than the reference position and the drawing area is limited (the vertical component of the virtual images arranged in the display area becomes shorter).

[0018] In a third aspect dependent on the first or second aspect, the control unit may set a vanishing point in linear perspective inside or outside the display area, and perform control to position the virtual image within a predetermined range visible to the viewer so that the virtual image follows a perspective image generated by emphasizing the sense of perspective in an image simulating a driving lane passing through the vanishing point.

[0019] Here, the "predetermined range" is, for example, a value determined based on a threshold value of whether or not a viewer can visually recognize each image as being arranged along the road surface, and is a value obtained by a test, a simulation, or the like. For example, as shown in an example of a display form in FIG. 6, the inclination of the perspective image VP simulating a driving lane is 40°, and the images (virtual images VT1, VT2) representing two speed limit signs are also arranged so that the inclination is 40°. If the difference between these inclinations becomes large, each image (virtual images VT1, VT2) may be visually recognized as not being along the road surface. In particular, when each image (virtual images VT1, VT2) is arranged so that it is vertical, that is, so that the inclination is 90°, the viewer cannot visually recognize each image (virtual images VT1, VT2) as being arranged along the road surface. Therefore, the "predetermined range" is set to a value (inclination) that does not allow such an arrangement, for example, 8° or more and less than 66° is set as the range.

[0020] For this reason, the control unit sets a vanishing point in linear perspective inside or outside the display area VA, and performs control to arrange the multiple virtual images VT1, VT2 (see FIG. 6) in a predetermined range (with an inclination of 8° or more and less than 66°) that can be visually recognized as being arranged along a perspective image VP generated by emphasizing the perspective in an image simulating a driving lane passing through the vanishing point. Note that the perspective image VP is an image generated by performing image processing to emphasize the perspective of the target image by changing the brightness, shading, color tone, etc. according to the display distance and the size of the target, or by performing image processing such as shading, overlapping, and blurring.

[0021] For example, as shown in Fig. 6, a vanishing point (a point where two or more lines that are originally parallel in linear perspective intersect when they are drawn as non-parallel) is set within or outside the angle of view of the image (virtual image), and speed limit sign images (virtual images VT1, VT2) are arranged on an image that mimics the driving lanes toward the vanishing point, so that the viewer can visually recognize these images (virtual images VT1, VT2) as if they are displayed along the road surface. In addition, the front-to-back relationship between the virtual images VT1 and VT2 can be expressed by arranging the speed limit sign image (virtual image VT1) of the road currently being driven on the lower side (the side that seems to be in front of the viewer) at the front side of the display area (imaging surface) and the speed limit sign image (virtual image VT2) of the nearest road on the upper side (the side that seems to be in the back of the viewer) at the back side of the display area (imaging surface).

[0022] According to the third aspect, the control unit sets a vanishing point in linear perspective inside or outside the display area, and controls the placement of the virtual image within a specified range that can be seen by the viewer so that the virtual image appears to be placed along a perspective image that is generated by emphasizing the perspective of an image that mimics a driving lane that passes through the vanishing point.This allows the viewer to see each image as if it were placed along the road surface, and makes it easy to grasp the chronological relationship between the viewed images.

[0023] In a fourth aspect dependent on the first to third aspects, the control unit may further control the display unit to change a relative size of the virtual image that gives the viewer a sense of perspective.

[0024] According to the fourth aspect, the control unit changes the relative size of the virtual images that give the viewer the perception of perspective, so that the viewer's eye position in the vertical direction (eye point EP) becomes lower than a reference position, and the viewer can view the image without compromising the sense of perspective even if the total number of virtual images that can be positioned in a direction that includes at least the vertical component of the display area is reduced.

[0025] In a fifth aspect which is dependent on the first to fourth aspects, the control unit may further control the display unit to change at least one of the size of the virtual image, the decoration of the virtual image, and the arrangement of the virtual image.

[0026] According to the fifth aspect, the control unit changes at least one of the size of the virtual image, the decoration of the virtual image, and the arrangement of the virtual image, so that the viewer's eye position in the height direction (eye point EP) becomes lower than the reference position, and the total number of virtual images that can be arranged in a direction that includes at least the vertical component of the display area can be reduced, allowing the viewer to view the image without losing the sense of perspective. For example, Fig. 7(a) shows an example of a display form in which the decoration of the virtual image V is simplified and controlled, and Fig. 7(b) shows an example of a display form in which the virtual image V is rearranged.

[0027] In a sixth aspect which is dependent on the first to fifth aspects, the control unit may perform control so that, when an end of a display area virtually set in front of the vehicle is set as the bottom end on the side closer to the vehicle and an end on the side farther from the vehicle is set as the top end, the top end is perceived by the viewer as being tilted and further back than the bottom end.

[0028] According to the sixth aspect, the control unit can perform display with enhanced expressiveness in the depth direction (front-rear direction of the vehicle) by using a display area VA, which is an inclined virtual image formation surface such that the upper end of the display area (e.g., PU in FIG. 1(b)) is deeper than the lower end (e.g., PL in FIG. 1(b)), and can realize more natural visibility of images with a sense of perspective. In addition, the display distance (imaging distance) can be adjusted according to the display position, and a display with a sense of depth can be realized over a fairly wide range on the upper end side, making it easier to grasp the relationship between the front and rear of the images.

[0029] A seventh aspect is a display control device that controls a head-up display device that projects and displays one or more virtual images superimposed on the foreground in a display area virtually set in front of a vehicle, and a processor changes the display mode of the virtual image when the angle (hereinafter also referred to as the depression angle) between a line parallel to the road surface based on the viewer's eye position in the vertical direction and a line connecting the viewer's eye position in the vertical direction and the center of the display area is equal to or less than a predetermined threshold value.

[0030] According to the seventh aspect, the processor changes the display mode of the virtual image when the angle between a line parallel to the road surface based on the viewer's eye position in the height direction and a line connecting the viewer's eye position in the height direction and the center of the display area is equal to or smaller than a predetermined threshold value. In this way, by changing the display mode of the virtual image even if the viewer's eye position in the height direction (eye point EP) is lower than the reference position, it is possible to provide a display control device that makes it easy to grasp the anteroposterior relationship of each image without compromising the sense of perspective (without weakening the expressive power of depth) even if the drawing possible area is limited (the vertical component of the virtual image placed in the display area becomes shorter).

[0031] In an eighth aspect dependent on the seventh aspect, the processor may change the display mode by reducing a total number of the virtual images that can be arranged in a direction that includes at least a vertical component of the display area.

[0032] According to the eighth aspect, a display control device can be provided in which the processor changes the display mode by reducing the total number of virtual images that can be arranged in a direction that includes at least the vertical component of the display area, making it easy to grasp the context of each image without compromising the sense of perspective even if the viewer's eye position in the vertical direction (eye point EP) is lower than the reference position and the drawing area is limited.

[0033] In a ninth aspect which is dependent on the seventh or eighth aspect, the processor may set a vanishing point in linear perspective inside or outside the display area, and perform control to position the virtual image within a predetermined range visible to the viewer so that the virtual image is aligned with a perspective image generated by emphasizing the sense of perspective in an image simulating a driving lane passing through the vanishing point.

[0034] According to the ninth aspect, a display control device can be provided in which the processor sets a vanishing point in linear perspective inside or outside the display area, and controls the placement of the virtual image within a specified range within which the viewer can see the virtual image as being positioned along a perspective image generated by emphasizing the perspective of an image that mimics a driving lane passing through the vanishing point, thereby making it possible to provide a display control device in which the images are seen by the viewer as if they were positioned along the road surface and which makes it easy to grasp the chronological relationship between the viewed images.

[0035] In a tenth aspect dependent on any of the seventh to ninth aspects, the processor may further vary a relative size of the virtual image to give the viewer a sense of perspective.

[0036] According to the tenth aspect, a display control device can be provided in which a processor changes the relative size of virtual images that give the viewer the perception of perspective, thereby lowering the viewer's eye position in the vertical direction (eye point EP) below a reference position, and allowing the viewer to view without compromising the sense of perspective even if the total number of virtual images that can be positioned in a direction that includes at least the vertical component of the display area is reduced.

[0037] In an eleventh aspect dependent on any of the seventh to tenth aspects, the processor may further change at least one of a size of the virtual image, a decoration of the virtual image, and a placement of the virtual image.

[0038] According to the eleventh aspect, a display control device can be provided in which the processor further changes at least one of the size of the virtual image, the decoration of the virtual image, and the arrangement of the virtual image, thereby lowering the viewer's eye position in the vertical direction (eye point EP) below a reference position, and allowing the viewer to view the virtual image without compromising the sense of perspective even if the total number of virtual images that can be arranged in a direction that includes at least the vertical component of the display area is reduced.

[0039] In a twelfth aspect which is dependent on the seventh to eleventh aspects, the processor may further control the angle and / or position of a relay optical system of the head-up display device, and may determine that the depression angle is equal to or less than the predetermined threshold value when the angle and / or position of the relay optical system falls within a predetermined range.

[0040] According to the twelfth aspect, the processor can further control the angle and / or position of the relay optical system of the head-up display device, and when the angle and / or position of the relay optical system falls within a predetermined range, the processor can determine that the depression angle is below a predetermined threshold, thereby changing the position of the display area in which the virtual image is placed in accordance with changes in the position of the eyebox.

[0041] A thirteenth aspect is a display control system including a head-up display device that projects and displays one or more virtual images superimposed on the foreground in a display area virtually set in front of a vehicle, and a processing device that generates the virtual images and arranges them in the display area, wherein the head-up display device has a display unit that displays the virtual image, and a control unit that controls the display of the virtual image on the display unit, and when there is a change in the viewer's eye position in the height direction (eye point EP), the processing device outputs a display command instructing the head-up display device to change the display mode of the virtual image and display it when the angle formed by a straight line parallel to the road surface based on the viewer's eye position in the height direction and a straight line connecting the viewer's eye position in the height direction and the center of the display area is equal to or greater than a predetermined threshold value.

[0042] According to the thirteenth aspect, when the angle between a straight line parallel to the road surface based on the viewer's eye position in the height direction (eye point EP) and a straight line connecting the viewer's eye position in the height direction (eye point EP) and the center of the display area is equal to or less than a predetermined threshold value, the processing device outputs a display command instructing the head-up display device to change the display mode of the virtual image and display it, and the head-up display device (control unit) that receives the display command performs control to change the display mode of the virtual image, for example, by reducing the total number of virtual images that can be arranged in a direction that includes at least the vertical component of the display area, thereby providing a display control system that makes it easy to grasp the anteroposterior relationship of each image without compromising the sense of perspective even if the drawing area is limited (the vertical component of the virtual images arranged in the display area is shortened).

[0043] Those skilled in the art will easily understand that the exemplified embodiments according to the present invention can be further modified without departing from the spirit of the present invention. [Brief description of the drawings]

[0044] [Figure 1] FIG. 1 is a diagram showing an example of application of a head-up display device of an embodiment of the present invention to a vehicle, where FIG. 1(a) is a diagram cited to explain the relationship between the viewer's eye position in the height direction and the display area, and FIG. 1(b) is a diagram cited to explain the relationship between a display area (virtual image formation surface) virtually set in front of the vehicle and the angle of view. [Diagram 2] FIG. 2 is a block diagram showing a configuration of a vehicle control system to which the head-up display device according to the embodiment of the present invention is applied. [Diagram 3] FIG. 3 is a diagram showing a cross-sectional structure of a head-up display device according to an embodiment of the present invention, in which FIG. 3(a) shows the case where the eye box is in a reference position, and FIG. 3(b) shows the case where a concave mirror, which is a reflective member, is rotated counterclockwise. [Figure 4] FIG. 4 is a flowchart showing the operation of the head-up display device according to the embodiment of the present invention. [Diagram 5]FIG. 5 is a diagram showing how an image appears (one example of a display form) when viewed from the eye position in the standard height direction of a viewer in the head-up display device according to the embodiment of the present invention. [Figure 6] FIG. 6 is a diagram showing how an image appears (one example of a display form) when the position of the viewer's eyes in the height direction changes in the head-up display device according to the embodiment of the present invention. [Figure 7] FIG. 7 is a diagram showing an example of the display form when the display mode is changed in the head-up display device of this embodiment, where FIG. 7(a) shows the display form when the decoration of the virtual image is simplified, and FIG. 7(b) shows the display form when the virtual image is rearranged. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0045] The best mode described below is used to easily understand the present invention. Therefore, those skilled in the art should be aware that the present invention is not unduly limited by the mode described below (hereinafter, referred to as the present mode).

[0046] A vehicle control system 1000 to which the head-up display device of the present embodiment is applied will be described below with reference to the accompanying drawings.

[0047] (Configuration of the embodiment) FIG. 1 is a diagram showing an example of application of a head-up display device (hereinafter, referred to as HUD device 100) of this embodiment to a vehicle, in which FIG. 1(a) is a diagram cited to explain the relationship between the viewer's eye position in the height direction and the display area, and FIG. 1(b) is a diagram cited to explain the relationship between a display area (virtual image formation surface) virtually set in front of the vehicle and the angle of view.

[0048] As shown in Fig. 1(a), the projection display of an image by the HUD device 100 involves reflecting display light L projected by the HUD device 100 provided inside the instrument panel 11 of the vehicle 10 in the direction of a viewer 14 of the vehicle 10 (a driver seated in the driver's seat of the vehicle 10) by the windshield 13 of the vehicle 10 to display an image (virtual image V). In other words, the HUD device 100 projects the display light L emitted from a display unit (120 in Fig. 2, which will be described later) onto the windshield 13, which is a member to be projected, and allows the viewer 14 to view the display light L in response to this projection. This allows the viewer 14 to view the image (virtual image V) superimposed on the scenery ahead of the vehicle 10.

[0049] In this case, the HUD device 100 projects display light L such that the image (virtual image V) is tilted at an angle of 45° or less (0° to 45°) with respect to the road surface on which the vehicle 10 is traveling, and the upper part of the image (virtual image V) appears to be tilted farther away from the viewer 14 than the lower part. This allows the viewer 14 to view the image (virtual image V) that is displayed mainly superimposed on the road surface ahead of the vehicle 10, and obtain various pieces of information indicated by the image (virtual image V).

[0050] Incidentally, a field angle AV is set in the HUD device 100. If a virtual range in space where the HUD device 100 can form an image (virtual image V) is defined as a virtual image formation surface (display area VA), the field angle AV is a field angle defined based on a virtual line connecting the eye point EP of the viewer 14 (the center of the eye box, which is a field angle defined based on a virtual line connecting the eye of the viewer 14 and the outer edge of the display area VA) and the outer edge of the display area VA. The field angle AV is an angle range in which the viewer 14 can view the image (virtual image V) as viewed from the eye point EP.

[0051] As shown in FIG. 1(b), the HUD device 100 is set so that the horizontal angle of view in the horizontal direction (e.g., about 10° to 12°) is larger than the vertical angle of view in the vertical direction (e.g., about 4° to 5°). When viewed from the eye point EP of the viewer 14, a forward range overlapping with a display area VA, which is a virtual image formation surface, is within the range of the angle of view AV. The display area VA corresponds to a display surface 120a (see FIGS. 3(a) and 3(b)) of the HUD device 100, which will be described later (in other words, the display area VA is in a conjugate relationship with the display surface 120a of the display unit 120, which will be described later), and an image (virtual image V) viewed in the display area VA corresponds to an image projected and displayed on the display surface 120a of the HUD device 100.

[0052] A display area VA is set on the display surface 120a of the display unit 120, and an image (virtual image V) is displayed within this display area VA. The image (virtual image V) includes virtual images such as a perspective image VP drawn by linear perspective that imitates information related to the foreground objects, such as a speed limit sign, a vehicle ahead (not shown), and a driving lane, in addition to foreground objects such as a real scene ahead of the vehicle 10. The "linear perspective" is a projection method that expresses a perspective in which an object appears smaller as it moves away from the front side to the back side, using lines, and is typically a perspective representation method using the convergence of parallel lines. In FIG. 1(b), for example, a virtual vanishing point (not shown) is set on the horizon, and parallel lines extending from the front side to the back side are drawn so as to converge to the vanishing point at infinity in principle.

[0053] In Fig. 1(b), the vertical direction of the display unit 120 (see Fig. 2) is the vertical angle of view, and the horizontal direction is the horizontal angle of view. In the display area VA, the area between the upper end PU and the lower end PL of the angle of view AV is divided into two equal parts by a boundary line (a dashed line in the figure), and the lower side of the area thus divided (the side that appears closer to the viewer 14) can be defined as the "near side," and the upper side (the side that appears farther from the viewer 14) can be defined as the "deep side." Note that the boundary line is merely an example and is not limited to this.

[0054] FIG. 2 is a block diagram showing an example of the configuration of a vehicle control system 1000 to which the HUD device 100 of the present embodiment is applied.

[0055] 2, the vehicle control system 1000 includes a HUD device 100, an in-vehicle sensing device 200, a driving assistance device 300, a navigation device 400, an operation input device 500, and an eye position detection unit 600. All of these enable bidirectional information transmission via a CAN (Control Area Network) 700, which is an in-vehicle LAN.

[0056] The on-vehicle sensing device 200 is a type of sensor required for recognizing the surrounding driving environment including the front of the vehicle 10, and includes a camera, LiDAR (Light Detection And Ranging), etc. It also includes an IMU (Inertial Measurement Unit) and a vehicle speed sensor that detect the behavior of the vehicle 10. Information recognized or detected by the on-vehicle sensing device 200 is transferred via the CAN 700 to the driving assistance device 300, the navigation device 400, and the HUD device 100 of this embodiment.

[0057] The camera captures at least the forward field of view (actual view) of the vehicle 10, and the LiDAR uses near-infrared light, visible light, and ultraviolet light to, for example, illuminate an obstacle or the like present in front of the vehicle 10 captured by the camera, capture the reflected light with an optical sensor, and determine the distance to the obstacle based on the time difference. The IMU uses a three-axis acceleration sensor and a three-axis angular velocity sensor (gyro sensor) to measure the driving situation and attitude of the vehicle (acceleration [m / s 2 ] to detect translational motion in three axes, and angular velocity [deg / s] to detect rotational motion.

[0058] The driving assistance device 300 has at least one of a driving assistance function using an ADAS (Advanced Driver-Assistance System) such as an LKAS (Lane Keeping Assist System) or an ACC (Adaptive Cruise Control System) that assists the viewer 14 in driving, and an automatic driving function that can act on behalf of the driver. The driving assistance device 300 recognizes or detects the driving environment and behavior in front of the vehicle 10 based on information acquired from the in-vehicle sensing device 200, and controls the vehicle 10 (the drive system, braking system, and steering system, not shown) based on the analysis result of the information obtained here, and can transfer the analysis result of the information to the HUD device 100, the navigation device 400, etc. via the CAN 700.

[0059] The navigation device 400 uses a Global Navigation Satellite System (GNSS) such as a Global Positioning System (GPS) and a gyro sensor (sensor 402 such as GPS), and an information processing unit 401 obtains map information from a map DB 404 or by wireless communication with the outside of the vehicle, and provides guidance to facilities in the vicinity of the vehicle 10 and route guidance, etc. The navigation device 400 transfers a signal to the HUD device 100 via the CAN 700 at an appropriate timing to prompt display output based on the guidance information.

[0060] The information processing unit 401 performs information processing such as searching for and guiding destinations and facilities based on the position information of the vehicle acquired from the in-vehicle sensing device 200, the driving support device 300, the operation input device 500, and built-in sensors such as a GPS and a gyro sensor (sensor 402 such as GPS). Then, based on the result of the information processing, for example, when it is detected that the vehicle is approaching an intersection where the vehicle should turn, the information processing unit 401 generates a virtual image V such as an arrow indicating to turn right or left at the intersection and transfers it to the HUD device 100. The information processing unit 401 also outputs a drive signal via the CAN 700 to control the rotation of a concave mirror 141 (see FIGS. 3(a) and (b)), which is a reflecting member of the HUD device 100 described later, in order to control the position of the above-mentioned eye box.

[0061] When the height direction eye position (eye point EP) of the viewer 14, which is set or detected by an operation input of the viewer 14, is changed, the information processing unit 401 outputs a command to the HUD device 100 to move the eye box. The information processing unit 401 can move the eye box by changing the angle and / or position of the relay optical system (e.g., the concave mirror 141) of the HUD device 100 according to the command (controlling the driving unit 110). Specifically, for example, if the eye point EP of the viewer 14 detected by the eye position detection unit 600 is relatively low, the information processing unit 401 moves the eye box downward. In a typical HUD device 100, when the eye box is lowered, the display area VA rises in accordance with the amount by which the eye box is lowered (the movement amount of the eye box and the movement amount of the display area VA are typically proportional). According to the control to lower the eyebox (control to raise the display area VA) in this way, the angle (depression angle) between a line parallel to the road surface based on the eye position of the viewer 14 in the height direction and a line connecting the eye position of the viewer 14 in the height direction and the center of the display area VA becomes smaller. Then, upon receiving the above-mentioned command (a command to adjust the height of the eyebox, or from another perspective, a command to adjust the height of the display area), the HUD device 100 performs control to change the display mode of the virtual image V and display it on the display unit 120 when the angle (depression angle) between a line parallel to the road surface based on the eye position of the viewer 14 in the height direction and a line connecting the eye position of the viewer 14 in the height direction and the center of the display area VA is equal to or smaller than a predetermined threshold value.

[0062] Here, the "position of the viewer's eyes in the height direction" refers to the eye point EP of the viewer 14, for example, as shown in FIG. 1(a), and can be detected by the eye position detection unit 600. The virtual image displayed by the HUD device 100 is correctly viewed in a predetermined limited area (hereinafter, also referred to as the eye box) (for example, as in (1) and (2) below). Therefore, since the eye point EP of the viewer 14 can be estimated to be within the eye box, information on the "position of the eye box" can be substituted for information on the "position of the eye box". Since the "position of the eye box" and the position of the display area are in a corresponding relationship (when the eye box moves, the display area also moves accordingly), in some embodiments, it can also be determined by the "position of the eye box" whether or not the angle formed by the straight line connecting the viewer's eye position in the height direction and the center of the display area is equal to or smaller than a predetermined threshold value.

[0063] The term "eye box" used in the description of the embodiment refers to (1) an area in which at least a part of the virtual image V of the image displayed by the display unit 120 can be seen, and outside the area, no part of the virtual image V of the image displayed by the display unit 120 can be seen, or (2) an area in which at least a part of the virtual image V of the image displayed by the display unit 120 can be seen with a predetermined luminance or more, and outside the area, the entire virtual image V of the image displayed by the display unit 120 is less than the predetermined luminance. That is, when the viewer 14 places his / her eyes (both eyes) outside the eye box, the viewer 14 cannot see the entire virtual image V of the image displayed by the display unit 120, or the visibility of the entire virtual image V of the image displayed by the display unit 120 is very low and difficult to perceive. The predetermined luminance is, for example, about 1 / 50 of the luminance of the virtual image V seen at the center of the eye box.

[0064] Furthermore, the "predetermined threshold" refers to, for example, the angular position of the concave mirror 141 (see FIGS. 3(a) and (b)), which is a reflecting member, when the eyebox is in a reference position. Furthermore, since the "position of the eyebox" is determined by the direction of the display light L emitted from the HUD device 100 (the position projected onto the projection target member), the position of the eyebox changes depending on the angle and / or position of a relay optical system (e.g., a concave mirror) within the HUD device 100 that determines the direction of the display light L emitted from the HUD device 100. Therefore, information regarding the "position of the eyebox" (information regarding the position of the viewer's eyes in the height direction) can be replaced with information regarding the angle and / or position of a relay optical system (e.g., a concave mirror) or the like.

[0065] In some embodiments, the determination of whether the angle between the viewer's eye position in the height direction and the line connecting the center of the display area is equal to or smaller than a predetermined threshold value can also be determined by the "angle and / or position of a relay optical system (e.g., a concave mirror) or the like." The concave mirror 141 is connected to the driving unit 110, and the driving unit 110 can rotate the concave mirror 141 within a predetermined angle range according to the eye position in the height direction of the viewer 14 set by the viewer 14 or detected by the eye position detection unit 600 under the control of the control unit 160, thereby changing the arrival point of the display light L in the windshield 13. In other words, the position in the display area VA where the image (virtual image V) is arranged can be changed according to the change in the position of the eye box. In addition, the "position of the viewer's eyes in the height direction" can be determined, for example, from angle information of a reflective member (concave mirror 141 (an example of a relay optical system)) of the HUD device 100 set by the viewer 14, or can be determined by image recognition from photographic information using a camera (eye position detection unit 600) provided on the vehicle 10.

[0066] The storage unit 403 is a memory in which a program area and a work area are allocated, and in which, for example, a static RAM, a dynamic RAM, a flash memory, etc. are implemented. Here, in the program area, for example, various programs necessary for navigation, such as destination and facility search and guidance, are written, and in the work area, information generated in the execution process of the various programs described above and display information including images to be displayed on the HUD device 100 are allocated and temporarily written.

[0067] The storage unit 403 also stores a judgment threshold for judging whether or not a condition for changing the display mode of the virtual image is satisfied. When there are two types of display modes of the virtual image, there is one judgment threshold, but when there are three or more types of display modes of the virtual image, the storage unit 403 may store two or more judgment thresholds. In order to judge the condition for changing the display mode of the virtual image, the storage unit 403 may previously store (1) one or more thresholds to be compared with an angle (depression angle) formed by a straight line parallel to the road surface based on the viewer's eye position in the height direction (eye point EP) and a straight line connecting the viewer's eye position in the height direction (eye point EP) and the center of the display area, (2) one or more thresholds to be compared with the viewer's eye position in the height direction (eye point EP), and (3) one or more thresholds to be compared with an angle and / or position of a relay optical system (for example, a concave mirror) or the like. Note that these thresholds may be changed during the manufacture of the HUD device 100, when the HUD device 100 is attached to a vehicle, or by calibration at a timing different from these.

[0068] The navigation device 400 has a map database (map DB 404). The map DB 404 can acquire and store the latest map information by, for example, communication with an external center (not shown) via a V2X (Vehicle to X) type communication system (not shown). Here, the map information stored in the map DB 404 is mapping data that is digitized to represent the driving environment of the vehicle 10. The mapping data is preferably digital data of a particularly high-precision dynamic map. A "dynamic map" is a digital map that combines a huge amount of dynamic information that changes from moment to moment, such as traffic regulations, construction information, accidents, congestion, pedestrians, and traffic lights information, with static information such as high-precision three-dimensional position information (road surface information, diagonal line information, three-dimensional structures).

[0069] The operation input device 500 is, for example, switches provided near a steering wheel (steering handle). The operation input device 500 allows the viewer 14 to operate these switches to switch the HUD device 100 ON / OFF, select an execution function of the driving support device 300 that executes ADAS such as LKAS and ACC, set the air volume of the air conditioner, and further set setting information (position of the eye box) that instructs the rotation operation of a reflecting member (concave mirror 141, which is an example of a relay optical system) by the drive unit 110 (see FIGS. 3(a) and (b)) of the HUD device 100.

[0070] The eye position detection unit 600 is configured with a camera that captures an image of the face of the viewer 14 and outputs the captured image, and is capable of detecting (determining) the eye position in the height direction of the viewer 14 by performing image processing on the captured image. As described above, the eye position in the height direction of the viewer can also be determined from angle information of the reflecting member (concave mirror 141, which is an example of a relay optical system) of the HUD device 100 that is set by the viewer 14.

[0071] The HUD device 100 includes a drive unit 110, a display unit 120, and a control unit 160.

[0072] The driving unit 110 is connected to a concave mirror 141 (see FIGS. 3(a) and (b)), which is a reflecting member, and can rotate the concave mirror 141 to a predetermined angular position based on a control signal generated by the control unit 160. The rotational position of the concave mirror 141 in the HUD device 100 has several steps, and can be adjusted within this range. The concave mirror 141 uses a stepping motor for its rotational drive, and can be rotated a predetermined number of steps corresponding to the position of the eye box (setting information) set by operating the operation input device 500.

[0073] The driving unit 110 controls the fixed angle position of the concave mirror (one example of a relay optical system) 141 by a rotation driving mechanism using a stepping motor, and therefore, the angle range set for the concave mirror 141 can be set by a predetermined number of steps in the clockwise direction and counterclockwise direction around a reference position. The driving unit 110 may change the position of the relay optical system, or may change the angle and position.

[0074] The display unit 120 is mainly composed of a light source 121 consisting of a light emitting diode mounted on a wiring board, and a TFT (Thin Film Transistor Liquid Crystal) type liquid crystal display element 122 located on the emission side (directly above) of the light source 121 so as to transmit illumination light from the light source 121 to form display light L. This means that the light source 121 is provided behind the liquid crystal display element 122, and the liquid crystal display element 122 displays a predetermined image by the light emitted from the light source 121.

[0075] The display unit 120 has a display surface 120a (see FIGS. 3(a) and (b)) disposed at an angle to the optical axis, and outputs display light L consisting of light in the visible wavelength range. For example, a light source 121 that emits white light can be applied, and the light emitted from this light source 121 is transmitted through a liquid crystal display element 122 to output display light L including a desired image. The liquid crystal display element 122 forms a desired image based on display image data (driving signal) generated by drawing calculation by the control unit 160.

[0076] The control unit 160 can display on the display unit 120 (liquid crystal display element 122) images to be displayed based on vehicle information transferred from the driving support device 300 or the navigation device 400, such as guidance images for lanes and the like, as well as the speed of the vehicle 10, remaining energy, time, route guidance images, and the like, using image data such as numerical values ​​and icons. Note that the images to be displayed are not limited to vehicle information such as the vehicle speed and remaining energy, and any display form can be adopted.

[0077] The control unit 160 outputs a drive signal to the liquid crystal display element 122 to prompt the display to be viewed by the viewer 14, based on image information generated and transferred by, for example, the information processing unit 401 of the navigation device 400 or the driving support device 300. The control unit 160 is also connected to a light source 121 that transmits and illuminates the liquid crystal display element 122, and can turn the liquid crystal display element 122 on and off or adjust the brightness by outputting a drive signal.

[0078] The control unit 160 can control the display position of the image (virtual image V) to be viewed, for example, according to setting information related to the position of the eyebox acquired by the viewer 14 operating the operation input unit 500. Specifically, the control unit 160 rotates and adjusts the angle of the reflecting member (concave mirror 141) adjusted by the driving unit 110, according to the setting information related to the position of the eyebox. Then, if the angle differs from the reference position, the control unit 160 performs control to change the display position of the image (virtual image V) in the display area VA according to the adjusted angle.

[0079] Here, the "reference position" refers to the angular position of the concave mirror 141 when the eye box is at the reference position. The concave mirror 141 is connected to the driving unit 110, and the driving unit 110 can rotate the concave mirror 141 within a predetermined angle range according to the eye position (eye point EP) of the viewer 14 in the height direction set by or detected by the viewer 14 under the control of the control unit 160, thereby changing the arrival point of the display light L in the windshield 13. In other words, the position in the display area VA where the image is arranged can be changed according to the change in the position of the eye box (the eye position in the height direction of the viewer 14).

[0080] The control unit 160 controls the display unit 120 to change the display mode of the virtual image V when the angle (depression angle) between a line parallel to the road surface based on the eye position (eye point EP) of the viewer 14 in the height direction and a line connecting the eye position (eye point EP) of the viewer 14 in the height direction and the center of the display area VA is less than a predetermined threshold value.

[0081] Here, the "position of the viewer's eyes in the height direction" refers to the viewer's eye point EP, for example, as shown in FIG. 1(a), and can be detected by an eye position detection unit. In another embodiment, the "position of the viewer's eyes in the height direction (approximate position)" can be estimated from the position of the eye box. Since the eye point EP of the viewer 14 can be estimated to be within the eye box, information on the "position of the eye box" can be substituted for information on the "position of the eye box". Since the "position of the eye box" and the position of the display area VA are in a corresponding relationship (when the eye box moves, the display area VA also moves accordingly), in some embodiments, the determination of whether the angle (depression angle) formed by the straight line connecting the position of the viewer's eyes in the height direction (eye point EP) and the center of the display area VA is equal to or smaller than a predetermined threshold value can also be determined by the "position of the eye box".

[0082] Furthermore, since the "position of the eyebox" is determined by the direction of the display light L emitted from the HUD device 100 (the position projected onto the projection target), the position of the eyebox changes depending on the angle and / or position of the relay optical system (e.g., the concave mirror 141) in the HUD device 100 that determines the direction of the display light L emitted from the HUD device 100. Therefore, information on the "position of the eyebox" (information on the eye position (eye point EP) of the viewer 14 in the height direction) can be replaced with information on the angle and / or position of the relay optical system (e.g., the concave mirror 141). In some embodiments, the determination of whether the angle formed by the straight line connecting the eye position (eye point EP) of the viewer 14 in the height direction and the center of the display area VA is equal to or smaller than a predetermined threshold value can also be determined by the "angle and / or position of the relay optical system (e.g., the concave mirror 141)". The concave mirror 141 is connected to the driving unit 110, and the driving unit 110, under the control of the control unit 160, can rotate the concave mirror 141 within a predetermined angle range based on the eye position (eye point EP) of the viewer 14 in the height direction set by the viewer 14 or detected by the eye position detection unit 600, thereby changing the arrival point of the display light L within the windshield 13. In other words, the position within the display area VA where the image (virtual image V) is placed can be changed with the change in the position of the eye box.

[0083] When the control unit 160 changes the display mode, the control unit 160 can control the display unit 120 to change the display mode by reducing the total number of virtual images V that can be arranged in a direction that includes at least the vertical component of the display area VA. Since the expressive power of the perspective is weakened by reducing the total number of virtual images V that can be arranged, the control unit 160 can also control the display unit 120 to change the relative size of the virtual images V that give the viewer 14 the perception of perspective, or to change at least one of the size of the virtual images V, the decoration of the virtual images V, and the arrangement of the virtual images V.

[0084] Here, the "vertical component of the display area" refers to the vertical angle of view having a vertical component of the angle of view AV within the angle range in which the viewer 14 can view an image (virtual image V) as seen from the eye point EP, as shown in the relationship between the display area VA and the angle of view AV in Fig. 1(b). Also, the "total number of virtual images that can be arranged" refers to the maximum number of virtual images that can be arranged in a direction that includes at least the vertical component of the display area, and is determined by, for example, the size of the virtual images V to be arranged, the interval between the virtual images V, and further the size of the vertical component of the display area (drawable area).

[0085] In addition, the control unit 160 can set a vanishing point in linear perspective inside or outside the display area VA (angle of view AV) and perform control to position the virtual image V within a specified range visible to the viewer 14 so that it is positioned along a perspective image generated by emphasizing the sense of perspective in an image that mimics a driving lane passing through the vanishing point.

[0086] Here, the "vanishing point" refers to the point where two or more lines that are originally parallel in linear perspective intersect when they are drawn as non-parallel, and refers to the line of sight of the viewer when looking through to the distance. The "predetermined range" is a value determined based on a threshold value indicating whether the viewer can see each image as if it is arranged along the road surface, and this is also a value obtained by testing, simulation, etc.

[0087] For example, as shown in an example of a display form in FIG. 6, the inclination of the perspective image VP simulating a driving lane is 40°, and the two vehicle speed limit sign images (VT1, VT2) are also arranged so that their inclination is 40°. If the difference between these inclinations becomes large, there is a possibility that each image (VT1, VT2) will be viewed as not being aligned with the road surface. In particular, if each image (VT1, VT2) is arranged so that it is vertical, that is, so that the inclination is 90°, the viewer 14 will not be able to view each image (VT1, VT2) as being aligned with the road surface. Therefore, the "predetermined range" is set to a value (inclination) that does not allow such an arrangement, and for example, the range is set to be equal to or greater than 8° and less than 66°.

[0088] For example, as shown in Fig. 6, a vanishing point (not shown) is set within or outside the viewing angle AV of an image (virtual image V), and speed limit sign images (virtual images VT1, VT2) are arranged on an image (perspective image VP) in which the perspective is emphasized by imitating the driving lanes toward the vanishing point, so that the viewer 14 can visually recognize these images (virtual images VT1, VT2) as if they are displayed along the road surface. In addition, the front-to-back relationship between the virtual images VT1 and VT2 can be expressed by arranging the speed limit sign image (virtual image VT1) of the currently traveling road on the lower side (the side that is perceived as the near side as seen by the viewer) on the near side of the display area (imaging surface) as seen by the viewer 14, and the speed limit sign image (virtual image VT2) of the nearest road on the upper side (the side that is perceived as the far side as seen by the viewer) on the near side of the display area (imaging surface).

[0089] In addition, the control unit 160 can perform display with enhanced expressiveness in the depth direction (forward / backward direction of the vehicle 10) by using the display area VA, which is an inclined imaging surface such that the upper end PU (see FIG. 1(b)) of the display area VA, which is virtually set in front of the vehicle 10, is deeper than the lower end PL (see FIG. 1(b)), thereby achieving more natural visibility of images with a sense of perspective.

[0090] FIG. 3 shows a schematic cross-sectional view of the internal structure of the HUD device 100, where FIG. 3(a) shows the case where the eyebox is in the reference position, and FIG. 3(b) shows the case where the concave mirror, which is the reflective member, is rotated counterclockwise.

[0091] As shown in FIGS. 3( a ) and 3 ( b ), the HUD device 100 is mainly composed of a display unit 120 , a first reflector 130 , a second reflector 140 , and a housing 150 .

[0092] The display unit 120 has a display surface 120a and outputs display light L consisting of light in the visible wavelength range, and can be, for example, a light source 121 that emits white light. The light emitted from the light source 121 is transmitted through a liquid crystal display element 122 to output display light L including a desired image. The liquid crystal display element 122 forms the desired image based on display image data (driving signal) generated by drawing calculation by the control unit 160.

[0093] The first reflector 130 includes a cold mirror 131. The cold mirror 131 reflects light in the visible wavelength range (450 to 750 nm) including the emission wavelength range of the display unit 120 with a high reflectance of, for example, 80% or more, and reflects light outside the visible wavelength range with a low reflectance. In this case, the cold mirror 131 is adapted to reflect light outside the visible wavelength range, particularly light in the infrared wavelength range (heat rays of external light such as sunlight), with a low reflectance of, for example, 15% or less. The cold mirror 131 is configured so that light that is not reflected by the reflective layer thereof passes through the cold mirror 131. The cold mirror 131 can prevent the liquid crystal display element 122 from being heated by sunlight or the like and becoming too hot.

[0094] The second reflector 140 includes a concave mirror 141 which is a reflecting member that reflects the display light L from the cold mirror 131 (that is, the liquid crystal display element 122), and a mirror holder 142 which holds the concave mirror 141.

[0095] The concave mirror 141 is formed by depositing a second reflective layer 141a on a resin substrate made of polycarbonate having a concave surface. The concave mirror 141 is provided at an angle such that the second reflective layer 141a faces the cold mirror 131 and the light-transmitting cover 153 and can be seen from the light-transmitting cover 153.

[0096] The concave mirror 141 reflects (projects) the display light L from the cold mirror 131 toward the light-transmitting cover 153 (the windshield 13 of the vehicle 10) while magnifying the light. This means that the concave mirror 141 magnifies the display light L reflected by the cold mirror 131, and projects the magnified display light L onto the windshield 13 through the light-transmitting cover 153. The concave mirror 141 is adhered to the mirror holder 142 by a double-sided adhesive member. The mirror holder 142 is made of a synthetic resin such as ABS (Acrylonitrile Butadiene Styrene), and is fixed to the housing 150.

[0097] The concave mirror 141 is connected to the drive unit 110 and can be rotated within a predetermined angle range depending on the situation. This makes it possible to change the point where the display light L reaches within the windshield 13. In other words, the position of the eye box of the HUD device 100 changes. This makes it possible to change the eye position in the height direction (eye point EP) at which the viewer 14 can view the image, and the display position within the display area VA where the image is displayed.

[0098] Here, the lower the eyebox position, the higher the display position of the image (virtual image V) in the display area VA. This means that the depression angle, which is the angle between a line parallel to the road surface based on the eye position (eyepoint EP) of the viewer 14 in the height direction and a line connecting the eye position (eyepoint EP) of the viewer 14 in the height direction and the center of the display area VA, changes in a smaller direction. On the other hand, the higher the eyebox position, the lower the display position of the image in the display area VA. This means that the depression angle, which is the angle between a line parallel to the road surface based on the eye position (eyepoint EP) of the viewer 14 in the height direction and a line connecting the eye position (eyepoint EP) of the viewer 14 in the height direction and the center of the display area VA, changes in a larger direction.

[0099] In other words, when concave mirror 141, which is in the reference position shown in Fig. 3(a), rotates clockwise, the position of the eyebox becomes lower as shown in Fig. 3(b), and conversely, when it rotates counterclockwise, the position of the eyebox becomes higher. Note that since the depression angle is uniquely determined by a predetermined formula depending on the position of the eyebox, calculating the display position of the image (virtual image V) at the position of the eyebox is equivalent to calculating the display position of the image (virtual image V) in the display area VA using the depression angle.

[0100] The housing 150 is made of, for example, a black light-blocking synthetic resin material, is formed in a roughly box-like shape, and holds and houses the projection unit consisting of the display unit 120, the first reflector 130, and the second reflector 140 in its internal space, the space portion 151, and is provided with an opening window portion 152 that opens at the upper part (windshield 13 side) of the concave mirror 141 in the second reflector 140.

[0101] Further, the housing 150 is provided with a light-transmitting cover 153, which is an emission part, so as to cover the open window part 152. This light-transmitting cover 153 is made of a light-transmitting synthetic resin material (e.g., acrylic resin), is formed in a curved shape (curved surface shape), and functions as a light-transmitting member through which the display light L reflected by the concave mirror 141 passes (passes through). That is, the display light L reflected by the cold mirror 131 and the concave mirror 141, i.e., the display light L output by the projection part, is projected onto the windshield 13 through the light-transmitting cover 153 formed on the housing 150, thereby displaying an image.

[0102] The HUD device 100 projects and outputs the display light L (image) onto the lower side of the windshield 13. Therefore, the viewer 14 can see the image superimposed on the foreground object of the vehicle 10 from the lower area of ​​the windshield 13. Therefore, the viewer 14 mainly sees an image (virtual image V) displayed on the road surface near the vehicle 10. In addition, due to the configuration of the projection unit of the HUD device 100, an image is output that appears to be inclined so that the upper part of the output image is located further back from the vehicle 10 than the lower part.

[0103] (Operation of the embodiment) Fig. 4 is a flowchart showing the operation of the HUD device 100 of this embodiment. Fig. 5 is a diagram showing how an image appears (one example of a display form) when viewed from the standard eye position (eye point EP) of the viewer 14 in the height direction in the HUD device 100 of this embodiment, and Fig. 6 is a diagram showing how an image appears (one example of a display form) when the eye position (eye point EP) of the viewer 14 in the height direction changes (when the eye point EP becomes lower) in the HUD device 100 of the embodiment of the present invention.

[0104] Hereinafter, the operation of the HUD device 100 of the present embodiment shown in FIG. 2 will be described with reference to FIGS.

[0105] First, the information processing unit 401 of the navigation device 400 generates an image (virtual image V) to be displayed on the HUD device 100 of this embodiment (step S101), and outputs a display command together with the generated image (virtual image V) to the HUD device 100. Here, the generated image (virtual image V) is, for example, information on a foreground object that is a real scene ahead of the vehicle 10, that is, a speed limit sign image (for example, speed limit sign images VT1, VT2, VT3 in FIG. 5), a perspective image VP obtained by performing image processing that emphasizes the perspective on an image that mimics a driving lane, an icon VM1 that indicates vehicle information such as vehicle speed and engine RPM, or an image that includes navigation information not shown.

[0106] The vehicle speed limit sign images VT1, VT2, and VT3 may be generated by the driving support device 300. The navigation information is information including surrounding facility guides and route guides generated by the navigation device 400 (information processing unit 401).

[0107] The HUD device 100 (control unit 160) of this embodiment, which has received a display command from the navigation device 400 (information processing unit 401), first determines whether or not there has been a change in the eye position (eye point EP) of the viewer 14 in the height direction (step S102). Here, the change in the eye position (eye point EP) of the viewer 14 in the height direction is determined from angle information of the concave mirror 141, which is a reflective member of the HUD device 100, which is set by operating the operation input device 500, or from information obtained by performing image recognition on photographic information acquired from a camera (on-vehicle sensing device 200) provided in the vehicle 10.

[0108] Here, if there is no change in the eye position (eye point EP) of the viewer 14 in the height direction (step S102 "NO"), the control unit 160 executes control to draw an image (virtual image V) generated by the navigation device 400 (information processing unit 401) in the display area VA (step S103 "drawing A of virtual image V"). Specifically, the control unit 160 executes drawing calculation to determine and draw a display position so that a virtual line (perspective image VP) simulating a driving lane has the same inclination as a driving lane in the real world and the speed limit sign images VT1, VT2, and VT3, which are the virtual image V, are positioned within a predetermined range from the virtual line (perspective image VP), and executes control to write the obtained display data to a VRAM (Video RAM) area (built in the control unit 160) having at least the same storage capacity as the display area VA.

[0109] When executing the above-mentioned drawing calculation, the control unit 160 draws the above-mentioned virtual images V (VP, VT1, VT2, VT3) in the display area VA, and also draws (writes into the built-in VRAM) images to be superimposed on the foreground, which is the scenery ahead of the vehicle 10, at desired positions, such as alphanumeric characters VS1 indicating the speed of the vehicle 10, an icon MV1 indicating the vehicle status such as the number of revolutions, or remaining energy, time, route guidance images, etc. (not shown). Then, after waiting for the display timing (step S104 "YES"), the control unit 160 reads the drawn image data from the VRAM and supplies it to the liquid crystal display element 122 of the display unit 120 to perform the desired display (step S105).

[0110] 5 is a diagram showing how an image is viewed from the standard eye position (eye point EP) of the viewer 14 in the height direction in the HUD device 100 of this embodiment (i.e., how an image is viewed by a viewer 14 with a standard height). According to FIG. 5, the vehicle 10, for example, in accordance with guidance from the navigation device 400, first travels along a road where a 80 km / h vehicle speed limit sign VT1 is applied, then travels along a road where a 60 km / h vehicle speed limit sign VT2 is applied in the near future, and travels along a road where a 80 km / h vehicle speed limit sign VT3 is applied in the near future. As a result, each vehicle speed limit sign image (virtual images VT1, VT2, VT3) is generated and displayed on the display unit 120 of the HUD device 100. Here, it is preferable that the three vehicle speed limit sign images VT1, VT2, VT3 are arranged along the road surface, and the images are drawn and displayed so that the context between the images can be understood.

[0111] According to an example of the display form shown in Fig. 5, a vanishing point in linear perspective is set inside or outside the display area VA, and three speed limit sign images VT1, VT2, and VT3 are arranged on a perspective image VP, which is a virtual line that imitates the driving lane toward the vanishing point. Here, the vanishing point is set so that the perspective image VP toward the vanishing point has the same inclination (rise angle) as the driving lane in the real world. This allows the viewer 14 to visually recognize each of the speed limit sign images VT1, VT2, and VT3 as if they are arranged along the road surface.

[0112] Although the speed limit sign images VT1, VT2, VT3 may be positioned at a position offset from the perspective image VP, if they are significantly offset, the speed limit sign images VT1, VT2, VT3 may be perceived as having a low relevance to the road surface, and may be perceived as not conforming to the road surface. Therefore, the speed limit sign images VT1, VT2, VT3 must be positioned within a predetermined range from the perspective image VP.

[0113] Here, the "predetermined range" is a value determined based on a threshold value of whether the viewer 14 can visually recognize each image as being arranged along the road surface, and is set by testing, simulation, etc. For example, the inclination of the driving lane in the perspective image VP in FIG. 5 in the real world is 50°, and the three vehicle speed limit sign images VT1, VT2, and VT3 are also arranged at an inclination of 50°. However, if the three vehicle speed limit sign images VT1, VT2, and VT3 are arranged vertically, that is, at an inclination of 90°, the viewer 14 cannot visually recognize each of the vehicle speed limit sign images VT1, VT2, and VT3 as being arranged along the road surface. Therefore, the predetermined range is set to a value (inclination) that does not allow such an arrangement, and is set to, for example, 8° or more and less than 66°. In this way, by using a representation method in which a virtual line (perspective image VP) that resembles a driving lane is generated toward a vanishing point and each speed limit sign image VT1, VT2, VT3 is displayed arranged along the perspective image VP, the viewer 14 can visually recognize each speed limit sign image VT1, VT2, VT3 as if it were arranged along the road surface.

[0114] In FIG. 5, the sign image VT1 of the nearby speed limit is arranged on the lower side (in front of the viewer 14), and the sign image VT3 of the distant speed limit is arranged on the upper side (in back of the viewer 14). It is known that human visual characteristics are such that a visual target on the upper side is visually recognized as being far away from the viewer 14, and a visual target on the lower side is visually recognized as being near. Therefore, the viewer 14 can visually recognize the sign image VT1 of the nearby speed limit as being displayed on the foreground and the sign image VT3 of the distant speed limit as being displayed on the background, which makes it easy to understand the anteroposterior relationship between the sign images of the respective speed limits. Furthermore, the HUD device 100 generates an inclined display area VA, so the display distance changes depending on the vertical position of the display area VA. Therefore, by providing a difference in the vertical position between the images, it becomes easier to understand the anteroposterior relationship between the images.

[0115] Returning to the flowchart of FIG. 4 for explanation. In step S102, if there is a change in the eye position (eye point EP) of the viewer 14 in the height direction (step S102 “YES”), the control unit 160 calculates the “depression angle” which is the angle between a line parallel to the road surface based on the eye position (eye point EP) of the viewer 14 in the height direction and a line connecting the eye position (eye point EP) of the viewer 14 in the height direction and the center of the display area VA (step S106). The depression angle is uniquely determined by a predetermined formula depending on the position of the eye box. Therefore, calculating the drawing position of the image (virtual image V: for example, the perspective image VP, the speed limit sign images VT1, VT2, VT3) at the position of the eye box is equivalent to calculating the drawing position of the image (virtual image V) in the display area VA using the depression angle.

[0116] Next, the control unit 160 judges whether the depression angle calculated in response to a change in the eye position (eye point EP) of the viewer 14 in the height direction is equal to or smaller than a threshold value (step S107). Here, the "threshold value" refers to, for example, the angular position of the reflecting member (concave mirror 141) when the eye box is in a reference position. The concave mirror 141 is connected to a drive unit, and the drive unit 110 can rotate the concave mirror 141 within a predetermined angle range based on the eye position (eye point EP) of the viewer 14 in the height direction set or detected by the viewer 14 under the control of the control unit 160, thereby changing the arrival point of the display light L in the windshield 13. In other words, the position in the display area VA where the image (virtual image V) is arranged can be changed in response to a change in the position of the eye box.

[0117] Here, if the depression angle is equal to or less than the threshold (step S107 "YES"), in other words, if the eye position (eye point EP) of the viewer 14 in the height direction becomes lower than the reference position and the angle (depression angle) between the eye position (eye point EP) of the viewer 14 in the height direction and the line connecting the eye position (eye point EP) of the viewer 14 in the height direction and the center of the display area VA becomes equal to or less than a predetermined threshold, the control unit 160 performs control to change the display mode of the image (virtual image V) and draw it (step S108 "drawing B of virtual image V"). When changing the display mode, the control unit 160 performs control to reduce the total number of virtual images V that can be arranged in a direction that includes at least the vertical component of the display area VA and draw them in the display area VA.

[0118] When performing the above-mentioned drawing, the control unit 160 not only arranges the above-mentioned images (virtual image V: for example, perspective image VP, speed limit sign images VT1, VT2, VT3) in the display area VA, but also draws (writes into a built-in VRAM area having at least the same capacity as the display area VA) images to be superimposed on the foreground, which is the scenery ahead of the vehicle 10, such as letters and numbers VS1 indicating the vehicle speed, and an icon MV1 indicating the vehicle status such as the number of revolutions, at desired positions. Then, after waiting for the display timing (step S104 "YES"), the drawn image data is read from the VRAM and supplied to the liquid crystal display element 122 of the display unit 120 to perform the desired display (step S105).

[0119] For example, Figure 6 shows an example of the display form of an image (virtual image V) displayed on the display unit 120 when the control unit 160 changes the display mode by reducing the total number of virtual images V that can be arranged in a direction that includes at least the vertical component of the display area VA and drawing them in the display area VA.

[0120] As shown in Fig. 6, when the eye position (eye point EP) of the viewer 14 in the height direction is lower than the reference position, the imageable area of ​​the display area VA is limited, and the vertical component (vertical angle of view AV) of the display area VA becomes narrower (VA → VB in Fig. 6). Therefore, the vertical component of the virtual image V arranged in the display area (imageable area VB) becomes shorter, and as a result, the expressive power of depth is weakened, as described above. In contrast, in the HUD device 100 of this embodiment, the control unit 160 controls to reduce the total number of virtual images V that can be placed in a direction that includes at least the vertical component of the drawing area VB (for example, from three speed limit sign images VT1, VT2, and VT3 to two speed limit sign images VT1 and VT2), making it easier to grasp the anteroposterior relationship of each image (virtual images V: speed limit sign images VT1 and VT2) without compromising the sense of perspective (without weakening the expressive power of depth) even if the viewer's 14 eye position in the vertical direction (eye point EP) is lower than the reference position and the drawing area is limited (the vertical component of the virtual images placed in the display area becomes shorter).

[0121] Here, the "vertical component of the display area" refers to the vertical angle of view having a vertical component of the angle of view AV within the angle range in which the viewer 14 can view an image (virtual image V) as seen from the eye point EP, as shown in the relationship between the display area VA and the angle of view AV in Fig. 1(b). Also, the "total number of virtual images that can be arranged" refers to the maximum number of virtual images that can be arranged in a direction that includes at least the vertical component of the display area, and is a value determined by, for example, the size of the virtual images to be arranged, the interval between the virtual images, and further the size of the vertical component of the display area (drawable area).

[0122] In addition, when arranging the image (virtual image V) in a direction including at least the vertical component of the display area VA, it is necessary to arrange two or more images (virtual images V) so that they can be viewed along the road surface and arranged at a certain interval. Here, the certain interval is, for example, a value determined by the difference between the imaging distance from the eye position (eye point EP) of the viewer 14 in the height direction to the bottom end of the virtual image V in the display area VA (the virtual image V1 arranged at the top among two or more images (virtual images V1, V2) arranged so that they can be viewed along the road surface) and the imaging distance from the eye position (eye point EP) of the viewer 14 in the height direction to the top end of the virtual image V (the virtual image V2 arranged below the virtual image V1), and is a value determined by an experiment or a simulation.

[0123] For example, in a HUD device 100 using a 4-8m imaging surface (wherein the imaging distance from the viewer's eye position in the height direction (eyepoint EP) to the top end of the display area VA is 8m, and the imaging distance from the viewer's eye position in the height direction (eyepoint EP) to the bottom end of the display area VA is 4m), it has been found that if the vertical component of the field of view AV, which indicates the distance (clearance) between the images (virtual images VT1, VT2), exceeds 0.8°, they cannot be viewed as the same information group, but that by setting it to a smaller value, they can be viewed as the same information group.

[0124] Whether the vertical component of the angle of view AV exceeds 0.8° can be used as a criterion for determining whether the images are viewed as the same information group or as independent information. Therefore, if it is desired that the viewer 14 view two or more virtual images V arranged along a perspective image VP simulating a traffic lane as the same information group, the virtual images need to be arranged so that the vertical component of the angle of view AV does not exceed 0.8° for the clearance between the virtual images, and if it is desired that the viewer 14 view two or more virtual images V along the perspective image VP as independent information, the virtual images need to be arranged so that the vertical component of the angle of view AV exceeds 0.8° for the clearance between the virtual images.

[0125] For this reason, in the HUD device 100 of this embodiment, the control unit 160 sets a vanishing point in linear perspective inside or outside the display area VA, and performs control to draw a number of virtual images V that allow the virtual images V to be arranged within a predetermined range (tilt angle of 8° or more and less than 66°) visible to the viewer 14 as if they are arranged along a perspective image VP that is generated by emphasizing the perspective of an image that mimics a driving lane passing through the vanishing point, and with a certain interval (clearance) between them so that they can be viewed as the same information group or as independent information.

[0126] For example, according to the display form shown in FIG. 6, even if the eye position (eye point EP) of the viewer 14 in the height direction becomes lower than the reference position and the drawing area of ​​the display area VA becomes narrower, the virtual images V1 and V2 are perceived as being arranged along the perspective image VP, and in order to maintain the distance (clearance) between the virtual images V1 and V2, an example is shown in which the total number of virtual images V that can be arranged in a direction that includes at least the vertical component of the display area VA is reduced from three (vehicle speed limit sign images VT1, VT2, VT3) shown in FIG. 5 to two (vehicle speed limit sign images VT1 and VT2).

[0127] FIG. 7 shows an example of a display when the display mode is changed in the HUD device 100 of this embodiment, where FIG. 7(a) shows the display form when the decoration of the virtual image V is simplified, and FIG. 7(b) shows the display form when the virtual image V is rearranged.

[0128] In the above-mentioned "drawing B of virtual image V" (step S108), the total number of virtual images V that can be arranged in a direction including at least the vertical component of the display area VA is reduced in order to change the display mode. However, in order to reduce the total number of virtual images V that can be arranged without reducing the sense of perspective, for example, a control may be performed to change the relative size of the vehicle speed limit sign image (VT1, VT2 in FIG. 6) displayed as the virtual image V. In addition, since the eye position (eye point EP) in the height direction of the viewer 14 becomes lower and the drawable area of ​​the display area VA becomes narrower (VA → VB), the character size may be changed, or, for example, as shown in FIG. 7(a), the display mode may be changed so that the number 80km / h indicating the vehicle speed limit is surrounded by a simple circle (right side of the arrow) by omitting the decorative double line or the control of the display attributes such as the sharpness and highlights to enhance the sense of perspective and simplifying the image.

[0129] Also, for example, as shown in Fig. 7(b), when the total number of virtual images V that can be arranged is reduced, the speed limit sign image VT1 (Fig. 6) displayed in the vicinity may not be displayed in the same position as before the change (Fig. 5), but may be repositioned to an arbitrary position that gives a sense of perspective, for example, by moving the speed limit sign image VT1 to the left so as to move it away from the center of the narrowed drawable area VB. As a result, in addition to the effect of perceiving a sense of perspective, the viewer 14 can easily grasp the front-to-back relationship of each of the displayed speed limit sign images (virtual images V1, V2 in Fig. 6).

[0130] In this way, when the control unit 160 changes the display mode, the total number of virtual images that can be arranged in a direction that includes at least the vertical component of the display area VA is reduced, and the display unit 120 displays the images. This makes it easier to grasp the relationship between the front and rear of each image without losing the sense of perspective (without weakening the ability to express depth) even if the eye position (eye point EP) of the viewer 14 in the height direction is lower than the reference position and the drawing area is limited (the vertical component of the virtual image arranged in the display area is shortened). In addition, at this time, the control unit 160 controls the display unit 120 to change at least one of the relative size of the virtual image V that gives the viewer 14 a sense of perspective, or the size of the virtual image V, the decoration of the virtual image V, and the arrangement of the virtual image V, and displays the images. This makes it easier to grasp the relationship between the front and rear of each image without losing the sense of perspective (without weakening the ability to express depth) even if the total number of virtual images that can be arranged is reduced.

[0131] Returning to the flow chart of Fig. 4, in step S107, if the calculated depression angle is equal to or greater than the threshold (step S107 "NO"), the control unit 160 performs control to change the total number of virtual images V that can be arranged along the perspective image VP in a direction that includes at least the vertical component of the display area VA from the display form shown in Fig. 6 to the display form shown in Fig. 5 (two vehicle speed limit sign images VT1 and VT2 → three vehicle speed limit sign images VT1, VT2, and VT3), or to increase the total number of virtual images V arranged along the perspective image VP in a direction that includes at least the vertical component of the display area VA (three or more virtual images V) and draw them (step S109 "drawing C of virtual image V").

[0132] When performing the above-mentioned drawing calculation, the control unit 160 not only arranges the above-mentioned virtual images V (VP, VT1, VT2, VT3) in the display area VA, but also draws (writes into a built-in VRAM having at least the same capacity as the display area VA) images to be superimposed on the foreground, which is the scenery ahead of the vehicle 10, such as letters or numbers VS1 indicating the vehicle speed, an icon MV1 indicating the vehicle status such as the number of revolutions, or an image of the remaining energy, time, route guidance, etc., not shown, at desired positions. Then, after waiting for the display timing (step S104 "YES"), the control unit 160 reads the drawn image data from the VRAM and supplies it to the liquid crystal display element 122 of the display unit 120 to perform the desired display (step S105).

[0133] (Modification) According to the HUD device 100 of the present embodiment described above, the information processing unit 401 of the navigation device 400 takes in setting information related to the position of the eye box set by the viewer 14 via the operation input device 500 and issues a display command to the HUD device 100. The HUD device 100, upon receiving this, changes the display mode of the virtual image V and controls the display on the display unit 120 when an angle (depression angle) formed between a straight line parallel to the road surface based on the eye position (eye point EP) of the viewer 14 in the height direction and a straight line connecting the eye position (eye point EP) of the viewer 14 in the height direction and the center of the display area VA is equal to or smaller than a predetermined threshold value, thereby making it easier to understand the anteroposterior relationship between the virtual images V. However, the functions of taking in the operation input and generating the image are not limited to the navigation device 400 (information processing unit 401) and may be replaced by the driving assistance device 300. Furthermore, the HUD device 100 may take in these functions independently of the navigation device 400 and the driving assistance device 300, and thereby perform these functions independently of the HUD device 100.

[0134] In addition, the HUD device 100 may be regarded as a display device that only controls image display, and the navigation device 400 (information processing unit 401) or the driving assistance device 300 may output a display command to instruct the HUD device 100 to change and display the display mode of the virtual image V when the angle (depression angle) between a straight line parallel to the road surface based on the eye position (eye point EP) of the viewer 14 in the height direction and a straight line connecting the eye position (eye point EP) of the viewer in the height direction and the center of the display area VA is equal to or smaller than a predetermined threshold value, and the HUD device 100 (control unit 160) that receives the display command may perform control to change the display mode of the virtual image V.

[0135] (Effects of the embodiment) As described above, the HUD device 100 of the present embodiment is a head-up display device that projects and displays one or more virtual images V superimposed on the foreground in a display area VA that is virtually set in front of the vehicle 10, for example, as shown in Fig. 1(a)(b). The head-up display device (HUD device 100) has a display unit 120 that displays the virtual image V, and a control unit 160 that controls the display of the virtual image V on the display unit 120, for example, as shown in Fig. 2. The control unit 160 controls the display of the virtual image V on the display unit 120 by changing the display mode of the virtual image V when an angle (depression angle) formed between a straight line parallel to the road surface based on the eye position (eye point EP) of the viewer 14 in the height direction and a straight line connecting the eye position (eye point EP) of the viewer 14 in the height direction and the center of the display area VA is equal to or smaller than a predetermined threshold value.

[0136] According to the HUD device 100 of this embodiment, when the depression angle is equal to or less than a predetermined threshold (the angular position of the reflective member (concave mirror) when the eye box is in the reference position), the control unit 160 controls the display unit 120 to change the display mode of the virtual image V (for example, by reducing the total number of virtual images V that can be arranged in a direction that includes at least the vertical component of the display area VA, and further by changing the relative size of the virtual images V that give the viewer 14 a sense of perspective, or by changing at least one of the size of the virtual images V, the decoration of the virtual images V, and the arrangement of the virtual images V). For example, by changing the display mode of the virtual images V and displaying them even if the eye position (eye point EP) of the viewer 14 in the height direction is lower than the reference position, it becomes easier to grasp the anteroposterior relationship of each image without losing the sense of perspective (without weakening the expressive power of depth) even if the drawable area (area in which the virtual image V can be drawn in the display area VA) is limited (the vertical component of the virtual image V drawn in the display area VA becomes shorter).

[0137] Furthermore, according to the HUD device 100 of this embodiment, when the control unit 160 changes the display mode, the total number of virtual images V (meaning the maximum number of virtual images V that can be arranged in a direction that includes at least the vertical component of the display area VA (the vertical angle of view having a vertical component within the angle of view AV that is the angular range in which the viewer 14 can view the image (virtual image V) as seen from the eye point EP)) that can be arranged in the display area VA is reduced, and the number of virtual images V displayed on the display unit 120 is controlled to be reduced. This makes it easier to grasp the relationship between the images without compromising the sense of perspective (without weakening the expressiveness of depth) even if the eye position of the viewer 14 in the vertical direction is lower than the reference position and the drawable area is restricted (the vertical component of the virtual images arranged in the display area is shortened).

[0138] When the angle (depression angle) between the eye position (eye point EP) of the viewer 14 in the height direction and the line connecting the center of the display area VA becomes equal to or larger than a predetermined threshold, the display mode is returned to the mode before the angle became equal to or smaller than the threshold (for example, as shown in FIG. 5 as an example of the display mode, the total number of virtual images that can be arranged in a direction including at least the vertical component of the display area VA is increased from two to three), or the total number of virtual images that can be arranged in a direction including at least the vertical component of the display area VA is increased from three to four. This makes it possible to provide a HUD device 100 that allows the viewer to easily grasp the relationship between the front and rear of each image without being affected by a change in the eye position (eye point EP) in the height direction of the viewer, in other words, without depending on the height of the viewer, and without impairing the sense of perspective (without weakening the expressive power of depth).

[0139] Furthermore, according to the HUD device 100 of this embodiment, the control unit 160 sets a vanishing point in linear perspective inside or outside the display area VA, and performs control to position the virtual image VV within a predetermined range that can be viewed by the viewer 14 as being positioned along the perspective image VP generated by emphasizing the perspective in an image simulating a driving lane passing through the vanishing point (this value is determined based on a threshold value for whether the viewer 14 can view each image as being positioned along the road surface, and the range is set, for example, so that the inclination of the image is 8° or more and less than 66°) so that each image is viewed by the viewer 14 as being positioned along the road surface, and it becomes easy to grasp the anteroposterior relationship between the viewed images.

[0140] For example, as shown in Fig. 6, a vanishing point (a point where two or more lines that are originally parallel in linear perspective intersect when they are drawn as non-parallel) is set within or outside the angle of view of an image (virtual image), and speed limit sign images (virtual images VT1, VT2) are arranged on an image (perspective image VP) that mimics the driving lanes toward the vanishing point, so that the viewer 14 can visually recognize these images (virtual images VT1, VT2) as if they are displayed along the road surface. In addition, the front-to-back relationship between the virtual images VT1 and VT2 can be expressed by arranging the speed limit sign image (virtual image VT1) of the road currently being traveled on the lower side (the side that is perceived as the near side as seen by the viewer) on the near side of the display area (imaging surface) and the speed limit sign image (virtual image VT2) of the nearest road on the upper side (the side that is perceived as the far side as seen by the viewer) on the near side of the display area (imaging surface).

[0141] Furthermore, according to the HUD device 100 of this embodiment, the control unit 160 can change the relative size of the virtual image V that gives the viewer 14 the perception of perspective, thereby enabling control so that the viewer 14 feels a sense of perspective without weakening the expressive power of depth even if the viewer's eye position in the vertical direction (eye point EP) becomes lower than the reference position and the total number of displays is reduced.

[0142] Furthermore, according to the HUD device 100 of this embodiment, the control unit 160 can control the viewer 14 to feel a sense of perspective by changing at least one of the size of the virtual image V, the decoration of the virtual image V, and the arrangement of the virtual image V, thereby weakening the expressive power of depth even if the total number of displays is reduced by lowering the eye position (eye point EP) in the height direction of the viewer from the reference position. For example, Fig. 7(a) shows an example of a display form when the decoration of the virtual image V is simplified, and Fig. 7(b) shows an example of a display form when the virtual image V is rearranged.

[0143] Furthermore, according to the HUD device 100 of this embodiment, the control unit 160 can perform display with improved expressiveness in the depth direction (front-rear direction of the vehicle) by using the display area VA, which is an inclined virtual image formation surface such that the upper end (e.g., PU in FIG. 1(b)) of the display area VA is deeper than the lower end (e.g., PL in FIG. 1(b)), and can achieve more natural visibility of images with a sense of perspective. Also, the display distance (imaging distance) can be adjusted according to the display position, and a display with a sense of depth can be achieved over a fairly wide range on the upper end side, making it easier to grasp the relationship between the front and rear of the images.

[0144] The display control device of this embodiment is a display control device that controls a head-up display device (HUD device 100) that projects and displays one or more virtual images superimposed on the foreground in a display area virtually set in front of a vehicle, as shown in Fig. 2. The display control device performs processing in which a processor (e.g., an information processing unit 401 of a navigation device 400) changes the display mode of a virtual image V when an angle (depression angle) formed between a straight line parallel to the road surface based on the eye position of the viewer 14 in the height direction and a straight line connecting the eye position of the viewer 14 in the height direction and the center of the display area VA is equal to or smaller than a predetermined threshold value.

[0145] According to the display control device of this embodiment, the processor performs processing to change the display mode of the virtual image V when the angle between a line parallel to the road surface based on the eye position of the viewer 14 in the height direction and a line connecting the eye position of the viewer 14 in the height direction and the center of the display area is equal to or less than a predetermined threshold. In this way, by changing the display mode of the virtual image V even if the eye position (eye point EP) of the viewer 14 in the height direction is lower than the reference position, it is possible to provide a display control device that makes it easy to grasp the anteroposterior relationship of each image without compromising the sense of perspective (without weakening the expressive power of depth) even if the drawable area is limited (the vertical component of the virtual image V placed in the display area becomes shorter).

[0146] In addition, according to the display control device of this embodiment, the processor changes the display mode by reducing the total number of virtual images V that can be arranged in a direction that includes at least the vertical component of the display area VA, thereby making it possible to provide a display control device that makes it easy to grasp the anteroposterior relationship of each image without compromising the sense of perspective, even if the eye position (eye point EP) of the viewer 14 in the vertical direction is lower than the reference position and the drawing area is limited.

[0147] Furthermore, according to the display control device of this embodiment, the processor sets a vanishing point in linear perspective inside or outside the display area VA, and performs control to place the virtual image V within a specified range that can be viewed by the viewer 14 so that the virtual image V is positioned along a perspective image that is generated by emphasizing the perspective of an image that mimics a driving lane that passes through the vanishing point, thereby providing a display control device that allows the viewer 14 to view each image as if it were positioned along the road surface and makes it easy to grasp the anteroposterior relationship between the viewed images.

[0148] Furthermore, according to the display control device of this embodiment, the processor changes the relative size of the virtual image V that gives the viewer 14 the perception of perspective, thereby making the eye position (eye point EP) of the viewer 14 in the vertical direction lower than the reference position, and a display control device can be provided that can reduce the total number of virtual images V that can be positioned in a direction that includes at least the vertical component of the display area VA and still allow the viewer to view the images without compromising the sense of perspective.

[0149] Furthermore, according to the display control device of this embodiment, the processor further changes at least one of the size of the virtual image V, the decoration of the virtual image V, and the arrangement of the virtual image V, thereby making the eye position (eye point EP) of the viewer 14 in the vertical direction lower than the reference position, and a display control device can be provided that can reduce the total number of virtual images V that can be arranged in a direction that includes at least the vertical component of the display area VA, and still allow the viewer to view the virtual images without compromising the sense of perspective.

[0150] Furthermore, according to the display control device of this embodiment, the processor can further control the angle and / or position of the relay optical system of the head-up display device (HUD device 100), and when the angle and / or position of the relay optical system falls within a predetermined range, it can determine that the depression angle is equal to or less than a predetermined threshold value, thereby changing the position of the display area VA in which the virtual image is placed in accordance with changes in the position of the eyebox.

[0151] 2, the display control system of this embodiment is a display control system (e.g., vehicle control system 1000) including a head-up display device (HUD device 100) that projects and displays one or more virtual images superimposed on the foreground in a display area virtually set in front of the vehicle, and a processing device (processor: e.g., information processing unit 401 of navigation device 400) that generates a virtual image V and arranges it in the display area VA. The head-up display device (HUD device 100) includes a display unit 120 that displays the virtual image V, and a control unit 160 that controls the display of the virtual image V on the display unit 120. Then, when there is a change in the vertical eye position (eye point EP) of the viewer 14, the processing device (information processing unit 401) outputs a display command to instruct the head-up display device (HUD device 100) to change the display mode of the virtual image V and display it when the angle (depression angle) formed by a straight line parallel to the road surface based on the vertical eye position (eye point EP) of the viewer 14 and a straight line connecting the viewer's vertical eye position (eye point EP) and the center of the display area VA is equal to or greater than a predetermined threshold value.

[0152] According to the display control system of this embodiment, a processing device (processor: for example, the information processing unit 401 of the navigation device 400) outputs a display command to instruct the head-up display device (HUD device 100) to change the display mode of the virtual image V and display it when an angle (depression angle) formed by a straight line parallel to the road surface based on the eye position (eye point EP) of the viewer 14 in the height direction and a straight line connecting the eye position (eye point EP) of the viewer in the height direction and the center of the display area VA is equal to or smaller than a predetermined threshold value, and the head-up display device (control unit 1 60) changes the display mode of the virtual images V (for example, by reducing the total number of virtual images V that can be arranged in a direction that includes at least the vertical component of the display area VA, and further by changing the relative size of the virtual images V that give the viewer 14 the perception of perspective, or by changing at least one of the size of the virtual images V, the decoration of the virtual images V, and the arrangement of the virtual images V), thereby making it possible to provide a display control system (for example, vehicle control system 1000) that makes it easy to grasp the anteroposterior relationship of each image without compromising the sense of perspective even when the drawing area is limited (the vertical component of the virtual images V arranged in the display area VA is shortened).

[0153] In the display control device and display control system of this embodiment, the functions executed by the processing device (processor: for example, the information processing unit 401 of the navigation device 400) are realized by a program executed by a computer. In this case, the information processing unit 401 includes a computer including at least one control device (for example, a processor) and at least one storage device (for example, a RAM) as hardware for executing the above-mentioned program.

[0154] The above-mentioned program may be recorded in one or more computer-readable recording media, not temporarily. The recording media may or may not be included in the information processing unit 401. In the latter case, the above-mentioned program may be supplied to the above-mentioned information processing unit 401 via any wired or wireless transmission medium.

[0155] At least a part of the above-mentioned functions can be realized by a logic circuit. For example, an integrated circuit in which a logic circuit is formed is also included in the scope of the present invention. In addition, the above-mentioned functions can be realized by, for example, a quantum computer. Each process described in this embodiment may be executed by an AI (Artificial Intelligence). In this case, the AI ​​may be one that operates in the information processing unit 401, or may be one that operates in another device (for example, a cloud server, etc.).

[0156] The present invention is not limited to the above-described exemplary embodiments, and those skilled in the art could easily modify the above-described exemplary embodiments to the extent that they fall within the scope of the claims. [Explanation of symbols]

[0157] 100 Head-up display device (HUD), 110 Drive unit, 120 Display unit, 120a Display surface, 121 Light source, 122 Liquid crystal display element, 141 Concave mirror, 160 Control unit, 200 Vehicle-mounted sensing device, 300 Driving support device, 400 Navigation device, 500 Operation input device, 600 Eye position detection unit, 700 CAN, 1000 Vehicle control system (display control system), V (V1, V2) Virtual image, VA Display area, VB Drawing area, VP Perspective image, VT1, VT2, VT3 Speed ​​limit sign image, VSI Alphanumeric characters indicating vehicle speed, VM1 Icon

Claims

1. A head-up display device that projects and displays one or more virtual images in a display area virtually set in front of a vehicle, the virtual images being superimposed on a foreground, A display unit that displays the virtual image; A control unit that controls displaying the virtual image on the display unit, The control unit is A head-up display device that controls the display mode of the virtual image to be changed and displayed on the display unit when the angle between a line parallel to the road surface based on the viewer's eye position in the height direction and a line connecting the viewer's eye position in the height direction and the center of the display area is less than a predetermined threshold value.

2. The control unit is The head-up display device according to claim 1 , wherein the display mode is changed by reducing a total number of the virtual images that can be arranged in a direction including at least a vertical component of the display area, and the virtual images are displayed on the display unit.

3. The control unit is 3. The head-up display device according to claim 2, wherein a vanishing point in linear perspective is set inside or outside the display area, and the virtual image is controlled to be positioned within a predetermined range visible to the viewer so that the virtual image is aligned with a perspective image generated by emphasizing the sense of perspective in an image simulating a driving lane passing through the vanishing point.

4. The control unit is 3. The head-up display device according to claim 2, further comprising control for changing a relative size of the virtual image that gives the viewer a sense of perspective and displaying the virtual image on the display unit.

5. The control unit is 3. The head-up display device according to claim 2, further comprising control for changing at least one of a size of the virtual image, a decoration of the virtual image, and a position of the virtual image to be displayed on the display unit.

6. The control unit is 6. A head-up display device according to claim 1, wherein, when an end of a display area virtually set in front of the vehicle is defined as a lower end on the side closer to the vehicle and an end of the display area farther from the vehicle is defined as an upper end, the upper end is displayed so that the upper end is seen by the viewer as being inclined and further back than the lower end.

7. A display control device controls a head-up display device that projects and displays one or more virtual images in a display area virtually set in front of a vehicle, the display control device comprising: The processor calculates the angle (hereinafter also referred to as the depression angle) between a line parallel to the road surface based on the viewer's eye position in the height direction and a line connecting the viewer's eye position in the height direction and the center of the display area. ) is equal to or less than a predetermined threshold, the display mode of the virtual image is changed.

8. The processor, The display control device according to claim 7 , wherein the display mode is changed by reducing a total number of the virtual images that can be arranged in a direction that includes at least a vertical component of the display area.

9. The processor, 9. The display control device according to claim 8, wherein a vanishing point in linear perspective is set inside or outside the display area, and the virtual image is controlled to be positioned within a predetermined range visible to a viewer so that the virtual image is aligned with a perspective image generated by emphasizing the sense of perspective in an image simulating a driving lane passing through the vanishing point.

10. The processor, The display control device according to claim 8 , further comprising: changing a relative size of the virtual image that gives the viewer a sense of perspective.

11. The processor, The display control device according to claim 8 , further comprising: changing at least one of a size of the virtual image, a decoration of the virtual image, and a layout of the virtual image.

12. The processor, The angle and / or position of a relay optical system of the head-up display device can be further controlled, and when the angle and / or position of the relay optical system is within a predetermined range, it is determined that the depression angle is equal to or less than the predetermined threshold value. The display control device according to claim 7.

13. A display control system including: a head-up display device that projects and displays one or more virtual images in a display area virtually set in front of a vehicle, the virtual images being superimposed on a foreground; and a processing device that generates the virtual images and arranges them in the display area, The head-up display device includes: A display unit that displays the virtual image; A control unit that controls displaying the virtual image on the display unit, The processing device includes: A display control system that outputs a display command to instruct the head-up display device to change the display mode of the virtual image when there is a change in the viewer's eye position in the height direction, and when the angle between a straight line parallel to the road surface based on the viewer's eye position in the height direction and a straight line connecting the viewer's eye position in the height direction and the center of the display area is equal to or greater than a predetermined threshold value.

Images