Display control device, head-up display device, and display control method

Abstract

To display an image which makes an observer easily sense depth feeling.SOLUTION: In a head-up display device for enabling an observer to visually recognize a virtual image of an image while overlapping a front scenery, a processor performs thickness display processing of displaying the virtual image as one or a plurality of perspective virtual images with depth expression having a thickness VT in a vertical direction when viewed from the observer and gradually reducing the thickness VT so as to converge to a disappearance point VP20, and the thickness disappearance point VP20 is arranged below a disappearance point VP0 of the front scenery when viewed from the observer.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present disclosure relates to a display control device, a head-up display device, a display control method, etc., that are used in a moving body such as a vehicle and that superimpose an image on the foreground of the moving body (the actual view in the forward direction of the moving body as seen by an occupant of the vehicle) and allow the image to be viewed.

[0002] Patent Document 1 describes a head-up display device (one example of a virtual image display device) in which display light projected onto a projection target such as the front windshield of a vehicle is reflected toward a vehicle occupant (observer) inside the vehicle, thereby allowing the observer to visually recognize a virtual image superimposed on the foreground of the vehicle. In particular, the head-up display device described in Patent Document 1 virtually places a display object (virtual image) at a predetermined position (here, the position is referred to as a target position) in the depth or in the up, down, left and right directions in the space of the foreground, and controls an image displayed inside the head-up display device as if the display object exists at the target position in the foreground, even if there is a change in the attitude of the vehicle or a change in the eye position of the observer. In other words, such a head-up display device forms an augmented reality in which a virtual object is added to and displayed in a real landscape (foreground).

[0003] In the example of FIG. 4 of Patent Document 1, a virtual object in the shape of an arrow pointing forward (in other words, an arrow extending from the rear to the front) is displayed at a position slightly above the road surface on which the vehicle is traveling (a position above the road surface). In such a case, as in the example of FIG. 5 of Patent Document 1, the observer perceives a sense of perspective as the width of the rear end of the arrow (typically the width in the left-right direction) gradually narrows toward the tip. In other words, the observer perceives the rear end of the arrow as being nearby and the tip of the arrow as being far away. A technique such as linear perspective (perspective projection) is typically used for the expression that creates such a sense of perspective. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2010-156608 A Summary of the Invention [Problem to be solved by the invention]

[0005] Incidentally, in the above-mentioned head-up display device, the area in which a virtual image can be displayed is typically a flat or curved area (here, the area will be referred to as a virtual image display area), and the virtual image display area is positioned so as to face directly toward the observer's line of sight. In other words, when viewed from the left or right of the vehicle, the angle between the virtual image display area in which the virtual image is displayed and the road surface on which the vehicle is traveling is set to approximately 90 degrees (the virtual image display area is not positioned so as to be approximately parallel to the road surface).

[0006] FIG. 11 is a diagram showing the foreground viewed by an observer through the front windshield WS of the host vehicle while the host vehicle is traveling, and the perspective image 1100 displayed in the virtual image display area 1000 overlapping the foreground. FIG. 11 shows the aspect of the perspective image expressed as being arranged parallel to the road surface using linear perspective. Here, the left line of the vehicle's driving lane 1010 is indicated by reference numeral 1011, the right line is indicated by reference numeral 1012, and the intersection of the extension of the left line 1011 and the extension of the right line 1012 is set as the vanishing point 1020. The perspective image 1100 in FIG. 11 is an arrow having a thickness 1120 in the height direction (Y-axis direction), and is expressed by linear perspective (one-point perspective) so that the intersection of the upper side 1101 passing through the upper part forming the thickness 1120 and the lower side 1102 passing through the lower part coincides with the vanishing point 1020. 11 includes a first thickness portion 1111 that is expressed as being in front of the viewer and has a first thickness 1121 in the height direction (Y-axis direction), and a second thickness portion 1112 that is expressed as being further back than the first thickness portion 1111 and has a second thickness 1122 in the height direction (Y-axis direction). In linear perspective (one-point perspective), the further away something is rendered, the smaller (shorter) it is, so the second thickness 1122 is shorter than the first thickness 1121. In other words, in linear perspective (one-point perspective), the distance in the height direction (Y-axis direction) between an upper side 1101 that passes through the upper part and a lower side 1102 that passes through the lower part, which form the thickness 1120, is expressed as gradually shorter as it becomes farther away, and is set to zero at the vanishing point 1020. This makes it possible to pictorially express that the perspective image 1100 is arranged parallel to the plane of the driving lane 1010.

[0007] However, as described above, even if a perspective image 1100 such as that shown in FIG. 11 is displayed in a virtual image display area that is not positioned so as to be approximately parallel to the road surface, that is, a virtual image display area that is raised toward the observer by approximately 90 degrees from the road surface on which the vehicle is traveling, the distance between the far end 1300 of the perspective image 1100 and the driving lane 1010 is longer than the distance between the near end 1400 and the driving lane 1010, and the far end 1300 of the perspective image 1100 is perceived as raised toward the observer relative to the parallel surface of the driving lane 1010, making it difficult to sense that it is parallel to the surface of the driving lane 1010, and there is room for improvement.

[0008] A summary of certain embodiments disclosed herein is provided below. It should be understood that these aspects are presented merely to provide the reader with an overview of these certain embodiments, and are not intended to limit the scope of the disclosure. Indeed, the disclosure may encompass a variety of aspects that are not set forth below.

[0009] The present disclosure relates to displaying an image that allows a user to easily sense a sense of depth. More specifically, the present disclosure relates to providing a display control device, a head-up display device, a display control method, and the like that display an image that allows a user to easily sense that the image is parallel to a plane of a driving lane.

[0010] Therefore, the display control device, head-up display device, display control method, etc. described in this specification adopt the following means to solve the above problems. The gist of this embodiment is that a display control device controls a head-up display device that allows an observer to visually recognize a virtual image of an image superimposed on a foreground, and executes a thickness display process that displays one or more perspective virtual images with a depth expression that has a thickness in the vertical direction as seen by the observer and that gradually reduces the thickness so as to converge to a vanishing point, and the thickness vanishing point is located below the vanishing point of the foreground as seen by the observer.

[0011] Therefore, the display control device described in this specification is a display control device that controls a head-up display device that allows the observer to view a virtual image of an image superimposed on the foreground, and executes a thickness display process that displays the image as one or more perspective virtual images with a depth representation that has thickness in the vertical direction as seen by the observer and that gradually reduces the thickness so as to converge to a vanishing point, and the thickness vanishing point is located below the vanishing point in the foreground as seen by the observer. [Brief description of the drawings]

[0012] [Figure 1] FIG. 1 is a diagram showing an example of application of a virtual image display system for a vehicle to a vehicle. [Diagram 2] FIG. 2 is a diagram showing a configuration of the head-up display device. [Diagram 3] FIG. 3 is a diagram conceptually showing a virtual object placed at a target position in a real scene, and an image displayed in a virtual image display area so that the virtual object is visually recognized at the target position in the real scene. [Figure 4] FIG. 4 is a diagram showing an example of a foreground viewed by an observer while the host vehicle is traveling, and a virtual image in the first width display manner displayed so as to be superimposed on the foreground. [Diagram 5] FIG. 5 is a diagram showing an example of a foreground viewed by an observer while the host vehicle is traveling, and a virtual image in the first width display manner displayed so as to be superimposed on the foreground. [Figure 6A] FIG. 6A is a diagram showing a modified example of a perspective image. [Figure 6B] FIG. 6B is a diagram showing a modified example of a perspective image. [Figure 6C] FIG. 6C is a diagram showing a modified example of a perspective image. [Figure 6D] FIG. 6D is a diagram showing a modified example of a perspective image. [Figure 7] FIG. 7 is a diagram showing an example of a foreground viewed by an observer while the host vehicle is traveling, and a virtual image in the first thickness display manner displayed superimposed on the foreground. [Figure 8A]FIG. 8A is a diagram showing a perspective virtual image V of a first thickness display mode having a second vanishing point. [Figure 8B] FIG. 8B is a diagram showing a perspective virtual image V of the first thickness display mode having a second vanishing point. [Figure 9] FIG. 9 is a block diagram of a vehicle virtual image display system according to some embodiments. [Figure 10] FIG. 10 is a flow diagram showing a display control method for executing a display control process for displaying a perspective virtual image in the first width display mode or the first thickness display mode. [Figure 11] FIG. 11 is a diagram relating to a conventional example, showing an example of a foreground viewed by an observer while the host vehicle is traveling, and a virtual image displayed superimposed on the foreground. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] 1 to 10, an exemplary configuration and operation of a vehicle virtual image display system will be described. Note that the present invention is not limited to the following embodiments (including the contents of the drawings). Of course, modifications (including removal of components) can be made to the following embodiments. In addition, in the following description, descriptions of known technical matters will be omitted as appropriate in order to facilitate understanding of the present invention.

[0014] Please refer to FIG. 1. FIG. 1 is a diagram showing an example of application of a virtual image display system for vehicles to a vehicle. In FIG. 1, the left-right direction (in other words, the width direction of the vehicle 1) of a vehicle (an example of a moving body, hereinafter also referred to as the vehicle itself) 1 is the X-axis (the positive direction of the X-axis is the left direction when the vehicle 1 faces forward), the up-down direction (in other words, the height direction of the vehicle 1) along a line segment that is perpendicular to the left-right direction and perpendicular to the ground or a surface equivalent to the ground (here, the road surface 6) is the Y-axis (the positive direction of the Y-axis is the upward direction), and the front-rear direction along a line segment that is perpendicular to each of the left-right direction and the up-down direction is the Z-axis (the positive direction of the Z-axis is the straight-ahead direction of the vehicle 1). This point is the same in other drawings.

[0015] A head-up display device (an example of a virtual image display device) 20 in the vehicle virtual image display system 10 is a head-up display (HUD) device provided in a dashboard 5 of a vehicle (an example of a moving body) 1. The HUD device 20 emits display light K toward a front windshield 2 (an example of a projection target member), and the front windshield 2 reflects the display light K of an image displayed by the HUD device 20 toward an eye box 200. The HUD device 20 notifies an occupant of the host vehicle 1 (the occupant is typically the driver of the host vehicle 1) of not only information related to the host vehicle 1 (hereinafter referred to as vehicle information) but also information other than the vehicle information in an integrated manner. The vehicle information includes not only information about the host vehicle 1 itself but also information outside the host vehicle 1 related to the operation of the host vehicle 1.

[0016] As shown in the figure, the vehicle virtual image display system 10 provided in a vehicle (host vehicle) 1 includes an eye position detection unit (gaze detection unit) 409 for pupil (or face) detection that detects the position and gaze direction of the left eye 700L and right eye 700R of an observer (typically a driver sitting in the driver's seat of the host vehicle 1), an external sensor 411 composed of a camera (e.g., a stereo camera) that captures an image in front of the host vehicle 1 (broadly speaking, the surroundings), a head-up display device (hereinafter also referred to as a HUD device) 20, and a display control device 30 that controls the HUD device 20.

[0017] In addition, the "eye box" used in the description of this embodiment is (1) a region in which the entire virtual image V of the image can be seen, and at least a part of the virtual image V of the image cannot be seen outside the region, (2) a region in which at least a part of the virtual image V of the image can be seen, and at least a part of the virtual image V of the image cannot be seen outside the region, (3) a region in which at least a part of the virtual image V of the image can be seen with a predetermined luminance or more, and at least a part of the virtual image V of the image is less than the predetermined luminance outside the region, or (4) a region in which at least a part of the virtual image V can be viewed stereoscopically when the HUD device 20 can display a virtual image V that can be viewed stereoscopically, and at least a part of the virtual image V cannot be viewed stereoscopically outside the region. That is, when the observer places his / her eyes (both eyes) outside the eye box 200, the observer cannot view the entire virtual image V of the image, the visibility of the entire virtual image V of the image is very low and difficult to perceive, or the virtual image V of the image cannot be viewed stereoscopically. The predetermined brightness is, for example, about 1 / 50 of the brightness of a virtual image of an image viewed at the center of the eyebox. The "eyebox" is set to be the same as the area (also called the iris) where the observer's viewpoint is expected to be located in the vehicle in which the HUD device 20 is mounted, or to include most of the iris (for example, 80% or more).

[0018] The virtual image display area VS is a flat, curved, or partially curved area where an image generated inside the HUD device 20 is formed as a virtual image V, and is also called an imaging surface. The virtual image display area VS is a position where a display surface (e.g., an exit surface of a liquid crystal light modulation element) 51 of a display 40 of the HUD device 20 described later is formed as a virtual image by a virtual image optical system 90 described later, that is, the virtual image display area VS corresponds to a display surface 51 of the HUD device 20 described later (in other words, the virtual image display area VS is in a conjugate relationship with the display surface 51 of the display 40 described later), and the virtual image viewed in the virtual image display area VS corresponds to an image displayed on a display surface 51 of the HUD device 20 described later. It is preferable that the virtual image display area VS itself has low visibility to the extent that it is not actually visible to the observer's eyes or is difficult to be visible.

[0019] The virtual image display area VS has an angle (tilt angle θt in Figure 1) between the horizontal direction (XZ plane) and the axis along the left-right direction (X-axis direction) of the vehicle 1, and the angle between the line segment connecting the center 205 of the eye box 200 and the upper end VS1 of the virtual image display area VS and the line segment connecting the center of the eye box and the lower end VS2 of the virtual image display area VS, which is defined as the vertical angle of view of the virtual image display area VS, and an angle (vertical arrangement angle θd in Figure 1) between the bisector of this vertical angle of view and the horizontal direction (XZ plane). The vertical arrangement angle θd is typically set downward from the horizontal direction (XZ plane), for example, +2 degrees (here, the positive direction of the vertical arrangement angle θd can also be said to be a depression angle where the bisector of the vertical angle of view is oriented downward from the center 205 of the eyebox 200, and the negative direction of the vertical arrangement angle θd can also be said to be an elevation angle where the bisector of the vertical angle of view is oriented upward from the center 205 of the eyebox 200).

[0020] In addition, the virtual image display region VS of this embodiment has a tilt angle θt of approximately 90 [degrees] so that it faces approximately directly when facing forward (positive direction of the Z axis). However, the tilt angle θt is not limited to this and can be changed within the range of 30≦θt<90 [degrees]. The tilt angle θt may be set to, for example, 60 [degrees], and the virtual image display region VS may be disposed so that the upper region is farther away than the lower region as viewed by the viewer. However, the range of the tilt angle θt is not limited to this.

[0021] 2 is a diagram showing one embodiment of the configuration of a head-up display device. The HUD device 20 is installed, for example, in a dashboard (reference numeral 5 in FIG. 1). The HUD device 20 has a display 40, a relay optical system 80, and a housing 22 that houses the display 40 and the relay optical system 80 and has a light exit window 21 that can emit display light K from the display 40 from the inside to the outside.

[0022] 2 is typically mainly composed of a liquid crystal display panel (light modulation element) 50 and a light source unit 60. A display surface 51 is the viewer-side surface of the liquid crystal display panel 50, and emits display light K of an image. The angle (including the tilt angle θt) of the virtual image display region VS can be set by setting the angle of the display surface 51 with respect to an optical axis Kp of the display light K traveling from the center of the display surface 51 toward the eye box 200 (the center 205 of the eye box 200) via the relay optical system 80 and the projected portion.

[0023] The liquid crystal display panel (light modulation element) 50 receives light from the light source unit 60, and emits display light K, which is spatially light modulated into a 2D image, toward the relay optical system 80 (second mirror 82). The liquid crystal display panel 50 is, for example, rectangular in shape, with the short side being the direction in which pixels corresponding to the vertical direction (Y-axis direction) of the virtual image V seen by the observer are arranged. The observer visually recognizes the transmitted light of the liquid crystal display panel 50 through the virtual image optical system 90. The virtual image optical system 90 is a combination of the relay optical system 80 and the front windshield 2 shown in FIG. 2. The light modulation element 50 (display 40) may be a self-luminous display, or may be a projection display (for example, a reflective display panel using LCoS or DMD) that projects an image onto a screen, but is not limited thereto. In this case, the display surface 51 is the screen of the projection display.

[0024] The light source unit 60 is composed of a light source (not shown) and an illumination optical system (not shown).

[0025] The light source (not shown) is, for example, a plurality of chip-type LEDs, and emits illumination light to the liquid crystal display panel (an example of a light modulation element) 22. The light source unit 60 is composed of, for example, four light sources, and is arranged in a row along the long side of the liquid crystal display panel 50. The light source unit 60 emits illumination light toward the liquid crystal display panel 50 under the control of the display control device 30. The configuration of the light source unit 60 and the arrangement of the light sources are not limited to this.

[0026] The illumination optical system (not shown) is composed of, for example, one or more lenses (not shown) arranged in the emission direction of the illumination light from the light source unit 60, and a diffusion plate (not shown) arranged in the emission direction of the one or more lenses.

[0027] The light source unit 60 may be configured to be capable of local dimming, and may change the degree of illumination for each area of ​​the display surface 51 under the control of the display control device 30. This allows the light source unit 60 to adjust the luminance of the image displayed on the display surface 51 for each area.

[0028] The relay optical system 80 is disposed on the optical path of the display light K (light traveling from the display 40 toward the eye box 200) emitted from the display 40, and is composed of one or more optical members that project the display light K from the display 40 onto the front windshield 2 outside the HUD device 20. The relay optical system 80 in FIG. 2 includes one concave first mirror 81 and one flat second mirror 82.

[0029] The first mirror 81 is, for example, a free curved surface shape having a positive optical power. In other words, the first mirror 81 may be a curved surface shape having different optical powers for each region, that is, the optical power added to the display light K may be different depending on the region (optical path) through which the display light K passes. Specifically, the optical power added by the relay optical system 80 may be different between the first display light K11, the second display light K12, and the third display light K13 (see FIG. 2) traveling from each region of the display surface 51 toward the eye box 200.

[0030] The second mirror 82 is, for example, a flat mirror, but is not limited thereto, and may be a curved surface having optical power. That is, the relay optical system 80 may combine a plurality of mirrors (for example, the first mirror 81 and the second mirror 82 in this embodiment) to add different optical power depending on the region (optical path) through which the display light K passes. The second mirror 82 may be omitted. That is, the display light K emitted from the display 40 may be reflected by the first mirror 81 to the projection target portion (front windshield) 2.

[0031] In addition, the relay optical system 80 includes two mirrors (reflective optical system) in this embodiment, but is not limited to this. The relay optical system 80 may include one or more refractive optical elements such as lenses, diffractive optical elements such as holograms, reflective optical elements different from those described above, or combinations thereof, which may be added to the above-mentioned reflective optical system, or may replace part or all of the above-mentioned reflective optical system.

[0032] Furthermore, the relay optical system 80 has the function of setting the distance to the virtual image display area VS by using this curved shape (an example of optical power), and the function of generating a virtual image that is an enlarged image of the image displayed on the display surface 51. In addition to this, it may also have the function of suppressing (correcting) distortion of the virtual image that may occur due to the curved shape of the front windshield 2, or other known optical functions.

[0033] Furthermore, the relay optical system 80 may be rotatable by being fitted with one or more actuators 86, 87 controlled by the display control device 30. For example, the actuator 86 may rotate (and / or move) the first mirror 81 about a first rotation axis AX1. The actuator 87 may rotate (and / or move) the second mirror 82 about the first rotation axis AX1.

[0034] The head-up display device 20 can display images in the vicinity of objects such as the road surface of the driving lane, a branch road, a road sign, an obstacle (pedestrian, bicycle, motorcycle, other vehicle, etc.), and a feature (building, bridge, etc.) that exist in the foreground, which is a real space (real scene) visually recognized through the front windshield 2 of the vehicle 1, a position overlapping the object, or a position set based on the object, so that an observer (typically an observer seated in the driver's seat of the vehicle 1) can perceive visual augmented reality (AR). In the description of this embodiment, an image whose display position can be changed according to a real object existing in the real scene or a depth or a target position in the up, down, left, and right directions in a predetermined space in the real scene determined by the display control device 30 is defined as an AR image, and an image whose display position is set regardless of the position of the real object is defined as a non-AR image. In addition, in the description of this embodiment, an image consisting of one or more images expressed by a perspective projection that creates a sense of perspective is defined as a perspective image, and an image not expressed by a perspective projection is defined as a non-perspective image. A perspective image is typically an AR image.

[0035] A display control device 30, which will be described later, can control the aspect of the image V displayed by the HUD device 20 (perceived by the observer) by executing, for example, image rendering processing (graphic processing), display drive processing, and the like.

[0036] 3 is a conceptual diagram showing a virtual object arranged at a target position in a real scene, and an image displayed in a virtual image display area so that the virtual object is visually recognized at the target position in the real scene. As shown in FIG. 3, the depth direction is the Z-axis direction as viewed from the observer 700, the left-right direction (width direction of the vehicle 1) is the X-axis direction, and the up-down direction (up-down direction of the vehicle 1) is the Y-axis direction. Note that the direction moving away from the observer is the positive direction of the Z-axis, the left direction as viewed from the observer is the positive direction of the X-axis, and the up direction as viewed from the observer is the positive direction of the Y-axis.

[0037] The viewer 700 perceives a virtual object FU at a predetermined target position PT in the real scene by viewing the virtual image V formed (focused) in the virtual image display area VS via the projection unit 2. The viewer views the virtual image V of the image of the display light K reflected by the projection unit 2. At this time, if the virtual image V is, for example, an arrow indicating a course, the arrow of the virtual image V is displayed in the virtual image display area VS so that the virtual object FU is positioned and viewed at the predetermined target position PT in the foreground of the host vehicle 1.

[0038] (Perspective virtual image V10 of the first width display mode) 4 is a diagram showing an example of the foreground viewed by the observer while the vehicle 1 is traveling, and a virtual image of the first width display mode displayed superimposed on the foreground. The virtual image V10 of the first width display mode (hereinafter also referred to as the perspective virtual image V10) has a vanishing point VP10 (also referred to as the first vanishing point VP10, width vanishing point, or first width vanishing point). That is, the figure of the perspective virtual image V10 has a shape with a vanishing point VP10 that gives a sense of depth, and for example, an extension line V1A of the left side of the stem of the "arrow" and an extension line V1B of the right side intersect at the vanishing point VP10.

[0039] As shown in the figure, when the perspective virtual image V10 is disposed downward from the viewer's perspective (at a position where the vertical arrangement angle θd is in a positive direction), the vanishing point VP10 is disposed above the position of the virtual image V10. On the other hand, when the perspective virtual image V10 is disposed upward from the viewer's perspective (at a position where the vertical arrangement angle θd is in a negative direction), the vanishing point VP10 is disposed below the position of the virtual image.

[0040] The viewer gets a sense of depth from the perspective virtual image V10 as seen by the viewer based on the relationship between the positions of the vanishing point VP10 and the perspective virtual image V10. In this way, the perspective virtual image V10 has the vanishing point VP10, making it easy for the viewer to estimate the depth position of the perspective virtual image V10.

[0041] On the other hand, as shown in Fig. 4, the driving lane 6 of the vehicle 1 has a vanishing point VP0 (actual scene vanishing point VP0). In this specific example, there is a driving lane 6 going straight ahead as seen by the viewer, and the extensions of the boundaries 7 and 8 on both sides of the driving lane 6 essentially intersect (become a point) at the vanishing point VP0. In this way, since the driving lane (road surface) 6 has a vanishing point VP0, the viewer gets a sense of depth in the actual scene (driving lane 6).

[0042] In the vehicle virtual image display system 10 according to this embodiment, as shown in FIG. 4, the vanishing point VP10 of the perspective virtual image V10 is located in front of (below when viewed from the observer) the vanishing point VP0 of the real scene (the driving lane 6).

[0043] In general, when creating an image, including a painting, using linear perspective (perspective projection), a vanishing point is used to position various objects in the image at their respective depth positions. For example, a straight line is virtually drawn radially from a predetermined vanishing point, and the contour line of each object is aligned with the straight line, so that each object is perceived as being fixed at a predetermined depth position. Note that multiple vanishing points can be provided, but for simplicity, the following description will be given assuming that one vanishing point is provided in one image.

[0044] When a general linear perspective method is used, a virtual image is displayed so that the extension lines of each boundary line forming the outline of a virtual object perceived by an observer intersect at the vanishing point VP0 in the foreground (driving lane 6). In other words, the virtual image is generated so that the position of the vanishing point VP10 of the virtual image coincides with the position of the vanishing point VP0 in the foreground (driving lane 6). However, in this embodiment, the perspective virtual image V10 is generated so that the position of the vanishing point VP10 of the perspective virtual image V10 is closer to the viewer than the position of the vanishing point VP0 in the foreground (driving lane 6) (first width display process).

[0045] This makes it possible for the viewer to easily sense that the perceived placement angle of the virtual object FU10 (the angle with respect to the road surface with the X2 axis in FIG. 3 as the axis of rotation) is approximately parallel to the driving lane 6.

[0046] The virtual object FU10 is an "arrow", and the target position PT at which the virtual object FU10 is placed (set) is usually set to match the height of the surface of the foreground (driving lane 6) (i.e., the set height is set to 0 m). However, this is just one example and is not limiting. In another example, the target position PT may be set to a position higher than the height 1 of the surface of the foreground (driving lane 6) (i.e., the set height may be set to 0.5 m or 1 m). In another example, the target position PT may be set to a position lower than the height of the surface of the foreground (driving lane 6) (i.e., the set height may be set to -1 m or -2 m).

[0047] (Perspective virtual image V10 of the first thickness display mode) Moreover, the perspective virtual image V10 in the first thickness display mode has one or more thicknesses VT, and has one or more vanishing points VP20 of the thicknesses VT (also called second vanishing points or second width vanishing points). That is, the figure of the perspective virtual image V10 in the first thickness display mode has one or more thicknesses VT, and the farther the distance that the viewer is to perceive, the gradually shorter the one or more thicknesses VT are toward the second vanishing point VP20. That is, the distance between the extension line V1C of the upper side forming the thickness VT and the extension line V1D of the lower side forming the thickness VT shown in FIG. 4 becomes zero at the second vanishing point VP20 (the extension line V1C of the upper side forming the thickness VT and the extension line V1D of the lower side intersect at the second vanishing point VP20).

[0048] As shown in Fig. 4, the perspective virtual image V10 in the first thickness display mode has two thicknesses VT1 and VT2. The perspective virtual image V10 in the first thickness display mode has a first thickness portion VY1 parallel to the up-down, left-right direction (XY plane) represented as being closest to the viewer of the stem of the arrow, and a second thickness portion VY2 parallel to the up-down, left-right direction (XY plane) represented as being closest to the viewer of the head of the arrow. The first thickness portion VY1 has a length (hereinafter also referred to as the first thickness) VT1 along the up-down direction (Y-axis direction) as seen by a viewer facing forward from the driver's seat, and the second thickness portion VY2 has a length (hereinafter also referred to as the second thickness) VT2 along the up-down direction (Y-axis direction) as seen by a viewer facing forward from the driver's seat. Here, the first thickness VT1 on the side near the viewer is longer than the second thickness VT2 of the second thickness portion VY2 which is positioned farther away than the first thickness portion VY1 having the first thickness VT1, and the first thickness VT1 and the second thickness VT2 are set based on the upper edge V1C and the lower edge V1D which converge to the second vanishing point VP20.

[0049] The viewer gets a sense of depth of the perspective virtual image V10 as seen by the viewer based on the change in thickness of the perspective virtual image V10 in the depth direction (the relationship between the lengths of the first thickness VT1 and the second thickness VT2). In this way, the perspective virtual image V10 has a second vanishing point VP20 related to the thickness VT, which makes it easy for the viewer to estimate the depth position of the perspective virtual image V10.

[0050] In the vehicle virtual image display system 10 according to the second embodiment, as shown in Fig. 4, the second vanishing point VP20 of the thickness VT of the perspective virtual image V10 is disposed on the near side (lower side as seen by the observer) than the vanishing point VP0 of the real scene (the driving lane 6). That is, when a general linear perspective method is used, the virtual image is displayed so that the extension lines V1C and V1D of the boundaries forming the thickness of the virtual object perceived by the observer intersect at the vanishing point VP0 of the foreground (the driving lane 6). That is, the virtual image is generated so that the position of the second vanishing point VP20 of the thickness VT of the virtual image coincides with the position of the vanishing point VP0 of the foreground (the driving lane 6). In the second embodiment, the perspective virtual image V10 is generated so that the position of the second vanishing point VP20 of the thickness VT of the perspective virtual image V10 is located on the near side of the position of the vanishing point VP0 of the foreground (the driving lane 6) (first thickness display process).

[0051] In the example of Fig. 4, the first vanishing point VP10 in the width direction (X-axis direction) of the perspective virtual image V10 in the first width display process and the second vanishing point VP20 in the thickness direction (Y-axis direction) of the perspective virtual image V10 in the first thickness display process coincide with each other, but are not limited to this. The second vanishing point VP20 may be above the first vanishing point VP10 and below the actual scene vanishing point VP0 as seen by the observer. This makes it possible to display an image that is easily perceived as being parallel to the plane of the travel lane while maintaining the thickness VT even in the distance (while suppressing the decrease in the thickness VT).

[0052] Fig. 5 is a diagram showing a modified example of the perspective virtual image V10 in the first thickness display mode. The perspective virtual image V10 in the first thickness display mode shown in Fig. 4 has two (one example of a plurality) separated first thickness parts VY1 having a first thickness VT1 and a second thickness part VY2 having a second thickness VT2, but is not limited thereto. As shown in FIG. 5, the perspective virtual image V10 in the first thickness display mode has one thickness portion VY3, and the near end of the one thickness portion VY3 as seen by the observer is designated as the third thickness VT3, and the far end is designated as the fourth thickness VT4. The third thickness VT3 and the fourth thickness VT4 in the one thickness portion VY3 may be set so that an extension line V1C of the upper side connecting the upper end of the third thickness VT3 and the upper end of the fourth thickness VT4, and an extension line V1D of the lower side connecting the lower end of the third thickness VT3 and the lower end of the fourth thickness VT4, converge to a second vanishing point VP20 below the actual scene vanishing point VP0.

[0053] 6A, 6B, 6C, and 6D are diagrams showing modified examples of perspective images. The perspective virtual image V10 in the above embodiment is composed of one straight "arrow", but is not limited to this. The perspective virtual image V20 shown in FIG. 6A may be bent from the middle of the image ("arrow"). In this case, the left perspective line V1A can be defined as an extension of the left side of the perspective virtual image V20 until it is bent, and the right perspective line V1B can be defined as an extension of the right side of the perspective virtual image V20 until it is bent.

[0054] 6B is composed of a first perspective virtual image V31 and a second perspective virtual image V32 that is disposed above the first perspective virtual image V31 and is set to be smaller in size than the first perspective virtual image V31. In this case, the left perspective line V1A can be defined as a straight line connecting the left end of the first perspective V31 and the left end of the second perspective V32, and the right perspective line V1B can be defined as a straight line connecting the right end of the first perspective V31 and the right end of the second perspective V32.

[0055] 6B, the perspective virtual image V30 has two thicknesses VT5 and VT6. The perspective virtual image V30 of the first thickness display mode has a first thickness portion VY1 parallel to the up-down, left-right direction (XY plane) of the first perspective virtual image V31 that is displayed as being near the viewer, and a second thickness portion VY2 parallel to the up-down, left-right direction (XY plane) of the second perspective virtual image V32 that is displayed as being farther away than the first perspective virtual image V31. The first thickness portion VY1 has a first thickness VT1 along the up-down direction (Y-axis direction) as seen by a viewer facing forward from the driver's seat, and the second thickness portion VY2 has a second thickness VT2 along the up-down direction (Y-axis direction) as seen by a viewer facing forward from the driver's seat. Here, the first thickness VT1 on the side near the viewer is longer than the second thickness VT2 of the second thickness portion VY2 which is positioned farther away than the first thickness portion VY1 having the first thickness VT1, and the first thickness VT1 and the second thickness VT2 are set based on the upper edge V1C and the lower edge V1D which converge to the second vanishing point VP20.

[0056] 6C may be configured with a plurality of perspective virtual images V41 to V49 divided into left and right. In this case, the left perspective line V1A can be defined as a straight line (approximate straight line) connecting the left ends of the left perspectives V41 to V44, and the right perspective line V1B can be defined as a straight line (approximate straight line) connecting the right ends of the right perspectives V45 to V48.

[0057] 6D may be set to a position facing a vanishing point VP0' that is shifted to the left or right of vanishing point VP0, which is the intersection of boundary lines 7, 8 of the travel lane 6, as seen by the observer (the vanishing point VP0' shown in FIG. 6D is disposed to the left of vanishing point VP0). Even in this case, the vanishing point VP10 (first vanishing point VP10) of the perspective virtual image V50 may be disposed in front of vanishing point VP0' (actual scene vanishing point VP0).

[0058] (Perspective virtual image V60 of the second width display mode) 7 is a diagram showing an example of the foreground viewed by the observer while the vehicle 1 is traveling, and a virtual image of the second width display mode displayed superimposed on the foreground. The virtual image V60 of the second width display mode (hereinafter also referred to as the perspective virtual image V60) has a vanishing point VP30 (third vanishing point VP30) on the farther side than the first vanishing point VP10 of the perspective virtual image V10 of the first width display mode. That is, the figure of the perspective virtual image V60 has a shape having the third vanishing point VP30 that gives a sense of depth, and for example, an extension line V1B of the left side of the stem of the "arrow" and an extension line V2B of the right side intersect at the third vanishing point VP30.

[0059] As shown in the figure, when the perspective virtual image V60 of the second width display mode is disposed downward (at a position where the vertical arrangement angle θd is in a positive direction) as seen by the viewer, the third vanishing point VP30 is disposed above the position of the first vanishing point VP10. On the other hand, when the perspective virtual image V60 of the second width display mode is disposed upward (at a position where the vertical arrangement angle θd is in a negative direction) as seen by the viewer, the third vanishing point VP30 is disposed below the position of the first vanishing point VP10.

[0060] The HUD device 20 (display control device 30) of some embodiments changes the display mode of the perspective virtual image V to the above-mentioned first width display mode having the first vanishing point VP10 or the above-mentioned second width display mode having the third vanishing point VP30 set on the far side of the first vanishing point VP10 according to the type of the perspective virtual image V. The HUD device 20 (display control device 30) of some embodiments (10) displays the perspective virtual image V in the second width display mode having the third vanishing point VP30 if the perspective virtual image V is a moving image that moves or expands in the perspective direction (one example of a second width display process), and (15) displays the perspective virtual image V in the first width display mode having the first vanishing point VP10 if the perspective virtual image V is a still image, a moving image that does not move much in the perspective direction, or a moving image that does not expand much in the perspective direction.

[0061] 8A and 8B are diagrams showing a perspective virtual image V in a second width display manner having a third vanishing point VP30. In some embodiments, the HUD device 20 (display control device 30) displays the perspective virtual image V in the second width display manner having the third vanishing point VP30 (an example of a second width display process) when the perspective virtual image V is (11) a moving image that is perceived as gradually expanding and contracting in the perspective direction (in the example of FIG. 8A, a moving image expressed in a perspective projection that gradually extends from the near side toward the third vanishing point VP30 on the far side) or (12) a moving image that is perceived as gradually moving in the perspective direction (in the example of FIG. 8B, a moving image expressed in a perspective projection that gradually moves from the near side toward the third vanishing point VP30 on the far side). On the other hand, when the perspective virtual image V is (16) a still image, (17) a moving image that is not perceived as stretching much in the perspective direction, or (18) a moving image that is not perceived as moving much in the perspective direction, the HUD device 20 (display control device 30) displays it in a first width display manner having a first vanishing point VP10 (an example of a first width display process). Specifically, the "(17) moving image that is not perceived as stretching much in the perspective direction, or (18) a moving image that is not perceived as moving much in the perspective direction" is, for example, a moving image that moves in the left-right direction (X-axis direction), or a moving image that moves in the up-down direction (Y-axis direction) by an infinitesimal distance less than a predetermined threshold in a direction that can be perceived by humans as the depth direction (Z-axis direction) (more specifically, a moving image in which the distance of movement in the up-down direction (Y-axis direction) is 1 / 10 or less of the distance of movement in the left-right direction (X-axis direction)), etc.

[0062] Furthermore, the HUD device 20 (display control device 30) of some embodiments changes the display mode of the perspective virtual image V to the above-mentioned first width display mode having the first vanishing point VP10 or the above-mentioned second width display mode having the third vanishing point VP30 set on the far side of the first vanishing point VP10, depending on the moving speed (deformation speed) of the moving image of the perspective virtual image V. For example, the HUD device 20 (display control device 30) of some embodiments (20) displays the perspective virtual image V in the second width display mode having the third vanishing point VP30 if the moving speed (deformation speed) of the moving image is fast (an example of a second width display process), and (25) displays the perspective virtual image V in the first width display mode having the first vanishing point VP10 if the moving speed of the moving image is slow (an example of a first width display process).

[0063] In the above embodiment, the HUD device 20 (display control device 30) displays the perspective virtual image V in a second width display manner having a third vanishing point VP30 when the speed of the image (21) gradually expanding and contracting in the perspective direction or gradually moving in the perspective direction is equal to or greater than a predetermined image speed threshold and is determined to be fast (an example of a second width display process). On the other hand, in the above embodiment, the HUD device 20 (display control device 30) displays the perspective virtual image V in a first width display manner having a first vanishing point VP10 when the speed of the image (26) gradually expanding and contracting in the perspective direction or gradually moving in the perspective direction is less than a predetermined image speed threshold and is determined to be slow (an example of a first width display process).

[0064] In addition, the HUD device 20 (display control device 30) of some embodiments changes the display mode of the perspective virtual image V to the first width display mode having the first vanishing point VP10 or the second width display mode having the third vanishing point VP30 set on the farther side than the first vanishing point VP10, depending on the brightness of the area ahead (surroundings) of the vehicle 1 (an example of a second width display process). For example, the HUD device 20 (display control device 30) of some embodiments (30) displays the image in the second width display mode having the third vanishing point VP30 when it is determined that the area ahead (surroundings) of the vehicle 1 is bright, and (35) displays the image in the first width display mode having the first vanishing point VP10 when it is determined that the area ahead (surroundings) of the vehicle 1 is dark (an example of a first width display process).

[0065] In addition, in some embodiments, the HUD device 20 (display control device 30) changes the display mode of the perspective virtual image V to the above-mentioned first width display mode having a first vanishing point VP10, or the above-mentioned second width display mode having a third vanishing point VP30 set farther away from the first vanishing point VP10, depending on the length of the perspective virtual image V in the up-down direction (Y-axis direction), which is the depth direction (Z-axis direction) as perceived by humans. For example, in some embodiments, the HUD device 20 (display control device 30) displays (40) the length of the perspective virtual image V in the up-down direction (Y-axis direction), which is the distance in the depth direction (Z-axis direction) as perceived by humans, in a second width display manner having a third vanishing point VP30 (an example of a second width display processing), and (45) displays (45) the length of the perspective virtual image V in the up-down direction (Y-axis direction), which is the distance in the depth direction (Z-axis direction) as perceived by humans, in a first width display manner having a first vanishing point VP10 (an example of a first width display processing).

[0066] In addition, the HUD device 20 (display control device 30) of some embodiments changes the display mode of the perspective virtual image V to the first width display mode having the first vanishing point VP10 or the second width display mode having the third vanishing point VP30 set on the far side of the first vanishing point VP10, depending on the length of the period during which the display continues. For example, the HUD device 20 (display control device 30) of some embodiments (50) displays the perspective virtual image V of a moving image or still image in the second width display mode having the third vanishing point VP30 when the period from the start of displaying the perspective virtual image V of the moving image or still image to the end of displaying the perspective virtual image V is equal to or longer than a predetermined threshold value (an example of a second width display process), and (55) displays the perspective virtual image V of a moving image or still image in the first width display mode having the first vanishing point VP10 when the period from the start of displaying the perspective virtual image V of the moving image or still image to the end of displaying the perspective virtual image V of the moving image or still image is shorter than a predetermined threshold value (an example of a first width display process).

[0067] In addition, the HUD device 20 (display control device 30) of some embodiments may change the display mode in the width direction of the perspective virtual image V from the first width display mode having the first vanishing point VP10 to the second width display mode having the third vanishing point VP30 set on the far side of the first vanishing point VP10, and may change the display mode in the thickness direction of the perspective virtual image V from the first thickness display mode having the second vanishing point VP20 to the second width display mode having a fourth vanishing point (not shown) set on the far side of the second vanishing point VP20. Here, the first vanishing point VP10 and the second vanishing point VP20 may coincide with each other, but are not limited thereto. The second vanishing point VP20 may be above the first vanishing point VP10 and below the real scene vanishing point VP0 as seen by the observer. In addition, the third vanishing point (also called the thickness vanishing point or the first thickness vanishing point) VP30 and the fourth vanishing point (not shown, also called the second thickness vanishing point) may coincide, but are not limited to this. The fourth vanishing point (not shown) may be above the third vanishing point VP30 and below the actual scene vanishing point VP0 as seen by the observer.

[0068] FIG. 9 is a block diagram of a vehicle virtual image display system, according to some embodiments. The display controller 30 includes one or more I / O interfaces 31, one or more processors 33, one or more image processing circuits 35, and one or more memories 37. The various functional blocks described in FIG. 9 may be configured in hardware, software, or a combination of both. FIG. 9 is just one embodiment, and the components shown may be combined into fewer components, or there may be additional components. For example, the image processing circuit 35 (e.g., a graphics processing unit) may be included in one or more processors 33.

[0069] As shown, the processor 33 and the image processing circuit 35 are operatively coupled to the memory 37. More specifically, the processor 33 and the image processing circuit 35 can execute a program stored in the memory 37 to control the vehicle virtual image display system 10 (the display 40), for example, to generate and / or transmit image data. The processor 33 and / or the image processing circuit 35 can include at least one general-purpose microprocessor (e.g., a central processing unit (CPU)), at least one application specific integrated circuit (ASIC), at least one field programmable gate array (FPGA), or any combination thereof. The memory 37 can include any type of magnetic medium, such as a hard disk, any type of optical medium, such as a CD and a DVD, any type of semiconductor memory, such as a volatile memory, and a non-volatile memory. The volatile memory can include DRAM and SRAM, and the non-volatile memory can include ROM and NVRAM.

[0070] As shown in the figure, the processor 33 is operably connected to the I / O interface 31. The I / O interface 31 communicates with, for example, a vehicle ECU 401 (described later) and / or other electronic devices (reference numerals 403 to 419 described later) provided in the vehicle in accordance with the CAN (Controller Area Network) standard (also referred to as CAN communication). Note that the communication standard adopted by the I / O interface 31 is not limited to CAN, and may include, for example, wired communication interfaces such as CANFD (CAN with Flexible Data Rate), LIN (Local Interconnect Network), Ethernet (registered trademark), MOST (Media Oriented Systems Transport: MOST is a registered trademark), UART, or USB, or in-vehicle communication (internal communication) interfaces that are short-range wireless communication interfaces within several tens of meters, such as a personal area network (PAN) such as a Bluetooth (registered trademark) network, a local area network (LAN) such as an 802.11x Wi-Fi (registered trademark) network, etc. In addition, the I / O interface 31 may include an external communication (external communication) interface such as a wide area communication network (e.g., an Internet communication network) based on cellular communication standards such as a wireless wide area network (WWAN0, IEEE802.16-2004 (WiMAX: Worldwide Interoperability for Microwave Access)), IEEE802.16e-based (Mobile WiMAX), 4G, 4G-LTE, LTE Advanced, and 5G.

[0071] As shown in the figure, the processor 33 is interoperably connected to the I / O interface 31, thereby enabling information to be exchanged with various other electronic devices connected to the vehicle virtual image display system 10 (I / O interface 31). For example, a vehicle ECU 401, a road information database 403, a vehicle position detection unit 405, an operation detection unit 407, an eye position detection unit 409, an outside sensor 411, a brightness detection unit 413, an IMU 415, a mobile information terminal 417, and an external communication device 419 are operably connected to the I / O interface 31. The I / O interface 31 may include a function of processing (converting, calculating, analyzing) information received from other electronic devices connected to the vehicle virtual image display system 10.

[0072] The display 40 is operatively coupled to the processor 33 and the image processing circuitry 35. Thus, the image displayed by the light modulation elements 50 may be based on image data received from the processor 33 and / or the image processing circuitry 35. The processor 33 and the image processing circuitry 35 control the image displayed by the light modulation elements 50 based on information obtained from the I / O interface 31.

[0073] The vehicle ECU 401 obtains the status of the host vehicle 1 (e.g., mileage, vehicle speed, accelerator pedal opening, brake pedal opening, engine throttle opening, injector fuel injection amount, engine RPM, motor RPM, steering angle, shift position, drive mode, various warning states, posture (including roll angle and / or pitching angle), vehicle vibration (including vibration magnitude, frequency, and / or frequency)) from sensors and switches installed in the host vehicle 1, and collects and manages (which may also include control) the said status of the host vehicle 1, and as part of its function, can output a signal indicating a numerical value of the said status of the host vehicle 1 (e.g., the vehicle speed of the host vehicle 1) to the processor 33 of the display control device 30. In addition to or instead of simply transmitting a value detected by a sensor or the like (e.g., the pitching angle is 3 degrees in the forward tilt direction) to the processor 33, the vehicle ECU 401 may transmit to the processor 33 a determination result based on one or more states of the host vehicle 1 including the value detected by the sensor (e.g., that the host vehicle 1 satisfies a predetermined condition for a forward tilt state) and / or an analysis result (e.g., that the vehicle has been in a forward tilt state due to braking in combination with information on the brake pedal opening degree). For example, the vehicle ECU 401 may output to the display control device 30 a signal indicating a determination result that the host vehicle 1 satisfies a predetermined condition previously stored in a memory (not shown) of the vehicle ECU 401. The I / O interface 31 may acquire the above-mentioned information from a sensor or a switch provided on the host vehicle 1 without going through the vehicle ECU 401.

[0074] In addition, the vehicle ECU 401 may output an instruction signal to the display control device 30 to indicate the image to be displayed by the vehicle virtual image display system 10, and at this time, the coordinates, size, type, display mode, notification necessity of the image, and / or necessity-related information that serves as the basis for determining the notification necessity of the image may be added to the instruction signal and transmitted.

[0075] The road information database 403 is included in a navigation device (not shown) provided in the vehicle 1, or an external server connected to the vehicle 1 via an external communication interface (I / O interface 31), and may read out information about the surroundings of the vehicle 1 (information related to real objects around the vehicle 1), such as road information (lanes, white lines, stop lines, pedestrian crossings, road widths, number of lanes, intersections, curves, forks, traffic regulations, etc.) on which the vehicle 1 is traveling, and information about features (buildings, bridges, rivers, etc.), such as presence or absence, position (including distance to the vehicle 1), direction, shape, type, detailed information, etc.), and transmit the information to the processor 33. The road information database 403 may also calculate an appropriate route (navigation information) from a departure point to a destination, and output a signal indicating the navigation information or image data indicating the route to the processor 33.

[0076] The vehicle position detection unit 405 is a GNSS (Global Navigation Satellite System) or the like provided in the vehicle 1, which detects the current position and direction of the vehicle 1, and outputs a signal indicating the detection result via the processor 33 or directly to the road information database 403, a mobile information terminal 417 described later, and / or an external communication device 419. The road information database 403, the mobile information terminal 417 described later, and / or the external communication device 419 may select / generate information about the surroundings of the vehicle 1 by acquiring the position information of the vehicle 1 from the vehicle position detection unit 405 continuously, intermittently, or for each predetermined event, and output the information to the processor 33.

[0077] The operation detection unit 407 is, for example, a hardware switch provided on a CID (Center Information Display) or an instrument panel of the vehicle 1, or a software switch combining an image and a touch sensor, and outputs operation information based on an operation by an occupant of the vehicle 1 (a user sitting in the driver's seat and / or a user sitting in the passenger seat) to the processor 33. For example, the operation detection unit 407 outputs to the processor 33 display area setting information based on an operation to move the virtual image display area 100 by a user's operation, eye box setting information based on an operation to move the eye box 200, information based on an operation to set the eye position 700 of the observer, and the like.

[0078] Eye position detection unit 409 may include a camera such as an infrared camera that detects eye position 700 (see FIG. 1) of an observer sitting in the driver's seat of host vehicle 1, and may output a captured image to processor 33. Processor 33 may obtain a captured image (one example of information from which eye position 700 can be estimated) from eye position detection unit 409, and analyze the captured image using a method such as pattern matching to detect the coordinates of observer's eye position 700, and output a signal indicating the coordinates of the detected eye position 700 to processor 33.

[0079] Furthermore, eye position detection unit 409 may output an analysis result of analyzing the image captured by the camera (for example, a signal indicating to which spatial region the observer's eye position 700 belongs in which a plurality of preset display parameters correspond) to processor 33. Note that the method of acquiring the observer's eye position 700 of the host vehicle 1 or information that allows the observer's eye position 700 to be estimated is not limited to these, and the information may be acquired using a known eye position detection (estimation) technique.

[0080] Furthermore, the eye position detection section 409 may detect the moving speed and / or moving direction of the viewer's eye position 700, and output a signal indicating the moving speed and / or moving direction of the viewer's eye position 700 to the processor 33.

[0081] In addition, when the eye position detection unit 409 detects (10) a signal indicating that the observer's eye position 700 is outside the eye box 200, (20) a signal that estimates that the observer's eye position 700 is outside the eye box 200, or (30) a signal that predicts that the observer's eye position 700 will be outside the eye box 200, it may determine that a specified condition is satisfied and output a signal indicating that state to the processor 33.

[0082] (20) Signals indicating that the observer's eye position 700 is outside the eye box 200 include (21) a signal indicating that the observer's eye position 700 cannot be detected, (22) a signal indicating that the observer's eye position 700 cannot be detected after movement of the observer's eye position 700 has been detected, and / or (23) a signal indicating that either of the observer's eye positions 700R, 700L is in the vicinity of the boundary 200A of the eye box 200 (the vicinity includes, for example, within a specified coordinate range from the boundary 200A).

[0083] (30) Signals predicting that the observer's eye position 700 will be outside the eye box 200 include (31) a signal indicating that the newly detected eye position 700 is equal to or greater than an eye position movement distance threshold pre-stored in memory 37 compared to the previously detected eye position 700 (the movement of the eye position within a specified unit time is greater than a specified range), and (32) a signal indicating that the movement speed of the eye position is equal to or greater than an eye position movement speed threshold pre-stored in memory 37.

[0084] The eye position detection unit 409 may also have a function as the gaze direction detection unit 409. The gaze direction detection unit 409 may include an infrared camera or a visible light camera that captures an image of the face of an observer sitting in the driver's seat of the vehicle 1, and may output the captured image to the processor 33. The processor 33 can specify the gaze direction (and / or the gaze position) of the observer by acquiring an image (one example of information that can estimate the gaze direction) from the gaze direction detection unit 409 and analyzing the captured image. The gaze direction detection unit 409 may analyze an image captured by a camera and output a signal indicating the gaze direction (and / or the gaze position) of the observer, which is the analysis result, to the processor 33. The method of acquiring information that can estimate the gaze direction of the observer of the vehicle 1 is not limited to these, and may be acquired using other known gaze direction detection (estimation) techniques such as an EOG (Electro-oculogram) method, a corneal reflection method, a scleral reflection method, a Purkinje image detection method, a search coil method, and an infrared fundus camera method.

[0085] The external sensor 411 detects real objects present around the vehicle 1 (in front, on the sides, and behind). The real objects detected by the external sensor 411 may include, for example, obstacles (pedestrians, bicycles, motorcycles, other vehicles, etc.), the road surface of a driving lane, dividing lines, roadside objects, and / or features (buildings, etc.) described below. The external sensor is composed of, for example, a detection unit made up of a radar sensor such as a millimeter wave radar, ultrasonic radar, or laser radar, a camera, or a combination of these, and a processing device that processes (data fuses) detection data from the one or more detection units. Conventional well-known methods are applied to object detection by these radar sensors and camera sensors. By object detection by these sensors, the presence or absence of a real object in a three-dimensional space, and if a real object exists, the position (relative distance from the vehicle 1, the left-right position when the traveling direction of the vehicle 1 is the forward-rearward direction, the up-down position, etc.), size (size in the lateral direction (left-right direction), height direction (up-down direction), etc.), moving direction (lateral direction (left-right direction), depth direction (front-rear direction)), moving speed (lateral direction (left-right direction), depth direction (front-rear direction)), and / or type, etc. of the real object may be detected. One or more external sensors 411 can detect a real object in front of the vehicle 1 at each detection period of each sensor, and output real object information (presence or absence of a real object, and if a real object exists, information on the position, size, and / or type of each real object, etc.) which is an example of real object information, to the processor 33. Note that the real object information may be transmitted to the processor 33 via another device (for example, the vehicle ECU 401). In addition, when a camera is used as a sensor, an infrared camera or a near-infrared camera is preferable so that a real object can be detected even when the surroundings are dark, such as at night. Furthermore, when using a camera as a sensor, a stereo camera that can obtain distance and the like from parallax is desirable.

[0086] The brightness detection unit 413 detects the illuminance or brightness of a predetermined range of the foreground present in front of the vehicle interior as the outside brightness (an example of brightness information), or detects the illuminance or brightness inside the vehicle interior as the inside vehicle brightness (an example of brightness information). The brightness detection unit 413 is, for example, a phototransistor or a photodiode, and is mounted on the instrument panel, the rearview mirror, the HUD device 20, or the like of the vehicle 1 shown in FIG.

[0087] The IMU 415 may include a combination of one or more sensors (e.g., an accelerometer and a gyroscope) configured to detect the position, orientation, and changes therein (speed of change, acceleration of change) of the host vehicle 1 based on inertial acceleration. The IMU 415 may output the detected values ​​(the detected values ​​include signals indicating the position, orientation, and changes therein (speed of change, acceleration of change) of the host vehicle 1, etc.) and the results of analyzing the detected values ​​to the processor 33. The results of the analysis may be signals indicating a determination result as to whether or not the detected values ​​satisfy a predetermined condition, and may be, for example, a signal indicating that the behavior (vibration) of the host vehicle 1 is small based on a value regarding the change (speed of change, acceleration of change) of the position or orientation of the host vehicle 1.

[0088] The mobile information terminal 417 is a smartphone, a notebook computer, a smartwatch, or other information device that can be carried by an observer (or another passenger of the vehicle 1). The I / O interface 31 can communicate with the mobile information terminal 417 by pairing with the mobile information terminal 417, and acquires data recorded in the mobile information terminal 417 (or a server via the mobile information terminal). The mobile information terminal 417 has, for example, the same functions as the above-mentioned road information database 403 and the vehicle position detection unit 405, and may acquire the road information (an example of real object related information) and transmit it to the processor 33. The mobile information terminal 417 may also acquire commercial information (an example of real object related information) related to commercial facilities in the vicinity of the vehicle 1 and transmit it to the processor 33. In addition, the mobile information terminal 417 may transmit schedule information of the holder of the mobile information terminal 417 (e.g., the observer), information on incoming calls at the mobile information terminal 417, information on received emails, etc. to the processor 33, and the processor 33 and the image processing circuit 35 may generate and / or transmit image data related to these.

[0089] The external communication device 419 is a communication device that exchanges information with the vehicle 1, and is, for example, another vehicle connected to the vehicle 1 by vehicle-to-vehicle communication (V2V: Vehicle To Vehicle), a pedestrian (a mobile information terminal carried by a pedestrian) connected to the vehicle 1 by vehicle-to-pedestrian communication (V2P: Vehicle To Pedestrian), and a network communication device connected by vehicle-to-roadside communication (V2I: Vehicle To Roadside Infrastructure), and in a broad sense, includes everything connected to the vehicle 1 by communication (V2X: Vehicle To Everything). The external communication device 419 may obtain the positions of, for example, pedestrians, bicycles, motorcycles, other vehicles (such as a preceding vehicle), road surfaces, dividing lines, roadside objects, and / or features (such as buildings), and output them to the processor 33. Moreover, the external communication device 419 has the same function as the vehicle position detection unit 405 described above, and may acquire position information of the vehicle 1 and transmit it to the processor 33, and may also have the function of the road information database 403 described above, and may acquire the road information (an example of real object related information) and transmit it to the processor 33. Note that the information acquired from the external communication device 419 is not limited to the above.

[0090] The software components stored in the memory 37 include a display parameter setting module 502 , a graphics module 504 , a light source driving module 506 , and an actuator driving module 508 .

[0091] 10 is a flow diagram showing a method S100 of executing a display control process for displaying a perspective virtual image in a first display mode (first width display mode, first thickness display mode) or a second display mode (second width display mode, second thickness display mode). The method S100 is executed in a HUD device 20 including a display and a display control device 30 that controls the HUD device 20. Some operations in the method S100 are optionally combined, the procedures of some operations are optionally changed, and some operations are optionally omitted.

[0092] The display control device 30 (graphics module 504 described later) acquires, for example, an appropriate route (navigation information) from the road information database 403 to the destination, real object information (presence or absence of real objects, and if real objects exist, information such as the position, size, and / or type of each real object) from the external sensor 411, and selects and sets the virtual objects to be displayed (S110).

[0093] As shown in S131, the display parameter setting module 502 changes the display mode of the perspective virtual image V based on one or more pieces of information acquired from the I / O interface 31. The display parameter setting module 502 (11) sets the perspective virtual image V to a second width display mode having a third vanishing point VP30 (an example of a second width display process) when the perspective virtual image V is set to a moving image that moves or expands in the perspective direction, and (12) sets the perspective virtual image V to a first width display mode having a first vanishing point VP10 (an example of a first width display process) when the perspective virtual image V is set to a still image, a moving image that does not move much in the perspective direction, or a moving image that does not expand or contract much in the perspective direction, based on one or more pieces of information acquired from the I / O interface 31. In addition, in some embodiments, based on one or more pieces of information acquired from the I / O interface 31, the display parameter setting module 502 (16) sets the perspective virtual image V to a second thickness display mode having a fourth vanishing point (not shown) (an example of a second thickness display process) when the perspective virtual image V is set to a moving image that moves or stretches in the perspective direction, and (17) sets the perspective virtual image V to a first thickness display mode having a second vanishing point VP20 (an example of a first thickness display process) when the perspective virtual image V is set to a still image, a moving image that does not move much in the perspective direction, or a moving image that does not stretch much in the perspective direction.

[0094] Furthermore, as shown in S132, based on one or more pieces of information acquired from the I / O interface 31, the display parameter setting module 502 (21) sets the moving speed of the perspective virtual image V animation to a second width display mode having a third vanishing point VP30 (an example of a second width display processing) when the moving speed of the perspective virtual image V animation is set to be faster than a predetermined threshold, and (22) sets the moving speed of the perspective virtual image V animation to a first width display mode having a first vanishing point VP10 (an example of a first width display processing) when the moving speed of the perspective virtual image V animation is set to be faster than a predetermined threshold. In addition, in some embodiments, the display parameter setting module 502, as shown in S132, based on one or more pieces of information acquired from the I / O interface 31, (26) if the movement speed of the moving image of the perspective virtual image V is set faster than a predetermined threshold, sets it to a second thickness display mode having a fourth vanishing point (not shown) (an example of a second thickness display processing), and (27) if the movement speed of the moving image of the perspective virtual image V is set faster than a predetermined threshold, sets it to a first thickness display mode having a second vanishing point VP20 (an example of a first thickness display processing).

[0095] Furthermore, as shown in S133, based on information (but not limited to this) acquired from the brightness detection unit 413, the display parameter setting module 502 sets the perspective virtual image V to a second width display mode having a third vanishing point VP30 (an example of a second width display processing) if it is determined that the area ahead (surroundings) of the vehicle 1 is bright, and sets the perspective virtual image V to a first width display mode having a first vanishing point VP10 (an example of a first width display processing) if it is determined that the area ahead (surroundings) of the vehicle 1 is dark. In addition, in some embodiments, as shown in S133, based on information (but not limited to this) obtained from the brightness detection unit 413, the display parameter setting module 502 sets the perspective virtual image V to a second thickness display mode having a fourth vanishing point (not shown) (an example of a second thickness display processing) if it is determined that the area in front of (surrounding) the host vehicle 1 is bright, and sets the perspective virtual image V to a first thickness display mode having a second vanishing point VP20 (an example of a first thickness display processing) if it is determined that the area in front of (surrounding) the host vehicle 1 is dark.

[0096] Further, as shown in S134, based on road information acquired from the road information database 403 and position information (but not limited to) of a real object existing in the foreground acquired from the outside sensor 411, the display parameter setting module 502 sets (41) the perspective virtual image V to a second width display mode having a third vanishing point VP30 if the length in the up-down direction (Y-axis direction) of the perspective virtual image V indicating the guide route or a real object existing in the foreground, which is a distance in the depth direction (Z-axis direction) as perceived by humans, is the same as or longer than a predetermined threshold value pre-stored in the memory 37 (an example of a second width display processing), and (42) sets the perspective virtual image V to a first width display mode having a first vanishing point VP10 if the length in the up-down direction (Y-axis direction) of the perspective virtual image V, which is a distance in the depth direction (Z-axis direction) as perceived by humans, is shorter than a predetermined threshold value (an example of a first width display processing). In some embodiments, as shown in S134, based on road information acquired from the road information database 403 and position information (but not limited to) of a real object in the foreground acquired from the outside sensor 411, the display parameter setting module 502 sets the perspective virtual image V to a second thickness display mode having a fourth vanishing point (not shown) if the length in the up-down direction (Y-axis direction) of the perspective virtual image V indicating the guide route or a real object in the foreground, which is a distance in the depth direction (Z-axis direction) as perceived by humans, is the same as or longer than a predetermined threshold value pre-stored in the memory 37 (an example of a second thickness display process), and sets the perspective virtual image V to a first thickness display mode having a second vanishing point VP20 if the length in the up-down direction (Y-axis direction) of the perspective virtual image V, which is a distance in the depth direction (Z-axis direction) as perceived by humans, is shorter than a predetermined threshold value (an example of a first thickness display process).

[0097] Furthermore, as shown in S135, based on one or more pieces of information acquired from the I / O interface 31, (51) if the period from the start to the end of the display of the perspective virtual image V of a moving image or still image is the same as or longer than a predetermined threshold pre-stored in the memory 37, the display parameter setting module 502 sets the perspective virtual image V to a second width display mode having a third vanishing point VP30 (an example of a second width display processing), and (52) if the period from the start to the end of the display of the perspective virtual image V of a moving image or still image is shorter than a predetermined threshold pre-stored in the memory 37, the display parameter setting module 502 sets the perspective virtual image V to a first width display mode having a first vanishing point VP10 (an example of a first width display processing). In addition, in some embodiments, as shown in S135, based on one or more pieces of information acquired from the I / O interface 31, the display parameter setting module 502 sets (56) the perspective virtual image V to a second thickness display mode having a fourth vanishing point (not shown) if the period from when the perspective virtual image V of a moving image or still image starts to when it ends is the same as or longer than a predetermined threshold pre-stored in the memory 37 (an example of a second thickness display processing), and (57) sets the perspective virtual image V to a first thickness display mode having a second vanishing point VP20 if the period from when the perspective virtual image V of a moving image or still image starts to when it ends is shorter than a predetermined threshold pre-stored in the memory 37 (an example of a first thickness display processing).

[0098] The graphics module 504 includes various known software components for generating image data by performing image processing such as rendering, and driving the display 40. The graphics module 504 may also include various known software components for changing the type (moving image, still image, shape), arrangement (position coordinates, angle), size, display distance (in the case of 3D), and visual effects (e.g., brightness, transparency, saturation, contrast, or other visual characteristics) of the image to be displayed. The graphics module 504 can generate image data to be viewed by an observer in terms of the type of image (one example of a display parameter), the position coordinates of the image (one example of a display parameter), the angle of the image (a pitching angle about the X-direction axis, a yaw rate angle about the Y-direction axis, a rolling angle about the Z-direction axis, etc., one example of a display parameter), the size of the image (one example of a display parameter), the color of the image (one example of a display parameter set by hue, saturation, brightness, etc.), and the intensity of the perspective representation of the image (one display parameter set by the position of the vanishing point, etc.), and drive the light modulation element 50.

[0099] The light source driving module 506 includes various known software components for executing driving of the light source unit 24. The light source driving module 506 may drive the light source unit 24 based on the set display parameters.

[0100] The actuator driving module 508 includes various known software components for executing the actuation of the first actuator 28 and / or the second actuator 29. The actuator driving module 508 may drive the first actuator 28 and the second actuator 29 based on set display parameters.

[0101] The eye position detection module 510 detects the observer's eye position 700. The eye position detection module 510 includes various software components for performing various operations related to detecting coordinates indicating the observer's eye position 700 (positions in the X and Y axes directions, which are an example of a signal indicating the eye position 700), coordinates indicating the observer's eye height (position in the Y axis direction, which is an example of a signal indicating the eye position 700), detecting coordinates indicating the observer's eye height and depth position (positions in the Y and Z axes directions, which are an example of a signal indicating the eye position 700), and / or detecting coordinates indicating the observer's eye position 700 (positions in the X, Y, and Z axes directions, which is an example of a signal indicating the eye position 700).

[0102] The positions of the first vanishing point VP10, the second vanishing point VP20, the third vanishing point VP30, and the fourth vanishing point (not shown) may be adjusted by the viewer's eye position 700 detected by the eye position detection unit 409, or the attitude of the host vehicle 1 detected by the IMU 415. For example, the display control device 30 (display parameter setting module 502) may adjust the positions of the first vanishing point VP10, the second vanishing point VP20, the third vanishing point VP30, and the fourth vanishing point (not shown) in response to a change in the attitude (pitching angle (angle about the X-direction as an axis)) of the host vehicle 1. When the pitching angle is such that the front of the vehicle 1 descends, the display control device 30 (display parameter setting module 502) adjusts the relative positions of the first vanishing point VP10, the second vanishing point VP20, the third vanishing point VP30, and the fourth vanishing point (not shown) to be on the upper side as seen by the observer in the case where the pitching angle is such that the front of the vehicle 1 ascends, and can adjust the relative positions of the first vanishing point VP10, the second vanishing point VP20, the third vanishing point VP30, and the fourth vanishing point (not shown) to be on the lower side as seen by the observer in the case where the pitching angle is such that the front of the vehicle 1 ascends.

[0103] The operations of the above-mentioned processing processes can be implemented by executing one or more functional modules of an information processing device, such as a general-purpose processor or an application-specific chip. These modules, combinations of these modules, and / or combinations with known hardware that can replace their functions, are all included in the scope of protection of the present invention.

[0104] The functional blocks of the vehicle virtual image display system 10 are optionally implemented by hardware, software, or a combination of hardware and software to implement the principles of the various embodiments described. It will be understood by those skilled in the art that the functional blocks described in FIG. 9 may be optionally combined or one functional block may be separated into two or more sub-blocks to implement the principles of the embodiments described. Therefore, the description in this specification optionally supports any possible combination or division of the functional blocks described in this specification. [Explanation of symbols]

[0105] 1: Vehicle 2:Projected member 5: Dashboard 6: Road surface 6: Driving lane (road surface) 7: Border 8: Border 10: Virtual image display system for vehicles 20: HUD device (head-up display device) 21: Light exit window 22: Housing 24: Light source unit 28: First actuator 29: Second actuator 30: Display control device 31: I / O interface 33: Processor 35: Image processing circuit 37: Memory 40:Display unit 50: Light modulation element 51:Display surface 60: Light source unit 80: Relay optical system 81: First mirror 82: 2nd mirror 86: Actuator 87: Actuator 90: Virtual image optical system 100: Virtual image display area 200: Eye Box 200A: Boundary 205: Center 401: Vehicle ECU 403: Road information database 405: Vehicle position detection unit 407: Operation detection unit 409: Eye position detection unit (gaze direction detection unit) 411: External sensor 413: Brightness detection section 417: Mobile information terminal 419: External communication equipment 502: Display parameter setting module 504: Graphics module 506: Light source driving module 508: Actuator drive module 510: Eye position detection module 700: Observer 700L: Eye position (left eye) 700R: Eye position (right eye) 1000: Virtual image display area 1010: Driving lane 1011: Left line 1012: Right line 1020 :Vanishing point 1100 :Perspective image 1101: Top 1102: Bottom 1111: First thickness part 1112: Second thickness part 1120: Thickness 1121: First thickness 1122: Second thickness 1300: far end 1400: Near end AX1: First rotation axis FU: Virtual Object FU10: Virtual Objects K:Display light K11: 1st display light K12: 2nd display light K13: 3rd display light Kp: Optical axis PT: Target position V: Image V: Virtual image V10: Perspective Virtual Image V1A: Extension line (left perspective line) V1B: Extension line (right perspective line) V1C: Extension line (top) V1D: Extension line (bottom) V20: Perspective Virtual Image VP0: Real scene vanishing point VP0: Vanishing point VP0': Vanishing point VP10: 1st vanishing point (width vanishing point, 1st width vanishing point) VP20: 2nd vanishing point (thickness vanishing point, 1st thickness vanishing point) VP30: 3rd vanishing point (2nd width vanishing point) VS: Virtual image display area VS1: Upper end VS2: Bottom end VT: Thickness VT1: First thickness VT2: Second thickness VT3: The third thickness VT4: 4th thickness VT5: Thickness VT6: Thickness VY1: First thickness VY2: Second thickness VY3: Thickness WS: Front windshield θd: Vertical arrangement angle θt: Tilt angle

Claims

1. A display control device (30) that performs display control in a head-up display device that projects an image onto a projection target member, thereby allowing an observer to visually recognize a virtual image of the image superimposed on a foreground, comprising: one or more processors (33); A memory (37); one or more computer programs stored in the memory (37) and configured to be executed by the one or more processors (33); The processor (33) execute a thickness display process to display one or more perspective virtual images having a thickness (VT) in the vertical direction as seen by the observer and with a depth expression that gradually decreases so that the thickness (VT) converges to a thickness vanishing point (VP20) so that the observer can visually recognize one or more perspective images that are displayed as if they are arranged parallel to a road surface; execute a width display process to display one or more perspective virtual images having a width in the left-right direction as viewed from the observer, the width being gradually reduced so as to converge to a width vanishing point (VP10); the thickness vanishing point (VP20) and the width vanishing point (VP10) are located below the foreground vanishing point (VP0) as viewed from the observer, In the thickness display process, the thickness vanishing point (VP20) is disposed above the width vanishing point (V10) and below the foreground vanishing point (VP0) as viewed by an observer. A display control device (30).

2. A display control device (30) that performs display control in a head-up display device that projects an image onto a projection target member, thereby allowing an observer to visually recognize a virtual image of the image superimposed on a foreground, comprising: one or more processors (33); A memory (37); one or more computer programs stored in the memory (37) and configured to be executed by the one or more processors (33); The processor (33) execute a thickness display process to display one or more perspective virtual images having a thickness (VT) in the vertical direction as seen by the observer and with a depth expression that gradually decreases so that the thickness (VT) converges to a thickness vanishing point (VP20) so that the observer can visually recognize one or more perspective images that are displayed as if they are arranged parallel to a road surface; The thickness vanishing point (VP20) is disposed below the foreground vanishing point (VP0) as viewed from the observer, a first thickness display process for displaying the one or more perspective virtual images in a first thickness display mode having a first thickness vanishing point that is located below a vanishing point (VP0) of the foreground as viewed from the observer; a second thickness display process for displaying as the one or more perspective virtual images in a second thickness display mode having a second thickness vanishing point disposed above the first thickness vanishing point as viewed from the observer, If the perspective virtual image is a still image, the first thickness display process is executed; If the perspective virtual image is a moving image, the second thickness display process is executed. A display control device (30).

3. A display control device (30) that performs display control in a head-up display device that projects an image onto a projection target member, thereby allowing an observer to visually recognize a virtual image of the image superimposed on a foreground, comprising: one or more processors (33); A memory (37); one or more computer programs stored in the memory (37) and configured to be executed by the one or more processors (33); The processor (33) execute a thickness display process to display one or more perspective virtual images having a thickness (VT) in the vertical direction as seen by the observer and with a depth expression that gradually decreases so that the thickness (VT) converges to a thickness vanishing point (VP20) so that the observer can visually recognize one or more perspective images that are displayed as if they are arranged parallel to a road surface; The thickness vanishing point (VP20) is disposed below the foreground vanishing point (VP0) as viewed from the observer, a first thickness display process for displaying the one or more perspective virtual images in a first thickness display mode having a first thickness vanishing point that is located below a vanishing point (VP0) of the foreground as viewed from the observer; a second thickness display process for displaying as the one or more perspective virtual images in a second thickness display mode having a second thickness vanishing point disposed above the first thickness vanishing point as viewed from the observer, If the perspective virtual image is a moving image that does not expand or contract in a perspective direction as viewed by the observer, or a moving image that does not move in a perspective direction, the first thickness display process is executed; executes the second thickness display process if the perspective virtual image is a moving image that is perceived as expanding or contracting in a perspective direction or moving in a perspective direction when viewed by the observer. A display control device (30).

4. A display control device (30) that performs display control in a head-up display device that projects an image onto a projection target member, thereby allowing an observer to visually recognize a virtual image of the image superimposed on a foreground, comprising: one or more processors (33); A memory (37); one or more computer programs stored in the memory (37) and configured to be executed by the one or more processors (33); The processor (33) execute a thickness display process to display one or more perspective virtual images having a thickness (VT) in the vertical direction as seen by the observer and with a depth expression that gradually decreases so that the thickness (VT) converges to a thickness vanishing point (VP20) so that the observer can visually recognize one or more perspective images that are displayed as if they are arranged parallel to a road surface; The thickness vanishing point (VP20) is disposed below the foreground vanishing point (VP0) as viewed from the observer, a first thickness display process for displaying the one or more perspective virtual images in a first thickness display mode having a first thickness vanishing point that is located below a vanishing point (VP0) of the foreground as viewed from the observer; a second thickness display process for displaying as the one or more perspective virtual images in a second thickness display mode having a second thickness vanishing point disposed above the first thickness vanishing point as viewed from the observer, When the surroundings are dark, the first thickness display process is executed; When the surroundings are bright, the second thickness display process is executed. A display control device (30).

5. A display control device (30) that performs display control in a head-up display device that projects an image onto a projection target member, thereby allowing an observer to visually recognize a virtual image of the image superimposed on a foreground, comprising: one or more processors (33); A memory (37); one or more computer programs stored in the memory (37) and configured to be executed by the one or more processors (33); The processor (33) execute a thickness display process to display one or more perspective virtual images having a thickness (VT) in the vertical direction as seen by the observer and with a depth expression that gradually decreases so that the thickness (VT) converges to a thickness vanishing point (VP20) so that the observer can visually recognize one or more perspective images that are displayed as if they are arranged parallel to a road surface; The thickness vanishing point (VP20) is disposed below the foreground vanishing point (VP0) as viewed from the observer, a first thickness display process for displaying the one or more perspective virtual images in a first thickness display mode having a first thickness vanishing point that is located below a vanishing point (VP0) of the foreground as viewed from the observer; a second thickness display process for displaying as the one or more perspective virtual images in a second thickness display mode having a second thickness vanishing point disposed above the first thickness vanishing point as viewed from the observer, When the moving speed or the deformation speed of the perspective virtual image is slow, the first thickness display process is executed; When the moving speed or the deformation speed of the perspective virtual image is fast, the second thickness display process is executed. A display control device (30).

6. A display control device (30) that performs display control in a head-up display device that projects an image onto a projection target member, thereby allowing an observer to visually recognize a virtual image of the image superimposed on a foreground, comprising: one or more processors (33); A memory (37); one or more computer programs stored in the memory (37) and configured to be executed by the one or more processors (33); The processor (33) execute a thickness display process to display one or more perspective virtual images having a thickness (VT) in the vertical direction as seen by the observer and with a depth expression that gradually decreases so that the thickness (VT) converges to a thickness vanishing point (VP20) so that the observer can visually recognize one or more perspective images that are displayed as if they are arranged parallel to a road surface; The thickness vanishing point (VP20) is disposed below the foreground vanishing point (VP0) as viewed from the observer, a first thickness display process for displaying the one or more perspective virtual images in a first thickness display mode having a first thickness vanishing point that is located below a vanishing point (VP0) of the foreground as viewed from the observer; a second thickness display process for displaying as the one or more perspective virtual images in a second thickness display mode having a second thickness vanishing point disposed above the first thickness vanishing point as viewed from the observer, When a period from when the perspective virtual image starts to be displayed until when the perspective virtual image ends is shorter than a predetermined threshold value, the first thickness display process is executed; executes the second thickness display process when a period from when the display of the perspective virtual image starts to when the display of the perspective virtual image ends is longer than the predetermined threshold value. A display control device (30).

7. A display control device (30) according to any one of claims 1 to 6, A light modulation element (50) that emits display light; A head-up display device (20) comprising: a relay optical system (80) that directs the display light from the light modulation element (50) to a projection target.

8. An image display control method, comprising: a thickness display process is executed to display one or more perspective virtual images having a thickness (VT) in the vertical direction as viewed by the observer and with a depth expression that gradually decreases so that the thickness (VT) converges to a thickness vanishing point (VP20), so that the observer can visually recognize one or more perspective images that are displayed as if they are arranged parallel to a road surface; execute a width display process to display one or more perspective virtual images having a width in the left-right direction as viewed from the observer, the width being gradually reduced so as to converge to a width vanishing point (VP10); The thickness vanishing point (VP20) and the width vanishing point (VP10) are located below the foreground vanishing point (VP0) as seen by the observer, In the thickness display process, the thickness vanishing point (VP20) is disposed above the width vanishing point (V10) and below the foreground vanishing point (VP0) as viewed by an observer. Display control method.

9. An image display control method, comprising: a thickness display process is executed to display one or more perspective virtual images having a thickness (VT) in the vertical direction as viewed by the observer and with a depth expression that gradually decreases so that the thickness (VT) converges to a thickness vanishing point (VP20), so that the observer can visually recognize one or more perspective images that are displayed as if they are arranged parallel to a road surface; The one or more processors (33) a first thickness display process for displaying the one or more perspective virtual images in a first thickness display mode having a first thickness vanishing point that is located below a vanishing point (VP0) in the foreground as viewed from the observer; a second thickness display process for displaying as the one or more perspective virtual images in a second thickness display mode having a second thickness vanishing point disposed above the first thickness vanishing point as viewed from the observer, If the perspective virtual image is a still image, the first thickness display process is executed; If the perspective virtual image is a moving image, the second thickness display process is executed. Display control method.

10. An image display control method, comprising: a thickness display process is executed to display one or more perspective virtual images having a thickness (VT) in the vertical direction as viewed by the observer and with a depth expression that gradually decreases so that the thickness (VT) converges to a thickness vanishing point (VP20), so that the observer can visually recognize one or more perspective images that are displayed as if they are arranged parallel to a road surface; The one or more processors (33) a first thickness display process for displaying the one or more perspective virtual images in a first thickness display mode having a first thickness vanishing point that is located below a vanishing point (VP0) in the foreground as viewed from the observer; a second thickness display process for displaying as the one or more perspective virtual images in a second thickness display mode having a second thickness vanishing point disposed above the first thickness vanishing point as viewed from the observer, If the perspective virtual image is a moving image that does not expand or contract in a perspective direction as viewed by the observer, or a moving image that does not move in a perspective direction, the first thickness display process is executed; executes the second thickness display process if the perspective virtual image is a moving image that is perceived as expanding or contracting in a perspective direction or moving in a perspective direction when viewed by the observer. Display control method.

11. An image display control method, comprising: a thickness display process is executed to display one or more perspective virtual images having a thickness (VT) in the vertical direction as viewed by the observer and with a depth expression that gradually decreases so that the thickness (VT) converges to a thickness vanishing point (VP20), so that the observer can visually recognize one or more perspective images that are displayed as if they are arranged parallel to a road surface; The one or more processors (33) a first thickness display process for displaying the one or more perspective virtual images in a first thickness display mode having a first thickness vanishing point that is located below a vanishing point (VP0) in the foreground as viewed from the observer; a second thickness display process for displaying as the one or more perspective virtual images in a second thickness display mode having a second thickness vanishing point disposed above the first thickness vanishing point as viewed from the observer, When the surroundings are dark, the first thickness display process is executed; When the surroundings are bright, the second thickness display process is executed. Display control method.

Images