Display control device, head-up display device, and display control method

Abstract

To provide a display controller which displays an image that is easy-to-realize so as to be parallel to a face of a travel lane, a head-up display device, and a display control method and so on.SOLUTION: A head-up display device, which causes an observer to visually recognize a virtual image of an image while overlaying the same on a foreground, executes: first display processing for displaying as a perspective virtual image of a first display mode having a first vanishing point VP1 that is located below a vanishing point VP0 of the foreground when viewed from the observer; and second display processing for displaying as a perspective virtual image of a second display mode having a second vanishing point VP2 that is located above the first vanishing point VP1 when viewed from the observer.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] The present disclosure relates to a display control device, a head-up display device, a display control method, etc., that are used in a moving body such as a vehicle and that superimpose an image on the foreground of the moving body (the actual view in the forward direction of the moving body as seen by an occupant of the vehicle) and allow the image to be viewed.

[0002] Patent Document 1 describes a head-up display device (one example of a virtual image display device) in which display light projected onto a projection target such as the front windshield of a vehicle is reflected toward a vehicle occupant (observer) inside the vehicle, thereby allowing the observer to visually recognize a virtual image superimposed on the foreground of the vehicle. In particular, the head-up display device described in Patent Document 1 virtually places a display object (virtual image) at a predetermined position (here, the position is referred to as a target position) in the depth or in the up, down, left and right directions in the space of the foreground, and controls an image displayed inside the head-up display device as if the display object exists at the target position in the foreground, even if there is a change in the attitude of the vehicle or a change in the eye position of the observer. In other words, such a head-up display device forms an augmented reality in which a virtual object is added to and displayed in a real landscape (foreground).

[0003] In the example of FIG. 4 of Patent Document 1, a virtual object in the shape of an arrow pointing forward (in other words, an arrow extending from the rear to the front) is displayed at a position slightly above the road surface on which the vehicle is traveling (a position above the road surface). In such a case, as in the example of FIG. 5 of Patent Document 1, the observer perceives a sense of perspective as the width of the rear end of the arrow (typically the width in the left-right direction) gradually narrows toward the tip. In other words, the observer perceives the rear end of the arrow as being nearby and the tip of the arrow as being far away. A technique such as linear perspective (perspective projection) is typically used for the expression that creates such a sense of perspective. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2010-156608 A Summary of the Invention [Problem to be solved by the invention]

[0005] Incidentally, in the above-mentioned head-up display device, the area in which a virtual image can be displayed is typically a flat or curved area (here, the area will be referred to as a virtual image display area), and the virtual image display area is positioned so as to face directly toward the observer's line of sight. In other words, when viewed from the left or right of the vehicle, the angle between the virtual image display area in which the virtual image is displayed and the road surface on which the vehicle is traveling is set to approximately 90 degrees (the virtual image display area is not positioned so as to be approximately parallel to the road surface).

[0006] Fig. 10 is a diagram showing the foreground viewed by an observer through the front windshield WS of the host vehicle while the host vehicle is traveling, and a perspective image 1100 displayed in a virtual image display area 1000 overlapping the foreground. Fig. 10 shows an aspect of a perspective image expressed as if arranged parallel to the road surface using linear perspective, in which the left line of the host vehicle's driving lane 1010 is indicated by reference numeral 1011, the right line is indicated by reference numeral 1012, and the intersection of the extension of the left line 1011 and the extension of the right line 1012 is set as a vanishing point 1020. The perspective image 1100 in Fig. 10 is a trapezoid, and is expressed by linear perspective (one-point perspective) so that the intersection of the extension of the left side 1101 and the extension of the right side 1102 of the trapezoid coincides with the vanishing point 1020. This gives a pictorial representation that the perspective image 1100 is arranged parallel to the plane of the driving lane 1010.

[0007] However, as described above, even if a perspective image 1100 such as that shown in FIG. 10 is displayed in a virtual image display area that is not positioned so as to be approximately parallel to the road surface, that is, a virtual image display area that is raised toward the observer by approximately 90 degrees from the road surface on which the vehicle is traveling, the distance between the far end 1110 of the perspective image 1100 and the driving lane 1010 is longer than the distance between the near end 1120 and the driving lane 1010, and the far end 1110 of the perspective image 1100 is perceived as raised toward the observer relative to the parallel surface of the driving lane 1010, making it difficult to sense that it is parallel to the surface of the driving lane 1010, and there is room for improvement.

[0008] A summary of certain embodiments disclosed herein is provided below. It should be understood that these aspects are presented merely to provide the reader with an overview of these certain embodiments, and are not intended to limit the scope of the disclosure. Indeed, the disclosure may encompass a variety of aspects that are not set forth below.

[0009] The present disclosure relates to providing a display control device, a head-up display device, a display control method, and the like that display an image that is easily perceived as being parallel to the plane of the driving lane.

[0010] Therefore, the display control device, head-up display device, display control method, etc. described in this specification adopt the following means to solve the above problems. The gist of this embodiment is to execute a first display process for displaying a perspective virtual image in a first display mode having a first vanishing point located below a vanishing point in the foreground as seen by the observer, and a second display process for displaying a perspective virtual image in a second display mode having a second vanishing point located above the first vanishing point as seen by the observer.

[0011] Therefore, the display control device described in this specification is a display control device that performs display control in a head-up display device that projects an image onto a projection target member, thereby allowing an observer to view a virtual image of the image superimposed on the foreground, and is equipped with one or more processors, a memory, and one or more computer programs stored in the memory and configured to be executed by the one or more processors, and the processor performs a first display process to display a perspective virtual image of a first display mode having a first vanishing point that is located below a vanishing point in the foreground as seen by the observer, and a second display process to display a perspective virtual image of a second display mode having a second vanishing point that is located above the first vanishing point as seen by the observer. [Brief description of the drawings]

[0012] [Figure 1] FIG. 1 is a diagram showing an example of application of a virtual image display system for a vehicle to a vehicle. [Diagram 2] FIG. 2 is a diagram showing a configuration of the head-up display device. [Diagram 3] FIG. 3 is a diagram conceptually showing a virtual object placed at a target position in a real scene, and an image displayed in a virtual image display area so that the virtual object is visually recognized at the target position in the real scene. [Figure 4] FIG. 4 is a diagram showing an example of a foreground viewed by an observer while the host vehicle is traveling, and a virtual image in the first display manner displayed so as to be superimposed on the foreground. [Figure 5A] FIG. 5A is a diagram showing a modified example of a perspective image. [Figure 5B] FIG. 5B is a diagram showing a modified example of a perspective image. [Figure 5C] FIG. 5C is a diagram showing a modified example of a perspective image. [Figure 5D] FIG. 5D is a diagram showing a modified example of a perspective image. [Figure 6] FIG. 6 is a diagram showing an example of a foreground viewed by an observer while the host vehicle is traveling, and a virtual image in the second display manner displayed superimposed on the foreground. [Figure 7A] FIG. 7A is a diagram showing a perspective virtual image V of a second display mode having a second vanishing point. [Figure 7B] FIG. 7B is a diagram showing a perspective virtual image V of a second display mode having a second vanishing point. [Figure 8] FIG. 8 is a block diagram of a vehicle virtual image display system according to some embodiments. [Figure 9] FIG. 9 is a flow diagram showing a display control method for executing a display control process for displaying a perspective virtual image in the first display mode or the second display mode. [Figure 10] FIG. 10 is a diagram relating to a conventional example, showing an example of a foreground viewed by an observer while the host vehicle is traveling, and a virtual image displayed superimposed on the foreground. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] 1 to 8, an exemplary configuration and operation of a virtual image display system for a vehicle will be described. Note that the present invention is not limited to the following embodiments (including the contents of the drawings). Of course, modifications (including removal of components) can be made to the following embodiments. In addition, in the following description, descriptions of known technical matters will be omitted as appropriate in order to facilitate understanding of the present invention.

[0014] Please refer to FIG. 1. FIG. 1 is a diagram showing an example of application of a virtual image display system for vehicles to a vehicle. In FIG. 1, the left-right direction (in other words, the width direction of the vehicle 1) of a vehicle (an example of a moving body, hereinafter also referred to as the vehicle itself) 1 is the X-axis (the positive direction of the X-axis is the left direction when the vehicle 1 faces forward), the up-down direction (in other words, the height direction of the vehicle 1) along a line segment that is perpendicular to the left-right direction and perpendicular to the ground or a surface equivalent to the ground (here, the road surface 6) is the Y-axis (the positive direction of the Y-axis is the upward direction), and the front-rear direction along a line segment that is perpendicular to each of the left-right direction and the up-down direction is the Z-axis (the positive direction of the Z-axis is the straight-ahead direction of the vehicle 1). This point is the same in other drawings.

[0015] A head-up display device (an example of a virtual image display device) 20 in the vehicle virtual image display system 10 is a head-up display (HUD) device provided in a dashboard 5 of a vehicle (an example of a moving body) 1. The HUD device 20 emits display light K toward a front windshield 2 (an example of a projection target), and the front windshield 2 reflects the display light K of an image displayed by the HUD device 20 toward an eye box 200. The HUD device 20 notifies an occupant of the vehicle 1 (the occupant is typically the driver of the vehicle 1) of not only information related to the vehicle 1 (hereinafter referred to as vehicle information) but also information other than the vehicle information in an integrated manner. The vehicle information includes not only information about the vehicle 1 itself but also information outside the vehicle 1 related to the operation of the vehicle 1.

[0016] As shown in the figure, the vehicle virtual image display system 10 provided in a vehicle (host vehicle) 1 includes an eye position detection unit (gaze detection unit) 409 for pupil (or face) detection that detects the position and gaze direction of the left eye 700L and right eye 700R of an observer (typically a driver sitting in the driver's seat of vehicle 1), an external sensor 411 composed of a camera (e.g., a stereo camera) that captures an image in front of the vehicle 1 (broadly speaking, the surroundings), a head-up display device (hereinafter also referred to as a HUD device) 20, and a display control device 30 that controls the HUD device 20.

[0017] In addition, the "eye box" used in the description of this embodiment is (1) a region in which the entire virtual image V of the image can be seen, and at least a part of the virtual image V of the image cannot be seen outside the region, (2) a region in which at least a part of the virtual image V of the image can be seen, and at least a part of the virtual image V of the image cannot be seen outside the region, (3) a region in which at least a part of the virtual image V of the image can be seen with a predetermined luminance or more, and at least a part of the virtual image V of the image is less than the predetermined luminance outside the region, or (4) a region in which at least a part of the virtual image V can be viewed stereoscopically when the HUD device 20 can display a virtual image V that can be viewed stereoscopically, and at least a part of the virtual image V cannot be viewed stereoscopically outside the region. That is, when the observer places his / her eyes (both eyes) outside the eye box 200, the observer cannot view the entire virtual image V of the image, the visibility of the entire virtual image V of the image is very low and difficult to perceive, or the virtual image V of the image cannot be viewed stereoscopically. The predetermined brightness is, for example, about 1 / 50 of the brightness of a virtual image of an image viewed at the center of the eyebox. The "eyebox" is set to be the same as the area (also called the iris) where the observer's viewpoint is expected to be located in the vehicle in which the HUD device 20 is mounted, or to include most of the iris (for example, 80% or more).

[0018] The virtual image display area VS is a flat, curved, or partially curved area where an image generated inside the HUD device 20 is formed as a virtual image V, and is also called an imaging surface. The virtual image display area VS is a position where a display surface (e.g., an exit surface of a liquid crystal light modulation element) 51 of a display 40 (to be described later) of the HUD device 20 is formed as a virtual image, that is, the virtual image display area VS corresponds to a display surface 51 (to be described later) of the HUD device 20 (in other words, the virtual image display area VS is in a conjugate relationship with the display surface 51 of the display 40 (to be described later).) And, it can be said that the virtual image viewed in the virtual image display area VS corresponds to an image displayed on a display surface 51 (to be described later) of the HUD device 20. It is preferable that the virtual image display area VS itself has low visibility to the extent that it is not actually visible to the observer's eyes or is difficult to be visible.

[0019] The virtual image display area VS has an angle (tilt angle θt in Figure 1) between the horizontal direction (XZ plane) and the axis along the left-right direction (X-axis direction) of the vehicle 1, and the angle between the line segment connecting the center 205 of the eye box 200 and the upper end VS1 of the virtual image display area VS and the line segment connecting the center of the eye box and the lower end VS2 of the virtual image display area VS, which is defined as the vertical angle of view of the virtual image display area VS, and an angle (vertical arrangement angle θd in Figure 1) between the bisector of this vertical angle of view and the horizontal direction (XZ plane). The vertical arrangement angle θd is typically set downward from the horizontal direction (XZ plane), for example, +2 degrees (here, the positive direction of the vertical arrangement angle θd can also be said to be a depression angle where the bisector of the vertical angle of view is oriented downward from the center 205 of the eyebox 200, and the negative direction of the vertical arrangement angle θd can also be said to be an elevation angle where the bisector of the vertical angle of view is oriented upward from the center 205 of the eyebox 200).

[0020] In addition, the virtual image display region VS of this embodiment has a tilt angle θt of approximately 90 [degrees] so that it faces approximately directly when facing forward (positive direction of the Z axis). However, the tilt angle θt is not limited to this and can be changed within the range of -10≦θt<90 [degrees]. The tilt angle θt may be set to, for example, 60 [degrees], and the virtual image display region VS may be disposed so that the upper region is farther away than the lower region as viewed from the viewer. However, the range of the tilt angle θt is not limited to this.

[0021] 2 is a diagram showing one embodiment of the configuration of a head-up display device. The HUD device 20 is installed, for example, in a dashboard (reference numeral 5 in FIG. 1). The HUD device 20 has a display 40, a relay optical system 80, and a housing 22 that houses the display 40 and the relay optical system 80 and has a light exit window 21 that can emit display light K from the display 40 from the inside to the outside.

[0022] 2 is typically mainly composed of a liquid crystal display panel (light modulation element) 50 and a light source unit 60. A display surface 51 is the viewer-side surface of the liquid crystal display panel 50, and emits display light K of an image. The angle (including the tilt angle θt) of the virtual image display region VS can be set by setting the angle of the display surface 51 with respect to an optical axis Kp of the display light K traveling from the center of the display surface 51 toward the eye box 200 (the center 205 of the eye box 200) via the relay optical system 80 and the projected portion.

[0023] The liquid crystal display panel (light modulation element) 50 receives light from the light source unit 60, and emits display light K, which is spatially light modulated into a 2D image, toward the relay optical system 80 (second mirror 82). The liquid crystal display panel 50 is, for example, rectangular in shape, with the short side being the direction in which pixels corresponding to the vertical direction (Y-axis direction) of the virtual image V seen by the observer are arranged. The observer visually recognizes the transmitted light of the liquid crystal display panel 50 through the virtual image optical system 90. The virtual image optical system 90 is a combination of the relay optical system 80 and the front windshield 2 shown in FIG. 2. The light modulation element 50 (display 40) may be a self-luminous display, or may be a projection display (for example, a reflective display panel using LCoS or DMD) that projects an image onto a screen, but is not limited thereto. In this case, the display surface 51 is the screen of the projection display.

[0024] The light source unit 60 is composed of a light source (not shown) and an illumination optical system (not shown).

[0025] The light source (not shown) is, for example, a plurality of chip-type LEDs, and emits illumination light to the liquid crystal display panel (an example of a light modulation element) 22. The light source unit 60 is composed of, for example, four light sources, and is arranged in a row along the long side of the liquid crystal display panel 50. The light source unit 60 emits illumination light toward the liquid crystal display panel 50 under the control of the display control device 30. The configuration of the light source unit 60 and the arrangement of the light sources are not limited to this.

[0026] The illumination optical system (not shown) is composed of, for example, one or more lenses (not shown) arranged in the emission direction of the illumination light from the light source unit 60, and a diffusion plate (not shown) arranged in the emission direction of the one or more lenses.

[0027] The light source unit 60 may be configured to be capable of local dimming, and may change the degree of illumination for each area of ​​the display surface 51 under the control of the display control device 30. This allows the light source unit 60 to adjust the luminance of the image displayed on the display surface 51 for each area.

[0028] The relay optical system 80 is disposed on the optical path of the display light K (light traveling from the display 40 toward the eye box 200) emitted from the display 40, and is composed of one or more optical members that project the display light K from the display 40 onto the front windshield 2 outside the HUD device 20. The relay optical system 80 in FIG. 2 includes one concave first mirror 81 and one flat second mirror 82.

[0029] The first mirror 81 is, for example, a free curved surface shape having a positive optical power. In other words, the first mirror 81 may be a curved surface shape having different optical powers for each region, that is, the optical power added to the display light K may be different depending on the region (optical path) through which the display light K passes. Specifically, the optical power added by the relay optical system 80 may be different between the first display light K11, the second display light K12, and the third display light K13 (see FIG. 2) traveling from each region of the display surface 51 toward the eye box 200.

[0030] The second mirror 82 is, for example, a flat mirror, but is not limited thereto, and may be a curved surface having optical power. That is, the relay optical system 80 may combine a plurality of mirrors (for example, the first mirror 81 and the second mirror 82 in this embodiment) to add different optical power depending on the region (optical path) through which the display light K passes. The second mirror 82 may be omitted. That is, the display light K emitted from the display 40 may be reflected by the first mirror 81 to the projection target portion (front windshield) 2.

[0031] In addition, the relay optical system 80 includes two mirrors (reflective optical system) in this embodiment, but is not limited to this. The relay optical system 80 may include one or more refractive optical elements such as lenses, diffractive optical elements such as holograms, reflective optical elements different from those described above, or combinations thereof, which may be added to the above-mentioned reflective optical system, or may replace part or all of the above-mentioned reflective optical system.

[0032] Furthermore, the relay optical system 80 has the function of setting the distance to the virtual image display area VS by using this curved shape (an example of optical power), and the function of generating a virtual image that is an enlarged image of the image displayed on the display surface 51. In addition to this, it may also have the function of suppressing (correcting) distortion of the virtual image that may occur due to the curved shape of the front windshield 2, or other known optical functions.

[0033] Furthermore, the relay optical system 80 may be rotatable by being fitted with one or more actuators 86, 87 controlled by the display control device 30. For example, the actuator 86 may rotate (and / or move) the first mirror 81 about a first rotation axis AX1. The actuator 87 may rotate (and / or move) the second mirror 82 about the first rotation axis AX1.

[0034] The head-up display device 20 can display images in the vicinity of objects such as the road surface of the driving lane, a branch road, a road sign, an obstacle (pedestrian, bicycle, motorcycle, other vehicle, etc.), and a feature (building, bridge, etc.) that exist in the foreground, which is a real space (real scene) visually recognized through the front windshield 2 of the vehicle 1, a position overlapping the object, or a position set based on the object, so that an observer (typically an observer seated in the driver's seat of the vehicle 1) can perceive visual augmented reality (AR). In the description of this embodiment, an image whose display position can be changed according to a real object existing in the real scene or a depth or a target position in the up, down, left, and right directions in a predetermined space in the real scene determined by the display control device 30 is defined as an AR image, and an image whose display position is set regardless of the position of the real object is defined as a non-AR image. In addition, in the description of this embodiment, an image consisting of one or more images expressed by a perspective projection that creates a sense of perspective is defined as a perspective image, and an image not expressed by a perspective projection is defined as a non-perspective image. A perspective image is typically an AR image.

[0035] A display control device 30, which will be described later, can control the aspect of the image V displayed by the HUD device 20 (perceived by the observer) by executing, for example, image rendering processing (graphic processing), display drive processing, and the like.

[0036] 3 is a conceptual diagram showing a virtual object arranged at a target position in a real scene, and an image displayed in a virtual image display area so that the virtual object is visually recognized at the target position in the real scene. As shown in FIG. 3, the depth direction is the Z-axis direction as viewed from the observer 700, the left-right direction (width direction of the vehicle 1) is the X-axis direction, and the up-down direction (up-down direction of the vehicle 1) is the Y-axis direction. Note that the direction moving away from the observer is the positive direction of the Z-axis, the left direction as viewed from the observer is the positive direction of the X-axis, and the up direction as viewed from the observer is the positive direction of the Y-axis.

[0037] The viewer 700 perceives a virtual object FU at a predetermined target position PT in the real scene by viewing the virtual image V formed (focused) in the virtual image display area VS via the projection unit 2. The viewer views the virtual image V of the image of the display light K reflected by the projection unit 2. At this time, if the virtual image V is, for example, an arrow indicating a course, the arrow of the virtual image V is displayed in the virtual image display area VS so that the virtual object FU is positioned and viewed at the predetermined target position PT in the foreground of the host vehicle 1.

[0038] (Perspective virtual image V10 of the first display mode) 4 is a diagram showing an example of the foreground viewed by an observer while the vehicle 1 is traveling, and a virtual image of a first display mode displayed superimposed on the foreground. A virtual image V10 of the first display mode (hereinafter also referred to as a perspective virtual image V10) has a vanishing point VP1 (first vanishing point VP1). That is, the figure of the perspective virtual image V10 has a shape with a vanishing point VP1 that gives a sense of depth, and for example, an extension line V1A of the left side of the stem of an "arrow" and an extension line V1B of the right side intersect at the vanishing point VP1.

[0039] As shown in the figure, when the perspective virtual image V10 is disposed below the viewer (at a position where the vertical arrangement angle θd is in a positive direction), the vanishing point VP1 is disposed above the position of the virtual image V10. On the other hand, when the perspective virtual image V10 is disposed above the viewer (at a position where the vertical arrangement angle θd is in a negative direction), the vanishing point VP1 is disposed below the position of the virtual image.

[0040] The viewer gets a sense of depth from the perspective virtual image V10 as seen by the viewer based on the relationship between the positions of the vanishing point VP1 and the perspective virtual image V10. In this way, the perspective virtual image V10 has the vanishing point VP1, making it easy for the viewer to estimate the depth position of the perspective virtual image V10.

[0041] On the other hand, as shown in Fig. 4, the driving lane 6 of the vehicle 1 has a vanishing point VP0 (actual scene vanishing point VP0). In this specific example, there is a driving lane 6 going straight ahead as seen by the viewer, and the extensions of the boundaries 7 and 8 on both sides of the driving lane 6 essentially intersect (become a point) at the vanishing point VP0. In this way, since the driving lane (road surface) 6 has a vanishing point VP0, the viewer gets a sense of depth in the actual scene (driving lane 6).

[0042] In the vehicle virtual image display system 10 according to this embodiment, as shown in FIG. 4, the vanishing point VP1 of the perspective virtual image V10 is located in front of (below when viewed from the observer) the vanishing point VP0 of the real scene (the driving lane 6).

[0043] In general, when creating an image, including a painting, using linear perspective (perspective projection), a vanishing point is used to position various objects in the image at their respective depth positions. For example, a straight line is virtually drawn radially from a predetermined vanishing point, and the contour line of each object is aligned with the straight line, so that each object is perceived as being fixed at a predetermined depth position. Note that multiple vanishing points can be provided, but for simplicity, the following description will be given assuming that one vanishing point is provided in one image.

[0044] When a general linear perspective method is used, a virtual image is displayed so that the extension lines of each boundary line forming the outline of a virtual object perceived by an observer intersect at the vanishing point VP0 in the foreground (driving lane 6). In other words, the virtual image is generated so that the position of the vanishing point VP1 of the virtual image coincides with the position of the vanishing point VP0 in the foreground (driving lane 6). In this embodiment, however, the perspective virtual image V10 is generated so that the position of the vanishing point VP1 of the perspective virtual image V10 is closer to the viewer than the position of the vanishing point VP0 in the foreground (driving lane 6) (first display process).

[0045] This makes it possible for the viewer to easily sense that the perceived placement angle of the virtual object FU10 (the angle with respect to the road surface with the X2 axis in FIG. 3 as the axis of rotation) is approximately parallel to the driving lane 6.

[0046] The virtual object FU10 is an "arrow", and the target position PT at which the virtual object FU10 is placed (set) is usually set to match the height of the surface of the foreground (driving lane 6) (i.e., the set height is set to 0 m). However, this is just one example and is not limiting. In another example, the target position PT may be set to a position higher than the height 1 of the surface of the foreground (driving lane 6) (i.e., the set height may be set to 0.5 m or 1 m). In another example, the target position PT may be set to a position lower than the height of the surface of the foreground (driving lane 6) (i.e., the set height may be set to -1 m or -2 m).

[0047] 5A, 5B, 5C, and 5D are diagrams showing modified examples of perspective images. The perspective virtual image V10 in the above embodiment is composed of one straight "arrow", but is not limited to this. The perspective virtual image V20 shown in FIG. 5A may be bent from the middle of the image ("arrow"). In this case, the left perspective line V1A can be defined as an extension of the left side of the perspective virtual image V20 until it is bent, and the right perspective line V1B can be defined as an extension of the right side of the perspective virtual image V20 until it is bent.

[0048] 5B is composed of a first perspective virtual image V31 and a second perspective virtual image V32 that is disposed above the first perspective virtual image V31 and is set to be smaller in size than the first perspective virtual image V31. In this case, the left perspective line V1A can be defined as a straight line connecting the left end of the first perspective V31 and the left end of the second perspective V32, and the right perspective line V1B can be defined as a straight line connecting the right end of the first perspective V31 and the right end of the second perspective V32.

[0049] 5C may be configured with a plurality of perspective virtual images V41 to V49 divided into left and right. In this case, the left perspective line V1A can be defined as a straight line (approximate straight line) connecting the left ends of the left perspectives V41 to V44, and the right perspective line V1B can be defined as a straight line (approximate straight line) connecting the right ends of the right perspectives V45 to V48.

[0050] 5D may be set to a position facing a vanishing point VP0' (the vanishing point VP0' shown in FIG. 5D is disposed to the left of the vanishing point VP0) that is shifted to the left and right of the vanishing point VP0, which is the intersection of the boundary lines 7 and 8 of the travel lane 6, as seen by the observer. Even in this case, the vanishing point VP1 (the first vanishing point VP1) of the perspective virtual image V50 may be disposed in front of the vanishing point VP0' (the real scene vanishing point VP0).

[0051] (Perspective virtual image V60 of the second display mode) 6 is a diagram showing an example of the foreground viewed by the observer while the vehicle 1 is traveling, and a virtual image of the second display mode displayed superimposed on the foreground. The virtual image V60 of the second display mode (hereinafter also referred to as the perspective virtual image V60) has a vanishing point VP2 (second vanishing point VP2) on the farther side than the first vanishing point VP1 of the perspective virtual image V10 of the first display mode. That is, the figure of the perspective virtual image V60 has a shape having the second vanishing point VP2 that gives the perception of depth, and for example, an extension line V1B of the left side of the stem of the "arrow" and an extension line V2B of the right side intersect at the second vanishing point VP2.

[0052] As shown in the figure, when the perspective virtual image V60 of the second display mode is disposed downward (at a position where the vertical arrangement angle θd is in a positive direction) as viewed by the viewer, the second vanishing point VP2 is disposed above the position of the first vanishing point VP1. On the other hand, when the perspective virtual image V60 of the second display mode is disposed upward (at a position where the vertical arrangement angle θd is in a negative direction) as viewed by the viewer, the second vanishing point VP2 is disposed below the position of the first vanishing point VP1.

[0053] The HUD device 20 (display control device 30) of some embodiments changes the display mode of the perspective virtual image V to the above-mentioned first display mode having the first vanishing point VP1 or the above-mentioned second display mode having the second vanishing point VP2 set on the far side of the first vanishing point VP1 according to the type of the perspective virtual image V. The HUD device 20 (display control device 30) of some embodiments (10) displays the perspective virtual image V in the second display mode having the second vanishing point VP2 if the perspective virtual image V is a moving image that moves or expands in the perspective direction (one example of a second display process), and (15) displays the perspective virtual image V in the first display mode having the first vanishing point VP1 if the perspective virtual image V is a still image, a moving image that does not move much in the perspective direction, or a moving image that does not expand much in the perspective direction.

[0054] 7A and 7B are diagrams showing a perspective virtual image V in a second display mode having a second vanishing point VP2. In some embodiments, the HUD device 20 (display control device 30) displays the perspective virtual image V in a second display mode having a second vanishing point VP2 when the perspective virtual image V is (11) a moving image that is perceived as gradually expanding and contracting in the perspective direction (in the example of FIG. 7A, a moving image expressed by a perspective projection that gradually extends from the near side toward the second vanishing point VP2 on the far side) or (12) a moving image that is perceived as gradually moving in the perspective direction (in the example of FIG. 7B, a moving image expressed by a perspective projection that gradually moves from the near side toward the second vanishing point VP2 on the far side) (an example of a second display process). On the other hand, when the perspective virtual image V is (16) a still image, (17) a moving image that is not perceived as stretching much in the perspective direction, or (18) a moving image that is not perceived as moving much in the perspective direction, the HUD device 20 (display control device 30) displays it in a first display mode having a first vanishing point VP1 (an example of a first display process). Specifically, the "(17) moving image that does not stretch much in the perspective direction, or (18) a moving image that does not move much in the perspective direction" is, for example, a moving image that moves in the left-right direction (X-axis direction), or a moving image that moves in the up-down direction (Y-axis direction) by an infinitesimal distance less than a predetermined threshold in a direction that can be perceived by humans as the depth direction (Z-axis direction) (more specifically, a moving image in which the distance of movement in the up-down direction (Y-axis direction) is 1 / 10 or less of the distance of movement in the left-right direction (X-axis direction)), etc.

[0055] Furthermore, the HUD device 20 (display control device 30) of some embodiments changes the display mode of the perspective virtual image V to the above-mentioned first display mode having the first vanishing point VP1 or the above-mentioned second display mode having the second vanishing point VP2 set on the farther side than the first vanishing point VP1, depending on the moving speed (deformation speed) of the moving image of the perspective virtual image V. For example, the HUD device 20 (display control device 30) of some embodiments (20) displays the perspective virtual image V in the second display mode having the second vanishing point VP2 if the moving speed (deformation speed) of the moving image is fast (an example of a second display process), and (25) displays the perspective virtual image V in the first display mode having the first vanishing point VP1 if the moving speed of the moving image is slow (an example of a first display process).

[0056] In the above embodiment, the HUD device 20 (display control device 30) displays the perspective virtual image V in a second display mode having a second vanishing point VP2 when the speed at which the image (21) gradually expands and contracts in the perspective direction or gradually moves in the perspective direction is equal to or greater than a predetermined image speed threshold and is determined to be fast (an example of a second display process). On the other hand, in the above embodiment, the HUD device 20 (display control device 30) displays the perspective virtual image V in a first display mode having a first vanishing point VP1 when the speed at which the image (26) gradually expands and contracts in the perspective direction or gradually moves in the perspective direction is less than a predetermined image speed threshold and is determined to be slow (an example of a first display process).

[0057] In addition, the HUD device 20 (display control device 30) of some embodiments changes the display mode of the perspective virtual image V to the first display mode having the first vanishing point VP1 or the second display mode having the second vanishing point VP2 set on the farther side than the first vanishing point VP1 according to the brightness of the area ahead (surroundings) of the vehicle 1 (an example of a second display process). For example, the HUD device 20 (display control device 30) of some embodiments (30) displays the image in the second display mode having the second vanishing point VP2 when it is determined that the area ahead (surroundings) of the vehicle 1 is bright, and (35) displays the image in the first display mode having the first vanishing point VP1 when it is determined that the area ahead (surroundings) of the vehicle 1 is dark (an example of a first display process).

[0058] In addition, in some embodiments, the HUD device 20 (display control device 30) changes the display mode of the perspective virtual image V to the above-mentioned first display mode having a first vanishing point VP1, or the above-mentioned second display mode having a second vanishing point VP2 set farther away than the first vanishing point VP1, depending on the length of the perspective virtual image V in the up-down direction (Y-axis direction), which is the depth direction (Z-axis direction) as perceived by humans. For example, in some embodiments, the HUD device 20 (display control device 30) displays (40) the length of the perspective virtual image V in the up-down direction (Y-axis direction), which is the distance in the depth direction (Z-axis direction) as perceived by humans, in a second display mode having a second vanishing point VP2 (an example of a second display processing), and (45) displays (45) the length of the perspective virtual image V in the up-down direction (Y-axis direction), which is the distance in the depth direction (Z-axis direction) as perceived by humans, in a first display mode having a first vanishing point VP1 (an example of a first display processing).

[0059] In addition, the HUD device 20 (display control device 30) of some embodiments changes the display mode of the perspective virtual image V to the above-mentioned first display mode having the first vanishing point VP1 or the above-mentioned second display mode having the second vanishing point VP2 set on the far side of the first vanishing point VP1 according to the length of the period during which the display continues. For example, the HUD device 20 (display control device 30) of some embodiments (50) displays the perspective virtual image V of a moving image or a still image in the second display mode having the second vanishing point VP2 when the period from the start of displaying the perspective virtual image V of the moving image or a still image to the end of displaying the perspective virtual image V is equal to or longer than a predetermined threshold value (an example of a second display process), and (55) displays the perspective virtual image V of a moving image or a still image in the first display mode having the first vanishing point VP1 when the period from the start of displaying the perspective virtual image V of the moving image or a still image to the end of displaying the perspective virtual image V of the moving image or a still image is shorter than a predetermined threshold value (an example of a first display process).

[0060] FIG. 8 is a block diagram of a vehicle virtual image display system, according to some embodiments. The display controller 30 includes one or more I / O interfaces 31, one or more processors 33, one or more image processing circuits 35, and one or more memories 37. The various functional blocks described in FIG. 8 may be configured in hardware, software, or a combination of both. FIG. 8 is just one embodiment, and the components shown may be combined into fewer components, or there may be additional components. For example, the image processing circuit 35 (e.g., a graphics processing unit) may be included in the one or more processors 33.

[0061] As shown, the processor 33 and the image processing circuit 35 are operatively coupled to the memory 37. More specifically, the processor 33 and the image processing circuit 35 can execute a program stored in the memory 37 to control the vehicle virtual image display system 10 (the display 40), for example, to generate and / or transmit image data. The processor 33 and / or the image processing circuit 35 can include at least one general-purpose microprocessor (e.g., a central processing unit (CPU)), at least one application specific integrated circuit (ASIC), at least one field programmable gate array (FPGA), or any combination thereof. The memory 37 can include any type of magnetic medium, such as a hard disk, any type of optical medium, such as a CD and a DVD, any type of semiconductor memory, such as a volatile memory, and a non-volatile memory. The volatile memory can include DRAM and SRAM, and the non-volatile memory can include ROM and NVRAM.

[0062] As shown in the figure, the processor 33 is operably connected to the I / O interface 31. The I / O interface 31 communicates with, for example, a vehicle ECU 401 (described later) and / or other electronic devices (reference numerals 403 to 419 described later) provided in the vehicle in accordance with the CAN (Controller Area Network) standard (also referred to as CAN communication). Note that the communication standard adopted by the I / O interface 31 is not limited to CAN, and may include, for example, wired communication interfaces such as CANFD (CAN with Flexible Data Rate), LIN (Local Interconnect Network), Ethernet (registered trademark), MOST (Media Oriented Systems Transport: MOST is a registered trademark), UART, or USB, or in-vehicle communication (internal communication) interfaces that are short-range wireless communication interfaces within several tens of meters, such as a personal area network (PAN) such as a Bluetooth (registered trademark) network, a local area network (LAN) such as an 802.11x Wi-Fi (registered trademark) network, etc. In addition, the I / O interface 31 may include an external communication (external communication) interface such as a wide area communication network (e.g., an Internet communication network) based on cellular communication standards such as a wireless wide area network (WWAN0, IEEE802.16-2004 (WiMAX: Worldwide Interoperability for Microwave Access)), IEEE802.16e-based (Mobile WiMAX), 4G, 4G-LTE, LTE Advanced, and 5G.

[0063] As shown in the figure, the processor 33 is interoperably connected to the I / O interface 31, thereby enabling information to be exchanged with various other electronic devices connected to the vehicle virtual image display system 10 (I / O interface 31). For example, a vehicle ECU 401, a road information database 403, a vehicle position detection unit 405, an operation detection unit 407, an eye position detection unit 409, an outside sensor 411, a brightness detection unit 413, an IMU 415, a mobile information terminal 417, and an external communication device 419 are operably connected to the I / O interface 31. The I / O interface 31 may include a function of processing (converting, calculating, analyzing) information received from other electronic devices connected to the vehicle virtual image display system 10.

[0064] The display 40 is operatively coupled to the processor 33 and the image processing circuitry 35. Thus, the image displayed by the light modulation elements 50 may be based on image data received from the processor 33 and / or the image processing circuitry 35. The processor 33 and the image processing circuitry 35 control the image displayed by the light modulation elements 50 based on information obtained from the I / O interface 31.

[0065] The vehicle ECU 401 acquires the status of the vehicle 1 (e.g., mileage, vehicle speed, accelerator pedal opening, brake pedal opening, engine throttle opening, injector fuel injection amount, engine RPM, motor RPM, steering angle, shift position, drive mode, various warning states, posture (including roll angle and / or pitching angle), vehicle vibration (including vibration magnitude, frequency, and / or frequency)) from sensors and switches installed in the vehicle 1, and collects and manages (which may also include control) the said status of the vehicle 1, and as part of its function, can output a signal indicating a numerical value of the said status of the vehicle 1 (e.g., the vehicle speed of the vehicle 1) to the processor 33 of the display control device 30. In addition to or instead of simply transmitting a value detected by a sensor or the like (e.g., the pitching angle is 3 degrees in the forward tilt direction) to the processor 33, the vehicle ECU 401 may transmit to the processor 33 a determination result based on one or more states of the vehicle 1 including the value detected by the sensor (e.g., that the vehicle 1 satisfies a predetermined condition for a forward tilt state) and / or an analysis result (e.g., that the vehicle has been in a forward tilt state due to braking, in combination with information on the brake pedal opening degree). For example, the vehicle ECU 401 may output to the display control device 30 a signal indicating a determination result that the vehicle 1 satisfies a predetermined condition previously stored in a memory (not shown) of the vehicle ECU 401. The I / O interface 31 may acquire the above-mentioned information from a sensor or a switch provided in the vehicle 1 without going through the vehicle ECU 401.

[0066] In addition, the vehicle ECU 401 may output an instruction signal to the display control device 30 to indicate the image to be displayed by the vehicle virtual image display system 10, and at this time, the coordinates, size, type, display mode, notification necessity of the image, and / or necessity-related information that serves as the basis for determining the notification necessity of the image may be added to the instruction signal and transmitted.

[0067] The road information database 403 is included in a navigation device (not shown) provided in the vehicle 1, or an external server connected to the vehicle 1 via an external communication interface (I / O interface 31), and may read out information about the surroundings of the vehicle 1 (information related to real objects around the vehicle 1), such as road information (lanes, white lines, stop lines, crosswalks, road widths, number of lanes, intersections, curves, forks, traffic regulations, etc.) on which the vehicle 1 is traveling, and information about features (buildings, bridges, rivers, etc.), such as presence or absence, position (including distance to the vehicle 1), direction, shape, type, detailed information, etc.), and transmit the information to the processor 33. The road information database 403 may also calculate an appropriate route (navigation information) from a departure point to a destination, and output a signal indicating the navigation information or image data indicating the route to the processor 33.

[0068] The vehicle position detection unit 405 is a GNSS (Global Navigation Satellite System) or the like provided in the vehicle 1, which detects the current position and direction of the vehicle 1, and outputs a signal indicating the detection result to the road information database 403, a mobile information terminal 417 described later, and / or an external communication device 419 via the processor 33 or directly. The road information database 403, the mobile information terminal 417 described later, and / or the external communication device 419 may select / generate information about the surroundings of the vehicle 1 by acquiring the position information of the vehicle 1 from the vehicle position detection unit 405 continuously, intermittently, or for each predetermined event, and output the information to the processor 33.

[0069] The operation detection unit 407 is, for example, a hardware switch provided on a CID (Center Information Display) or an instrument panel of the vehicle 1, or a software switch combining an image and a touch sensor, and outputs operation information based on an operation by an occupant of the vehicle 1 (a user sitting in the driver's seat and / or a user sitting in the passenger seat) to the processor 33. For example, the operation detection unit 407 outputs to the processor 33 display area setting information based on an operation to move the virtual image display area 100 by a user's operation, eye box setting information based on an operation to move the eye box 200, information based on an operation to set the observer's eye position 700, and the like.

[0070] Eye position detection unit 409 may include a camera such as an infrared camera that detects eye position 700 (see FIG. 1) of an observer sitting in the driver's seat of vehicle 1, and may output the captured image to processor 33. Processor 33 may obtain a captured image (one example of information from which eye position 700 can be estimated) from eye position detection unit 409, and analyze the captured image using a method such as pattern matching to detect the coordinates of observer's eye position 700, and output a signal indicating the coordinates of the detected eye position 700 to processor 33.

[0071] Furthermore, eye position detection unit 409 may output an analysis result of analyzing the image captured by the camera (for example, a signal indicating to which spatial region the observer's eye position 700 belongs in relation to a plurality of preset display parameters) to processor 33. Note that the method of acquiring the observer's eye position 700 in vehicle 1 or information that allows the observer's eye position 700 to be estimated is not limited to these, and the information may be acquired using a known eye position detection (estimation) technique.

[0072] Furthermore, the eye position detection section 409 may detect the moving speed and / or moving direction of the viewer's eye position 700, and output a signal indicating the moving speed and / or moving direction of the viewer's eye position 700 to the processor 33.

[0073] In addition, when the eye position detection unit 409 detects (10) a signal indicating that the observer's eye position 700 is outside the eye box 200, (20) a signal that estimates that the observer's eye position 700 is outside the eye box 200, or (30) a signal that predicts that the observer's eye position 700 will be outside the eye box 200, it may determine that a specified condition is satisfied and output a signal indicating that state to the processor 33.

[0074] (20) Signals indicating that the observer's eye position 700 is outside the eye box 200 include (21) a signal indicating that the observer's eye position 700 cannot be detected, (22) a signal indicating that the observer's eye position 700 cannot be detected after movement of the observer's eye position 700 has been detected, and / or (23) a signal indicating that either of the observer's eye positions 700R, 700L is in the vicinity of the boundary 200A of the eye box 200 (the vicinity includes, for example, within a specified coordinate range from the boundary 200A).

[0075] (30) Signals predicting that the observer's eye position 700 will be outside the eye box 200 include (31) a signal indicating that the newly detected eye position 700 is equal to or greater than an eye position movement distance threshold pre-stored in memory 37 compared to the previously detected eye position 700 (the movement of the eye position within a specified unit time is greater than a specified range), and (32) a signal indicating that the movement speed of the eye position is equal to or greater than an eye position movement speed threshold pre-stored in memory 37.

[0076] The eye position detection unit 409 may also have a function as the gaze direction detection unit 409. The gaze direction detection unit 409 may include an infrared camera or a visible light camera that captures an image of the face of an observer sitting in the driver's seat of the vehicle 1, and may output the captured image to the processor 33. The processor 33 can specify the gaze direction (and / or the gaze position) of the observer by acquiring an image (one example of information that can estimate the gaze direction) from the gaze direction detection unit 409 and analyzing the captured image. The gaze direction detection unit 409 may analyze an image captured by a camera and output a signal indicating the gaze direction (and / or the gaze position) of the observer, which is the analysis result, to the processor 33. The method of acquiring information that can estimate the gaze direction of the observer of the vehicle 1 is not limited to these, and may be acquired using other known gaze direction detection (estimation) techniques such as an EOG (Electro-oculogram) method, a corneal reflection method, a scleral reflection method, a Purkinje image detection method, a search coil method, and an infrared fundus camera method.

[0077] The external sensor 411 detects real objects present around the vehicle 1 (in front, on the sides, and behind). The real objects detected by the external sensor 411 may include, for example, obstacles (pedestrians, bicycles, motorcycles, other vehicles, etc.), the road surface of a driving lane described below, dividing lines, roadside objects, and / or features (buildings, etc.). The external sensor is composed of, for example, a detection unit consisting of a radar sensor such as a millimeter wave radar, ultrasonic radar, or laser radar, a camera, or a combination of these, and a processing device that processes (data fuses) detection data from the one or more detection units. Conventional well-known methods are applied to object detection by these radar sensors and camera sensors. By object detection by these sensors, the presence or absence of a real object in a three-dimensional space, and if a real object exists, the position (relative distance from the vehicle 1, position in the left and right direction when the traveling direction of the vehicle 1 is the forward and backward direction, position in the up and down direction, etc.), size (size in the lateral direction (left and right direction), height direction (up and down direction), etc.), moving direction (lateral direction (left and right direction), depth direction (front and back direction)), moving speed (lateral direction (left and right direction), depth direction (front and back direction)), and / or type, etc. of the real object may be detected. One or more external sensors 411 can detect a real object in front of the vehicle 1 at each detection period of each sensor, and output real object information (presence or absence of a real object, and if a real object exists, information on the position, size, and / or type of each real object, etc.) which is an example of real object information, to the processor 33. Note that the real object information may be transmitted to the processor 33 via another device (for example, the vehicle ECU 401). In addition, when a camera is used as a sensor, an infrared camera or a near-infrared camera is preferable so that a real object can be detected even when the surroundings are dark, such as at night. Furthermore, when using a camera as a sensor, a stereo camera that can obtain distance and the like from parallax is desirable.

[0078] The brightness detection unit 413 detects the illuminance or brightness of a predetermined range of the foreground present in front of the cabin of the vehicle 1 as the outside brightness (an example of brightness information), or detects the illuminance or brightness inside the cabin as the inside brightness (an example of brightness information). The brightness detection unit 413 is, for example, a phototransistor or a photodiode, and is mounted on the instrument panel, the rearview mirror, the HUD device 20, or the like of the vehicle 1 shown in FIG.

[0079] The IMU 415 may include a combination of one or more sensors (e.g., an accelerometer and a gyroscope) configured to detect the position, orientation, and changes therein (speed of change, acceleration of change) of the vehicle 1 based on inertial acceleration. The IMU 415 may output the detected values ​​(the detected values ​​include signals indicating the position, orientation, and changes therein (speed of change, acceleration of change) of the vehicle 1) and the results of analyzing the detected values ​​to the processor 33. The results of the analysis may be signals indicating a determination result as to whether or not the detected values ​​satisfy a predetermined condition, and may be, for example, a signal indicating that the behavior (vibration) of the vehicle 1 is small based on a value regarding the change (speed of change, acceleration of change) of the position or orientation of the vehicle 1.

[0080] The mobile information terminal 417 is a smartphone, a notebook computer, a smartwatch, or other information device that can be carried by the observer (or other passengers of the vehicle 1). The I / O interface 31 can communicate with the mobile information terminal 417 by pairing with the mobile information terminal 417, and acquires data recorded in the mobile information terminal 417 (or a server via the mobile information terminal). The mobile information terminal 417 has, for example, the same functions as the above-mentioned road information database 403 and the vehicle position detection unit 405, and may acquire the road information (an example of real object related information) and transmit it to the processor 33. The mobile information terminal 417 may also acquire commercial information (an example of real object related information) related to commercial facilities in the vicinity of the vehicle 1 and transmit it to the processor 33. In addition, the mobile information terminal 417 may transmit schedule information of the holder of the mobile information terminal 417 (e.g., the observer), information on incoming calls at the mobile information terminal 417, information on received emails, etc. to the processor 33, and the processor 33 and the image processing circuit 35 may generate and / or transmit image data related to these.

[0081] The external communication device 419 is a communication device that exchanges information with the vehicle 1, and is, for example, another vehicle connected to the vehicle 1 through vehicle-to-vehicle communication (V2V: Vehicle To Vehicle), a pedestrian (a mobile information terminal carried by a pedestrian) connected to the vehicle 1 through vehicle-to-pedestrian communication (V2P: Vehicle To Pedestrian), and a network communication device connected through vehicle-to-roadside communication (V2I: Vehicle To Roadside Infrastructure), and in a broad sense includes everything connected to the vehicle 1 through communication (V2X: Vehicle To Everything). The external communication device 419 may obtain the positions of, for example, pedestrians, bicycles, motorcycles, other vehicles (such as a preceding vehicle), road surfaces, dividing lines, roadside objects, and / or features (such as buildings), and output them to the processor 33. Moreover, the external communication device 419 may have the same function as the vehicle position detection unit 405 described above, and may acquire position information of the vehicle 1 and transmit it to the processor 33, and may also have the function of the road information database 403 described above, and may acquire the road information (an example of real object related information) and transmit it to the processor 33. Note that the information acquired from the external communication device 419 is not limited to the above.

[0082] The software components stored in the memory 37 include a display parameter setting module 502 , a graphics module 504 , a light source driving module 506 , and an actuator driving module 508 .

[0083] 9 is a flow diagram showing a method S100 for executing a display control process for displaying a perspective virtual image in a first display mode or a second display mode. The method S100 is executed in a HUD device 20 including a display and a display control device 30 that controls the HUD device 20. Some operations in the method S100 are optionally combined, the procedure of some operations is optionally changed, and some operations are optionally omitted.

[0084] The display control device 30 (graphics module 504 described later) acquires, for example, an appropriate route (navigation information) from the road information database 403 to the destination, real object information (presence or absence of real objects, and if real objects exist, information such as the position, size, and / or type of each real object) from the external sensor 411, and selects and sets the virtual objects to be displayed (S110).

[0085] As shown in S131, the display parameter setting module 502 changes the display mode of the perspective virtual image V based on one or more pieces of information acquired from the I / O interface 31. The display parameter setting module 502 (10) sets the perspective virtual image V to a second display mode having a second vanishing point VP2 when the perspective virtual image V is set to a moving image that moves or expands in the perspective direction (an example of a second display process), and (15) sets the perspective virtual image V to a first display mode having a first vanishing point VP1 when the perspective virtual image V is set to a still image, a moving image that does not move much in the perspective direction, or a moving image that does not expand or contract much in the perspective direction (an example of a first display process), based on one or more pieces of information acquired from the I / O interface 31.

[0086] Furthermore, as shown in S132, based on one or more pieces of information acquired from the I / O interface 31, the display parameter setting module 502 sets (20) the moving speed of the moving image of the perspective virtual image V to a second display mode having a second vanishing point VP2 (an example of a second display processing) when the moving speed of the moving image of the perspective virtual image V is set to be faster than a predetermined threshold, and sets (25) the moving speed of the moving image of the perspective virtual image V to a first display mode having a first vanishing point VP1 (an example of a first display processing).

[0087] Furthermore, as shown in S133, based on information (but not limited to this) acquired from the brightness detection unit 413, the display parameter setting module 502 sets (30) the perspective virtual image V to a second display mode having a second vanishing point VP2 if it is determined that the area ahead (surroundings) of the vehicle 1 is bright (an example of a second display processing), and (35) sets (36) the perspective virtual image V to a first display mode having a first vanishing point VP1 if it is determined that the area ahead (surroundings) of the vehicle 1 is dark (an example of a first display processing).

[0088] Further, as shown in S134, based on road information acquired from the road information database 403 and position information (but not limited to this) of a real object existing in the foreground acquired from the outside sensor 411, the display parameter setting module 502 sets the perspective virtual image V to a second display mode having a second vanishing point VP2 (an example of a second display processing) if (40) the length in the up-down direction (Y-axis direction) of the perspective virtual image V indicating the guide route or a real object existing in the foreground, which is a distance in the depth direction (Z-axis direction) as perceived by humans, is the same as or longer than a predetermined threshold value pre-stored in the memory 37, and (45) sets the perspective virtual image V to a first display mode having a first vanishing point VP1 (an example of a first display processing) if the length in the up-down direction (Y-axis direction) of the perspective virtual image V, which is a distance in the depth direction (Z-axis direction) as perceived by humans, is shorter than a predetermined threshold value.

[0089] Furthermore, as shown in S135, based on one or more pieces of information acquired from the I / O interface 31, (50) if the period from the start to the end of the display of the perspective virtual image V of a moving image or still image is the same as or longer than a predetermined threshold pre-stored in the memory 37, the display parameter setting module 502 sets the perspective virtual image V to a second display mode having a second vanishing point VP2 (an example of a second display processing), and (55) if the period from the start to the end of the display of the perspective virtual image V of a moving image or still image is shorter than a predetermined threshold pre-stored in the memory 37, the display parameter setting module 502 sets the perspective virtual image V to a first display mode having a first vanishing point VP1 (an example of a first display processing).

[0090] The graphics module 504 includes various known software components for generating image data by performing image processing such as rendering, and driving the display 40. The graphics module 504 may also include various known software components for changing the type (moving image, still image, shape), arrangement (position coordinates, angle), size, display distance (in the case of 3D), and visual effects (e.g., brightness, transparency, saturation, contrast, or other visual characteristics) of the image to be displayed. The graphics module 504 can generate image data to be viewed by an observer in terms of the type of image (one example of a display parameter), the position coordinates of the image (one example of a display parameter), the angle of the image (a pitching angle about the X-direction axis, a yaw rate angle about the Y-direction axis, a rolling angle about the Z-direction axis, etc., one example of a display parameter), the size of the image (one example of a display parameter), the color of the image (one example of a display parameter set by hue, saturation, brightness, etc.), and the intensity of the perspective representation of the image (one display parameter set by the position of the vanishing point, etc.), and drive the light modulation element 50.

[0091] The light source driving module 506 includes various known software components for executing driving of the light source unit 24. The light source driving module 506 may drive the light source unit 24 based on the set display parameters.

[0092] The actuator driving module 508 includes various known software components for executing the actuation of the first actuator 28 and / or the second actuator 29. The actuator driving module 508 may drive the first actuator 28 and the second actuator 29 based on set display parameters.

[0093] The eye position detection module 510 detects the observer's eye position 700. The eye position detection module 510 includes various software components for performing various operations related to detecting coordinates indicating the observer's eye position 700 (positions in the X and Y axes directions, which are an example of a signal indicating the eye position 700), coordinates indicating the observer's eye height (position in the Y axis direction, which is an example of a signal indicating the eye position 700), detecting coordinates indicating the observer's eye height and depth position (positions in the Y and Z axes directions, which are an example of a signal indicating the eye position 700), and / or detecting coordinates indicating the observer's eye position 700 (positions in the X, Y, and Z axes directions, which is an example of a signal indicating the eye position 700).

[0094] The positions of the first vanishing point VP1 and the second vanishing point VP2 may be adjusted by the eye position 700 of the observer detected by the eye position detection unit 409, or the attitude of the host vehicle 1 detected by the IMU 415. For example, the display control device 30 (display parameter setting module 502) may adjust the positions of the first vanishing point VP1 and the second vanishing point VP2 in response to a change in the attitude (pitching angle (angle with the X-direction as an axis)) of the host vehicle 1. When the pitching angle is such that the front of the host vehicle 1 is lowered, the display control device 30 (display parameter setting module 502) may adjust the relative positions of the first vanishing point VP1 and the second vanishing point VP2 with respect to the virtual image display area VS to be on the upper side as seen by the observer, and when the pitching angle is such that the front of the host vehicle 1 is raised, the display control device 30 (display parameter setting module 502) may adjust the relative positions of the first vanishing point VP1 and the second vanishing point VP2 with respect to the virtual image display area VS to be on the lower side as seen by the observer.

[0095] The operations of the above-mentioned processing processes can be implemented by executing one or more functional modules of an information processing device, such as a general-purpose processor or an application-specific chip. These modules, combinations of these modules, and / or combinations with known hardware that can replace their functions, are all included in the scope of protection of the present invention.

[0096] The functional blocks of the vehicle virtual image display system 10 are optionally implemented by hardware, software, or a combination of hardware and software to implement the principles of the various embodiments described. It will be understood by those skilled in the art that the functional blocks described in FIG. 8 may be optionally combined or one functional block may be separated into two or more sub-blocks to implement the principles of the embodiments described. Therefore, the description in this specification optionally supports any possible combination or division of the functional blocks described in this specification. [Explanation of symbols]

[0097] 1: Vehicle 2: Projected part (front windshield) 5: Dashboard 6: Driving lane (road surface) 7: Border 8: Border 10: Virtual image display system for vehicles 20: HUD device (head-up display device) 21: Light exit window 22: Housing 24: Light source unit 28: First actuator 29: Second actuator 30: Display control device 31: I / O interface 33: Processor 35: Image processing circuit 37: Memory 40:Display unit 50: Light modulation element 51:Display surface 60: Light source unit 80: Relay optical system 81: First mirror 82: 2nd mirror 86: Actuator 87: Actuator 90: Virtual image optical system 100: Virtual image display area 200: Eye Box 200A: Boundary 205: Center 401: Vehicle ECU 403: Road information database 405: Vehicle position detection unit 407: Operation detection unit 409: Eye position detection unit (gaze direction detection unit) 411: External sensor 413: Brightness detection section 417: Mobile information terminal 419: External communication equipment 502: Display parameter setting module 504: Graphics module 506: Light source driving module 508: Actuator drive module 510: Eye position detection module 700: Eye position (observer) 700L: Left eye (eye position) 700R: Right eye (eye position) AX1: First rotation axis FU: Virtual Object FU10: Virtual Objects K:Display light Kp: Optical axis PT: Target position V: Perspective virtual image V10: Perspective Virtual Image V1A: Left perspective line V20: Perspective Virtual Image V1B: Right perspective line V30: Perspective Virtual Image V31: 1st perspective virtual image V32: Second perspective virtual image VP0: Real scene vanishing point VP0': Vanishing point VP1: 1st vanishing point VP2: 2nd vanishing point VS: Virtual image display area VS1: Upper end VS2: Bottom end θd: Vertical arrangement angle θt: Tilt angle

Claims

1. A display control device (30) that performs display control in a head-up display device that projects a perspective image, which is expressed as being arranged parallel to a road surface, onto a projection member, thereby allowing an observer to visually recognize a virtual image of the perspective image superimposed on a foreground, comprising: one or more processors (33); A memory (37); one or more computer programs stored in the memory (37) and configured to be executed by the one or more processors (33); The processor (33) a first display process for displaying a perspective virtual image in a first display mode having a first vanishing point (VP1) disposed below a vanishing point (VP0) of the foreground as viewed from the observer; a second display process for displaying a perspective virtual image in a second display mode having a second vanishing point (VP2) disposed above the first vanishing point (VP1) as seen by the observer; The processor (33) If the perspective virtual image is a still image, the first display process is executed; If the perspective virtual image is a moving image, the second display process is executed. A display control device (30).

2. A display control device (30) that performs display control in a head-up display device that projects a perspective image, which is expressed as being arranged parallel to a road surface, onto a projection member, thereby allowing an observer to visually recognize a virtual image of the perspective image superimposed on a foreground, comprising: one or more processors (33); A memory (37); one or more computer programs stored in the memory (37) and configured to be executed by the one or more processors (33); The processor (33) a first display process for displaying a perspective virtual image in a first display mode having a first vanishing point (VP1) disposed below a vanishing point (VP0) of the foreground as viewed from the observer; a second display process for displaying a perspective virtual image in a second display mode having a second vanishing point (VP2) disposed above the first vanishing point (VP1) as seen by the observer; The processor (33) If the perspective virtual image is a moving image that does not expand or contract in a perspective direction as viewed by an observer, or a moving image that does not move in a perspective direction, the first display process is executed; executes the second display process if the perspective virtual image is a moving image that is perceived as expanding or contracting in a perspective direction or moving in a perspective direction when viewed by an observer. A display control device (30).

3. A display control device (30) that performs display control in a head-up display device that projects a perspective image, which is expressed as being arranged parallel to a road surface, onto a projection member, thereby allowing an observer to visually recognize a virtual image of the perspective image superimposed on a foreground, comprising: one or more processors (33); A memory (37); one or more computer programs stored in the memory (37) and configured to be executed by the one or more processors (33); The processor (33) a first display process for displaying a perspective virtual image in a first display mode having a first vanishing point (VP1) disposed below a vanishing point (VP0) of the foreground as viewed from the observer; a second display process for displaying a perspective virtual image in a second display mode having a second vanishing point (VP2) disposed above the first vanishing point (VP1) as seen by the observer; The processor (33) When the surroundings are dark, the first display process is executed; When the surroundings are bright, the second display process is executed. A display control device (30).

4. A display control device (30) that performs display control in a head-up display device that projects a perspective image, which is expressed as being arranged parallel to a road surface, onto a projection member, thereby allowing an observer to visually recognize a virtual image of the perspective image superimposed on a foreground, comprising: one or more processors (33); A memory (37); one or more computer programs stored in the memory (37) and configured to be executed by the one or more processors (33); The processor (33) a first display process for displaying a perspective virtual image in a first display mode having a first vanishing point (VP1) disposed below a vanishing point (VP0) of the foreground as viewed from the observer; a second display process for displaying a perspective virtual image in a second display mode having a second vanishing point (VP2) disposed above the first vanishing point (VP1) as seen by the observer; The processor (33) When the moving speed or the deformation speed of the perspective virtual image is slow, the first display process is executed; When the moving speed or the deformation speed of the perspective virtual image is fast, the second display process is executed. A display control device (30).

5. A display control device (30) that performs display control in a head-up display device that projects a perspective image, which is expressed as being arranged parallel to a road surface, onto a projection member, thereby allowing an observer to visually recognize a virtual image of the perspective image superimposed on a foreground, comprising: one or more processors (33); A memory (37); one or more computer programs stored in the memory (37) and configured to be executed by the one or more processors (33); The processor (33) a first display process for displaying a perspective virtual image in a first display mode having a first vanishing point (VP1) disposed below a vanishing point (VP0) of the foreground as viewed from the observer; a second display process for displaying a perspective virtual image in a second display mode having a second vanishing point (VP2) disposed above the first vanishing point (VP1) as seen by the observer; The processor (33) executes the first display process when a period from when the perspective virtual image starts to when the perspective virtual image ends is shorter than a predetermined threshold value; executing the second display process when a period from when the display of the perspective virtual image starts to when the display of the perspective virtual image ends is longer than the predetermined threshold value; A display control device (30).

6. In the second display process, The second vanishing point (VP2) is located below the foreground vanishing point (VP0); A display control device (30) according to any one of the preceding claims.

7. A display control device (30) according to any one of claims 1 to 6, A light modulation element (50) that emits display light; A head-up display device (20) comprising: a relay optical system (80) that directs the display light from the light modulation element (50) to a projection target.

8. A display control method for performing display control in a head-up display device, which projects a perspective image, which is expressed as being arranged parallel to a road surface, onto a projection member, thereby allowing an observer to visually recognize a virtual image of the perspective image superimposed on a foreground, comprising: a first display process for displaying a perspective virtual image in a first display mode having a first vanishing point (VP1) disposed below a vanishing point (VP0) in the foreground as seen by an observer; a second display process for displaying a perspective virtual image in a second display mode having a second vanishing point (VP2) disposed above the first vanishing point (VP1) as seen by the observer; When the surroundings are dark, the first display process is executed; When the surroundings are bright, the second display process is executed. Display control method.

Images