Display control device, head-up display device, and display control method

Abstract

To reduce situations in which display light is not directed to actual eye positions, while improving light use efficiency.SOLUTION: A processor can at least set a part region D (thereafter, called a light emitting region DM) emitting illumination light equal to or more than prescribed light intensity of a light source element 60 (60A) in a first light emitting region DM1, and a second light emitting region DM2 wider than the first light emitting region DM1 on the basis of at least information on eye positions of an observer, or information enabling an estimation of the eye positions. When a head-up display device 20 starts up, the processor implements start-up light-emitting processing of setting the light emitting region DM in the second light emitting region DM2, and thereafter, implements reduction processing of reducing the light emitting region DM to the first light emitting region DM1.SELECTED DRAWING: Figure 9

Description

[Technical field]

[0001] The present disclosure relates to a display control device for a head-up display device that is used in a vehicle and allows an image to be visually superimposed on a view in front of the vehicle, a head-up display device, a display control method, and the like. [Background technology]

[0002] Patent Document 1 describes a head-up display device that is optically designed to distribute display light toward the entire area called the eyebox, which includes most or all of the iris where the eyes of passengers (observers) of various builds seated in the driver's seat of a vehicle are assumed to be located. According to this, if the passenger (observer) has their eyes positioned within the eyebox, they can view the image (virtual image) displayed by the head-up display device in a desired display mode (for example, within a desired brightness range). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2012-203176 A Summary of the Invention [Problem to be solved by the invention]

[0004] The head-up display device of Patent Document 1 directs display light to the entire eyebox, so the display light directed to other areas of the eyebox where the observer's eyes are not present is wasted. In other words, there is a problem that the utilization efficiency of the display light emitted by the head-up display device (here, the utilization efficiency of the display light (also called light utilization efficiency) is the ratio of the display light visually recognized by the observer to the emitted display light) is low. From another perspective, there is also a problem of high power consumption.

[0005] The present applicant has invented a head-up display device that detects the eye position of the observer, directs display light to the eye position, and reduces the amount of display light directed to other than the eye position (see Patent Application No. 2020-165113 for details. This technology is not publicly known). Such a head-up display device needs to detect the eye position of the observer, and when the vehicle starts the startup operation, the eye position cannot be detected (the eye position cannot be detected with high accuracy) unless the startup operations of the head-up display device, the display control device, the eye position detection unit, etc. are completed, so there is a problem that the display cannot be started (in other words, there is room for improvement in terms of the time it takes to start displaying after the vehicle (head-up display device) starts the startup operation). In addition, during the startup operation of the head-up display device, the display control device, the eye position detection unit, etc., or immediately after the startup operation is completed, the operation of the display control and the accuracy of the eye position detection are difficult to stabilize, and there is a risk that the display light will not be directed normally to the actual eye position. [Means for solving the problem]

[0006] A summary of certain embodiments disclosed herein is provided below. It should be understood that these aspects are presented merely to provide the reader with an overview of these certain embodiments, and are not intended to limit the scope of the disclosure. Indeed, the disclosure may encompass a variety of aspects that are not set forth below.

[0007] The present disclosure relates to an outline of improving light utilization efficiency. The present disclosure also relates to shortening the time required to start display while improving light utilization efficiency. The present disclosure also relates to reducing a situation in which display light is not directed to an actual eye position while improving light utilization efficiency.

[0008] Therefore, the display control device, head-up display device, and display control method described in this specification employ the following means to solve the above problems. In this embodiment, when the HUD device or the eye position detection unit is in an activated state, the gist of the present embodiment is that in a planar illumination unit that illuminates a light modulation element, the area from which illumination light having a predetermined light intensity or more is emitted is made relatively wide and then narrowed.

[0009] A display control device according to a first aspect of the present specification includes a light source element having a plurality of partial regions and capable of emitting illumination light for each partial region, a light modulation element that modulates the illumination light incident from the light source element and emits it as display light, and a display control device that displays the illumination light emitted from the different partial regions modulated by the light modulation element at different positions in an eye box. partial viewing area and a focusing optical system that focuses light toward a projection target, and controls a head-up display device that projects display light onto a projection target to allow an observer to view a virtual image from within an eyebox. The display control device includes a processor that acquires information about the observer's eye position (hereinafter referred to as eye position information) or information that allows the eye position to be estimated (hereinafter referred to as eye position estimation information) and controls a light source element based at least on the eye position information or the eye position estimation information, and the processor is capable of setting partial regions (hereinafter referred to as light-emitting regions) that emit illumination light of a predetermined light intensity or greater to at least a first light-emitting region and a second light-emitting region that is wider than the first light-emitting region, and when the head-up display device is started up, the light-emitting region is set to the second light-emitting region. By doing so, the range of the partial viewing area toward which the display light is directed is set as the second spotlighting area. The start-up light emission process is then performed, and the light emission area is then reduced to the first light emission area. By this, the range of the partial viewing area to which the display light is directed is set to the first spotlighting area which is narrower than the second spotlighting area. Execute the reduction process. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram illustrating an example of application of a vehicular display system to a vehicle, according to some embodiments. [Diagram 2] FIG. 2 is a diagram illustrating a configuration of a head-up display device according to some embodiments. [Diagram 3]FIG. 3 is a diagram illustrating optical paths in a head-up display device, according to some embodiments. [Figure 4A] FIG. 4A is a diagram illustrating partial viewing zones formed by virtually dividing an eyebox, and a state in which display light is directed to a plurality of partial viewing zones in accordance with the eye position of a viewer, according to some embodiments. [Figure 4B] FIG. 4B is a diagram illustrating a relationship between partial viewing areas of the eyebox and positions at which display light based on illumination light emitted from each partial region of a directional light source unit is focused, according to some embodiments. [Figure 4C] FIG. 4C is a diagram showing how an illumination laser beam is irradiated onto a light-emitting region of a light diffusion optical system to direct display light to the spot lighting area of ​​FIG. 4A. [Figure 5A] FIG. 5A is a diagram showing an example of a distribution of light amount (luminance) when display light is distributed in a spot lighting area according to some embodiments. [Figure 5B] FIG. 5B is a diagram showing a state in which the eye position has moved after the state of FIG. 5A. [Figure 5C] FIG. 5C is a diagram showing a state in which the spot lighting area has switched from one corresponding to the previous partial viewing area to one corresponding to the new partial viewing area E after the state of FIG. 5B. [Figure 6] FIG. 6 is a block diagram of a vehicle display system according to some embodiments. [Figure 7A] FIG. 7A is a flow diagram illustrating a method for performing operations for setting a spot lighting area (a location where display light is distributed) based on an observer's eye position, according to some embodiments. [Figure 7B] FIG. 7B is a flow diagram that explains a portion of FIG. 7A in more detail. [Figure 7C] FIG. 7C is a flow diagram that explains a portion of FIG. 7A in more detail. [Figure 7D] FIG. 7D is a flow diagram that explains a portion of FIG. 7A in more detail. [Figure 8A]FIG. 8A is a diagram illustrating partial viewing zones formed by virtually dividing an eyebox, and a state in which display light is directed to a plurality of partial viewing zones in accordance with the eye position of a viewer, according to some embodiments. [Figure 8B] FIG. 8B is a diagram illustrating a schematic relationship between partial viewing areas of the eyebox and positions at which display light based on illumination light emitted from each partial area of ​​the directional light source unit is concentrated, according to some embodiments. [Figure 9] FIG. 9 is a diagram showing an embodiment of the first light-emitting region and the second light-emitting region DM2 in the planar illumination unit. [Figure 10] FIG. 10 is a diagram illustrating light paths in a head-up display device, according to some embodiments. [Figure 11] FIG. 11 is a diagram showing an embodiment of the light source array shown in FIG. [Figure 12A] FIG. 12A is a diagram showing one embodiment of a light emission pattern of a light source array. [Figure 12B] FIG. 12B is a diagram showing one embodiment of a light emission pattern of a light source array. [Figure 13A] FIG. 13A is a diagram showing a modified example of the relationship between the partial viewing areas of the eyebox and the positions at which display light based on illumination light emitted from each partial area of ​​the directional light source unit is concentrated according to some embodiments. [Figure 13B] FIG. 13B is a diagram showing a modified example of the relationship between the partial viewing areas of the eyebox and the positions at which display light based on illumination light emitted from each partial area of ​​the directional light source unit is concentrated, according to some embodiments. [Figure 13C] FIG. 13C is a diagram showing a modified example of the relationship between the partial viewing areas of the eyebox and the positions at which display light based on illumination light emitted from each partial area of ​​the directional light source unit is concentrated, according to some embodiments. [Figure 13D] FIG. 13D is a diagram showing a modified example of the relationship between the partial viewing areas of the eyebox and the positions at which display light based on illumination light emitted from each partial area of ​​the directional light source unit is concentrated, according to some embodiments. [Figure 14]FIG. 14 is a diagram showing an aspect of the first light-emitting region (and the second light-emitting region) in the planar illumination unit before and after the spot area reduction process. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] 1 to 14, the configurations and operations of an exemplary vehicle display system, a head-up display device, and a display control device are described below. Note that the present invention is not limited to the following embodiments (including the contents of the drawings). Of course, modifications (including the deletion of components) can be made to the following embodiments. In addition, in the following description, descriptions of known technical matters are omitted as appropriate to facilitate understanding of the present invention.

[0012] 1 is mounted on a vehicle 1 and includes a head-up display device (hereinafter also referred to as HUD device) 20, a display control device 30 that controls the HUD device 20, a projection target 2 that includes a non-specular reflection element 3 (optional) described below and receives display light K from the HUD device 20, and an eye position detection unit 411 that detects an eye position 700 of an observer. The HUD device 20 projects display light K toward a front windshield (one example of the projection target 2) of the vehicle that is located in front of an observer (typically, a driver seated in the driver's seat of the vehicle 1) (the front can also be rephrased as the forward direction, or the traveling direction of the vehicle).

[0013] The non-specular reflection element 3 can be configured, for example, by a holographic element or a diffractive optical element (DOE). The holographic element is an optical member made of, for example, a hologram material such as a photopolymer or dichromated gelatin, which diffracts incident light in a desired direction, and is created by recording an interference pattern between an object light and a reference light. The non-specular reflection element 3 is, for example, sheet-shaped, is contained in the projection target portion 2, and diffracts the display light K projected by the HUD device 20 to the eye box 200, which is a predetermined area inside the vehicle 1. For example, the non-specular reflection element 3 described later can diffract the display light K received from the HUD device 20 to increase the size of the image (virtual image) visually recognized by the observer. The observer can visually recognize the virtual image V of the display image (not shown) displayed by the HUD device 20 on the far side of the projection target portion 2 by placing the eye 700 in the eye box 200. As described above, the non-specular reflection element 3 may be omitted. In other words, the vehicle display system 10 may not allow the user to see non-specularly reflected light from the projected portion 2 (non-specular reflection element 3), but may allow the user to see specularly reflected light from the projected portion 2 (not including the non-specular reflection element 3).

[0014] The "eye box" used in the description of this embodiment means (1) a region within which the entire virtual image V of the image can be seen, and at least a portion of the virtual image V of the image cannot be seen outside the region; (2) a region within which at least a portion of the virtual image V of the image can be seen, and no portion of the virtual image V of the image can be seen outside the region; (3) a region within which the entire virtual image V of the image can be seen at a predetermined brightness or higher, and at least a portion of the virtual image V of the image is less than the predetermined brightness outside the region; (4) a region within which at least a portion of the virtual image V of the image can be seen at a predetermined brightness or higher, and the entire virtual image V of the image is less than the predetermined brightness outside the region; or (5) a region within which at least a portion of the virtual image V can be viewed stereoscopically, when the HUD device 20 is capable of displaying a virtual image V that can be viewed stereoscopically, and no portion of the virtual image V can be viewed stereoscopically outside the region. That is, if the observer places his / her eyes (both eyes) outside the eyebox 200, the observer cannot see the entire virtual image V of the image, the entire visibility of the virtual image V of the image is very low and difficult to perceive, or the virtual image V of the image cannot be viewed stereoscopically. The predetermined brightness is, for example, about 1 / 50 of the brightness of the virtual image of the image viewed at the center of the eyebox. The "eyebox" is set to be the same as the area (also called the iris) where the observer's viewpoint is expected to be located in the vehicle in which the HUD device 20 is mounted, or to include most of the iris (for example, 80% or more).

[0015] The virtual image display area VS is a flat, curved, or partially curved area where an image generated inside the HUD device 20 is formed as a virtual image V, and is also called an imaging surface. The virtual image display area VS is a position where a display surface (e.g., an exit surface of a liquid crystal light modulation element) 52 of a display 50 of the HUD device 20 described later is formed as a virtual image, that is, the virtual image display area VS corresponds to a display surface 52 of the HUD device 20 described later (in other words, the virtual image display area VS is in a conjugate relationship with the display surface 52 of the display 50 described later). It can be said that the virtual image V viewed in the virtual image display area VS corresponds to an image displayed on a display surface 52 of the HUD device 20 described later. It is preferable that the virtual image display area VS itself has low visibility to the extent that it is not actually visible to the observer's eyes or is difficult to be visible.

[0016] In the virtual image display area VS, the angle (tilt angle θt in FIG. 1) between the horizontal direction (XZ plane) and the axis along the left-right direction (X-axis direction) of the vehicle 1 is set as the axis, and the angle between the line segment connecting the center 205 of the eye box 200 and the upper end VS1 of the virtual image display area VS and the line segment connecting the center 205 of the eye box 205 and the lower end VS2 of the virtual image display area VS is set as the vertical angle of view of the virtual image display area VS, and the angle (vertical arrangement angle θv in FIG. 1) between the bisector of this vertical angle of view and the horizontal direction (XZ plane) is set. The vertical arrangement angle θv is typically set downward from the horizontal direction (XZ plane), for example, +2 degrees (here, the positive direction of the vertical arrangement angle θv can be said to be a depression angle where the bisector of the vertical angle of view is oriented downward from the center 205 of the eyebox 200, and the negative direction of the vertical arrangement angle θv can be said to be an elevation angle where the bisector of the vertical angle of view is oriented upward from the center 205 of the eyebox 200).

[0017] In addition, the virtual image display region VS of this embodiment has a tilt angle θt of approximately 90 degrees so that it faces approximately directly when facing forward (positive direction of the Z axis). However, the tilt angle θt is not limited to this and can be set in the range of 40≦θt<90 degrees. In this case, for example, the tilt angle θt may be set to 60 degrees, and the virtual image display region VS may be disposed so that the upper region is farther away than the lower region as viewed from the viewer.

[0018] FIG. 2 is a diagram showing the configuration of the HUD device 20 of this embodiment. The HUD device 20 has a display 50 having a display surface 52 for displaying an image, a relay optical system (collecting optical system) 40, and a housing 22 that houses the display 50 and the relay optical system 40 and has a light exit window 21 that can emit display light K from the display 50 from the inside to the outside. The display 50 of this embodiment described later is a 2D display, but may be a 3D display. That is, the HUD device 20 may display parallax images corresponding to the eye points (left and right eyes) arranged in the eye box 200, allowing the observer to perceive a virtual image of a stereoscopic image based on the virtual image display area VS. The 3D display does not have to be a binocular parallax type 3D display, and may be, for example, a light field display. A part of the relay optical system 40 may be omitted.

[0019] The display 50 in FIG. 2 is composed of a light modulation element 51 and a directional light source unit 60. The display surface 52 is the surface of the light modulation element 51 on the observer side, and emits display light K of an image. The angle of the virtual image display region VS (including the tilt angle θt) can be set by setting the angle of the display surface 52 with respect to the optical axis Kp of the display light K traveling from the center of the display surface 52 toward the eye box 200 (the center 205 of the eye box 200) via the relay optical system 40 and the projected unit 2. In addition, the directional light source unit 60, which will be described later, can change the position where the virtual image V can be viewed (easily viewed) in the eye box 200 by changing the direction of the illumination light.

[0020] The relay optical system 40 is disposed on the optical path of the display light K (light traveling from the display 50 toward the eye box 200) emitted from the display 50, and is composed of one or more optical members that project the display light K from the display 50 onto the front windshield 2 outside the HUD device 20. The relay optical system 40 in FIG. 2 includes one concave first mirror 41 and one flat second mirror 42.

[0021] The first mirror 41 is, for example, a free curved surface shape having a positive optical power. In other words, the first mirror 41 may be a curved surface shape having different optical powers for each region, that is, the optical power added to the display light K may be different depending on the region (optical path) through which the display light K passes. Specifically, the optical power added by the relay optical system 40 may be different between the first display light K1, the second display light K2, and the third display light K3 (see FIG. 2) traveling from each region of the display surface 52 toward the eye box 200.

[0022] The second mirror 42 is, for example, a flat mirror, but is not limited thereto, and may be a curved surface having optical power. That is, the relay optical system 40 may combine a plurality of mirrors (for example, the first mirror 41 and the second mirror 42 in this embodiment) to add different optical power depending on the region (optical path) through which the display light K passes. The second mirror 42 may be omitted. That is, the display light K emitted from the display 50 may be reflected by the first mirror 41 to the projection target portion (front windshield) 2.

[0023] Further, in this embodiment, the relay optical system 40 includes two mirrors, but is not limited to this and may additionally or alternatively include one or more refractive optical elements such as lenses, diffractive optical elements such as holograms, reflective optical elements, or combinations thereof.

[0024] In addition, the relay optical system 40 of this embodiment has the function of setting the distance to the virtual image display area VS and the function of generating a virtual image that is an enlarged image of the image displayed on the display surface 52, by using this curved shape (an example of optical power), but in addition to this, it may also have the function of suppressing (correcting) distortion of the virtual image that may occur due to the curved shape of the front windshield 2.

[0025] Furthermore, the relay optical system 40 may be provided with one or more actuators (not shown) that are controlled by the display control device 30, and may be rotatable (and / or movable).

[0026] The light modulation element 51 is, for example, an LCD that is a transmissive display panel, and receives illumination light from the directional light source unit 60 and emits spatially light-modulated display light K toward the relay optical system 40 (second mirror 42). The light modulation element 51 is typically rectangular, for example, with a short side in which pixels corresponding to the vertical direction (Y-axis direction) of the virtual image V seen by the observer are arranged, and a long side in which pixels corresponding to the horizontal direction (X-axis direction) of the virtual image V seen by the observer that is perpendicular to the short side are arranged. The observer visually recognizes the transmitted light of the light modulation element 51 (reflected light, diffracted light, etc. may be considered depending on the type of the light modulation element 51) through the virtual image optical system 90. The virtual image optical system 90 is a combination of the relay optical system 40 and the front windshield 2 (non-regular reflection element 3 included therein) shown in FIG. 2. The light modulation element 51 (display 50) may be a reflective display panel using LCoS or DMD.

[0027] (First embodiment) Please refer to FIG. 3. FIG. 3 is a diagram showing an example of a main configuration of a HUD device 20 (a directional light source unit 60 of the first embodiment) having a function of performing viewpoint tracking spotlighting control. In the HUD device 20 (a directional light source unit 60) shown in FIG. 3, an eye box 200 is divided into n (25 in this embodiment) partial viewing zones E (E(1,1) to E(5,5) in FIG. 4A) which are divided into five in the left-right direction (X-axis direction) and five in the up-down direction (Y-axis direction) when an observer (a driver of a vehicle 1) faces the front of the vehicle 1. The HUD device 20 (a directional light source unit 60) performs viewpoint tracking spotlighting control in which a light emission pattern (a spotlighting area SP described later) is made different for each partial viewing zone E where the eye position of the driver (user) is detected, and a display light K is locally directed toward the partial viewing zone E where the eye position of the driver (user) is detected.

[0028] The display control device 30, which will be described later, detects the eye position of the viewer from an eye position detection unit 411, which will be described later, and controls the directional light source unit 60 so that a bright image (virtual image V) is visible only at the eye position of the viewer and its surroundings. These will be described in detail later.

[0029] The directional light source unit (light source element) 60 of the first embodiment is composed of a light source 70, a scanning device (scanner) 75, a first field lens (illumination optical system) 76, a second field lens (illumination optical system) 77, and a light diffusion optical system 80. The scanning device 75 can scan the laser light emitted from the light source 70 on m (m>n) partial regions D (the partial regions D in this embodiment are 121, from D(1,11) to D(11,11)) of the light diffusion optical system 80, and the display control device 30 controls the light source 70 to irradiate a strong laser light onto a partial region D (light-emitting region DM) corresponding to the partial viewing region E where the eye position detected by the eye position detection unit 411 exists, thereby allowing the observer to view an image (virtual image V). The light-emitting region DM is a partial region D that emits illumination light of a predetermined light intensity toward the light modulation element 51 among the partial regions D.

[0030] The light source 70 is, for example, a plurality of laser diodes each capable of emitting three different colors (e.g., R, G, B), and mixes a plurality of laser beams (e.g., white) using a dichroic mirror or the like, and directs the mixture as one illumination laser beam 74 toward the scanning device 75. The light source 70 is capable of adjusting the intensity of the illumination laser beam 74 (including turning it off) under the control of the display control device 30 described later. More specifically, the light source 70 is capable of adjusting the intensity of the illumination laser beam 74 in a time-division manner (including turning it off) in accordance with the scanning position of the scanning device 75 described later under the control of the display control device 30 described later.

[0031] The scanning device (scanner) 75 is an element that two-dimensionally deflects the illumination laser beam 74 emitted from the light source 70, and is an optical scanning device such as a polygon mirror, a galvanometer mirror, or a MEMS (Micro Electro Mechanical System) mirror device using MEMS technology. The scanning device 75 is a minute mirror that is configured to be able to oscillate using two axes perpendicular to each other. The scanning device 75, for example, scans at high speed in the main scanning direction and scans at low speed in the sub-scanning direction perpendicular to the main scanning direction.

[0032] The first field lens 76 is disposed on the optical path of the illumination laser beam 74 scanned by the scanning device 75 between the scanning device 75 and the light diffusion optical system 80. The first field lens 76 has a function of collimating the received illumination laser beam 74 in a direction parallel to the optical axis of the first field lens 76. The first field lens 76 is not an essential component and may be omitted. However, by providing the first field lens 76, the display light K obtained by modulating the illumination laser beam 74 emitted from each light-emitting region DM of the light diffusion optical system 80 can be directed accurately to each partial viewing zone E of the eye box 200. Although the first field lens 76 is a refractive optical system, one or more other refractive optical systems, reflective optical systems, or diffractive optical systems may be additionally or alternatively provided on the optical path of the illumination laser beam 74 between the scanning device 75 and the light diffusion optical system 80.

[0033] The second field lens (collecting optical system) 77 is disposed on the optical path of the illumination laser beam 74 diffused by each partial region D (light-emitting region DM) between the light diffusion optical system 80 and the light modulation element 51, and functions as a collecting optical system that collects the illumination light emitted from the different (light-emitting region DM) and the display light K modulated by the light modulation element 51 toward different regions in the eye box 200. The second field lens 77 of this embodiment is a front-stage optical member (backlight optical member) composed of one or more field lenses, and is designed so that the illumination laser beam 74 diffused by each partial region D (light-emitting region DM) of the light diffusion optical system 80 transmits and illuminates the entire surface of the light modulation element 51. The second field lens 77 is also designed so that the display light K emitted from the light modulation element 51 reaches a partial viewing area E at an arbitrary position in the eye box 200 via the virtual image optical system 90 (HUD optical member composed of the relay optical system 40 and the front windshield 2). The collecting optical system that collects the display light K obtained by modulating the illumination light emitted from the different partial regions D (light-emitting regions DM) by the light modulation element 51 toward the different partial viewing zones E in the eye box 200 may include various optical systems such as one or more refractive optical systems, reflective optical systems, and diffractive optical systems instead of (or in addition to) the field lens. Note that the display light K passes through the relay optical system 40 and the projected unit 2 before reaching the eye box 200 from the light modulation element 51. At this time, the display light K is deflected by the relay optical system 40 and the projected unit 2 to a certain extent before being collected toward different regions in the eye box 200. Therefore, in a broad sense, it can be said that the collecting optical system includes the relay optical system 40 and the projected unit 2 in addition to the second field lens (collecting optical system) 77. Also, the collecting optical system may be configured with the relay optical system 40 and the projected unit 2 without the second field lens (collecting optical system) 77.

[0034] The diffusion plate 78 is disposed on the optical path of the illumination laser beam 74 between the light modulation element 51 and the second field lens 77. The diffusion plate 78 diffuses the received illumination laser beam 74 and emits it to the light modulation element 51. This reduces luminance unevenness in the display light K generated by the illumination laser beam 74 emitted from the light-emitting region DM viewed in each partial viewing zone E of the eyebox 200. The diffusion plate 78 may be any optical member having a function of diffusing light, and may have a surface formed of, for example, a bead member, a fine uneven structure, or a rough surface. Alternatively, the diffusion plate 78 may be a dot sheet or a transparent milky white sheet.

[0035] First, an example in which a lens array 81 is used as the light diffusion optical system 80 will be described. The lens array 81 of this embodiment is divided into m partial regions D, and a micro lens having a curvature is formed in each partial region D. When an illumination laser beam 74 is incident on each partial region D, the illumination laser beam 74 refracted by the micro lens is emitted as diffused light. More specifically, the illumination laser beam 74 incident on any of the m partial regions D of the lens array 81 (light diffusion optical system 80) from the light source 70 is refracted by the micro lens, and after passing through the second field lens 77 and the diffusion plate 78, is irradiated on the entire surface of the light modulation element 51 (the "entire surface" here includes the entire region in which an image can be displayed and a part of the region (the entire surface of the use region) used to display the image). The optically modulated display light K passes through the virtual image optical system 90 and is irradiated on a predetermined region of the eye box 200 including at least the partial region D where the eye position 700 is present.

[0036] Next, an example in which a hologram recording medium 82 is used as the light diffusion optical system 80 will be described. The hologram recording medium 82 of this embodiment is divided into m partial regions D, and interference fringes are formed in each partial region D. When an illumination laser beam 74 is incident on each partial region D, the illumination laser beam 74 diffracted by the interference fringes is emitted as diffused light. More specifically, the illumination laser beam 74 incident on any one of the m partial regions D of the hologram recording medium 82 (light diffusion optical system 80) from the light source 70 is diffracted by the interference fringes, and after passing through the second field lens 77 and the diffusion plate 78, is irradiated onto the entire surface of the light modulation element 51, and the optically modulated display light K passes through the virtual image optical system 90 and is irradiated onto a predetermined region of the eye box 200 corresponding to the predetermined partial region D. The light diffusion optical system 80 may be a Fresnel lens.

[0037] FIG. 4C is a diagram showing how the scanning device (scanner) 75 scans the light diffusion optical system 80. In FIG. 4C, the first field lens 76 is omitted. The light diffusion optical system 80 has a third direction (α-axis) corresponding to the first direction (X-axis) in which the partial viewing areas E of the eyebox 200 are arranged horizontally, and a fourth direction (β-axis) corresponding to the second direction (Y-axis) in which the partial viewing areas E are arranged vertically. The scanning device 75 performs main scanning along the third direction (α-axis) and sub-scanning along the fourth direction (β-axis). In the example of FIG. 4C, the scanning device 75 passes through different partial areas D on the forward and backward paths of the main scanning, and irradiates a predetermined partial area D with the illumination laser beam 74 under the control of the display control device 30. Specifically, the scanning device 75 irradiates the illumination laser beam 74 on a predetermined partial region D (light-emitting region DM) selected by the display control device 30 while scanning at high speed from partial region D(11,1) to D(1,1) on the outward path, and does not irradiate the illumination laser beam 74 on a partial region D not selected by the display control device 30, and then, on the return path, irradiates the illumination laser beam 74 on the predetermined partial region D (light-emitting region DM) selected by the display control device 30 while scanning from partial region D(1,2) to D(11,2). However, the manner in which the scanning device (scanner) 75 scans the light diffusion optical system 80 is not limited to this.

[0038] In another example, the scanning device 75 may irradiate the illumination laser beam 74 only on the forward path (or, alternatively, only on the return path). Specifically, the scanning device 75 may irradiate the illumination laser beam 74 on a predetermined partial region D (light-emitting region DM) selected by the display control device 30 while scanning from partial region D(11,1) to D(1,1) on the forward path, and then scan up to partial region D(11,2) without irradiating the illumination laser beam 74, and again on the forward path, irradiate the illumination laser beam 74 on the predetermined partial region D (light-emitting region DM) selected by the display control device 30 while scanning from partial region D(11,2) to D(2,2).

[0039] In another example, the scanning device 75 may irradiate the illumination laser beam 74 to the same partial region D on the forward and return paths. Specifically, the scanning device 75 irradiates the illumination laser beam 74 to a predetermined partial region D (light-emitting region DM) selected by the display control device 30 while scanning from the partial region D(11,1) to D(1,1) on the forward path, and then irradiates the illumination laser beam 74 to the predetermined partial region D (light-emitting region DM) selected by the display control device 30 while scanning again from the partial region D(1,1) to D(11,1) on the return path. Note that the position in the partial region D to which the illumination laser beam 74 is irradiated differs between the forward and return paths. That is, the illumination laser beam 74 may be irradiated to an upper position (negative direction of β) of the predetermined partial region D on the forward path, and irradiated to a lower position (positive direction of β) of the predetermined partial region D on the return path.

[0040] 4A is a diagram illustrating partial viewing zones that are virtually divided within an eye box, and a state in which display light is directed to a plurality of partial viewing zones corresponding to the eye positions of a viewer. Here, it is assumed that the right eye 700R of the viewer is detected in the partial viewing zone E(2,3), and the left eye 700L is detected in the partial viewing zone E(4,3). The partial viewing area E (hereinafter also referred to as the right-eye spotlighting area SPR) to which the right-eye display light K is directed is a horizontally long elliptical area centered on the partial viewing area E(2,3) and spanning nine partial viewing areas E: the upper part of the upper partial viewing area E(2,2), the lower part of the lower partial viewing area E(2,4), the right part of the right partial viewing area E(1,3), the left part of the left partial viewing area E(3,3), the upper left part of the upper left partial viewing area E(3,2), the lower left part of the lower left partial viewing area E(3,4), the lower right part of the lower right partial viewing area E(1,4), and the upper right part of the upper right partial viewing area E(1,2). Similarly, the partial viewing area E to which the left-eye display light K is directed (hereinafter also referred to as the left-eye spotlighting area SPL) is a horizontally long elliptical area centered on the partial viewing area E(4,3) and spanning nine partial viewing areas E, including the upper part of the upper partial viewing area E(4,2), the lower part of the lower partial viewing area E(4,4), the right part of the right partial viewing area E(3,3), the left part of the left partial viewing area E(5,3), the upper left part of the upper left partial viewing area E(5,2), the lower left part of the lower left partial viewing area E(5,4), the lower right part of the lower right partial viewing area E(3,4), and the upper right part of the upper right partial viewing area E44(3,2).

[0041] FIG. 4B is a diagram illustrating the relationship between the state of the partial viewing area E of the eye box 200 and the position where the display light K based on the illumination light emitted from each partial area D of the directional light source unit 60 is collected. In FIG. 4B, the position where the display light K based on the illumination light emitted from each partial area D is collected is depicted by a dot (a hollow circle). However, this shows the position where the main ray of the display light K based on the illumination light emitted from each partial area D is directed, and the display light K is also irradiated around the dot (hollow circle). In the example of FIG. 4B, the directional light source unit 60 arranges the partial areas D so that the display light K can be irradiated at nine different points within one partial viewing area E. Specifically, there is one point at the center of the partial viewing area E, four points at the four corners of the partial viewing area E, and four points at the midpoints of each of the four corners of the partial viewing area E, for a total of nine points. In one embodiment shown in FIG. 4B, the display control device 30 controls the directional light source unit 60 so that the display light K is irradiated to all nine different locations in the partial viewing area E where the eye position 700 is present. The light-emitting area DM includes a main partial area DA that emits illumination light that is the source of the display light K that is condensed at the center of the partial viewing area E where the eye position 700 is present, and a sub-partial area DB that emits illumination light that is the source of the display light K that is condensed at a position farther away from the center of the partial viewing area E where the eye position 700 is present than the main partial area DA. That is, when the left eye position 700L is present in the partial viewing area E(4,3), the main partial area DA is the partial area D(8,6), and the sub-partial areas DB are the partial areas D(7,5), D(8,5), D(9,5), D(7,6), D(9,6), D(7,7), D(8,7), and D(9,7). On the other hand, when the right eye position 700R is located in the partial viewing area E(2,3), the main partial area DA is the partial area D(4,6), and the sub partial areas DB are the partial areas D(3,5), D(4,5), D(5,5), D(3,6), D(5,6), D(3,7), D(4,7), and D(5,7).

[0042] FIG. 4C is a diagram showing a state in which the illumination laser beam 74 is irradiated onto the light emitting region DM of the light diffusion optical system 80 in order to direct the display light K to the spot lighting area SP of FIG. 4A. The display control device 30 drives the light source 70 in accordance with the timing at which the scanning device 75 scans the light emitting region DM that emits illumination light of a predetermined light intensity among the m partial regions D. As a result, as shown in FIG. 4A, the display light K of the light modulation element 51 is directed to the spot lighting area SP (spot lighting area SPR for the right eye, spot lighting area SPL for the left eye) of the eye box 200 described above that corresponds to the light emitting region DM of the light diffusion optical system 80. Note that the light emitting region DM corresponding to the above-mentioned spot lighting area SPR for the right eye is referred to as the light emitting region DMR for the right eye, and the light emitting region DM corresponding to the above-mentioned spot lighting area SPL for the left eye is referred to as the light emitting region DML for the left eye. Therefore, the display light K obtained by modulating the illumination light emitted from the right-eye light-emitting region DMR by the light modulation element 51 is irradiated onto the right-eye spotlighting area SPR, and the display light K obtained by modulating the illumination light emitted from the left-eye light-emitting region DML by the light modulation element 51 is irradiated onto the left-eye spotlighting area SPL. Furthermore, the display control device 30 does not irradiate the illumination laser beam 74 onto the partial region D other than the right-eye light-emitting region DMR and the left-eye light-emitting region DML, or irradiates the illumination laser beam 74 with a very low light intensity. This allows the display light K to be efficiently irradiated onto the viewing zone E where the viewer's eyes are located, and power consumption to be kept low.

[0043] FIG. 5A is a diagram showing an example of the distribution of the light quantity (luminance) when the display light K is distributed in the spot lighting area SP. When the display control device 30 estimates that the eye position 700 is in the partial viewing area E(2,3), the display control device 30 drives the light source 70 in accordance with the timing at which the scanning device 75 scans the light emitting area DM so that the light emitting area DM (partial areas D(3,5), D(4,5), D(5,5), D(3,6), D(4,6), D(5,6), D(3,7), D(4,7), D(5,7)) of the light diffusion optical system 80 is irradiated with the illumination laser beam 74 so as to become a predetermined spot lighting area SP corresponding to the partial viewing area E(2,3). As a result, as shown in FIG. 4A, the display light K of the light modulation element 51 is directed to the predetermined spot lighting area SP of the eye box 200 corresponding to the light emitting area DM of the light diffusion optical system 80. The symbol Ld1 (Ld) in FIG. 5A indicates the light amount distribution in the left-right direction (X-axis direction) of the eye box 200 (the same applies to FIGS. 5B and 5C described later). The light quantity distribution Ld1 is a combination of display light based on illumination light emitted from the light emitting region DM (partial regions D(3,5), D(4,5), D(5,5), D(3,6), D(4,6), D(5,6), D(3,7), D(4,7), D(5,7), and a light quantity of a predetermined threshold value Lth (for example, 50% of the maximum light quantity (brightness) in the spot lighting area SP) or more is ensured within the boundary of the spot lighting area SP. Furthermore, the light quantity distribution Ld extends beyond the boundary of the spot lighting area SP of the partial viewing zone E and spreads, and the light intensity does not decrease sharply but gradually (with a gradient). Therefore, even when the viewpoint moves outside the boundary, the display (virtual image) becomes faint and disappears, and does not disappear abruptly. This suppresses (reduces) discomfort and anxiety.

[0044] In this embodiment, the sub-partial region DB emits illumination light that is the source of the display light K that is condensed at a position away from the center of the partial viewing region E(2,3). In particular, in the aspects shown in Figs. 5A, 5B, and 5C, the sub-partial region DB is arranged so that all of the display light K is condensed at the boundary of the partial viewing region E(2,3), so that even if the eye moves to the vicinity (or outside) of the partial viewing region E to which the display light K is directed, the image (virtual image V) with relatively high brightness due to the sub-partial region DB can be viewed. Fig. 5B shows a case where the eye position 700 moves from the partial viewing region E(2,3) to E(3,3) after the state of Fig. 5A. Even if system latency occurs and the spot lighting area SP does not switch (remains as the spot lighting area SP corresponding to the partial viewing area E(2,3)) at the moment of moving from the partial viewing area E(2,3) to E(3,3), the display light K with a relatively high luminance can be viewed by the light from the sub partial area DB that emits the illumination light that is the source of the display light K that is condensed at a position away from the center of the partial viewing area E(2,3). Fig. 5C is a diagram showing a state in which the spot lighting area SP has switched from one corresponding to the previous partial viewing area E(2,3) to one corresponding to the new partial viewing area E(3,3) after the state of Fig. 5B.

[0045] In another preferred example, the spot lighting area SP is illuminated with uniform (including substantially uniform) light so as to achieve a predetermined luminance level. Specifically, the light emitting regions DM are arranged, the emitted light intensity is adjusted, and / or the entire or partial optical adjustment of the focusing optical system is performed so that the light quantity distribution Ld has a top hat shape. This allows the image (virtual image V) to be viewed with a substantially uniform luminance within the illuminated partial viewing zone E even if the eye position moves. However, in this embodiment, the light quantity distribution Ld may be a Gaussian distribution.

[0046] In some embodiments, the display control device 30 (processor 33) may set the emission intensity of the sub partial region DB to be higher than 0.7 times and lower than 1.0 times the emission intensity of the main partial region DA. In this way, since the emission intensity of the sub partial region DB is relatively high, even when the eye position 700 is near the boundary of the partial viewing zone E, the display light K with relatively high luminance can be viewed by the light from the sub partial region DB.

[0047] In some embodiments, the display control device 30 (processor 33) may set the emission intensity of the sub partial region (DB) higher than that of the main partial region (DA). In this way, since the emission intensity of the sub partial region DB is relatively high, even when the eye position 700 is near the boundary of the partial viewing zone E, the display light K having a relatively high luminance can be viewed by the light from the sub partial region DB.

[0048] FIG. 6 is a block diagram of a vehicle display system 10, according to some embodiments. The display controller 30 includes one or more I / O interfaces 31, one or more processors 33, one or more image processing circuits 35, and one or more memories 37. The various functional blocks described in FIG. 6 may be configured in hardware, software, or a combination of both. FIG. 6 is only one embodiment, and the components shown may be combined into fewer components, or there may be additional components. For example, the image processing circuitry 35 (e.g., a graphics processing unit) may be included in one or more processors 33.

[0049] As shown in the figure, the processor 33 and the image processing circuit 35 are operatively coupled to the memory 37. More specifically, the processor 33 and the image processing circuit 35 can execute a program stored in the memory 37 to operate the vehicle display system 10 (the HUD device 20, the light modulation element 51, and the directional light source unit 60), for example, by controlling the directional light source unit 60 (the control of the directional light source unit 60 includes, for example, control of the illuminating laser beam 74 (illumination light) emitted by the light source 70, or control of the light source array 80A described later), generating and / or transmitting image data, and the like. The processor 33 and / or the image processing circuit 35 can include at least one general-purpose microprocessor (e.g., a central processing unit (CPU)), at least one application specific integrated circuit (ASIC), at least one field programmable gate array (FPGA), or any combination thereof. The memory 37 can include any type of magnetic medium such as a hard disk, any type of optical medium such as a CD and a DVD, any type of semiconductor memory such as a volatile memory, and a non-volatile memory. Volatile memory may include DRAM and SRAM, and non-volatile memory may include ROM and NVRAM.

[0050] As shown in the figure, the processor 33 is operably connected to the I / O interface 31. The I / O interface 31 communicates with, for example, a vehicle ECU 401 (described later) provided in the vehicle or other electronic devices (reference numerals 403 to 419 described later) according to the CAN (Controller Area Network) standard (also referred to as CAN communication). Note that the communication standard adopted by the I / O interface 31 is not limited to CAN, and includes, for example, wired communication interfaces such as CANFD (CAN with Flexible Data Rate), LIN (Local Interconnect Network), Ethernet (registered trademark), MOST (Media Oriented Systems Transport: MOST is a registered trademark), UART, or USB, or in-vehicle communication (internal communication) interfaces that are short-range wireless communication interfaces within several tens of meters, such as a personal area network (PAN) such as a Bluetooth (registered trademark) network, a local area network (LAN) such as an 802.11x Wi-Fi (registered trademark) network, etc. In addition, the I / O interface 31 may include an external communication (external communication) interface such as a wide area communication network (e.g., an Internet communication network) based on cellular communication standards such as a wireless wide area network (WWAN0, IEEE802.16-2004 (WiMAX: Worldwide Interoperability for Microwave Access)), IEEE802.16e-based (Mobile WiMAX), 4G, 4G-LTE, LTE Advanced, and 5G.

[0051] As shown in the figure, the processor 33 is interoperably connected to the I / O interface 31, thereby enabling information to be exchanged with various other electronic devices connected to the vehicle display system 10 (I / O interface 31). For example, a vehicle ECU 401, a road information database 403, a vehicle position detection unit 405, an external sensor 407, an operation detection unit 409, an eye position detection unit (gaze direction detection unit) 411, an IMU 413, a brightness sensor 415, a mobile information terminal 417, and an external communication device 419 are operably connected to the I / O interface 31. The I / O interface 31 may include a function of processing (converting, calculating, analyzing) information received from other electronic devices connected to the vehicle display system 10.

[0052] The light modulation element 51 is, for example, a liquid crystal display (LCD) and is operatively coupled to the processor 33 and the image processing circuitry 35. Thus, the image displayed by the light modulation element 51 may be based on image data received from the processor 33 and / or the image processing circuitry 35. The processor 33 and the image processing circuitry 35 control the image displayed by the light modulation element 51 based on information obtained from the I / O interface 31.

[0053] The vehicle ECU 401 acquires information on the state of the vehicle 1 from sensors and switches provided in the vehicle 1 (e.g., the ON / OFF state of a start switch (e.g., an accessory switch: ACC or an ignition switch: IGN) (an example of first start-up information), mileage, vehicle speed, accelerator pedal opening, brake pedal opening, engine throttle opening, injector fuel injection amount, engine RPM, motor RPM, steering angle, shift position, drive mode, information on headlights including headlight on / off and illumination range (an example of dimming reference information), various warning states, attitude (including roll angle and / or pitch angle), vehicle vibration (including vibration magnitude, frequency, and / or frequency)), etc., and collects and manages (which may also include control) the state of the vehicle 1, and as part of its function, can output a signal indicating a numerical value of the state of the vehicle 1 (e.g., the vehicle speed of the vehicle 1) to the processor 33 of the display control device 30. In addition to or instead of simply transmitting a value detected by a sensor or the like (e.g., the pitching angle is 3 degrees in the forward tilt direction) to the processor 33, the vehicle ECU 401 may transmit to the processor 33 a determination result based on one or more states of the vehicle 1 including the value detected by the sensor (e.g., that the vehicle 1 satisfies a predetermined condition for a forward tilt state) and / or an analysis result (e.g., that the vehicle has been in a forward tilt state due to braking, in combination with information on the brake pedal opening degree). For example, the vehicle ECU 401 may output to the display control device 30 a signal indicating a determination result that the vehicle 1 satisfies a predetermined condition previously stored in a memory (not shown) of the vehicle ECU 401. The I / O interface 31 may acquire the above-mentioned information from a sensor or a switch provided in the vehicle 1 without going through the vehicle ECU 401.

[0054] In addition, the vehicle ECU 401 may output an instruction signal to the display control device 30 to indicate the image to be displayed by the vehicle display system 10, and at this time, the coordinates, size, color, gradation, type, display mode, notification necessity of the image, necessity-related information on which the notification necessity is determined, and / or a signal to adjust the brightness of the virtual image (an example of a dimming signal) may be added to the instruction signal and transmitted.

[0055] The road information database 403 is included in a navigation device (not shown) provided in the vehicle 1, or an external server connected to the vehicle 1 via an external communication interface (I / O interface 31), and may read out information about the surroundings of the vehicle 1 (information related to real objects around the vehicle 1) such as road information (lanes, white lines, stop lines, pedestrian crossings, road widths, number of lanes, intersections, curves, forks, traffic regulations, etc.) on which the vehicle 1 is traveling, information about tunnels that may include tunnel entrances and exits (one example of dimming reference information), and information about land features (buildings, bridges, rivers, etc.), including their presence or absence, positions (including distances to the vehicle 1), directions, shapes, types, and detailed information, and transmit the information to the processor 33. The road information database 403 may also calculate an appropriate route (navigation information) from the departure point to the destination, and output a signal indicating the navigation information or image data indicating the route to the processor 33.

[0056] The vehicle position detection unit 405 is a GNSS (Global Navigation Satellite System) or the like provided in the vehicle 1, which detects the current position and direction of the vehicle 1, and outputs a signal indicating the detection result to the road information database 403, a mobile information terminal 417 described later, and / or an external communication device 419 via the processor 33 or directly. The road information database 403, the mobile information terminal 417 described later, and / or the external communication device 419 may select / generate information about the surroundings of the vehicle 1 by acquiring the position information of the vehicle 1 from the vehicle position detection unit 405 continuously, intermittently, or for each predetermined event, and output the information to the processor 33.

[0057] The external sensor 407 detects real objects present around the vehicle 1 (in front, on the sides, and behind). The real objects detected by the external sensor 407 may include, for example, obstacles (pedestrians, bicycles, motorcycles, other vehicles, etc.), the road surface of a driving lane, dividing lines, roadside objects, and / or features (buildings, etc.) described below. The external sensor is composed of, for example, a detection unit made up of a radar sensor such as a millimeter wave radar, ultrasonic radar, or laser radar, a camera, or a combination of these, and a processing device that processes (performs data fusion) detection data from the one or more detection units. Conventional well-known methods are applied to object detection by these radar sensors and camera sensors. By object detection by these sensors, the presence or absence of a real object in a three-dimensional space, and if a real object exists, the position (relative distance from the vehicle 1, position in the left and right direction when the traveling direction of the vehicle 1 is the forward and backward direction, position in the up and down direction, etc.), size (size in the lateral direction (left and right direction), height direction (up and down direction), etc.), moving direction (lateral direction (left and right direction), depth direction (front and back direction)), moving speed (lateral direction (left and right direction), depth direction (front and back direction)), and / or type, etc. of the real object may be detected. One or more external sensors 407 can detect a real object in front of the vehicle 1 at each detection period of each sensor, and output real object information (presence or absence of a real object, and if a real object exists, information on the position, size, and / or type of each real object, etc.) which is an example of real object information, to the processor 33. Note that the real object information may be transmitted to the processor 33 via another device (for example, the vehicle ECU 401). In addition, when a camera is used as a sensor, an infrared camera or a near-infrared camera is preferable so that a real object can be detected even when the surroundings are dark, such as at night. Furthermore, when using a camera as a sensor, a stereo camera that can obtain distance and the like from parallax is desirable.

[0058] The operation detection unit 409 is, for example, a hardware switch provided on a CID (Center Information Display) or an instrument panel of the vehicle 1, or a software switch combining an image and a touch sensor, and outputs operation information based on an operation by an occupant of the vehicle 1 (a user sitting in the driver's seat and / or a user sitting in the passenger seat) to the processor 33. For example, the operation detection unit 409 outputs, to the processor 33, display area setting information based on an operation to move the virtual image display area VS, eye box setting information based on an operation to move the eye box 200, image brightness setting information (an example of a dimming signal) based on an operation to change the brightness of the virtual image, information based on an operation to set the observer's eye position 700, and the like, by the user's operation. Note that the HUD device 20 may be switched on / off by the operation of the operation detection unit 409. That is, the operation detection unit 409 may output a signal (an example of first activation information) to turn on / off the HUD device 20 to the processor 33.

[0059] Eye position detection unit 411 may include a camera such as an infrared camera that detects eye position 700 of an observer sitting in the driver's seat of vehicle 1, and may output a captured image to processor 33. Processor 33 may obtain a captured image (one example of information from which eye position 700 can be estimated) from eye position detection unit 411, and analyze the captured image using a method such as pattern matching to detect the coordinates of observer's eye position 700, and output a signal indicating the coordinates of the detected eye position 700 to processor 33.

[0060] Furthermore, eye position detection unit 411 may output the analysis results of the image captured by the camera (for example, a signal indicating to which spatial region the observer's eye position 700 belongs in which a plurality of pre-set display parameters 600 (described later) correspond) to processor 33. Note that the method of acquiring the observer's eye position 700 in vehicle 1 or information that allows the observer's eye position 700 to be estimated is not limited to these, and may be acquired using a known eye position detection (estimation) technique.

[0061] Furthermore, the eye position detection section 411 may detect the moving speed and / or moving direction of the viewer's eye position 700, and output to the processor 33 a signal indicating the moving speed and / or moving direction of the viewer's eye position 700.

[0062] Furthermore, when the eye position detection unit 411 detects (10) a signal indicating that the viewer's eye position 700 is outside the eye box 200, (20) a signal estimating that the viewer's eye position 700 is outside the eye box 200, (30) a signal estimating that the viewer's eye position 700 is predicted to be outside the eye box 200, (40) a signal indicating that the viewer's eye position 700 cannot be detected, or (50) a signal estimating that the detection accuracy of the viewer's eye position 700 is degraded, the eye position detection unit 411 may determine that a predetermined condition is satisfied and output a signal indicating the corresponding state to the processor 33. Furthermore, the eye position detection unit 411 may output (60) a signal indicating that the eye position detection unit 411 has started normal operation (completed a startup operation) (an example of startup information (second startup information) of the eye position detection unit) or, conversely, (70) a signal indicating that the eye position detection unit 411 has not started normal operation (has not started) (an example of startup information (second startup information) of the eye position detection unit) to the processor 33.

[0063] (20) Signals indicating that the observer's eye position 700 is estimated to be outside the eye box 200 include, for example, (21) a signal indicating that the observer's eye position 700 cannot be detected, (22) a signal indicating that the observer's eye position 700 cannot be detected after movement of the observer's eye position 700 has been detected, and / or (23) a signal indicating that either of the observer's eye positions 700R, 700L is in the vicinity of the boundary 200A of the eye box 200 (the vicinity includes, for example, within a specified coordinate range from the boundary 200A).

[0064] (30) Examples of signals predicting that the observer's eye position 700 will be outside the eye box 200 include, for example, (31) a signal indicating that the newly detected eye position 700 is equal to or greater than an eye position movement distance threshold pre-stored in memory 37 compared to the previously detected eye position 700 (the movement of the eye position within a specified unit time is greater than a specified range), and (32) a signal indicating that the movement speed of the eye position is equal to or greater than an eye position movement speed threshold pre-stored in memory 37.

[0065] (40) The signal indicating that the observer's eye position 700 cannot be detected includes, for example, (41) a signal indicating that the observer's eye position 700 cannot be detected, (42) a signal indicating that the observer's eye position 700 cannot be detected after movement of the observer's eye position 700 has been detected, and / or (43) a signal indicating that either the observer's eye position 700R, 700L cannot be detected, etc.

[0066] (50) Examples of signals indicating that the detection accuracy of the observer's eye position 700 has decreased include, for example, (51) a signal indicating that the newly detected eye position 700 is equal to or greater than an eye position movement distance threshold pre-stored in memory 37 compared to the previously detected eye position 700 (the movement of the eye position within a specified unit time is greater than a specified range), (52) a signal indicating that the movement speed of the eye position is equal to or greater than an eye position movement speed threshold pre-stored in memory 37, and / or (53) a signal indicating that either of the observer's eye positions 700R, 700L cannot be detected.

[0067] The eye position detection unit 411 may also have a function as the gaze direction detection unit 411. The gaze direction detection unit 411 may include an infrared camera or a visible light camera that captures an image of the face of an observer sitting in the driver's seat of the vehicle 1, and may output the captured image to the processor 33. The processor 33 can specify the gaze direction (and / or the gaze position) of the observer by acquiring an image (one example of information that can estimate the gaze direction) from the gaze direction detection unit 411 and analyzing the captured image. The gaze direction detection unit 411 may analyze an image captured by a camera and output a signal indicating the gaze direction (and / or the gaze position) of the observer, which is the analysis result, to the processor 33. The method of acquiring information that can estimate the gaze direction of the observer of the vehicle 1 is not limited to these, and may be acquired using other known gaze direction detection (estimation) techniques such as an EOG (Electro-oculogram) method, a corneal reflection method, a scleral reflection method, a Purkinje image detection method, a search coil method, and an infrared fundus camera method.

[0068] The IMU 413 may include a combination of one or more sensors (e.g., an accelerometer and a gyroscope) configured to detect the position, orientation, and changes therein (speed of change, acceleration of change) of the vehicle 1 based on inertial acceleration. The IMU 413 may output the detected values ​​(the detected values ​​include signals indicating the position, orientation, and changes therein (speed of change, acceleration of change) of the vehicle 1) and the results of analyzing the detected values ​​to the processor 33. The results of the analysis may be signals indicating a determination result as to whether or not the detected values ​​satisfy a predetermined condition, and may be, for example, a signal indicating that the behavior (vibration) of the vehicle 1 is small based on a value regarding the change (speed of change, acceleration of change) of the position or orientation of the vehicle 1.

[0069] The brightness sensor 415 detects the illuminance or brightness of a predetermined range of the foreground present in front of the cabin of the vehicle 1 as the outside brightness (an example of dimming reference information), or detects the illuminance or brightness inside the cabin as the inside brightness (an example of dimming reference information). The brightness sensor 415 is, for example, a phototransistor or a photodiode, and is mounted on the instrument panel, the rearview mirror, the HUD device 20, or the like of the vehicle 1 shown in FIG.

[0070] The mobile information terminal 417 is a smartphone, a notebook computer, a smartwatch, or other information device that can be carried by the observer (or other passengers of the vehicle 1). The I / O interface 31 can communicate with the mobile information terminal 417 by pairing with the mobile information terminal 417, and acquires data recorded in the mobile information terminal 417 (or a server via the mobile information terminal). The mobile information terminal 417 has, for example, the same functions as the above-mentioned road information database 403 and the vehicle position detection unit 405, and may acquire the road information (an example of real object related information; an example of dimming reference information) and transmit it to the processor 33. The mobile information terminal 417 may also acquire commercial information (an example of real object related information) related to commercial facilities in the vicinity of the vehicle 1 and transmit it to the processor 33. In addition, the mobile information terminal 417 may transmit schedule information of the holder of the mobile information terminal 417 (e.g., the observer), information on incoming calls at the mobile information terminal 417, information on received emails, etc. to the processor 33, and the processor 33 and the image processing circuit 35 may generate and / or transmit image data related to these.

[0071] The external communication device 419 is a communication device that exchanges information with the vehicle 1, and is, for example, another vehicle connected to the vehicle 1 by vehicle-to-vehicle communication (V2V: Vehicle To Vehicle), a pedestrian (a mobile information terminal carried by a pedestrian) connected to the vehicle 1 by vehicle-to-pedestrian communication (V2P: Vehicle To Pedestrian), and a network communication device connected by vehicle-to-roadside infrastructure (V2I: Vehicle To Roadside Infrastructure), and in a broad sense, includes everything connected to the vehicle 1 by communication (V2X: Vehicle To Everything). The external communication device 419 may obtain the positions of, for example, pedestrians, bicycles, motorcycles, other vehicles (such as a preceding vehicle), road surfaces, dividing lines, roadside objects, and / or features (such as buildings), and output them to the processor 33. Moreover, the external communication device 419 has the same function as the vehicle position detection unit 405 described above, and may acquire position information of the vehicle 1 and transmit it to the processor 33, and further has the function of the road information database 403 described above, and may acquire the road information (an example of real object related information; an example of dimming reference information) and transmit it to the processor 33. Note that the information acquired from the external communication device 419 is not limited to the above.

[0072] The software components stored in the memory 37 include an eye position detection module 502, an eye position estimation module 504, an eye position prediction module 506, an eye position state determination module 508, an activation determination module 510, a release condition determination module 512, a spot lighting area setting module 514, a display parameter setting module 516, a graphics module 518, a light source driving module 520, an actuator driving module 522, and a dimming value setting module 524, etc.

[0073] 7A, 7B, 7C, and 7D are flow diagrams illustrating a method S100 for performing operations for setting a spotlighting area based on an activation state of a HUD device 20, etc., according to some embodiments. The method S100 is performed in a HUD device 20 including a display, and a display control device 30 that controls the HUD device 20. Some operations of the method S100 described below are optionally combined, the sequence of some operations is optionally changed, and some operations are optionally omitted.

[0074] First, FIG. 7A is referred to. The start-up determination module 510 in FIG. 6 acquires start-up information indicating the start-up state of at least one of the vehicle 1, the HUD device 20, the display control device 30, and the eye position detection unit 411, and determines whether the HUD device 20 or the eye position detection unit 411 is in the start-up state based on the start-up information (step S110). Step S110 will be described in detail later with reference to FIG. 7B and FIG. 7C. Note that the "start-up state" here refers to a state from when the entire vehicular display system 10, the HUD device 20, or the eye position detection unit 411 starts operating due to power being supplied or a start switch being turned on, until the vehicular display system 10 starts normal operation, that is, a state until the vehicular display system 10 can normally set the spot lighting area SP according to the eye position 700 of the observer.

[0075] (Step S140) Next, the spotlighting area setting module 514 in FIG. 6 sets the spotlighting area SP based on the determination in step S110.

[0076] (Step S150) In step S140, if it is determined in step S110 that the spotlighting area is not activated, the spotlighting area setting module 514 sets the spotlighting area SP to a first spotlighting area SP1. In step S150, the spotlighting area setting module 514 sets the spotlighting area SP to an area (first spotlighting area SP1) narrower than a second spotlighting area SP2 described later, in accordance with (so as to include at least) the eye position 700 detected by the eye position detection module 502 described later, the eye position 700 estimated by the eye position estimation module 504, and / or the eye position 700 predicted by the eye position prediction module 506 described later.

[0077] (Start-up light emission process (step S160)) On the other hand, in step S140, when it is determined that the spotlighting area SP is in the activated state in step S110, the spotlighting area setting module 514 sets the spotlighting area SP to a second spotlighting area SP2 wider than the first spotlighting area SP1 (activation light emission process (step S160)). In the activation light emission process (step S160), the spotlighting area setting module 514 sets the spotlighting area SP to the second spotlighting area SP2 pre-stored in the memory 37, not based on the eye position 700 detected by the eye position detection module 502 described later, the eye position 700 estimated by the eye position estimation module 504, and / or the eye position 700 predicted by the eye position prediction module 506. Here, the second spotlighting area SP2 may be the entire area of ​​the eye box 200. As a result, if the eye position 700 is within the eye box 200, the observer can view the virtual image V even when at least one of the HUD device 20, the display control device 30, and the eye position detection unit 411 has just been started and their operations are unstable. Also, it is possible to reduce the situation where the display light K is not directed to the eye position 700.

[0078] In some embodiments, the second spotlighting area SP2 in the startup light emission process (step S160) may be a part of the eyebox 200. The second spotlighting area SP2 in step S160 may be a square including the center of the eyebox 200, a rectangle elongated in the left-right direction of the vehicle 1, or an ellipse elongated in the left-right direction of the vehicle 1. In addition, when the first spotlighting area SP1 is a plurality of areas (two in this case) such as the right-eye spotlighting area SPR and the left-eye spotlighting area SPL, the second spotlighting area SP2 in step S160 may also be a plurality of areas. In this case, the second spotlighting area SP2 (a right-eye spotlighting area SPR wider than the right-eye spotlighting area SPR that is the first spotlighting area SP1, and a left-eye spotlighting area SPL wider than the left-eye spotlighting area SPL that is the first spotlighting area SP1) may be set so that the display light K is directed to two predetermined partial viewing zones E stored in advance in the memory 37. In addition, the two predetermined partial viewing zones E pre-stored in memory 37 may be, for example, areas centered on two points separated by an average interpupillary distance of 64 mm with the center of the eye box 200 as the center of symmetry (but are not limited to this).

[0079] In the start-up light emission process (step S160), the release condition determination module 512 in Fig. 6 determines whether or not the release condition of the start-up light emission process is satisfied, and if it is determined that the release condition is satisfied, the spot lighting area setting module 514 makes the spot lighting area SP narrower than the second spot lighting area SP2 (spot area reduction process (step S170)). The spot area reduction process (step S170) will be described in detail later with reference to Fig. 7D.

[0080] (Step S120) Next, referring to Fig. 7B, the start-up determination module 510 in Fig. 6 acquires first start-up information (one example of the start-up information) indicating the start-up state of at least one of the vehicle 1, the HUD device 20, and the display control device 30, and determines whether or not they are in the start-up state based on the first start-up information (step S120).

[0081] In step S120, the start-up determination module 510 may detect, for example, from the vehicle ECU 401 or the like, a signal indicating the start-up state of the vehicle 1 (for example, a signal (an example of first start-up information) indicating the ON / OFF state of a start-up switch (for example, an accessory switch: ACC or an ignition switch: IGN)), and if it detects the start-up of the vehicle 1, may determine that the HUD device 20 is in an activated state (step S121).

[0082] Also, in step S120, the startup determination module 510 may detect, for example, a signal to turn on / off the HUD device 20 (an example of first startup information) from the operation detection unit 409, and when a signal to turn on the HUD device 20 is detected, determine that the HUD device 20 is in an activated state (step S123).

[0083] Also, in step S120, the startup determination module 510 may, for example, provide a voltage detection circuit (not shown) (or a current detection circuit (not shown)) in the HUD device 20, detect a signal (an example of first startup information) indicating the presence or absence of power supply to the HUD device 20 from the voltage detection circuit (or the current detection circuit), and if a signal indicating that power has been supplied to the HUD device 20 is detected, determine that the HUD device 20 is in an activated state (step S125).

[0084] Also, in step S120, the startup determination module 510 may, for example, provide a voltage detection circuit (not shown) (or a current detection circuit (not shown)) in the display control device 30 (processor 33), detect a signal (an example of first startup information) indicating the presence or absence of power supply to the display control device 30 (processor 33) from the voltage detection circuit (or the current detection circuit), and if a signal indicating that power has been supplied to the display control device 30 (processor 33) is detected, determine that the HUD device 20 is in an activated state (step S127).

[0085] The method for determining whether the HUD device 20 is in an activated state is not limited to the above, and any known means may be used.

[0086] (Step S130) Next, referring to Fig. 7C, the start-up determination module 510 in Fig. 6 acquires second start-up information (an example of the start-up information) indicating the start-up state of the eye position detection unit 411, and determines whether or not the eye position detection unit 411 is in the start-up state based on the second start-up information (step S130).

[0087] In step S130, if start-up determination module 510 is capable of detecting from eye position detection unit 411 a signal (an example of start-up information (second start-up information) of the eye position detection unit) indicating that eye position detection unit 411 has started normal operation (start-up operation has been completed), and has not detected from eye position detection unit 411 a signal (an example of start-up information (second start-up information) of the eye position detection unit) indicating that eye position detection unit 411 has started normal operation (start-up operation has been completed), start-up determination module 510 may determine that eye position detection unit 411 is in an activated state (step S131).

[0088] Furthermore, in step S130, startup determination module 510 is capable of periodically (or irregularly) detecting a signal indicating that eye position detection unit 411 is in startup operation (an example of startup information (second startup information) of the eye position detection unit) from eye position detection unit 411, and may determine that eye position detection unit 411 is in an startup state when it periodically (or irregularly) detects a signal indicating that eye position detection unit 411 is in startup operation (an example of startup information (second startup information) of the eye position detection unit) (step S133).

[0089] Furthermore, in step S130, startup determination module 510 is capable of periodically (or irregularly) detecting a signal indicating that startup of eye position detection unit 411 has not been completed (an example of startup information (second startup information) of the eye position detection unit) from eye position detection unit 411, and may determine that eye position detection unit 411 is in an activated state when a signal indicating that startup of eye position detection unit 411 has not been completed (an example of startup information (second startup information) of the eye position detection unit) is detected periodically (or irregularly) (step S135).

[0090] Furthermore, in step S130, startup determination module 510 is capable of periodically (or irregularly) detecting a signal indicating that startup of eye position detection unit 411 has not been completed (an example of startup information (second startup information) of the eye position detection unit) from eye position detection unit 411, and may determine that eye position detection unit 411 is in an activated state when a signal indicating that startup of eye position detection unit 411 has not been completed (an example of startup information (second startup information) of the eye position detection unit) is detected periodically (or irregularly) (step S135).

[0091] Also, in step S130, after it is determined that the HUD device 20 (display control device 30) is in an activated state in step S120, if the activation determination module 510 has not detected eye position information from the eye position detection module 502 (or information from which the eye position can be estimated from the eye position estimation module 504, or information from which the eye position can be predicted from the eye position prediction module 506) a predetermined number of times (once or a predetermined multiple times), the activation determination module 510 may determine that the eye position detection unit 411 is in an activated state (step S137).

[0092] The method for determining whether eye position detection unit 411 is in an activated state is not limited to the above, and any known means may be used.

[0093] (Spot area reduction process (step S170)) Next, referring to Fig. 7D, the release condition determination module 512 in Fig. 6 determines whether the release condition of the startup light emission process S160 is satisfied, and if the release condition is satisfied, reduces the spotlighting area SP from the second spotlighting area SP2 to the normal first spotlighting area SP1.

[0094] In step S170, the release condition determination module 512 in FIG. 6 may determine that the release condition is met if a predetermined time pre-stored in the memory 37 has elapsed since it was determined in step S110 that the device is in an activated state (an example of step S180).

[0095] Also, in step S170, the release condition determination module 512 in FIG. 6 is capable of detecting that the display control device 30 (processor 33) has completed the startup sequence, and after it is determined in step S110 that the display control device 30 (processor 33) is in the started state, if it detects that the display control device 30 (processor 33) has completed the startup sequence, it may determine that the release condition is met (an example of step S180).

[0096] Also, in step S170, the release condition determination module 512 in FIG. 6 is capable of detecting, for example, a signal (an example of release information) from eye position detection unit 411 indicating that eye position detection unit 411 has started normal operation (that start-up operation has been completed), and when it detects a signal (an example of release information) from eye position detection unit 411 indicating that eye position detection unit 411 has started normal operation (that start-up operation has been completed), it may determine that the release condition is met (an example of step S180).

[0097] Also, in step S170, the release condition determination module 512 in FIG. 6 is capable of periodically (or irregularly) detecting a signal (an example of release information) indicating that eye position detection unit 411 is in startup operation, and may determine that the release condition is met when eye position detection unit 411 no longer periodically (or irregularly) detects a signal (an example of release information) indicating that eye position detection unit 411 is in startup operation (an example of step S180).

[0098] Also, in step S170, the release condition determination module 512 in FIG. 6 is capable of periodically (or irregularly) detecting a signal indicating that startup of eye position detection unit 411 has not been completed (an example of the release information) from eye position detection unit 411, and may determine that the release condition has been met when the signal indicating that startup of eye position detection unit 411 has not been completed (an example of the release information) is no longer detected periodically (or irregularly) (an example of step S180).

[0099] Also, in step S170, the release condition determination module 512 in FIG. 6 is capable of periodically (or irregularly) detecting a signal indicating that startup of eye position detection unit 411 has not been completed (an example of the release information) from eye position detection unit 411, and may determine that the release condition has been met when the signal indicating that startup of eye position detection unit 411 has not been completed (an example of the release information) is no longer detected periodically (or irregularly) (an example of step S180).

[0100] Also, in step S170, after it is determined in step S120 that the HUD device 20 (display control device 30) is in an activated state, the release condition determination module 512 in FIG. 6 may determine that the release condition is met, for example, when eye position information from the eye position detection module 502 (or information from which the eye position can be estimated from the eye position estimation module 504, or information from which the eye position can be predicted from the eye position prediction module 506) is detected a predetermined number of times (once or a predetermined multiple times) (an example of step S180).

[0101] (Spot area reduction process (step S190)) In step S170, when it is determined that the release condition is satisfied in step S180, the spotlighting area setting module 514 sets the spotlighting area SP from the second spotlighting area SP2 to a first spotlighting area SP1 narrower than the second spotlighting area SP2. In the spotlighting area reduction process (step S190), the spotlighting area setting module 514 sets the spotlighting area SP to an area (first spotlighting area SP1) narrower than the second spotlighting area SP2 in accordance with (at least including) the eye position 700 detected by the eye position detection module 502, the eye position 700 estimated by the eye position estimation module 504, and / or the eye position 700 predicted by the eye position prediction module 506, which will be described later. This makes it possible to reduce the display light K directed toward the eye box 200 area other than the eye position 700, and to suppress power consumption. In addition, it is possible to reduce the occurrence of stray light due to the display light K not directed toward the eye position 700.

[0102] (First spot area reduction process (step S191)) In step S170, if it is determined in step S180 that the cancellation condition is satisfied, the spotlighting area setting module 514 may instantly switch the spotlighting area SP from the second spotlighting area SP2 to the first spotlighting area SP1.

[0103] (Second spot area reduction process (step S192)) Also, in step S170, if it is determined that the cancellation condition is satisfied in step S180, the spotlighting area setting module 514 may gradually reduce the spotlighting area SP from the second spotlighting area SP2, and switch it to a first spotlighting area SP1 that matches the eye position 700 detected by the eye position detection module 502 described below, the eye position 700 estimated by the eye position estimation module 504, and / or the eye position 700 predicted by the eye position prediction module 506.

[0104] The eye position detection module 502 in FIG. 6 detects the eye position 700 of the observer (obtains information indicating the eye position 700) via the eye position detection unit 411. The eye position detection module 502 includes various software components for executing various operations related to detecting coordinates indicating the eye position 700 of the observer (positions in the X and Y axis directions, which are an example of a signal indicating the eye position 700), detecting coordinates indicating the height of the observer's eyes (positions in the Y axis direction, which are an example of a signal indicating the eye position 700), detecting coordinates indicating the height and depth direction position of the observer's eyes (positions in the Y and Z axis directions, which are an example of a signal indicating the eye position 700), and / or detecting coordinates indicating the eye position 700 of the observer (positions in the X, Y, and Z axis directions, which are an example of a signal indicating the eye position 700). The information indicating the eye position 700 is an example of startup information (second startup information) of the eye position detection unit.

[0105] The eye position 700 detected by the eye position detection module 502 includes the respective positions 700R and 700L of the right eye and the left eye, a predetermined one of the right eye position 700R and the left eye position 700L, one of the right eye position 700R and the left eye position 700L that is detectable (easy to detect), or a position calculated from the right eye position 700R and the left eye position 700L (for example, the midpoint between the right eye position and the left eye position), etc. For example, the eye position detection module 502 determines the eye position 700 based on the observation position acquired from the eye position detection unit 411 immediately before the timing of updating the display settings.

[0106] In addition, the eye position detection unit 411 may detect the movement direction and / or movement speed of the observer's eye position 700 based on multiple observation positions with different detection timings of the observer's eyes acquired from the eye position detection unit 411, and output a signal indicating the movement direction and / or movement speed of the observer's eye position 700 to the processor 33.

[0107] The eye position estimation module 504 acquires information that allows the eye position to be estimated. The information that allows the eye position to be estimated is, for example, the captured image acquired from the eye position detection unit 411, the position of the driver's seat of the vehicle 1, the position of the observer's face, the height of the seat, or the observation positions of the eyes of multiple observers. The eye position estimation module 504 estimates the eye position 700 of the observer of the vehicle 1 from the information that allows the eye position to be estimated. The eye position estimation module 504 includes various software components for executing various operations related to estimating the observer's eye position 700, such as estimating the observer's eye position 700 from the captured image acquired from the eye position detection unit 411, the position of the driver's seat of the vehicle 1, the position of the observer's face, the height of the seat, or the observation positions of the eyes of multiple observers. That is, the eye position estimation module 504 may include table data, an arithmetic expression, and the like for estimating the observer's eye position 700 from the information that allows the eye position to be estimated. The above-mentioned information that allows the eye position to be estimated is an example of startup information (second startup information) of the eye position detection unit.

[0108] The eye position estimation module 504 may calculate the estimated position of the eye position 700 based on the eye observation position acquired from the eye position detection unit 411 immediately before the timing of updating the spot lighting area and one or more eye observation positions acquired in the past, for example, by a method such as a weighted average.

[0109] The eye position prediction module 506 acquires information that can predict the viewer's eye position 700. The information that can predict the viewer's eye position 700 is, for example, the latest observation position acquired from the eye position detection unit 411, or one or more observation positions acquired in the past. The eye position prediction module 506 includes various software components for performing various operations related to predicting the eye position 700 based on the information that can predict the viewer's eye position 700. Specifically, for example, the eye position prediction module 506 predicts the viewer's eye position 700 at the timing when the viewer visually recognizes the image to which the new display setting is applied. The eye position prediction module 506 may predict the next value using one or more past observation positions using a prediction algorithm such as the least squares method, a Kalman filter, an α-β filter, or a particle filter. The information that can predict the viewer's eye position 700 is an example of startup information (second startup information) of the eye position detection unit.

[0110] The spotlighting area setting module 514 controls an area (spotlighting area SP) to which the display light K is directed, based on the acquired information indicating the eye position 700 (information that allows the eye position to be estimated, or information that allows the viewer's eye position 700 to be predicted). The spotlighting area setting module 514 sets an area including the partial viewing area E where the viewer's left eye position 700L exists and a part or all of the surrounding partial viewing area E as a left-eye spotlighting area SPL, and sets an area including the partial viewing area E where the viewer's right eye position 700R exists and a part or all of the adjacent partial viewing area E as a right-eye spotlighting area SPR. Specifically, as shown in FIG. 4B , the spotlighting area setting module 514 controls the directional light source unit 60 so that the display light K is directed toward a right-eye spotlighting area SPR including a partial viewing area E(2,3) where the observer's right eye 700R is located and parts of the surrounding partial viewing areas E(1,2), E(2,2), E(2,3), E1,3), E(3,3), E(1,4), E(2,4), and E(2,5), and toward a right-eye spotlighting area SPR including a partial viewing area E(4,3) where the observer's left eye 700L is located and parts of the surrounding partial viewing areas E(3,2), E(4,2), E(5,2), E3,3), E(5,3), E(3,4), E(4,4), and E(5,4). In this case, the spot lighting area setting module 514 sets a main partial area DA (D(4,6)) that directs the display light K to the center of the partial viewing area E(2,3) where the right eye 700R of the observer is located, and sub partial areas DB (D(3,5), D(4,5), D(5,5), D(3,6), D(5,6), D(3,7), D(4,7), and D(5,7)) that direct the display light K to areas other than the center in the partial viewing area E(2,3) where the right eye 700R is located, is caused to emit light at a predetermined light intensity or higher, and the main partial area DA (D(8,6)) that directs display light K toward the center of the partial viewing area E(4,3) where the viewer's left eye 700L is located, and the sub partial areas DB (D(7,5), D(8,5), D(9,5), D(7,6), D(9,6), D(7,7), D(8,7), and D(9,7)) that direct display light K toward an area that is not at the center of the partial viewing area E(2,3) where the left eye 700L is located are caused to emit light at a predetermined light intensity or higher.The spot lighting area setting module 514 does not irradiate the illumination laser beam 74 (or irradiates an illumination laser beam 74 with very low light intensity) to partial areas D other than the right eye light-emitting area DMR (main partial area DA + sub partial area DB for the right eye) and the left eye light-emitting area DML (main partial area DA + sub partial area DB for the left eye).

[0111] Also, the spotlighting area setting module 514 may set a single binocular spotlighting area SPC including the partial viewing area E where the observer's left eye position 700L is located and the partial viewing area E where the observer's right eye position 700R is located. Specifically, the spotlighting area setting module 514 directs the display light K to the binocular spotlighting area SPC including the partial viewing area E(2,3) where the observer's right eye 700R is located, the partial viewing area E(4,3) where the left eye 700L is located, and the surrounding partial viewing areas E(1,2), E(2,2), E(3,2), E(4,2), E(5,2), E(1,3), E(3,3), E(3,5), E(1,4), E(2,4), E(3,4), E(4,4), and a part of E(5,4), as shown in FIG. 8A. In this case, the spot lighting area setting module 514 causes the main partial area DA(D(4,6)) that directs the display light K to the center of the partial viewing area E(2,3) where the viewer's right eye 700R is located, the main partial area DA(D(8,6)) that directs the display light K to the center of the partial viewing area E(4,3) where the left eye 700L is located, and the sub partial areas DB(D(5,4) to D(7,4), D(2,5) to D(9,5), D(2,6), D(3,6), D(5,6) to D(7,6), D(9,6), D(2,7) to D(9,7), and D(5,8) to D(7,8)) surrounding the main partial area DA(D(4,6)) to emit light at a predetermined light intensity or higher. The spot lighting area setting module 514 does not irradiate the illumination laser beam 74 (or irradiates an illumination laser beam 74 with very low light intensity) to partial areas D other than these light-emitting areas DMC for both eyes (main partial area DA for the right eye + main partial area DA for the right eye + sub partial area DB).

[0112] 9 is a diagram showing an embodiment of the first light-emitting region DM1 (first light-emitting region DML1 for the left eye, first light-emitting region DMR1 for the right eye) and the second light-emitting region DM2 (second light-emitting region DML2 for the left eye, second light-emitting region DMR2 for the right eye) in the planar illumination unit SE. If the vertical length of the light-emitting region DM is designated by the symbol PV and the horizontal length by the symbol PH, the first light-emitting region DM1 is shorter than the second light-emitting region DM2 in both the vertical length PV and the horizontal length PH. The second light-emitting region DM2 includes the first light-emitting region DM1 and its surrounding light-emitting region DM. In this way, when the eye position 700 is positioned forward (second position 702 based on the first position 701), by widening the light-emitting area DM of the light source element 60 (light source element 60A described later), the virtual image V of the image that can be displayed on the entire display surface 52 of the light modulation element 51 (here, "the entire" includes 100% of the displayable area of ​​the light modulation element 51, or a predetermined display area in the displayable area (e.g., 70% of the entire area)) can be viewed by the observer at the desired brightness (or higher). In the example of FIG. 9, a first left-eye light-emitting region DML1 (second left-eye light-emitting region DML2) for the left eye 700L and a first right-eye light-emitting region DMR1 (second right-eye light-emitting region DMR2) for the right eye 700R are provided, but this is not limited to this. The spotlighting area setting module 514 in another example may change the width of only the light-emitting region DM directed toward one of the eyes, based on the eye position (Z-axis coordinate) in the longitudinal direction of the vehicle 1. Furthermore, the spotlighting area setting module 514 in another example may change the width of a single connected binocular light-emitting region DMC directed toward a single connected binocular spotlighting area SPC including the partial viewing area E where the left eye position 700L exists and the partial viewing area E where the right eye position 700R exists, based on the eye position (Z-axis coordinate) in the longitudinal direction of the vehicle 1.

[0113] The spot lighting area setting module 514 may control the light amount distribution Ld in the light emitting region DM of the planar lighting unit SE based on the acquired eye position (Z-axis coordinate) in the front-rear direction of the vehicle 1.

[0114] In some embodiments, the spot lighting area setting module 514 may set the emission intensity of the sub partial region DB to be greater than 0.7 times the emission intensity of the main partial region DA. In this way, since the emission intensity of the sub partial region DB is relatively strong, even when the eye position 700 is near the boundary of the partial viewing zone E, the display light K with a relatively high luminance can be viewed by the light from the sub partial region DB.

[0115] In some embodiments, the spot lighting area setting module 514 may set the emission intensity of the sub partial region (DB) higher than that of the main partial region (DA). In this way, since the emission intensity of the sub partial region DB is relatively high, even if the eye position 700 is near the boundary of the partial viewing zone E, the display light K with a relatively high brightness can be viewed by the light from the sub partial region DB. In addition, the uniformity of the brightness of the virtual image V that can be displayed in the entire virtual image display region VS can be improved.

[0116] The display parameter setting module 516 in FIG. 6 determines to which of a plurality of predetermined partial viewing areas E the eye position 700 detected by the eye position detection module 502, the eye position 700 estimated by the eye position estimation module 504, or the eye position 700 predicted by the eye position prediction module 506 belongs, and sets the display parameters 600 corresponding to the eye position 700. The display parameter setting module 516 includes various software components for executing various operations related to determining to which of a plurality of spatial areas (partial viewing areas E) the X-axis coordinate, the Y-axis coordinate, the Z-axis coordinate, or a combination thereof indicated by the eye position 700 belongs, and setting the display parameters 600 corresponding to the partial viewing area E to which the eye position 700 belongs. That is, the display parameter setting module 516 may include table data, arithmetic expressions, and the like for specifying the display parameters 600 from the X-axis coordinate, the Y-axis coordinate, the Z-axis coordinate, or a combination thereof indicated by the eye position 700.

[0117] Display parameter setting module 516 may apply a corresponding display parameter to each partial viewing area E. For example, when light emitting region DM emits light in a time-division manner, display parameter setting module 516 may switch to different display parameters 600 for each timing at which one light emitting region DM emits light. In the example of FIG. 4B, display parameters 600 corresponding to light emitting region DM(7,6) may be applied to light modulation element 51 at the timing at which light emitting region DM(7,6) emits light, and display parameters 600 corresponding to light emitting region DM(8,6) may be applied to light modulation element 51 at the timing at which light emitting region DM(8,6) emits light.

[0118] Further, another display parameter setting module 516 may apply one common display parameter to a plurality of light emitting regions DM. For example, in order to direct the display light K to the right eye spotlighting area SPR shown in FIG. 4B, the display parameter setting module 516 applies a common display parameter 600 to the light modulation element 51 at each timing of emitting light from the right eye light emitting region DMR (partial regions D(8,6), D(7,5), D(8,5), D(9,5), D(7,6), D(9,6), D(7,7), D(8,7), D(9,7)). Specifically, the display parameter setting module 516 may apply the display parameters 600 corresponding to the main partial area DA (D(8,6) in the example of Figure 4B) that directs the display light K toward the center of the spot lighting area SP to the light modulation element 51 at each timing when the light emitting areas DM (partial areas D(8,6), D(7,5), D(8,5), D(9,5), D(7,6), D(9,6), D(7,7), D(8,7), D(9,7)) are caused to emit light.

[0119] In some embodiments, the display parameter setting module 516 may switch the display parameters 600 at the timing when the right-eye light-emitting region DMR is illuminated and the timing when the left-eye light-emitting region DML is illuminated. If the display parameters 600 are parameters that cause different images to be displayed, the display parameter setting module 516 displays an image for the right eye at the timing when the right-eye light-emitting region DMR is illuminated and displays an image for the left eye at the timing when the left-eye light-emitting region DML is illuminated, thereby allowing the viewer to perceive a stereoscopic image (virtual image) due to left-right parallax.

[0120] Further, the other display parameter setting module 516 may apply two or more display parameters to one spot lighting area SP. In this case, the display parameter setting module 516 sets in advance a display parameter 600 common to a plurality of partial regions D. For example, in the example of FIG. 4B, if a common first display parameter is set to partial regions D(7,5), D(8,5), and D(9,5), a common second display parameter (different from the first display parameter) is set to partial regions D(7,6), D(8,6), and D(9,6), and a common third display parameter (different from the first and second display parameters) is set to partial regions D(7,7), D(8,7), and D(9,7), the display parameters are The data setting module 516 applies a first display parameter to the light modulation element 51 when the partial regions D(7,5), D(8,5), and D(9,5) emit light, respectively, applies a second display parameter to the light modulation element 51 when the partial regions D(7,6), D(8,6), and D(9,6) emit light, respectively, and applies a third display parameter to the light modulation element 51 when the partial regions D(7,7), D(8,7), and D(9,7) emit light, respectively.

[0121] The types of the display parameters 600 include (1) an arrangement of an image that has a predetermined positional relationship with a real object located outside the vehicle 1 as viewed from the eye position 700 (parameters for controlling the display device 50 to control the arrangement of the image, and / or parameters for controlling an actuator), (2) parameters for pre-distorting an image to reduce distortion of an image that may be caused by the virtual image optical system 90 as viewed from the eye position 700 (parameters for controlling the display device 50 to pre-distort an image displayed on the display device 50, also called warping parameters), and (3) parameters for visually recognizing a desired stereoscopic image as viewed from the eye position (parameters for controlling the display device 50, parameters for controlling the light source of the display device 50, parameters for controlling an actuator, or parameters including a combination of these). However, the display parameters 600 set (selected) by the display parameter setting module 516 may be parameters that are preferably changed according to the eye position 700 of the observer, and the types of the display parameters 600 are not limited to these.

[0122] The display parameter setting module 516 selects one or more display parameters 600 corresponding to the eye position 700 for each type of display parameter. When the eye position 700 includes, for example, a right eye position 700R and a left eye position 700L, the display parameter setting module 516 may select two display parameters: a display parameter 600 corresponding to the right eye position 700R and a display parameter 600 corresponding to the left eye position 700L. On the other hand, when the eye position 700 is, for example, a predetermined one of the right eye position 700R and the left eye position 700L, a detectable (easy to detect) one of the right eye position 700R and the left eye position 700L, or a position calculated from the right eye position 700R and the left eye position 700L (for example, a midpoint between the right eye position 700R and the left eye position 700L), the display parameter setting module 516 may select one display parameter 600 corresponding to the eye position 700. The display parameter setting module 516 may also select the display parameters 600 corresponding to the eye position 700 and the display parameters 600 set around the eye position 700. That is, the display parameter setting module 516 may select three or more display parameters 600 corresponding to the eye position 700.

[0123] The graphics module 518 in FIG. 6 includes various known software components for generating image data by performing image processing such as rendering, and driving the display device 50. The graphics module 518 may also include various known software components for changing the type, arrangement (position coordinates, angle), size, display distance (in the case of 3D), and visual effects (e.g., brightness, transparency, saturation, contrast, or other visual characteristics) of the image to be displayed. The graphics module 518 may generate image data to be viewed by an observer in the type of image (one of the display parameters), the position coordinates of the image (one of the display parameters), the angle of the image (such as a pitch angle with the X-direction as an axis, a yaw rate angle with the Y-direction as an axis, and a rolling angle with the Z-direction as an axis, which is one of the display parameters), and the size of the image (one of the display parameters), and drive the HUD device 20.

[0124] The light source driving module 520 includes various known software components for executing driving of the directional light source unit 60. The light source driving module 520 can drive the directional light source unit 60 based on the spot lighting area SP set by the spot lighting area setting module 514, the display parameters 600 set by the display parameter setting module 516, and the brightness set by the dimming value setting module 524 described later.

[0125] The actuator driving module 522 includes various known software components for executing the driving of the first actuator 28 and / or the second actuator 29. The actuator driving module 522 can drive the first actuator 28 and the second actuator 29 based on the set display parameters 600.

[0126] The operations of the above-mentioned processing processes can be implemented by executing one or more functional modules of an information processing device, such as a general-purpose processor or an application-specific chip. These modules, combinations of these modules, and / or combinations with known hardware that can replace their functions, are all included in the scope of protection of the present invention.

[0127] The functional blocks of the vehicle display system 10 are optionally implemented by hardware, software, or a combination of hardware and software to implement the principles of the various embodiments described. It will be understood by those skilled in the art that the functional blocks described in FIG. 6 may be optionally combined or one functional block may be separated into two or more sub-blocks to implement the principles of the embodiments described. Therefore, the description in this specification optionally supports any possible combination or division of the functional blocks described in this specification.

[0128] Second embodiment FIG. 10 is a diagram showing a configuration of a directional light source unit 60A in the second embodiment. FIG. 11 is a diagram showing one aspect of the light source array 80A shown in FIG. 10. The directional light source unit 60 is composed of a field lens 77A and a light source array 80A in which a plurality of light sources (81 in this embodiment as shown in FIG. 11) are arranged in an array. The light source array 80A (surface illumination unit SE) is arranged in m (m>n) partial regions D (the partial regions D in this embodiment are 121 from D(1,1) to D(11,11)). The display control device 30 partially drives (partially drives) the light source so that strong display light K is directed toward the spot lighting area SP corresponding to the partial viewing region E in which the eye position detected by the eye position detection unit 411 exists. As in the case of the directional light source unit 60A described above, the viewer is allowed to view an image (virtual image V), and light can be prevented from being directed toward the partial viewing region E in which the viewer's eyes do not exist.

[0129] The field lens 77A is disposed on the optical path of illumination light emitted from a light source disposed in each partial region D between the light source array 80A and the light modulation element 51. The second field lens 77 of this embodiment is a front-stage optical member (backlight optical member) composed of one or more field lenses, and designed so that the illumination laser beam 74 diffused by each partial region D of the light diffusion optical system 80 transmits and illuminates the entire surface of the light modulation element 51. The field lens 77A is also designed so that the display light K emitted from the light modulation element 51 reaches a partial viewing area E at an arbitrary position in the eye box 200 via the virtual image optical system 90 (a HUD optical member composed of the relay optical system 40 and the front windshield 2).

[0130] The diffusion plate 78 is disposed on the optical path of the illumination light between the light modulation element 51 and the field lens 77A. The diffusion plate 78 diffuses the received illumination light and emits it to the light modulation element 51. This makes it possible to reduce luminance unevenness in the display light K generated by the illumination light emitted from the multiple light-emitting regions DM viewed in the partial viewing area E of the eye box 200. The diffusion plate 78 may be any optical member that has a function of diffusing light, and the surface of the diffusion plate 78 may be, for example, a bead member, a fine uneven structure, or a rough surface. Alternatively, the diffusion plate 78 may be a dot sheet or a transparent milky white sheet.

[0131] In the light source array 80A of this embodiment, one light source such as an LED is arranged in each of m partial regions D (multiple light sources may be arranged), and the illumination light emitted by the light source arranged in any of the m partial regions D is irradiated onto the entire surface of the light modulation element 51 after passing through the field lens 77 and the diffuser 78, and the optically modulated display light K passes through the virtual image optical system 90 and is then irradiated onto a predetermined area in a predetermined eye box 200 corresponding to the predetermined partial region D.

[0132] The relationship between the vertical length PV and the horizontal length PH of the light emitting region DM will be described with reference to FIG. 11, FIG. 12A, and FIG. 12B. Although the light emitting patterns are different in FIG. 11, FIG. 12A, and FIG. 12B, if the bounding box including all of the partial regions D of a group that are caused to emit light with a predetermined light intensity or more is defined as the light emitting region DM, it can be said that the size of the light emitting region DM in FIG. 11, FIG. 12A, and FIG. 12B is the same. The horizontal direction of the light emitting region DM corresponds to the left-right direction (X-axis direction) of the eye box 200, and the vertical direction of the light emitting region DM corresponds to the up-down direction (Y-axis direction) of the eye box 200. In this embodiment, the horizontal length PH of the light emitting region DM is set to be twice or more the vertical length PV. As a result, the horizontal length of the spot lighting area SP corresponding to the light emitting region DM is set to be approximately twice or more the vertical length. This makes it possible to reduce the vertical size of the spot lighting area SP to reduce the power of the light source, while also increasing the horizontal size of the spot lighting area SP so that the image (virtual image) can be viewed at the desired brightness even if the eye position 700 moves horizontally.

[0133] That is, in some embodiments, the size of the spot lighting area SP may be set to be larger than the partial viewing area E. Furthermore, in some embodiments, the left-right length of the spot lighting area SP may be set to be more than twice that of the partial viewing area E. Specifically, the light emitting area DM is arranged, the intensity of light emitted by the light emitting area DM is adjusted (balanced), and / or the entire or partial optical adjustment of the focusing optical system is performed so that the partial viewing area E has a vertical length of 26 [mm] and a horizontal length of 10 [mm] and the spot lighting area SP has a vertical length of 52 [mm] and a horizontal length of 20 [mm].

[0134] 13A to 13D are diagrams showing modified examples of the relationship between the partial viewing area of ​​the eyebox and the position at which the display light based on the illumination light emitted from each partial area of ​​the directional light source unit is concentrated in some embodiments.

[0135] In the example of FIG. 13A, the directional light source unit 60 arranges the partial regions D so that the partial region D can be irradiated at five different locations within one partial viewing area E. Specifically, there is one location at the center of the partial viewing area E and four locations at the four corners of the partial viewing area E, for a total of five locations. In one embodiment shown in FIG. 13A, the display control device 30 controls the directional light source unit 60 so that the display light K is irradiated at all five different locations within the partial viewing area E where the eye position 700 is present. The light-emitting area DM includes a main partial region DA that emits illumination light that is the source of the display light K that is condensed at the center of the partial viewing area E where the eye position 700 is present, and a sub-partial region DB that emits illumination light that is the source of the display light K that is condensed at a position farther away from the center of the partial viewing area E where the eye position 700 is present than the main partial region DA. That is, when the left eye position 700L is in the partial viewing zone E(4,3), the main partial region DA is the partial region D(8,6), and the sub partial regions DB are the partial regions D(7,5), D(9,5), D(7,7), and D(9,7). On the other hand, when the right eye position 700R is in the partial viewing zone E(2,3), the main partial region DA is the partial region D(4,6), and the sub partial regions DB are the partial regions D(3,5), D(5,5), D(3,7), and D(5,7).

[0136] In the example of FIG. 13B, the directional light source unit 60 arranges the partial regions D so that the partial region D can be irradiated at three different locations within one partial viewing area E. Specifically, there is one location at the center of the partial viewing area E and two locations at the four corners of the partial viewing area E, for a total of three locations. In one embodiment shown in FIG. 13B, the display control device 30 controls the directional light source unit 60 so that the display light K is irradiated at all three different locations within the partial viewing area E where the eye position 700 is present. The light-emitting area DM includes a main partial region DA that emits illumination light that is the source of the display light K that is condensed at the center of the partial viewing area E where the eye position 700 is present, and a sub-partial region DB that emits illumination light that is the source of the display light K that is condensed at a position farther away from the center of the partial viewing area E where the eye position 700 is present than the main partial region DA. That is, when the left eye position 700L is in the partial viewing zone E(4,3), the main partial region DA is the partial region D(8,6), and the sub partial regions DB are the partial regions D(7,5) and D(9,7). On the other hand, when the right eye position 700R is in the partial viewing zone E(2,3), the main partial region DA is the partial region D(4,6), and the sub partial regions DB are the partial regions D(3,5) and D(5,7).

[0137] In the example of FIG. 13C, the directional light source unit 60 arranges the partial regions D so that the partial regions D can be irradiated at four different locations in one partial viewing area E. Specifically, the four locations are at the four corners of the partial viewing area E. In one embodiment shown in FIG. 13C, the display control device 30 controls the directional light source unit 60 so that the display light K is irradiated at all four different locations in the partial viewing area E where the eye position 700 is present. The light-emitting area DM does not include a main partial region DA that emits illumination light that is the source of the display light K that is condensed at the center of the partial viewing area E where the eye position 700 is present, but only includes a sub-partial region DB that emits illumination light that is the source of the display light K that is condensed at a position farther away from the center of the partial viewing area E where the eye position 700 is present than the main partial region DA. That is, when the left eye position 700L is present in the partial viewing area E(4,3), there is no main partial region DA, and the sub-partial regions DB are the partial regions D(7,5), D(9,5), D(7,7), and D(9,7). On the other hand, when the right eye position 700R is in the partial viewing area E(2,3), there is no main partial area DA, and the sub partial areas DB are the partial areas D(3,5), D(5,5), D(3,7), and D(5,7).

[0138] In the example of FIG. 13D, the directional light source unit 60 arranges the partial regions D so that the light can be irradiated to two different locations within one partial viewing area E. Specifically, the two locations are other than the four corners of the partial viewing area E. In one embodiment shown in FIG. 13D, the display control device 30 controls the directional light source unit 60 so that the display light K is irradiated to all two different locations within the partial viewing area E where the eye position 700 is present. The light-emitting area DM does not include the main partial region DA that emits illumination light that is the source of the display light K that is condensed at the center of the partial viewing area E where the eye position 700 is present, but only includes a sub-partial region DB that emits illumination light that is the source of the display light K that is condensed at a position that is farther away from the center of the partial viewing area E where the eye position 700 is present than the main partial region DA and is not a boundary of the partial viewing area E. That is, when the left eye position 700L is in the partial viewing zone E(4,3), there is no main partial region DA, and the sub partial regions DB are the partial regions D(7.5,6) and D(8.5,6) which are not on the boundaries of the partial viewing zone E. On the other hand, when the right eye position 700R is in the partial viewing zone E(2,3), there is no main partial region DA, and the sub partial regions DB are the partial regions D(3.5,6) and D(4.5,6).

[0139] 4B, 13A, and 13B, the light source element 60 (60A) in some embodiments may have a main partial area DA that emits illumination light for directing the display light K to the center of the partial viewing area E. In this case, the processor 33 may set the light emission pattern so that the light emitting area DM includes the main partial area DA that emits illumination light for directing the display light to the center of the partial viewing area E.

[0140] 4B, 13A, 13B, 13C, and 13D, the light source element 60 (60A) in some embodiments has two or more sub-partial regions DB that are point-symmetric with respect to positions at which illumination light can be emitted so that the display light is directed to the center of the partial viewing area E. In this case, the processor 33 sets the light emission pattern so that the light-emitting area DM includes two or more sub-partial regions DB that are point-symmetric with respect to positions at which illumination light can be emitted so that the display light is directed to the center of the partial viewing area E.

[0141] In addition, in some embodiments, the light source element 60 (60A) may be arranged at a position where it can emit illumination light so that the display light is directed to the boundary of the partial viewing area E, with the above-mentioned two or more sub-partial areas DB being point-symmetrical with respect to a position where it can emit illumination light so that the display light is directed to the center of the partial viewing area E, as shown in Figures 4B, 13A, 13B, and 13C.

[0142] In addition, in some embodiments, the light source element 60 (60A) may be arranged at a position where the above-mentioned two or more sub-partial regions DB, which are point-symmetric with respect to positions at which illumination light can be emitted so that display light is directed to the center of the partial viewing area E, can be emitted to two or all four of the four corners of the partial viewing area E that are point-symmetric, as shown in Figures 4B, 13A, 13B, and 13C.

[0143] In some embodiments, the light source element 60 (60A), as shown in Figures 13C and 13D, may not have a main partial area DA that emits illumination light so that the display light is directed to the center of the partial viewing area E, but may only have two or more sub-partial areas DB described above that are point-symmetric with respect to a position at which illumination light can be emitted so that the display light is directed to the center of the partial viewing area E.

[0144] The partial regions D in the light source element 60 (60A) do not need to be arranged regularly overall, and may be arranged differently, for example, between a position corresponding to the center (and its periphery) of the eyebox 200 and a position corresponding to the periphery of the eyebox 200. The partial regions D in the light source element 60 (60A) may be arranged differently between a position corresponding to the center (and its periphery) in the left-right direction of the eyebox 200 and a position corresponding to the periphery in the left-right direction of the eyebox 200. The partial regions D in the light source element 60 (60A) may be arranged differently between a position corresponding to the center (and its periphery) in the up-down direction of the eyebox 200 and a position corresponding to the periphery in the up-down direction of the eyebox 200.

[0145] In the above embodiment, the light source element 60 (60A) has m partial regions D, which is more than n partial viewing regions E, but is not limited to this. That is, the number (n) of partial viewing regions E and the number (m) of partial regions D may be equal.

[0146] In the above embodiment, the display control device 30 (processor 33) changes the size of the light-emitting region DM in the planar illumination unit SE based on the eye position (Z-axis coordinate) in the front-rear direction of the vehicle 1, but this is not limited to this. The display control device 30 (processor 33) may change the size of the light-emitting region DM in the planar illumination unit SE based on (1) the eye position (X-coordinate) in the left-right direction of the vehicle 1, (2) the eye position (Y-coordinate) in the up-down direction of the vehicle 1, or (3) any combination of these eye positions (X-coordinate, Y-coordinate), in addition to the eye position (Z-axis coordinate) in the front-rear direction of the vehicle 1.

[0147] 14 is a diagram showing an aspect of the first light-emitting region and the second light-emitting region in the planar illumination unit before and after the spot area reduction process. In some embodiments, when the processor 33 reduces the light-emitting region DM corresponding to the spot lighting area SP from the second light-emitting region DM2 (corresponding to the second spot lighting area SP2) to the first light-emitting region DM1 (corresponding to the first spot lighting area SP1) (spot area reduction process S170), the processor 33 may increase the light intensity of at least a part of the first light-emitting region DM1 (light intensity increase process). The light intensity increase process may include an aspect in which the light intensity of the main partial region DA (hollow circle in FIG. 14) of the first light-emitting region DM1 is increased, and the light intensity of the sub-partial region DB (hollow square in FIG. 14) of the first light-emitting region DM1 is not increased (maintained). Conversely, the light intensity increasing process may include an aspect in which the light intensity of the sub partial region DB (hollow square mark in FIG. 14) of the first light-emitting region DM1 is increased, and the light intensity of the main partial region DA (hollow circle mark in FIG. 14) of the first light-emitting region DM1 is not increased (maintained). According to this aspect, by executing the light intensity increasing process that increases the light intensity of at least a part of the first light-emitting region DM1 in conjunction with the light-emitting area reducing process that reduces the light-emitting region DM, it is possible to improve the light utilization efficiency while displaying the virtual image V at a desired dimming value.

[0148] In some embodiments, in the spot area reduction process S170, the light intensity increase process may increase the light intensity of the entire first light-emitting region DM1. According to this embodiment, the light intensity increase process increases the light intensity of the main partial region DA (indicated by the hollow circle in FIG. 14) of the first light-emitting region DM1, and also increases the light intensity of the sub-partial region DB (indicated by the hollow square in FIG. 14) of the first light-emitting region DM1.

[0149] Furthermore, the processor 33 in some embodiments may set the average light intensity in the first light-emitting region DM1 after the spot area reduction process S170 to be higher than the average light intensity in the second light-emitting region DM3 before the spot area reduction process S170. According to this embodiment, the average light intensity of the first light-emitting region DM1 (hollow circle and hollow square in FIG. 14) after the spot area reduction process S170 is made higher than the average light intensity of the second light-emitting region DM2 (hollow circle, hollow square, and hollow triangle in FIG. 14) before the spot area reduction process S170.

[0150] Furthermore, in the spot area reduction process S170 of some embodiments, the processor 33 may gradually reduce the light intensity of the second light-emitting region DM2 not included in the first light-emitting region DM1 in the spot area reduction process S170. According to this embodiment, the light intensity of the second light-emitting region DM2 (hollow triangular mark in FIG. 14) not included in the first light-emitting region DM1 is gradually reduced. This makes it possible to prevent a sudden decrease in luminance of the virtual image V caused by a sudden decrease in the light intensity of the first light-emitting region DM1 not included in the second light-emitting region DM2 from being visually recognized when the observer's eye position 700 is in or near the partial viewing zone E corresponding to the first light-emitting region DM1 (hollow triangular mark in FIG. 14) not included in the second light-emitting region DM2 depending on the observer's eye position 700. [Explanation of symbols]

[0151] 1: Vehicle 2: Front windshield (projected area) 3: Non-specular reflection element 10: Vehicle display system 20:HUD device 21: Light exit window 22: Housing 28: First actuator 29: Second actuator 30: Display control device 31: I / O interface 33: Processor 35: Image processing circuit 37: Memory 40: Relay optical system 41: First mirror 42: 2nd mirror 50: Display 51: Light modulation element 52:Display surface 60: Directional light source unit (light source element) 60A: Directional light source unit (light source element) 64: Average pupillary distance 70:Light source 74: Laser beam for lighting 75 : Scanning device 76: First field lens 77: Second field lens 77A: Field lens 78: Diffuser 80: Light diffusion optical system 80A: Light source array 81: Lens array 82: Holographic recording media 90: Virtual image optical system 200: Eye Box 200A: Boundary 205: Eye box center 401: Vehicle ECU 403: Road information database 405: Vehicle position detection unit 407: External sensor 409: Operation detection unit 411: Eye position detection unit (gaze direction detection unit) 415: Brightness sensor 417: Mobile information terminal 419: External communication equipment 502: Eye position detection module 504: Eye position estimation module 506: Eye position prediction module 508: Eye position state determination module 510: Startup determination module 512: Release condition determination module 514: Spot lighting area setting module 516: Display parameter setting module 518: Graphics module 520: Light source driving module 522: Actuator drive module 524: Dimming value setting module 600: Display parameters 700: Eye position 700L: Left eye 700R: Right eye D: Partial area DA: Main partial area DM: Light emitting area DM1: First light-emitting region DM2: Second light-emitting region DM3: Second light-emitting region DMC: Light emitting area DML: Light emitting area for left eye DML1: First left eye light emitting area DML2: Second light-emitting area for the left eye DMR: Right eye light emitting region DMR1: First right eye light emitting region DMR2: Second light-emitting region for the right eye E: partial viewing area K:Display light K1: 1st display light K2: 2nd display light K3: 3rd display light Kp: Optical axis Ld:Light amount distribution Ld1:Light amount distribution Lth: Predetermined threshold S160: Start-up light emission process S170: Spot area reduction process SP: Spot lighting area SP1: 1st spot lighting area SP2: Second spot lighting area SPC: Spot Lighting Area SPL: Left eye spot lighting area SPR: Spot lighting area for right eye V: Virtual image VS: Virtual image display area VS1: Upper end VS2: Bottom end

Claims

1. a light source element (60, 60A) having a plurality of partial regions (D) and capable of emitting illumination light for each of the partial regions (D); a light modulation element (51) that modulates the illumination light incident from the light source element (60, 60A) and emits the modulated light as display light; a focusing optical system (2, 40, 77, 77A) that focuses the display light obtained by modulating the illumination light emitted from the different partial regions (D) by the light modulation element (51) toward a different partial viewing region (E) in an eye box (200), and controls a head-up display device (20) in which a viewer views a virtual image within the range of the eye box by projecting the display light onto a projection target, a processor (33) that acquires information on an eye position of an observer (hereinafter referred to as eye position information) or information that enables the eye position to be estimated (hereinafter referred to as eye position estimation information) and controls the light source elements (60, 60A) based at least on the eye position information or the eye position estimation information, The processor (33) The partial region (D) (hereinafter referred to as the light-emitting region (DM)) that emits the illumination light having a predetermined light intensity or more can be set to at least a first light-emitting region (DM1) and a second light-emitting region (DM2) that is wider than the first light-emitting region (DM1); When the head-up display device (20) is started up, a start-up light emission process is executed in which the range of the partial viewing area (E) toward which the display light is directed is set to a second spot lighting area (SP2) by setting the light-emitting area (DM) to the second light-emitting area (DM2); Then, a reduction process is performed to reduce the light emitting region (DM) to the first light emitting region (DM1) so that the range of the partial viewing area (E) toward which the display light is directed is set to a first spot lighting area (SP1) narrower than the second spot lighting area (SP2). A display control device (30).

2. The processor (33) In the startup light emission process, acquiring the eye position information and / or the eye position estimation information; and setting the partial area (D) for directing the display light to a predetermined position indicated by the eye position information and / or the eye position estimation information in the first light-emitting area (DM1); After setting the first light-emitting area (DM1), the reduction process is executed to reduce the light-emitting area (DM) from the second light-emitting area (DM2) to the first light-emitting area (DM1); A display control device (30) according to claim 1.

3. The processor (33) in the startup light emission process The light emitting region (DM) is gradually narrowed in a stepwise manner from the second light emitting region (DM2) to the first light emitting region (DM1); A display control device (30) according to claim 1 or 2.

4. The processor (33) The average light intensity in the second light-emitting region (DM2) during the startup light-emitting process is set to be lower than the average light intensity in the first light-emitting region (DM1) during the startup light-emitting process. A display control device (30) according to claim 1.

5. The processor (33) Obtaining information about the right and left eye positions of an observer in the eye box (200) or information from which the right and left eye positions can be estimated; The first light-emitting region (DM1) includes a right-eye light-emitting region (DMR) set according to the partial viewing area (E) in which the right eye position is located, and a left-eye light-emitting region (DML) set according to the partial viewing area (E) in which the left eye position is located, The second light-emitting region (DM2) includes a binocular light-emitting region (DMC) that is larger than the right eye light-emitting region (DMR) and the left eye light-emitting region (DML), In the startup light emission process, the light emitting region (DM) is changed from the binocular light emitting region (DMC) to the right eye light emitting region (DMR) and the left eye light emitting region (DML). A display control device (30) according to claim 1.

6. a light source element (60, 60A) having a plurality of partial regions (D) and capable of emitting illumination light for each of the partial regions (D); a light modulation element (51) that modulates the illumination light incident from the light source element (60, 60A) and emits the modulated light as display light; a focusing optical system (2, 40, 77, 77A) that focuses the display light obtained by modulating the illumination light emitted from the different partial regions (D) by the light modulation element (51) toward a different partial viewing region (E) in an eye box (200), and projects the display light onto a projection surface, thereby allowing an observer to view a virtual image from within the range of the eye box, a processor (33) that acquires information on an eye position of an observer (hereinafter referred to as eye position information) or information that enables the eye position to be estimated (hereinafter referred to as eye position estimation information) and controls the light source elements (60, 60A) based at least on the eye position information or the eye position estimation information, The processor (33) The partial region (D) (hereinafter referred to as the light-emitting region (DM)) that emits the illumination light having a predetermined light intensity or more can be set to at least a first light-emitting region (DM1) and a second light-emitting region (DM2) that is wider than the first light-emitting region (DM1); When the head-up display device (20) is started up, a startup light emission process is executed in which the light-emitting region (DM) is set to the second light-emitting region (DM2) to set the range of the partial viewing region (E) toward which the display light is directed to a second spot lighting area (SP2), and after the light-emitting region (DM) is set to the second light-emitting region (DM2), a reduction process is executed in which the light-emitting region (DM) is reduced to the first light-emitting region (DM1) to change the range of the partial viewing region (E) toward which the display light is directed to a first spot lighting area (SP1) narrower than the second spot lighting area (SP2). A head-up display device (20).

7. The processor (33) In the startup light emission process, Obtaining the eye position information and / or the eye position estimation information; and setting the partial area (D) for directing the display light to a predetermined position indicated by the eye position information and / or the eye position estimation information in the first light-emitting area (DM1); After setting the first light-emitting area (DM1), the reduction process is executed to reduce the light-emitting area (DM) from the second light-emitting area (DM2) to the first light-emitting area (DM1); A head-up display device (20) according to claim 6.

8. The processor (33) in the startup light emission process The light emitting region (DM) is gradually narrowed in a stepwise manner from the second light emitting region (DM2) to the first light emitting region (DM1); A head-up display device (20) according to claim 6 or claim 7.

9. The processor (33) The average light intensity in the second light-emitting region (DM2) during the startup light-emitting process is set to be lower than the average light intensity in the first light-emitting region (DM1) during the startup light-emitting process. A head-up display device (20) according to claim 6.

10. The processor (33) Obtaining information about the right and left eye positions of an observer in the eye box (200) or information from which the right and left eye positions can be estimated; The first light-emitting region (DM1) includes a right-eye light-emitting region (DMR) set according to the partial viewing area (E) in which the right eye position is located, and a left-eye light-emitting region (DML) set according to the partial viewing area (E) in which the left eye position is located, The second light-emitting region (DM2) includes a binocular light-emitting region (DMC) that is larger than the right eye light-emitting region (DMR) and the left eye light-emitting region (DML), In the startup light emission process, the light emitting region (DM) is changed from the binocular light emitting region (DMC) to the right eye light emitting region (DMR) and the left eye light emitting region (DML). A head-up display device (20) according to claim 6.

11. a light source element (60, 60A) having a plurality of partial regions (D) and capable of emitting illumination light for each of the partial regions (D); a light modulation element (51) that modulates the illumination light incident from the light source element (60, 60A) and outputs the modulated light as display light; a focusing optical system (2, 40, 77, 77A) that focuses the display light, which is obtained by modulating the illumination light emitted from the different partial regions (D) by the light modulation element (51), toward a different region in an eye box (200), and projects the display light onto a projection surface, thereby allowing an observer to view a virtual image within the range of the eye box, Acquiring information regarding the viewer's eye position (hereinafter referred to as eye position information) or information that enables the eye position to be estimated (hereinafter referred to as eye position estimation information). Controlling the light source elements (60, 60A) based at least on the eye position information or the eye position estimation information; The partial region (D) (hereinafter referred to as the light-emitting region (DM)) that emits the illumination light having a predetermined light intensity or more is set at least in a first light-emitting region (DM1) and a second light-emitting region (DM2) that is wider than the first light-emitting region (DM1); and executing a startup light emission process for changing the light emitting area (DM) to the second light emitting area (DM2) and then to the first light emitting area (DM1) when the head-up display device (20) is started up. Display control method.

12. The startup light emission process includes: acquiring the eye position information and / or the eye position estimation information; Setting the partial area (D) that directs the display light to a predetermined position indicated by the eye position information and / or the eye position estimation information in the first light-emitting area (DM1); and changing the light emitting area (DM) from the second light emitting area (DM2) to the first light emitting area (DM1) after setting the first light emitting area (DM1). The display control method according to claim 11.

Images