Head-up display device

Abstract

To output an image with the relatively high luminance to a region that can be visually recognized by an occupant even in a scene in which relatively big change may occur at a viewpoint position of the occupant while suppressing the temperature rise in a light source.SOLUTION: A head-up display device comprises: a display unit which displays an image; and a control unit which controls the display unit. The control unit comprises: a viewpoint position detection part which detects a viewpoint position of an occupant; a light emission control part which emits light related to an image via the display unit with a partial region according to the viewpoint position as an illumination region in the entire region of an eye box on the basis of the viewpoint position detected by the viewpoint position detection part; a parameter acquisition part which acquires a value of a vehicle parameter; and a region size change part which changes the size of the illumination region on the basis of the value of the vehicle parameter acquired by the parameter acquisition part.SELECTED DRAWING: Figure 2A

Description

[Technical field]

[0001] The present disclosure relates to a head-up display (HUD) device. [Background technology]

[0002] In a head-up display device, lighting the entire eyebox all the time would be wasteful, so from the perspective of power saving, a viewpoint-tracking spotlighting control is known that illuminates a partial area of ​​the eyebox as an illumination area depending on the driver's viewpoint position. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6586646 specification Summary of the Invention [Problem to be solved by the invention]

[0004] However, a configuration in which the entire eye box is constantly illuminated is disadvantageous not only in terms of power saving but also in terms of the temperature rise of the light source due to heat generation. In this regard, the above-described viewpoint tracking spotlighting control is advantageous in that it can prevent the temperature rise of the light source.

[0005] However, since the change in the viewpoint position of the occupant can occur in various ways, there may be cases where it is difficult to realize viewpoint tracking spotlighting control with high accuracy. In particular, when the vehicle state changes, for example, when the vehicle switches from automatic driving to manual driving, or when the occupant operates the vehicle, such as the air conditioning panel or the transmission, the occupant's movement is accompanied, so the occupant's viewpoint position is likely to change significantly. In such cases, the detection accuracy of the viewpoint position and the tracking accuracy and responsiveness of the viewpoint tracking spotlighting control may be reduced, and there is a risk that an image with a relatively high brightness (virtual image) cannot be output to an area visible to the occupant (area within the eye box).

[0006] Therefore, an object of the present disclosure is to output an image with a relatively high brightness to an area visible to an occupant, while preventing the light source from becoming too hot, even in situations where a relatively large change in the occupant's viewpoint position may occur. [Means for solving the problem]

[0007] According to one aspect, there is provided a head-up display device that is mounted in a vehicle and allows an occupant of the vehicle to visually recognize a virtual image including vehicle information, A display unit for displaying the image; A control unit that controls the display unit, The control unit is A viewpoint position detection unit that detects the viewpoint position of an occupant; a light emission control unit that, based on the viewpoint position detected by the viewpoint position detection unit, sets a partial area of ​​an entire area of ​​an eyebox corresponding to the viewpoint position as an illumination area, and irradiates light related to the image via the display unit; A parameter acquisition unit that acquires values ​​of vehicle parameters; Based on the values ​​of the vehicle parameters acquired by the parameter acquisition unit, The size of the illumination area is gradually changed over time. A head-up display device is provided, comprising: a region resizing unit. Effect of the Invention

[0008] According to the present disclosure, it is possible to output an image with a relatively high brightness to an area visible to the occupant, while preventing the light source from becoming too hot, even in situations where a relatively large change in the occupant's viewpoint position may occur. [Brief description of the drawings]

[0009] [Figure 1] 1 is a diagram showing a head-up display according to an embodiment of the present invention mounted on a vehicle as viewed from the side of the vehicle. [Figure 2A]FIG. 1A is a diagram showing an overview of viewpoint-tracking spotlighting control and an example of the main configuration of a HUD device equipped with a function for expanding the lighting area when a specific operation event is detected / predicted, and FIG. 1B is a diagram showing the state when a specific operation event is detected. [Figure 2B] 13A and 13B are diagrams illustrating a state in which an illumination area is expanded in response to a specific operation event. [Diagram 3] 3A and 3B are diagrams showing an example of the configuration of an eyebox used in a HUD device. [Figure 4] FIG. 4A is a diagram showing a configuration example of a stereoscopic display (3D) HUD device to which this embodiment is applied, and FIG. 4B is a diagram for explaining the principle of displaying a stereoscopic image (virtual image). [Diagram 5] FIG. 5(A) is a diagram showing the transition (change) of the gaze point detection state over time, and FIGS. 5(B) to 5(F) are diagrams showing the expansion and contraction of the illumination area accompanying a specific operation event. [Figure 6] 6(A) to 6(H) are diagrams showing an example of the expansion and contraction of the illumination area. [Figure 7] 7(A) to (D) are diagrams showing an example of contraction and expansion of the illumination area. [Figure 8] 8A and 8B are diagrams showing another example of a change in the viewpoint position, and an expansion of the illumination area by estimating or predicting the movement of the viewpoint. [Figure 9] 10 is a flowchart showing an example of the operation of the HUD device for expanding and reducing the illumination area. [Figure 10] 10A and 10B are diagrams illustrating an example of expansion and contraction of an illumination area suitable for the operation example of FIG. 9. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Each embodiment will be described in detail below with reference to the accompanying drawings.

[0011] FIG. 1 is a diagram showing a schematic view of a HUD device 1 according to an embodiment mounted on a vehicle as viewed from the side of the vehicle.

[0012] In the HUD device 1, as shown in Fig. 1, when the display light is irradiated onto the windshield 2, the driver of the vehicle VC sees a display image (virtual image) V(C) obtained by the irradiation in front of the windshield 2. This allows the driver to visually recognize the virtual image V(C) superimposed on the scenery ahead. Therefore, the driver can grasp vehicle information, etc. with less viewpoint movement compared to when looking at a meter in the instrument panel 9, improving convenience and safety. In a modified example, a combiner or the like may be used instead of the windshield 2.

[0013] In this embodiment, the HUD device 1 is configured to perform viewpoint tracking spotlighting control and the like.

[0014] FIG. 2A(A) is a diagram showing an overview of the viewpoint-tracking spotlighting control and an example of a main configuration of a HUD device 1 having a function of expanding the lighting area when a specific operation event is detected / predicted, and FIG. 2A(B) is a diagram showing a state in which a change in the viewpoint position occurs due to a specific operation event.

[0015] In the HUD device 1 shown in FIG. 2A(A), the eye box EB is divided into four partial regions Z10 to Z13, and the position of the driver's (passenger's) eyes (viewpoint) can be detected using these partial regions as units.

[0016] 2A(A) (and (B)), the arrangement direction of partial regions Z10 to Z13 (left-right direction on the paper) is the X direction (vehicle width direction) in real space as seen by the driver driving the vehicle, the direction along the perpendicular line to the paper is the Y direction (vehicle height direction), and the Z direction is the direction along the vehicle's traveling direction (front-rear or depth direction). Note that the real space outside the vehicle as seen by the driver is the space below the windshield 2 shown in FIGS. 2A(A) and (B).

[0017] The HUD device 1 includes a control unit 100, a display unit 145, and a reflection optical system (light collecting optical system) 150.

[0018] The control unit 100 has a partial illumination control unit 110. The partial illumination control unit 110 has a viewpoint position detection unit 112, a parameter acquisition unit 113, an event processing unit 114, a light emission control unit 116, and an area size change unit 118. In the figure, S1a indicates viewpoint detection information. Also, S2 indicates specific operation event notification information (notification signal) indicating that a specific operation event has been detected or predicted.

[0019] The camera 90 captures an image of the face, eyes, etc. of a person (driver). Information S0 obtained by capturing the image is supplied to the viewpoint position detection unit 112.

[0020] Furthermore, the ECU (electronic control unit) 160 centrally manages the in-vehicle electronic devices. The ECU 160 collects vehicle information, information about the surroundings of the vehicle (surrounding information), and vehicle parameter information used to detect a specific operation event described later. Thus, the ECU 160 functions as a collector of various types of information. The various types of information collected by the ECU 160 are provided to the parameter acquisition unit 113 and the light emission control unit 116. In FIG. 2A(A), the various types of information provided are indicated by the symbol S3. The functions of the ECU 160 may be realized by a plurality of ECUs. Furthermore, some or all of the functions of the ECU 160 may be realized by the control unit 100.

[0021] The viewpoint position detection unit 112 detects the viewpoint position of the occupant based on the imaging information (imaging signal) S0 from the camera 90 that images the face or pupils of the occupant. The viewpoint position of the occupant may correspond to the position of the occupant's eyes. For example, the viewpoint position detection unit 112 may detect the viewpoint position by using pattern recognition, a corneal reflex method, or the like. The viewpoint position detection unit 112 may be provided separately from the HUD device 1.

[0022] The parameter acquisition unit 113 acquires vehicle parameter information used for detecting a specific operation event described later from the ECU 160, and provides the acquired vehicle parameter information (see S5) to the event processing unit 114. The vehicle parameter information may include values ​​of one or more vehicle parameters used for detecting a specific operation event. In this embodiment, the value of the vehicle parameter may be a value representing the presence or absence of an operation by an occupant on an operation unit (described later) related to the specific operation event, the amount of operation, etc. In this case, the value of the vehicle parameter may be a value directly representing the presence or absence of an operation by an occupant, the amount of operation, etc., or may be a value indirectly representing the presence or absence of an operation by an occupant. For example, in the case of an operation to the reverse gear (reverse position) of the transmission, the value of the vehicle parameter may be sensor information from a sensor that detects the position (shift position) of the shift knob, or may be information regarding the activation state of a rear monitoring camera (not shown) that is activated when the transmission is in the reverse gear.

[0023] The event processing unit 114 detects or predicts a specific operation event based on the vehicle parameter information. The event processing unit 114 may be provided separately from the HUD device 1. For example, a part or all of the functions of the event processing unit 114 may be realized by the ECU 160.

[0024] The specific operation event may include, among various operations performed by an occupant (in this embodiment, the "driver" as an example) on the vehicle, an operation event related to an operation that is likely to cause a relatively large change in the driver's viewpoint position.

[0025] For example, the specific operation event may include an operation event for an operation unit (such as an air conditioning panel or a transmission) arranged on the instrument panel 9 (see FIG. 1) or a console box of the vehicle. This is because, compared with an operation unit (not shown) arranged around the steering wheel, these operation units are arranged at a position farther away from the occupants (particularly the driver), and therefore the driver's viewpoint position during operation is likely to change relatively greatly. Alternatively, the specific operation event may include an operation event requiring a relatively complicated procedure. For example, an operation requiring a relatively complicated procedure, such as operating a lever while pressing a specific button, is likely to change the driver's viewpoint position relatively greatly. In addition, if the vehicle has an automatic driving mode (for example, a mode that realizes automatic driving of level 3 or higher according to the level classification based on the Society of Automotive Engineers), the specific operation event may include an operation event for switching from the automatic driving mode to the manual driving mode. This is because, when switching from the automatic driving mode to the manual driving mode, the driver's posture is likely to change significantly, and therefore the driver's viewpoint position is likely to change relatively greatly. The specific operation event may be different for each occupant. This is because the manner in which the viewpoint position changes may differ even for the same operation event due to the occupant's habits, body type, etc. The specific operation event for each occupant may be determined through machine learning or the like.

[0026] In addition, the specific operation event does not necessarily have to be an operation event related to the in-vehicle electronic device, and may include, for example, an operation event corresponding to a mobile terminal brought into the vehicle by the occupant. In this case, the specific operation event related to the mobile terminal by the occupant may be detected or predicted based on communication between the mobile terminal and the in-vehicle device realized via Bluetooth (registered trademark) or the like.

[0027] The method of predicting a specific operation event may be realized by utilizing machine learning, etc. For example, a driver (passenger) has individual characteristics in the operation manner (operation action, etc.) while driving, and if it is learned that a certain driver has a tendency to frequently change his / her posture while driving as a precursor to a specific operation event, which may cause the viewpoint to change up and down (in the height direction of the vehicle), the specific operation event can be estimated (predicted) by utilizing the learning result (estimation or prediction by learning).

[0028] The light emission control unit 116 generates and outputs a light emission control signal S4 based on the input signal S1a, thereby controlling the light emission state (lighting / non-lighting, luminance when emitting light, etc.) of the display pixels in the display unit 145.

[0029] In this embodiment, the light emission control unit 116 executes viewpoint tracking spot lighting control (hereinafter also simply referred to as "spot lighting control") based on the detection result of the viewpoint position from the viewpoint position detection unit 112. In a spot lighting control state, the light emission control unit 116 irradiates light related to an image (virtual image) via the display unit 145, using a partial area corresponding to the viewpoint position out of the entire area of ​​the eye box EB as an illumination area based on the detection result of the viewpoint position from the viewpoint position detection unit 112. Details of the spot lighting control will be described later.

[0030] The region size change unit 118 changes the size of the illumination region of the entire region of the eye box EB based on vehicle parameter information among various information S3 supplied from the ECU 160. The change in the illumination region may be realized by changing a partial region to be the illumination region. The region size change unit 118 gives an instruction (see S6) to the light emission control unit 116 so that the display light of the image reaches only the illumination region after the change. In response to this, the light emission control unit 116 generates and outputs a light emission control signal S4, thereby controlling the light emission state (on / off, luminance when emitting light, etc.) of the display pixels in the display unit 145 so that the display light of the image reaches only the illumination region after the change.

[0031] Specifically, the area size change unit 118 executes the expansion process of the illumination area when a specific operation event is detected by the event processing unit 114. The expansion process of the illumination area may be executed in a manner that replaces (interrupts) the spotlighting control by the light emission control unit 116. Therefore, in the expansion process of the illumination area, the illumination area is determined regardless of the detection result of the viewpoint position from the viewpoint position detection unit 112. The details of the expansion process of the illumination area will be described later.

[0032] When the region size change unit 118 expands the illumination region by the expansion process of the illumination region, the region size change unit 118 maintains the expanded state of the illumination region until a predetermined end condition is satisfied thereafter. The predetermined end condition is arbitrary, but may be satisfied, for example, when a predetermined time Tth0 has elapsed from the start point of the expansion process of the illumination region (for example, the point of time when the event processing unit 114 detects a specific operation event). When the predetermined end condition is satisfied, the region size change unit 118 executes a reduction process to change the illumination region to its original size (immediately before the start of the expansion process of the illumination region). The predetermined time Tth0 may be set to be equal to or greater than the time required for the viewpoint position of the occupant, which changes due to the specific operation event, to stop (stabilize), and may be adapted by testing or the like. In addition, the predetermined time Tth0 may differ for each attribute (type) of the specific operation event. When the reduction process by the region size change unit 118 is completed by returning the illumination region to its original size, the spotlighting control by the light emission control unit 116 may be resumed.

[0033] The display unit 145 has a light source (backlight) 120, an optical system (first optical system) 130 of the display unit, and a liquid crystal panel 140 that displays an image. Note that the liquid crystal panel is only an example, and other non-self-luminous display means using non-self-luminous elements may be used. Furthermore, self-luminous display means using LED elements, organic EL elements, or the like as pixels may be used.

[0034] The light source 120 has a plurality of light source elements (here, LED elements) 122a to 122d mounted (arranged) on the mounting surface of the substrate 121 in a planar shape (matrix shape).

[0035] Although not shown, the optical system (first optical system) 130 of the display unit is configured by combining a plurality of lenses. For example, a first field lens having a curved (e.g., arc-shaped) cross-sectional shape on the light-incident side and a flat cross-sectional shape on the light-exiting side, and a second field lens having a flat cross-sectional shape on the light-incident side and a curved (e.g., arc-shaped) cross-sectional shape on the light-exiting side are arranged in close proximity. The first and second field lenses appropriately refract the light emitted from the light source elements 122a to 122d (hereinafter, also referred to as "light source elements 122" when not distinguished), and the liquid crystal panel 140 is illuminated from the back by the light. By controlling the light transmittance of each pixel of the liquid crystal panel 140, for example, the twist angle of the liquid crystal element, it is possible to control the light emission / non-emission of each pixel, or the intensity of the light emitted from each pixel (in other words, the light emission intensity of each pixel).

[0036] The reflective optical system (light-collecting optical system) 150 functions as a second optical system. In Fig. 2A(A), the reflective optical system (light-collecting optical system) 150 is illustrated in a simplified manner, but specifically, for example, it may include a reflecting mirror, a concave mirror (curved mirror: reference numeral 171 in Fig. 4(B)), or the like.

[0037] The light emitted from the display unit 145 travels while being reflected by the reflecting mirrors of the reflecting optical system (light-collecting optical system) 150, and the light (display light: symbol K in Figure 3) is reflected by the vehicle windshield 2 (see Figures 1 and 3) and projected onto the eye box EB (viewpoints A1, A2).

[0038] In FIG. 2A(A), a viewpoint A1 (right eye) is located at a position of a partial region Z10 of the eyebox EB, and a viewpoint A2 (left eye) is located at a position of a partial region Z12 of the eyebox EB.

[0039] In FIG. 2A(A), the light source elements 122b and 122d are turned on and in a light-emitting state, and the light source elements 122a and 122c are turned off and in a non-light-emitting state.

[0040] A light ray L1 (indicated by a dashed line) generated by the light emitted by the light source element 122b is finally projected onto the left eye A2. A light ray R1 (indicated by a solid line) generated by the light emitted by the light source element 122d is finally projected onto the right eye A1. For example, when the pixels corresponding to the light source elements 122a and 122d on the display surface of the liquid crystal panel 140 form an image having parallax for the right eye / left eye, the parallax image is projected onto each of the driver's eyes A1 and A2. This allows the driver to view a virtual image.

[0041] Next, FIG. 2A(B) will be referred to. In FIG. 2A(B), the viewpoints A1 and A2 are located in the partial regions Z11 and Z13. Correspondingly, the light source elements 122a and 122c are turned on and in a light-emitting state, and the light source elements 122b and 122d are turned off and in a non-light-emitting state. A light ray L2 (indicated by a solid line) generated by the light source element 122a emitting light is ultimately projected onto the left eye A2. Also, a light ray R2 (indicated by a dashed line) generated by the light source element 122c emitting light is ultimately projected onto the right eye A1.

[0042] In this state, it is assumed that the viewpoints A1 and A2 move in the X direction (to the right on the paper) due to a specific operation event. Note that the viewpoint A1' at the time of detecting / predicting the specific operation event is located in a partial region Z12 of the eyebox EB, and the viewpoint A2' is located outside the eyebox EB.

[0043] In this state, the light rays R2 and L2 are not projected onto the viewpoints A1' and A2', and the driver cannot see the display (virtual image) by the HUD device 1. However, since the viewpoint A1' is within the eye box EB, it is possible to change the illumination range to irradiate light onto the viewpoint A1' and make it visible.

[0044] Next, reference is made to Fig. 2B, which is a diagram showing a state in which the illumination area is expanded in response to a specific operation event.

[0045] The position of the viewpoint (viewpoints A1', A2') in FIG. 2B is the same as that in FIG. 2A(B). Here, the event processing unit 114 detects a specific operation event and sends a specific operation event detection signal (detection information) S2 to the area size changing unit 118. In response to this, the area size changing unit 118 performs an expansion process of the illumination area as a measure against a relatively large change in the viewpoint position that may occur due to the specific operation event. Note that the expansion of the illumination area can be realized, for example, by expanding the area of ​​the pixels or light source 120 that contribute to the display of the virtual image on the display unit 145 (in other words, by controlling to increase the number of pixels or light source elements 122 in the light-emitting state) and controlling to illuminate an illumination area wider than the illumination area of ​​the eye box EB that was illuminated immediately before the specific operation event as a reference.

[0046] In Fig. 2B, light source elements 122b and 122d that were turned off in Fig. 2A(B) are turned on, and all light source elements 122a to 122d are turned on (emitting light). In Fig. 2A(B), illumination was provided only by light rays L2 and R2, but in Fig. 2B, illumination is also provided by light rays L1 and R1, and the illumination range (illumination area) in the eye box EB is expanded (enlarged).

[0047] Focusing on the eye box EB, the light ray L1 is projected onto the viewpoint A1' (located in the partial region Z12 of the eye box EB) after the change accompanying the specific operation event. This prevents a situation in which the driver is completely unable to view the display (virtual image) by the HUD device 1 due to the specific operation event. In other words, it is possible to suppress (reduce or prevent) the possibility that the display suddenly disappears completely due to the specific operation event.

[0048] Next, reference will be made to Fig. 3. Fig. 3(A) and (B) are diagrams showing an example of the configuration of an eyebox used in a HUD device.

[0049] In FIG. 3(A), the eyebox EB is divided into a plurality of (here, nine as an example) partial regions Z1 to Z9, and the position of the driver's viewpoint A is detected for each of the partial regions Z1 to Z9.

[0050] Display light K of an image is emitted from the light collecting optical system 150 (see Figs. 2A and 2B) of the HUD device 1, a part of which is reflected by the windshield 2 and enters the driver's viewpoint (eye) A. When viewpoint A is within the eye box EB, the driver can visually recognize a virtual image of the image.

[0051] 3B, display light K of an image is emitted from the light collecting optical system 150 of the HUD device 1, a part of which is reflected by the windshield 2 and enters the left and right viewpoints (left and right eyes) A1, A2 of the driver. When the viewpoints A1, A2 are within the eye box EB, the driver can visually recognize a virtual image of the image.

[0052] Here, the width direction (left-right direction) of the vehicle as viewed from the driver's viewpoints (eyes) A1 and A2 is defined as the X direction (or horizontal direction), the height direction perpendicular to the X direction and perpendicular to the road surface is defined as the Y direction (vertical direction), and the forward direction (front-rear direction) is defined as the Z direction. In FIG. 3B, the inside of the eye box EB is provided with 12 partial regions W1 to W12 partitioned by vertical boundaries, and each partial region W1 to W12, which has a vertically long rectangular shape, is adjacently arranged (arranged) in the horizontal direction (X direction). By detecting the viewpoint position using each partial region W1 to W12 as a unit (unit of minimum resolution), the horizontal positions of the viewpoints A1 and A2 in the eye box EB can be detected. Based on such detection results, spot lighting control can be suitably performed.

[0053] Next, reference is made to Fig. 4. Fig. 4(A) is a diagram showing a configuration example of a stereoscopic display type (3D) HUD device to which this embodiment is applied, and Fig. 4(B) is a diagram for explaining the principle of displaying a stereoscopic image (virtual image). In Fig. 4, the same reference numerals are used to denote the same parts as in the previous drawings.

[0054] 4(A), an image (parallax image) ZA for the right eye and an image (parallax image) ZB for the left eye are displayed on the liquid crystal panel 140. Note that a "parallax image" is an image that reproduces the parallax (the difference between the images perceived by each eye) that occurs when the left and right eyes are in different positions.

[0055] When the light emission side of the liquid crystal panel 140 is defined as the front, an optical member 160 is disposed in front of the liquid crystal panel 140. This optical member 160 functions as a light beam separating member, and as shown in the upper left, specifically, can be configured by a lenticular lens 170 or a parallax barrier (parallax barrier) 180 having slits SL. The parallax barrier 180 has a plurality of slits 185a to 185n. However, these are merely examples and the present invention is not limited to these.

[0056] If the light (reproduced light for each eye that reproduces an image) L10 and R10 separated by the optical member 160 having a light beam separation function enters both eyes A1 and A2 located at the image forming point of the light, a person sees an apparent three-dimensional image IM at the point where convergence (intersection of light) occurs. In other words, this can be considered as the three-dimensional image IM being generated by the 3D display. In the example of FIG. 4(A), the convergence angle is θc.

[0057] As described with reference to FIG. 2A etc., behind the liquid crystal panel 140, there are arranged the control unit 100, the light source (light source for backlight such as LED) 120, and the light source optical system (first optical system) 130.

[0058] As shown in Fig. 4(A), in the spot lighting control state, the entire surface of the liquid crystal panel 140 is not illuminated, but the necessary areas ZA and ZB according to the viewpoint position of the occupant are selectively illuminated. Note that reference numerals 147 and 147' indicate pixels corresponding to (the centers of) the viewpoints A1 and A2. This selective illumination realizes partial illumination (spot lighting control) on the eye box EB.

[0059] Please refer to FIG. 4(B). In FIG. 4(B), an eye box EB is set in front of the viewer (passenger), and eye point P(C) is located at the center of the eye box EB. If virtual image formation planes PS(L) and PS(R) corresponding to the left and right eyes are set in front of the windshield 2, a virtual image V(C) is located at the center of the overlapping area. The convergence angle of the virtual image V(C) is θd, and the virtual image V(C) is recognized as a three-dimensional image by the viewer (passenger).

[0060] This stereoscopic virtual image V(C) is displayed (formed) as follows: Reproduction light L10, R10 for each of the left and right eyes of the virtual stereoscopic image IM generated by the 3D display shown in Fig. 3(A) is reflected (reflected at least once) by a curved mirror (concave mirror or the like) 171 included in the light collecting optical system 150 of the HUD device 1, and is thereby projected (projected) onto the windshield 2 as display light K, and the reflected light reaches both eyes of the viewer and forms an image in front of the windshield 2, thereby displaying (forming) the virtual image V(C).

[0061] Next, reference will be made to Fig. 5. Fig. 5(A) is a diagram showing a transition (change) including the stop of the spotlighting control state caused by a specific operation event, and Fig. 5(B) to (F) are diagrams showing the expansion and contraction of the lighting area corresponding to the specific operation event.

[0062] The control to expand or reduce the lighting area when detecting / predicting a specific operation event, as described below, can be applied to both single-eye HUDs that project the same image to each of the driver's eyes, and binocular (parallax, hologram, or light reproduction) HUDs that project different images (parallax images) to each eye.

[0063] In Fig. 5(A), the period from time t0 to time t2 is a "spot lighting control state" in which a specific operation event is neither detected nor predicted, and at time t2, a specific operation event is detected and predicted. The spot lighting control stop state continues from time t2 to time t5. After time t5, the spot lighting control state is entered.

[0064] 5(B), (C), (D), (E), and (F) show the illumination areas in the eye box EB at the time points t1, t3, t4, t6, and t7, respectively. The illumination areas are shown as hatched rectangular (including square and rectangular) areas Q1 to Q5. Moreover, the viewpoints A1 and A2 in the spotlighting control state are shown by solid lines, and the viewpoints A1 and A2 in the specific operation event state are shown by dashed lines.

[0065] In FIG. 5(B) (corresponding to time t1), viewpoints A1 and A2 are located near the upper right diagonal on the eyebox EB (here, the eyebox is defined as a rectangular area in a planar view seen from the front).

[0066] By performing spotlighting control, the display light of the image is irradiated (projected) onto a rectangular illumination area Q1 of a predetermined size centered on the viewpoints A1 and A2 whose positions have been detected. If the viewpoint moves, the viewpoint position is detected as needed, and the illumination area Q1 moves to follow the viewpoint while maintaining its size. This state is the spotlighting control state.

[0067] Note that the above-mentioned "predetermined size" here refers to the illumination size when the viewpoint position has been detected and the illumination area is not in contact with the eye box EB (in other words, in the normal state of spot lighting).

[0068] In FIG. 5(C) (corresponding to time t3), a specific operation event has already been detected and predicted. The direction of movement of the viewpoint that may occur due to a specific operation event may differ for each specific operation event. In this case, the viewpoint moves diagonally downward to the right on the eye box EB. In FIG. 5(C), the size of the illumination area Q2 is expanded (enlarged) compared to the size of the illumination area Q1 in FIG. 5(A) (in other words, the size of the illumination area immediately before the specific operation event). It is expanded almost uniformly (isotropic expansion) in the directions perpendicular to each of the four sides that determine the outline of the rectangular illumination area.

[0069] In Fig. 5(C), the viewpoint A1 is outside the illumination area Q2, but the viewpoint A2 is located inside Q2, and an image can be projected onto the viewpoint A2. This prevents the driver from being unable to see the display (virtual image) at all when a specific operation event is detected / predicted.

[0070] Here, the control for expanding the illumination area is performed by the control unit 100 described above (specifically, the partial illumination control unit 110: see FIG. 2A(A)).

[0071] When the display unit 145 for displaying an image forms pixels using self-luminous elements such as LEDs or organic EL, the illumination area can be expanded by controlling the light emission state of the pixels, and when the display unit 145 is a light-transmitting type combining a liquid crystal panel 140 and a backlight (light source 120) as in the examples of Figures 2A and 2B, the illumination area can be expanded by controlling the light emission state (on, off, etc.) of the light source 120 (light source element 122). In other words, for example, by changing the pixels (non-emitting pixels) around the pixel that was emitting light just before the specific operation event to an emitting state (a state that can contribute to the visibility of a virtual image), it is possible to expand the illumination area around the illumination area just before the specific operation event.

[0072] Here, it is preferable to "expand based on the illumination area immediately before the specific operation event". If the time for which the viewpoint position changes due to the specific operation event is short, the viewpoint is likely to remain near the original illumination area, and expanding to the periphery of the original area increases the possibility of projecting light onto the viewpoint. In the example of FIG. 5(C), the illumination area is expanded around the illumination area Q1 in FIG. 5(B) at time t1 immediately before the occurrence of the specific operation event, thereby realizing a larger illumination area Q2, which makes it possible to project light onto the viewpoint A2 as described above.

[0073] Also, the illumination area Q2 can be expanded by projecting light onto the entire area that protrudes (expands) from the illumination area Q1 all at once. However, when the illumination area in the eye box EB is expanded, a part of the light beam heading toward the expanded illumination area may enter the original illumination area. As described above, the entire eye box EB may be illuminated all at once, but in this case, it is assumed that the brightness of the image (virtual image) viewed in the original illumination area will increase suddenly, and the driver (passenger) may feel uncomfortable due to the change in brightness.

[0074] Therefore, when expanding the illumination area, the control unit 100 may perform at least one of continuous expansion control and stepwise expansion control, for example, gradually expanding the illumination area over a first predetermined time Tth1. In this case, it is possible to suppress the sense of incongruity that may occur when the illumination area is expanded to the entire eye box EB at once. In other words, it is possible to perform "spotlighting expansion control" caused by a specific operation event in a manner that causes less sense of incongruity.

[0075] If the illumination area is expanded continuously on the time axis, the sense of incongruity can be further reduced. In addition, if the illumination area is expanded stepwise (in steps), efficient expansion can be performed easily (with less strain on the device) with regularity according to a specified expansion algorithm. Note that by combining continuous expansion and stepwise expansion, it is possible to expand the illumination area with a higher degree of freedom.

[0076] In this way, even if the viewpoint changes temporarily and significantly due to a specific operation event that is likely to cause a movement of the viewpoint (at least one of the left and right eyes), if the viewpoint is located in the expanded illumination area, the driver (passenger) can visually recognize the display (virtual image), and the possibility of preventing the display from suddenly disappearing is increased. In addition, since the possibility of visually recognizing the display necessary for the operation of the vehicle is increased, the reliability in terms of safety is also increased.

[0077] Also, when transitioning from FIG. 5(B) to FIG. 5(C), for example, the light source element 122 may be made to emit light up to a predetermined light intensity in one go by binary control of the light source 120 being turned off / on. However, it is expected that a large change in luminance may cause discomfort. Therefore, in the examples of FIG. 5(B) and (C), the amount of light is gradually increased. A hollow arrow with an expanded tip is shown between FIG. 5(B) and (C), which indicates that the amount of light is gradually increased (this also applies to the arrow drawn between FIG. 5(C) and (D)).

[0078] Specifically, the pixels (display pixels) of the display unit 145 are not controlled to be switched on and off at a desired luminance (binary control of on / off), but are controlled to be raised to a desired luminance over time (in other words, multi-value lighting control). This can further reduce the sense of incongruity.

[0079] Next, Fig. 5(D) will be described. In Fig. 5(D), at time t4, the illumination area is expanded to the entire area on the eye box EB (in the figure, the entire illumination area is denoted by the symbol Q3).

[0080] In addition, in FIG. 5(D), due to a specific operation event, the viewpoints A1 and A2 move further diagonally downward to the right from FIG. 5(C). However, by expanding the lighting area to the entire range, images are projected onto both viewpoints A1 and A2, and the driver can see the image (virtual image).

[0081] Here, referring to Fig. 5(A), time t4 is a time after the first predetermined time Tth1 has elapsed from the detection / prediction time t2 of the specific operation event. In other words, in Fig. 5(D), the illumination area of ​​the eye box EB is gradually expanded to its entire area over the first predetermined time Tth1 from the detection / prediction time of the specific operation event.

[0082] Here, as described above, at the stage of Figure 5 (C), the illumination range may be expanded (enlarged) to the entire area of ​​the eye box EB in one go. In this case, however, it is expected that the brightness of the image (virtual image) viewed in the original illumination area will increase suddenly, and the driver (passenger) may feel uncomfortable due to the change in brightness.

[0083] Therefore, after Fig. 5(B), the state is changed to Fig. 5(C), and then to Fig. 5(D) (a state in which the illumination range is expanded to the entire eye box). In other words, the control to expand the illumination range to the entire area is carried out over a first predetermined time Tth1. The size of the illumination area can be gradually expanded (for example, Fig. 5(C)). This makes it possible to suppress (reduce) the sense of discomfort.

[0084] An example of the expansion control of the illumination area has been described above, but specific examples and modified examples of the expansion will be described later.

[0085] Next, reference is made to Fig. 5(E) and (F). Fig. 5(E) shows a state at time t6. Since a predetermined time Tth0 has passed since the detection / prediction time t2 of the specific operation event, the illumination area is reduced to the original size illumination area Q4. At this time, the position (e.g., the center position) of the illumination area Q4 may be determined according to the detection result of the viewpoint position detection unit 112. The size of the illumination area Q4 is larger than the illumination area Q1 in Fig. 5(B) and has not completely returned to the original size. In the next Fig. 5(F), it has been reduced to the original size. Here, Fig. 5(F) shows a state at time t7.

[0086] When a predetermined time Tth0 has elapsed from the detection / prediction time t2 of the specific operation event, the spotlighting control is switched to one based on the viewpoint position detected at that time (in principle, partial lighting of the same size as the lighting area before the specific operation event). This makes it possible to return to the original spotlighting control state without discomfort after the spotlighting expansion control associated with the specific operation event. In a modified example, the size may be reduced to the same size as the original partial lighting size at the stage of FIG. 5(E).

[0087] It should be noted that the viewpoint of the driver (passenger) may converge to a certain position while repeating a certain degree of movement.

[0088] 5(E) and (F), the spot lighting control state is restored at time t7, which is the time after the second predetermined time Tth2 has elapsed from time t5, which is the time when the predetermined time Tth0 has elapsed from time t2 when the specific operation event was detected / predicted (the time when the time required for the viewpoint to stabilize has elapsed). As a result, during the period before the second predetermined time Tth2 has elapsed, the illuminated area can be gradually reduced (for example, FIG. 5(E)), and the gradual reduction in the illuminated area suppresses (reduces) the discomfort felt by the driver.

[0089] In addition, when transitioning from Fig. 5(D) to (E) and from Fig. 5(E) to (F), the amount of illumination light (illumination brightness) is gradually reduced, thereby suppressing (reducing) the discomfort that may occur when transitioning to normal spot lighting all at once. Note that tapered white arrows are shown between Fig. 5(D) and (E) and between Fig. 5(E) and (F), which indicate that the amount of light is gradually reduced. Other examples of reducing the illumination area will be described later.

[0090] Next, FIG. 6 will be referred to. FIG. 6(A)-(H) are diagrams showing an example of the expansion and contraction of the illumination area. As shown in the figure, the eye box EB has a predetermined shape (here, a square or a rectangle). However, the shape is not limited to this. More precisely, the "eye box" is a predetermined area (the shape of the predetermined area includes, for example, a cube, a rectangular parallelepiped, and an ellipsoid) that is defined in the left-right direction (X-axis direction), the up-down direction (Y-axis direction), and the depth direction (Z-axis direction) inside the vehicle, and is set to be the same as the area (also called the iris) where the position of the observer's viewpoint is expected to be located in the vehicle in which the HUD device 1 is mounted, or to include most of the iris (for example, 80% or more). In the drawings used to explain this embodiment, the concept of the depth direction is omitted, and the eye box is shown as a rectangle defined in the left-right direction and the up-down direction. The shape of the eye box defined in the left-right direction and the up-down direction can be set to a square or a rectangle (including a rectangle and a square), as well as a polygon, an ellipse, or the like. In FIG. 6 (as well as FIG. 7), the illumination areas are shown as hatched rectangular (including square and rectangular) areas Q6, Q7, etc., and areas Q1 to Q5 are as shown in FIG.

[0091] First, the expansion pattern in FIG. 6 will be described. When expanding the illumination area, for example, continuously or stepwise, the control unit 100 can perform any of isotropic expansion, which expands the illumination area uniformly around its periphery, anisotropic expansion, which expands the illumination area non-uniformly, and a combination of these. Taking an illumination area having a square or rectangular outer shape as an example, any of "isotropic expansion" which expands the illumination area along the perpendicular lines to each of the four sides, "anisotropic expansion" which expands the illumination area along the perpendicular lines to each of one side, two sides, or three sides, and "expansion by a combination of isotropic expansion and anisotropic expansion" can be performed. In addition, when any side of the outer shape of the illumination area reaches the corresponding side of the outer shape of the eye box EB, expansion is performed for the other unreached sides, and when the illumination area has expanded to the entire area of ​​the eye box EB, the expansion can be completed.

[0092] FIG. 6 shows three expansion patterns (specific examples that differ from FIG. 5 in at least some parts). One expansion pattern is an example of expansion to FIG. 6(A), (B), (E), and (H). This example is a pattern in which FIG. 6(E) is inserted into the previously shown FIG. 5(B), (C), and (D) (which correspond to FIG. 6(A), (B), and (H)).

[0093] The second expansion pattern is a pattern in which the data is expanded as shown in Fig. 6(A), (C), (F), and (H). The third expansion pattern is a pattern in which the data is expanded as shown in Fig. 6(A), (D), (G), and (H). Efficient expansion is achieved by using isotropic and anisotropic expansion.

[0094] In the example of FIG. 6, the rules for expansion are clarified, and effects such as easier control by software are obtained.

[0095] Next, a reduction pattern will be described. When reducing the illumination area, for example, the control unit 100 performs control to reduce a wide illumination area expanded in association with a specific operation event to a narrow illumination area of ​​the same size as the illumination area before the specific operation event, corresponding to the subsequent viewpoint position.

[0096] At this time, any of isotropic shrinkage in which the illumination area shrinks uniformly, anisotropic shrinkage in which the illumination area shrinks non-uniformly, and a combination of these can be performed. Specifically, for example, a first shrinkage pattern (first method) can be envisioned in which a control is performed to restore a narrow illumination area of ​​the same size as the illumination area before the specific operation event by combining "isotropic shrinkage" in which the illumination area shrinks along the perpendicular lines to each of the four sides of a square or rectangular illumination area and "anisotropic shrinkage" in which the illumination area shrinks along the perpendicular lines to each of one side, two sides, or three sides.

[0097] In addition, when the contraction along one of two orthogonal line segments is defined as vertical contraction and the contraction along the other is defined as horizontal contraction, the original lighting size can be restored by appropriately implementing these (including the case of implementing them in combination). In addition, as the "two orthogonal line segments", if the plane viewed ignoring the depth of the eye box EB can be grasped as a rectangle (including a square or a rectangle) erected perpendicular to the road surface, a line segment in the height direction of the vehicle and a line segment in the width direction of the vehicle can be adopted. Specifically, for example, when the contraction along the vertical side of a square or a rectangle is defined as vertical contraction and the contraction along the horizontal side is defined as horizontal contraction, two reduction patterns (second method) can be assumed in which vertical contraction is implemented and then horizontal contraction is implemented to implement control to restore the lighting area to a narrower size than the lighting area before the specific operation event.

[0098] Also, a third shrinkage pattern (third method) can be envisaged in which horizontal shrinkage is performed followed by vertical shrinkage to control the return to a narrower illumination area that is the same size as the illumination area before the specific operation event.

[0099] According to these methods, the rules for reduction are clarified, and the effects of facilitating control by software can be obtained.

[0100] In Fig. 6, three reduction patterns are obtained by tracing the three expansion patterns described above in reverse. Note that the reduction patterns shown in Fig. 6 are for the second and third methods described above (the reduction pattern for the first method will be described with reference to Fig. 7).

[0101] In the example of Figure 6, starting from Figure 6(H), any one of the following can be implemented: a first reduction pattern that reduces to Figures 6(E), (B), and (A); a second reduction pattern that reduces to Figures 6(F), (C), and (A); or a third reduction pattern that reduces to Figures 6(G), (D), and (A).

[0102] Next, reference will be made to Fig. 7. Fig. 7(A) to (D) are diagrams showing an example of contraction and expansion of the illumination area.

[0103] In Fig. 7, two contraction (or expansion) patterns are shown. Fig. 7(A) and (B) The reduction patterns in Fig. 7(A), (C), and (D) show cases where isotropic shrinkage is performed after anisotropic shrinkage. The reduction patterns in Fig. 7(A), (C), and (D) show cases where anisotropic shrinkage is performed after isotropic shrinkage. These correspond to the first reduction pattern described above. Moreover, by tracing each reduction pattern in reverse, two types of expansion patterns can be obtained. Note that these patterns are merely examples and are not limiting.

[0104] Next, Fig. 8 will be referred to. Fig. 8(A) and (B) are diagrams showing another example of change in the viewpoint position (an example considering a return to the spot lighting control state from the detection / prediction of a specific operation event) and an expansion of the lighting area by estimating or predicting the movement of the viewpoint, etc. In Fig. 8, the eye box shown in Fig. 3(B) is used.

[0105] In Fig. 8(A), the viewpoints A1 and A2 are initially located in partial regions W7 and W8 of the eyebox EB (spot lighting control state). After that, the viewpoints move significantly to the right (positive X direction), and one of the viewpoints (viewpoint A2) moves out of the eyebox EB (viewpoint movement (1)). Next, the viewpoints A1 and A2 move to the left (negative X direction), and the viewpoint A2, which had been out of the eyebox, returns to the eyebox EB, but the return is somewhat small and does not return to its original position.

[0106] Additionally, the viewpoints A1 and A2 move slightly to the lower right (viewpoint movement (3)), and then return to the left of their original positions (viewpoint movement (4)).

[0107] A specific operation event may involve such various viewpoint movements. Even in this case, if control to expand the illumination area is performed when a specific operation event is detected / predicted, the driver is more likely to be able to see the display (virtual image) by the HUD device 1, which is also beneficial.

[0108] In addition, when expanding the lighting range in response to a specific operation event, the direction of movement of the viewpoint, etc. can be estimated or predicted, and control can be performed to expand the lighting range more in the estimated (predicted) direction (including expanding only in that direction).

[0109] In other words, when expanding the illumination area continuously or stepwise, the control unit 100 may estimate or predict the change direction of the viewpoint position caused by the specific operation event based on at least one of the movement direction of the viewpoint for each attribute of the specific operation event, the past learning result regarding the movement of the viewpoint, and the driving situation at the time of detection / prediction of the specific operation event, and may perform control to expand the illumination area so as to include at least the estimated or predicted change direction. That is, the change direction of the viewpoint position caused by the specific operation event (movement direction of the viewpoint at the time of detection / prediction of the specific operation event) may be estimated or predicted, and the illumination area may be expanded by utilizing the content of the estimation (prediction).

[0110] Furthermore, if there is a certain trend in the direction of change in the viewpoint position caused by a specific operation event for each attribute of the specific operation event and / or for each occupant, the learning results of that trend can be used to estimate or predict the direction of change in the viewpoint position caused by a specific operation event (the direction of movement of the viewpoint when a specific operation event is detected / predicted).

[0111] By performing such estimation (prediction) and gradually expanding to include the direction of change in the viewpoint position caused by the estimated (predicted) specific operation event, efficient expansion is possible. For example, by expanding the estimated (predicted) direction more thoroughly (increasing the degree of expansion) than the non-estimated (non-predicted) direction, the possibility of projecting an image onto the viewpoint that is changing due to the specific operation event increases, and the possibility that the expansion of the illumination area will efficiently lead to an increase in the visibility of the virtual image can be increased. In addition, since the necessary area can be illuminated with focus, it is also beneficial in terms of power saving (suppressing the temperature rise of the light source 120).

[0112] In the example of FIG. 8(A), the range F1 was initially illuminated, but after making the above estimation (prediction), The illumination area is expanded greatly to the right (positive X direction), which is the estimated (predicted) direction (expanded to range F2), while the illumination area is expanded slightly to the left (negative X direction) (expanded to range F3).

[0113] In FIG. 8(A), by enlarging (extending) the illumination area in this manner, it is possible for the driver to view (or have an increased opportunity to view) the display (virtual image) in any of the above-mentioned viewpoint movements (1), (2) or (3), (4).

[0114] Next, let us refer to Fig. 8(B). In Fig. 8(B), the viewpoints A1 and A2 move significantly to the right (positive X direction) and go outside the eye box EB (viewpoint movement (5)). After that, the viewpoints continue to move outside the eye box EB (viewpoint movement (6)). Then, they move significantly to the left (negative X direction), passing far past their original positions and moving to a position significantly to the left of their original positions (viewpoint movement (7)).

[0115] In the example of FIG. 8(B), a specific operation event similar to that in FIG. 8(A) is detected, and the illumination range is expanded from F1 to an area including F2 and F3, and further expanded to an area including F4 (the entire area of ​​the eye box EB).

[0116] In Fig. 8(B), by implementing such an expansion (enlargement) of the illumination area, it is possible to allow the driver to view the display (virtual image) (or to increase the opportunities for viewing) even in the above-mentioned cases of viewpoint movement (5), (6), and (7). In addition, since the illumination is expanded to cover the entire area of ​​the eye box EB after focusing on illuminating the necessary area, it is also beneficial in terms of power saving.

[0117] In this embodiment, various modifications are possible.

[0118] For example, when reducing the amount of illumination light when reducing a wide illumination area to a narrow illumination area, the control unit 100 may reduce the illumination light by slowly lowering the brightness of pixels or light source elements 122 that have little effect on the brightness viewed at the viewpoint position.

[0119] This control is an example of a control method that assumes that light from multiple pixels (light sources) is directed toward one viewpoint. A portion of the light beam directed toward an illumination area in the eye box EB may also be directed toward other illumination areas. In other words, a portion of the light beam directed toward the partial area Z2 may enter the partial area Z5 in FIG. 3(A). If the pixels or light sources that mainly direct the light beams toward the partial areas Z1, Z2, Z3, . . . , Z9 are respectively the illumination elements C1, C2, C3, . . . , and C9, the partial area Z5 is illuminated with the light beam directed from the illumination element C5 and a portion of the light beams from the illumination elements C2, C4, C6, and C8 that are directed toward the partial areas Z2, Z4, Z6, and Z8 adjacent to the partial area Z5. Therefore, when reducing a wide illumination area (e.g., partial areas Z2, Z4, Z5, Z6, Z8) to a narrow illumination area (e.g., partial area Z5), if the light amount of the illumination elements C2, C4, C6, C8 is reduced, the brightness of the image viewed in partial area Z5 decreases. It is expected that the change (decrease) in the viewed brightness may give the viewer a sense of incongruity.

[0120] Therefore, when changing the lighting range from an expanded range to a reduced range, for example, by gradually (over time) reducing the luminance of light sources (here, C2, C3, etc. excluding lighting elements C1) that have little effect on the luminance of the virtual image viewed at the eye position (subregion X1), it is possible to make it difficult to sense the reduction in luminance of the image due to the light that reached the eye position X1 (light from lighting elements C2, C3, etc.), and thereby reduce the sense of discomfort.

[0121] In some embodiments, when the illumination range is changed from the expanded range to the contracted range, for example, when the luminance of the light source (here, C1, C2, C9 excluding the illumination element C5) that has a small effect on the luminance of the virtual image viewed at the eye position (in the example of FIG. 3, the partial region Z5) is gradually (over time) reduced, the luminance of the light source (for example, the illumination elements C1, C3, C7, C9 corresponding to the partial regions Z1, Z3, Z7, Z9 relatively far from the partial region Z5) that has a small effect on the luminance of the virtual image viewed at the eye position may be reduced faster, and the luminance of the light source (for example, the illumination elements C2, C4, C6, C8 corresponding to the partial regions Z2, Z4, Z6, Z8 relatively close to the partial region Z5) that has a large effect may be reduced slower. Here, reducing the luminance faster includes, for example, making the speed at which the luminance is gradually reduced more rapidly than the other, making the timing of reducing the speed faster than the other, or a combination of these, etc. (not limited to these).

[0122] Furthermore, when reducing a wide illumination area to a narrow illumination area, the control unit 100 may slowly reduce the luminance of pixels or light source elements 122 that have a small effect on the luminance viewed at the viewpoint, and slowly increase the luminance of pixels or light source elements 122 that have a large effect on the luminance viewed at the viewpoint. This control is another example of a control method that is based on the premise that light from multiple pixels is directed to one viewpoint.

[0123] In other words, when changing the lighting range from an expanded range to a reduced range, for example, the luminance of light sources (here, C2, C3, etc. excluding lighting element C1) that have little effect on the luminance of the virtual image viewed at the eye position (subregion X1) is gradually (over time) reduced, and the luminance of light sources (here, lighting element C1) that have a large effect on the luminance of the virtual image viewed at the eye position X1 is gradually (over time) increased.

[0124] This allows the brightness of the image (virtual image) to be maintained substantially constant even while the illumination range is being changed. Therefore, for example, the change in brightness (difference in brightness) of the image (virtual image) seen from eye position X1 when the illumination range is expanded and when it is reduced can be reduced. This further reduces the sense of discomfort.

[0125] Furthermore, the control unit 100 may variably set the first predetermined time Tth1 shown in Fig. 5(A) based on the attribute of the specific operation event. In other words, the time required to expand the illumination area to the entire area (maximum area) of the eye box EB can be variably controlled according to the attribute of the specific operation event, and control suitable for the attribute of the specific operation event can be performed.

[0126] Alternatively, the first predetermined time Tth1 may be varied based on the vehicle state at that time. For example, when the vehicle is traveling on a bumpy road and the occurrence frequency of vehicle pitching is equal to or higher than a predetermined value, when the driver's fatigue level is determined to be high, or when the contrast of the image captured by the camera 90 is equal to or lower than a predetermined value, it is preferable to shorten the first predetermined time Tth1 and increase the expansion speed to the maximum area compared to normal cases (control to reach the maximum area faster). In these cases, it is desirable from the viewpoint of safety to ensure that the driver (passenger) can visually recognize the display (virtual image).

[0127] Also, for example, since the sensitivity of the human eye is higher at night than during the day, it is possible to implement control such as control for expanding slowly over a longer period of time to reduce discomfort. Also, in cases where light and darkness change suddenly, such as in a tunnel, the speed at which the illumination area expands may be increased so that the maximum area can be reached quickly. This is because, since it is more difficult to visually recognize the display (virtual image) in rainy weather than in sunny weather, it is preferable to increase the speed at which the illumination area expands when a specific operation event is detected / predicted, and prioritize ensuring that the display is visible over the occurrence of discomfort due to changes in display luminance.

[0128] In this way, it is possible to implement optimal control adapted to various situations.

[0129] Furthermore, the control unit 100 may variably set the second predetermined time Tth2 shown in FIG. 5(A) based on the attribute of the specific operation event.

[0130] When the illumination area is reduced, the same effect can be obtained by implementing the same control as in the case of expansion.

[0131] In the present embodiment, as described above, the light source 120 has a plurality of light source elements 122a to 122d mounted (arranged) in a planar (matrix) configuration on the mounting surface of the substrate 121. Therefore, if all of the light source elements 122 are always turned on in order to always maximize the illumination area, the visibility of the image generated by the display unit 145 will be improved, but this may result in an increase in temperature of the light source 120 (or an increase in the size of the cooling mechanism for the light source 120).

[0132] In this regard, in this embodiment, as described above, spot lighting control is executed, so that the increase in temperature of the light source 120 due to heat generation from the light source element 122 (or the increase in size of the cooling mechanism for the light source 120) can be effectively suppressed.

[0133] However, as described above, since the spot lighting control is executed based on the detection result of the viewpoint position of the occupant by the viewpoint position detection unit 112, inconveniences may occur if the spot lighting control is continued when the accuracy or reliability of the detection result of the viewpoint position by the viewpoint position detection unit 112 is low. That is, there is a risk that the occupant may not be able to view the image (the image generated by the display unit 145) with appropriate display luminance.

[0134] The accuracy and reliability of the detection result of the viewpoint position by the viewpoint position detection unit 112 is likely to decrease when the viewpoint position of the occupant changes relatively significantly.

[0135] Taking this into consideration, a configuration (hereinafter also referred to as a "comparative configuration") may be considered in which, when the occupant's viewpoint position changes relatively significantly based on the viewpoint position detection result by the viewpoint position detection unit 112, spot lighting control is stopped and processing to expand the lighting area is performed.

[0136] However, in such a comparative configuration, the process of expanding the illumination area is initiated only after a relatively large change in the occupant's viewpoint position is detected, and it is likely that an initial period will occur during which the occupant is unable to view the image (the image generated by the display unit 145) at an appropriate display brightness.

[0137] In contrast, according to this embodiment, as described above, while executing the spotlighting control, only when a specific operation event is detected or predicted, the spotlighting control is temporarily stopped and the illumination area expansion process is executed. This makes it possible to allow the occupant to view an image (an image generated by the display unit 145) with appropriate display luminance even in a scene (including a vehicle state) in which a relatively large change in the occupant's viewpoint position may occur, while suppressing the light source 120 from becoming too hot.

[0138] In addition, since the specific operation event is likely to be detected or predicted before the occupant's viewpoint position changes relatively greatly or while the occupant's viewpoint position is changing relatively greatly, the possibility that the expansion process of the illumination area can be started earlier than in the comparative configuration can be increased. As a result, even if the occupant's viewpoint position changes relatively greatly, it is possible to allow the occupant to view the image (the image generated by the display unit 145) with appropriate display luminance throughout the entire period of the change. That is, even in a scene where a relatively large change may occur in the occupant's viewpoint position, it is possible to reduce the possibility that a period in which it becomes difficult for the occupant to view the virtual image (the virtual image related to the image generated by the display unit 145) or the display quality of the virtual image decreases occurs initially (temporarily). In particular, when predicting a specific operation event, it may be possible to expand the illumination area prior to the specific operation event. In this case, even if the occupant's viewpoint position changes relatively greatly, it is possible to allow the occupant to view the image (the image generated by the display unit 145) with appropriate display luminance throughout the entire period of the change.

[0139] The various effects of the present embodiment contribute to safe driving. In addition, the driver can visually recognize the display by the good HUD device 1 even when performing operations related to specific operation events in various driving scenes, and driving comfort is also ensured.

[0140] Next, an operation example will be described with reference to Fig. 9 and Fig. 10. Fig. 9 is a flowchart showing an example of an operation procedure of the HUD device 1 for expanding and reducing the illumination area. Fig. 10 is a diagram showing an example of the expansion and reduction of the illumination area suitable for the operation example of Fig. 9. The process shown in Fig. 9 may be repeatedly executed at predetermined intervals, for example, when the vehicle is in an activated state.

[0141] In step S900, the control unit 100 detects the viewpoint position of the occupant by the viewpoint position detection unit 112 based on the imaging information (imaging signal) S0 from the camera 90. The method of detecting the viewpoint position of the occupant may be as described above.

[0142] In step S902, the control unit 100 acquires various information S3 supplied from the ECU 160.

[0143] In step S904, the control unit 100 determines whether or not the illumination area is in a full illumination state (a state in which the entire illumination area is illuminated) or in transition to a full illumination state due to the illumination area expansion control started in step S904 described later. If the determination result is "YES", the process proceeds to step S912, and otherwise the process proceeds to step S906.

[0144] In step S906, the control unit 100 judges whether or not a specific operation event is detected or predicted. The method of detecting or predicting a specific operation event may be as described above. If the judgment result is "YES", the process proceeds to step S908, and otherwise, the process proceeds to step S910.

[0145] In step S908, the control unit 100 executes (starts) the expansion control of the illumination area (the maximum area is the entire eye box) to realize a full lighting state (a state in which the entire illumination area is illuminated). Note that the expansion control of the illumination area is accompanied by the stopping (temporary stopping) of the spotlighting control, as described above.

[0146] In step S910, the control unit 100 executes spot lighting control based on the viewpoint position of the occupant obtained in step S900. The spot lighting control may be as described above.

[0147] In step S912, the control unit 100 judges whether or not a predetermined time Tth0 has elapsed since the current specific operation event was detected or predicted. If the judgment result is "YES", the process proceeds to step S916, otherwise the process proceeds to step S914.

[0148] In step S914, the control unit 100 continues the expansion control of the illumination area started in step S904 to realize or maintain a full lighting state (a state in which the entire illumination area is illuminated). For example, in the example shown in FIG. 10, the control unit 100 gradually increases the size of the illumination area as shown by the arrows R100 to R102 in (a). Specifically, the control unit 100 controls the LED array of the light source 120 so that the illumination pattern is changed stepwise from the original illumination pattern PT1 based on the viewpoint detection through illumination patterns PT2 and PT3 to emit light in an illumination pattern PT4 that illuminates the entire eye box. Note that, as an example, each of the illumination patterns PT1 to PT14 shown in FIG. 10 is an illumination pattern by the light source elements 122 arranged in a 7×5 array, with white outlines representing "on" and black outlines representing "off". The two lit light source elements 122 in the illumination pattern PT1 generate light that is projected onto the left and right eyes of the occupant, respectively.

[0149] In step S916, the control unit 100 ends the expansion control of the illumination area and resumes the spotlighting control. At this time, the control unit 100 may execute a reduction process to gradually reduce the illumination area as described above. For example, in the example shown in FIG. 10, the control unit 100 gradually reduces the size of the illumination area as shown by the arrows R110 to R112 in (b). Specifically, the control unit 100 controls the LED array of the light source 120 to emit light in the original illumination pattern PT14 based on the viewpoint detection by gradually changing the illumination pattern through the illumination pattern PT11 (=PT4 in (a)) that illuminates the entire eye box, the illumination patterns PT12, and PT13.

[0150] 9, when a specific operation event is detected or predicted, a full lighting state (a state in which the entire lighting area is illuminated) can be realized regardless of the result of the detection of the viewpoint position by the viewpoint position detection unit 112. This can reduce inconveniences that may occur due to a specific operation event causing a relatively large change in the viewpoint position of the occupant (deterioration in the detection accuracy of the viewpoint position by the viewpoint position detection unit 112, and the possibility that the occupant may not be able to view the image related to the vehicle information at an appropriate brightness).

[0151] 9, the viewpoint position detection process by the viewpoint position detection unit 112 continues even during the execution of the expansion control of the illumination area, but is not limited to this. For example, the viewpoint position detection process by the viewpoint position detection unit 112 may be stopped during part or the entire execution period of the expansion control of the illumination area.

[0152] Although each embodiment has been described above in detail, the present invention is not limited to the specific embodiment, and various modifications and changes are possible within the scope of the claims. In addition, it is also possible to combine all or a plurality of the components of the above-described embodiments. [Explanation of symbols]

[0153] 1 HUD device 2 Windshield 9. Instrument Panel 90 Camera 100 Control section 110 Partial lighting control unit 112 Viewpoint position detection unit 113 Parameter Acquisition Unit 114 Event Processing Unit 116 Light emission control unit 118 Area size change section 120 light source 121 Substrate 122 Light source element 130 Optical system (first optical system) 140 Liquid crystal panel 145 Display section

Claims

1. A head-up display device that is mounted on a vehicle and allows an occupant of the vehicle to visually recognize a virtual image including vehicle information, A display unit for displaying the image; A control unit that controls the display unit, The control unit is A viewpoint position detection unit that detects the viewpoint position of an occupant; a light emission control unit that, based on the viewpoint position detected by the viewpoint position detection unit, sets a partial area of ​​an entire area of ​​an eyebox corresponding to the viewpoint position as an illumination area, and irradiates light related to the image via the display unit; A parameter acquisition unit that acquires values ​​of vehicle parameters; and an area size changing unit that gradually changes the size of the illumination area over time based on the values ​​of the vehicle parameters acquired by the parameter acquisition unit.

2. The control unit further includes an event processing unit that detects or predicts a specific operation event that may cause a change in the viewpoint position among various operation events that may be performed by an occupant, The head-up display device according to claim 1 , wherein the area size changing unit increases a size of the illumination area when the specific operation event is detected or predicted by the event processing unit.

3. 3. The head-up display device according to claim 2, wherein when the specific operation event is detected or predicted by the event processing unit, the area size changing unit maximizes a size of the illumination area so that the illumination area corresponds to an entire area of ​​an eye box, and maintains the maximized size of the illumination area until a predetermined time has elapsed.

4. The head-up display device according to claim 2 or 3, wherein the specific operation event includes an operation event performed on an operation unit disposed on an instrument panel or a console box.

5. The vehicle is capable of switching between an automatic driving mode and a manual driving mode, The head-up display device according to claim 2 , wherein the specific operation event includes an operation event related to a switching operation from an autonomous driving mode to a manual driving mode.

Images