Head-up display device, display control device, and display control method

The HUD system dynamically adjusts illumination area to maintain visibility and reduce discomfort by expanding when viewpoint loss occurs, addressing the challenge of viewpoint deviation in HUD systems.

JP7584001B2Active Publication Date: 2024-11-15NIPPON SEIKI CO LTD
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
JP2022510551
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Priority Date
2021-03-22
Filing Date
2021-03-23
Publication Date
2024-11-15
Estimated Expiration
2041-03-23

AI Technical Summary

Technical Problem

Existing head-up display (HUD) systems face issues with viewpoint tracking spotlighting, where the driver's viewpoint deviation leads to temporary loss, causing discomfort and reduced visibility of the virtual image, especially in larger HUD devices that aim for energy savings by reducing the partial illumination area.

Method used

The HUD system expands the illumination area when viewpoint loss is detected or predicted, using control units to adjust the lighting area dynamically, including continuous or staged expansion, multi-value lighting control, and estimation of viewpoint movement to maintain visibility and reduce discomfort.

Benefits of technology

This approach ensures continuous visibility of the virtual image by adapting the illumination area to the driver's viewpoint, reducing discomfort and enhancing safety by maintaining image visibility even during temporary viewpoint deviations.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007584001000001
    Figure 0007584001000001
  • Figure 0007584001000002
    Figure 0007584001000002
  • Figure 0007584001000003
    Figure 0007584001000003
Patent Text Reader

Abstract

Through the present invention, during visual point tracking spot lighting control for radiating light to a partial region of an eye box in accordance with the visual point position of an operator, problems such as a virtual image becoming invisible are suppressed and discomfort is reduced when the visual point is lost or visual point loss becomes prone to occur. A head-up display (HUD) device which is mounted to a vehicle and which causes a virtual image of an image to be visually recognized has a display unit 145 for displaying an image, and a control unit 100 for performing visual point tracking spot lighting control for radiating light to a partial region of an eye box EB in accordance with the visual point position of an operator, the control unit 100 performing control so as to expand a region of a light source or a pixel that contributes to display of a virtual image in the display unit 145 when visual point loss in which the visual point position cannot be detected occurs, or when visual point loss becomes prone to occur, and to illuminate a wider illumination region than a partial illumination region of the eye box EB that was illuminated immediately prior to the expansion.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a head-up display (HUD) device, a display control device, a display control method, etc. that projects (projects) image display light onto a projection member such as a windshield or combiner of a vehicle (vehicle) and displays a virtual image in front of a driver, etc. [Background technology]

[0002] In a HUD device, lighting the entire eyebox at all times would be wasteful, so from the perspective of energy conservation, it has been proposed to implement viewpoint-tracking spot lighting (partial lighting), which illuminates a partial area of ​​the eyebox according to the driver's viewpoint (for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6586646 Summary of the Invention [Problem to be solved by the invention]

[0004] The present inventors have investigated detecting the viewpoint position of a driver (which can be broadly interpreted as a pilot, crew member, etc.) and implementing viewpoint-tracking spotlighting, and have recognized the following new problems.

[0005] There may be cases where the driver's viewpoint moves and the HUD device temporarily loses the viewpoint. In this case, if the driver's viewpoint (at least one of the two eyes) moves out of the partial illumination area, the driver will not be able to see the virtual image. This may give the driver a sudden feeling and lead to a sense of discomfort.

[0006] However, even if the driver's viewpoint is outside the partial illumination area, if it is actually located within the eye box, it is possible to continue viewing the virtual image as long as light is projected onto the area where the viewpoint is located.

[0007] In addition, even if the driver's viewpoint moves out of the partial illumination area and out of the eye box, the viewpoint should return to the eye box afterwards, but it does not necessarily return to the original position at this time. It is also possible that the viewpoint may return to a position quite far from the original position for some reason.

[0008] In addition, even when returning to the vicinity of the original position, it is conceivable that the viewpoint moves by trial and error, and the viewpoint position gradually stabilizes and eventually stops. When the viewpoint is moving by trial and error and the viewpoint position is outside the original partial illumination area, no light is projected onto the viewpoint on the way back, and a state occurs in which the virtual image cannot be seen.

[0009] However, the fact that the viewpoint returns to the eyebox means that the viewpoint is located inside the eyebox, and if the lighting can be devised to project light (illumination light) onto the viewpoint, it will be possible to see the virtual image.

[0010] Also, viewpoint loss may occur when the viewpoint moves significantly for some reason, or it may occur when the viewpoint moves rapidly and instantaneously, even if the distance moved is relatively short.

[0011] There are many different types of viewpoint loss. It is important to take measures that take into account the circumstances of viewpoint loss, if necessary.

[0012] Furthermore, in recent years, for example, HUD devices capable of displaying virtual images over a fairly wide range in front of the vehicle have been developed, and such HUD devices tend to become larger. From the viewpoint of energy saving, it is preferable to reduce the area of ​​the partial illumination region, but in this case, there is an increased possibility that the driver (user) will not be able to see the virtual image when the driver loses his / her line of sight. Therefore, it tends to be difficult to achieve both energy saving performance and increased visibility.

[0013] Thus, there is room for improvement in implementing viewpoint tracking spotlighting, and the above-mentioned Patent Document 1 is silent on this point.

[0014] One of the objectives of the present invention is to prevent virtual images from becoming invisible and reduce the sense of discomfort when the driver loses their viewpoint while performing viewpoint-tracking spotlighting control, which irradiates a partial area of ​​the eyebox according to the driver's viewpoint position.

[0015] Other objects of the present invention will become apparent to those skilled in the art by referring to the following exemplary aspects and best modes, as well as the accompanying drawings. For example, another object of the present invention is to prevent the virtual image from becoming invisible or the quality of the virtual image from deteriorating due to the display light not being properly irradiated to the viewpoint, not only when viewpoint loss occurs but also when a state in which viewpoint loss is likely to occur, thereby reducing the sense of discomfort. [Means for solving the problem]

[0016] In order to facilitate an understanding of the outline of the present invention, embodiments according to the present invention will be exemplified below.

[0017] In a first aspect, a head-up display device includes: A head-up display (HUD) device mounted on a vehicle and allowing a virtual image to be viewed, A display unit for displaying the image; a control unit that performs viewpoint tracking spotlighting control to irradiate a partial area of ​​an eye box with light according to a viewpoint position of a driver; having The control unit is When a viewpoint loss occurs, in which it is not possible to detect the viewpoint position, the area of ​​pixels or light sources contributing to the display of the virtual image on the display unit is expanded, and a lighting area wider than the partial lighting area of ​​the eye box that was illuminated immediately before the viewpoint loss is controlled to be illuminated.

[0018] In the first aspect, when viewpoint-tracking spotlighting is being performed and the viewpoint position is lost (viewpoint lost), control is performed to illuminate a lighting area larger than the partial lighting area that was spotlit (partially illuminated) before the viewpoint was lost.

[0019] For example, when the viewpoint is lost, the illumination area may be expanded to cover the entire area (front surface) of the eyebox.

[0020] When the display unit that displays the image forms pixels with self-luminous elements such as LEDs or organic EL, the illumination area can be expanded by controlling the light emission state of the pixels, and when the display unit is a light-transmitting display unit that combines a liquid crystal panel and a backlight (light source), the illumination area can be expanded by controlling the light emission state (on, off, etc.) of the light source (light source element). In other words, for example, by changing the pixels (non-emitting pixels) around the pixel that was emitting light just before the viewpoint was lost to an emitting state (a state that can contribute to the visibility of a virtual image), it is possible to expand the illumination area around the illumination area just before the viewpoint was lost.

[0021] Here, it is preferable to "expand based on the lighting area immediately before the viewpoint is lost". If the viewpoint is lost for a short period of time, the viewpoint is likely to remain near the original lighting area, and expanding around the original area increases the possibility of projecting light onto the viewpoint.

[0022] Even if the viewpoint (at least one of the left and right eyes) is temporarily lost due to a movement of the viewpoint, if the viewpoint is located in the expanded illumination area, the driver (user) can view the display (virtual image), and the possibility of preventing the display from suddenly disappearing is increased. This provides the effect of reducing the occurrence of discomfort. In addition, the possibility of viewing the display necessary for the operation of the vehicle is increased, thereby improving safety reliability.

[0023] In addition, this embodiment can be applied to both single-eye HUDs that project the same image to each of the driver's eyes, and binocular (parallax, hologram, or optical reproduction) HUDs that project different images (parallax images) to each eye.

[0024] Moreover, the "viewpoint loss" in this embodiment can be assumed to be both a viewpoint loss in which at least one of the eyes is inside the eyebox, and a viewpoint loss in which both eyes are outside the eyebox.

[0025] In a second embodiment dependent on the first embodiment, The control unit is The illumination area may be controlled to be expanded continuously or in a stepwise manner.

[0026] According to the second aspect, by gradually expanding the illumination area, it is possible to suppress the sense of incongruity that may occur when the illumination area is expanded to cover the entire eyebox at once. In other words, it is possible to achieve "spotlighting expansion control" that produces less sense of incongruity.

[0027] If the illumination area is expanded continuously on the time axis, the sense of incongruity can be further reduced. In addition, if the illumination area is expanded stepwise (in steps), efficient expansion can be performed easily (with less strain on the device) with regularity according to a specified expansion algorithm. Note that by combining continuous expansion and stepwise expansion, it is possible to expand the illumination area with a higher degree of freedom.

[0028] In a third embodiment dependent on the second embodiment, The control unit is When the viewpoint loss continues for a first predetermined time or longer, a control for illuminating the entire eyebox may be performed.

[0029] When the partial illumination area in the eye box is expanded, a part of the light beam heading toward the expanded illumination area may enter the original illumination area. As described in the first embodiment, the entire eye box may be illuminated at once, but in this case, it is assumed that the brightness of the image (virtual image) viewed in the original illumination area may increase suddenly, and it cannot be said that there is no possibility that the driver (user) may feel uncomfortable due to the change in brightness.

[0030] Therefore, in this embodiment, the control to expand the illumination range to the entire area is performed on the condition that the first predetermined time has elapsed. This allows the size of the partial illumination area to be gradually expanded during the period before the first predetermined time has elapsed. This makes it possible to suppress (reduce) the sense of discomfort.

[0031] In a fourth aspect dependent on the second or third aspect, The control unit is When the illumination area is expanded continuously or stepwise, control may be performed to increase the amount of illumination light.

[0032] In this embodiment, when the illumination area is expanded, the amount of light from the light source in the direction in which the illumination area is expanded is controlled (gradually increased to a desired luminance), thereby making it possible to further reduce the sense of discomfort.

[0033] For example, the illumination area (partial illumination area) immediately before the viewpoint is lost is designated as A, and the expanded area is designated as B. The luminance of area B can be increased to the same luminance as area A in one go. In this case, however, it cannot be denied that the luminance change in the peripheral visual field will be somewhat large, as described in the third embodiment above. In this case, it cannot be said that there is no possibility that the driver (user) will feel uncomfortable.

[0034] Therefore, in this embodiment, the pixels of the display unit (display pixels) are not controlled to be switched on / off at a desired brightness (binary control of on / off), but are controlled to be raised to a desired brightness over time (in other words, multi-value lighting control), which can further reduce the sense of discomfort.

[0035] In a fifth aspect dependent on any one of the second to fourth aspects, The control unit is When expanding the lighting area continuously or stepwise, a control may be implemented to estimate or predict the direction in which the viewpoint was lost based on at least one of the direction of movement of the viewpoint before the viewpoint was lost, past learning results regarding the movement of the viewpoint, and the driving conditions at the time the viewpoint was lost, and to expand the lighting area to include at least the estimated or predicted direction.

[0036] In this aspect, the direction in which viewpoint loss occurred (the direction in which the viewpoint moved when the viewpoint was lost) is estimated or predicted, and the illumination area is expanded by utilizing the details of the estimation (prediction).

[0037] Even when the gaze point is lost, it may be possible to estimate (or predict) the direction of gaze point movement. For example, when only one of the eyes is lost, the direction of gaze point loss can be estimated (predicted) by tracking the movement trajectory of the other eye (estimation based on gaze point movement direction).

[0038] In addition, drivers (users) have individual characteristics in their gestures and actions while driving. For example, if it is known from past learning that a certain driver frequently changes his / her posture while driving, which causes the viewpoint to change up and down (in the direction of the vehicle's height), then it is possible to utilize the results of this learning to estimate (predict) that the viewpoint has moved in the vertical direction, causing viewpoint loss (estimation or prediction through learning).

[0039] In addition, if the vehicle is currently traveling on a bumpy road (such as an unpaved road) (this can be determined, for example, from vehicle information collected by the ECU, etc.), it can be estimated (predicted) that the viewpoint may have momentarily shifted upward, downward, or diagonally, causing the vehicle to become lost (estimate or prediction based on the driving situation).

[0040] By performing such estimation (prediction) and expanding to include the estimated (predicted) direction, efficient expansion is possible. For example, by expanding the estimated (predicted) direction more thoroughly (increasing the degree of expansion) than the non-estimated (non-predicted) direction, the possibility of projecting an image onto the viewpoint during a lost state increases, and the possibility that the expansion of the illumination area will efficiently lead to increased visibility of the virtual image increases. In addition, since it is possible to focus on illuminating the necessary areas, it is also beneficial in terms of energy conservation.

[0041] In a sixth aspect dependent on any one of the second to fifth aspects, The eye box has a predetermined outer shape, The control unit is When the illumination area is expanded continuously or stepwise, isotropic expansion, which expands the illumination area uniformly around its periphery; Or anisotropic expansion, where the expansion is not uniform, or performing either the isotropic expansion or the anisotropic expansion in combination; When the viewpoint position is detected again while the expansion is being performed, or when the illumination region expands to cover the entire area of ​​the eyebox, control may be performed to terminate the expansion.

[0042] In the sixth aspect, the eyebox and the outline (contour) of the illumination area have a predetermined shape, and the illumination area can be expanded by isotropic expansion, anisotropic expansion, or a combination of these. Note that "expanding the illumination area uniformly around its periphery" can also be rephrased as "expanding the illumination area to a region similar to the original shape of the illumination area and having a larger area." Also, "expanding non-uniformly around its periphery" can also be rephrased as "expanding the illumination area to a region not similar to the original shape of the illumination area and having a larger area."

[0043] When an edge of the outline of the illumination area reaches an edge of the eye box, further expansion is not possible, so expansion is performed for other unreached edges, and the expansion ends when the viewpoint is redetected during expansion or when expansion has reached the entire eye box.

[0044] According to this aspect, the rules for expansion are clarified, and the effects of facilitating control by software are obtained.

[0045] In a seventh aspect dependent on any one of the first to sixth aspects, The control unit is When the viewpoint position is redetected after viewpoint loss occurs, control may be implemented to reduce the wide illumination area expanded after viewpoint loss to a narrow illumination area of ​​the same size as the partial illumination area before viewpoint loss, corresponding to the redetected viewpoint position.

[0046] According to the seventh aspect, when the viewpoint is restored from being lost, the same (normal) spotlighting state as before the viewpoint was lost can be restored without any sense of incongruity.

[0047] In an eighth embodiment dependent on the sixth embodiment, The eye box has a predetermined outer shape, The control unit is When the viewpoint position is redetected after the viewpoint loss occurs, a control is performed to reduce the wide illumination area expanded after the viewpoint loss to a narrow illumination area having the same size as the partial illumination area before the viewpoint loss, the narrow illumination area corresponding to the redetected viewpoint position; In addition, when reducing the illumination area, A control is performed to restore the illumination area to a narrow illumination area having the same size as the partial illumination area before the viewpoint is lost by performing a combination of anisotropic shrinkage in which the illumination area is uniformly reduced and anisotropic shrinkage in which the illumination area is reduced non-uniformly. Or, When a contraction along one of two orthogonal line segments is defined as a vertical contraction and a contraction along the other is defined as a horizontal contraction, a vertical contraction is performed, and then a horizontal contraction is performed, thereby performing a control to return the partial illumination area to a narrow illumination area having the same size as the partial illumination area before the viewpoint is lost. Or, A control may be performed in which horizontal contraction is performed, and then vertical contraction is performed to return the partial illumination area to a narrow illumination area having the same size as the partial illumination area before the viewpoint is lost.

[0048] In the eighth aspect, when the outer shape (contour) of the eye box and the illumination area is a predetermined shape, such as a rectangle (including a square and a rectangle), the illumination area can be reduced by using any of the following methods: a first method of performing isotropic reduction followed by anisotropic reduction, a second method of performing vertical reduction followed by horizontal reduction, or a third method of performing horizontal reduction followed by vertical reduction.

[0049] According to the eighth aspect, the rules for reduction are clarified, and effects such as easy control by software can be obtained.

[0050] In a ninth aspect dependent on the seventh or eighth aspect, The control unit is The time required for the wide illumination area to become the narrow illumination area may be controlled to be equal to or longer than a second predetermined time.

[0051] When the viewpoint is redetected, the lighting transitions to partial lighting based on the redetected viewpoint position (in principle, partial lighting of the same size as the partial lighting area before the viewpoint was lost); however, the driver's (user's) viewpoint may continue to move to a certain extent and converge to a certain position.

[0052] Therefore, in the ninth aspect, the normal spotlighting state is restored after a predetermined time has elapsed (after the time required for the viewpoint to stabilize has elapsed). This makes it possible to suppress (reduce) the viewpoint from moving out of the lighting area due to a small movement of the viewpoint on the way back.

[0053] In a tenth aspect dependent on any one of the seventh to ninth aspects, The control unit is When reducing the wide illumination area to the narrow illumination area, control may be performed to reduce the amount of illumination light.

[0054] According to the tenth aspect, by gradually reducing the amount of illumination light (illumination brightness) when reducing the illumination area, it is possible to suppress (reduce) the sense of discomfort that may occur when switching to normal spot lighting all at once.

[0055] In an eleventh aspect dependent on the tenth aspect, The control unit is In the case where the amount of illumination light is reduced when reducing the wide illumination area to the narrow illumination area, Illumination light may be reduced by slowly decreasing the luminance of pixels or light source elements that have a small effect on the luminance perceived at the viewpoint position.

[0056] In the eleventh aspect, an example of a control method based on the premise that light from a plurality of pixels is directed toward one viewpoint will be shown.

[0057] When changing the lighting range from an expanded range to a reduced range, for example, by gradually (over time) reducing the luminance of light sources (here, C2, C3, etc. excluding light source element C1) that have little effect on the luminance of the virtual image viewed at the eye position (X1), it is possible to make it difficult to sense the reduction in luminance of the image due to the light that reached the eye position X1 slightly (light from light sources C2, C3, etc.), and to reduce the sense of discomfort.

[0058] In a twelfth aspect dependent on any one of the seventh to ninth aspects, The control unit is When reducing the wide illumination area to the narrow illumination area, The luminance of pixels or light source elements that have a small effect on the luminance viewed at the viewpoint position may be decreased over time, and the luminance of pixels or light source elements that have a large effect on the luminance viewed at the viewpoint position may be increased over time.

[0059] In the twelfth aspect, another example of a control method based on the premise that light from a plurality of pixels is directed toward one viewpoint will be described.

[0060] In this embodiment, when the illumination range is changed from an expanded range to a reduced range, for example, the luminance of light sources (here, C2, C3, etc. excluding light source element C1) that have little effect on the luminance of the virtual image viewed at eye position (X1) is gradually (over time) reduced, and the luminance of light source C1 that has a large effect on the luminance of the virtual image viewed at eye position X1 is gradually (over time) increased.

[0061] This allows the brightness of the image (virtual image) to be maintained substantially constant even while the illumination range is being changed. Therefore, for example, the change in brightness (difference in brightness) of the image (virtual image) seen from eye position X1 when the illumination range is expanded and when it is reduced can be reduced. This further reduces the sense of discomfort.

[0062] In a thirteenth aspect dependent on the third aspect, The control unit may variably control the first predetermined time period based on at least one of a driving condition of the vehicle, a surrounding environment, and a condition of a driver.

[0063] In the thirteenth aspect, the time required to expand the illumination area to the entire area (maximum area) of the eye box is variably controlled in response to various factors, and control suitable for the driving scene is implemented.

[0064] For example, when the vehicle is traveling on a bumpy road and the occurrence frequency of vehicle pitching is equal to or higher than a predetermined value, when the driver's fatigue level is determined to be high, when the contrast of the image captured by the camera is equal to or lower than a predetermined value, etc., it is preferable to shorten the first predetermined time and increase the expansion speed to the maximum area compared to normal (control to reach the maximum area faster). In these cases, it is desirable from the viewpoint of safety to ensure that the driver (user) can visually recognize the display (virtual image).

[0065] Also, for example, since the sensitivity of the human eye is higher at night than during the day, it is possible to control the expansion slowly over a longer period of time to reduce discomfort. In addition, in cases where light and darkness change suddenly, such as in a tunnel, the speed at which the illumination area expands may be increased so that the maximum area can be reached quickly. This is because it is more difficult to visually recognize the display (virtual image) in rainy weather than in sunny weather, so it is preferable to increase the speed at which the illumination area expands when lost, and give priority to ensuring that the display is visible rather than to the occurrence of discomfort due to changes in display luminance.

[0066] In this way, according to the thirteenth aspect, it is possible to carry out optimal control adapted to various situations.

[0067] In a fourteenth aspect dependent on the ninth aspect, The control unit may variably control the second predetermined time based on at least one of a driving state of the vehicle, a surrounding environment, and a state of the driver.

[0068] When the illumination area is reduced, the same effect can be obtained by implementing the same control as described in the thirteenth embodiment.

[0069] In a fifteenth aspect dependent on any one of the first to fourteenth aspects, The control unit is Regarding the light amount distribution of the illumination light in the illumination area (including the wide illumination area and the narrow illumination area), The amount of light in the peripheral portion of the illumination area is controlled so as to be less than the amount of light in the central portion, but to ensure a luminance equal to or greater than a predetermined value. Or, The amount of light may be controlled to be uniform within the illumination area.

[0070] In the fifteenth aspect, a preferred example of the distribution of the amount of light (luminance) in the illumination area (including the partial illumination area) is shown. When the viewpoint moves out of the partial illumination area (the spotlighting area), it can be assumed that the luminance becomes extremely small, and the visibility of the image (virtual image) drops extremely and suddenly.

[0071] In this case, it is desirable to reduce the discomfort felt by the viewer when the virtual image suddenly disappears (the brightness drops drastically) when the eye (point of view) approaches the boundary (edge) between the partially illuminated area and the non-illuminated area.

[0072] In one preferred example, if there is a rectangular illumination area, the amount of light is large in the center and small in the periphery (near the boundary), but the amount of light (brightness) is low enough to ensure a certain amount (above a predetermined value) of light, and the virtual image is prevented from suddenly disappearing when the viewpoint moves outside the above-mentioned boundary. In this case, when the viewpoint moves outside the boundary, a faint virtual image is seen and then the image disappears, which makes it difficult to feel uncomfortable.

[0073] In another preferred example, the illumination area (lighting area) is illuminated with uniform (including approximately uniform) light so as to achieve a predetermined luminance level. In this case, the amount of light also decreases from the edge (boundary) vicinity, but a certain amount of light is ensured, so that when the viewpoint moves outside the boundary, a faint virtual image appears and then disappears, making it difficult to feel uncomfortable.

[0074] In this way, according to the fifteenth aspect, the sense of discomfort felt when the display (virtual image) suddenly disappears when the viewpoint moves outside the illumination area is reduced.

[0075] Those skilled in the art will easily understand that the exemplified embodiments according to the present invention can be further modified without departing from the spirit of the present invention. For example, the present invention can reduce the sense of discomfort not only when viewpoint loss occurs, but also when a state in which viewpoint loss is likely to occur, by preventing the virtual image from becoming invisible or the quality of the virtual image from deteriorating due to the display light not being properly irradiated to the viewpoint. In a sixteenth aspect of the present invention, a head-up display device is a head-up display (HUD) device mounted on a vehicle and allowing a virtual image of an image to be visually recognized, A display unit for displaying the image; a control unit that performs viewpoint tracking spotlighting control to irradiate a partial area of ​​an eye box with light according to a viewpoint position of a driver; having The control unit is When a state in which viewpoint loss, in which the viewpoint position cannot be detected, is likely to occur, is detected, the area of ​​pixels or light sources contributing to the display of the virtual image on the display unit is expanded, and a lighting area wider than the partial lighting area of ​​the eye box that was illuminated immediately before the expansion is controlled to be illuminated. According to the sixteenth aspect, by detecting or predicting a state where viewpoint loss is likely to occur, measures can be taken to prevent a displayed image (such as a virtual image) from becoming invisible or to prevent a deterioration in display quality before viewpoint loss actually occurs. Therefore, quick and reliable measures can be taken. In a seventeenth aspect dependent on the sixteenth aspect, The control unit is When the moving speed of the viewpoint is equal to or greater than a first threshold, it may be determined that the viewpoint loss is likely to occur. In the seventeenth aspect, when the viewpoint (eye) position moves at a high speed at a speed equal to or greater than a threshold, it is determined that viewpoint loss is likely to occur, and measures are taken. For example, such a situation is likely to occur when the driver's face suddenly swings widely. In this aspect, it is possible to quickly and reliably respond to such a situation. In an eighteenth embodiment dependent from the sixteenth embodiment, The control unit is When a sudden change in brightness of the surroundings including the front of the vehicle is detected or a sudden change is predicted, And, If a sudden change in the brightness inside the vehicle is detected or predicted, And, When a rear seat monitor or a child monitor provided in the vehicle is turned on or is in an on state, And, When it is determined that the driving environment of the vehicle is an environment that increases the fluctuation of the face or eye position of the driver, When at least one of the above conditions is met, it may be determined that the state is one in which viewpoint loss is likely to occur. According to the eighteenth aspect, measures can be taken in advance in response to various driving situations where viewpoint loss is likely to occur. A sudden change in brightness is likely to occur, for example, when a vehicle enters or exits a tunnel. When a sudden change in brightness occurs, the driver tends to close his eyes or look away from the road ahead, which can be one of the causes of viewpoint loss. In addition, sudden changes in brightness inside the vehicle are likely to occur when, for example, highly directional light from the headlights of oncoming vehicles or highly directional light from street lights using LEDs enters the vehicle. This situation can also be a cause of viewpoint loss. In addition, when a rear seat monitor or a child monitor is turned on, the driver tends to look at the monitoring terminal, which can be one of the reasons for losing focus. In addition, when the vehicle is traveling on rough roads, such as a road with many bumps or a muddy unpaved road, or a road with a series of sharp curves, the driver is likely to lose focus due to shaking of the body or face. According to this aspect, measures to prevent loss of viewpoint can be implemented in such various driving situations. In a nineteenth aspect dependent on any one of the sixteenth to eighteenth aspects, The expansion of the illumination area may include a multi-stage expansion process. This can reduce the discomfort felt by the driver and also reduce the burden on the device. In a nineteenth aspect dependent on any one of the sixteenth to twentieth aspects, The control unit is When the state in which the viewpoint loss is likely to occur is resolved, control may be performed to reduce the expanded wide illumination area to a narrow illumination area having the same size as the partial illumination area before expansion. According to the nineteenth aspect, when the state in which the possibility of viewpoint loss is high is eliminated, the illumination area is reduced. This makes it possible to suppress, for example, a rise in temperature of the light source and also to reduce the driving power of the light source. In a twenty-first aspect dependent on the seventeenth aspect, The control unit is When the moving speed of the viewpoint becomes less than the first threshold value, Or, A value smaller than the first threshold value or equal to or smaller than a second threshold value larger than the first threshold value So, A control is performed to reduce the expanded wide illumination area to a narrow illumination area having the same size as the partial illumination area before expansion. In the 21st aspect, when the necessity of expanding the illumination area is determined based on the result of comparing the movement speed of the viewpoint with the first threshold, the necessity of reducing the illumination area is also determined by comparing with the first threshold or with a second threshold different from the first threshold. Since it is a comparative determination, it is easy to realize. The second threshold may be smaller than the first threshold. In this case, it is not determined that the illumination area needs to be reduced unless the moving speed of the viewpoint falls below the first threshold and then becomes equal to or less than the second threshold. Therefore, the illumination area can be stably maintained in an expanded state. In this case, the effect of reducing the probability that the viewpoint will fall out of the illumination range is obtained. On the other hand, the second threshold may be greater than the first threshold. In this case, when the speed of movement of the viewpoint becomes equal to or less than the second threshold, it is determined that the illumination area needs to be reduced. This shortens the period during which the illumination area is expanded, thereby achieving effects such as suppressing the temperature rise of the light source and reducing power consumption. In a twenty-second aspect dependent on the twenty-first or twenty-first aspect, The reduction of the illumination area may include a multi-stage reduction process, thereby reducing the discomfort felt by the driver. In a twenty-third aspect, a display control device has the control unit in any one of the sixteenth to twenty-first aspects. According to the 23rd aspect, the display control device can include a control unit capable of dealing with viewpoint loss, thereby improving, for example, the portability or ease of installation of the control unit. In a 24th aspect, a display control method includes: A step of performing viewpoint tracking spotlighting control for irradiating a partial area of ​​an eye box with light according to a viewpoint position of a driver; When a viewpoint loss occurs in which the viewpoint position cannot be detected, or when a state in which the viewpoint loss is likely to occur is detected, an area of ​​pixels or light sources contributing to display of an image is expanded, and an illumination area wider than the partial illumination area of ​​the eye box that was illuminated immediately before the expansion is performed is illuminated; When the viewpoint position is redetected after the viewpoint loss occurs, or when the state in which the viewpoint loss is likely to occur is resolved, reducing the expanded wide illumination area to a narrow illumination area having the same size as the partial illumination area before the expansion; Includes. According to the twenty-fourth aspect, when viewpoint loss occurs or when viewpoint loss is likely to occur, effective measures can be implemented ex post or in advance. [Brief description of the drawings]

[0076] [Figure 1] FIG. 1(A) is a diagram showing an overview of viewpoint tracking spotlighting control and an example of the main configuration of a HUD device equipped with a function for expanding the lighting area when viewpoint loss occurs, and FIG. 1(B) is a diagram showing a state in which viewpoint loss has occurred. [Diagram 2] FIG. 2A is a diagram showing a state in which the illumination area is expanded after the viewpoint is lost, and FIG. 2B is a diagram showing a preferable example of the light amount distribution of the illumination light in the illumination area. [Diagram 3] 3A and 3B are diagrams showing an example of the configuration of an eyebox used in a HUD device. [Figure 4] FIG. 4(A) is a diagram showing an example of the configuration of a stereoscopic display (3D) HUD device to which the present invention is applied, and FIG. 4(B) is a diagram for explaining the principle of displaying a stereoscopic image (virtual image). [Diagram 5] FIG. 5(A) is a diagram showing the transition (change) of the viewpoint detection state over time, and FIGS. 5(B) to (F) are diagrams showing the expansion and contraction of the illumination area in the eyebox in response to movement of the viewpoint. [Figure 6] 6(A) to 6(H) are diagrams showing an example of the expansion and contraction of the illumination area. [Figure 7] 7(A) to (D) are diagrams showing an example of contraction and expansion of the illumination area. [Figure 8] Figures 8(A) and (B) are diagrams showing other examples of changes in the viewpoint position (examples that also take into account the case where the viewpoint returns to a redetected state from a lost state), as well as the expansion of the lighting area by estimating or predicting the movement of the viewpoint. [Figure 9] 11 is a flowchart showing an example of an operation procedure of the HUD device for expanding and reducing the illumination area. [Figure 10] FIG. 10 is a diagram illustrating an example of the configuration of a HUD device. [Figure 11] FIG. 11 is a diagram illustrating an example of the configuration of the display unit. [Figure 12] 12A and 12B are diagrams showing an overview of viewpoint tracking spotlighting control using the display unit shown in FIG. [Figure 13] FIG. 13 is a diagram showing an example of an arrangement of an LED array in a display unit. [Figure 14] FIG. 14(A) is a diagram showing the illumination area in the eye box before the illumination area is expanded, and FIG. 14(B) is a diagram showing the illumination area in the eye box after the illumination area is expanded, and an example of the configuration of a control unit that expands or reduces the illumination area. [Figure 15] FIG. 15(A) is a diagram showing an example of an LED lighting pattern when the illumination area is expanded in a stepwise manner, and FIG. 15(B) is a diagram showing an example of an LED lighting pattern when the illumination area is reduced in a stepwise manner. [Figure 16] FIG. 16 is a flowchart showing an example of a procedure for expanding or reducing the illumination area. [Figure 17] FIG. 17 is a diagram showing the illumination area in the eye box after the illumination area is expanded, and another example of the configuration of the control unit that expands or reduces the illumination area. [Figure 18] FIG. 18 is a diagram showing an example of the configuration of a vehicle display system. [Figure 19] FIG. 19 is a diagram illustrating an example of the configuration of a display control device. [Figure 20] FIG. 20 is a diagram showing an example of the expansion and contraction of the illumination area when a vehicle passes through a tunnel. [Figure 21] FIG. 21 is a flowchart showing another example of the procedure for expanding or reducing the illumination area. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0077] (First embodiment) First, measures taken when viewpoint loss occurs in the first embodiment will be described with reference to Figures 1 to 9. Next, measures taken in a state where viewpoint loss is likely to occur in the second embodiment will be described with reference to Figures 10 to 21. The best mode described below is used to easily understand the present invention. Therefore, those skilled in the art should be aware that the present invention is not unduly limited by the mode described below.

[0078] Please refer to Fig. 1. Fig. 1(A) is a diagram showing an overview of viewpoint tracking spotlighting control and an example of a main configuration of a HUD device equipped with a function for expanding the lighting area when viewpoint loss occurs, and Fig. 1(B) is a diagram showing a state in which viewpoint loss occurs.

[0079] In the HUD device 1 shown in FIG. 1(A), the eye box EB is divided into four partial regions Z10 to Z13, and it is possible to detect the position of the driver's (user's) eyes (viewpoint) using these partial regions as units.

[0080] 1(A) (and (B)), the arrangement direction of partial regions Z10 to Z13 (left and right direction on the paper) is the X direction (vehicle width direction) in real space as seen by the driver driving the vehicle, the direction along the perpendicular line to the paper is the Y direction (vehicle height direction), and the Z direction is the direction along the vehicle's traveling direction (front-rear or depth direction). Note that the real space outside the vehicle as seen by the driver is the space below the windshield 2 shown in FIGS. 1(A) and (B).

[0081] The HUD device 1 includes a control unit 100, a display unit 145, and a reflection optical system (light collecting optical system) 150. The configuration of each unit will be described below in order.

[0082] The control unit 100 has a partial lighting (spot lighting) control unit 110. The partial lighting (spot lighting) control unit 110 has a viewpoint position detection unit 112, a viewpoint loss (lost) detection unit 114, and a light emission control unit 116. In the figure, S1a and S1b indicate viewpoint detection information. Also, S2 indicates viewpoint lost notification information (notification signal) indicating that a viewpoint loss (viewpoint lost) has occurred.

[0083] The camera 90 captures an image of the face, eyes, etc. of a person (driver). Information S0 obtained by capturing the image is supplied to the viewpoint position detection unit 112.

[0084] Furthermore, ECU (electronic control unit) 160 centrally manages the on-board electronics. This ECU 160 collects vehicle information, information about the surroundings of the vehicle (ambient information), and information such as the fatigue level of the driver and other passengers (biological information). Thus, ECU 160 functions as a collector of various types of information. The various types of information collected by ECU 160 are provided to light emission control unit 116. In FIG. 1(A), the various types of information provided are indicated by the symbol S3.

[0085] In addition, the light emission control unit 116 can adjust, for example, adaptively changing the contents of the process of expanding the illumination area when the viewpoint is lost, and the process of reducing the illumination area after the viewpoint is re-detected, based on various information S3 supplied from the ECU 160 (this will be described later).

[0086] The viewpoint position detection unit 112 detects the viewpoint position based on imaging information (imaging signal) S0 from the camera 90 that captures an image of a person's face or pupils. The viewpoint position detection unit 120 can be provided separately from the HUD device 1.

[0087] The viewpoint loss (viewpoint lost) detection unit 114 detects the occurrence of viewpoint loss. The light emission control unit 116 generates and outputs a light emission control signal S4 based on the input signals S1a, S2, S3, etc., and thereby controls the light emission state (lighting / non-lighting, brightness when emitting light, etc.) of the display pixels in the display unit 145. The viewpoint loss (viewpoint lost) detection unit 114 can also be provided separately from the HUD device 1.

[0088] The display unit (display device) 145 has a light source (backlight) 120, an optical system (first optical system) 130 of the display unit, and a liquid crystal panel 140 that displays an image. Note that the liquid crystal panel is only an example, and other non-self-luminous display means using non-self-luminous elements may be used. Furthermore, self-luminous display means using LED elements, organic EL elements, etc. as pixels may be used.

[0089] The light source 120 has a plurality of light source elements (here, LED elements) 120a to 122d mounted (arranged) on the mounting surface of the substrate 121 in a planar shape (matrix shape).

[0090] Although not shown, the optical system (first optical system) 130 of the display unit is configured by combining a plurality of lenses. For example, a first field lens having a curved (e.g., arc-shaped) cross-sectional shape on the light-incident side and a flat cross-sectional shape on the light-exiting side, and a second field lens having a flat cross-sectional shape on the light-incident side and a curved (e.g., arc-shaped) cross-sectional shape on the light-exiting side are arranged in close proximity to each other. The first and second field lenses appropriately refract the light emitted from the light source elements 122a to 122d, and the liquid crystal panel 140 is illuminated from the back by the light. By controlling the light transmittance of each pixel of the liquid crystal panel 140, for example, the twist angle of the liquid crystal element, it is possible to control the light emission / non-emission of each pixel, or the intensity of the light emitted from each pixel (in other words, the light emission intensity of each pixel).

[0091] The reflective optical system (light-collecting optical system) 150 functions as a second optical system. In Fig. 1(A), the reflective optical system (light-collecting optical system) 150 is depicted in a simplified manner, but specifically, it is composed of, for example, a reflecting mirror or a concave mirror (curved mirror: reference numeral 171 in Fig. 4(B)).

[0092] The light emitted from the display unit 145 travels while being reflected by the reflecting mirrors of the reflecting optical system (light-collecting optical system) 150, and the light (display light: symbol K in Figure 3) is reflected by the vehicle windshield 2 (see Figure 3) and projected onto the eye box EB (viewpoints A1, A2).

[0093] In FIG. 1A, a viewpoint A1 (right eye) is located at a partial region Z10 of the eyebox EB, and a viewpoint A2 (left eye) is located at a partial region Z12 of the eyebox EB.

[0094] In FIG. 1A, the light source elements 122b and 122d are turned on and in a light-emitting state, and the light source elements 122a and 122c are turned off and in a non-light-emitting state.

[0095] A light ray L1 (indicated by a dashed line) generated by the light emitted by the light source element 122b is finally projected onto the left eye A2. A light ray R1 (indicated by a solid line) generated by the light emitted by the light source element 122d is finally projected onto the right eye A1. For example, when the pixels corresponding to the light source elements 122a and 122d on the display surface of the liquid crystal panel 140 form an image having parallax for the right eye / left eye, the parallax image is projected onto each of the driver's eyes A1 and A2. This allows the driver to view a stereoscopic virtual image.

[0096] Next, FIG. 1(B) will be referred to. In FIG. 1(B), viewpoints A1 and A2 are located in partial regions Z11 and Z13. Correspondingly, light source elements 122a and 122c are turned on and in a light-emitting state, and light source elements 122b and 122d are turned off and in a non-light-emitting state. A light ray L2 (shown by a solid line) generated by light emission from light source element 122a is ultimately projected onto the left eye A2. Also, a light ray R2 (shown by a dashed line) generated by light emission from light source element 122c is ultimately projected onto the right eye A1.

[0097] In this state, it is assumed that the viewpoints A1 and A2 move in the X direction (to the right on the paper), and the viewpoints cannot be detected, resulting in a viewpoint loss. Note that when the viewpoint is lost, the viewpoint A1' is located in a partial region Z12 of the eyebox EB, and the viewpoint A2' is located outside the eyebox EB.

[0098] In this state, the light rays R2 and L2 are not projected onto the viewpoints A1' and A2', and the driver cannot see the display (virtual image) by the HUD device 1. However, since the viewpoint A1' is within the eye box EB, it is possible to change the illumination range to irradiate light onto the viewpoint A1' and make it visible.

[0099] Next, reference will be made to Fig. 2. Fig. 2(A) is a diagram showing a state in which the illumination area is expanded after the viewpoint is lost, and Fig. 2(B) is a diagram showing a preferable example of the light amount distribution of the illumination light in the illumination area.

[0100] The position of the viewpoints (viewpoints A1', A2') in FIG. 2(A) is the same as that in FIG. 1(B), and this is a viewpoint lost state. Here, the viewpoint loss detection unit 114 detects that the viewpoint has been lost, and sends a viewpoint lost detection signal (detection information) S2 to the light emission control unit 116. In response to this, the light emission control unit 116 performs an expansion process of the illumination area as a measure against the viewpoint loss. Note that the expansion of the illumination area can be realized, for example, by expanding the area of ​​pixels or light sources that contribute to the display of a virtual image on the display unit 145 (in other words, by controlling to increase the number of pixels or light source elements in an illuminated state), and controlling to illuminate an illumination area wider than the partial illumination area of ​​the eye box that was illuminated immediately before the viewpoint was lost, based on that area.

[0101] In Fig. 2(A), the light source elements 122b and 122d that were turned off in Fig. 1(B) are turned on, and all of the light source elements 122a to 122d are turned on (emitting light). In Fig. 1(B), illumination was provided only by light rays L2 and R2, but in Fig. 2(A), illumination is also provided by light rays L1 and R1, and the illumination range (illumination area) in the eye box EB is expanded (enlarged).

[0102] Focusing on the eye box EB, the light ray L1 is projected to the viewpoint A1' (located in the partial area Z12 of the eye box) after the viewpoint is lost. This prevents the driver from being completely unable to see the display (virtual image) by the HUD device 1. In other words, the situation in which the display suddenly disappears completely due to the occurrence of viewpoint loss is suppressed (reduced or prevented).

[0103] Next, reference is made to Fig. 2(B). Fig. 2(B) shows a preferred example of the distribution of the amount of light (luminance) in the illumination area (including the partial illumination area). When the viewpoint moves out of the partial illumination area (the spotlighting area), it can be expected that the luminance becomes extremely small, and the visibility of the image (virtual image) drops extremely and suddenly.

[0104] In this case, it is desirable to reduce the discomfort felt by the viewer when the virtual image suddenly disappears (the brightness drops drastically) when the eye (point of view) approaches the boundary (edge) between the partially illuminated area and the non-illuminated area.

[0105] Therefore, in one preferred example, if there is a rectangular illumination area, the amount of light is high in the center and low in the periphery (near the boundary), but the amount of light (brightness) is low enough to ensure a certain amount (above a specified value), and the virtual image is prevented from suddenly disappearing when the viewpoint moves outside the above-mentioned boundary. In this case, when the viewpoint moves outside the boundary, a faint virtual image is seen and then the image disappears, which makes it difficult to feel uncomfortable.

[0106] In another preferred example, the illumination area (lighting area) is illuminated with uniform (including approximately uniform) light so as to achieve a predetermined luminance level. In this case, the amount of light also decreases from the edge (boundary) vicinity, but a certain amount of light is ensured, so that when the viewpoint moves outside the boundary, a faint virtual image appears and then disappears, making it difficult to feel uncomfortable.

[0107] The light quantity distributions DB1 and DB2 shown in Fig. 2(B) correspond to the other preferable example. DB1 indicates the light quantity distribution of a partial illumination area corresponding to the partial area Z13 of the eyebox EB, and DB2 indicates the light quantity distribution in the illumination area (entire illumination area) when the illumination area is expanded to the entire area of ​​the eyebox EB (partial areas Z10 to Z13). In both distributions, a light quantity equal to or greater than a predetermined threshold value Lth is ensured at the inner boundary of the eyebox EB or near the boundary of the outer shape of the eyebox EB itself (in other words, the periphery or edge of the illumination area).

[0108] This allows the amount of light to extend beyond the boundary, and the light intensity does not decrease abruptly, but gradually (with a gradient). Therefore, even when the viewpoint moves outside the boundary, the display (virtual image) fades and disappears, not suddenly. This suppresses (reduces) the sense of discomfort and anxiety.

[0109] In this way, according to the example of FIG. 2(B), the sense of incongruity caused by the virtual image suddenly disappearing when the viewpoint moves outside the illumination area is reduced.

[0110] Next, reference will be made to Fig. 3. Fig. 3(A) and (B) are diagrams showing an example of the configuration of an eyebox used in a HUD device.

[0111] In FIG. 3(A), the eyebox EB is divided into a plurality of (here, nine) partial regions Z1 to Z9, and the position of the driver's viewpoint A is detected for each of the partial regions Z1 to Z9.

[0112] Display light K of an image is emitted from the light collecting optical system 150 (see Figs. 1 and 2) of the HUD device 1, a part of which is reflected by the windshield 2 and enters the driver's viewpoint (eye) A. When viewpoint A is within the eye box, the driver can visually recognize a virtual image of the image.

[0113] 3B, display light K of an image is emitted from the light collecting optical system 150 of the HUD device 1, a part of which is reflected by the windshield 2 and enters the left and right viewpoints (left and right eyes) A1, A2 of the driver. When the viewpoints A1, A2 are within the eye box EB, the driver can visually recognize a virtual image of the image.

[0114] Here, the width direction (left-right direction) of the vehicle 1 as viewed from the driver's viewpoints (eyes) A1 and A2 is defined as the X direction (or lateral direction), the height direction perpendicular to the X direction and along the perpendicular line to the road surface is defined as the Y direction (vertical direction), and the forward direction (front-rear direction) is defined as the Z direction. In FIG. 3B, the inside of the eye box EB is provided with eight partial regions E1 to E8 partitioned by vertical boundaries, and each partial region E1 to E8, which has a vertically long rectangular shape, is arranged (arranged) adjacent to each other in the lateral direction (X direction). By detecting the viewpoint position using each partial region E1 to E8 as a unit (unit of minimum resolution), the lateral positions of the viewpoints A1 and A2 in the eye box EB can be detected.

[0115] The information on the viewpoint position detected using each of the partial areas Z1 to Z9 or E1 to E8 can be used for control other than the viewpoint-tracking spotlighting. Also, the "partial area in the eye box EB" and the "partial illumination area" in spotlighting are different concepts, and the sizes of the two do not necessarily match.

[0116] Next, reference is made to Fig. 4. Fig. 4(A) is a diagram showing a configuration example of a stereoscopic display (3D) HUD device to which the present invention is applied, and Fig. 4(B) is a diagram for explaining the principle of displaying a stereoscopic image (virtual image). In Fig. 4, the same reference numerals are used to designate the same parts as in the previous drawings.

[0117] 4(A), an image (parallax image) QR for the right eye and an image (parallax image) QL for the left eye are displayed on a display surface 145 of a liquid crystal panel (broadly speaking, a flat panel display) 140. Note that a "parallax image" is an image that reproduces the parallax (the difference between the images perceived by each eye) that occurs when the left and right eyes are in different positions.

[0118] When the light emission side of the liquid crystal panel 140 is defined as the front, an optical member 160 is disposed in front of the liquid crystal panel 140. This optical member 160 functions as a light beam separating member, and as shown in the upper left, specifically, can be configured by a lenticular lens 170 or a parallax barrier (parallax barrier) 180 having slits SL. The parallax barrier 180 has a plurality of slits 185a to 185n. However, these are merely examples and the present invention is not limited to these.

[0119] If the light (reproduced light for each eye that reproduces an image) L10 and R10 separated by the optical member 160 having a light beam separation function enters both eyes A1 and A2 located at the image forming point of the light, a person sees an apparent three-dimensional image IM at the point where convergence (intersection of light) occurs. In other words, this can be considered as the three-dimensional image IM being generated by a 3D display. In the example of FIG. 4(A), the convergence angle is θc.

[0120] As described with reference to FIG. 1 etc., behind the liquid crystal panel 140, there are arranged the control unit 100, the light source (light source for backlight such as LED) 120, and the light source optical system (first optical system) 130.

[0121] As shown in Fig. 4(A), the entire display surface 145 of the liquid crystal panel 140 is not illuminated, but only the necessary areas ZA and ZB are selectively illuminated. Note that reference numerals 147 and 147' indicate pixels corresponding to (the centers of) the viewpoints A1 and A2. This selective illumination realizes partial illumination (spot lighting) on ​​the eye box EB.

[0122] Please refer to Fig. 4(B). In Fig. 4(B), an eye box EB is set in front of the viewer (such as the driver of a vehicle), and eye point P(C) is located at the center of the eye box EB. If virtual image formation planes PS(L) and PS(R) corresponding to the left and right eyes are set in front of the windshield 2, a virtual image V(C) is located at the center of the overlapping area. The convergence angle of the virtual image V(C) is θd, and the virtual image V(C) is recognized as a three-dimensional image by the viewer (user).

[0123] This stereoscopic virtual image V(C) is displayed (formed) as follows: Reproduction light L10, R10 for each of the left and right eyes of the virtual stereoscopic image IM generated by the 3D display shown in Fig. 3(A) is reflected (reflected at least once) by a curved mirror (concave mirror or the like) 171 included in the light collecting optical system 150 of the HUD device 1, and is thereby projected (projected) onto the windshield 2 as display light K, and the reflected light reaches both eyes of the viewer and forms an image in front of the windshield 2, thereby displaying (forming) the virtual image V(C).

[0124] Next, reference is made to Fig. 5. Fig. 5(A) is a diagram showing the transition (change) of the viewpoint detection state over time, and Fig. 5(B) to (F) are diagrams showing the expansion and contraction of the illumination area in the eyebox in response to the movement of the viewpoint.

[0125] In Fig. 5(A), the period from time t0 to t2 is a "viewpoint detection state" in which the viewpoint is detected, and at time t2, the position of the viewpoint cannot be detected and a viewpoint loss (viewpoint lost) occurs. This viewpoint lost state continues until time t5. At time t5, the viewpoint is detected again, and thereafter, the state becomes a viewpoint detection state.

[0126] Incidentally, as the "viewpoint loss", it is possible to imagine both a viewpoint loss in which at least one of both eyes is inside the eyebox, and a viewpoint loss in which both eyes are outside the eyebox.

[0127] In addition, the control to expand or reduce the lighting area when the viewpoint is lost, as described below, can be applied to both single-eye HUDs that project the same image to each of the driver's eyes, and binocular (parallax, hologram, or light reproduction) HUDs that project different images (parallax images) to each eye.

[0128] 5(B), (C), (D), (E), and (F) show the illumination areas in the eye box EB at times t1, t3, t4, t6, and t7, respectively. The illumination areas are shown as rectangular (including square and rectangular) areas Q1 to Q5 with sand patterns. Also, the viewpoints A1 and A2 in the viewpoint detection state are shown by solid lines, and the viewpoints A1 and A2 in the viewpoint lost state are shown by dashed lines.

[0129] In FIG. 5(B) (corresponding to time t1), viewpoints A1 and A2 are located near the upper right diagonal on the eyebox EB (here, the eyebox is defined as a rectangular area in a planar view seen from the front).

[0130] By performing spotlighting control, the display light of the image is irradiated (projected) onto a rectangular partial illumination area Q1 of a predetermined size, centered on the viewpoints A1 and A2 whose positions have been detected. If the viewpoint moves, the viewpoint position is detected as needed, and the partial illumination area Q1 moves to follow the viewpoint while maintaining its size. This state is the normal spotlighting state.

[0131] The above-mentioned "predetermined size" here refers to the illumination size when the viewpoint position has been detected and the partial illumination area is not in contact with the eye box EB (in other words, in the normal state of spot lighting).

[0132] In Fig. 5(C) (corresponding to time t3), viewpoint loss occurs, and the HUD device 1 is unable to detect the viewpoint position. In reality, the viewpoint moves diagonally downward to the right on the eye box EB. In Fig. 5(C), the size of the partial illumination area Q2 is expanded (enlarged) compared to the size of the partial illumination area Q1 in Fig. 5(A) (in other words, the size of the partial illumination area immediately before viewpoint loss). It is expanded almost uniformly (isotropic expansion) in the directions perpendicular to each of the four sides that determine the outline of the rectangular illumination area.

[0133] In Fig. 5(C), the viewpoint A1 is outside the partial illumination area Q2, but the viewpoint A2 is located inside Q2, and an image can be projected onto the viewpoint A2. This prevents the driver from being unable to see the display (virtual image) at all when the viewpoint is lost.

[0134] Here, the control for expanding such a partial illumination region is performed by the control unit 100 described above (specifically, the partial illumination control unit 110: see FIG. 1(A)).

[0135] When the display unit 140 that displays the image forms pixels with self-luminous elements such as LEDs or organic EL, the illumination area can be expanded by controlling the light emission state of the pixels, and when the display unit 140 is a light-transmitting display device that combines a liquid crystal panel and a backlight (light source) as in the examples of Figures 1 and 2, the illumination area can be expanded by controlling the light emission state (on, off, etc.) of the light source (light source element). In other words, for example, by changing the pixels (non-emitting pixels) surrounding the pixel that was emitting light just before the viewpoint was lost to an emitting state (a state that can contribute to the visibility of a virtual image), it is possible to expand the illumination area around the periphery based on the illumination area just before the viewpoint was lost.

[0136] Here, it is preferable to "expand based on the illumination area immediately before the viewpoint is lost". If the viewpoint is lost for a short period of time, the viewpoint is likely to remain near the original illumination area, and expanding to the periphery of the original area increases the possibility of projecting light onto the viewpoint. In the example of Fig. 5(C), the illumination area is expanded around the partial illumination area Q1 in Fig. 5(B) at time t1 immediately before the viewpoint is lost, which realizes a larger partial illumination area Q2 and makes it possible to project light onto the viewpoint A2 as described above.

[0137] Also, the partial illumination area Q2 can be expanded by projecting light onto the entire area that protrudes (expands) from the partial illumination area Q1 all at once. However, when the partial illumination area in the eye box EB is expanded, a part of the light beam heading toward the expanded illumination area may also enter the original illumination area. As described above, the entire eye box may be illuminated all at once, but in this case, it is assumed that the brightness of the image (virtual image) viewed in the original illumination area will increase suddenly, and it cannot be said that there is no possibility that the driver (user) will feel uncomfortable due to the change in brightness.

[0138] Therefore, when expanding the illumination area, the control unit 100 may perform at least one of continuous expansion control and stepwise expansion control, and gradually expand the illumination area over time. In this case, it is possible to suppress the sense of incongruity that may occur when the illumination area is expanded to cover the entire eyebox in one go. In other words, it is possible to perform "spotlighting expansion control" that produces less sense of incongruity.

[0139] If the illumination area is expanded continuously on the time axis, the sense of incongruity can be further reduced. In addition, if the illumination area is expanded stepwise (in steps), efficient expansion can be performed easily (with less strain on the device) with regularity according to a specified expansion algorithm. Note that by combining continuous expansion and stepwise expansion, it is possible to expand the illumination area with a higher degree of freedom.

[0140] In this way, even if the viewpoint (at least one of the left and right eyes) moves and the viewpoint is temporarily lost, if the viewpoint is located in the expanded illumination area, the driver (user) can view the display (virtual image), and the possibility of preventing the display from suddenly disappearing is increased. This provides the effect of reducing the occurrence of discomfort. In addition, the possibility of viewing the display necessary for the operation of the vehicle is increased, thereby improving safety reliability.

[0141] Also, when moving from FIG. 5(B) to FIG. 5(C), for example, the light source element may be made to emit light up to a predetermined light intensity all at once by binary control of turning the light source off / on. However, it is expected that a large change in luminance may cause discomfort. Therefore, in the examples of FIG. 5(B) and (C), the light amount is gradually increased. A hollow arrow with an expanded tip is shown between FIG. 5(B) and (C), which indicates that the light amount is gradually increased (this also applies to the arrow drawn between FIG. 5(C) and (D)).

[0142] Specifically, the pixels (display pixels) of the display unit 140 are not controlled to be switched on and off at a desired luminance (binary control of on / off), but are controlled to be raised to a desired luminance over time (in other words, multi-value lighting control). This can further reduce the sense of discomfort.

[0143] Next, Fig. 5(D) will be described. In Fig. 5(D), at time t4, the illumination area is expanded to the entire area on the eye box EB (in the figure, the entire illumination area is denoted by the symbol Q3).

[0144] In addition, in Figure 5 (D), the viewpoints A1 and A2 have moved further diagonally downward to the right from Figure 5 (C), but by expanding the lighting area to the entire range, images are projected onto both viewpoints A1 and A2, and the driver can see, for example, a three-dimensional image (virtual image).

[0145] Here, referring to Fig. 5(A), time t4 is a time point after the first predetermined time Tth1 has elapsed from the time t2 when the viewpoint loss occurs. In other words, in Fig. 5(D), when the viewpoint loss continues for the first predetermined time Tth1 or more, control is performed to illuminate the entire eye box EB.

[0146] Here, as described above, at the stage of Figure 5 (C), the illumination range may be expanded (enlarged) to the entire area of ​​the eye box EB in one go. However, in this case, it is expected that the brightness of the image (virtual image) viewed in the original illumination area will increase suddenly, and it cannot be said that there is no possibility that the driver (user) will feel uncomfortable due to the change in brightness.

[0147] Therefore, after Fig. 5(B), the state of Fig. 5(C) is entered, and if the gaze point is still not detected again, the state of Fig. 5(D) (a state in which the illumination range is expanded to the entire eye box) is entered. In other words, the control of expanding the illumination range to the entire area is performed on the condition that the first predetermined time Tth1 has elapsed. This allows the size of the partial illumination area to be gradually expanded (for example, Fig. 5(C)) during the period before the first predetermined time Tth1 has elapsed. Therefore, the sense of discomfort can be suppressed (reduced).

[0148] An example of the expansion control of the partial illumination region has been described above, but specific examples and modified examples of the expansion will be described later. In addition, the expansion control may be performed by estimating (or predicting) the moving direction of the viewpoint, which will also be described later.

[0149] Next, reference is made to Figures 5(E) and (F). Figure 5(E) shows the state at time t6. Since the viewpoint is redetected at time t5, the illumination area is reduced to a partial illumination area Q4 so as to correspond to the redetected viewpoint position. However, the size of the partial illumination area Q4 is larger than the partial illumination area Q1 in Figure 5(B) and has not returned to its original size completely. In the next step 5(F), it is reduced to its original size. Here, Figure 5(F) shows the state at time t7.

[0150] When the viewpoint is redetected, the lighting changes to partial lighting based on the redetected viewpoint position (in principle, partial lighting of the same size as the partial lighting area before the viewpoint was lost). This allows the lighting to return to the same (normal) spot lighting state as before the viewpoint was lost without any sense of incongruity when the viewpoint is recovered from being lost. At the stage in Figure 5(E), the lighting may be reduced to the same size as the original partial lighting size in one go.

[0151] However, there may be cases where the driver's (user's) viewpoint repeats a certain amount of movement and then converges to a certain position.

[0152] Therefore, in the examples of Fig. 5 (E) and (F), the normal spotlighting state is restored at time t7, which is the time after the second predetermined time Tth2 has elapsed from time t5 when the viewpoint is redetected (the time when the time required for the viewpoint to stabilize has elapsed). This makes it possible to suppress (reduce) the viewpoint from moving out of the illumination area due to a small movement of the viewpoint on the way back. In addition, during the period before the second predetermined time Tth2 has elapsed, the illumination area can be gradually reduced (for example, Fig. 5 (E)), and the gradual reduction in the illumination area suppresses (reduces) the discomfort felt by the driver.

[0153] In addition, when transitioning from Fig. 5(D) to (E) and from Fig. 5(E) to (F), the amount of illumination light (illumination brightness) is gradually reduced, thereby suppressing (reducing) the discomfort that may occur when transitioning to normal spot lighting all at once. Note that tapered white arrows are shown between Fig. 5(D) and (E) and between Fig. 5(E) and (F), which indicate that the amount of light is gradually reduced. Other examples of reducing the illumination area will be described later.

[0154] Next, FIG. 6 will be referred to. FIG. 6(A)-(H) are diagrams showing an example of the expansion and contraction of the illumination area. As shown in the figure, the eye box EB has a predetermined shape (here, a square or a rectangle). However, the present invention is not limited to this. More precisely, the "eye box" is a predetermined area (the shape of the predetermined area includes, for example, a cube, a rectangular parallelepiped, and an ellipsoid) that is defined in the left-right direction (X-axis direction), the up-down direction (Y-axis direction), and the depth direction (Z-axis direction) inside the vehicle, and is set to be the same as the area (also called an iris) where the position of the observer's viewpoint is expected to be located in the vehicle in which the HUD device 20 is mounted, or to include most of the iris (for example, 80% or more). In the drawings used to explain this embodiment, the concept of the depth direction is omitted, and the eye box is shown as a rectangle or the like defined in the left-right direction and the up-down direction. The shape of the eye box defined in the left-right direction and the up-down direction can be set to a square or a rectangle (including a rectangle and a square), as well as a polygon, an ellipse, or the like.

[0155] First, the expansion pattern in Fig. 6 will be described. When expanding the illumination area, for example, continuously or stepwise, the control unit 100 (or 110) can implement any of isotropic expansion, which expands the illumination area uniformly around its periphery, anisotropic expansion, which expands the illumination area non-uniformly, and a combination of these. Taking an illumination area EB having a square or rectangular outer shape as an example, any of "isotropic expansion" which expands the illumination area along the perpendicular lines to each of the four sides, "anisotropic expansion" which expands the illumination area along the perpendicular lines to each of one side, two sides, or three sides, and "expansion by a combination of isotropic expansion and anisotropic expansion" can be implemented. Also, when any side of the outer shape of the illumination area reaches the corresponding side of the outer shape of the eye box EB, expansion is implemented for the other unreached sides, and control can be implemented to end the expansion when the viewpoint position is redetected during the expansion or when the illumination area expands to the entire area of ​​the eye box EB.

[0156] FIG. 6 shows three expansion patterns (specific examples that differ from FIG. 5 in at least some parts). One expansion pattern is an example of expansion to FIG. 6(A), (B), (E), and (H). This example is a pattern in which FIG. 6(E) is inserted into the previously shown FIG. 5(B), (C), and (D) (which correspond to FIG. 6(A), (B), and (H)).

[0157] The second expansion pattern is a pattern in which the data is expanded as shown in Fig. 6(A), (C), (F), and (H). The third expansion pattern is a pattern in which the data is expanded as shown in Fig. 6(A), (D), (G), and (H). Efficient expansion is achieved by using isotropic and anisotropic expansion.

[0158] In the example of FIG. 6, the rules for expansion are clarified, and effects such as easier control by software are obtained.

[0159] Next, a description will be given of a reduction pattern. When reducing the illumination area, for example, when the viewpoint position is redetected after the viewpoint is lost, the control unit 100 or 110 performs control to reduce the wide illumination area expanded after the viewpoint is lost to a narrow illumination area of ​​the same size as the partial illumination area before the viewpoint is lost, corresponding to the redetected viewpoint position.

[0160] At this time, any of isotropic shrinkage, which shrinks the illumination area uniformly, anisotropic shrinkage, which shrinks the illumination area non-uniformly, and a combination of these can be performed. Specifically, for example, a first shrinkage pattern (first method) can be envisioned in which a combination of "isotropic shrinkage," which shrinks the illumination area along the perpendicular lines to each of the four sides of an illumination area having a square or rectangular outer shape, and "anisotropic shrinkage," which shrinks the illumination area along the perpendicular lines to each of one side, two sides, or three sides, is performed to restore the illumination area to a narrow illumination area of ​​the same size as the partial illumination area before the viewpoint was lost.

[0161] In addition, when the contraction along one of two orthogonal line segments is defined as vertical contraction and the contraction along the other is defined as horizontal contraction, the original lighting size can be restored by appropriately implementing these (including the case of implementing them in combination). In addition, as the "two orthogonal line segments", if the plane viewed ignoring the depth of the eye box can be grasped as a rectangle (including a square or a rectangle) erected perpendicular to the road surface, the line segment in the height direction of the vehicle and the line segment in the width direction of the vehicle can be adopted. Specifically, for example, when the contraction along the vertical side of a square or a rectangle is defined as vertical contraction and the contraction along the horizontal side is defined as horizontal contraction, two reduction patterns (second method) can be assumed in which vertical contraction is implemented and then horizontal contraction is implemented to perform control to restore the partial lighting area to the same size as the partial lighting area before the viewpoint was lost.

[0162] In addition, a third shrinkage pattern (third method) can be envisioned in which horizontal shrinkage is performed, followed by vertical shrinkage, to control the return to a narrower illumination area of ​​the same size as the partial illumination area before the viewpoint was lost.

[0163] According to these methods, the rules for reduction are clarified, and the effects of facilitating control by software can be obtained.

[0164] In Fig. 6, three reduction patterns are obtained by tracing the three expansion patterns described above in reverse. Note that the reduction patterns shown in Fig. 6 are for the second and third methods described above (the reduction pattern for the first method will be described with reference to Fig. 7).

[0165] In the example of Figure 6, starting from Figure 6(H), any one of the following can be implemented: a first reduction pattern that reduces to Figures 6(E), (B), and (A); a second reduction pattern that reduces to Figures 6(F), (C), and (A); or a third reduction pattern that reduces to Figures 6(G), (D), and (A).

[0166] Next, reference will be made to Fig. 7. Fig. 7(A) to (D) are diagrams showing an example of contraction and expansion of the illumination area.

[0167] In FIG. 7, two reduction (or expansion) patterns are shown. The reduction patterns in FIG. 7(A), (B), and (D) show cases where isotropic reduction is performed after anisotropic reduction. The reduction patterns in FIG. 7(A), (C), and (D) show cases where anisotropic reduction is performed after isotropic reduction. These correspond to the first reduction pattern described above. Moreover, by tracing each reduction pattern in reverse, two expansion patterns can be obtained. Note that these patterns are merely examples and are not limited to these.

[0168] Next, let us refer to Fig. 8. Fig. 8(A) and (B) are diagrams showing another example of change in the viewpoint position (an example considering a return from a viewpoint lost state to a viewpoint redetection state), and an extension of the illumination area by estimating or predicting the viewpoint movement, etc. In Fig. 8, the eyebox shown in Fig. 3(B) is used.

[0169] In Fig. 8(A), the viewpoints A1 and A2 are initially located in partial regions W7 and W8 of the eyebox EB (viewpoint detection state). After that, the viewpoints move significantly to the right (positive X direction), and one of the viewpoints (viewpoint A2) moves out of the eyebox EB (viewpoint movement (1)). Next, the viewpoints A1 and A2 move to the left (negative X direction), and the viewpoint A2 that was out of the eyebox returns to the eyebox EB, but the return is somewhat small and does not return to its original position.

[0170] Additionally, the viewpoints A1 and A2 move slightly to the lower right (viewpoint movement (3)), and then return to the left of their original positions (viewpoint movement (4)).

[0171] Such various viewpoint movements are possible. Even in this case, if the control to expand the illumination area is performed when the viewpoint is lost, the driver is more likely to be able to see the display (virtual image) by the HUD device 1, for example, while the viewpoint is returning to the original position, which is also beneficial in this respect.

[0172] In addition, when expanding the illumination range, it is also possible to estimate or predict the direction of movement of the viewpoint, and perform control to expand the range more in the estimated (predicted) direction (including expanding only in that direction).

[0173] In other words, when expanding the lighting area continuously or stepwise, the control unit 100 (or 110) may estimate or predict the direction in which the viewpoint was lost based on at least one of the direction of movement of the viewpoint before the viewpoint was lost, past learning results regarding the movement of the viewpoint, and the driving conditions at the time the viewpoint was lost, and perform control to expand the lighting area to include at least the estimated or predicted direction.

[0174] In other words, the direction in which the viewpoint is lost (the direction in which the viewpoint moves when the viewpoint is lost) may be estimated or predicted, and the illumination area may be expanded by utilizing the contents of the estimation (prediction).

[0175] Even when the gaze point is lost, it may be possible to estimate (or predict) the direction of gaze point movement. For example, when only one of the eyes is lost, the direction of gaze point loss can be estimated (predicted) by tracking the movement trajectory of the other eye (estimation based on gaze point movement direction).

[0176] In addition, drivers (users) have individual characteristics in their gestures and actions while driving. For example, if it is known from past learning that a certain driver frequently changes his / her posture while driving, which causes the viewpoint to change up and down (in the direction of the vehicle's height), then it is possible to utilize the results of this learning to estimate (predict) that the viewpoint has moved in the vertical direction, causing viewpoint loss (estimation or prediction through learning).

[0177] In addition, if the vehicle is currently traveling on a bumpy road (such as an unpaved road) (this can be determined, for example, from vehicle information collected by the ECU, etc.), it can be estimated (predicted) that the viewpoint may have momentarily shifted upward, downward, or diagonally, causing the vehicle to become lost (estimate or prediction based on the driving situation).

[0178] By performing such estimation (prediction) and expanding to include the estimated (predicted) direction, efficient expansion is possible. For example, by expanding the estimated (predicted) direction more thoroughly (increasing the degree of expansion) than the non-estimated (non-predicted) direction, the possibility of projecting an image onto the viewpoint during a lost state increases, and the possibility that the expansion of the illumination area will efficiently lead to increased visibility of the virtual image increases. In addition, since it is possible to focus on illuminating the necessary areas, it is also beneficial in terms of energy conservation.

[0179] In the example of FIG. 8(A), initially, the range F1 is illuminated, but after the above estimation (prediction) is performed, the illumination area is greatly expanded (expanded to range F2) to the right (positive X direction), which is the estimated (predicted) direction, while the illumination area is slightly expanded (expanded to range F3) to the left (negative X direction).

[0180] In FIG. 8(A), by enlarging (extending) the illumination area in this manner, it is possible for the driver to view (or have an increased opportunity to view) the display (virtual image) in any of the above-mentioned viewpoint movements (1), (2) or (3), (4).

[0181] Next, let us refer to Fig. 8(B). In Fig. 8(B), the viewpoints A1 and A2 move significantly to the right (positive X direction) and go outside the eye box EB (viewpoint movement (5)). After that, the viewpoints continue to move outside the eye box EB (viewpoint movement (6)). Then, they move significantly to the left (negative X direction), passing far past their original positions and moving to a position significantly to the left of their original positions (viewpoint movement (7)).

[0182] In the example of Fig. 8(B), the same estimation (prediction) of the viewpoint movement as in Fig. 8(A) is performed, and the illumination range is expanded from F1 to an area including F2 and F3. However, since the viewpoint is not detected again after the expansion, the illumination range is further expanded to an area including F4 (the entire area of ​​the eye box EB).

[0183] In Fig. 8(B), by implementing such an expansion (extension) of the illumination area, it is possible to allow the driver to view the display (virtual image) (or to increase the opportunities for viewing) even in the above-mentioned viewpoint movements (5), (6), and (7). In addition, after focusing on illuminating the required area, the illumination is expanded to cover the entire eye box EB, which is also beneficial in terms of energy saving.

[0184] Although the change control of the illumination area of ​​the present invention has been described above using several examples, the present invention is not limited to these and various modifications and applications are possible.

[0185] For example, when reducing the amount of illumination light when reducing a wide illumination area to a narrow illumination area, the control unit 100 (or 110) may reduce the illumination light by slowly lowering the brightness of pixels or light source elements that have little effect on the brightness viewed at the viewpoint position.

[0186] This control is an example of a control method that assumes that light from multiple pixels (light sources) is directed toward one viewpoint. A part of the light flux directed toward a partial illumination region in the eye box may also be directed toward other partial illumination regions. In other words, a part of the light flux directed toward partial region Z2 may enter partial region Z5 in FIG. 3(A). If the pixels or light sources that mainly direct light flux to partial regions Z1, Z2, Z3, . . . , Z9 are lighting elements C1, C2, C3, . . . , and C9, respectively, partial region Z5 is illuminated with the light flux directed from lighting element C5 and parts of the light flux from lighting elements C2, C4, C6, and C8 that are directed toward partial regions Z2, Z4, Z6, and Z8 adjacent to partial region Z5. Therefore, when reducing a wide illumination area (e.g., partial areas Z2, Z4, Z5, Z6, Z8) to a narrow illumination area (e.g., partial area Z5), if the light amount of the illumination elements C2, C4, C6, and C8 is reduced, the brightness of the image viewed in illumination area Z5 decreases. It is expected that the change (decrease) in the viewed brightness may give the viewer a sense of incongruity.

[0187] Therefore, when changing the lighting range from an expanded range to a reduced range, for example, by gradually (over time) reducing the luminance of light sources (here, C2, C3, etc. excluding lighting elements C1) that have little effect on the luminance of the virtual image viewed at the eye position (subregion X1), it is possible to make it difficult to sense the reduction in luminance of the image due to the light that reached the eye position X1 (light from lighting elements C2, C3, etc.), and thereby reduce the sense of discomfort.

[0188] In some embodiments, when the illumination range is changed from the expanded range to the contracted range, for example, when the luminance of the light source (here, C1, C2, C9 excluding the illumination element C5) that has a small effect on the luminance of the virtual image viewed at the eye position (in the example of FIG. 3, the partial region Z5) is gradually (over time) reduced, the luminance of the light source (for example, the illumination elements C1, C3, C7, C9 corresponding to the partial regions Z1, Z3, Z7, Z9 relatively far from the partial region Z5) that has a small effect on the luminance of the virtual image viewed at the eye position may be reduced faster, and the luminance of the light source (for example, the illumination elements C2, C4, C6, C8 corresponding to the partial regions Z2, Z4, Z6, Z8 relatively close to the partial region Z5) that has a large effect may be reduced slower. Here, reducing the luminance faster includes, for example, making the speed at which the luminance is gradually reduced more rapidly than the other, making the timing of reducing the speed faster than the other, or a combination of these, etc. (not limited to these).

[0189] Furthermore, when reducing a wide illumination area to the narrow illumination area, the control unit 100 (or 110) may slowly reduce the luminance of pixels or light source elements that have a small effect on the luminance viewed at the viewpoint, and slowly increase the luminance of pixels or light source elements that have a large effect on the luminance viewed at the viewpoint. This control is another example of a control method that assumes that light from multiple pixels is directed to one viewpoint.

[0190] In other words, when changing the lighting range from an expanded range to a reduced range, for example, the luminance of light sources (here, C2, C3, etc. excluding lighting element C1) that have little effect on the luminance of the virtual image viewed at the eye position (subregion X1) is gradually (over time) reduced, and the luminance of light sources (here, lighting element C1) that have a large effect on the luminance of the virtual image viewed at the eye position X1 is gradually (over time) increased.

[0191] This allows the brightness of the image (virtual image) to be maintained substantially constant even while the illumination range is being changed. Therefore, for example, the change in brightness (difference in brightness) of the image (virtual image) seen from eye position X1 when the illumination range is expanded and when it is reduced can be reduced. This further reduces the sense of discomfort.

[0192] Furthermore, the control unit 100 (or 110) may variably control the first predetermined time Tth1 shown in FIG. 5(A) based on at least one of the driving conditions of the vehicle, the surrounding environment, and the condition of the driver.

[0193] In other words, the time required to expand the illumination area to cover the entire area (maximum area) of the eyebox can be variably controlled according to various factors, enabling control appropriate for the driving scene.

[0194] For example, when the vehicle is traveling on a bumpy road and the occurrence frequency of vehicle pitching is equal to or higher than a predetermined value, when the driver's fatigue level is determined to be high, when the contrast of the image captured by the camera is equal to or lower than a predetermined value, etc., it is preferable to shorten the first predetermined time Tth1 and increase the expansion speed to the maximum area compared to normal (control to reach the maximum area faster). In these cases, it is desirable from the viewpoint of safety to ensure that the driver (user) can visually recognize the display (virtual image).

[0195] Also, for example, since the sensitivity of the human eye is higher at night than during the day, it is possible to control the expansion slowly over a longer period of time to reduce discomfort. In addition, in cases where light and darkness change suddenly, such as in a tunnel, the speed at which the illumination area expands may be increased so that the maximum area can be reached quickly. This is because it is more difficult to visually recognize the display (virtual image) in rainy weather than in sunny weather, so it is preferable to increase the speed at which the illumination area expands when lost, and give priority to ensuring that the display is visible rather than to the occurrence of discomfort due to changes in display luminance.

[0196] In this way, it is possible to implement optimal control adapted to various situations.

[0197] Furthermore, the control unit 100 (or 110) may variably control the second predetermined time Tth2 shown in FIG. 5(A) based on at least one of the driving conditions of the vehicle, the surrounding environment, and the condition of the driver.

[0198] When the illumination area is reduced, the same effect can be obtained by implementing the same control as in the case of expansion.

[0199] Next, reference is made to FIG. 9, which is a flow chart showing an example of an operation procedure of the HUD device for expanding and reducing the illumination area.

[0200] When viewpoint tracking spotlighting control is being performed (step S1), for example, if it is detected that viewpoint loss for a threshold time or more occurs (step S2), an expansion process for the lighting area is performed (step S3). Note that the reason for detecting "occurrence of viewpoint loss for a threshold time or more" is to take measures for viewpoint loss that continues for a certain period of time, since performing the expansion process every time a very short viewpoint loss occurs may cause discomfort due to a change in brightness.

[0201] In step S3, the illumination area is expanded (the maximum area is the entire area of ​​the eye box). At this time, the expansion may be continuous or stepwise. Also, the illumination area may be expanded to the entire area of ​​the eye box if the viewpoint loss continues for more than a first predetermined time. It is also possible to gradually increase the amount of light. Also, at least one of isotropic expansion, anisotropic expansion (including anisotropic expansion by estimating or predicting the viewpoint loss direction), and a combination of these may be implemented. Also, it is possible to perform processing that is different from the normal processing due to environmental factors (driving scene, day / night, weather, etc.) and biological condition factors (fatigue level of the driver, etc.) (for example, variably controlling the first predetermined time).

[0202] In step S4, it is determined whether the viewpoint (viewpoint position) has been detected again. If N, the process returns to step S3, and if Y, the process proceeds to step S5.

[0203] In step S5, the illumination area is reduced by a combination of isotropic and anisotropic reduction. In addition, when the second predetermined time has elapsed, the illumination area may be restored to its original size. The amount of light may be gradually reduced. In addition, it is possible to perform processing that differs from the normal processing (for example, variably controlling the second predetermined time) depending on environmental factors (driving scene, time of day, weather, etc.) and biological condition factors (fatigue level of the driver, etc.).

[0204] As described above, according to the present invention, when viewpoint loss occurs while implementing viewpoint-tracking spotlighting control in which light is irradiated onto a partial area of ​​the eye box depending on the driver's viewpoint position, it is possible to prevent virtual images from becoming invisible and reduce the sense of discomfort.

[0205] Second Embodiment In the second embodiment, a countermeasure for a situation where viewpoint loss is likely to occur will be described. Please refer to Fig. 10. Fig. 10 is a diagram showing an example of the configuration of a HUD device. In Fig. 10, the same reference numerals are used for the parts common to the previous figures. However, in Fig. 10, the HUD device is indicated by the reference numeral 1'.

[0206] The HUD device 1′ includes a display unit (display device) 145, a folding mirror (reflecting mirror) 169, a curved mirror (concave mirror) 171, and a window unit 143 for emitting light, within a housing 141. Display light is emitted toward the windshield 2 through the window unit 143. The display unit 145 includes a light source (backlight) 120, a light source optical system 130, a diffusion plate 137, and a liquid crystal panel 149. The position of the driver's viewpoint (eye) A is captured by a camera 90 serving as a viewpoint detection unit. A virtual image V is displayed on a virtual image display surface PS in front of the vehicle.

[0207] Next, reference is made to Fig. 11. Fig. 11 is a diagram showing an example of the configuration of the display unit. The display unit 145 includes a light source substrate 121, LEDs 122 as light sources (light source elements), a lens 130 as an optical system of the light source, a diffusion plate 137, and a liquid crystal panel 149. The LEDs 122 are arranged in a lattice pattern on the light source substrate 121, thereby forming an LED array.

[0208] Next, FIG. 12 is referred to. FIG. 12(A) and FIG. 12(B) are diagrams showing an outline of the viewpoint tracking spotlighting control using the display unit shown in FIG. 11. In the diagram, x', y', and z' indicate coordinates applied to the display unit. FIG. 12(A) shows a configuration seen from the direction along the z' axis, and FIG. 12(B) shows a configuration seen from the direction along the X' axis. In FIG. 12(A), LEDs 120g and 120i are emitting light. In FIG. 12(B), LEDs 122e, 122g, and 122f are emitting light. As a result, as shown in the upper part of FIG. 12(A), display light is projected onto the left and right eyes A1 and A2 of the driver, and spotlighting is realized. In addition, the symbol EB indicates an eye box. In addition, the symbol 169 indicates a folding mirror in FIG. 10, and the symbol 171 indicates a curved mirror such as a concave mirror. The symbol VP is a display image that is conjugate with the virtual image V in FIG. 10.

[0209] Next, reference is made to Fig. 13. Fig. 13 is a diagram showing an example of an arrangement of an LED array in a display unit. In Fig. 13, LEDs 122 are arranged in a lattice pattern on a light source substrate 121 to form an LED array. In the example of Fig. 13, LEDs 122g and 122i emit light. The light emission of the LEDs 122 is controlled by a light emission control unit 116 of the control unit 100. This is the same as the configuration in Fig. 1 shown above.

[0210] Next, Fig. 14 will be referred to. Fig. 14(A) is a diagram showing the illumination area in the eye box before the illumination area is expanded, and Fig. 14(B) is a diagram showing the illumination area in the eye box after the illumination area is expanded, and an example of the configuration of a control unit that expands or reduces the illumination area. 14A, the eye box EB is divided into a plurality of regions Z20 to Z28. The driver's left eye A1 is located in the divided region Z24. In the drawing, the reference character R100 indicates the irradiation region of the display light.

[0211] 14(B) will be referred to. The display control device 350 has a control unit 100. The control unit 100 has a viewpoint speed detection unit 305, a viewpoint speed determination unit 307, and a light emission control unit .

[0212] In FIG. 14(B), the viewpoint moves from divided area Z24 of the eye box EB to Z22. The viewpoint speed detection unit 305 detects the viewpoint movement speed vs. The viewpoint speed determination unit 307 compares the viewpoint movement speed vs with a first threshold value θv1 and a second threshold value θv2. The comparison determination can be realized with a simple configuration, and is therefore easy to implement. Also, high-speed and accurate determination is possible. When the viewpoint movement speed vs becomes equal to or greater than the first threshold value θv1, the control unit 100 determines that a state in which viewpoint loss, in which the viewpoint position cannot be detected, is likely to occur, has occurred, and expands the illumination area of ​​the display light via the light emission control unit 116. The above-mentioned situation is likely to occur when the driver's face suddenly swings widely, etc. In this way, when the control unit 100 detects a state in which viewpoint loss, in which the viewpoint position cannot be detected, is likely to occur, the control unit 100 expands the area of ​​pixels or light sources that contribute to displaying a virtual image on the display unit 145, and controls to illuminate an illumination area wider than the partial illumination area of ​​the eye box EB that was illuminated immediately before the expansion. Specifically, for example, in the LED unit shown above, the illumination range is expanded by individually controlling the emission of LEDs.

[0213] 14B, the movement speed (eye speed) vs of the viewpoint (eye) becomes equal to or greater than the first threshold θv1, so the illumination area of ​​the display light is expanded by the control of the control unit 100. The expanded illumination area 200 illuminates almost the entire area of ​​the eye box EB.

[0214] By expanding the illumination area of ​​the display light, the deterioration of the display quality of the image (virtual image) caused by the image (virtual image) becoming invisible or the display light not being properly irradiated to the viewpoint (eye) is suppressed. In addition, since the comparison judgment is performed at high speed and with high accuracy, the illumination range can be expanded quickly and reliably.

[0215] Furthermore, when the moving speed vs of the viewpoint becomes equal to or smaller than the second threshold value θv2, the control unit 100 reduces the enlarged illumination area R200 to a size equivalent to the illumination area R100 shown in Fig. 14(A). This provides the effects of reducing power consumption, suppressing the temperature rise of the light source, and reducing the load on the device.

[0216] When reducing the illumination area, the moving speed vs of the viewpoint (eye) may be compared with the first threshold θv1, and the illumination area may be reduced when the moving speed vs is less than the first threshold θv1, in the same manner as in the case of expansion. In this case, the second threshold θv2 is not necessary.

[0217] In addition, when the second threshold θv2 is used, the second threshold θv2 may be smaller than the first threshold θv1. In this case, the need to reduce the illumination area is not determined unless the moving speed of the viewpoint falls below the first threshold θv1 and further becomes equal to or less than the second threshold θv2. Therefore, the state in which the illumination area is expanded can be stably maintained. In this case, the effect of reducing the probability that the viewpoint will go out of the illumination range is obtained. On the other hand, the second threshold θv2 may be greater than the first threshold θv1. In this case, when the moving speed of the viewpoint becomes equal to or less than the second threshold θv2, it is determined that the illumination area needs to be reduced. This shortens the period during which the illumination area is expanded, thereby obtaining effects such as suppressing the temperature rise of the light source and reducing power consumption.

[0218] Next, reference is made to Fig. 15. Fig. 15(A) is a diagram showing an example of an LED lighting pattern when the illumination area is expanded in stages, and Fig. 15(B) is a diagram showing an example of an LED lighting pattern when the illumination area is reduced in stages. On the light source substrate 121, a plurality of LEDs 122 are arranged in a lattice pattern to form an LED array. In FIG. 15(A), the number of lit LEDs 122 increases as the state transitions from state A1 to state A4. In state A1, two LEDs are lit, and in state A2, the number of lit LEDs increases to nine. In state A3, the number of lit LEDs increases to 15. In state A3, all LEDs are lit. In other words, all LEDs are lit. This allows for a gradual expansion of the illumination range. In Fig. 15(B), the lighting pattern changes inversely to that in Fig. 15(A), and the number of lit LEDs 122 decreases as the state transitions from B1 to B4. This realizes a gradual reduction in the illumination range.

[0219] In this way, the expansion of the illumination area may include a multi-stage reduction process, which reduces the sense of discomfort felt by the driver and also reduces the burden on the device. The reduction of the illumination area may include a multi-stage reduction process, thereby reducing the discomfort felt by the driver.

[0220] Next, reference is made to Fig. 16. Fig. 16 is a flowchart showing an example of a procedure for expanding or reducing the illumination area. In step S100, viewpoint position information is acquired. In step S101, viewpoint speed is detected. In step S102, it is determined whether the LED array is in a fully lit state. If the result is Y, the process proceeds to step S103.

[0221] In step S103, it is determined whether the viewpoint speed is equal to or less than the second threshold value θv2. If it is Y, in step S104, the lighting area (the number of lit LEDs) is reduced. The lighting pattern of the LED array is determined based on the detected viewpoint position. If N, the process proceeds to step S105. In step S105, the all-illuminated state is maintained.

[0222] If the answer is N in step S102, the process proceeds to step S106. In step S106, it is determined whether the viewpoint speed is equal to or greater than the first threshold value θv1. If the answer is Y, the lighting area (the number of lit LEDs) is expanded in step S107. If the answer is N, the current light emission state is maintained. In other words, a lighting pattern is determined based on the viewpoint position, the LED array emits light in that lighting pattern, and that light emission state is maintained.

[0223] Next, Fig. 17 is referred to. Fig. 17 is a diagram showing another example of the configuration of the lighting area in the eye box after the lighting area is expanded, and the control unit that expands or reduces the lighting area. In Fig. 17, the control unit 100 has a situation determination unit 309 as a means for determining that a state in which viewpoint loss is likely to occur has occurred. The situation determination unit 309 is provided with various information related to the driving situation from the ECU 160, for example.

[0224] When a sudden change in brightness of the surroundings including the front of the vehicle is detected, or when such a sudden change is predicted, in other words, specifically when the vehicle is passing through a tunnel, the situation determination unit 309 of the control unit 100 determines whether or not the vehicle is in a state where ... And, When a sudden change in brightness inside the vehicle is detected or predicted, specifically when highly directional light from the headlights of an oncoming vehicle enters the vehicle or when highly directional light from an LED street light enters the vehicle, And, When the rear seat monitor or child monitor installed in the vehicle is turned on or is in the turned on state, And, When it is determined that the driving environment of the vehicle is an environment that increases the fluctuation of the position of the face or eyes of the driver, specifically, when the vehicle is driving on a rough road, When at least one of the above conditions is met, it can be determined that the state is prone to viewpoint loss. The illumination area is expanded based on this determination result.

[0225] By making such situation judgments, it becomes possible to take measures in advance in response to various driving situations in which viewpoint loss is likely to occur. A sudden change in brightness is likely to occur, for example, when a vehicle enters or exits a tunnel. When a sudden change in brightness occurs, the driver tends to close his or her eyes or look away from the road ahead, which can be a cause of loss of focus. In addition, sudden changes in brightness inside the vehicle are likely to occur when, for example, highly directional light from the headlights of oncoming vehicles or highly directional light from street lights using LEDs enters the vehicle. This situation can also be a cause of viewpoint loss. In addition, when a rear seat monitor or a child monitor is turned on, the driver tends to look at the monitoring terminal, which can be one of the reasons for losing focus. In addition, when the vehicle is traveling on rough roads, such as a road with many bumps or a muddy unpaved road, or a road with a series of sharp curves, the driver is likely to lose focus due to shaking of the body or face. According to the configuration of FIG. 17, measures against loss of viewpoint can be implemented in such various driving scenes.

[0226] Next, reference is made to Fig. 18. Fig. 18 is a diagram showing an example of the configuration of a vehicle display system. The vehicle display system has ECU 160, antenna AN, communication unit 298, map database 306, a monitoring terminal 310 such as a liquid crystal display device that can be used for a rear seat monitor or a child monitor, an ambient brightness sensor (ambient brightness detection unit) 312 that detects the brightness of the surroundings including the front, an interior brightness sensor (interior brightness detection unit) 314 that detects the brightness inside the vehicle, an ambient imaging camera 316 that captures the actual surroundings including the front, a camera (viewpoint detection unit) 90 that detects the viewpoint position, and a monitoring camera 320 for a rear seat monitor or child monitor that monitors people or children sitting in the rear seat.

[0227] In the example of FIG. 18, a driver 201 can know the state of a child 203 seated in the back seat (assumed to be a child seat here) via a monitoring terminal 310. The viewpoint image capture signal S0, various information (including vehicle speed information) S3 output from the ECU 160, the surrounding brightness detection signal S5 output from the surrounding brightness sensor 312, the interior brightness detection signal S6 output from the interior brightness sensor 314, surrounding captured image information S7 output from the surrounding image capture camera 316, and monitoring information S8 output from the monitoring camera 320 are each supplied to a display control device 350. Based on these various signals and information, the display control device 350 detects or predicts the occurrence of a state in which viewpoint loss is likely to occur, and takes preventative measures such as those shown in FIG. 14(B) or FIG. 17.

[0228] Next, reference is made to Fig. 19. Fig. 19 is a diagram showing an example of the configuration of a display control device. In Fig. 19, parts common to the configuration of the control unit shown in Fig. 1(A) and Fig. 2(A) are given the same reference numerals. A display control device 350 in Fig. 19 has a control unit 100, similar to the configurations in Fig. 1(A) and Fig. 2(A). However, in Fig. 19, a viewpoint speed detection unit 305, a viewpoint speed determination unit 307, and a situation determination unit 309 are added to the configurations in Fig. 1(A) and Fig. 2(A). The various signals (or information) S5 to S9 shown in FIG.

[0229] The display control device 350 can be provided inside the HUD device 1' (broadly speaking, a device capable of displaying various images) shown in Fig. 10. However, the present invention is not limited to this. It is also possible to install the display control device 350 outside the HUD device 1'. In this case, the display control device 350 and the HUD device 1' are connected by a wireless or wired communication means. Such modifications and applications can be made as appropriate.

[0230] Next, reference is made to Fig. 20. Fig. 20 is a diagram showing an example of the expansion and contraction of the illumination area when a vehicle passes through a tunnel. 20, a vehicle 401 passes through a tunnel 403. The brightness state around the vehicle 401 is a "bright state" from time t0 to t2, a "transition state from light to dark" from time t2 to t3, a "dark state" from time t3 to t4, a "transition state from dark to light" from time t4 to t6, and a "bright" state from time t6 onwards.

[0231] At time t0, it is detected that there is a tunnel 403 ahead of the vehicle 401. This detection can be realized, for example, by using GPS communication and the map database 306. The presence of the tunnel 403 can also be detected by capturing an image of the front using the surroundings imaging camera 316 and performing image processing based on the captured image. The surroundings brightness sensor 312 may also detect a sudden decrease in brightness in the tunnel ahead.

[0232] Before time t0, the illumination range in the viewpoint tracking spotlighting is set to the normal range (illumination range R100 in FIG. 14(A)).

[0233] When the presence of the tunnel 403 is detected, the display control device 350 determines that a state in which viewpoint loss is likely to occur is predicted, and expands the illumination range at time t0 (see illumination range R200 in FIG. 17). The expanded illumination range continues until time t3. After time t3, the illumination range is reduced and returned to the normal range.

[0234] At time t4, a sudden rise in the surrounding brightness is detected by, for example, the surrounding brightness sensor 312. The display control device 350 determines that a state in which viewpoint loss is likely to occur is predicted, and expands the illumination range at time t4 (see illumination range R200 in FIG. 17). The expanded illumination range continues until time t7. After time t7, the illumination range is reduced and returned to the normal range.

[0235] In this way, it is possible to predict a sudden drop in the surrounding brightness and expand the illumination range in the viewpoint tracking spot lighting before the vehicle 401 enters the tunnel 403. Similarly, it is possible to predict a sudden increase in the surrounding brightness and expand the illumination range in the viewpoint tracking spot lighting before the vehicle 401 exits the tunnel 403. Therefore, it is possible to sufficiently prevent a situation in which the driver is unable to view the displayed image (virtual image) or the quality of the displayed image is degraded.

[0236] Next, reference is made to Fig. 21. Fig. 21 is a flow chart showing another example of the procedure for expanding or reducing the illumination area. In Fig. 21, steps common to Fig. 16 are denoted by the same reference numerals. In the flowchart of FIG. 21, steps S101, S103, and S106 in FIG. 16 are replaced with steps S101', S103', and S106'. In S101', the situation determination unit 309 performs situation determination. In step S103', it is determined whether the illumination range needs to be reduced. In step S106', it is determined whether the illumination range needs to be expanded. The other steps are the same as in FIG. 16.

[0237] As described above, according to the present invention, when viewpoint loss occurs or when a state where viewpoint loss is likely to occur occurs during the execution of viewpoint tracking spotlighting control that irradiates a partial region of the eye box with light according to the viewpoint position of the driver, it is possible to suppress the driver from having difficulty in viewing the virtual image or the deterioration of the display quality of the virtual image, thereby reducing the sense of discomfort. This contributes to safe driving, and also ensures driving comfort by allowing the driver to always see the display provided by the HUD device in a good quality in various driving situations.

[0238] The present invention can be used in both monocular HUD devices, in which display light for the same image is incident on each of the left and right eyes, and parallax HUD devices, in which images with parallax are incident on each of the left and right eyes.

[0239] In this specification, the term "vehicle" may be broadly interpreted as a vehicle, and the HUD device may also include a simulator (e.g., an aircraft simulator).

[0240] The present invention is not limited to the above-described exemplary embodiments, and those skilled in the art could easily modify the above-described exemplary embodiments to the extent that they fall within the scope of the claims. [Explanation of symbols]

[0241] 2...windshield (projected member, reflective translucent member), 90...camera (image capture unit for detecting eyes and face for viewpoint detection), 100...control unit, 110...partial lighting (spot lighting) control unit, 112...viewpoint position detection unit, 114...loss of viewpoint detection unit, 116...light emission control unit, 120...light source (backlight), 121...light source board, 122a to 122d...light source elements (LEDs, etc.), 130...light source optical system (first light system), 140···light collecting optical system (second optical system, optical system including a reflector which projects an image onto a windshield or the like), 160···ECU (electronic control unit, general control unit, information collecting unit), EB···eye box, Z1 to Z9, W1 to W12···partial regions (divided regions) of the eye box, 306···map database, 305···viewpoint speed detection unit, 307···viewpoint speed determination unit, 309···situation determination unit, 350···display control device.

Claims

1. A head-up display (HUD) device that is mounted on a vehicle and allows a user to visually recognize a virtual image of an image, A display unit that displays the image; a control unit that performs viewpoint tracking spotlighting control to irradiate a partial area of ​​an eye box with light according to a viewpoint position of a driver; having The control unit is When a viewpoint loss occurs, which makes it impossible to detect the viewpoint position, an area of ​​pixels or light sources contributing to displaying the virtual image on the display unit is expanded, and a lighting area wider than a partial lighting area of ​​the eye box that was illuminated immediately before the viewpoint loss is illuminated is illuminated. Head-up display device.

2. The control unit is The head-up display device according to claim 1 , wherein at least one of a control for expanding the illumination area continuously and a control for expanding the illumination area in a stepwise manner is performed.

3. The control unit is The head-up display device according to claim 2 , wherein when the gaze point loss continues for a first predetermined time or more, a control is implemented to illuminate the entire area of ​​the eye box.

4. The control unit is The head-up display device according to claim 2 or 3, wherein control is performed to increase the amount of illumination light when the illumination area is expanded continuously or stepwise.

5. The control unit is When expanding the illumination area continuously or stepwise, a direction in which the viewpoint was lost is estimated or predicted based on at least one of the direction of movement of the viewpoint before the viewpoint was lost, past learning results regarding the movement of the viewpoint, and the driving situation at the time of the viewpoint being lost, and a control is performed to expand the illumination area so as to include at least the estimated or predicted direction. The head-up display device according to any one of claims 2 to 4.

6. The eye box has a predetermined outer shape, The control unit is When the illumination area is expanded continuously or stepwise, isotropic expansion, which expands the illumination area uniformly around its periphery; or anisotropic expansion, which expands the illumination area non-uniformly around its periphery; Or, the expansion is a combination of the isotropic expansion and the anisotropic expansion; Implement one of the following: When the viewpoint position is detected again during the expansion, or when the illumination area expands to the entire area of ​​the eye box, control is performed to terminate the expansion. The head-up display device according to any one of claims 2 to 5.

7. The control unit is When the viewpoint position is redetected after the viewpoint loss occurs, a control is performed to reduce the wide illumination area expanded after the viewpoint loss to a narrow illumination area having the same size as the partial illumination area before the viewpoint loss, which corresponds to the redetected viewpoint position. The head-up display device according to any one of claims 1 to 6.

8. The eye box has a predetermined outer shape, The control unit is When the viewpoint position is redetected after the viewpoint loss occurs, a control is performed to reduce the wide illumination area expanded after the viewpoint loss to a narrow illumination area having the same size as the partial illumination area before the viewpoint loss, the narrow illumination area corresponding to the redetected viewpoint position; In addition, when reducing the illumination area, A control is performed to restore the illumination area to a narrow illumination area having the same size as the partial illumination area before the viewpoint is lost by performing a combination of anisotropic shrinkage in which the illumination area is uniformly reduced and anisotropic shrinkage in which the illumination area is reduced non-uniformly. Or, When a contraction along one of two orthogonal line segments is defined as a vertical contraction and a contraction along the other is defined as a horizontal contraction, a vertical contraction is performed, and then a horizontal contraction is performed, thereby performing a control to return the partial illumination area to a narrow illumination area having the same size as the partial illumination area before the viewpoint is lost. Or, performing horizontal contraction, and then performing vertical contraction to perform control to return the partial illumination area to a narrow illumination area having the same size as the partial illumination area before the viewpoint is lost; The head-up display device according to claim 6.

9. The control unit is 9. The head-up display device according to claim 7, wherein the control is performed so that a time required for the wide illumination area to become the narrow illumination area is equal to or longer than a second predetermined time.

10. The control unit is The head-up display device according to claim 7 , further comprising: a control for reducing an amount of illumination light when the wide illumination area is reduced to the narrow illumination area.

11. The control unit is In the case where the amount of illumination light is reduced when reducing the wide illumination area to the narrow illumination area, The illumination light is reduced by slowly lowering the brightness of pixels or light source elements that have a small effect on the brightness viewed at the viewpoint position. The head-up display device according to claim 10.

12. The control unit is When reducing the wide illumination area to the narrow illumination area, The luminance of a pixel or light source element that has a small effect on the luminance viewed at the viewpoint position is reduced over time, and the luminance of a pixel or light source element that has a large effect on the luminance viewed at the viewpoint position is increased over time. The head-up display device according to any one of claims 7 to 9.

13. The head-up display device according to claim 3 , wherein the control unit variably controls the first predetermined time period based on at least one of a driving condition of the vehicle, a surrounding environment, and a condition of a driver.

14. The head-up display device according to claim 9 , wherein the control unit variably controls the second predetermined time period based on at least one of a driving state of the vehicle, a surrounding environment, and a state of a driver.

15. The control unit is Regarding the light amount distribution of the illumination light in the illumination area, The amount of light in the peripheral portion of the illumination area is controlled so as to be less than the amount of light in the central portion, but to ensure a luminance equal to or greater than a predetermined value. Or, Controlling the amount of light so as to be uniform within the illumination area; A head-up display device according to any one of claims 1 to 14.

16. A head-up display (HUD) device that is mounted on a vehicle and allows a user to visually recognize a virtual image of an image, A display unit that displays the image; a control unit that performs viewpoint tracking spotlighting control to irradiate a partial area of ​​the eye box with light according to the viewpoint position of the driver; having The control unit is When a state in which a viewpoint loss in which the viewpoint position cannot be detected is likely to occur is detected, an area of ​​pixels or light sources contributing to display of the virtual image on the display unit is expanded, and a lighting area wider than a partial lighting area of ​​the eye box that was illuminated immediately before the expansion is controlled to be illuminated. Head-up display device.

17. The control unit is When the moving speed of the viewpoint is equal to or greater than a first threshold, it is determined that the viewpoint loss is likely to occur. The head-up display device according to claim 16.

18. The control unit is When a sudden change in brightness of the surroundings including the front of the vehicle is detected or a sudden change is predicted, And, If a sudden change in the brightness inside the vehicle is detected or predicted, And, When a rear seat monitor or a child monitor provided in the vehicle is turned on or is in an on state, And, When it is determined that the driving environment of the vehicle is an environment that increases the fluctuation of the face or eye position of the driver, When at least one of the above conditions is met, it is determined that the state is prone to cause the viewpoint loss. The head-up display device according to claim 16.

19. The expansion of the illumination region includes a multi-stage expansion process. The head-up display device according to any one of claims 16 to 18.

20. The control unit is When the state where the viewpoint loss is likely to occur is resolved, a control is performed to reduce the expanded wide illumination area to a narrow illumination area having the same size as the partial illumination area before the expansion.

20. A head-up display device according to any one of claims 16 to 19.

21. The control unit is When the moving speed of the viewpoint becomes less than the first threshold value, Or, When the value is smaller than the first threshold value or equal to or less than a second threshold value larger than the first threshold value, performing control to reduce the expanded wide illumination area to a narrow illumination area having the same size as the partial illumination area before the expansion; The head-up display device according to claim 17.

22. The reduction of the illumination area includes a multi-stage reduction process.

22. The head-up display device according to claim 20 or 21.

23. A display control device comprising the control unit according to any one of claims 1 to 21.

24. A step of performing viewpoint tracking spotlighting control for irradiating a partial area of ​​an eye box with light according to a viewpoint position of a driver; When a viewpoint loss occurs in which the viewpoint position cannot be detected, or when a state in which the viewpoint loss is likely to occur is detected, an area of ​​pixels or light sources contributing to display of an image is expanded, and an illumination area wider than the partial illumination area of ​​the eye box that was illuminated immediately before the expansion is performed is illuminated; When the viewpoint position is redetected after the viewpoint loss occurs, or when the state in which the viewpoint loss is likely to occur is resolved, reducing the expanded wide illumination area to a narrow illumination area having the same size as the partial illumination area before the expansion; A display control method comprising:

Citation Information

Patent Citations

  • Head-up display device

    JP2018045143A

  • Head-up display and mobile body equipped with head-up display

    JP6586646B2

  • Image-generation device for a head-up display and method for controlling such a device

    US20200012098A1

  • Head-up display and moving body comprising head-up display

    WO2017002344A1