Display control device, head-up display device

Abstract

According to the present invention, in the case in which a loss of the viewpoint position of a driver occurs when a viewpoint position follow-up warping control is executed to update warping parameters according to the viewpoint position of the driver, and then the viewpoint position is re-detected, it is suppressed that the appearance of an image instantaneously changes in accordance with the update of the warping parameters and the driver is caused to feel uneasy. When the viewpoint loss in which at least one position of the right and left viewpoints becomes unclear is detected, a control unit 180, which executes the viewpoint position follow-up warping control, maintains, in a viewpoint loss period, the warping parameters set immediately before the viewpoint loss period, and, when the viewpoint position is re-detected after the viewpoint loss period, invalidates at least one warping process using the warping parameters corresponding to the re-detected viewpoint position.

Description

[Technical field]

[0001] The present invention relates to a head-up display (HUD) device, a display control device, etc. that projects (projects) display light of an image onto a projection target member such as a vehicle windshield or combiner, and displays a virtual image in front of a driver, etc. [Background technology]

[0002] In a HUD device, an image correction process (hereinafter referred to as a warping process) is known that distorts a projected image in advance so that the image has characteristics opposite to those of a virtual image caused by the optical system, the curved surface shape of a windshield, etc. The warping process in a HUD device is described in, for example, Patent Document 1.

[0003] Moreover, performing warping processing based on the viewpoint position of the driver (viewpoint tracking warping) is described in, for example, Patent Document 2. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2015-87619 A [Patent Document 2] JP 2014-199385 A Summary of the Invention [Problem to be solved by the invention]

[0005] The inventors have considered implementing viewpoint-tracking warping control, which updates warping parameters according to the viewpoint position of a driver (which can be broadly interpreted as a pilot, crew member, etc.), and have recognized the new problem described below.

[0006] There may be a case where the driver's viewpoint position moves and the HUD device temporarily loses the viewpoint position, after which the viewpoint position is redetected and the warping parameters are updated based on the redetected viewpoint position.

[0007] Here, the image (virtual image) displayed after the warping process would ideally be a flat virtual image in which distortions caused by optical systems, etc. have been completely corrected. In the case of viewpoint-tracking warping, ideally, a virtual image without distortion would always be obtained, even when the viewpoint position of the driver (user) changes; however, it is not possible to completely remove distortion.

[0008] For this reason, if simple viewpoint-position tracking warping is performed based on the redetected viewpoint position after the viewpoint position is lost, even if a virtual image of the same image is displayed, the appearance of the virtual image from the driver's perspective (the appearance of the virtual image, the impression given by the virtual image, etc.) may change instantly, causing the driver to feel uncomfortable.

[0009] A typical example of viewpoint loss is when the driver's viewpoint moves outside the eye box for some reason and then returns to the eye box. However, this is not limited to this, and the driver's viewpoint may also move instantaneously within the eye box. There are many different types of viewpoint loss. It is also important to take measures to consider the situation of viewpoint loss as necessary.

[0010] Furthermore, in recent years, for example, HUD devices capable of displaying virtual images over a fairly wide range in front of a vehicle have been developed, and such HUD devices tend to become larger. Although efforts have been made to reduce distortion by improving the design of the optical system, for example, it is difficult to achieve a uniform distortion reduction effect in the entire area of ​​the eyebox. For example, it is possible to imagine a situation in which, although the degree of distortion can be significantly reduced when the driver's viewpoint is in the central area of ​​the eyebox, the degree of residual distortion becomes large to a certain extent when the viewpoint is in the periphery of the eyebox. This point can also be one of the factors that greatly change the appearance of the virtual image due to the warping process after the viewpoint position is lost.

[0011] One object of the present invention is to prevent a HUD device from experiencing a sense of discomfort when performing a viewpoint-tracking warping process, in which the driver's viewpoint position is lost while performing viewpoint-tracking warping control that updates warping parameters in accordance with the driver's viewpoint position, and the viewpoint position is subsequently redetected, in which the appearance of the image changes instantaneously as the warping parameters are updated.

[0012] Other objects of the present invention will become apparent to those skilled in the art by referring to the following exemplary aspects and best modes, as well as the accompanying drawings. [Means for solving the problem]

[0013] In order to facilitate an understanding of the outline of the present invention, embodiments according to the present invention will be exemplified below.

[0014] In a first aspect, a display control device includes: A display control device that is mounted on a vehicle and has a display unit that displays an image, and an optical system that includes an optical member that reflects display light of the image and projects it onto the projection target, and controls a head-up display (HUD) device that projects the image onto the projection target provided in the vehicle, thereby allowing a driver to visually recognize a virtual image of the image, a control unit that updates a warping parameter in accordance with a viewpoint position of a driver in an eyebox and performs a viewpoint-position tracking warping control by using the warping parameter to pre-distort an image to be displayed on the display unit so that the image has a distortion characteristic opposite to a distortion characteristic of a virtual image of the image, The control unit is When a viewpoint loss is detected in which the position of at least one of the right and left viewpoints of the driver becomes unclear, during a viewpoint lost period, the warping parameters set immediately before the viewpoint lost period are maintained; When the position of the viewpoint is redetected after the viewpoint lost period, at least one warping process using the warping parameters corresponding to the redetected viewpoint position is invalidated.

[0015] In the first mode, during a period of viewpoint loss (sometimes referred to as viewpoint loss or viewpoint position loss), the previous warping parameters are maintained, and when the viewpoint position is redetected, warping using new warping parameters is not performed immediately, but is performed after a delay.

[0016] In other words, when the viewpoint position is redetected after the viewpoint is lost, the control unit disables at least one warping process using the warping parameters corresponding to the redetected viewpoint position (setting the disablement period after redetection), and then enables (performs) the warping process using the warping parameters corresponding to the viewpoint position after the disablement period has ended.

[0017] For example, consider a case where the eyebox is divided into multiple partial regions, and a method is adopted in which the viewpoint position is detected for each partial region. For example, even if the viewpoint is in "partial region A" just before the viewpoint is lost, the viewpoint is lost, and then when the viewpoint position is redetected, it is found that the viewpoint has moved to "partial region B," the warping process that applies the warping parameters corresponding to the position of "partial region B" is invalidated (invalidated at least once). When the viewpoint position moves from partial region B to partial region C, the warping process that applies the warping parameters corresponding to the position of partial region C can be invalidated again (invalidated a second time).

[0018] The number of invalidations can be adaptively determined in consideration of the manner in which the viewpoint is lost (eg, the length of the viewpoint lost period), the traveling state of the vehicle (eg, the vehicle speed), and the like.

[0019] In a second embodiment dependent on the first embodiment, The control unit is comparing the viewpoint lost time with a threshold value, and if the viewpoint lost time is shorter than the threshold value, A control may be implemented to make a period during which the warping process is disabled longer than when the viewpoint lost time is longer than the threshold value.

[0020] In a second aspect, the control unit measures the time during which the viewpoint position is lost (lost time or lost time), and when the lost time is shorter than a predetermined threshold, performs control to lengthen the invalidation period.

[0021] In the second mode, the period during which the warping parameters are fixed and the appearance of the image (virtual image) is kept constant is set longer, so that the driver can be informed that re-detection after the viewpoint loss has been successful and that the corresponding processing is now being performed. In other words, by lengthening the period during which the warping parameters are fixed and performing image processing using warping parameters updated over time, even if the appearance of the image (virtual image) changes, the driver can be made to recognize that the change does not occur suddenly in accordance with the movement of the driver's eyes, but that it is a change that has been made with a certain amount of time.

[0022] This makes it easier for the driver to intuitively know that although the driver lost track of his or her viewpoint due to the movement of his or her eyes, the HUD device system successfully redetected the viewpoint and performed processing to address the loss of viewpoint.

[0023] In other words, it is possible to convey to the user (the driver) that the HUD device (system) is properly processing the situation after the viewpoint is lost. This gives the driver a sense of security and mental stability, and therefore has the effect of making it difficult for discomfort to occur or reducing the discomfort.

[0024] In a third embodiment dependent on the first embodiment, The control unit is comparing the viewpoint lost time with a threshold value, and if the viewpoint lost time is shorter than the threshold value, A control may be implemented to make a period during which the warping process is disabled shorter than a period when the viewpoint lost time is longer than the threshold value.

[0025] In the third aspect, the control unit measures the time when the viewpoint position is lost, and when the lost time is shorter than a predetermined threshold, the control unit performs control to shorten the invalidation period. Although the direction of control is opposite to that of the second aspect, the second and third aspects each have different effects, and therefore each aspect can be selectively applied according to the desired effect.

[0026] When the viewpoint loss time is short, it is considered that the change in the viewpoint position is small (the movement distance of the viewpoint is relatively short). Therefore, the invalidation time of the warping process using the new warping parameters is set shorter than when the viewpoint loss time is long. As a result, for example, after performing the necessary minimum invalidation (timing delay), an appropriately warped and corrected image (virtual image) corresponding to the viewpoint position can be quickly displayed, thereby reducing discomfort and suppressing the occurrence of discomfort.

[0027] In other words, when the viewpoint loss period is short, since it is estimated that the movement distance of the viewpoint position is small, it can be estimated that there is little significant difference in the distortion mode of the virtual image before and after the update of the warping parameters. Considering this point, it is possible to prevent a sharp (in other words, a change in the appearance of the virtual image in a relatively short time) change in the appearance of the virtual image immediately after the re-detection of the viewpoint position, and then quickly return to the normal viewpoint tracking warping control, thereby surely obtaining the effect of improving visibility.

[0028] In a fourth aspect subordinate to any one of the first to third aspects, the control unit When the update cycle of the warping parameters before the occurrence of the viewpoint loss and during the viewpoint loss period is set as the first update cycle RT1, and the update cycle of the warping parameters during the period when the warping process is invalidated is set as the second update cycle RT2, the parameter update cycle may be changed so that RT1 < RT2.

[0029] In the fourth aspect, during the invalidation period, in parallel with the process of maintaining the parameters immediately before the viewpoint loss, a process of lengthening the update cycle of the warping parameters (a process of changing the update cycle of the warping parameters) is used in combination.

[0030] For example, if the frame rate of an image (virtual image) is 60 fps (frames per second), image processing (image display processing) of 60 frames is performed per second (in other words, one frame period is 1 / 60 seconds). As an example, it is assumed that the warping parameters are also normally updated for each frame.

[0031] Here, in the invalidation period after the viewpoint is lost, if the parameter is updated every two frames, the update period will be 2 / 60 seconds, and if it is updated every three frames, the update period will be 3 / 60 seconds, which is a long update period. In this way, by switching to an update in units of multiple frames, it is possible to lengthen (increase) the update period. By lengthening the update period, the reflection of the updated warping parameters in the image (virtual image) will be delayed. In other words, the sensitivity of the reflection of the updated parameters in the display will be reduced. After the invalidation period has passed, the update period of the changed parameters will be returned to the original (returned from RT2 to RT1), but since the return of the update period is not actually done instantly and requires a certain amount of time, even if the parameters are switched, the reflection of the switched parameters in the actual display will be delayed.

[0032] This makes it easy to appropriately provide a certain amount of delay (a delay long enough that the driver can perceive a delayed change in the appearance of the actual display (in other words, a delay that has the effect of substantially slightly extending the width of the invalidation period)). It is also expected to have the effect of making it easier to design the timing control circuit.

[0033] In addition, the range of control variations is expanded, and flexible responses are possible, such as variably controlling the degree of increase in the parameter update cycle, or devising the timing for returning the increased update cycle to its original state, etc. It also becomes easy to set the amount of delay in actual display control over a fairly wide range.

[0034] In a fifth aspect dependent from the fourth aspect, The control unit is After changing the parameter update period from RT1 to RT2, At the end of the period during which the warping process is disabled, the RT2 is returned to the RT1. Or, Returning from the RT2 to the RT1 when a predetermined time has elapsed from the end of the period during which the warping process is disabled. Or, The change in the parameter update period may be started from the end timing of the period during which the warping process is disabled, and may be gradually returned from the RT2 to the RT1 over time.

[0035] The fifth aspect describes an example of an aspect in which, after the process of lengthening the update period in the fourth aspect is carried out, the update period is returned to the original period.

[0036] In the first example, the long update period is returned to the original short update period in synchronization with the switching (update) of the warping parameters. Even in this case, since a certain amount of time is required to change the update period, the delay for that amount is reliably secured.

[0037] In a second example, the parameter update cycle is restored when a predetermined time has elapsed since the warping parameter was switched (updated). In this example, the timing of restoring the update cycle is delayed by a predetermined time from the parameter switching (updating) timing, and the reflection of the changed parameters in the display is further delayed, making it easier to reliably realize a delay of an appropriate length that can be perceived by the human eye.

[0038] In the third example, when the update period is returned to the original state, it is gradually returned to the original state over a predetermined time. In other words, when the update period is returned from 1 / 15 seconds to 1 / 60 seconds, for example, it is not returned immediately, but is controlled so as to gradually switch in stages in units of a predetermined time, such as 1 / 30 seconds, 1 / 45 seconds, and 1 / 60 seconds. By controlling the update period to be switched gradually on the time axis, the delay in reflecting the changed parameters in the display can be managed with higher accuracy.

[0039] In a sixth aspect dependent on any one of the first to fifth aspects, and a low speed state determination unit that determines whether the speed of the vehicle is in a low speed state. The control unit is A period during which the warping process is disabled when the vehicle is in the low speed state including a stopped state may be longer than a period during which the warping process is disabled when the vehicle is in a speed faster than the low speed state.

[0040] In the sixth aspect, when the vehicle is in a low speed state, the invalidation period is set longer (in other words, the timing of switching the warping parameters is delayed further) than when the vehicle is in a medium or high speed state. When the vehicle is in a low speed state, the driver is sensitive to visual fluctuations in the front and the like, and is easy to detect such fluctuations. Therefore, at this time, the reflection of the new parameters in the image after the viewpoint is lost is delayed more significantly, and measures are taken to prevent discomfort caused by an instantaneous change in the appearance of the display from occurring. When the vehicle speed leaves the low speed state and becomes faster, the invalidation period is shortened (this can also include the case where the invalidation period is eliminated), and control is implemented that emphasizes accelerating the correction of image distortion based on the viewpoint position redetected after the loss. This makes it possible to perform appropriate warping control corresponding to the vehicle speed.

[0041] In a seventh aspect dependent on any one of the first to sixth aspects, The control unit is The period during which the warping process is disabled is changed according to the vehicle speed. In this case, When the speed of the vehicle is within a range of a first speed value U1 (U1>0) or more and a second speed value U2 that is greater than the first speed value, the period during which the warping process is disabled is controlled to be decreased with respect to the vehicle speed as the vehicle speed increases. Or, A control is implemented in which the degree of the decrease is made gentle when the vehicle speed is in a range close to the first speed value, and the degree of the decrease is made steeper as the vehicle speed moves away from the first speed value. Or, A control may be implemented in which the degree of decrease is made gentle when the vehicle speed is in a range close to the first speed value, and the degree of decrease is made steeper as the vehicle speed moves away from the first speed value, and the degree of decrease is made gentler as the vehicle speed approaches the second speed value.

[0042] In the seventh aspect, when control is performed to shorten (in other words, reduce) the period during which the warping process is disabled in response to an increase in vehicle speed, the control may be performed when the vehicle speed is in a range of equal to or greater than a first speed U1 (>0) and equal to or less than a second speed value U2 that is greater than the first speed value (first control). In this case, the control may not be performed when the vehicle speed is in a range less than the first speed U1 and greater than the second speed U2, so as not to place an excessive burden on the system of the HUD device. Also, by reducing the period of disabling relative to the speed, more flexible and appropriate warping process according to the speed is possible.

[0043] Also, when the vehicle is in a low-speed state where the driver is likely to perceive visual changes in the image (virtual image) (in other words, when the vehicle speed is in a range close to the first speed value U1), control may be implemented to reduce the extent to which the invalidation period is decreased so as to suppress abrupt updates to the warping parameters (second control). In this case, more accurate control is achieved.

[0044] In addition to the second control, a control (inverse S-shaped characteristic control) may be implemented to gradually decrease the invalidation period as the second speed value U2 is approached, so that the decrease stops and becomes constant when the second speed value U2 is reached, thereby preventing the change from suddenly hitting a plateau and causing discomfort (third control). This can further improve the visibility of the virtual image.

[0045] In an eighth aspect dependent on any one of the first to seventh aspects, The head-up display device includes: When adjusting the position of the eye box in accordance with the height position of the viewpoint of the driver, the optical member may be left unchanged and a reflection position of the display light of the image on the optical member may be changed.

[0046] In an eighth aspect, when the HUD device that implements the above control adjusts the height position of the eyebox in accordance with the height position of the driver's eyes (point of view), the HUD device does not rotate the optical member that projects light onto the projection target member, for example, by using an actuator, but instead adjusts the reflection position of the light on the optical member.

[0047] Recent HUD devices have tended to be developed on the premise that virtual images are displayed over a fairly wide range in front of the vehicle, and in this case, the device inevitably becomes large. Naturally, the optical components also become large. If the optical components are rotated using an actuator or the like, the error may actually reduce the accuracy of the control of the height position of the eye box. To prevent this, the position where the light beam is reflected by the optical components is changed.

[0048] In such large optical components, the reflection surface is optimally designed as a free-form surface to prevent the distortion of the virtual image from becoming apparent as much as possible, but as described above, for example, when the driver's viewpoint is located around the eye box, the distortion may become apparent. Therefore, in such a case, by implementing control to temporarily disable (delay) the application of the parameters corresponding to the viewpoint position after the redetection within a predetermined range, it is possible to make it difficult for discomfort caused by the change in appearance due to the distortion of the virtual image to occur, and the above control can be effectively utilized to improve visibility.

[0049] In a ninth aspect dependent on any one of the first to eighth aspects, A virtual image display surface corresponding to the image display surface of the display unit is disposed so as to be superimposed on a road surface in front of the vehicle. Or, The virtual image display surface may be arranged at an angle to the road surface so that the distance between the near end, which is the end closest to the vehicle, and the road surface is small, and the distance between the far end, which is the end farther from the vehicle, and the road surface is large.

[0050] In a ninth aspect, in the HUD device, a virtual image display surface (corresponding to a display surface such as a screen as a display unit) disposed in front of the vehicle is provided so as to be superimposed on the road surface or inclined with respect to the road surface. The former is sometimes called a road surface superimposed HUD, and the latter is sometimes called an inclined surface HUD.

[0051] These devices use a wide virtual image display surface superimposed on the road surface or a wide virtual image display surface inclined to the road surface to perform various displays, for example, within a range of 5m to 100m ahead of the vehicle, and it is preferable that the HUD device and eye box are enlarged to detect the viewpoint position with high accuracy over a wider range than before and perform image correction using appropriate warping parameters. However, when viewpoint loss occurs, the highly accurate warping parameter switching control may actually reduce the visibility of the image (virtual image) after the viewpoint is redetected. Therefore, the application of the control method of the present invention is effective.

[0052] Those skilled in the art will easily understand that the exemplified embodiments according to the present invention can be further modified without departing from the spirit of the present invention. [Brief description of the drawings]

[0053] [Figure 1] FIG. 1(A) is a diagram for explaining an overview of the warping process and the manner in which a virtual image (and a virtual image display surface) displayed after the warping process is distorted, and FIG. 1(B) is a diagram showing an example of a virtual image viewed by a driver through the windshield. [Diagram 2] FIG. 2A is a diagram for explaining an overview of the viewpoint position tracking warping process, and FIG. 2B is a diagram showing an example of the configuration of an eyebox the inside of which is divided into a plurality of partial regions. [Diagram 3]3(A) to 3(F) are diagrams showing examples of virtual images distorted in different ways after warping processing. [Figure 4] FIG. 4 is a diagram showing an example of viewpoint loss and redetection of the viewpoint position in an eyebox whose interior is divided into a plurality of partial regions. [Diagram 5] FIG. 5 is a diagram illustrating an example of a system configuration of the HUD device. [Figure 6] 6(A) to 6(C) are timing charts showing an example of control for providing a period during which warping processing based on a redetected viewpoint position is disabled. [Figure 7] 7(A) to (D) are timing charts showing another example of control for providing a period during which warping processing based on a redetected viewpoint position is invalidated (when parameter update period change processing is also used). [Figure 8] Figures 8(A) and (B) are timing charts showing another example of control for setting a period during which warping processing based on a redetected viewpoint position is disabled (a first control example when the viewpoint lost period is shorter than a threshold value). [Figure 9] Figures 9(A) and (B) are timing charts showing another example of control for setting a period during which warping processing based on a redetected viewpoint position is disabled (a second control example when the viewpoint lost period is shorter than a threshold value). [Figure 10] FIG. 10 is a diagram showing an example of characteristics in the case where the period during which the warping process based on the redetected viewpoint position is invalidated is variably controlled according to the vehicle speed. [Figure 11] FIG. 11 is a flowchart showing an example of a procedure (first control example) of warping image correction control for accommodating viewpoint loss. [Figure 12] FIG. 12 is a flowchart showing an example of a procedure (second control example) of warping image correction control for accommodating viewpoint loss. [Figure 13] FIG. 13 is a flowchart showing an example of a procedure (third control example) of warping image correction control for accommodating viewpoint loss. [Figure 14]FIG. 14(A) is a diagram showing an example of display by a road surface superimposed HUD, FIG. 14(B) is a diagram showing an example of display by an inclined surface HUD, and FIG. 14(C) is a diagram showing an example of the configuration of the main parts of a HUD device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0054] The best mode described below is used to easily understand the present invention. Therefore, those skilled in the art should be aware that the present invention is not unduly limited by the mode described below.

[0055] Please refer to Fig. 1. Fig. 1(A) is a diagram for explaining an overview of the warping process and a state of distortion of a virtual image (and a virtual image display surface) displayed through the warping process, and Fig. 1(B) is a diagram showing an example of a virtual image visually recognized by a driver through a windshield.

[0056] As shown in FIG. 1A, the HUD device 100 has a display unit (e.g., a light-transmitting screen) 101, a reflecting mirror 103, and a curved mirror (e.g., a concave mirror, the reflecting surface of which may be a free-form surface) 105 as an optical member that projects display light. An image displayed on the display unit 101 is projected onto a virtual image display area 5 of a windshield 2 as a projection target member via the reflecting mirror 103 and the curved mirror 105. In FIG. 1, reference numeral 4 indicates a projection area. Note that the HUD 100 may be provided with a plurality of curved mirrors, and may include a refractive optical element such as a lens, a diffractive optical element, or other functional optical element in addition to or instead of a part (or all) of the mirror (reflective optical element) of the present embodiment.

[0057] A portion of the image display light is reflected by the windshield 2 and enters the viewpoint (eye) A of the driver or the like located inside (or on) a predetermined eye box EB (here, a rectangular shape of a specified area), and is focused in front of the vehicle 1, thereby displaying a virtual image V on a virtual virtual image display surface PS corresponding to the display surface 102 of the display unit 101.

[0058] The image on the display unit 101 is distorted due to the influence of the shape of the curved mirror 105, the shape of the windshield 2, etc. In order to offset the distortion, a distortion having the opposite characteristics to the distortion is applied to the image. This predistortion type image correction is referred to in this specification as warping processing or warping image correction processing.

[0059] Ideally, the warping process would result in the virtual image V displayed on the virtual image display surface PS being a flat image with no curvature. However, in the case of, for example, a large HUD device 100 that projects display light onto a wide projection area 4 on the windshield 2 and sets the virtual image display distance over a fairly wide range, it is undeniable that some degree of distortion will remain, and this is unavoidable.

[0060] In the upper left of FIG. 1, PS' indicated by a broken line indicates a virtual image display surface from which distortion has not been completely removed, and V' indicates a virtual image displayed on that virtual image display surface PS'.

[0061] Furthermore, the degree or manner of distortion of the virtual image V that remains distorted differs depending on the position of the viewpoint A on the eye box EB. Since the optical system of the HUD device 100 is designed assuming that the viewpoint A is located near the center, there is a tendency for the distortion of the virtual image to be relatively small when the viewpoint A is near the center, and for the distortion of the virtual image to become greater toward the periphery.

[0062] FIG. 1(B) shows an example of a virtual image V visually recognized by a driver through a windshield 2. In FIG. 1(B), the virtual image V having a rectangular outer shape is provided with, for example, 5 vertically and 5 horizontally, totaling 25 reference points (reference pixel points) G(i,j) (where i and j are both variables that can take values ​​of 1 to 5). Each reference point in the image (original image) is given a distortion with an inverse characteristic to the distortion caused in the virtual image V by the warping process, and therefore, ideally, the distortion given in advance and the distortion that actually occurs are offset, and ideally, a virtual image V without a curve as shown in FIG. 1(B) is displayed. The number of reference points G(i,j) can be appropriately increased by an interpolation process or the like. In FIG. 1(B), reference numeral 7 denotes a steering wheel.

[0063] Next, let us refer to Fig. 2. Fig. 2(A) is a diagram for explaining an overview of the viewpoint position tracking warping process, and Fig. 2(B) is a diagram showing an example of the configuration of an eyebox whose inside is divided into a plurality of partial regions. In Fig. 2, the same reference symbols are used for the parts common to Fig. 1 (this also applies to the following figures).

[0064] As shown in FIG. 2(A), the eyebox EB is divided into a plurality of (here, nine) partial regions Z1 to Z9, and the position of the driver's viewpoint A is detected for each of the partial regions Z1 to Z9.

[0065] Display light K of an image is emitted from the projection optical system 118 of the HUD device 100, a part of which is reflected by the windshield 2 and enters the driver's viewpoint (eye) A. When viewpoint A is within the eye box, the driver can visually recognize a virtual image of the image.

[0066] The HUD device 100 has a ROM 210, which incorporates an image conversion table 212. The image conversion table 212 stores warping parameters WP that determine polynomials, multipliers, constants, etc. for image correction (warping image correction) using a digital filter, for example. The warping parameters WP are provided corresponding to each of the partial regions Z1 to Z9 in the eye box EB. In FIG. 2(A), warping parameters WP(Z1) to WP(Z9) corresponding to each partial region are shown. Note that in the figure, only symbols WP(Z1), WP(Z4), and WP(Z7) are shown.

[0067] When viewpoint A moves, the position of viewpoint A among the partial regions Z1 to Z9 is detected. Then, one of the warping parameters WP(Z1) to WP(Z9) corresponding to the detected partial region is read from ROM 210 (updating the warping parameter), and warping processing is performed using the warping parameter.

[0068] Figure 2(B) shows an eyebox EB with more partial regions than the example in Figure 2(A). The eyebox EB is divided into 6 partial regions vertically and 10 horizontally, for a total of 60 partial regions. Each partial region is displayed as Z(X, Y), with its coordinate position in the X and Y directions as parameters.

[0069] Next, reference is made to Fig. 3. Fig. 3(A) to (F) are diagrams showing examples of virtual images distorted in different ways after warping processing. As described above, the appearance of virtual image V after warping processing differs depending on where driver's viewpoint A is located in the eyebox.

[0070] 3(A), the virtual image V displayed in the virtual image display area 5 of the windshield 2 is ideally displayed without distortion and without curvature. However, in actual virtual image V, distortion remains even after warping processing, and the degree and form of the distortion change depending on the position of the viewpoint A.

[0071] In the example of Fig. 3(B), although there is distortion, it is relatively minor, and the appearance of the virtual image V is close to that of Fig. 3(A). In the example of Fig. 3(C), the tendency of the distortion is similar to that of Fig. 3(B), but the degree of distortion is large, and the appearance is not similar to that of Fig. 3(A).

[0072] In addition, in the example of Figure 3(D), the degree of distortion is similar to that of Figure 3(C), but the manner in which the distortion occurs (the tendency of the distortion, or the manner in which the virtual image appears after the distortion occurs) is different from that of Figure 3(C).

[0073] In the example of Fig. 3(E), the degree of distortion is greater and there is no balance between the left and right of the virtual image V. In the example of Fig. 3(F), the virtual image V is distorted in a similar manner to Fig. 3(E), but the appearance is quite different from Fig. 3(F).

[0074] In this way, even virtual images V (virtual images V after warping processing) displaying the same content look quite different depending on the position of the viewpoint A. For example, when the viewpoint A is located in the center of the eyebox, the viewed virtual image V has relatively little distortion as shown in Fig. 3(B), but when the viewpoint A moves from the center to the periphery, the distortion becomes relatively large as shown in Fig. 3(E), for example.

[0075] In this state (the case of Figure 3(E) where viewpoint A is located in one partial area on the periphery of the eye box), if, for example, viewpoint A moves to another partial area and the appearance of virtual image V changes, for example, as shown in Figure 3(B) (change a1), or as shown in Figure 3(F) (change a2), in either case, the appearance changes significantly, increasing the likelihood that the driver (user) will feel uncomfortable.

[0076] Next, reference is made to Fig. 4. Fig. 4 is a diagram showing an example of viewpoint loss and redetection of the viewpoint position in an eyebox whose interior is divided into a plurality of partial regions. Fig. 4 illustrates each of viewpoint movements (1) to (6) as examples of how the viewpoint position is redetected after viewpoint loss occurs.

[0077] A typical example of viewpoint loss is when the driver's viewpoint A leaves the eyebox EB while driving, detection of its position is interrupted, and then viewpoint A returns to the inside of the eyebox EB. In FIG. 4, movements (1) to (6) are illustrated as examples of viewpoint movement in this case. In viewpoint movement (1), viewpoint A moves from inside the central region CT of the eyebox EB to outside the eyebox EB, and the movement distance is long and the time for which the viewpoint is lost is also large. In this case, even when viewpoint A returns to the inside of the eyebox EB, it can be assumed that it will be unstable (fluctuate a lot), for example, moving through movements (2) and (3) and then remaining at (4).

[0078] Also, viewpoint loss can occur in cases such as viewpoint movement examples (5) and (6) where viewpoint A does not go outside the eyebox EB but moves instantaneously through multiple partial regions. In viewpoint movement examples (5) and (6), the movement distance is shorter, the viewpoint loss time is shorter, and the viewpoint movement is relatively stable compared to the movement modes (1) to (4) above. There are various modes of viewpoint loss, and it is desirable to deal with them flexibly.

[0079] In this embodiment, the basic measure is to maintain the warping parameters immediately before the viewpoint loss occurs, to start the invalidation period from the timing when the viewpoint A is redetected, and to invalidate at least one warping process using the warping parameters corresponding to the redetected viewpoint position during the invalidation period. During the invalidation period, the warping parameters immediately before the viewpoint loss are maintained.

[0080] In addition, it may be possible to assume that warping parameters corresponding to the position of the center (symbol CP) of the eye box EB are adopted during the viewpoint lost period. In this case, the parameters are shifted from the parameters immediately before the viewpoint lost to the parameters corresponding to the center CP, and then shifted to the parameters corresponding to the position after redetection in a stepwise manner. Since there is a high possibility that the appearance of the virtual image will change due to the switching of the warping parameters, this is not adopted in this embodiment. As described above, during the viewpoint lost period and the subsequent invalidation period, the warping parameters immediately before the viewpoint lost are maintained to suppress the change in the appearance of the virtual image. By setting the invalidation period to an appropriate length, for example, only the warping associated with the redetection of the viewpoint A immediately after the viewpoint movement (2) in FIG. 4 can be invalidated, or further, the warping associated with the redetection of the viewpoint A immediately after the viewpoint movement (3) can also be invalidated (for example, the examples of FIG. 6 and FIG. 7).

[0081] In addition, during the invalidation period, a process for changing the update period of the warping parameters (a process for lengthening the update period) can also be used. In this case, an applied process can be used in which the period in which the parameter update period is increased is continued for a while even after the invalidation period ends, and an applied process can also be used in which the update period is gradually returned over time when returning it to its original state (examples of Figs. 7(A) to (D)).

[0082] Also, the invalidation period may be variably controlled in response to whether the viewpoint lost period is longer or shorter than a threshold value (threshold value for comparison and judgment). For example, after viewpoint loss due to viewpoint movement (5) in FIG. 4 (a relatively short movement of the viewpoint), it is possible to give the driver (user) a sense of security by making it appear that the viewpoint has been redetected (example of FIG. 8). Alternatively, conversely, it is possible to end invalidation relatively quickly and quickly perform warping processing using parameters according to the viewpoint position after the movement, thereby suppressing a redundant invalidation period (example of FIG. 9). Also, adaptive control based on the vehicle speed may be performed by changing the invalidation period according to the vehicle speed (example of FIG. 10). Details of these contents will be described later.

[0083] Next, reference will be made to Fig. 5. Fig. 5 is a diagram showing an example of a system configuration of a HUD device. A viewpoint detection camera 110 that detects the position of the driver's viewpoint A (eyes, pupils) is provided in a vehicle 1. In addition, an operation input unit 130 is provided in the vehicle 1 to enable the driver to set information required for the HUD device 100, and a vehicle ECU 140 that can collect various information about the vehicle 1 is also provided.

[0084] The HUD device 100 also includes a light source 112, a light projecting unit 114, a projection optical system 118, a viewpoint position detecting unit (viewpoint position determining unit) 120, a bus 150, a bus interface 170, a display control unit 180, an image generating unit 200, a ROM 210 incorporating an image conversion table 212, and a VRAM 220 that stores image (original image) data 222 and temporarily stores image data 224 after warping processing. The display control unit (display control device) 180 is composed of one or more processors, one or more image processing circuits, and one or more memories, and can control the HUD device 100 (display unit 116), for example, by generating and / or transmitting image data, by executing a program stored in the memory. The processor and / or image processing circuitry may include at least one general-purpose microprocessor (e.g., a central processing unit (CPU)), at least one application specific integrated circuit (ASIC), at least one field programmable gate array (FPGA), or any combination thereof. The memory may include any type of magnetic media, such as hard disks, any type of optical media, such as CDs and DVDs, any type of semiconductor memory, such as volatile memory, and non-volatile memory. Volatile memory may include DRAM and SRAM, and non-volatile memory may include ROM and NVRAM.

[0085] The viewpoint position detection unit 120 has a viewpoint coordinate detection unit 122, and an in-eyebox partial area detection unit 124 that detects (determines) in which partial area within the eyebox the viewpoint A is located, based on the detected coordinates.

[0086] The display control unit 180 also has a speed detection unit (which also serves as a low speed state determination unit) 182 that detects (determines) the speed of the vehicle 1, a warping control unit 184 (which has a warping management unit 185), a timer 190, a memory control unit 192, and a warping processing unit (warping image correction processing unit) 194.

[0087] The warping management unit 185 also includes a viewpoint loss detection unit (or viewpoint lost detection unit) 186 that detects the occurrence of viewpoint loss, a warping parameter switching delay unit (invalidation period setting unit) 187, a warping parameter update period change unit 188, and a temporary eyebox partial region storage unit 189 that temporarily stores information about the partial region of the eyebox corresponding to the detected viewpoint position.

[0088] Here, when the viewpoint position is redetected for the first time after the viewpoint is lost, the warping parameter switching delay unit (disablement period setting unit) 187 does not immediately switch the warping parameters based on the redetected viewpoint position at that timing, but instead temporarily delays the switching of the warping parameters, thereby implementing control to disable the switching of the warping parameters.

[0089] In addition, the warping parameter update period change unit 188 performs control to change the warping parameter update period (specifically, to lengthen the update period) at least during the invalidation period in parallel with the process of setting the invalidation period by delaying the warping parameter switching timing. By changing the update period, for example, it is possible to appropriately delay the timing of reflecting the updated warping parameters in the actual display.

[0090] The basic operation is as follows: That is, using partial area information (information indicating which partial area of ​​the eyebox the viewpoint A is in) sent from the viewpoint position detection unit 120 as an address variable, the memory control unit 192 accesses the ROM 210, reads out the corresponding warping parameters, the warping processing unit (warping image correction processing unit) 194 performs warping processing on the original image using the read warping parameters, the image generation unit 200 generates an image in a predetermined format based on the data after the warping processing, and the image is supplied to, for example, the light source 112, the light projection unit 114, etc.

[0091] However, if warping is performed simply following the movement of the viewpoint A, as described above, when the viewpoint position is redetected after the viewpoint is lost, the appearance of the virtual image V changes instantaneously, which may cause the driver to feel visual discomfort. Therefore, control is implemented to intentionally reduce (suppress) the sensitivity of the warping process. There are several modes of this control. Each will be explained in order below.

[0092] Please refer to FIG. 6. FIGS. 6(A) to 6(C) are timing charts showing an example of control for providing a period for invalidating warping processing based on a redetected viewpoint position. As shown in FIG. 6(A), in the period from time t10 to t11, the position of viewpoint A is in a partial region Z(n, m) of the eyebox EB (n, m are natural numbers specifying a partial region in the eyebox), viewpoint loss (viewpoint lost) occurs in the period from time t11 to t12, and thereafter, at time t12, viewpoint A is redetected to be in a partial region Z(r, s) of the eyebox EB (where r, s are natural numbers specifying a partial region in the eyebox, and the partial region specified here is different from Z(n, m)). The viewpoint loss period (viewpoint lost period) is denoted as T0.

[0093] Also, as shown in FIG. 6C, in this embodiment, the value of the update period of the warping parameters is fixed to RT1 and is not changed.

[0094] As shown by the solid line in Fig. 6(B), during the viewpoint lost period (time t11 to t12), the warping parameter is maintained at the parameter value WP1 immediately before the viewpoint lost occurs. As an alternative, as shown by the dashed line, it is possible to maintain the parameter value during the viewpoint lost period at a value corresponding to the center position of the eye box EB. However, in this case, if the driver is looking at a virtual image even during the viewpoint lost period, the appearance of the virtual image may change due to the change in the parameter value, which may cause discomfort, so this alternative is not adopted.

[0095] Furthermore, at time t12, the position of viewpoint A is redetected, but parameters are not immediately switched to those corresponding to the redetected position, and instead the parameter value WP1 is maintained for a predetermined period from redetection point t12 (here, the period until time t13), and at time t13, the parameter value is changed to a value WP2 based on the redetected position. The period from times t12 to t13 is the invalidation period Ta, and when the viewpoint position is redetected at least once during this invalidation period Ta, the change in parameters based on the redetected position (application of parameters) is invalidated, and warping is performed using the original parameters that are maintained.

[0096] By providing the invalidation period, the parameters are fixed for a short time, and the warping parameters are not switched instantly. Also, during the invalidation period, even if the viewpoint position is unstable and moves through multiple partial regions of the eye box, the parameters are not changed based on continuous redetection of the viewpoint position. This stabilizes the warping process. Therefore, for example, when the driver's viewpoint A leaves the eye box and then returns to the eye box, the appearance of the virtual image V changes instantly, suppressing the occurrence of discomfort.

[0097] Next, reference will be made to Fig. 7. Fig. 7(A) to (D) are timing charts showing another example of control for providing a period during which warping processing based on a redetected viewpoint position is invalidated (such as when parameter update period change processing is used in combination).

[0098] As shown in FIG. 7(A), at times t1 to t3, a loss of view (viewpoint loss) occurs. The loss-of-view period (viewpoint-loss period) is denoted as T1. Also, as shown in FIG. 7(A), when the loss-of-view period T1 is compared with a predetermined threshold value (threshold value for determining the length of the loss-of-view period) Th, it is equal to or greater than the threshold value Th. In other words, in the example of FIG. 7, Th ≦ T1 holds (however, in FIG. 7(A), a case where Th < T1 is shown as a specific example).

[0099] Also, as shown in FIG. 7(B), the period from time t3 to t4 is an invalidation period Ta. The warping parameter WP1 may instantaneously switch to WP2 at time t4, or may gradually switch over time as shown by the characteristic lines Tk1 or Tk2 of the broken line. Note that Tk1 and Tk2 correspond to the characteristic lines G1 and G2 in FIG. 7(D) (described later). When switching of parameters such as the characteristic lines Tk1 and Tk2 is performed, the point in time when the parameter value has completed switching to WP2 is time t5, which is a point in time further delayed by a time (period) Tb from time t4 (this point will also be described later).

[0100] Also, in the example of FIG. 7, the change process of the update period of the warping parameter is also mentioned. FIG. 7(C) shows a case where the update period of the warping parameter is fixed to RT1 and no change in the update period is made. FIG. 7(D) shows that during the invalidation period Ta from time t3 to t4, a change process (used in combination) of the update period of the warping parameter is performed to change the update period of the warping parameter from RT1 to RT2 (specifically, to increase the update period).

[0101] For example, if the frame rate of an image (virtual image) is 60 fps (frames per second), image processing (image display processing) of 60 frames is performed in one second (in other words, one frame period is 1 / 60 second). As an example, assume a case where it is normal to update the warping parameter every frame.

[0102] Here, in the invalidation period Ta after the viewpoint is lost, if the parameter is updated every two frames, the update period will be 2 / 60 seconds, and if it is updated every three frames, the update period will be 3 / 60 seconds, and the update period will be longer. In this way, by switching to an update in units of multiple frames, it is possible to lengthen (increase) the update period. By lengthening the update period, the reflection of the updated warping parameters in the image (virtual image) will be delayed. In other words, the sensitivity of the reflection of the updated parameters in the display will be reduced. After the invalidation period has passed, the update period of the changed parameters will be returned to the original (returned from RT2 to RT1), but in reality, the return of the update period is not instantaneous and requires a certain amount of time, so even if the warping parameters are switched, the reflection of the switched parameters in the actual display will be delayed.

[0103] This makes it easy to appropriately provide a certain amount of delay (a delay long enough that the driver can perceive a delayed change in the appearance of the actual display (in other words, a delay that has the effect of substantially slightly extending the width of the invalidation period)). It is also expected to have the effect of making it easier to design the timing control circuit.

[0104] In addition, the range of control variations is expanded, and flexible responses are possible, such as variably controlling the degree of increase in the parameter update cycle, or devising the timing for returning the increased update cycle to its original state, etc. It also becomes easy to set the amount of delay in actual display control over a fairly wide range.

[0105] A modified example of the timing for returning the increased update cycle to its original state is shown in FIG. 7(D). In FIG. 7(D), as shown by the dashed (thick) characteristic line G1, the time point at which the update cycle is returned to its original state may be changed from time t4 to time t5. In this case, the period during which the sensitivity of the parameters to be reflected in the display is reduced will be extended. In this case, in addition to delaying the parameter switching to provide an invalidation period, the timing for returning the update cycle to its original state (returning from RT2 to RT1) is delayed to further delay the reflection of the updated parameters to the actual display, making it easier to create the necessary delay and reducing the burden on the timing circuit, etc.

[0106] Also, as shown by the dashed (thin) characteristic line G2 in Fig. 7(D), the process of returning the update period to the original state starts at time t4, but the update period may be returned little by little thereafter with some time to spare. For example, when returning the parameter update period from 1 / 15 second (=RT2) to 1 / 60 second (=RT1), the period is not returned immediately, but is controlled to be gradually switched in stages, such as 1 / 30 second, 1 / 45 second, and 1 / 60 second in units of a predetermined time. By controlling the update period to be gradually switched on the time axis, the delay in reflecting the changed parameters in the display can be managed with higher accuracy.

[0107] Next, reference is made to Fig. 8. Fig. 8(A) and (B) are timing charts showing another example of control for providing a period during which warping processing based on a redetected viewpoint position is disabled (a first control example when the viewpoint lost period is shorter than a threshold value).

[0108] In the example of FIG. 8, the control unit (symbol 184 or 185 in FIG. 5) measures the viewpoint lost time (viewpoint lost time: sometimes simply referred to as lost time) during which the viewpoint is lost, for example using a timer 190, and when the viewpoint lost time is shorter than a predetermined threshold value Th, performs control to make the invalidation period (invalidation period) longer than when the lost time is longer than the threshold value Th (for example, the example of FIG. 7).

[0109] In Fig. 8, viewpoint lost time T10 (times t1 to t6) is less than threshold value Th. Viewpoint lost occurs at time t1, and the position of viewpoint A is redetected at time t6. In the example of Fig. 7 described above, an invalidation period Ta is provided after redetection, but in the example of Fig. 8, the invalidation period is further extended by a period Td. The invalidation period becomes a period Te (=Ta+Td).

[0110] By setting a longer period during which the warping parameters are fixed and the appearance of the image (virtual image) is kept constant, the driver can be informed that redetection after the viewpoint loss has been successful and that the corresponding processing is now being carried out. In other words, by lengthening the period during which the warping parameters are fixed and performing image processing using warping parameters updated over time, even if the appearance of the image (virtual image) changes, the driver can be made to recognize that the change does not occur suddenly in conjunction with the movement of the driver's eyes, but that it has been changed with a certain amount of time.

[0111] This makes it easier for the driver to intuitively know that although the driver lost track of his or her viewpoint due to the movement of his or her eyes, the HUD device system successfully redetected the viewpoint and performed processing to address the loss of viewpoint.

[0112] In other words, it is possible to convey to the user (the driver) that the HUD device (system) is properly processing the situation after the viewpoint is lost. This gives the driver a sense of security and mental stability, and therefore has the effect of making it difficult for discomfort to occur or reducing the discomfort.

[0113] Next, reference is made to Fig. 9. Fig. 9(A) and (B) are timing charts showing another example of control for providing a period during which warping processing based on a redetected viewpoint position is disabled (a second control example when the viewpoint lost period is shorter than a threshold value).

[0114] In the example of FIG. 9, the control unit (symbol 184 or 185 in FIG. 5) measures the viewpoint lost time (viewpoint lost time: sometimes simply referred to as lost time) using, for example, a timer 190, and when the viewpoint lost time is shorter than a predetermined threshold value Th, performs control to shorten the invalidation period (invalidation period) compared to when the lost time is longer than the threshold value Th (for example, the example of FIG. 7). The direction of control in FIG. 9 is opposite to that in FIG. 8. However, since the effects obtained are different, each of the examples of FIG. 8 and FIG. 9 can be selectively applied depending on the expected effect.

[0115] When the viewpoint lost time is short, it is considered that the change in viewpoint position is small (the moving distance of the viewpoint is relatively short), so in the example of FIG. 9, the invalidation time of the warping process using the new warping parameters is set shorter than when the viewpoint lost time is longer than the threshold Th (example of FIG. 7). As shown in FIG. 9(B), the warping parameters are switched from WP1 to WP2 at time t9. The period Tf from time t6 to t9 is the invalidation period. The invalidation period Tf in FIG. 9 is set shorter than the invalidation period Ta in FIG. 7.

[0116] This makes it possible, for example, to perform the minimum necessary invalidation (timing delay) and then quickly display an appropriate warping-corrected image (virtual image) according to the viewpoint position, thereby reducing the sense of discomfort or suppressing the occurrence of discomfort.

[0117] In other words, when the viewpoint lost period is short, it is estimated that the distance moved by the viewpoint position is small, and therefore it can be estimated that there is little difference in the manner of distortion of the virtual image before and after the warping parameters are updated. Taking this into consideration, abrupt (in other words, in a fairly short period of time) changes in the appearance of the virtual image immediately after redetection of the viewpoint position are prevented, and then normal viewpoint tracking warping control is quickly restored, thereby reliably achieving an improvement in visibility.

[0118] Next, reference is made to Fig. 10. Fig. 10 is a diagram showing an example of characteristics in a case where the period during which the warping process based on the redetected viewpoint position is invalidated is variably controlled according to the vehicle speed. In the example of Fig. 10, the invalidation period after the viewpoint is lost is adaptively controlled according to the vehicle speed of the vehicle 1.

[0119] 5 described above also functions as a low speed state determination unit. When the low speed information determination unit 182 determines (for example, using a vehicle speed determination threshold) that the vehicle 1 is in a low speed state (stopped state or running state at a low speed), the control units (184, 185) perform control to make the period during which the warping process using the new parameters is invalidated (invalidation period) longer than the invalidation period in a state faster than the low speed state.

[0120] In Figure 10, when the vehicle speed is in a low speed state of 0 to U1, the values ​​of the invalidation periods Ta, Te, and Tf (corresponding to Figures 7(B), 8, and 9, respectively) are N1, and when the vehicle speed is in a medium speed state of U1 to U2, the values ​​are smaller than N1, and the same is true when the vehicle speed is in a high speed state of U2 or above.

[0121] When the vehicle is traveling at a low speed, the driver (user) is sensitive to changes in vision, such as the view ahead, and is easily able to detect such changes. Therefore, at this time, the reflection of the new parameters in the image after the viewpoint is lost is delayed significantly, and measures are taken to prevent discomfort caused by an instantaneous change in the appearance of the display. When the vehicle speed leaves the low speed state and becomes faster, the invalidation periods Ta, Te, and Tf are shortened (this may include the case where the invalidation periods are eliminated), and control is implemented that emphasizes speeding up the correction of image distortion based on the viewpoint position that is redetected after the viewpoint is lost. This enables more flexible and appropriate warping control that corresponds to the vehicle speed.

[0122] In the control example of FIG. 10, the control unit (184 or 185 in FIG. 5) changes the period during which the warping process is disabled (disabled period) according to the vehicle speed of the vehicle 1. In this case, when the speed of the vehicle 1 is in a range of a first speed value U1 (U1>0) or more and a second speed value U2 or less that is greater than the first speed value, the control is performed to reduce the period during which the warping process is disabled (disabled period) relative to the vehicle speed as the vehicle speed increases (this is the control shown by the characteristic lines Q2, Q3, and Q4, and this is referred to as the first control). In addition, in this case, when the vehicle speed is in a range of less than the first speed U1 and more than the second speed U2, the control is not performed and the value of the disabled period is fixed to N1 or N2. This makes it possible to prevent an excessive burden from being placed on the system of the HUD device.

[0123] Furthermore, when the control shown by characteristic line Q3 is implemented, the degree of reduction in the invalidation period is made gentle when the vehicle speed is in the range close to the first speed value U1, and the degree of reduction becomes steeper as the vehicle speed moves away from the first speed value U1.

[0124] In other words, when the vehicle is in a low-speed state where the driver is likely to perceive visual changes in the image (virtual image) (in other words, when the vehicle speed is in a range close to the first speed value U1), control is implemented to gradually reduce the invalidation period so as to suppress abrupt updates to the warping parameters (this is referred to as the second control). In this case, more accurate control is achieved.

[0125] In addition, when the control shown by the characteristic line Q4 is performed, in the range where the vehicle speed is close to the first speed value U1, the degree of decrease is made gentle when decreasing the invalidation period, and as the vehicle speed moves away from the first speed value, the degree of decrease is made steeper, and as the vehicle speed approaches the second speed value U2, the degree of decrease is made gentler (control with an inverse S-shaped characteristic, which is called the third control). In this third control, in addition to the second control described above, the degree of decrease of the invalidation period is made gentler as the vehicle speed approaches the second speed value U2, and the decrease is stopped and becomes constant when the second speed value U2 is reached, thereby suppressing the sudden (abrupt) plateau of the change and the occurrence of discomfort. This can further improve the visibility of the virtual image.

[0126] Next, Fig. 11 is referred to. Fig. 11 is a flow chart showing an example of a procedure of warping image correction control corresponding to viewpoint loss (first control example: corresponding to Fig. 6 and Fig. 7). The viewpoint position is monitored (step S1), and the presence or absence of viewpoint loss (viewpoint lost) is judged (step S2).

[0127] If the answer is N in step S2, the process returns to step S1. If the answer is Y, the warping parameters immediately before the viewpoint was lost are maintained (step S3). Then, it is determined whether the viewpoint position has been redetected after the viewpoint was lost (step S4). If the answer is N, the process returns to step S3, and if the answer is Y, the process proceeds to step S5.

[0128] In step S5, a delay process (invalidation process) of updating (switching) to the warping parameters corresponding to the redetected viewpoint position is performed, thereby invalidating at least one warping process using the parameters corresponding to the redetected viewpoint position. At this time, a parameter update period change process (processing in FIG. 7(D)) that lengthens the parameter update period may be used in combination.

[0129] In step S6, it is determined whether or not a predetermined time (invalidation period) Ta has elapsed. If the answer is No, the process returns to step S5, and if the answer is Yes, the process proceeds to step S7.

[0130] In step S7, the warping parameters are updated (switched) to those corresponding to the viewpoint position after redetection. In principle, the invalidation period ends at this time (however, when the control of the characteristic lines Tk1 and Tk2 in FIG. 7(B) is performed, it is also possible to design the control unit so that the substantial invalidation period is extended until the timing when the parameters are completely switched). Here, if the parameter update period has not been changed in step S5, the process in step S7 ends and the process proceeds to step S8.

[0131] Furthermore, if the parameter update period has been changed in step S5, a process of restoring the parameter update period to the original period is also performed in step S7. As a method for restoring the period, the three methods shown in Fig. 7(B) (any of the following (1) to (3)) can be considered. (1) The parameter update period is restored at the timing of parameter switching (in other words, in synchronization with parameter switching) (the process indicated by the solid line in FIG. 7B). (2) The parameter update period is temporarily maintained even after the parameters are switched, and then returned to the original period (processing according to the characteristic line Tk1 in FIG. 7(B)). (3) The value of the parameter update period is returned to its original value while being changed over time (processing according to the characteristic line Tk2 in FIG. 7(B)).

[0132] Then, in step S8, it is determined whether or not to end the image correction. If the answer is Y, the image correction ends, and if the answer is N, the process returns to step S1.

[0133] Next, reference is made to Fig. 12. Fig. 12 is a flowchart showing an example of a procedure for warping image correction control in response to viewpoint loss (third control example: corresponding to Figs. 8 and 9). In Fig. 12, steps S4-1 and S4-2 indicated by thick lines in the figure are added in addition to the procedure in Fig. 11. Since the rest is the same as Fig. 11, a description of the common processing will be omitted.

[0134] In step S4-1, it is determined whether the viewpoint loss period is shorter than the threshold value. If it is N, the process proceeds to step S5.

[0135] If it is Y, in step S4-2, Te (> Ta) is adopted as the invalidation period (the delay time for parameter switching) (in the case of FIG. 8), or Tf (< Ta) is adopted (in the case of FIG. 9). Note that in step S6, it is determined whether the time corresponding to either Te or Tf has elapsed.

[0136] Next, referring to FIG. 13. FIG. 13 is a flowchart showing an example of the procedure for warping image correction control corresponding to viewpoint loss (the third control example: corresponding to FIG. 10).

[0137] In step S10, the vehicle speed is detected. In step S11, the running state of the vehicle (including stop) is determined. For example, discrimination of each state such as low speed, medium speed, and high speed can be performed.

[0138] In step S12, the viewpoint is monitored, and in step S13, it is determined whether there is a viewpoint loss. If it is N, the process returns to step S12, and if it is Y, the process proceeds to step S14.

[0139] In step S14, warping processing such as control example 1 in FIG. 11 or control example 2 in FIG. 12 described above is performed. Subsequently, in step S15, it is determined whether to end the image correction. If it is Y, the process ends, and if it is N, the process returns to step S10.

[0140] Next, referring to FIG. 14. FIG. 14(A) is a diagram showing an example of display by a road surface superimposed HUD, FIG. 14(B) is a diagram showing an example of display by an inclined surface HUD, and FIG. 14(C) is a diagram showing a configuration example of the main part of the HUD device.

[0141] Figure 14(A) shows an example of virtual image display using a road surface overlay HUD, in which a virtual virtual image display surface PS corresponding to the image display surface (reference number 117 in Figure 5, or reference number 163 in Figure 14(C)) of a display unit (reference number 116 in Figure 5, or reference number 161 in Figure 14(C)) is positioned so as to be overlaid on the road surface 41 in front of the vehicle 1.

[0142] Figure 14 (B) shows an example of virtual image display using an inclined surface HUD, in which the virtual image display surface PS is positioned at an angle with respect to the road surface 41 so that the distance between the near end, which is the end of the virtual image display surface PS closer to the vehicle 1, and the road surface 41 is small, and the distance between the far end, which is the end farther from the vehicle 1, and the road surface 41 is large.

[0143] These devices use a wide virtual image display surface PS superimposed on the road surface 41 or a wide virtual image display surface PS inclined to the road surface 41 to perform various displays, for example, within a range of 5m to 100m ahead of the vehicle 1, and it is preferable that the HUD device and eye box EB are enlarged to detect the viewpoint position with high accuracy over a wider range than before and perform image correction using appropriate warping parameters. However, when viewpoint loss occurs, the highly accurate warping parameter switching control may actually reduce the visibility of the image (virtual image) after the viewpoint is redetected. Therefore, the application of the control method of the present invention is effective.

[0144] Next, Fig. 14(C) will be referred to. HUD device 107 in Fig. 14(C) has control unit 171, light projecting unit 151, screen 161 as a display unit having image display surface 163, reflecting mirror 133, curved mirror (concave mirror or the like) 131 whose reflecting surface is designed as a free-form surface, and actuator 173 that drives display unit 161.

[0145] In the example of FIG. 14(C), when the HUD device 107 adjusts the position of the eye box EB in accordance with the height position of the driver's viewpoint A, it does not move the curved mirror 131, which is an optical member that projects the display light onto the windshield 2 (no actuator is provided for the curved mirror 131), but instead changes the reflection position of the display light 51 of the image on the optical member 131.

[0146] In other words, when adjusting the height position of the eye box EB according to the height position of the driver's eyes (viewpoint A), the optical member 131 that projects light onto the projection target member 2 is not rotated by, for example, an actuator, but is instead adjusted by changing the reflection position of the light on the optical member 131. Note that the height direction is the Y direction in the figure (the direction along the perpendicular to the road surface 41, and the direction away from the road surface 41 is the positive direction). Note that the X direction is the left-right direction of the vehicle 1, and the Z direction is the front-rear direction (or forward direction) of the vehicle 1.

[0147] Recent HUD devices have tended to be developed on the premise that a virtual image is displayed over a fairly wide range in front of the vehicle, and in this case, the device inevitably becomes large. Naturally, the optical member 131 also becomes large. If the optical member 131 is rotated using an actuator or the like, the error may actually reduce the accuracy of control of the height position of the eye box EB. To prevent this, the position where the light beam is reflected by the reflecting surface of the optical member 131 is changed.

[0148] In such a large optical element 131, efforts are made to prevent distortion of the virtual image as much as possible by optimally designing the reflective surface as a free-form surface, etc. However, as described above, in cases such as when the driver's viewpoint A is located near the eye box EB, distortion may inevitably become apparent.

[0149] Therefore, in such a case, by implementing control to temporarily disable (delay) the application of parameters corresponding to the viewpoint position after the above-mentioned redetection within a specified range, it is possible to reduce the discomfort caused by changes in appearance due to distortion of the virtual image, and by effectively utilizing the above-mentioned control, the visibility of the virtual image can be improved.

[0150] As described above, according to the present invention, when implementing viewpoint position tracking warping control that updates warping parameters according to the driver's viewpoint position, it is possible to effectively prevent the appearance of the image from instantly changing due to the update of the warping parameters after a loss of viewpoint (viewpoint lost), which may cause the driver to feel uncomfortable.

[0151] The present invention can be used in both monocular HUD devices, in which display light for the same image is incident on each of the left and right eyes, and parallax HUD devices, in which images with parallax are incident on each of the left and right eyes.

[0152] In this specification, the term "vehicle" may be broadly interpreted as a vehicle. In addition, terms related to navigation (e.g., signs, etc.) are also broadly interpreted, taking into consideration the perspective of navigation information in a broad sense that is useful for driving a vehicle. In addition, the HUD device includes those used as simulators (e.g., aircraft simulators).

[0153] The present invention is not limited to the above-described exemplary embodiments, and those skilled in the art could easily modify the above-described exemplary embodiments to the extent that they fall within the scope of the claims. [Explanation of symbols]

[0154] 1...vehicle (own vehicle), 2...projected member (reflective translucent member, windshield, etc.), 4...projection area, 5...virtual image display area, 7...steering wheel, 51...display light, 100...HUD device, 110...viewpoint detection camera 110, 112...light source, 114...light projecting unit, 116...display unit, 117...display surface (image display surface), 118...projection optical system, 120...viewpoint position detection unit (viewpoint position determination unit), 122... viewpoint coordinate detection unit, 124... partial area detection unit within eye box, 130... operation unit, 131... curved mirror (concave mirror, etc.), 133... reflecting mirror (including reflecting mirror, correction mirror, etc.), 140... vehicle ECU, 150... bus, 151... light source unit, 161... display unit (screen, etc.), 63... display surface (image display surface), 170... bus interface, 171... control unit , 173... actuator for driving a display unit, 180... display control unit (display control device), 182... speed detection unit, 184... warping control unit, 185... warping management unit, 186... viewpoint loss (viewpoint lost) detection unit, 187... warping parameter switching delay unit (invalidation period setting unit), 188... warping parameter update period change unit, 189... one of partial area information of an eye box time storage unit, 192...memory control unit, 194...warping processing unit, 200...image generation unit, 210...ROM, 220...VRAM, 212...image conversion table, 222...image (original image) data, 224...image data after warping processing, EB...eye box, Z (Z1 to Z9, etc.)...partial region of the eye box, WP...warping parameter, PS...virtual image display surface, V...virtual image

Claims

1. A display control device that is mounted on a vehicle and controls a head-up display (HUD) device that projects an image to be displayed on a display unit onto a projection member provided in the vehicle, thereby allowing a driver to visually recognize a virtual image of the image, a control unit that updates a warping parameter in accordance with a viewpoint position of a driver in an eyebox and performs a viewpoint-position tracking warping control by using the warping parameter to pre-distort an image to be displayed on the display unit so that the image has a distortion characteristic opposite to a distortion characteristic of a virtual image of the image; The control unit is When a viewpoint loss is detected in which the position of at least one of the right and left viewpoints of the driver becomes unclear, during a viewpoint lost period, the warping parameters set immediately before the viewpoint lost period are maintained; when the position of the viewpoint is redetected after the viewpoint lost period, invalidating at least one warping process using a warping parameter corresponding to the redetected viewpoint position; A display control device comprising:

2. The control unit is The viewpoint lost period is compared with a threshold value, and if the viewpoint lost period is shorter than the threshold value, A control is performed to make a period during which the warping process is disabled longer than a period during which the viewpoint lost period is longer than the threshold value. The display control device according to claim 1 .

3. The control unit is The viewpoint lost period is compared with a threshold value, and if the viewpoint lost period is shorter than the threshold value, A control is performed to make a period during which the warping process is disabled shorter than a period during which the viewpoint lost period is longer than the threshold value. The display control device according to claim 1 .

4. The control unit is a warping parameter update period before the viewpoint loss occurs and during the viewpoint lost period is defined as a first update period RT1, and a warping parameter update period during a period in which the warping process is disabled is defined as a second update period RT2, the parameter update period is changed so that RT1<RT2.

4. The display control device according to claim 1, wherein the display control device is a display control device.

5. The control unit is After changing the parameter update period from the first update period RT1 to the second update period RT2, At the end timing of the period during which the warping process is disabled, the second update period RT2 is returned to the first update period RT1. Or, returning the second update period RT2 to the first update period RT1 at a timing when a predetermined time has elapsed from the end timing of the period during which the warping process is disabled; Or, starting to change the parameter update period from the end timing of the period during which the warping process is disabled as a starting point, and gradually returning the parameter update period from the second update period RT2 to the first update period RT1 over time; 5. The display control device according to claim 4.

6. and a low speed state determination unit that determines whether the speed of the vehicle is in a low speed state. The control unit is a period during which the warping process is disabled when the vehicle is in the low speed state including a stopped state is set to be longer than a period during which the warping process is disabled when the vehicle is in a speed state faster than the low speed state; 6. The display control device according to claim 1, wherein the display control device is a display control device.

7. The control unit is The period during which the warping process is disabled is changed according to the vehicle speed. In this case, When the speed of the vehicle is within a range of a first speed value U1 (U1>0) or more and a second speed value U2 that is greater than the first speed value, a period during which the warping process is disabled is controlled to be decreased with respect to the vehicle speed as the vehicle speed increases. Or, A control is implemented in which the degree of the decrease is made gentle when the vehicle speed is in a range close to the first speed value, and the degree of the decrease is made steeper as the vehicle speed moves away from the first speed value. Or, A control is implemented in which the degree of the decrease is made gentle in a range in which the vehicle speed is close to the first speed value, and the degree of the decrease is made steeper as the vehicle speed moves away from the first speed value, and the degree of the decrease is made gentler as the vehicle speed approaches the second speed value.

6. The display control device according to claim 1, wherein the display control device is a display control device.

8. The head-up display device includes: an optical system including an optical member that reflects the display light of the image and projects the image onto the projection target; A display control device as described in any one of claims 1 to 7, characterized in that when adjusting the position of the eye box in accordance with the height position of the driver's viewpoint, the optical element is not moved, and the reflection position of the display light of the image on the optical element is changed.

9. A virtual image display surface corresponding to the image display surface of the display unit is disposed so as to be superimposed on a road surface in front of the vehicle. Or, the virtual image display surface is disposed at an oblique angle with respect to the road surface such that a distance between a near end portion, which is an end portion close to the vehicle, and the road surface is small, and a distance between a far end portion, which is an end portion farther from the vehicle, and the road surface is large.

9. The display control device according to claim 1, wherein the display control device is a display control device.

10. A display control device according to any one of claims 1 to 9, A display unit for displaying an image; an optical system including an optical member that reflects display light of the image and projects it onto the projection target member,

Images