Display control device, head-up display device

Abstract

The present invention provides a display control device that suppresses the loss of the driver's viewing position when performing viewing position following deformation control for updating deformation parameters corresponding to the driver's viewing position. When the control unit (180) that performs the viewing position following deformation control detects a viewing point loss where the position of at least one of the driver's left and right viewing points is unknown, the deformation parameters set before the viewing point loss period are maintained during the viewing point loss period. When the position of the viewing point is detected again after the viewing point loss period, at least one deformation process using the deformation parameters corresponding to the viewing point position detected again is invalidated.

Description

Technical Field

[0001] The present invention relates to a head-up display (HUD) device, a display control device, etc., which project (cast) the display light of an image onto a projection member such as a windshield or a combiner of a vehicle and display a virtual image in front of a driver or the like. Background Art

[0002] In an HUD device, an image correction process (hereinafter referred to as a distortion process) is known, which preliminarily distorts a projected image to have characteristics opposite to the distortion of a virtual image caused by the curved surface shape of an optical system, a windshield, or the like. For example, the distortion process in an HUD device is described in Patent Document 1.

[0003] In addition, a distortion process based on the viewpoint position of a driver (viewpoint following distortion) is described in, for example, Patent Document 2.

[0004] Prior Art Documents

[0005] Patent Documents

[0006] Patent Document 1: Japanese Patent Application Laid-Open No. 2015-87619

[0007] Patent Document 2: Japanese Patent Application Laid-Open No. 2014-199385 Summary of the Invention

[0008] The inventors have studied a viewpoint position following distortion control that updates distortion parameters according to the viewpoint position of a driver (operator, crew member, etc., which can be widely interpreted), and have recognized the following new problems.

[0009] There are sometimes cases where the viewpoint position of a driver moves, the HUD device temporarily loses the viewpoint position, and then the viewpoint position is detected again, and the distortion parameters are updated according to the detected viewpoint position again.

[0010] Here, the image (virtual image) displayed after the distortion process is ideally a flat virtual image in which the distortion caused by an optical system or the like is completely corrected. In the viewpoint position following distortion, it is ideal to always obtain a virtual image without distortion even when the position of the driver's (user's) viewpoint changes, but the distortion cannot be completely removed.

[0011] Therefore, after the loss (loss) of the viewpoint position occurs, if a simple viewpoint position following distortion is performed based on the re-detected viewpoint position, even when displaying a virtual image of the same image, the appearance of the virtual image observed by the driver (the appearance of the virtual image, the impression received from the virtual image, etc.) will instantaneously change, and sometimes it will cause a sense of discomfort to the driver.

[0012] In addition, as a way of losing the viewing point, typically, it can be conceived that the viewing point moves outside the viewing area for some reason and then returns to the inside of the viewing area, but it is not limited to this, and there can also be a situation where the driver's viewing point instantaneously moves within the viewing area. Even when the viewing point is lost, the ways are diverse. As needed, it is also important to adopt countermeasures considering the situation of losing the viewing point.

[0013] Moreover, in recent years, for example, HUD devices capable of performing virtual image display in a quite large range in front of the vehicle have been developed, and such HUD devices tend to be enlarged. Although research has been conducted on the design of the optical system and the like to reduce distortion, for example, it is difficult to achieve the same distortion reduction effect uniformly throughout the entire area of the viewing area. For example, it can also be conceived that the following situation occurs: when the driver's viewing point is located in the central area of the viewing area, the degree of distortion can be suppressed considerably, but when the viewing point is located at the periphery of the viewing area, the remaining degree of distortion becomes somewhat larger. This also becomes one of the reasons for a significant change in the presentation manner of the virtual image based on the deformation process after the viewing point position is lost.

[0014] One of the objects of the present invention is, in a HUD device, when performing viewing point position following deformation processing and when performing viewing point position following deformation control for updating deformation parameters according to the driver's viewing point position, when the driver's viewing point position is lost and then the viewing point position is detected again, to suppress the instantaneous change in the appearance of the image accompanying the update of the deformation parameters and cause discomfort to the driver.

[0015] Other objects of the present invention will be apparent to those skilled in the art by referring to the following exemplified modes and preferred embodiments and the drawings.

[0016] Hereinafter, in order to easily understand the outline of the present invention, modes according to the present invention are exemplified.

[0017] In a first mode, a display control device includes: a display unit mounted on a vehicle and displaying an image; and an optical component that reflects display light of the image and projects it onto the projection component. The display control device controls a head-up display (HUD) device, and the head-up display device enables the driver to visually recognize a virtual image of the image by projecting the image onto the projection component provided in the vehicle.

[0018] The display control device has a control unit for performing viewing point position following deformation control that updates deformation parameters according to the driver's viewing point position in the viewing area and uses the deformation parameters to deform the image displayed on the display unit in a manner having characteristics opposite to the distortion characteristics of the virtual image of the image in advance.

[0019] When the control unit detects the loss of the position of at least one of the left and right viewpoints of the driver, during the period of viewpoint loss, the deformation parameters set before the period of viewpoint loss are maintained.

[0020] When the position of the viewpoint is detected again after the period of viewpoint loss, at least one deformation process using the deformation parameters corresponding to the position of the viewpoint detected again is invalidated.

[0021] In the first mode, during the period of viewpoint loss (which may also be described as loss of viewpoint or loss of viewpoint position), the previous deformation parameters are maintained. And at the moment when the viewpoint position is detected again, instead of immediately performing the deformation based on the new deformation parameters, the implementation is delayed.

[0022] In other words, when the control unit detects the position of the viewpoint again after the viewpoint loss, at least one deformation process using the deformation parameters corresponding to the position of the viewpoint detected again is invalidated (setting during the invalidation period after redetection), and then the deformation process using the deformation parameters corresponding to the position of the viewpoint after the end of the invalidation period is validated (implemented).

[0023] For example, assume a case where the viewpoint is divided into multiple partial regions and the position of the viewpoint is detected in units of each partial region. For example, even if it is determined that the viewpoint was located in "partial region A" before the viewpoint loss and the viewpoint loss occurs, and then at the moment when the position of the viewpoint is detected again, the viewpoint moves to "partial region B", the deformation process using the deformation parameters corresponding to the position of "partial region B" is invalidated (invalidated at least once). When the viewpoint position further moves from partial region B to partial region C, the deformation process using the deformation parameters corresponding to the position of this partial region C can also be invalidated again (second invalidation).

[0024] In addition, it is also possible to appropriately determine the number of invalidations in consideration of the mode of viewpoint loss (for example, the length of the period of viewpoint loss) and the driving state of the vehicle (for example, vehicle speed), etc.

[0025] In the second mode subordinate to the first mode,

[0026] when the control unit compares the viewpoint loss time with a threshold value and the viewpoint loss time is shorter than the threshold value,

[0027] the following control may also be implemented: compared with the case where the viewpoint loss time is longer than the threshold value, the period during which the deformation process is invalidated is extended.

[0028] In the second mode, the control unit measures the time when the viewpoint position is lost (lost time or elapsed time), and when the lost time is shorter than a specified threshold value, it performs control with a longer invalidation period.

[0029] In the second mode, by setting a longer period during which the appearance of the fixed image (virtual image) without changing the deformation parameters remains constant, it is possible to indicate the success of redetection after the driver's viewpoint is lost and make the driver recognize that the corresponding processing is currently being performed. In other words, by extending the fixed period of the deformation parameters and performing image processing with the deformation parameters updated over time, even when the appearance of the image (virtual image) changes, the driver is made to recognize that this change does not occur suddenly with the movement of the driver's eyes, but is a change with more time available.

[0030] As a result, it is easy for the driver to perceive that although the viewpoint position is lost due to eye movement, the HUD device system has successfully redetected the viewpoint position and performed processing corresponding to the viewpoint loss.

[0031] In other words, on the HUD device side (system side), it is possible to display to the driver as the user that the processing after the viewpoint loss is correctly performed. This gives the driver a sense of security and mental stability, thereby achieving the effect of being less likely to generate a sense of discomfort or reducing the sense of discomfort.

[0032] In the third mode subordinate to the first mode,

[0033] when the control unit compares the viewpoint loss time with the threshold value and the viewpoint loss time is shorter than the threshold value,

[0034] it is also possible to perform the following control: compared with the case where the viewpoint loss time is longer than the threshold value, shorten the period during which the deformation processing is invalidated.

[0035] In the third mode, the control unit measures the loss time (elapsed time) of the viewpoint position, and when the loss time is shorter than a specified threshold value, it performs control with a shorter invalidation period. Although the direction of control is opposite to that of the second mode, since the effects obtained by the second and third modes are different, each mode can be selectively applied according to the desired effect. Considering that when the viewpoint loss time is short, the change in the viewpoint position is small (the moving distance of the viewpoint is relatively short), the invalidation time of the deformation processing by the new deformation parameters is set shorter than when the viewpoint loss time is long. As a result, for example, after performing the necessary minimum invalidation (time delay), a suitable deformed and corrected image (virtual image) corresponding to the viewpoint position can be quickly displayed, and it is possible to reduce the sense of discomfort and suppress the generation of the sense of discomfort.

[0036] In other words, when the period during which the viewpoint is lost is short, it is presumed that the moving distance of the viewpoint position is small. Therefore, it can be presumed that there are rarely large differences in the form of distortion of the virtual image before and after the update of the deformation parameter. Considering this, it is possible to prevent a sudden change in the appearance of the virtual image after the re-detection of the viewpoint position (in other words, within a relatively short period of time), and then quickly return to the normal viewpoint following deformation control, thereby reliably obtaining an improved effect of visual recognition.

[0037] In a fourth mode subordinate to any one of the first to third modes,

[0038] When the control unit sets the update period of the deformation parameter before the occurrence of viewpoint loss and during the period of viewpoint loss as the first update period RT1, and sets the update period of the deformation parameter during the period of invalidating the deformation process as the second update period RT2, it can change the parameter update period to RT1 < RT2.

[0039] In the fourth mode, during the invalidation period, while maintaining the process of the parameters before the viewpoint loss, a process of extending the update period of the deformation parameter (a process of changing the update period of the deformation parameter) is used in combination.

[0040] For example, if the frame rate of the image (virtual image) is 60 fps (frames per second), then image processing (image display processing) of 60 frames is performed per second (in other words, the period of one frame is 1 / 60 second). As an example, assume that the update of the deformation parameter is also usually performed for each frame.

[0041] Here, if the parameter update is performed every two frames during the invalidation period after the viewpoint loss, the update period is 2 / 60 seconds. Additionally, if the parameter update is performed every three frames, the update period is 3 / 60 seconds, thereby making the update period longer. In this way, by switching to an update in units of multiple frames, the update period can be extended (increased). Since the update period becomes longer, the reflection of the updated deformation parameter on the image (virtual image) becomes slower. In other words, the sensitivity of the updated parameter reflected on the display becomes dull. If the invalidation period has passed, the update period of the changed parameter will return to the original (from RT2 to RT1), but restoring the update period to the original is not instantaneously completed in reality and requires a certain amount of time. Therefore, even if the parameter is switched, the reflection of the switched parameter in the actual display will be delayed.

[0042] Therefore, it is easy to appropriately set a delay of a certain time range (a delay with a degree that the driver can perceive the appearance change in the actual display (in other words, a delay that effectively slightly extends the amplitude of the invalidation period)). Effects such as facilitating the design of the time control circuit can also be expected.

[0043] In addition, research is conducted on aspects such as variably controlling the degree of increase in the parameter update period, or the timing when the increased update period is restored to the original, etc., so that the range of deformation of the control is expanded, and flexible response becomes possible. In actual display control, the delay amount is also easily set to be quite wide.

[0044] In a fifth mode subordinate to the fourth mode,

[0045] after the control unit changes the deformation update period from the RT1 to the RT2,

[0046] at the end of the period during which the deformation process is invalidated, it is restored from the RT2 to the RT1,

[0047] Or,

[0048] at the time when a predetermined time has further elapsed from the end of the period during which the deformation process is invalidated, it is restored from the RT2 to the RT1,

[0049] Or,

[0050] The change of the parameter update period can be started from the end of the period during which the deformation process is invalidated, and can be gradually restored from the RT2 to the RT1 as time passes.

[0051] In the fifth mode, an example is described in which after performing the process of extending the update period in the fourth mode, the update period is restored to the original.

[0052] In the first example, in synchronization with the switching (update) of the deformation parameter, the longer update period is restored to the original shorter update period. Even in this case, since it takes a certain amount of time to change the update period, the delay in this part can be reliably ensured.

[0053] In the second example, at the time point when a predetermined time has further elapsed from the time point of the switching (update) of the deformation parameter, the deformation update period is restored to the original. In this example, the time point for restoring the update period is delayed by a predetermined time from the time point of the switching (update) of the parameter, and the reflection of the changed parameter on the display is further delayed, so that it is easy to reliably achieve an appropriate length of delay perceptible to the human eye.

[0054] In a third example, when restoring the update cycle to its original state, the update cycle is gradually restored to its original state as time elapses. In other words, when restoring the update cycle from, for example, 1 / 15 second to 1 / 60 second, it is not restored immediately, but rather, control is implemented in a manner of gradually switching in stages of 1 / 30 second, 1 / 45 second, and 1 / 60 second at regular time intervals. By gradually switching the update cycle on the time axis, the delay reflected in the changed parameter display can be managed with higher precision.

[0055] In a sixth mode that is subordinate to any one of the first to fifth modes,

[0056] it further has a low-speed state determination unit that determines whether the speed of the vehicle is in a low-speed state.

[0057] During a period when the control unit invalidates the deformation process when the vehicle is in the low-speed state including the stop state, the period is longer than the period when invalidating the deformation process in a state faster than the low-speed state.

[0058] In the sixth mode, when the vehicle is in a low-speed state, the invalidation period is set longer than when it is in a medium-speed state or a high-speed state (in other words, the moment of switching the deformation parameter is made more delayed). When the vehicle is in a low-speed state, the driver is sensitive to visual changes ahead and is likely to notice such changes. Therefore, at this time, a countermeasure is provided to more greatly delay the reflection of the new parameter after the loss of the viewpoint on the image and make it less likely to cause a sense of incongruity due to an instantaneous change in the appearance of the display. When the vehicle speed leaves the low-speed state and becomes faster, the implemented control focuses on reducing the invalidation period (which can include the case of eliminating the invalidation period), and correcting the distortion of the image faster based on the viewpoint position detected again after the loss. Thus, appropriate deformation control can be performed corresponding to the vehicle speed.

[0059] In a seventh mode that is subordinate to any one of the first to sixth modes,

[0060] the control unit changes the period of invalidating the deformation process according to the vehicle speed. In this case,

[0061] control is implemented such that when the speed of the vehicle is in a range greater than or equal to a first speed value U1 (U1 > 0) and less than or equal to a second speed value U2 greater than the first speed value, the period of invalidating the deformation process decreases with respect to the vehicle speed as the vehicle speed increases.

[0062] Or,

[0063] control is implemented such that in a range where the vehicle speed is close to the first speed value, the degree of decrease slows down, and as the vehicle speed moves away from the first speed value, the degree of decrease intensifies.

[0064] Alternatively,

[0065] the following control may also be implemented: within a range where the vehicle speed is close to the first speed value, the degree of reduction slows down; as the vehicle speed moves away from the first speed value, the degree of reduction becomes more drastic; and as the vehicle speed approaches the second speed value, the degree of reduction slows down.

[0066] In the seventh mode, when shortening (in other words, reducing) the invalidation deformation process period is implemented corresponding to an increase in vehicle speed, this control (the first control) can be implemented when the vehicle speed is within a range greater than or equal to the first speed U1 (>0) and less than or equal to the second speed value U2 greater than the first speed value. Control is not implemented within the ranges where the vehicle speed is less than the first speed U1 and exceeds the second speed U2 to avoid overburdening the system of the HUD device. Additionally, by reducing the invalidation period with respect to speed, more flexible and appropriate deformation processing corresponding to speed can be performed.

[0067] Furthermore, when in a low-speed state where the driver is likely to feel visual changes in the image (virtual image) (in other words, the vehicle speed is within a range close to the first speed value U1), control (the second control) can be implemented in a manner that suppresses rapid updates of the deformation parameter, making the degree of reduction of the invalidation period slow down. In this case, higher-precision control is achieved.

[0068] In addition, in addition to the above second control, control (control with an inverse S-shaped characteristic) can also be implemented where the degree of reduction of the invalidation period slows down as it approaches the second speed value U2, and by stopping the reduction and making it constant when reaching the second speed value U2, the sudden change to the top is suppressed to avoid a sense of incoordination (the third control). Therefore, the visual recognition of the virtual image can be further improved.

[0069] In the eighth mode, which is subordinate to any one of the first to seventh modes,

[0070] when the head-up display device adjusts the position of the viewing area according to the height position of the driver's viewpoint, the reflection position of the display light of the image on the optical component can be changed without moving the optical component.

[0071] In the eighth mode, when adjusting the height position of the viewing area corresponding to the height position of the driver's eyes (viewpoint), the HUD device implementing the above control does not use, for example, an actuator to rotate the optical component that projects light onto the projection component, but instead responds by changing the reflection position of light in the optical component.

[0072] In recent years, HUD devices have tended to be developed on the premise of displaying a virtual image within a relatively large range in front of a vehicle. In such a case, the device inevitably becomes large-sized. Naturally, the optical components also become large-sized. When rotating an optical component using an actuator or the like, due to errors, the control accuracy of the height position of the viewing area may instead decrease. To prevent this, it is addressed by changing the position of the light reflected by the optical component.

[0073] In such large optical components, by optimally designing the reflecting surface as a free-form surface or the like, the distortion of the virtual image is minimized as much as possible. However, as described above, for example, when the driver's viewing point is located around the viewing area, there are sometimes cases where the distortion becomes obvious anyway. Therefore, in such a case, by implementing control to temporarily invalidate (delay) the parameter corresponding to the viewing point position after re-detection within a specified range, it is possible to make it difficult to generate a sense of incongruity caused by a change in appearance due to the distortion of the virtual image, and it is possible to effectively utilize the above control to improve visual recognition.

[0074] In a ninth aspect subordinate to any one of the first to eighth aspects,

[0075] A virtual virtual image display surface corresponding to the image display surface of the display unit is configured to overlap with the road surface in front of the vehicle.

[0076] Or,

[0077] It may also be disposed inclined with respect to the road surface in such a manner that the end portion closer to the vehicle of the virtual image display surface, i.e., the proximal end portion, has a smaller distance from the road surface, while the end portion farther from the vehicle, i.e., the distal end portion, has a larger distance from the road surface.

[0078] In the ninth aspect, a virtual virtual image display surface (corresponding to a display surface such as a screen of a display unit) disposed in front of the vehicle or the like in the HUD device is overlapped with the road surface or disposed inclined with respect to the road surface. The former is sometimes referred to as a road surface overlapping HUD, and the latter is sometimes referred to as an inclined surface HUD.

[0079] They use a relatively wide virtual image display surface that overlaps with the road surface or a relatively wide virtual image display surface that is disposed inclined with respect to the road surface. For example, various displays can be performed within a range of 5 m to 100 m in front of the vehicle. The HUD device is enlarged and thus the viewing area is also enlarged. It is preferable to detect the viewing point position with high precision within a wider range than before and perform image correction using appropriate deformation parameters. However, if the viewing point is lost, the switching control of the high-precision deformation parameters will instead reduce the visual recognition of the image (virtual image) after the viewing point is re-detected. Therefore, the application of the control method of the present invention is effective.

[0080] Those skilled in the art should easily understand that the embodiments of the present invention according to the examples can be further modified without departing from the spirit of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] Figure 1 In (A) of FIG. is a diagram for explaining an outline of the deformation process and a distortion mode of a virtual image (and a virtual image display surface) displayed after the deformation process. Figure 1 In (B) of FIG. is a diagram showing an example of a virtual image visually recognized by a driver through a windshield.

[0082] Figure 2 In (A) of FIG. is a diagram for explaining an outline of the viewpoint position following deformation process. Figure 2 In (B) of FIG. is a diagram showing a structural example of a viewpoint area in which the interior is divided into a plurality of partial areas.

[0083] Figure 3 In (A) to (F) of FIG. are diagrams showing examples of virtual images with different distortion modes after the deformation process.

[0084] Figure 4 is a diagram showing an example of viewpoint loss and re-detection of the viewpoint position in a viewpoint area in which the interior is divided into a plurality of partial areas.

[0085] Figure 5 is a diagram showing an example of the system configuration of a HUD device.

[0086] Figure 6 In (A) to (C) of FIG. is a timing chart showing an example of control for setting a period during which the deformation process based on the re-detected viewpoint position is invalidated.

[0087] Figure 7 In (A) to (D) of FIG. represents a timing chart showing other examples of control for setting a period during which the deformation process based on the re-detected viewpoint position is invalidated (in the case of a change process of the parameter update cycle).

[0088] Figure 8 In (A) and (B) of FIG. is a timing chart showing another example of control for setting a period during which the deformation process based on the re-detected viewpoint position is invalidated (the first control example when the viewpoint loss period is shorter than the threshold).

[0089] Figure 9 In (A) and (B) of FIG. is a timing chart showing another example of control for setting a period during which the deformation process based on the re-detected viewpoint position is invalidated (the second control example when the viewpoint loss period is shorter than the threshold).

[0090] Figure 10It is a diagram showing a characteristic example in the case of representing a period during which deformation processing based on a re-detected viewpoint position is variably controlled according to the vehicle speed.

[0091] Figure 11 It is a flowchart showing a process example (first control example) of distortion image correction control corresponding to viewpoint loss.

[0092] Figure 12 It is a flowchart showing a process example (second control example) of distortion image correction control corresponding to viewpoint loss.

[0093] Figure 13 It is a flowchart showing a process example (third control example) of distortion image correction control corresponding to viewpoint loss.

[0094] Figure 14 (A) in it is a diagram showing a display example of a road surface overlapping HUD, Figure 14 (B) in it is a diagram showing a display example of an inclined surface HUD, Figure 14 (C) in it is a diagram showing a structural example of the main part of the HUD device.

[0095] Symbol Explanation

[0096] 1 Vehicle (this vehicle)

[0097] 2 Projection member (reflection and light transmission member, windshield, etc.)

[0098] 4 Projection area

[0099] 5 Virtual image display area

[0100] 7 Steering wheel

[0101] 51 Display light

[0102] 100 HUD device

[0103] 110 Viewpoint detection camera

[0104] 112 Light source

[0105] 114 Light projection unit

[0106] 116 Display unit

[0107] 117 Display surface (image display surface)

[0108] 118 Projection optical system

[0109] 120 Viewpoint position detection unit (viewpoint position determination unit)

[0110] 122 Viewpoint coordinate detection unit

[0111] Partial Area Detection Unit within the 124 Viewpoint Area

[0112] 130 Operation Unit

[0113] 131 Curved Mirror (Concave Mirror, etc.)

[0114] 133 Reflecting Mirror (Including Reflecting Mirror, Correction Mirror, etc.)

[0115] 140 Vehicle ECU

[0116] 150 Bus

[0117] 151 Light Source Unit

[0118] 161 Display Unit (Screen, etc.)

[0119] 63 Display Surface (Image Display Surface)

[0120] 170 Bus Interface

[0121] 171 Control Unit

[0122] 173 Actuator for Driving the Display Unit

[0123] 180 Display Control Unit (Display Control Device)

[0124] 182 Speed Detection Unit

[0125] 184 Deformation Control Unit

[0126] 185 Deformation Management Unit

[0127] 186 Viewpoint Loss Detection Unit

[0128] 187 Switching Delay Unit for Deformation Parameters (Invalidation Period Setting Unit)

[0129] 188 Update Cycle Change Unit for Deformation Parameters

[0130] 189 Temporary Storage Unit for Partial Area Information of the Viewpoint Area

[0131] 192 Memory Control Unit

[0132] 194 Deformation Processing Unit

[0133] 200 Image Generation Unit

[0134] 210 ROM

[0135] 220 VRAM

[0136] 212 Image Transformation Table

[0137] 222 Image (Original Image) Data

[0138] Image data after the 224 transformation process

[0139] EB viewing area

[0140] Partial areas of the Z (such as Z1 to Z9) viewing area

[0141] WP transformation parameter

[0142] PS virtual image display surface

[0143] V virtual image Detailed implementation mode

[0144] The best implementation mode described below is used for easy understanding of the present invention. Therefore, those skilled in the art should note that the present invention is not improperly limited by the implementation mode described below.

[0145] Refer to Figure 1 . Figure 1 In (A) of [], it is a diagram for explaining the outline of the transformation process and the distortion mode of the virtual image (and virtual image display surface) displayed after the transformation process. Figure 1 In (B) of [], it is a diagram showing an example of the virtual image visually recognized by the driver through the windshield.

[0146] As Figure 1 shown in (A) of [], the HUD device 100 has: a display unit (for example, a light-transmissive screen) 101; a mirror 103; and a curved mirror 105 as an optical component for projecting display light (for example, it may also be a concave mirror, and the reflecting surface may be a free-form surface). The image displayed on the display unit 101 is projected onto the virtual image display area 5 of the windshield 2 as the projection component through the mirror 103 and the curved mirror 105. In Figure 1 , the symbol 4 represents the projection area. In addition, the HUD device 100 may be provided with multiple curved mirrors. In addition to the curved mirror (reflective optical element) of the present embodiment, or instead of a part (or all) of the mirror (reflective optical element) of the present embodiment, it may also include refractive optical elements such as lenses, diffractive optical elements and other functional optical elements.

[0147] A part of the display light of the image is reflected by the windshield 2 and enters the viewpoints (eyes) A of the driver or the like located inside (or on the EB) of the preset viewing area EB (here, it is a quadrilateral shape with a specified area). By imaging in front of the vehicle 1, a virtual image V is displayed on the virtual virtual image display surface PS corresponding to the display surface 102 of the display unit 101.

[0148] The image of the display unit 101 is distorted by the shape of the curved mirror 105, the shape of the windshield 2, etc. To cancel this distortion, the image is given a distortion with characteristics opposite to that distortion. This type of image correction by pre-distortion is referred to as deformation processing or deformed image correction processing in this specification.

[0149] Ideally, through the deformation processing, the virtual image V displayed on the virtual image display surface PS should be a flat image without curvature. However, for example, in a large HUD device 100 that projects display light onto a relatively wide projection area 4 of the windshield 2 and sets the virtual image display distance within a relatively wide range, it is inevitable that a certain degree of deformation remains, which is unavoidable.

[0150] At Figure 1 In the upper left, PS' represented by a dotted line indicates the virtual image display surface where the distortion has not been completely removed, and V' represents the virtual image displayed on this virtual image display surface PS'.

[0151] In addition, the degree or manner of distortion of the virtual image V with remaining distortion varies depending on the position of the viewing point A within the viewing point area EB. The optical system of the HUD device 100 is designed assuming that the viewing point A is near the central part. Therefore, when the viewing point A is near the central part, the distortion of the virtual image is relatively small, and the distortion of the virtual image becomes larger as it gets closer to the peripheral part.

[0152] Figure 1 In (B) of Figure 1 In (B) of Figure 1 In (B) of Figure 1 In (B) of

[0153] Next, refer to Figure 2 . Figure 2 In (A) of Figure 2 In (B) of Figure 2 In Figure 1The same parts are labeled with the same reference signs (the same applies to the subsequent figures).

[0154] As Figure 2 shown in (A) of Figure 2 , the viewing area EB is divided into a plurality of (nine in this case) partial areas Z1 to Z9, and the position of the driver's viewing point A is detected for each of the partial areas Z1 to Z9.

[0155] The display light K of the image is emitted from the projection optical system 118 of the HUD device 100, and a part of it is reflected by the windshield 2 and incident on the driver's viewing point (eye) A. When the viewing point A is within the viewing area, the driver can visually recognize the virtual image of the image.

[0156] The HUD device 100 has a ROM 210, and the ROM 210 incorporates an image transformation table 212. The image transformation table 212 stores, for example, distortion parameters WP such as polynomials, multipliers, and constants for determining image correction (distorted image correction) based on a digital filter. The distortion parameters WP are correspondingly set for each of the partial areas Z1 to Z9 in the viewing area EB. In Figure 2 (A) of Figure 2 , the distortion parameters corresponding to the partial areas are represented as WP(Z1) to WP(Z9). In addition, in the figure, as symbols, only WP(Z1), WP(Z4), and WP(Z7) are shown.

[0157] When the viewing point A moves, it is detected which of the plurality of partial areas Z1 to Z9 the viewing point A is located in. Then, one of the distortion parameters WP(Z1) and WP(Z9) corresponding to the detected partial area is read from the ROM 210 (update of the distortion parameter), and the distortion process is performed using the distortion parameter.

[0158] Figure 2 (B) of Figure 2 shows an example of the viewing area EB in which the number ratio of the partial areas is increased compared to Figure 2 (A) of Figure 2 . The viewing area EB is divided into a total of sixty partial areas, six vertically and ten horizontally. Each partial area is represented as Z(X, Y) with the coordinate positions in the X direction and Y direction as parameters.

[0159] Next, refer to Figure 3 . Figure 3 (A) to (F) of Figure 3 are diagrams showing examples of virtual images with different distortion modes after the distortion process. As described above, the appearance of the virtual image V after the distortion process is different depending on the position of the driver's viewing point A in the viewing area.

[0160] As Figure 3As shown in (A) in [reference], the virtual image V displayed in the virtual image display area 5 of the windshield 2 is ideally displayed in a form that is free of distortion and bending. However, in the actual virtual image V, even after the distortion correction process, some distortion remains, and the degree and form of this distortion vary depending on the position of the viewing point A.

[0161] In Figure 3 the example of (B) in [reference], although there is distortion, it is relatively mild, and the virtual image V is close to Figure 3 the appearance in (A) in [reference]. In Figure 3 the example of (C) in [reference], it can be said that the tendency of distortion is the same as that in Figure 3 (B) in [reference], but the degree of distortion has increased, and it cannot be said to have the same appearance as Figure 3 (A) in [reference].

[0162] In addition, in Figure 3 the example of (D) in [reference], the degree of distortion is the same as that in Figure 3 (C) in [reference], but the form of distortion (the tendency of distortion, or the form of the appearance of the virtual image after distortion occurs) is different from that in Figure 3 (C) in [reference].

[0163] In Figure 3 the example of (E) in [reference], the degree of distortion is even greater, and the left and right sides of the virtual image V are not balanced. In Figure 3 the example of (F) in [reference], the virtual image V has a distortion pattern similar to that in Figure 3 (E) in [reference], but the appearance is very different from that in Figure 3 (F) in [reference].

[0164] Thus, even for the virtual image V (the virtual image V after the distortion correction process) that displays the same content, the viewing method varies considerably depending on the position of the viewing point A. For example, when the viewing point A is located at the central part of the viewing point area, the visually recognized virtual image V has relatively little distortion as shown in Figure 3 (B) in [reference], but when the viewing point A moves from the central part to the peripheral part, for example, as shown in Figure 3 (E) in [reference], the distortion is relatively large.

[0165] In this state (the state where the viewing point A is located in a partial area of the peripheral part of the viewing point area Figure 3 in the case of (E) in [reference]), for example, assuming that the viewing point A moves to another partial area, for example, the appearance of the virtual image V changes as shown in Figure 3 (B) in [reference] (change a1), or changes as shown in Figure 3 (F) in [reference] (change a2). In either case, the appearance has changed considerably, increasing the likelihood of the driver (user) experiencing a sense of discomfort.

[0166] Next, referring toFigure 4 。 Figure 4 This is a diagram showing an example of the loss of a viewing point and the re-detection of the viewing point position in a viewing point area that is internally divided into multiple partial areas. In Figure 4 , as a method of re-detecting the viewing point position after the loss of the viewing point, each of the viewing point movements (1) to (6) is illustrated.

[0167] A typical example of the loss of a viewing point is when the driver's viewing point A deviates from the viewing point area EB during driving and the detection of its position is interrupted, and then the viewing point A returns to within the viewing point area EB. In Figure 4 , as a method of the viewing point movement at this time, the movements (1) to (6) are illustrated. The viewing point movement (1) is the movement of the viewing point A from the inside of the central area CT of the viewing point area EB to the outside of the viewing point area EB, the movement distance is long, and the loss time of the viewing point is also long. In this case, when the viewing point A returns to within the viewing point area EB, it can be assumed that it becomes unstable (with many changes), for example, by passing through the movements (2) and (3) and stopping at (4).

[0168] In addition, as in the viewing point movement examples (5) and (6), the loss of the viewing point also occurs when the viewing point A is not outside the viewing point area EB but instantaneously moves in multiple partial areas. In the viewing point movement examples (5) and (6), compared with the movement methods (1) to (4) above, the movement distance is short, the loss time of the viewing point is also short, and the viewing point movement is relatively stable. There are various ways of losing the viewing point, and it is hoped to respond flexibly.

[0169] In the present embodiment, the following countermeasures are basically adopted: during the period when the viewing point is lost (the viewing point is lost), the previous deformation parameters are maintained, and from the moment when the viewing point A is re-detected, the invalidation period starts. During the invalidation period, at least one deformation process using the deformation parameters corresponding to the re-detected viewing point position is invalidated. During the invalidation period, the deformation parameters before the loss of the viewing point are maintained.

[0170] In addition, during the viewing point loss period, it is also conceivable to adopt deformation parameters corresponding to the position of the center (symbol CP) of the viewing point area EB. However, in this case, the parameters are a staged process of temporarily transferring from the parameters before the viewing point loss to the parameters corresponding to the center CP, and then to the parameters corresponding to the position after the re-detection. Since there is a high possibility that the appearance of the virtual image will change due to the switching of the deformation parameters, this is not adopted in the present embodiment. As described above, during the viewing point loss period and the subsequent invalidation period, by maintaining the deformation parameters before the viewing point loss, control for suppressing the change in the appearance of the virtual image is implemented. By setting the invalidation period to an appropriate length, for example, it is possible to only make the Figure 4The redetection of the viewpoint A after the viewpoint movement (2) becomes invalid, or it is also possible to invalidate the deformation accompanying the redetection of the viewpoint A after the viewpoint movement (3) (for example, Figure 6 , Figure 7 example).

[0171] In addition, during the invalidation period, it is also possible to combine the process of changing the update period of the deformation parameter (the process of extending the update period). In addition, at this time, it is also possible to perform an application process in which the period of increasing the parameter update period continues for a while after the end of the invalidation period. In addition, when restoring the update period to the original, it is also possible to combine the application process that gradually recovers over time ( Figure 7 (A) to (D) examples).

[0172] In addition, the invalidation period can also be variably controlled according to whether the viewpoint loss period is longer than a threshold value (a threshold value for comparison determination) or shorter than the threshold value. For example, after the viewpoint loss (a relatively short loss of the viewpoint movement) based on Figure 4 the viewpoint movement (5), it is possible to demonstrate to the driver (user) that the viewpoint has been redetected and give a sense of security ( Figure 8 example), or, conversely, it is also possible to end the invalidation relatively quickly and quickly perform the deformation process based on the parameters corresponding to the moved viewpoint position, suppressing the long invalidation period ( Figure 9 example). In addition, it is also possible to implement speed-based adaptive control by making the invalidation period change according to the vehicle speed ( Figure 10 example). The details of these contents will be described later.

[0173] Next, refer to Figure 5 . Figure 5 is a diagram showing an example of the system configuration of the HUD device. A viewpoint detection camera 110 for detecting the position of the driver's viewpoint A (eyes, pupils) is provided on the vehicle 1. In addition, an operation input unit 130 is provided on the vehicle 1 so that the driver can set necessary information and the like for the HUD device 100, and a vehicle ECU 140 capable of collecting various information of the vehicle 1 is provided.

[0174] In addition, the HUD device 100 includes: a light source 112, a light projecting unit 114, a projection optical system 118, a viewpoint position detection unit (viewpoint position determination unit) 120, a bus 150, a bus interface 170, a display control unit 180, an image generation unit 200, a ROM 210 with a built-in image transformation table 212, and a VRAM 220 that stores image (original image) data 222 and temporarily stores the transformed image data 224. The display control unit (display control device) 180 is composed of one or more processors, one or more image processing circuits, and one or more memories, etc. By executing the program stored in the memory, it can control the HUD device 100 (display unit 116), such as generating and / or transmitting image data. The processor and / or the image processing circuit may include at least one general-purpose microprocessor (e.g., a central processing unit (CPU)), at least one application-specific integrated circuit (ASIC), at least one field-programmable gate array (FPGA), or any combination thereof. The memory includes any type of magnetic medium such as a hard disk, any type of optical medium such as a CD and a DVD, any type of semiconductor memory such as a volatile memory, and a non-volatile memory. The volatile memory may include DRAM and SRAM, and the non-volatile memory may include ROM and NVRAM.

[0175] The viewpoint position detection unit 120 includes a viewpoint coordinate detection unit 122 and a partial area detection unit 124 of the viewpoint area that detects (determines) which partial area of the viewpoint area the viewpoint A is located in based on the detected coordinates.

[0176] In addition, the display control unit 180 includes: a speed detection unit 182 (also serving as a low-speed state determination unit for determining a low-speed state) that detects (determines) the speed of the vehicle 1, a deformation control unit 184 (including a deformation management unit 185), a timer 190, a memory control unit 192, and a deformation processing unit (deformed image correction processing unit) 194.

[0177] In addition, the deformation management unit 185 includes: a viewpoint loss detection unit 186 (or viewpoint loss detection unit) that detects the occurrence of a viewpoint loss (viewpoint loss); a switching delay unit (invalid period setting unit) 187 of the deformation parameter; an update period change unit 188 of the deformation parameter; and a temporary storage unit 189 of the partial area of the viewpoint area that temporarily stores the partial area information of the viewpoint area corresponding to the detected viewpoint position.

[0178] Here, when the viewpoint position is first detected again after the viewpoint loss, the switching delay unit 187 (invalid period setting unit) of the deformation parameter does not immediately switch the deformation parameter based on the viewpoint position detected again at that moment, but temporarily delays the switching of the deformation parameter, thereby implementing the control to invalidate the switching of the deformation parameter.

[0179] In addition, the update period change unit 188 of the deformation parameter performs control to change the update period of the deformation parameter (specifically, extend the update period) at least during the invalidation period in parallel with the setting process of the invalidation period of the delay based on the switching time of the deformation parameter. By changing this update period, for example, it is possible to appropriately delay the reflection time of the updated deformation parameter to the actual display.

[0180] The basic operation is as follows. That is, the following operations are performed: The partial area information (information indicating which partial area of the view area the viewpoint A is located in) sent from the viewpoint position detection unit 120 is used as an address variable, the memory control unit 192 accesses the ROM 210, and reads out the corresponding deformation parameter. The deformation processing unit (deformed image correction processing unit) 194 uses the read deformation parameter to perform deformation processing on the original image. Based on the data after the deformation processing, the image generation unit 200 generates an image in a specified format and supplies the image to, for example, the light source 112, the light projection unit 114, etc.

[0181] However, if the deformation is simply performed by tracking the movement of the viewpoint A, as described above, when the viewpoint position is detected again after the viewpoint is lost, the appearance of the virtual image V will change instantaneously, and there will be a situation where the driver has a visual discomfort. Therefore, control is implemented to intentionally blunt (suppress) the sensitivity of the deformation processing. There are several ways of this control. The following will be described in sequence.

[0182] Refer to Figure 6 。 Figure 6 (A) to (C) in are timing charts showing an example of control for setting a period during which the deformation processing based on the viewpoint position detected again is invalidated. As shown in (A) in Figure 6 During the period from time t10 to t11, the position of the viewpoint A is located in the partial area Z(n, m) (n, m are natural numbers that determine the partial area within the view area) of the view area EB. At time t11 to t12, a viewpoint loss (viewpoint disappearance) occurs. After that, at time t12, it is detected again that the viewpoint A is located in the partial area Z(r, s) of the view area EB (where r, s are natural numbers that determine the partial area within the view area, and the partial area determined here is different from Z(n, m)). In addition, the viewpoint loss period (viewpoint disappearance period) is marked as T0.

[0183] In addition, as shown in (C) in Figure 6 In the present embodiment, the value of the update period of the deformation parameter is fixed to RT1 and not changed.

[0184] As shown in Figure 6As shown by the solid line in (B) in [reference], during the period of lost view (from time t11 to t12), the deformation parameter maintains the parameter value WP1 before the occurrence of lost view. As an alternative, as shown by the dashed line, it is also possible to consider maintaining the parameter value during the lost view period as the value corresponding to the center position of the view area EB. However, in this case, when the driver observes the virtual image during the lost view period, there may be a sense of incongruity due to the change in the appearance of the virtual image accompanying the change in the parameter value. Therefore, this alternative is not adopted.

[0185] In addition, at time t12, the position of the view point A is detected again. However, the parameter is not immediately switched to the parameter corresponding to the redetected position. During a specified period starting from the redetection time t12 (here, the period until time t13), the parameter value WP1 is maintained. At time t13, the parameter value is changed to the value WP2 based on the redetected position. The period from time t12 to t13 is the invalidation period Ta. During this invalidation period Ta, when the view point position is redetected at least once, the change (application of the parameter) of the parameter based on the redetected position is invalidated, and the deformation using the originally maintained parameter is implemented.

[0186] By setting the invalidation period, the parameter is fixed for a short period of time, and the instantaneous switching of the deformation parameter is not performed. In addition, during this invalidation period, even if the view point position is unstable, for example, it moves through multiple partial areas of the view area, the change of the parameter based on the continuous redetection of the view point position is not performed. As a result, the deformation process is stabilized. Therefore, for example, when the driver's view point A leaves the view area and then returns to the view area again, the situation where the appearance of the virtual image V changes instantaneously and causes a sense of incongruity can be suppressed.

[0187] Next, refer to Figure 7 . Figure 7 (A) to (D) in [reference] are timing diagrams showing other examples of the control for setting the period during which the deformation process based on the redetected view point position is invalidated (such as the case of changing the parameter update cycle, etc.).

[0188] As Figure 7 shown in (A) in [reference], from time t1 to t3, view point loss (lost view) occurs. The view point loss period is marked as T1. In addition, as Figure 7 shown in (A) in [reference], when the view point loss period T1 is compared with a specified threshold value (the threshold value for determining the length of the view point loss period) Th, it is greater than or equal to the threshold value Th. In other words, in the Figure 7 example, Th ≤ T1 holds (where, in Figure 7 (A) in [reference], as a specific example, the case of Th < T1 is shown).

[0189] In addition, asFigure 7 As shown in (B) in [reference], the period from time t3 to t4 is the invalidation period Ta. The deformation parameter WP1 may instantaneously (at time t4) switch to WP2, or as shown by the characteristic lines Tk1 or Tk2 of the dashed line, there may be a gradual switch over time. In addition, Tk1 and Tk2 correspond to the characteristic lines G1 and G2 in (D) in [reference] (described later). When switching parameters such as the characteristic lines Tk1 and Tk2 are implemented, the moment when the switch of the parameter value to WP2 is completed becomes the moment t5 which is the time (period) Tb further delayed from time t4 (this will also be described later). Figure 7 In [reference], the period from time t3 to t4 is the invalidation period Ta. When implementing the change process of the update period of the deformation parameter (specifically, extending the update period) from RT1 to RT2 during the invalidation period Ta from time t3 to t4.

[0190] In addition, in Figure 7 the example of [reference], the process of changing the update period of the deformation parameter is also mentioned. Figure 7 (C) in [reference] shows the case where the update period of the deformation parameter is fixed to RT1 and no change in the update period is made. Figure 7 (D) in [reference] implements (and uses) the change process of the update period of the deformation parameter, which changes the update period of the deformation parameter from RT1 to RT2 (specifically, extends the update period) during the invalidation period Ta from time t3 to t4.

[0191] For example, if the frame rate of the image (virtual image) is 60 fps (frames per second), then image processing (image display processing) of 60 frames is implemented per second (in other words, the period of one frame is 1 / 60 second). As an example, it is assumed that the update of the deformation parameter is also usually implemented for each frame.

[0192] Here, if parameter update is implemented every two frames during the invalidation period after the loss of the viewing point, the update period is 2 / 60 seconds. In addition, if parameter update is implemented every three frames, the update period is 3 / 60 seconds, so that the update period becomes longer. In this way, by switching to update in units of multiple frames, the update period can be extended (increased). Since the update period becomes longer, the reflection of the updated deformation parameter on the image (virtual image) becomes slower. In other words, the sensitivity of the updated parameter reflected on the display becomes dull. If the invalidation period has passed, the update period of the changed parameter will return to the original (from RT2 to RT1), but restoring the update period to the original is not instantaneously completed in reality and requires a certain amount of time. Therefore, even if the deformation parameter is switched, the reflection of the switched parameter in the actual display will be delayed.

[0193] Therefore, it is easy to appropriately set a delay of a certain time range (a delay with a length that the driver can perceive the appearance change in the actual display (in other words, a delay that effectively slightly extends the range of the invalidation period)). Effects such as facilitating the design of the moment control circuit can also be expected.

[0194] In addition, research is conducted on variably controlling the degree of increase in the parameter update period, or the timing when the increased update period is restored to the original, etc., so that the range of deformation of the control is expanded and flexible response becomes possible. In reality, the delay amount in display control is also likely to be set quite wide.

[0195] Figure 7 (D) in shows a modified example of the timing when the increased update period is restored to the original. In Figure 7 (D) in, as shown by the characteristic line G1 of the dotted line (thick line), the timing for restoring the update period can also be changed from time t4 to time t5. In this case, the period during which the sensitivity of the reflection of the parameter display is dulled is extended. In this case, in addition to setting an invalidation period for delaying the parameter switch, the timing for restoring the update period to the original (from RT2 to RT1) is also delayed, further delaying the reflection of the updated parameter in the actual display, thereby making it easier to generate the necessary delay and reducing the burden on the timing circuit, etc.

[0196] In addition, in Figure 7 (D) in, as shown by the characteristic line G2 of the dotted line (thin line), the process of restoring the update period starts from time t4, but afterwards, there can also be a time margin and the update period can be restored little by little. For example, when returning the parameter update period from 1 / 15 second (=RT2) to 1 / 60 second (=RT1), instead of returning immediately, control is implemented to gradually switch to 1 / 30 second, 1 / 45 second, 1 / 60 second in units of a specified time. By gradually switching the update period on the time axis, the delay reflected in the changed parameter display can be managed with higher precision.

[0197] Next, refer to Figure 8 . Figure 8 (A) and (B) in are timing charts showing another example of control for setting a period during which the deformation process based on the viewpoint position detected again is invalidated (the first control example when the viewpoint loss period is shorter than the threshold).

[0198] In the Figure 8 example, the control unit ( Figure 5 the symbol 184 or 185 in) uses, for example, a timer 190 to measure the viewpoint loss time (viewpoint loss time: sometimes simply referred to as loss time) of the lost viewpoint, and implements control such that the period during which invalidation is performed (invalidation period) is longer when the viewpoint loss time is shorter than a predetermined threshold Th than when the loss time is longer than the threshold Th (for example, Figure 7 example).

[0199] In Figure 8In the above description, the viewpoint loss time T10 (time t1-t6) is less than the threshold value Th. The viewpoint loss occurs at time t1, and the position of the viewpoint A is detected again at time t6. Figure 7 In the example, an invalidation period Ta is set after re-detection, but Figure 8 In the example of , the invalidation period is further extended by a period Td. The invalidation period is a period of Te (=Ta+Td).

[0200] By setting the period for which the appearance of the image (virtual image) is kept fixed and maintained for a certain period without changing the deformation parameters for a longer period, the driver can be informed that the re-detection after the viewpoint is lost is successful, and can be made aware that the corresponding processing is currently being implemented. In other words, by extending the fixed period of the deformation parameters and performing image processing with the deformation parameters updated over time, even if the appearance of the image (virtual image) changes, the driver can be made aware that the change is not sudden due to the movement of the driver's eyes, but is a change with more time.

[0201] This allows the driver to easily perceive that, although the viewpoint position has been lost due to the movement of his or her own eyes, the system of the HUD device has successfully detected the viewpoint position again and implemented processing corresponding to the viewpoint loss.

[0202] In other words, the HUD device side (system side) can demonstrate to the driver as a user that the processing after the viewpoint is lost is correctly implemented. This gives the driver a sense of security and mental stability, thereby achieving the effect of not easily generating or reducing the sense of discomfort.

[0203] Next, refer to Figure 9 . Figure 9 (A) and (B) are timing charts showing another example of control for setting a period for invalidating deformation processing based on the re-detected viewpoint position (a second control example when the viewpoint loss period is shorter than the threshold).

[0204] exist Figure 9 In the example, the control unit ( Figure 5 184 or 185 in the figure) for example, the viewpoint loss time (viewpoint loss time: referred to as loss time in some cases) of the lost viewpoint is counted using the timer 190, and when the viewpoint loss time is shorter than the predetermined threshold value Th, control is performed so that the invalidation period (invalidation period) is shorter than when the loss time is longer than the threshold value Th (for example, Figure 7 for example). Figure 9 The control direction and Figure 8 However, since the effect obtained is different, Figure 8 , Figure 9Each example can be selectively applied according to the desired effect.

[0205] When the viewpoint loss time is short, considering that the change in the viewpoint position is small (the moving distance of the viewpoint is short), therefore, in Figure 9 the example, compared with the case where the viewpoint loss time is longer than the threshold Th ( Figure 7 the example), the invalidation time of the deformation process according to the new deformation parameter is set shorter. As Figure 9 shown in (B) of Figure 9 , the invalidation period Tf of Figure 7 is set shorter than the invalidation period Ta of

[0206] Thus, for example, after implementing the necessary minimum invalidation (time delay), a suitable deformed correction image (virtual image) corresponding to the viewpoint position can be quickly displayed, which can reduce the sense of discomfort and suppress the generation of the sense of discomfort.

[0207] In other words, when the viewpoint loss period is short, it is presumed that the moving distance of the viewpoint position is small. Therefore, it can be presumed that there are few large differences in the distortion forms of the virtual images before and after the update of the deformation parameters. Considering this, to prevent a sharp change in the appearance of the virtual image after the re-detection of the viewpoint position (in other words, within a relatively short time), and then quickly return to the normal viewpoint following deformation control, the improved effect of visual recognition can be reliably obtained.

[0208] Next, refer to Figure 10 . Figure 10 is a diagram showing a characteristic example in the case of variably controlling the period during which the deformation process based on the re-detected viewpoint position is invalidated according to the vehicle speed. In Figure 10 the example, the invalidation period after the viewpoint loss is adaptively controlled according to the vehicle speed of the vehicle 1.

[0209] The previously described Figure 5 speed detection unit 182 also functions as a low-speed state determination unit. When it is determined by this speed detection unit 182 that the vehicle 1 is in a low-speed state (a stopped state or a low-speed driving state) (for example, determined using a vehicle speed determination threshold), the control units (184, 185) perform the following control: the period during which the deformation process according to the new parameter is invalidated (invalidation period) is longer than the invalidation period in a state faster than the low-speed state.

[0210] In Figure 10 , when the vehicle speed is in the low-speed state of 0 to U1, the invalidation periods Ta, Te, Tf (corresponding to (B) in Figure 7 respectively, Figure 8 ,Figure 9 The value corresponding to) is N1. In the medium-speed state of U1 to U2, this value is less than N1, and the same is true in the high-speed state where the vehicle speed is greater than or equal to U2.

[0211] In the low-speed state, the driver (user) is sensitive to visual changes in the front and the like and can easily detect such changes. Therefore, at this time, a countermeasure is provided that delays the reflection of a new parameter on the image to a greater extent after the loss of the viewpoint and is not likely to cause a sense of disharmony due to instantaneous changes in the display appearance. When the vehicle speed leaves the low-speed state and becomes faster, the implemented control focuses on reducing the invalidation periods Ta, Te, and Tf (which can include the case of eliminating the invalidation period), and correcting the image distortion faster based on the viewpoint position detected again after the loss. Thus, more flexible and appropriate deformation control can be performed corresponding to the vehicle speed.

[0212] In addition, in Figure 10 the control example, the control unit ( Figure 5 the symbol 184 or 185) corresponds to the period of the invalidation deformation process for the vehicle speed change of vehicle 1 (invalidation period). In this case, when the speed of vehicle 1 is greater than or equal to the first speed value U1 (U1 > 0) and less than or equal to the second speed value U2 greater than the first speed value, control is implemented such that the period of the invalidation deformation process (invalidation period) decreases with respect to the vehicle speed as the vehicle speed increases (the control represented by characteristic lines Q2, Q3, and Q4, this is the first control). In addition, at this time, no control is implemented in the range where the vehicle speed is less than the first speed U1 and exceeds the second speed U2, and the value of the invalidation period is fixed to N1 or N2. Thus, an excessive burden can be prevented from being imposed on the system of the HUD device.

[0213] In addition, in the case of implementing the control represented by characteristic line Q3, in the range where the vehicle speed is close to the first speed value U1, control is implemented such that the degree of decrease when the invalidation period decreases is slow, and the degree of decrease increases as the vehicle speed moves away from the first speed value U1.

[0214] In other words, when in the low-speed state where the driver is likely to feel the visual change of the image (virtual image) (in other words, in the range where the vehicle speed is close to the first speed value U1), control is implemented in a manner that suppresses the rapid update of the deformation parameter and slows down the degree of decrease of the invalidation period (this is the second control). In this case, higher-precision control is achieved.

[0215] In addition, when implementing the control represented by the characteristic line Q4, within the range where the vehicle speed is close to the first speed value U1, a control is implemented in which the degree of decrease during the invalidation period slows down as the vehicle speed approaches the first speed value U1, and the degree of decrease intensifies as the vehicle speed moves away from the first speed value U1, and the degree of decrease slows down as the vehicle speed approaches the second speed value U2 (control with an inverse S-shaped characteristic, which is regarded as the third control). In this third control, in addition to the above-described second control, a control is also implemented in which the degree of decrease in the invalidation period slows down as it approaches the second speed value U2, and by stopping the decrease and making it constant when reaching the second speed value U2, a sudden change to the top is suppressed to avoid a sense of disharmony. Therefore, the visual recognition of the virtual image can be further improved.

[0216] Next, referring to Figure 11 ,。 Figure 11 FIG. shows a process example of the deformed image correction control corresponding to the loss of the viewing point (first control example: corresponding to Figure 6 、 Figure 7 ) flowchart. The viewing point position is monitored (step S1), and it is determined whether there is a loss of the viewing point (loss of the viewing point) (step S2).

[0217] When the result in step S2 is "No", the process returns to step S1. When the result is "Yes", the deformation parameters before the loss of the viewing point are maintained (step S3). Next, it is determined whether the viewing point position is detected again after the loss of the viewing point (step S4). When the result is "No", the process returns to step S3, and when the result is "Yes", the process jumps to step S5.

[0218] In step S5, a delay process (invalidating process) for updating (switching) to the deformation parameters corresponding to the viewing point position detected again is implemented, whereby at least one deformation process using the parameters corresponding to the viewing point position detected again is invalidated. At this time, a parameter update cycle change process ( Figure 7 (D) process) for extending the parameter update cycle may also be used in combination.

[0219] In step S6, it is determined whether a predetermined time (invalidation period) Ta has elapsed. When the result is "No", the process returns to step S5, and when the result is "Yes", the process jumps to step S7.

[0220] In step S7, an update (switch) to the deformation parameters corresponding to the viewing point position detected again is implemented. In principle, the invalidation period ends at this time (however, when implementing the control of the characteristic lines Tk1 and Tk2 in Figure 7 , the control unit can also be designed to extend the substantial invalidation period to the moment when the parameters are completely switched). Here, when the parameter update cycle is not changed in step S5, the process in step S7 ends and jumps to step S8.

[0221] In addition, when the parameter update period is changed in step S5, in step S7, the parameter update period is also restored to the original process in parallel. As a method of return, consider Figure 7 any of the three methods shown in (B) below ((1) to (3) below).

[0222] (1) At the moment of parameter switching (in other words, synchronously with the parameter switching), restore the parameter update period to the original ( Figure 7 the process shown by the solid line in (B) below).

[0223] (2) Temporarily maintain the parameter update period even after parameter switching, and then restore it to the original (according to Figure 7 the process of characteristic line Tk1 in (B) below).

[0224] (3) Restore the parameter update period to the original while changing its value over time ( Figure 7 the process of characteristic line Tk2 in (B) below).

[0225] Next, in step S8, it is determined whether the image correction is completed. If it is "Yes", the process ends; if it is "No", return to step S1.

[0226] Next, refer to Figure 12 . Figure 12 is a flowchart showing an example of a process of deformed image correction control corresponding to viewpoint loss (third control example: corresponding to Figure 8 , Figure 9 ). In Figure 12 , in addition to the steps of Figure 11 , steps S4-1 and S4-2 shown by thick lines in the figure are added. The rest is the same as Figure 11 , so the description of the common process is omitted.

[0227] In step S4-1, it is determined whether the viewpoint loss period (viewpoint loss period) is shorter than a threshold value. If it is "No", jump to step S5.

[0228] When it is "Yes", in step S4-2, as the invalidation period (delay time of parameter switching), use Te (Ta) ( Figure 8 in the case of Figure 9 ), or use Tf (<Ta) (

[0229] in the case of Figure 13 . Figure 13 shows an example of a process of deformed image correction control corresponding to viewpoint loss (third control example: corresponding to Figure 10 ) flowchart.

[0230] In step S10, the vehicle speed is detected. In step S12, the driving state of the vehicle (including stop) is determined. For example, discrimination of each state of low speed, medium speed, and high speed is implemented.

[0231] In step S12, the viewing point is monitored, and in step S13, it is determined whether there is a loss of the viewing point (viewpoint loss). If the result is "no", the process returns to step S12; if "yes", the process jumps to step S14.

[0232] In step S14, the control example 1 described previously is implemented, or Figure 11 a modified process such as control example 2 is performed. Then, in step S15, it is determined whether the image correction is completed. If the result is "yes", the process ends; if "no", the process returns to step S10. Figure 12

[0233] Next, refer to Figure 14 . Figure 14 (A) in is a diagram showing a display example of a road surface overlapping HUD, Figure 14 (B) in is a diagram showing a display example of an inclined surface HUD, Figure 14 (C) in is a diagram showing a structural example of the main part of the HUD device.

[0234] Figure 14 (A) in represents a virtual virtual image display surface PS corresponding to the image display surface ( Figure 5 symbol 116 of or Figure 14 symbol 161 of (C) in ) configured to overlap with the road surface 41 in front of the vehicle 1, which is a virtual image display example of a road surface overlapping HUD. Figure 5 symbol 117 of or Figure 14 symbol 163 of (C) in ) of the virtual image display surface PS is arranged such that the proximal end, which is the end closer to the vehicle 1 side of the virtual image display surface PS, has a smaller distance from the road surface 41, and the distal end, which is the end farther from the vehicle 1 side, has a larger distance from the road surface 41. This is a virtual image display example of an inclined surface HUD.

[0235] Figure 14 (B) in represents a virtual image display example of an inclined surface HUD in which the virtual image display surface PS is arranged inclined with respect to the road surface 41 in such a way that the distance between the proximal end, which is the end closer to the vehicle 1 side of the virtual image display surface PS, and the road surface 41 is small, and the distance between the distal end, which is the end farther from the vehicle 1 side, and the road surface 41 is large.

[0236] They use a relatively wide virtual image display surface PS that overlaps with the road surface 41 or a relatively wide virtual image display surface PS that is inclined with respect to the road surface 41. For example, various displays can be performed within a range of 5 m to 100 m in front of the vehicle 1. The HUD device becomes larger, and thus the viewing area EB also becomes larger. It is preferable to detect the viewing point position with high precision within a wider range than before, and perform image correction using appropriate deformation parameters. However, if the viewing point is lost, the switching control of the high-precision deformation parameters will instead reduce the visual recognition of the image (virtual image) after the viewing point is detected again. Therefore, the application of the control method of the present invention is effective.

[0237] Next, refer to Figure 14 in (C) of Figure 14 The HUD device 107 in (C) of

[0238] In Figure 14 the example of (C) of

[0239] When the HUD device 107 adjusts the position of the viewing area EB at the height position corresponding to the driver's viewing point A, instead of moving the curved mirror 131 (without providing an actuator for the curved mirror 131) which is an optical component that projects the display light onto the windshield 2, it responds by changing the reflection position of the display light 51 of the image on the optical component.

[0240] In other words, when adjusting the height position of the viewing area EB at the height position corresponding to the driver's eyes (viewing point), instead of using an actuator, for example, to rotate the optical component that projects light onto the projection component 2, it responds by changing the reflection position of the light in the optical component. In addition, the height direction refers to the Y direction in the figure (the direction perpendicular to the road surface 41, and the direction away from the road surface 41 is the positive direction). In addition, the X direction is the left-right direction of the vehicle 1, and the Z direction is the front-rear direction (or forward direction) of the vehicle 1.

[0241] In such a large-sized optical component, by optimally designing its reflecting surface as a free-form surface or the like, the distortion of the virtual image is minimized as much as possible. However, as described above, for example, when the driver's viewing point A is located around the viewing area EB, there are cases where the distortion becomes obvious anyway.

[0242] Therefore, in such a case, by implementing control to temporarily invalidate (delay) the application of the parameter corresponding to the viewing point position after re-detection within a specified range, it is possible to prevent the uncomfortable feeling caused by the change in appearance due to the distortion of the virtual image, and the above control can be effectively utilized to improve visual recognition.

[0243] As described above, according to the present invention, when implementing the viewing point position following deformation control that updates the deformation parameter according to the driver's viewing point position, it is possible to effectively suppress the uncomfortable feeling caused to the driver by the instantaneous change in the appearance of the image due to the update of the deformation parameter after the occurrence of viewing point loss (viewpoint dropout).

[0244] The present invention can be used in either a monocular type HUD device or a parallax type HUD device. In the monocular type HUD device, the display light of the same image is incident on the left eye and the right eye. In the parallax type HUD device, images with parallax are incident on the left eye and the right eye.

[0245] In this specification, the term "vehicle" can also be interpreted in a broad sense as a means of transportation. In addition, terms related to navigation (such as signs, etc.) should also be interpreted in a broad sense. For example, from the perspective of generalized navigation information that helps the operation of the vehicle. In addition, the HUD device also includes a device used as a simulator (such as an aircraft simulator).

[0246] The present invention is not limited to the above-exemplified embodiments. In addition, those skilled in the art can easily change the above-exemplified embodiments to the scope included in the claims.

Claims

1. A display control device, characterized in that, the display control device controls a head-up display device mounted on a vehicle and configured to project an image displayed on a display unit onto a projection member provided in the vehicle, so that a driver visually recognizes a virtual image of the image; the display control device includes a control unit configured to update a deformation parameter according to a viewpoint position of the driver in a viewpoint area, and perform viewpoint position following deformation control of deforming an image displayed on the display unit in advance in a manner having a characteristic opposite to a distortion characteristic of the virtual image of the image by using the deformation parameter; when the control unit detects a viewpoint loss in which a position of at least one of the left and right viewpoints of the driver is unknown, during the viewpoint loss period, the deformation parameter set before the viewpoint loss period is maintained; when the position of the viewpoint is detected again after the viewpoint loss period, at least one deformation process using a deformation parameter corresponding to the viewpoint position detected again is invalidated.

2. The display control device according to claim 1, characterized in that, when the control unit compares the viewpoint loss period with a threshold value and the viewpoint loss period is shorter than the threshold value, the following control is performed: compared with a case where the viewpoint loss period is longer than the threshold value, the period for invalidating the deformation process is extended.

3. The display control device according to claim 1, characterized in that, when the control unit compares the viewpoint loss period with a threshold value and the viewpoint loss period is shorter than the threshold value, the following control is performed: compared with a case where the viewpoint loss period is longer than the threshold value, the period for invalidating the deformation process is shortened.

4. The display control device according to any one of claims 1 to 3, characterized in that, when the control unit sets an update period of the deformation parameter before the occurrence of the viewpoint loss and during the viewpoint loss period as a first update period RT1, and sets an update period of the deformation parameter during the period for invalidating the deformation process as a second update period RT2, the parameter update period is changed to RT1 < RT2.

5. The display control device according to claim 4, characterized in that, after the control unit changes the update period of the deformation parameter from the first update period RT1 to the second update period RT2, at the end moment of the period for invalidating the deformation process, it is restored from the second update period RT2 to the first update period RT1, or, at a moment when a predetermined time further elapses from the end moment of the period for invalidating the deformation process, it is restored from the second update period RT2 to the first update period RT1; or, starting from the end moment of the period for invalidating the deformation process, the change of the parameter update period is started, and it is gradually restored from the second update period RT2 to the first update period RT1 as time elapses.

6. The display control device according to any one of claims 1 to 3, characterized in that, it further includes a low-speed state determination unit configured to determine whether the speed of the vehicle is in a low-speed state. The period during which the control unit invalidates the deformation process when the vehicle is in the low-speed state including the stop state is longer than the period during which the deformation process is invalidated in a state faster than the low-speed state.

7. The display control device according to any one of claims 1 to 3, wherein: the control unit changes the period during which the deformation process is invalidated according to the vehicle speed. In this case, the following control is implemented: when the vehicle speed is in the range greater than or equal to the first speed value U1 (U1 > 0) and less than or equal to the second speed value U2 greater than the first speed value, the period during which the deformation process is invalidated decreases with respect to the vehicle speed as the vehicle speed increases, or the following control is implemented: the degree of decrease is slowed down in the range where the vehicle speed is close to the first speed value, and the degree of decrease is intensified as the vehicle speed moves away from the first speed value, or the following control is implemented: in the range where the vehicle speed is close to the first speed value, the degree of decrease is slowed down, as the vehicle speed moves away from the first speed value, the degree of decrease is more intensified, and as the vehicle speed approaches the second speed value, the degree of decrease is slowed down.

8. The display control device according to any one of claims 1 to 3, wherein: the head-up display device includes an optical system, and the optical system includes an optical component that reflects the display light of the image and projects it onto the projection component, when the head-up display device adjusts the position of the viewing area according to the height position of the driver's viewing point, the reflection position of the display light of the image on the optical component is changed without moving the optical component.

9. The display control device according to any one of claims 1 to 3, wherein: the virtual virtual image display surface corresponding to the image display surface of the display unit is arranged to overlap with the road surface in front of the vehicle, or it is arranged to be inclined with respect to the road surface in such a way that the distance between the proximal end, which is the end of the virtual image display surface closer to the vehicle, and the road surface becomes smaller, and the distance between the distal end, which is the end of the virtual image display surface farther from the vehicle, and the road surface becomes larger.

10. A head-up display device, wherein: it includes: the display control device according to any one of claims 1 to 9; a display unit that displays an image; and an optical system that includes an optical component that reflects the display light of the image and projects it onto the projection component.

Images