Display control device, display device, and image display control method
The display control device addresses image fluctuations due to vehicle pitching and rolling by adjusting correction amounts based on viewer height, enhancing stability and reducing discomfort.
Patent Information
- Application Number
- JP2021027376
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2021-02-24
- Publication Date
- 2024-11-06
- Estimated Expiration
- 2041-02-24
AI Technical Summary
Existing image correction methods for vehicles fail to adequately address fluctuations in image position due to pitching and rolling, particularly when viewer height varies, leading to reduced suppression effectiveness and increased discomfort.
A display control device that adjusts image correction amounts based on viewer height position, using variable or fixed correction methods to optimize image stability during vehicle attitude changes.
Enhances image stability and reduces viewer discomfort by dynamically adjusting image correction to match viewer height, improving the accuracy of superimposed images on vehicle displays.
Smart Images

Figure 0007578022000001 
Figure 0007578022000002 
Figure 0007578022000003
Abstract
Description
[Technical field]
[0001] The present invention relates to a display control device, a display device, an image display control method, and the like. [Background technology]
[0002] Image correction (pitching correction, rolling correction) for correcting image position fluctuations caused by pitching and rolling of a vehicle is described in, for example, Patent Document 1.
[0003] Vehicle pitching is a state in which the vehicle rotates back and forth around its left-right axis (an axis that passes through the center of the vehicle (e.g., the center of gravity) and extends in the left-right direction of the vehicle), causing a so-called vertical sway, and the vehicle's posture becomes tilted forward or backward.
[0004] Vehicle rolling is a state in which the vehicle rotates left and right around the front-to-rear axis (an axis that passes through the center of the vehicle (e.g., the center of gravity) and extends in the fore-and-aft direction of the vehicle), causing a so-called lateral sway, and the vehicle's posture becomes tilted to the left or right. [Prior art documents] [Patent documents]
[0005] [Patent Document 1] Patent No. 5161760 (Figs. 3 to 7, etc.) Summary of the Invention [Problem to be solved by the invention]
[0006] The present inventors have conducted studies and have found the following problems. (1) When pitching or rolling occurs, the vehicle (car body) rises or sinks, causing the position of the displayed image to fluctuate. (2) Here, the degree of the image position change differs depending on whether the viewer's (driver's, etc.) viewpoint (eye) position is high or low. In other words, when the distance from the central axis of pitching or rolling to the viewpoint (eye) differs, the greater the distance, the greater the effect of the vehicle's sway (attitude change), and the greater the change in the displayed image in the vehicle height direction, resulting in greater fluctuation in the image display position. (3) Therefore, even if pitch correction or rolling correction is performed with a uniform correction amount without taking into account the above-mentioned variation in the height position of the viewpoint, there is a limit to the effect of suppressing the fluctuation of the display position of the image. If the suppression effect is low, the viewer's discomfort increases. (4) Furthermore, if the effect of suppressing fluctuations in the display position of an image is low, the accuracy of superimposing an image (a superimposed image of a vehicle ahead, such as a warning mark) on a forward object (such as a vehicle ahead) displayed by a head-up display (HUD) device may decrease.
[0007] Such a problem was identified by the inventors' investigations. However, the above-mentioned Patent Document 1 does not mention such a problem, nor does it mention any measures to address it.
[0008] One of the objectives of the present invention is to improve the reduction in the effect of suppressing fluctuations in the display position of an image due to pitching and rolling of a vehicle, which is caused by differences in the viewpoint (eye) height of the viewer of the image (driver or passenger).
[0009] Other objects of the present invention will become apparent to those skilled in the art by referring to the following exemplary aspects and best modes, as well as the accompanying drawings. [Means for solving the problem]
[0010] In order to facilitate an understanding of the outline of the present invention, embodiments according to the present invention will be illustrated below.
[0011] In a first aspect, a display control device includes: A display control device that is mounted on a vehicle and controls image display on a display device that allows a viewer riding on the vehicle to view an image, A vehicle attitude information acquisition unit that acquires attitude information of the vehicle; a viewpoint position information acquisition unit that acquires viewpoint height position estimation information that can estimate a viewpoint height position of the viewer; a control unit that controls image correction including at least one of pitching correction that corrects the image so as to suppress a fluctuation in a display position of the image due to pitching of the vehicle and rolling correction that corrects the image so as to suppress a fluctuation in a display position of the image due to rolling of the vehicle, The control unit is When the viewpoint height position is estimated to be higher than the standard position as a result of estimation based on the viewpoint height position estimation information acquired by the viewpoint position information acquisition unit, a correction amount of the image correction is made larger than the correction amount at the standard position, and when the viewpoint height position is estimated to be lower, a control is performed to make the correction amount smaller than the correction amount at the standard position. death, The control unit is The amount of image correction is varied according to the degree of the estimated viewpoint height position, If the estimated viewpoint height position is higher than the standard position, the amount of image correction is increased as the viewpoint height position becomes higher; If the estimated viewpoint height position is lower than the standard position, the amount of image correction is decreased as the viewpoint height position becomes lower; The control unit is When the amount of image correction is varied according to the degree of the estimated viewpoint height position, the image correction is stopped when the viewpoint height position is outside a predetermined height range.
[0012] In the first mode, image correction is performed when the viewpoint height position is within a predetermined height range, and image correction is stopped when the viewpoint height position is outside the predetermined height range (in other words, the amount of image correction is set to zero).
[0013] If the viewpoint height is too high, the vehicle's attitude change will have a large effect on the image display position, and the image may fluctuate unnaturally even if image correction is performed. In addition, if image correction is performed too forcefully, the unnatural image fluctuations caused by the image correction may give the viewer a sense of discomfort. Taking these points into consideration, it was decided not to perform image correction.
[0014] In addition, if the viewpoint height position is too low, the effect of the vehicle's attitude change is also small, so it is considered that stopping image correction will have little effect. Also, if image correction is forcibly performed to accommodate small image position changes, the minute changes in the image position that accompany the image correction may give the viewer a sense of discomfort. Taking these points into consideration, it was decided not to perform image correction.
[0015] In a second aspect, a display control device includes: A display control device that is mounted on a vehicle and controls image display on a display device that allows a viewer riding on the vehicle to view an image, A vehicle attitude information acquisition unit that acquires attitude information of the vehicle; a viewpoint position information acquisition unit that acquires viewpoint height position estimation information that can estimate a viewpoint height position of the viewer; a control unit that controls image correction including at least one of pitching correction that corrects the image so as to suppress a fluctuation in a display position of the image due to pitching of the vehicle and rolling correction that corrects the image so as to suppress a fluctuation in a display position of the image due to rolling of the vehicle, The control unit is when the viewpoint height position is estimated to be higher than a standard position as a result of estimation based on the viewpoint height position estimation information acquired by the viewpoint position information acquisition unit, a correction amount of the image correction is made larger than the correction amount at the standard position, and when the viewpoint height position is estimated to be lower, a correction amount of the image correction is made smaller than the correction amount at the standard position; The control unit is When the estimated viewpoint height position is in a range higher than the standard position, a correction amount of a predetermined fixed value is set as a correction amount of the image correction; When the estimated viewpoint height position is in a range lower than the standard position, the amount of image correction is varied according to the degree of the viewpoint height position, and the amount of image correction is reduced as the viewpoint height position becomes lower.
[0016] In the second aspect, variable control of the amount of image correction is performed when the viewpoint height position is in a range lower than the standard position, and when the viewpoint height position is in a range higher than the standard position, the amount of correction is set to a predetermined fixed value (for example, the fixed value is an integer including zero).
[0017] By using a fixed correction amount when the viewpoint height position is higher than the standard position, there is no need to switch the correction amount significantly, and the burden on the control unit (e.g., the processing burden of calculating and obtaining the correction amount) can be reduced.
[0018] In a third aspect, a display control device includes: A display control device that is mounted on a vehicle and controls image display on a display device that allows a viewer riding on the vehicle to view an image, A vehicle attitude information acquisition unit that acquires attitude information of the vehicle; a viewpoint position information acquisition unit that acquires viewpoint height position estimation information that can estimate a viewpoint height position of the viewer; a control unit that controls image correction including at least one of pitching correction that corrects the image so as to suppress a fluctuation in a display position of the image due to pitching of the vehicle and rolling correction that corrects the image so as to suppress a fluctuation in a display position of the image due to rolling of the vehicle, The control unit is when the viewpoint height position is estimated to be higher than a standard position as a result of estimation based on the viewpoint height position estimation information acquired by the viewpoint position information acquisition unit, a correction amount of the image correction is made larger than the correction amount at the standard position, and when the viewpoint height position is estimated to be lower, a correction amount of the image correction is made smaller than the correction amount at the standard position; The control unit is When the estimated viewpoint height position is in a range lower than the standard position, a correction amount of a predetermined fixed value is set as the correction amount of the image correction; When the estimated viewpoint height position is in a range higher than the standard position, the amount of image correction may be varied according to the degree of the viewpoint height position, and the amount of image correction may be increased as the viewpoint height position becomes higher.
[0019] In the third aspect, variable control of the amount of image correction is performed when the viewpoint height position is in a range higher than the standard position, and when the viewpoint height position is in a range lower than the standard position, the amount of correction is set to a predetermined fixed value (for example, the fixed value is an integer including zero).
[0020] By using a fixed correction amount when the viewpoint height position is lower than the standard position, the effort of frequently changing the correction amount by small increments can be eliminated, thereby reducing the burden on the control unit (e.g., the processing burden of calculating and obtaining the correction amount).
[0021] In a fourth aspect dependent on any one of the first to third aspects, The viewpoint height position estimation information is In a case where the display device includes an optical system including a rotatable curved mirror that projects an image onto a projection target, the information may be position information of the curved mirror, Or, The information may be image display position information on the image display surface of the display unit when the display device comprises a display unit having an image display surface and an optical system including a non-rotatable curved mirror included in the display device for projecting an image onto a projection target.
[0022] In a fourth aspect, the viewpoint height position estimation information can be the position information of a rotatable curved mirror (such as a concave mirror), or, in the case where the curved mirror (such as a concave mirror) is non-rotatable, image display position information on the display unit.
[0023] Since the height position of the eyebox changes due to the rotation of a rotatable curved mirror (e.g., a concave mirror), position information of the curved mirror (e.g., the number of rotations of an actuator, which is a component of the rotation mechanism, or the rotation angle of the curved mirror, etc.) can be used as objective data for estimating the height position of the viewer's viewpoint.
[0024] Furthermore, when a curved mirror (such as a concave mirror) cannot rotate, the incident position of the display light of the image on the reflective surface of the curved mirror that reflects the light can be adjusted to control the reflection direction of the light from the reflective surface and change the height position of the eyebox. The change in the incident position of the display light on the reflective surface is ultimately realized by adjusting the image display position on the image display surface of the display unit that displays the image. Therefore, the image display position information can be used as objective data for estimating the height position of the viewer's viewpoint.
[0025] By utilizing the viewpoint height position estimation information, it is possible to identify the viewpoint height position more easily and quickly compared to when the viewpoint height position is captured by a camera and detected by image analysis.
[0026] In a fifth aspect dependent on any one of the first to fourth aspects, The viewpoint position information acquisition unit is In addition to the viewpoint height position estimation information that can estimate the viewpoint height position of the viewer, Obtaining viewpoint height position detection information that is detected based on an image of the viewer captured by a camera and has a higher viewpoint height position resolution than the viewpoint height position estimation information; The control unit is The viewpoint height position estimated based on the viewpoint height position estimation information may be fine-tuned using the viewpoint height position detection information.
[0027] In the fifth aspect, the viewpoint height position estimation information and viewpoint height position detection information, which has a higher resolution of the viewpoint height position, are used in combination. For example, the height position of the eyebox is estimated based on the viewpoint height position estimation information, and the viewpoint height position in the eyebox can be determined using the viewpoint height position detection information (for example, the viewpoint height position in the eyebox can be classified and determined as "high", "medium", or "low"). This allows the viewpoint height position to be determined (specified) more precisely, and the detection accuracy of the viewpoint height position can be improved. Therefore, more appropriate image correction can be performed.
[0028] In a sixth aspect dependent on the fifth aspect, The control unit is Viewpoint tracking warping is performed for each of a plurality of divided regions within the eyebox, and viewpoint position information obtained during the warping, which indicates which of the plurality of regions the viewer's viewpoint is located in, may also be used as the viewpoint height position detection information.
[0029] In the sixth aspect, the viewpoint position information obtained by imaging with a camera is used in common for viewpoint tracking warping and pitching / rolling correction taking into account the viewpoint height position. This eliminates the need to obtain viewpoint height position detection information separately for pitching correction or rolling correction, thereby reducing the burden on the control unit, etc.
[0030] In a seventh aspect dependent on any one of the first to sixth aspects, The control unit is When setting the amount of image correction, The pitching correction and the rolling correction may be handled separately, and a correction amount appropriate for each correction may be set.
[0031] According to the seventh aspect, it is possible to perform appropriate image correction for each of pitching and rolling.
[0032] In an eighth aspect, the display device comprises: A display control device according to any one of the first to seventh aspects; a correction unit that performs image correction including at least one of the pitching correction and the rolling correction; a display unit for displaying the corrected image; has.
[0033] According to the eighth aspect, it is possible to realize a high-performance display device capable of displaying an image with little positional fluctuation even when the vehicle pitches or rolls. The display device is broadly interpreted and may include, for example, a head-up display (HUD) device, a flat panel display, and the like.
[0034] In a ninth aspect dependent on the eighth aspect, The display device includes: The head-up display device may further include an optical system that projects the display light of the image displayed on the display unit onto a projection member provided in the vehicle, and has the function of allowing at least one of the driver and the passenger to view a virtual image.
[0035] According to the ninth aspect, it is possible to realize a high-performance head-up display (HUD) device capable of displaying an image with little positional fluctuation even when the vehicle pitches or rolls.
[0036] In a tenth aspect, an image display control method includes: An image display control method for a display device mounted on a vehicle, comprising: acquiring attitude information of the vehicle, the attitude information including at least one of pitching and rolling; acquiring viewpoint height position estimation information capable of estimating a viewpoint height position of a viewer of an image displayed by the display device; a step of variably controlling an amount of image correction for correcting a positional change of an image displayed by the display device due to a change in the attitude of the vehicle by making an amount of image correction larger than an amount of image correction at the standard position when it is estimated that the viewpoint height position of the viewer is higher than a standard position, and making an amount of image correction smaller than an amount of image correction at the standard position when it is estimated that the viewpoint height position is lower than the standard position; correcting an image displayed by the display device in accordance with the variably controlled correction amount; varying an amount of correction of the image correction according to the degree of the viewpoint height position, and when the estimated viewpoint height position is higher than the standard position, increasing the amount of correction of the image correction as the viewpoint height position becomes higher, and when the viewpoint height position is lower than the standard position, decreasing the amount of correction of the image correction as the viewpoint height position becomes lower; When the amount of image correction is varied according to the degree of the viewpoint height position, stopping the image correction when the viewpoint height position is outside a predetermined height range; Includes.
[0037] In an eleventh aspect, an image display control method includes: An image display control method for a display device mounted on a vehicle, comprising: acquiring attitude information of the vehicle, the attitude information including at least one of pitching and rolling; acquiring viewpoint height position estimation information capable of estimating a viewpoint height position of a viewer of an image displayed by the display device; a step of variably controlling an amount of image correction for correcting a positional change of an image displayed by the display device due to a change in the attitude of the vehicle by making an amount of image correction larger than an amount of image correction at the standard position when it is estimated that the viewpoint height position of the viewer is higher than a standard position, and making an amount of image correction smaller than an amount of image correction at the standard position when it is estimated that the viewpoint height position is lower than the standard position; correcting an image displayed by the display device in accordance with the variably controlled correction amount; a step of setting a predetermined fixed value as the amount of correction for the image correction when the viewpoint height position is in a range higher than the standard position, and varying the amount of correction for the image correction according to the level of the viewpoint height position when the viewpoint height position is in a range lower than the standard position, and decreasing the amount of correction for the image correction as the viewpoint height position becomes lower; Includes.
[0038] In a twelfth aspect, an image display control method includes: An image display control method for a display device mounted on a vehicle, comprising: acquiring attitude information of the vehicle, the attitude information including at least one of pitching and rolling; acquiring viewpoint height position estimation information capable of estimating a viewpoint height position of a viewer of an image displayed by the display device; a step of variably controlling an amount of image correction for correcting a positional change of an image displayed by the display device due to a change in the attitude of the vehicle by making an amount of image correction larger than an amount of image correction at the standard position when it is estimated that the viewpoint height position of the viewer is higher than a standard position, and making an amount of image correction smaller than an amount of image correction at the standard position when it is estimated that the viewpoint height position is lower than the standard position; correcting an image displayed by the display device in accordance with the variably controlled correction amount; a step of setting a predetermined fixed value of the correction amount of the image correction when the viewpoint height position is in a range lower than the standard position, and varying the correction amount of the image correction according to the degree of the viewpoint height position when the viewpoint height position is in a range higher than the standard position, and increasing the correction amount of the image correction as the viewpoint height position becomes higher; Includes.
[0039] Those skilled in the art will easily understand that the exemplified embodiments according to the present invention can be further modified without departing from the spirit of the present invention. [Brief description of the drawings]
[0040] [Figure 1] FIG. 1A is a diagram for explaining pitching and pitching correction, and FIG. 1B is a diagram for explaining rolling and rolling correction. [Diagram 2] FIG. 2 is a diagram showing an example of the configuration of a display control device and an example of the configuration of a main part of a display system using the display control device. [Diagram 3]FIG. 3 is a diagram showing a configuration example of a head-up display (HUD) device including a display control device, and a configuration example of a display system including the HUD device. [Figure 4] FIG. 4A is a diagram showing an example of a correction table, and FIG. 4B is a diagram showing an example of setting the rate of change of the image correction amount with respect to the fluctuation of the viewpoint height position estimation information. [Diagram 5] 5A and 5B are diagrams showing other examples of the correction table. [Figure 6] 6(A) and (B) are diagrams showing still another example of the correction table, and FIG. 6(C) is a diagram showing an example of a look-up table. [Figure 7] FIG. 7(A) is a diagram showing an example of the main configuration of a display system that estimates the height position of the eyebox using viewpoint height position estimation information and determines (identifies) the viewpoint height position within the eyebox using viewpoint position detection information, FIG. 7(B) is a diagram showing an example of the internal configuration of a correction table, and FIG. 7(C) is a diagram showing an example of a lookup table that adopts the configuration of FIG. 7(B). [Figure 8] FIG. 8 is a diagram showing another example of a HUD device (an example in which the curved mirror is non-rotatable). [Figure 9] FIG. 9 is a flowchart illustrating an example of a procedure for image correction processing. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0041] The best mode described below is used to easily understand the present invention. Therefore, those skilled in the art should be aware that the present invention is not unduly limited by the mode described below.
[0042] (First embodiment) Please refer to Fig. 1. Fig. 1(A) is a diagram for explaining pitching and pitching correction, and Fig. 1(B) is a diagram for explaining rolling and rolling correction.
[0043] 1, the X direction indicates the left-right direction of the vehicle 1, the Y direction indicates the height direction of the vehicle 1 (the direction along the perpendicular line to the road surface), and the Z direction indicates the front (front-rear direction) of the vehicle 1. The XYZ Cartesian coordinate system is a coordinate system of real space based on the road surface.
[0044] Pitching of vehicle 1 is a state in which the vehicle rotates back and forth around the left-right axis of vehicle 1 (an axis that passes through the center of the vehicle (e.g., the center of gravity) and extends in the left-right direction of vehicle 1), causing a so-called vertical sway, and the posture of vehicle 1 becomes a backward-leaning posture (see Figure 1(A)) or a forward-leaning posture (see Figure 1(B)).
[0045] 1A, the vehicle 1 is traveling on a flat road surface 4, but due to sudden acceleration, for example, the front end of the body of the vehicle 1 is lifted off the road surface 4, and the body of the vehicle 1 is in a backward tilted posture. The pitching angle is θa.
[0046] 1B, for example, due to sudden braking, the rear end of the body of the vehicle 1 is lifted off the road surface 4, and the body of the vehicle 1 is in a forward tilted position. The pitch angle is θb.
[0047] When pitching occurs, a misalignment (mismatch) occurs between the XYZ Cartesian coordinate system (first coordinate system) set in real space based on the road surface 4 and the local X1Y1Z1 Cartesian coordinate system (second coordinate system) that fluctuates depending on the posture of the body of the vehicle 1. This misalignment causes a fluctuation in the display position of the image. Pitching correction is an image correction that suppresses this fluctuation (basically, a coordinate conversion correction that converts the second coordinate system into the first coordinate system).
[0048] In a HUD device, for example, the position where the image display light is projected onto the windshield (projected member) may change due to pitching or the like, which may cause the shape of the displayed image to fluctuate. In this case, however, this can be addressed by warping correction (a correction that applies in advance to the image a distortion that is opposite to the distortion caused by the optical system).
[0049] As shown by the solid lines in the upper part of Figures 1(A) and (B), before pitching occurs, a warning mark (virtual image) 70 is displayed superimposed on the object ahead (e.g., the vehicle ahead) 60, but when pitching occurs, as shown by the dashed lines, the position of the warning mark (virtual image) moves in the vertical direction (the height direction (Y direction) of the vehicle 1 in the XYZ Cartesian coordinate system). In Figure 1(A), the amount of displacement is indicated as ΔD1. In Figure 1(B), the amount of displacement is indicated as ΔD2. In addition, in the figures, the warning mark (virtual image) after the positional movement has occurred is indicated by the symbol 70'.
[0050] The instantaneous change in the position of the image being displayed may cause discomfort to the driver or other passengers. Also, as in the examples of Figures 1(A) and (B), the superimposition of the warning mark 70 on the upcoming object 60 may be reduced, which may also cause discomfort.
[0051] In addition, the rolling of the vehicle 1 is a state in which the vehicle 1 rotates left and right around the longitudinal axis (an axis that passes through the center of the vehicle (e.g., the center of gravity) and extends in the longitudinal direction of the vehicle 1), causing a so-called lateral sway, and the posture of the vehicle 1 becomes a right-leaning posture (see FIG. 1(C)) or a left-leaning posture (see FIG. 1(D)).
[0052] When rolling occurs, a misalignment (mismatch) occurs between the XYZ Cartesian coordinate system (first coordinate system) set in real space based on the road surface 4 and the local X1Y1Z1 Cartesian coordinate system (second coordinate system) that fluctuates depending on the posture of the body of the vehicle 1. This misalignment causes fluctuations in the display position of the image. Rolling correction is an image correction that suppresses this fluctuation (basically, a coordinate conversion correction that converts the second coordinate system into the first coordinate system).
[0053] 1C, for example, the steering wheel is turned to the right, causing the left end of the body of the vehicle 1 to lift off the road surface 4, and the body of the vehicle 1 to tilt to the right. The rolling angle is θc.
[0054] 1(D), for example, the steering wheel is turned to the left, causing the right end of the body of the vehicle 1 to lift off the road surface 4, and the body of the vehicle 1 to lean to the left. The rolling angle is θd.
[0055] As shown by the solid lines in the upper part of Figures 1(C) and (D), before rolling occurs, a warning mark (virtual image) 70 is displayed superimposed on the object ahead (e.g., the vehicle ahead) 60, but when rolling occurs, as shown by the dashed lines, the position of the warning mark (virtual image) moves in the left-right direction (the width direction (X direction) of the vehicle 1 in the XYZ Cartesian coordinate system) and the up-down direction (the height direction (Y direction) of the vehicle 1 in the XYZ Cartesian coordinate system).
[0056] In Fig. 1(C), the amount of displacement in the left-right direction (yawing component) is indicated as ΔL1, and the amount of displacement in the up-down direction is indicated as ΔD3. In Fig. 1(D), the amount of displacement in the left-right direction (yawing component) is indicated as ΔL2, and the amount of displacement in the up-down direction is indicated as ΔD4. In addition, the symbol 70' is attached to the attention mark (virtual image) after the position movement has occurred.
[0057] Regarding the left-right displacement amounts (yawing components) ΔL1 and ΔL2 in Figures 1(C) and (D), there is no variation due to differences in the height position (position along the height direction (Y direction) of vehicle 1 in the XYZ Cartesian coordinate system) of the viewpoint (eyes) of the driver (or other passengers) riding in vehicle 1.
[0058] However, for the vertical displacement amounts ΔD1 to ΔD4 in Figures 1(A) to 1(D), if the viewpoint height position differs, the displacement amount also varies, and therefore image correction using a uniform correction amount has a limit to the accuracy of position correction. Therefore, it is important to variably control the correction amount by taking into account the height position of the viewer's viewpoint (eyes).
[0059] Please refer to Fig. 2. Fig. 2 is a diagram showing an example of the configuration of a display control device and an example of the configuration of a main part of a display system using the display control device.
[0060] In the display system of Figure 2 (assuming that a HUD device is used as the display device), the height position (position in the Y direction) of the eye box EB can be adjusted to match the height position of the viewer's (driver's, etc.) viewpoint (eyes).
[0061] In Fig. 2, the eyebox EB in the standard position is shown by a solid line, a part of the eyebox EB' that is higher than the standard position (indicated as "high" in the figure) is shown by a dashed line, and the eyebox EB'' that is lower than the standard position (indicated as "low" in the figure) is shown by a dashed and dotted line. Note that the eyebox EB has a three-dimensional shape, but for ease of explanation, Fig. 2 shows it in a two-dimensional form.
[0062] In FIG. 2, the eye box EB is divided into a number of regions (Z1 to Z9), and the position of the viewpoint (eye) can be detected for each divided region. With this configuration, spot lighting (local image projection) is performed only on the region where the viewpoint (eye) is located, thereby reducing the power consumption of the light source. Also, by adopting viewpoint-tracking warping, which applies a warping to an image in advance that is the opposite of the image distortion caused by the optical system (including the windshield as the projection target member) for each divided region, it is possible to suppress image distortion while reducing power consumption.
[0063] The viewpoint (eye) position can be detected, for example, by capturing an image of the viewer's eyes (or a face, etc.) with the camera 15 and determining (identifying) the viewpoint (eye) position by carrying out image processing or the like in the viewpoint position detection unit 34. In the display system of FIG. 2, viewpoint (eye) position detection information G2 is supplied to a processor (display control device) 36.
[0064] On the other hand, the height position of the eye box EB (EB', EB'') can be specified based on the position information (in other words, rotation information: specifically, the number of rotations and the rotation angle) of the curved mirror (not shown in FIG. 2, reference numeral 28 in FIG. 3) controlled by the curved mirror rotation control unit 31. Here, the height position of the eye box EB is adjusted in advance, for example, so that the viewpoint (eye) 6 is located in an area Z5 (an area located in the center of the nine divided areas in the eye box) in the eye box EB. Therefore, the height position of the eye box EB reflects the height of the viewpoint (eye) 6. Therefore, the height position of the viewpoint (eye) can be estimated by referring to the position information (rotation information: the number of rotations and the rotation angle) of the curved mirror obtained from the curved mirror rotation control unit 31. In the display system of FIG. 2, the position information of the curved mirror is supplied to the processor (display control device) 36 as viewpoint (eye) position estimation information G1.
[0065] The processor (display control device) 36 has a control unit 38 that controls image display. The control unit 38 has a vehicle attitude information acquisition unit 40, a viewpoint position information acquisition unit 41, and a correction control unit (image correction control unit) 43 that comprehensively controls image correction.
[0066] The correction control unit (image correction control unit) 43 has a pitching / rolling correction control unit 42 and a warping control unit 44. Note that the pitching / rolling correction control unit 42 may have a single correction control unit that commonly controls the pitching correction and the rolling correction, or may have a correction control unit for the pitching correction and a correction control unit for the rolling correction separately.
[0067] The pitching and rolling correction control unit 42 refers to the viewpoint (eye) position estimation information G1, or further refers to the viewpoint (eye) position detection information G2, obtains the amount of correction corresponding to the viewpoint height position, and performs image correction (details will be described later).
[0068] Next, reference is made to Fig. 3. Fig. 3 is a diagram showing a configuration example of a head-up display (HUD) device equipped with a display control device, and a configuration example of a display system including the HUD device. In Fig. 3, parts common to Fig. 2 are given the same reference numerals.
[0069] 3, a head-up display (HUD) device 100 is housed, for example, inside a dashboard 7 of a vehicle 1. The HUD device 100 has a light source unit 24 (including a display unit 22 having a flat panel display or the like), a plane mirror 26 and a curved mirror (concave mirror or the like) 28 as an optical system, a rotation mechanism (including an actuator) 30 that rotates and drives the curved mirror (concave mirror or the like) 28, a curved mirror rotation control unit 31 that controls the rotation of the curved mirror, an input interface (input I / F) 32, a viewpoint position control unit 34, a processor (display control device) 36, a memory (storage device or storage unit) 50, a drawing memory 65, and an image processing unit 80.
[0070] 2, the processor (display control device) 36 has a control unit 38 that controls image display, and this control unit 38 has a vehicle attitude information acquisition unit 40, a viewpoint position information acquisition unit 41, and a correction control unit (image correction control unit) 43 that comprehensively controls image correction. The correction control unit (image correction control unit) 43 has a pitching / rolling correction control unit 42 and a warping control unit 44.
[0071] The memory 50 has a correction table 52. The correction table 52 includes a pitching correction table 54, a rolling correction table 56, and a warping correction table 58. When the amount of image correction is calculated by calculation without using the correction table, the memory 50 stores correction calculation data 59.
[0072] The image processing unit 80 has a bus 82, an image generation unit 84, a correction unit (correction processing unit) 86 that corrects image data using correction amount information supplied from the processor (display control device) 36, and an output buffer 88 that supplies the corrected image data to the light source unit 24.
[0073] In the example of FIG. 3, a camera 15 that captures an image of the viewpoint (eyes) and face of the viewer 5 (5', 5'') is disposed inside the dashboard 7 together with the HUD device 100.
[0074] Further, a roll angle sensor 9 and a pitch angle sensor 11 (which are configured by, for example, an acceleration sensor) are provided to acquire attitude information (roll angle, pitch angle) of the vehicle 1. Information on the roll angle and pitch angle detected by these sensors is supplied to an input interface (input I / F) 32 of the HUD device 100 via a vehicle ECU 13 as vehicle attitude information G0.
[0075] 2 is supplied to the input I / F 32 from the curved mirror rotation control unit 31. Also, viewpoint height position detection information G2 is supplied to the input I / F 32 from the viewpoint position detection unit 34. The vehicle attitude information G0 is acquired by a vehicle attitude information acquisition unit 40 of the processor (display control device) 36, and the rotation information (viewpoint height position estimation information) G1 and the viewpoint height position detection information G2 are acquired by a viewpoint position information acquisition unit 41 of the processor (display control device) 36, and the acquired information is used to determine the amount of correction for pitching correction and rolling correction.
[0076] To determine the amount of correction, the pitching and rolling correction control unit 42 can refer to a correction table 52 in the memory 50 as appropriate. The correction table 52 may be in the form of, for example, a look-up table (LUT).
[0077] Furthermore, as shown in the upper part of Fig. 3, the height position (position in the Y direction) of the viewpoint (eye) 6 of the viewer (driver or other passenger) 5 varies depending on the height and posture of the viewer. In Fig. 3, the viewer in the standard position is given the reference number 5, and the viewpoint (eye) at that time is given the reference number 6. The viewpoint (eye) higher than the standard position is given the reference number 6', and the viewer at that time is given the reference number 5'. The viewpoint (eye) lower than the standard position is given the reference number 6'', and the viewer at that time is given the reference number 5''.
[0078] Also, corresponding to the viewpoint (eye) 6 in the standard position, a virtual image V is displayed on the virtual image display surface PS. Also, corresponding to the viewpoint (eye) 6', a virtual image V' is displayed on the virtual image display surface PS'. Also, corresponding to the viewpoint (eye) 6'' a virtual image V'' is displayed on the virtual image display surface PS''. In other words, if the viewpoint (eye) position is higher than the standard position, the displayed virtual image (image) position will be lower, and if it is lower than the standard position, the displayed virtual image (image) position will be higher.
[0079] Corresponding to the change in the height position of the viewpoint (eye) 6 (6', 6''), the curved mirror (concave mirror or the like) 28 is rotated by the rotation mechanism 31, whereby the position of the curved mirror 28 changes appropriately.
[0080] The height position (position in the Y direction) of the eyebox EB changes in response to the change in the position of the curved mirror 28. In Fig. 3, the eyebox in the standard position is marked with EB, the eyebox higher than the standard position is marked with EB', and the eyebox lower than the standard position is marked with EB''.
[0081] In this way, the position information (rotation information: information on the number of rotations and the rotation angle) of the curved mirror 28 corresponds to the height position of the eyebox, and the height position of the eyebox corresponds to the height position of the viewer's viewpoint (eyes). Therefore, the position information (rotation information: information on the number of rotations and the rotation angle) of the curved mirror 28 can be used as objective information that can estimate the height position of the viewer's viewpoint (eyes) (however, this is not limited to this. An example of estimating the height position of the viewpoint (eyes) using other information will be described later).
[0082] Next, reference will be made to Fig. 4. Fig. 4(A) is a diagram showing an example of a correction table, and Fig. 4(B) is a diagram showing an example of setting the rate of change of the image correction amount with respect to the variation of the viewpoint height position estimation information.
[0083] The correction table can be constructed, for example, in the form of a look-up table (LUT). The correction table (LUT) in Fig. 4(A) is accessed using vehicle attitude information (pitch angle θp or roll angle θr) and viewpoint height position estimation information (rotation speed or rotation angle of the curved mirror) as address variables, and one correction amount (any one of correction amounts A to W) corresponding to these inputs is output. Regarding the values of correction amounts A to W, A > B > C ··· > W holds.
[0084] In Fig. 4(A), when the viewpoint (eye) position is at the standard position, correction amount C is selected. The rotation speed at this time is "c", and the rotation angle of the curved mirror is "c'", and this "c" and "c'" are the design centers. In other words, this "c" and "c'" serve as the reference threshold Sth1 for determining whether the viewpoint height is higher or lower than the standard position.
[0085] Here, for example, regarding the numerical values of the rotation speed, a < b < c < ··· < w holds. When the rotation speed is a or b (< c), the eye box EB is set higher than the standard position. Therefore, the viewpoint (eye) position is estimated to be higher than the standard position, and the virtual image to be displayed is displayed at a position lower (down position) than the standard position (see Fig. 3).
[0086] On the other hand, when the rotation speed is c ··· w (> c), the eye box EB is set lower than the standard position. Therefore, the viewpoint (eye) position is estimated to be lower than the standard position, and the virtual image to be displayed is displayed at a position higher (up position) than the standard position (see Fig. 3).
[0087] In this way, based on the viewpoint height position estimation information (rotation speed or rotation angle) capable of estimating the viewer's viewpoint height position, the viewer's viewpoint height position is estimated. When the estimated viewpoint height position is estimated to be higher than the standard position, the correction amount for image correction is increased, and when it is estimated to be lower than the standard position, the correction amount for image correction is decreased.
[0088] A high viewpoint height position means that the distance from the central axis of pitching or rolling to the viewpoint (eyes) is large. In this case, the influence of the vehicle's sway (attitude change) becomes large, and the change in the displayed image in the height direction of the vehicle 1 becomes large, which tends to increase the change in the display position of the image. Therefore, in order to effectively suppress the large position change, the correction amount of the image correction is increased.
[0089] On the other hand, when the viewpoint height position is lowered, the influence of the shaking (attitude fluctuation) of the vehicle 1 is reduced, so if the above-mentioned large correction amount is maintained, there is a possibility that over-correction will occur. Therefore, the correction amount of the image correction is reduced according to the decrease in the viewpoint height position.
[0090] In addition, by utilizing the viewpoint height position estimation information, it is possible to identify the viewpoint height position more easily and quickly compared to when the viewpoint height position is captured by a camera and detected by image analysis.
[0091] By carrying out such image correction taking into consideration the viewpoint height position, it is possible to suppress the sense of discomfort felt by the viewer, and also to suppress a decrease in the superimposition of the image on the forward object.
[0092] In the example of FIG. 4(A), the amount of image correction is varied according to the level of the estimated viewpoint height position, and when the estimated viewpoint height position is higher than the standard position, the amount of image correction is increased as the viewpoint height position becomes higher, and when the estimated viewpoint height position is lower than the standard position, the amount of image correction is decreased as the viewpoint height position becomes lower.
[0093] By adaptively controlling the amount of image correction in accordance with the degree of the estimated viewpoint height position, more appropriate image correction (pitching correction or rolling correction) that corresponds to the attitude change of vehicle 1 becomes possible.
[0094] Next, refer to Fig. 4(B) In Fig. 4(B), the horizontal axis represents the number of rotations of the curved mirror (a to w in Fig. 4(A)), and the vertical axis represents the amount of image correction (A to W in Fig. 4(A)).
[0095] As shown in the figure, when the number of rotations is in the range of a to c, the slope of characteristic line Q1, which indicates the rate of change of the correction amount relative to the amount of fluctuation in the viewpoint height position estimation information (here, the number of rotations), is steep, and the rate of change is large. On the other hand, when the number of rotations is in the range of c to w, the slope of characteristic line Q2, which indicates the rate of change of the correction amount relative to the amount of fluctuation in the viewpoint height position estimation information (number of rotations), is gentle, and the rate of change is small.
[0096] In other words, when the amount of image correction is varied depending on the degree of the estimated viewpoint height position, the rate of change of the amount of image correction with respect to the amount of fluctuation in the viewpoint height position estimation information (rotation speed, rotation angle, etc.: viewpoint height position in a broader sense) in a range where the estimated viewpoint height position is higher than the standard position is defined as a first rate of change (rate of change shown by characteristic line Q1), and the rate of change of the amount of image correction with respect to the amount of fluctuation in the viewpoint height position estimation information (rotation speed, rotation angle, etc.: viewpoint height position in a broader sense) in a range where the estimated viewpoint height position is lower than the standard position is defined as a second rate of change (rate of change shown by characteristic line Q2), the first rate of change (rate of change shown by characteristic line Q1) is set to be greater than the second rate of change (rate of change shown by characteristic line Q2).
[0097] In this way, when the amount of image correction is variably controlled according to the viewpoint height position, in the range that is easily affected by vehicle attitude fluctuations (the range where the viewpoint is higher than the standard position), the rate of change in the correction amount relative to the amount of fluctuation in the viewpoint height position estimation information is set large, making it possible to quickly follow large fluctuations in the image position, thereby effectively preventing large changes in the display position of the image.
[0098] On the other hand, when the viewpoint is lower than the standard position, the positional fluctuation of the image is smaller, so the amount of correction is made smaller to prevent over-correction, while small image fluctuations can be reliably suppressed.
[0099] Next, reference will be made to Fig. 5. Fig. 5(A) and (B) are diagrams showing other examples of the correction table. The correction table in Fig. 5 is basically the same as the correction table in Fig. 4(A).
[0100] However, in FIG. 5, when the viewpoint height position is outside a predetermined height range, image correction is stopped (the correction amount is set to zero), which is added, and in this respect it differs from FIG. 4(A).
[0101] In the correction table of FIG. 5(A), data on the amount of correction in the case of rotation numbers x, y, z (rotation angles x', y', z' of the curved mirror) (in other words, when the viewpoint (eye) position is too low with respect to the standard position) is added to the contents of the correction table of FIG. 4(A). The amount of correction in this case is zero. In other words, in FIG. 5(A), the number of rotations x (rotation angle x') is set as the correction stop threshold value Sth2, and when the number of rotations or the rotation angle is equal to or greater than this correction stop threshold value Sth2, the amount of correction is set to zero.
[0102] If the viewpoint (eye) height is too low, the effect of the attitude change of the vehicle 1 is also small, so it is considered that stopping image correction will have little effect. Also, if image correction is forcibly performed to accommodate small image position changes, the minute changes in the image position that accompany the image correction may give the viewer a sense of discomfort. Taking these points into consideration, it has been decided not to perform image correction.
[0103] The correction table in FIG. 5(B) further includes data on the amount of correction in the case of rotation speeds γ, δ (rotation angles γ', δ' of the curved mirror) (in other words, when the viewpoint (eye) position is too high with respect to the standard position). The amount of correction in this case is also zero. In other words, in FIG. 5(B), the rotation speed δ (rotation angle δ') is set to the correction stop threshold value Sth3, and when the rotation speed or rotation angle is equal to or less than this correction stop threshold value Sth3, the amount of correction is set to zero.
[0104] If the viewpoint (eye) height is too high, the display position of the image is greatly affected by the change in the attitude of the vehicle 1, and the change in the display position of the image increases. Therefore, even if image correction is performed, the effect of suppressing the change in the image position may be insufficient, and the image may change unnaturally. Furthermore, if image correction is performed forcibly, the unnatural change in the image position caused by the image correction may give the viewer a sense of discomfort. Taking these points into consideration, it has been decided not to perform image correction.
[0105] Next, reference will be made to Fig. 6. Fig. 6(A) and (B) are diagrams showing still another example of the image correction table, and Fig. 6(C) is a diagram showing an example of a look-up table.
[0106] Fig. 6(A) is a modified example of Fig. 5(A) described above. In Fig. 6(A), variable control of the correction amount of image correction is performed in a range (rotation numbers d to w) where the viewpoint height position is lower than the standard position, and in a range where the viewpoint height position is higher than the standard position (rotation number c), the correction amount is set to a predetermined fixed value (for example, the fixed value is an integer including zero).
[0107] As shown in the upper right of FIG. 6(A), the correction amount corresponding to the rotation speeds a and b is set to α (α is a fixed integer value including zero).
[0108] By using a fixed correction amount α in the range where the viewpoint height position is higher than the standard position, there is no need to significantly switch the correction amount, and it is possible to reduce the burden on the control unit (reference numeral 38 in FIG. 2) (e.g., the processing burden of calculating and obtaining the correction amount).
[0109] In FIG. 6(B), in contrast to FIG. 6(A), variable control of the amount of image correction is performed when the viewpoint height position is in a range higher than the standard position, and when the viewpoint height position is in a range lower than the standard position, the amount of correction is set to a predetermined fixed value (for example, the fixed value is an integer including zero).
[0110] As shown in FIG. 6(B), the correction amount corresponding to the rotation speeds d to w is set to β (β is a fixed integer value including zero).
[0111] By using a fixed correction amount within a range where the viewpoint height position is lower than the standard position, the effort of frequently changing the correction amount in small increments can be eliminated, thereby reducing the burden on the control unit 38 (e.g., the processing burden of calculating and obtaining the correction amount).
[0112] 6(A) and (B) are used as a lookup table (LUT), pitch angle / roll angle information and viewpoint height position estimation information (e.g., rotation speed information) are input as address variables to lookup table 201 as shown in Fig. 6(C), and the correction amount corresponding to the input is output from lookup table 201. By using the LUT, the correction amount can be obtained easily and quickly.
[0113] Second embodiment Next, Fig. 7 will be referred to. Fig. 7(A) is a diagram showing an example of the configuration of the main parts of a display system that estimates the height position of the eyebox using viewpoint height position estimation information and determines (specifies) the viewpoint height position in the eyebox using viewpoint position detection information, Fig. 7(B) is a diagram showing an example of the internal configuration of a correction table, and Fig. 7(C) is a diagram showing an example of a lookup table that employs the configuration of Fig. 7(B). In Fig. 7, parts common to Fig. 3 are given the same reference numerals.
[0114] 7, viewpoint height position estimation information and viewpoint height position detection information (actual measurement information by camera 15) that has a higher resolution of the viewpoint height position are used together. Note that the viewpoint height can be detected in detail by capturing an image of the viewer's viewpoint (eye) by camera 15 and detecting the coordinates, etc., of the captured viewpoint (eye) in the imaging area.
[0115] For example, the height position of the eyebox can be estimated based on viewpoint height position estimation information (information on the number of rotations and rotation angle of the curved mirror), and the viewpoint height position within the eyebox can be determined (for example, the viewpoint height position within the eyebox can be classified as "high," "medium," or "low") using viewpoint height position detection information (information obtained by image processing using an image captured by a camera).
[0116] For example, in the eyebox EB in Fig. 2 described above, the inside of the eyebox EB is divided into nine regions Z1 to Z9. Here, if the viewpoint (eye) 6 is located in the regions Z1 to Z3 (upper regions), it is classified as "high," if it is located in the regions Z4 to Z6 (middle regions), it is classified as "middle," and if it is located in the regions Z7 to Z9 (lower regions), it is classified as "low."
[0117] In other words, the viewpoint height position estimated by the viewpoint height position estimation information can be finely adjusted using the viewpoint height position detection information. This allows the viewpoint height position to be determined (specified) more precisely, improving the detection accuracy of the viewpoint height position. Therefore, more appropriate image correction can be performed.
[0118] In FIG. 7(A), each of the eye boxes EB (EB', EB'') is labeled U (high), M (medium), or D (low), which indicates that the height position of the viewpoint (eyes) can be determined in more detail for each of the eye boxes EB (EB', EB'').
[0119] An example of the correction table (partial excerpt) is shown in Fig. 7(B). In the above embodiment, the correction amount C was determined uniformly when the number of rotations was c (the rotation angle c' of the curved mirror), but in Fig. 7(B), the correction amounts CU, CM, and CD (CU>CM>CD) are determined according to the viewpoint height position (U (high), M (middle), and D (low)) in the eye box EB.
[0120] In the lookup table (LUT) 203 in Fig. 7(C), in addition to the two inputs in Fig. 6(C) described above, viewpoint height position information in the eyebox (viewpoint height position detection information) is added. These three inputs are used as address variables to access the correction table and output the correction amount.
[0121] (Third embodiment) In this embodiment, the processor (display control device) 36 of the HUD device 100 performs viewpoint-tracking spotlighting, and the warping correction control unit (reference numeral 44 in FIG. 3) in the control unit 38 performs viewpoint-tracking warping.
[0122] As explained above using Figure 2, the inside of the eye box EB is divided into, for example, multiple regions Z1 to Z9, and the viewpoint (eye) is imaged by the camera 15 and image processing is performed to detect which region the viewpoint (eye) is currently in.
[0123] Based on the results of the viewpoint position detection, image display light is locally projected onto the area where the viewpoint (eye) is located (which may include its surroundings) to reduce power consumption of the light source unit 24 (viewpoint tracking spot lighting), and warping parameters (parameters that determine the amount of distortion (correction amount) when a distortion with opposite characteristics to the distortion caused by the optical system is applied in advance) are switched for each area Z1 to Z9 (viewpoint tracking warping) to achieve appropriate warping corresponding to the viewpoint position.
[0124] The viewpoint position detection information obtained at this time (information on which of the multiple regions Z1 to Z9 the viewpoint (eye) is located in) is also utilized as viewpoint height position detection information in the previous second embodiment. In other words, the viewpoint position information obtained by imaging with camera 15 is used in common for viewpoint tracking lighting, viewpoint tracking warping, and pitching correction / rolling correction taking into account the viewpoint height position.
[0125] This eliminates the need to acquire viewpoint height position detection information separately for pitching correction or rolling correction as in the second embodiment, and reduces the burden on the control unit 38 and the like.
[0126] (Fourth embodiment) Next, reference is made to Fig. 8. Fig. 8 is a diagram showing another example of a HUD device (an example in which the curved mirror cannot be rotated). In Fig. 8, the same reference numerals are used for the parts common to Fig. 3 (however, the curved mirror in Fig. 8 is given a different reference numeral because its function is different from that in Fig. 3).
[0127] In the example of Figure 8, the HUD device 100 comprises a display unit 22 with an image display surface, and an optical system 33 including a non-rotatable curved mirror 29 that projects an image onto the projection target member, the windshield 2, and uses image display position information (P1, P2, etc.) on the image display surface of the display unit 22 as viewpoint height position estimation information.
[0128] In the above embodiment, the position information (rotation information) of the curved mirror (concave mirror, etc.) was used as viewpoint height position estimation information, but if the curved mirror cannot be rotated, the position information (rotation information) of the curved mirror cannot be used.
[0129] Therefore, in the example of FIG. 8, image display position information (P1, P2, etc.) on the image display surface of the display unit 22 is used as viewpoint height position estimation information (however, this is just one example and is not limited to this. Information such as the rotation angle of other mirrors such as an auxiliary mirror may also be used).
[0130] In the case where the curved mirror cannot be rotated, the incident position of the image display light on the reflective surface of the curved mirror that reflects light can be adjusted to control the reflection direction of the light from the reflective surface and change the height position of the eye box.
[0131] The change in the incident position of the display light on the reflective surface is ultimately achieved by adjusting the image display position on the image display surface of the display unit 22 that displays the image.
[0132] In Fig. 8, an up position (UP) and a down position (DN) are shown as the height position of the viewpoint (eyes). The image display position P1 corresponds to the down position (DN), and P2 corresponds to the up position (UP). Therefore, image display position information (for example, coordinate information P1, P2, etc. of the displayed image) can be used as objective data for estimating the height position of the viewer's viewpoint. In the example of Fig. 8, image display position information is supplied to the input I / F 32 as viewpoint height estimation information G1.
[0133] In this way, it is possible to obtain viewpoint height estimation information even when the position of the curved mirror is fixed. By utilizing this information, it is possible to specify the viewpoint height more easily and quickly than when the viewpoint height is detected by capturing an image of the viewpoint with a camera and analyzing the image.
[0134] Fifth embodiment In this embodiment, the control unit (reference numeral 36 in FIGS. 2 and 3) distinguishes between pitching correction and rolling correction when setting the amount of image correction, and sets an appropriate amount of correction for each correction.
[0135] For example, the correction table (LUT or the like) as shown above may be prepared separately for pitching correction and rolling correction, which makes it possible to perform appropriate image correction for each of pitching and rolling.
[0136] Sixth embodiment Next, reference will be made to Fig. 9. Fig. 9 is a flowchart showing an example of the procedure of image correction processing.
[0137] In step S1, vehicle behavior information (attitude information) is acquired. In step S2, information on the attitude angle of the vehicle (pitch angle θp, roll angle θr, etc.) is acquired. In step S3, the position (rotational position) of the curved mirror (concave mirror, etc.) is detected.
[0138] In step S4, it is detected whether the rotation angle of the curved mirror is outside the correction stop range. If it is Y, in step S5, the viewpoint height position in the eye box is detected as necessary, and then in step S6, the correction amount in the image correction is obtained. At this time, a lookup table may be referenced, or the amount may be calculated by a correction calculation.
[0139] Furthermore, in step S6, if it is estimated that the viewer's viewpoint height position is higher than the standard position, the amount of image correction is made larger than the amount of correction at the standard position, and if it is estimated that the viewer's viewpoint height position is lower than the standard position, the amount of image correction is made smaller than the amount of correction at the standard position, thereby variably controlling the amount of image correction.
[0140] In step S7, image correction is performed using the acquired correction amount, and in step S8, image drawing processing is performed based on the image data (image content) after the image correction.
[0141] If the answer is N in step S4, the process proceeds to step S9 where the correction amount is set to zero (in other words, pitching correction and rolling correction are not performed), and then the process proceeds to step S8.
[0142] In addition, in the above step S2, the viewpoint is assumed to be in a standard position, and the correction amount for the standard position is provisionally set, and in step S6, it is determined whether or not the provisionally set correction amount can be used as is, and if not, the correction amount may be reacquired (reset). Such modifications may be made as appropriate.
[0143] As described above, according to the present invention, it is possible to realize a high-performance display device capable of displaying an image with little positional fluctuation even when pitching or rolling of a vehicle occurs. The display device is broadly interpreted and can be applied to, for example, a head-up display (HUD) device and other display devices. As the display device, a liquid crystal display device, a micro LED device, and various other display devices can be used.
[0144] In addition, the display control device can be a display control ECU mounted on the HUD device, but is not limited to this, and it is also possible to use an ECU mounted on something other than the HUD device, such as a vehicle ECU.
[0145] In this specification, the term "vehicle" may be broadly interpreted as a vehicle. Terms related to navigation (e.g., signs, etc.) are also broadly interpreted, taking into consideration the perspective of navigation information in a broad sense that is useful for driving a vehicle. HUD devices also include those used as simulators (e.g., aircraft simulators, simulators as game devices, etc.).
[0146] The present invention is not limited to the above-described exemplary embodiments, and those skilled in the art could easily modify the above-described exemplary embodiments to the extent that they fall within the scope of the claims. [Explanation of symbols]
[0147] 1 Vehicle (own vehicle), 2 Projected member (reflective translucent member, windshield, etc.), 6 Viewpoint (eye) of viewer, 15 Camera, 28 Curved mirror (concave mirror, etc.), 30 Curved mirror rotation mechanism, 31 Curved mirror rotation control unit, 32...input interface (I / F), 34...viewpoint position detection unit, 36...processor (display control device), 38...display unit, 40...vehicle attitude information acquisition unit, 41...viewpoint position information acquisition unit, 42...pitching / rolling correction control unit, 43...correction control unit, 44...warping correction control unit, 50...memory (storage device, storage unit), 52...correction table, 54...pitching correction table, 56...rolling correction table, 58...warping correction table, 80...image processing unit, 86...correction unit (correction processing unit)
Claims
1. A display control device that is mounted on a vehicle and controls image display on a display device that allows a viewer riding on the vehicle to view an image, A vehicle attitude information acquisition unit that acquires attitude information of the vehicle; a viewpoint position information acquisition unit that acquires viewpoint height position estimation information that can estimate a viewpoint height position of the viewer; a control unit that controls image correction including at least one of pitching correction that corrects the image so as to suppress a fluctuation in a display position of the image due to pitching of the vehicle and rolling correction that corrects the image so as to suppress a fluctuation in a display position of the image due to rolling of the vehicle, The control unit is when the viewpoint height position is estimated to be higher than a standard position as a result of estimation based on the viewpoint height position estimation information acquired by the viewpoint position information acquisition unit, a correction amount of the image correction is made larger than the correction amount at the standard position, and when the viewpoint height position is estimated to be lower, a correction amount of the image correction is made smaller than the correction amount at the standard position; The control unit is The amount of image correction is varied according to the degree of the estimated viewpoint height position, If the estimated viewpoint height position is higher than the standard position, the amount of image correction is increased as the viewpoint height position becomes higher; If the estimated viewpoint height position is lower than the standard position, the amount of image correction is decreased as the viewpoint height position becomes lower; The control unit is When the amount of image correction is varied according to the degree of the estimated viewpoint height position, the image correction is stopped when the viewpoint height position is outside a predetermined height range. Display control device.
2. A display control device that is mounted on a vehicle and controls image display on a display device that allows a viewer riding on the vehicle to view an image, A vehicle attitude information acquisition unit that acquires attitude information of the vehicle; a viewpoint position information acquisition unit that acquires viewpoint height position estimation information that can estimate a viewpoint height position of the viewer; a control unit that controls image correction including at least one of pitching correction that corrects the image so as to suppress a fluctuation in a display position of the image due to pitching of the vehicle and rolling correction that corrects the image so as to suppress a fluctuation in a display position of the image due to rolling of the vehicle, The control unit is when the viewpoint height position is estimated to be higher than a standard position as a result of estimation based on the viewpoint height position estimation information acquired by the viewpoint position information acquisition unit, a correction amount of the image correction is made larger than the correction amount at the standard position, and when the viewpoint height position is estimated to be lower, a correction amount of the image correction is made smaller than the correction amount at the standard position; The control unit is When the estimated viewpoint height position is in a range higher than the standard position, a correction amount of a predetermined fixed value is set as a correction amount of the image correction; When the estimated viewpoint height position is in a range lower than the standard position, the amount of correction of the image correction is varied according to the degree of the viewpoint height position, and the amount of correction of the image correction is decreased as the viewpoint height position becomes lower. Display control device.
3. A display control device that is mounted on a vehicle and controls image display on a display device that allows a viewer riding on the vehicle to view an image, A vehicle attitude information acquisition unit that acquires attitude information of the vehicle; a viewpoint position information acquisition unit that acquires viewpoint height position estimation information that can estimate a viewpoint height position of the viewer; a control unit that controls image correction including at least one of pitching correction that corrects the image so as to suppress a fluctuation in a display position of the image due to pitching of the vehicle and rolling correction that corrects the image so as to suppress a fluctuation in a display position of the image due to rolling of the vehicle, The control unit is when the viewpoint height position is estimated to be higher than a standard position as a result of estimation based on the viewpoint height position estimation information acquired by the viewpoint position information acquisition unit, a correction amount of the image correction is made larger than the correction amount at the standard position, and when the viewpoint height position is estimated to be lower, a correction amount of the image correction is made smaller than the correction amount at the standard position; The control unit is When the estimated viewpoint height position is in a range lower than the standard position, a correction amount of a predetermined fixed value is set as the correction amount of the image correction; When the estimated viewpoint height position is in a range higher than the standard position, the amount of correction of the image correction is varied according to the degree of the viewpoint height position, and the higher the viewpoint height position, the greater the amount of correction of the image correction. Display control device.
4. The viewpoint height position estimation information is When the display device is equipped with an optical system including a rotatable curved mirror that projects an image onto a projection target, the position information of the curved mirror is Or, When the display device includes a display unit having an image display surface, and an optical system including a non-rotatable curved mirror that projects an image onto a projection target, the display unit is provided with image display position information on the image display surface of the display unit. The display control device according to any one of claims 1 to 3.
5. The viewpoint position information acquisition unit is In addition to the viewpoint height position estimation information that can estimate the viewpoint height position of the viewer, Obtaining viewpoint height position detection information that is detected based on an image of the viewer captured by a camera and has a higher viewpoint height position resolution than the viewpoint height position estimation information; The control unit is fine-adjusting the viewpoint height position estimated based on the viewpoint height position estimation information by using the viewpoint height position detection information; The display control device according to any one of claims 1 to 4.
6. The control unit is A viewpoint tracking warping is performed for each of a plurality of divided regions in the eyebox, and viewpoint position information obtained during the warping, which indicates which of the plurality of regions the viewer's viewpoint is located in, is also used as the viewpoint height position detection information. The display control device according to claim 5 .
7. The control unit is When setting the amount of image correction, The pitching correction and the rolling correction are handled separately, and a correction amount appropriate for each correction is set. The display control device according to any one of claims 1 to 6.
8. A display control device according to any one of claims 1 to 7, a correction unit that performs image correction including at least one of the pitching correction and the rolling correction; a display unit for displaying the corrected image; A display device having the above configuration.
9. The display device includes: The head-up display device further includes an optical system that projects a display light of an image displayed on the display unit onto a projection member provided in the vehicle, and has a function of allowing at least one of the driver and the passenger, who are the viewers, to visually recognize a virtual image. The display device according to claim 8.
10. An image display control method for a display device mounted on a vehicle, comprising: acquiring attitude information of the vehicle, the attitude information including at least one of pitching and rolling; acquiring viewpoint height position estimation information capable of estimating a viewpoint height position of a viewer of an image displayed by the display device; a step of variably controlling an amount of image correction for suppressing a fluctuation in a display position of an image displayed by the display device due to a fluctuation in an attitude of the vehicle, by making an amount of image correction larger than an amount of image correction at the standard position when it is estimated that the viewpoint height position of the viewer is higher than a standard position, and making an amount of image correction smaller than an amount of image correction at the standard position when it is estimated that the viewpoint height position is lower than the standard position; correcting an image displayed by the display device in accordance with the variably controlled correction amount; Including, varying an amount of correction of the image correction according to the degree of the viewpoint height position, and when the estimated viewpoint height position is higher than the standard position, increasing the amount of correction of the image correction as the viewpoint height position becomes higher, and when the viewpoint height position is lower than the standard position, decreasing the amount of correction of the image correction as the viewpoint height position becomes lower; When the amount of image correction is varied according to the degree of the viewpoint height position, stopping the image correction when the viewpoint height position is outside a predetermined height range; The image display control method further includes:
11. An image display control method for a display device mounted on a vehicle, comprising: acquiring attitude information of the vehicle, the attitude information including at least one of pitching and rolling; acquiring viewpoint height position estimation information capable of estimating a viewpoint height position of a viewer of an image displayed by the display device; a step of variably controlling an amount of image correction for suppressing a fluctuation in a display position of an image displayed by the display device due to a fluctuation in an attitude of the vehicle, by making an amount of image correction larger than an amount of image correction at the standard position when it is estimated that the viewpoint height position of the viewer is higher than a standard position, and making an amount of image correction smaller than an amount of image correction at the standard position when it is estimated that the viewpoint height position is lower than the standard position; correcting an image displayed by the display device in accordance with the variably controlled correction amount; Including, a step of setting a predetermined fixed value as the amount of correction for the image correction when the viewpoint height position is in a range higher than the standard position, and varying the amount of correction for the image correction according to the level of the viewpoint height position when the viewpoint height position is in a range lower than the standard position, and decreasing the amount of correction for the image correction as the viewpoint height position becomes lower; The image display control method further includes:
12. An image display control method for a display device mounted on a vehicle, comprising: acquiring attitude information of the vehicle, the attitude information including at least one of pitching and rolling; acquiring viewpoint height position estimation information capable of estimating a viewpoint height position of a viewer of an image displayed by the display device; a step of variably controlling an amount of image correction for suppressing a fluctuation in a display position of an image displayed by the display device due to a fluctuation in an attitude of the vehicle, by making an amount of image correction larger than an amount of image correction at the standard position when it is estimated that the viewpoint height position of the viewer is higher than a standard position, and making an amount of image correction smaller than an amount of image correction at the standard position when it is estimated that the viewpoint height position is lower than the standard position; correcting an image displayed by the display device in accordance with the variably controlled correction amount; Including, a step of setting a predetermined fixed value of the correction amount of the image correction when the viewpoint height position is in a range lower than the standard position, and varying the correction amount of the image correction according to the degree of the viewpoint height position when the viewpoint height position is in a range higher than the standard position, and increasing the correction amount of the image correction as the viewpoint height position becomes higher; The image display control method further includes:
Citation Information
Patent Citations
Denshikairo
JP1976061760A
Display unit for vehicle
JP1998203199A
Information display device
JP2010256878A
Head-up display device
JP2015060180A
Vehicle information projection system and projection device
JP2015087619A
