Display control unit, display device, and display control method

Abstract

To perform shake correction by a correction amount suited to an image to display and to perform display control capable of suppressing discomfort a viewer experiences when displaying a plurality of images different in correction amount in shake correction suppressing shake of images caused by vibration of a self vehicle.SOLUTION: A control part 81 of a display control unit 80 has: a vibration information acquisition part 93; and a shake correction part 87 which performs shake correction to suppress shake of images caused by vibration of a vehicle 1 and can adjust a correction amount in shake correction. The shake correction part 87 performs shake correction in a correction amount of single display including correction amount zero suited to the image when the image is displayed on its own, discriminates a plurality of images from an image whose correction amount is maximum and the other images when those images are displayed, and performs correction amount change processing to change correction amounts of the other images so that they approximate the maximum correction amount.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a display control device, a display device, a display control method, and the like, which are mounted on a vehicle such as an automobile. [Background technology]

[0002] Patent Document 1 describes that the display control unit executes a change process to change the content of the shake correction that corrects the shake of the virtual image according to at least one of the projection distance and the depression angle, specifically, executes the shake correction when the projection distance is equal to or greater than a predetermined distance, and does not execute the shake correction when the projection distance is shorter than the predetermined distance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 7042443 (

[0006] etc.). Summary of the Invention [Problem to be solved by the invention]

[0004] When pitching or the like occurs in the vehicle, the shaking of the image can be effectively suppressed by setting an appropriate correction amount and performing shake correction according to the image being displayed, thereby realizing a display that does not feel unnatural. Here, it is assumed that there are a plurality of images being displayed, and the correction amount in the shake correction for each image is different. In this case, if pitching occurs in the vehicle and each image shakes, the amount of shake correction for each image is different, so the amplitude of image shake after correction will differ for each image. In other words, each of the multiple images within the viewer's field of vision will fluctuate (waver) with a different amplitude, which may cause the viewer to feel uncomfortable (for example, to sense flickering). The above-mentioned Patent Document 1 does not mention this point, nor does it mention any measures to address it. The above problems have been identified by the present inventors.

[0005] One object of the present invention is to realize display control that performs shake correction using a correction amount appropriate for the image to be displayed in a shake correction that suppresses image shaking caused by vibration of the vehicle, and that can suppress the discomfort felt by the viewer when multiple images with different correction amounts are displayed.

[0006] Other objects of the present invention will become apparent to those skilled in the art by referring to the following exemplary aspects and best modes, as well as the accompanying drawings. [Means for solving the problem]

[0007] In order to facilitate an understanding of the outline of the present invention, embodiments according to the present invention will be exemplified below.

[0008] In a first aspect, a display control device is a display control device that is mounted on a vehicle and has a control unit that controls image display of a display device that allows an image to be viewed by a viewer who is a passenger in the vehicle, and the control unit has a vibration information acquisition unit that acquires vibration information related to vibration of the vehicle while traveling, and a vibration correction unit that performs vibration correction so as to suppress shaking of the image caused by the vibration and is capable of adjusting the correction amount in the vibration correction, and when there is one image to be displayed, the vibration correction unit performs vibration correction with a correction amount for single display that is suitable for the image, including a correction amount of zero, and when there are multiple images to be displayed, the vibration correction unit distinguishes the images into a maximum correction amount image, which has the maximum correction amount in the vibration correction, and other images, which have a correction amount smaller than the maximum correction amount, and performs a correction amount change process that changes the current correction amount in the vibration correction of the other images to approach the maximum correction amount or to be the same as the maximum correction amount.

[0009] In the first mode, when an image is displayed alone, shake correction is performed with an appropriate correction amount (correction amount for single display) according to, for example, the depression angle of the image, the display distance (projection distance), or the type of image, etc. In other words, it is possible to set an individual correction amount by placing importance on the characteristics (attributes) of each image. On the other hand, when multiple images are displayed, a correction change process is performed to reduce (or set to zero) the difference in correction amount between each image, and a cooperative correction amount is set to suppress the variation in the correction amount. In other words, when multiple images are displayed and an image with the maximum correction amount exists, the current correction amount (specifically, if the correction amount change process has not been performed in the past, it is the correction amount for a single display, and if the correction amount change process has been performed in the past, it is the correction amount for each image after the correction amount change process) for the other images (images with a correction amount smaller than the maximum correction amount) is made closer to the maximum correction amount or made the same (equal), thereby reducing the difference in the correction amount between each image. This makes it possible to suppress discomfort (such as a flickering sensation) felt by the viewer.

[0010] In addition, in the correction amount change process, the maximum correction amount is set as the target value (reference value) because by using the image with the least shaking as the reference, the shaking of the other images can be reduced more than at present, thereby improving the effect of suppressing the sense of discomfort.

[0011] In addition, in the correction amount change process, when an image with the maximum correction amount is newly added while one image is being displayed, the "current correction amount for shake correction of other images" is the "correction amount for the single display of the one image (which becomes the other image)."

[0012] On the other hand, if a correction change process was recently performed on multiple images and the correction amount was changed, and then an image with the maximum correction amount was newly added, making the latest correction amount change process necessary, the correction amount (the correction amount changed by the most recent correction change process) of the other images (each of the multiple images that were displayed before the new image was added) before the latest correction amount change process is used.

[0013] According to this aspect, in shake correction that suppresses image shaking caused by vibration of the vehicle, shake correction is performed using a correction amount appropriate for the image to be displayed, and a display control device can be realized that can suppress the discomfort felt by the viewer when multiple images with different correction amounts are displayed.

[0014] In a second aspect that is dependent on the first aspect, the control unit may set the amount of correction for the individual display larger for an image with a large depression angle as seen by the viewer than for an image with a small depression angle, or may set the amount of correction for the individual display larger for an image with a large display distance, which is the distance from the viewer's viewpoint or a predetermined reference point to the displayed image, than for a small image, or may set the amount of correction for the individual display larger for a type of image for which the viewer has high sensitivity to changes in the position of the displayed image than for a type of image for which the viewer has low sensitivity.

[0015] In the second aspect, an appropriate correction amount (correction amount for single display) is set according to the depression angle of the image, the display distance (projection distance), the type of image, or the like.

[0016] Since an image with a small depression angle as seen by a viewer may be perceived as being farther away than an image with a large depression angle, it is conceivable that when an image sways up and down, the viewer may perceive a larger change in position as occurring. Therefore, it is preferable to increase the amount of correction for an image with a small depression angle compared to an image with a large depression angle, thereby reducing the amplitude of the image shaking. Furthermore, since an image with a large display distance is perceived by the viewer as being located farther away, as in the case of the depression angle described above, it is preferable to increase the amount of correction compared to an image with a small display distance, strengthen the vibration suppression force, and reduce the vibration amplitude of the image. Furthermore, the viewer's sensitivity to positional variations in an image is not uniform, but varies depending on the type of displayed image (display content). For example, images of navigation arrows and signs (turn-by-turn display) are often drawn on the road surface during actual driving. A viewer such as a driver remembers this from past driving experiences. Also, a viewer such as a driver has experienced that if the appearance of the arrow drawn on the road surface changes slightly, the distance in the depth direction changes significantly. Therefore, when an image such as an arrow is displayed on a HUD device or the like, a viewer such as a driver, based on past experiences as described above, tends to perceive the arrow as if it were superimposed on the road surface at a greater distance, and also to perceive the change in appearance as if the distance in the depth direction has changed significantly. In other words, the viewer's sensitivity to positional changes in the image (the viewer's perception sensitivity) is quite high. On the other hand, vehicle speed displays (instrument displays showing the vehicle status) are usually seen in a fixed position near the driver, and the viewer's sensitivity to positional changes in the image (the viewer's perception sensitivity) is not so high. In this way, the viewer's sensitivity differs depending on the type of image. Therefore, for types of images to which the viewer has a high sensitivity (such as turn-by-turn displays), it is preferable to increase the correction amount in the shake correction and suppress the amplitude of the shake of the image compared to the display of instruments showing the vehicle status.

[0017] In this way, according to this embodiment, it is possible to individually set an appropriate correction amount for shake correction, taking into consideration the characteristics (attributes) of an image.

[0018] In a third aspect dependent on the first or second aspect, when the correction amount change process has been performed in the past and an image to be displayed is newly added, the control unit may perform a new correction amount change process, if there is no change to the image with the largest correction amount, perform the new correction amount change process on the newly added image, and if there is a change to the image with the largest correction amount, perform the new correction amount change process on each of the images other than the newly added image.

[0019] In the third aspect, a case is assumed in which a new image is added under a condition in which the correction amount change process has been performed in the past. Here, if the correction amount of the newly added image is smaller than the maximum correction amount of the currently displayed image, the maximum correction amount is not changed. Since the correction amount change process has already been performed on the currently displayed image, a new correction change process needs to be performed only on the newly added image.

[0020] On the other hand, if the newly added image is the image with the maximum correction amount, the maximum correction amount is changed. In this case, the correction amount (current correction amount) of each of the multiple images currently displayed is changed so that it approaches or becomes the same as the maximum correction amount for the newly added image.

[0021] According to this aspect, even if a new image is added and a new correction amount change process is required after a correction amount change process has been performed in the recent past, it is possible to respond appropriately to the situation, thereby achieving image display that is less unnatural.

[0022] In a fourth aspect dependent on any one of the first to third aspects, when a new image having a maximum correction amount is additionally displayed in a first display state in which one or more images are displayed, and the one or more images become equivalent to the other images, the correction amount change process is performed on the one or more images to result in a second display state, and then the new image is erased, and accordingly, a first timing arrives at which the correction amount after the correction amount change process for the one or more images can be returned to the correction amount in the first display state, the control unit may perform a correction amount return process at a second timing at which a predetermined time has elapsed from the first timing to return the correction amount to the correction amount in the first display state, or may perform a correction amount return process to return the correction amount in the first display state, provided that the vehicle speed of the vehicle is less than a predetermined threshold, when the condition is satisfied.

[0023] In the fourth aspect, it is assumed that after the correction change process is performed, the image with the largest correction amount that caused the correction change process disappears, and the correction amounts for the other images are returned to the correction amounts in the state before the correction change process (first display state).

[0024] In this case, the timing at which the image with the maximum correction amount that caused the correction change process disappears may be the first timing at which the correction amount can be returned. However, if the correction amounts for the other images are returned to their original amounts at the same time that the image with the maximum correction amount disappears, the viewer will simultaneously perceive two visual changes: that "the image disappears" and that "the amplitude of the shaking of the images that have not disappeared (the other images) has changed," which may increase the sense of discomfort. Therefore, in this embodiment, when a first timing arrives at which the correction amount can be returned to the correction amount in the first display state, after a predetermined time has elapsed, a correction amount return process is performed to return the current correction amount to the correction amount in the first display state.

[0025] Furthermore, even if the first timing arrives at which the correction amount can be returned, if the vehicle speed is equal to or greater than a predetermined speed, a viewer such as a driver needs to pay attention to the front and surroundings in order to drive safely. In such a situation, if the correction amount is returned to its original state, a change will occur in the shaking of the image being displayed, which may be noticeable to the viewer, which is undesirable from a safety standpoint. Therefore, in this embodiment, when the first timing arrives at which the correction amount can be returned, the condition is that the vehicle speed is less than a predetermined threshold, and if the condition is satisfied, the correction amount return process is performed. This makes it possible to return the current correction amount to the original correction amount comfortably or safely at an appropriate timing or under appropriate driving conditions.

[0026] In a fifth aspect, a display device includes a display unit that displays an image, and a display control device according to any one of the first to fourth aspects.

[0027] According to the fifth aspect, a projection type display device with high functionality and improved user-friendliness can be realized.

[0028] In a sixth aspect dependent on the fifth aspect, the display device may be a projection type display device that displays an image displayed on the display unit as a virtual image on a virtual image display surface by projecting display light of the image displayed on the display unit onto a projection target.

[0029] According to the sixth aspect, a projection type display device with high functionality and improved user-friendliness can be realized. Typically, a head-up display (HUD) device or a windshield display (WSD) device can be used as the projection type display device. The term "projection type display device" is to be interpreted in a broad sense. The projection target may be a windshield of a vehicle. The projection target may also be the glasses part of a glasses-type head-mounted display. In this case, the head-mounted display may be understood as a type of head-up display in the broad sense. Therefore, it can be included in the present invention.

[0030] In a seventh aspect, a display control method is a display control method for controlling an image display of a display device mounted on a vehicle and displaying an image to be viewed by a viewer who is a passenger of the vehicle, the display control method including a first step of acquiring vibration information related to vibration of the vehicle while traveling, and performing shake correction to suppress shaking of the image caused by the vibration, where if there is only one image to be displayed, shake correction is performed with a correction amount for single display including a correction amount of zero that is suitable for that image, and if there are multiple images to be displayed, a correction amount change process is performed to distinguish those images into a maximum correction amount image having the maximum correction amount in shake correction and other images having a correction amount smaller than the maximum correction amount, and to change the correction amount in shake correction of the other images to approach the maximum correction amount or to be the same as the maximum correction amount.

[0031] According to the seventh aspect, in shake correction that suppresses image shaking caused by vibration of the vehicle, shake correction is performed with a correction amount appropriate for the image to be displayed, and a display control method can be realized that can suppress the discomfort felt by the viewer when multiple images with different correction amounts are displayed.

[0032] Those skilled in the art will easily understand that the exemplified embodiments according to the present invention can be further modified without departing from the spirit of the present invention. [Brief description of the drawings]

[0033] [Figure 1] FIG. 1 is a diagram showing an example of the configuration of a head-up display (HUD) device capable of performing correction amount changing processing and correction amount restoration processing to which the present invention is applied. [Diagram 2] 2A and 2B are diagrams showing an example of fluctuation in the display position of an image in the up and down direction (image shaking) caused by pitching. [Diagram 3] FIG. 3 is a diagram illustrating an example of shake correction. [Figure 4] FIG. 4 is a diagram showing an example of a case where an appropriate correction amount is set based on the depression angle as seen by the viewer, the display distance, the type of image, or the like. [Diagram 5] 5(A) to 5(C) are diagrams showing an example of setting an appropriate correction amount for each image, and an example of correction amount change processing. [Figure 6] FIG. 6 is a diagram showing another example of setting an appropriate correction amount for each image, and another example of the correction amount change process. [Figure 7] FIG. 7 is a diagram showing yet another example of setting an appropriate correction amount for each image, and yet another example of the correction amount change process. [Figure 8] FIG. 8 is a timing chart showing an example of the correction amount changing process and the correction amount restoring process (corresponding to FIG. 5). [Figure 9] FIG. 9 is a timing chart showing another example of the correction amount changing process and the correction amount restoring process (corresponding to FIG. 6). [Figure 10]FIG. 10 is a timing chart showing still another example of the correction amount changing process and the correction amount restoring process (corresponding to FIG. 7). [Figure 11] FIG. 11 is a flowchart showing an example of a main procedure of display control. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0034] The best mode described below is used to easily understand the present invention. Therefore, those skilled in the art should be aware that the present invention is not unduly limited by the mode described below.

[0035] (First embodiment) Please refer to Fig. 1. It is a diagram showing an example of the configuration of a head-up display (HUD) device capable of performing correction amount changing processing and correction amount restoration processing to which the present invention is applied.

[0036] The height direction of the vehicle 1 is defined as the up-down direction (Y direction), the width direction of the vehicle 1 is defined as the left-right direction (X direction), and the front-rear direction is defined as the Z direction.

[0037] As shown at A-1 in FIG. 1, a vehicle 1 is traveling ahead on a straight road at a speed of 60 km / h.

[0038] Moreover, a projection type display device 111 is installed inside the dashboard 41. In the example of Fig. 1, the projection type display device 111 is a head-up display (HUD) device 100. It should be noted that a WSD (windshield display) device may be used in place of the HUD device 100 or together with the HUD device 100. In a broad sense, the WSD device is a type of HUD device. However, while an HUD device projects (projects) display light of an image onto a projection target (e.g., windshield 2 in Figure 1) via an optical system, a WSD device projects (projects) display light of an image emitted from a display unit (such as a liquid crystal display device) directly onto a projection target without going through an optical system, and in this respect the configurations are different.

[0039] In FIG. 1, the HUD device 100 displays, in the foreground as viewed by a viewer (here assumed to be the driver) 5 who is an occupant of the vehicle 1, an image (virtual image) AR1 of a display showing the vehicle status (specifically, a vehicle speed display SP of "60 km / h") as a non-superimposed image that is not superimposed on an object, and also displays, in the background, an image (virtual image) AR2 of a turn-by-turn display (specifically, a navigation display NVa consisting of the letters "Tokyo" and a "navigation arrow graphic" superimposed on the road surface 9) as a superimposed image so as to be superimposed on the road surface 9 which is the object.

[0040] In the example of FIG. 1, the vehicle 1 is a right-hand drive vehicle in which the steering wheel 4 is located on the right side. The user of the projection-type display device 111 equipped with the HUD device 100 is a driver 5 who drives the vehicle 1. In a broad sense, the "driver" can also be called a "passenger" who is in the vehicle, or a "viewer" who visually recognizes an image.

[0041] As shown in A-2 of FIG. 1, the display system 115 includes various sensors (such as an acceleration sensor) 75, an ECU (Electronic Control Unit) 77 capable of collecting various types of vehicle information, and the HUD device 100.

[0042] The HUD device 100 includes a display control device (processor) 80 and a projection unit 95. The display control device (processor) 80 also includes a control unit 81. The control unit 81 has a display content determination unit 83 that determines the image to be displayed, a display distance setting unit 84 that sets the display distance (projection distance), a display position determination unit 85 that determines the display position (arrangement) on the display surface (virtual image display surface) (which may also be equipped with a depression angle calculation unit 86 that calculates the depression angle of the image as seen by the viewer), a shake correction unit 87, a display state change unit 92 that changes the display state, and a vibration information acquisition unit 93 that acquires vibration information of the vehicle, such as pitching.

[0043] The shake correction unit 87 has a correction amount adjustment unit 88, which has a correction amount setting unit 89 that sets the correction amount for the shake correction processing, a correction amount change unit 90 that performs correction amount change processing, and a correction amount return processing unit 91 that performs correction amount return processing.

[0044] Next, reference is made to Fig. 2. Fig. 2(A) and (B) are diagrams showing an example of fluctuation in the display position of an image in the up and down direction (image shaking) caused by pitching.

[0045] 2A, the vehicle 1 is traveling on a flat road surface, but due to sudden acceleration, for example, the front end of the body of the vehicle 1 is lifted off the road surface, and the body of the vehicle 1 is in a backward tilted posture. The pitching angle is θa. 2B, for example, due to sudden braking, the rear end of the body of the vehicle 1 is lifted off the road surface, and the body of the vehicle 1 is in a forward tilted position. The pitch angle is θb. When pitching occurs, a misalignment (mismatch) occurs between the XYZ Cartesian coordinate system (first coordinate system) set in real space with the road surface as the reference and the local X1Y1Z1 Cartesian coordinate system (second coordinate system) that fluctuates depending on the posture of the body of the vehicle 1. This misalignment causes a fluctuation (shaking) in the display position of the image. Shake correction (pitching correction here) is an image correction that suppresses this fluctuation (basically a coordinate conversion correction that converts the second coordinate system into the first coordinate system).

[0046] As shown by the solid lines in the upper part of Figures 2(A) and (B), before pitching occurs, a warning mark (virtual image) 70 is displayed superimposed on the object ahead (e.g., the vehicle ahead) 60, but when pitching occurs, as shown by the dashed lines, the position of the warning mark (virtual image) moves in the vertical direction (the height direction (Y direction) of the vehicle 1 in the XYZ Cartesian coordinate system).

[0047] In Fig. 2(A), the amount of displacement is indicated as ΔD1. In Fig. 2(B), the amount of displacement is indicated as ΔD2. In addition, in the figures, the reference symbol 70' is attached to the attention drawing mark (virtual image) after the positional movement has occurred. The shake correction unit 87 previously shown in FIG. 1 has a function of correcting such positional deviation (shake) of an image (virtual image). In practice, the display position of the image on the display unit (liquid crystal panel, screen, etc.) 96 of the HUD device 100 is changed, or a concave mirror, which is a component of the display optical system 98, is rotated to change the emission direction of the display light for the image, thereby performing shake correction.

[0048] Next, shake correction (shake correction processing) will be described with reference to Fig. 3. Fig. 3 is a diagram showing an example of shake correction.

[0049] Between times t0 and t1, the vehicle 1 is in a traveling state. In A-1 of Fig. 3, in response to pitching of the vehicle 1, an angular velocity detection signal output from an angular velocity sensor (one of various sensors indicated by reference numeral 75 in Fig. 1) provided on the vehicle 1 has a vibration waveform.

[0050] As shown in A-2 of FIG. 3, the attitude change detection signal output from a sensor (one of the various sensors indicated by reference numeral 75 in FIG. 1) that detects the attitude change of the vehicle 1 also has a vibration waveform. Here, as shown in A-3 of FIG. 3, pitching correction is performed to generate a display correction value. This display correction value can be obtained by inverting the polarity of the vibration waveform of A-2 in FIG.

[0051] As shown in A-4 of FIG. 3, by performing shake correction (pitching correction) using a display correction value, the deviation of the display position of an image (virtual image) can be sufficiently suppressed. Therefore, as shown in A-5 of FIG. 3, the display position of the non-overlapping image AR1 is maintained at approximately the same position even after pitching occurs.

[0052] However, the above example is merely an example. It is also possible to adjust the amount of correction in shake correction by slightly suppressing the pitching correction function in the shake correction unit 87.

[0053] In the previous explanation, the transition was from A-2 in FIG. 3 to A-3 in FIG. 3, but here, the transition is assumed to be from A-2 in FIG. 3 to A-6 in FIG.

[0054] In A-6 of Fig. 3, the amplitude of the display correction value is approximately half that of A-3 of Fig. 3, and therefore the shake correction function (pitching correction function) is suppressed as shown in A-7 of Fig. 3. Therefore, as shown in A-8 of Fig. 3, the amplitude of the shake of the non-superimposed image AR1 is suppressed compared to A-5 of Fig. 3.

[0055] Next, let us consider the case of moving from 3-2 in Fig. 3 to A-9 in Fig. 3. In A-9 and A-10 in Fig. 3, the shake correction function (pitching correction function) is further suppressed. Therefore, as shown in A-11 in Fig. 3, the amplitude of the shake of the non-superimposed image AR1 increases compared to A-8 in Fig. 3. In this way, the amount of compensation for shake compensation can be adjusted as appropriate.

[0056] However, if the correction amount becomes large, the change in the display position on the display unit 96 and the amount of rotation of the concave mirror and the like included in the display optical system 98 will increase, increasing the burden on the HUD device 100. Therefore, rather than setting a uniformly large correction amount, it is better to be able to set individual correction amounts that can keep the shaking within an allowable range according to the characteristics (attributes) of the image.

[0057] Next, reference is made to Fig. 4. Fig. 4 is a diagram showing an example of a case where an appropriate correction amount is set based on the depression angle as seen by the viewer, the display distance, the type of image, etc. In Fig. 4, the same reference numerals are used to designate parts common to Fig. 1.

[0058] In A-1 of Fig. 4, the symbol HL is a straight line parallel (horizontal) to the road surface 9 with respect to the viewpoint 3 of the viewer 5. With respect to this straight line HL, the depression angles for the images AR1 and AR2 displayed on the display surface (virtual image display surface) PS are determined. The depression angle for the image AR1 is θa, and the depression angle for the image AR2 is θb (<θa).

[0059] As shown in A-2 of Fig. 4, the image AR1 is an image (virtual image) of the display showing the vehicle situation (specifically, the vehicle speed display SP of "60km / h") previously shown in Fig. 1. The image AR2 is an image (virtual image) of the turn-by-turn display as a superimposed image (specifically, the navigation display NVa consisting of the character string "Tokyo" and a "navigation arrow graphic" superimposed on the road surface). In the drawing, reference characters K1 and K2 respectively denote display lights for images AR1 and AR2 emitted from the HUD device 100. Reference character 2 denotes a windshield as a member on which the image is projected.

[0060] Here, we will return to A-1 in Fig. 4 to continue the explanation. In A-1 in Fig. 4, both images AR1 and AR2 are displayed on a display surface (virtual image display surface) PS, and the display distance (virtual image display distance or projection distance) is the same.

[0061] However, the depression angle of the image AR2 is smaller than that of the image AR1. Also, the image AR2 is perceived by the viewer 5 as being superimposed on the road surface 9 that is farther away, so the display distance (virtual image display distance) is substantially longer. If pitching causes a positional shift QS1 in the image AR2 on the display surface (virtual image display surface) PS, the viewer 5 actually perceives a larger depth shift QS2 as having occurred on the road surface 9. In other words, the viewer is highly sensitive to positional changes in the image AR2. Therefore, it is preferable to set a larger correction amount for image AR2 than for image AR1 to provide a difference.

[0062] Next, reference will be made to A-3 in Fig. 4. A-3 in Fig. 4 shows a display example in the case of adopting a configuration (two-surface HUD configuration) in which a plurality of display surfaces (virtual image display surfaces) for displaying an image (virtual image) are provided. In the example of A-3 in FIG. 4, a first display surface PS1 and a second display surface PS2 having a larger display distance are provided. The image AR1 is displayed on a first display surface PS1 located on the near side as viewed by a viewer 5, and the second image AR2 is displayed on a second display surface PS2 located on the far side.

[0063] For image AR2, which has a long display distance (projection distance), the impact on the viewer's vision when shaking due to pitching occurs is large, so it is preferable to set the amount of shake correction larger than that for image AR1.

[0064] Next, reference will be made to Fig. 5. Fig. 5(A) to (C) are diagrams showing an example of setting an appropriate correction amount for each image, and an example of correction amount change processing.

[0065] In Fig. 5(A), an image AR1 is displayed alone on a display surface (virtual image display surface) PS. In Fig. 5(A), the image AR1 is an image (virtual image) of a vehicle speed display SP. However, the present invention is not limited to this. The image AR1 may be displayed in the lower part (footer area) of the display area on the display surface PS of the HUD device 100, and may display basic vehicle information, road information, and the like. Specifically, examples include basic vehicle information such as that displayed on the speedometer, the current speed limit, map guidance information (small world) as an overhead view, caution icons such as "Stop" signs, and lane information at intersections.

[0066] In Fig. 5(A), the shake correction amount in the shake correction of image AR1 is set to "a". When shake correction is performed with this correction amount, for example, as shown on the right side of Fig. 5(A), the shake amplitude becomes L1.

[0067] In Fig. 5(B), the image AR2 is displayed alone on the display surface (virtual image display surface) PS. In Fig. 5(B), the image AR2 is an image (virtual image) of a turn-by-turn display (specifically, a navigation display NVa consisting of the characters "Tokyo" and a "navigation arrow graphic" superimposed on the road surface). However, the present invention is not limited to this. The image AR2 is information that is displayed superimposed on the real scene (such as scenery) in front of the vehicle 1. Specifically, for example, route guidance (turn-by-turn display) represented by fishbones or arrows, POI display icons that notify the location of stores and destinations by superimposed display, a display indicating the switching of the speed limit (TSR display), a display for setting the distance interval from the vehicle ahead during autonomous driving, a display that surrounds the object to be noted with a frame, etc. can be exemplified.

[0068] In FIG. 5(B), the amount of shake correction in the shake correction of the image AR2 is set to "b (>a)". When shake correction is performed with this correction amount, for example, as shown on the right side of FIG. 5(B), the amplitude of the shake becomes L2 (<L1).

[0069] In FIG. 5(C), the images AR1 and AR2 are both displayed on the display surface (virtual image display surface) PS. Here, as an example, it is assumed that the image AR2 is newly added while the image AR1 is being displayed.

[0070] In this case, the plurality of display images are distinguished into an image with the maximum correction amount and an image with a correction amount smaller than that (other images). In the example of FIG. 5(C), the image AR2 corresponds to the image with the maximum correction amount, and the image AR1 corresponds to the other images.

[0071] And in order to suppress the increase in the sense of discomfort such as flickering due to the difference in the correction amount, a correction amount change process is performed. Specifically, the correction amount (correction amount for single display) a for the image AR1, which is the other image, is changed so as to approach the correction amount (maximum correction amount) b for the image AR2 or become equal to it. As a result, the correction amount of the image AR1 becomes "c: provided that a < c ≦ b is satisfied". Thereby, the amplitude of the shake of the image AR1 becomes L3 (provided that L2 ≦ L3 < L1 is satisfied), and the amplitude of the shake of the image is reduced compared to the case of single display in FIG. 5(A).

[0072] Through such cooperative correction amount change control, the difference in the correction amount between images AR1 and AR2 is reduced or eliminated. As a result, the degree of shaking of the two images is made uniform, and discomfort such as a flickering feeling can be effectively reduced.

[0073] (Second Embodiment) Next, refer to FIG. 6. FIG. 6 is a diagram showing another example of setting an appropriate correction amount for each image and another example of the correction amount change process. In FIG. 6, the same reference numerals are given to the same parts as in FIG. 5.

[0074] In FIG. 6(A), an image (virtual image) AR3 of a display (TSR display: specifically, including a display SL of a sign indicating the speed limit and a display WL of a white line) indicating the switching of the speed limit is displayed on the display surface (virtual image display surface) PS.

[0075] The image AR3 is displayed in the central region in the vertical direction of the display surface (virtual image display surface) PS, has a smaller depression angle than the image AR1 in FIG. 5 described above, and has a larger depression angle than the image AR2. Therefore, the correction amount for shake correction in the case of single display of the image AR3 is d (where a < c < b is satisfied), and the amplitude of shaking is L4 (where L2 < L4 ≤ L1 is satisfied).

[0076] In FIG. 6(B), assume a case where a new image AR3 is added in the display state (the state where images AR1 and AR2 are displayed) shown in FIG. 5(C) above.

[0077] The image with the smallest depression angle is AR2. Therefore, the image with the largest correction amount is AR2. The images AR1 and AR3 correspond to "other images" with a correction amount smaller than the largest correction amount.

[0078] Regarding the image AR1, since the correction amount change process has already been performed in the example of FIG. 5(C), there is no change in the correction amount in FIG. 6(B) either, and as shown in FIG. 5(C), the correction amount is "c". On the other hand, for the image AR3, in order to suppress a sense of discomfort such as flickering, a cooperative correction amount change process is performed, and the correction amount is changed so as to approach or become equal to the maximum correction amount "b".

[0079] As a result, for the image AR3, the correction amount is changed from the correction amount "d" in FIG. 6(A) to a correction amount e (where d < e ≦ b is satisfied). Along with this, the amplitude of the shake of the image AR3 is reduced to L5 (where L2 ≦ L5 < L4 is satisfied).

[0080] By such cooperative correction amount change control for the image AR2, the difference in the correction amount between each of the images AR1 to AR3 is reduced or the difference disappears. Thus, the degree of shake of the three images is made uniform, and a sense of discomfort such as flickering can be effectively reduced.

[0081] (Third Embodiment) Next, refer to FIG. 7. FIG. 7 is a diagram showing still another example of setting an appropriate correction amount for each image and still another example of the correction amount change process.

[0082] In FIG. 7(A), an image (virtual image) AR4 of a display (TSR display: specifically, including a display SL of a sign indicating the restricted speed and a display WL of a white line) indicating the switching of the restricted speed is displayed on the display surface (virtual image display surface) PS.

[0083] The image AR4 is displayed in the upper partial region in the vertical direction of the display surface (virtual image display surface) PS, has a smaller depression angle than the image AR2 in FIG. 5 described above, and is more susceptible to the influence of the shake due to pitching.

[0084] Therefore, the correction amount of the shake correction in the case of the single display for the image AR4 is f (>b). The amplitude of the shake is L6 (<L2).

[0085] In FIG. 7(B), assume a case where a new image AR4 is added in the display state (the state where the images AR1 and AR2 are displayed) shown in FIG. 5(C) above.

[0086] The image with the smallest depression angle is AR4. Therefore, the image with the largest correction amount is AR4. Images AR1 and AR2 will fall under "other images" with a correction amount smaller than the largest correction amount.

[0087] Therefore, for both images AR1 and AR2, in order to suppress a sense of discomfort such as flickering, a cooperative correction amount change process is carried out. In other words, the current correction amounts "c" and "b" (the correction amounts shown in Fig. 5(C)) of images AR1 and AR2 are changed so as to approach or be equal to the correction amount "f", which is the largest correction amount.

[0088] As a result, for image AR1, the correction amount changes from the correction amount "c" shown in Fig. 5(C) to "correction amount h (where c < h ≤ f is satisfied)". Along with this, the amplitude of the shake of image AR1 becomes L8 (where L6 ≤ L8 < L3 is satisfied).

[0089] Also, for image AR2, the correction amount changes from the correction amount "b" shown in Fig. 5(C) to "correction amount g (where b < g ≤ f is satisfied)". Along with this, the amplitude of the shake of image AR2 becomes L7 (where L6 ≤ L7 < L2 is satisfied).

[0090] By such cooperative correction amount change control for images AR1 and AR2, the difference in the correction amounts between images AR1, AR2, and AR4 is reduced or eliminated. Therefore, the degree of shake of the three images becomes uniform, and a sense of discomfort such as flickering can be effectively reduced.

[0091] (Fourth Embodiment) Next, the change timing of the correction amount in the correction amount change process and the correction amount return process will be described. Refer to Fig. 8. Fig. 8 is a timing chart showing an example (corresponding to Fig. 5) of the correction amount change process and the correction amount return process.

[0092] First, the correction amount change process will be described. At time t1, image AR1 is displayed, and at time t2, image AR2 is displayed. Note that image AR2 disappears at time t5. Also, along with the disappearance of image AR2, the shake correction for image AR2 also ends at time t5.

[0093] In the shake correction for image AR1, the correction amount during the period from time t1 to t2 is the correction amount "a", and at time t2, it is changed to the correction amount "c (a < c ≤ b)" by the correction amount change process.

[0094] Also, in the shake correction for image AR2, the correction amount during the period from time t2 to t5 is the correction amount "b".

[0095] Next, the correction amount restoration process will be described. At time t5, image AR2 (the image that caused the correction amount change process) is erased. Therefore, time t5 becomes the timing at which it is possible to restore the changed correction amount.

[0096] However, if the correction amount restoration process is performed at time t5, "the disappearance of image AR2" and "the change in the correction amount for image AR1 due to the correction amount restoration process" may overlap, increasing the discomfort of the viewer.

[0097] Therefore, the correction amount restoration process for image AR1 (the process of returning the current correction amount to the correction amount before the correction amount change process) is performed at time t6, which is delayed by a predetermined time TD from time t5. At time t6, the correction amount of image AR1 returns from the correction amount "c" to the original correction amount "a".

[0098] By providing a delay of the predetermined time TD, the discomfort felt by the viewer can be suppressed. Instead of providing a delay of the predetermined time TD, from the perspective of ensuring safety during driving, the correction amount restoration process may be performed when the vehicle speed of the host vehicle 1 is less than a predetermined value.

[0099] (Fifth Embodiment) Next, refer to FIG. 9. FIG. 9 is a timing chart showing another example (corresponding to FIG. 6) of the correction amount change process and the correction amount return process.

[0100] An image AR1 is displayed at time t1, an image AR2 is displayed at time t2, and an image AR3 is displayed at time t3. Note that the image AR3 disappears at time t4, and the image AR2 disappears at time t5. Also, along with the disappearance of the images AR3 and AR2, the shake correction for the images AR3 and AR2 also ends at times t4 and t5.

[0101] The image with the maximum correction amount is the image AR2 (the maximum correction amount is "b"). Therefore, the images AR1 and AR3 correspond to "other images" with a correction amount smaller than the maximum correction amount.

[0102] As described above with reference to FIG. 8, the correction amount in the shake correction for the image AR1 is the correction amount "a" during the period from time t1 to t2, and at time t2, it is changed to the correction amount "c (a < c ≤ b)" by the correction amount change process.

[0103] Also, for the image AR3, at time t3, the correction amount "d" for single display is changed to the correction amount e (where d < e ≤ b) by the correction amount change process.

[0104] Also, at time t5, the image AR2 (the image that caused the correction amount change process) is deleted, and at time t6, which is delayed by a predetermined time TD from time t5, the correction amount of the image AR1 returns from the correction amount c to the original correction amount a. By providing a delay of the predetermined time TD, the discomfort felt by the viewer can be suppressed. Note that instead of providing a delay of the predetermined time TD, from the viewpoint of ensuring safety during driving, the correction amount return process may be performed when the vehicle speed of the host vehicle 1 is less than a predetermined value.

[0105] (Sixth Embodiment) Next, refer to FIG. 10. FIG. 10 is a timing chart showing still another example (corresponding to FIG. 7) of the correction amount change process and the correction amount return process.

[0106] The image AR1 is displayed at time t10, the image AR2 is displayed at time t11, and the image AR4 is displayed at time t12. Note that the image AR4 disappears at time t14, and the image AR2 disappears at time t13. Also, with the disappearance of the images AR4 and AR2, the shake correction for the images AR4 and AR2 also ends at times t14 and t13.

[0107] The image with the maximum correction amount is the image AR4 (the maximum correction amount is "f"). Therefore, the images AR1 and AR2 correspond to "other images" with a correction amount smaller than the maximum correction amount.

[0108] The correction amount in the shake correction for the image AR1 is the correction amount "a" during the period from time t10 to t11. At time t11, a correction amount change process (the first time) is performed, and during the period from time t11 to t12, the correction amount becomes "c (where a < c ≤ b is satisfied)".

[0109] At time t12, since the image AR4 with the maximum correction amount is displayed, for the image AR1, a correction amount change process (the second time) is performed, and the correction amount becomes "h (where c < h ≤ f is satisfied)".

[0110] For the image AR2, the correction amount is "b" during the period from time t11 to t12. During the period from time t12 to t13, due to the correction amount change process associated with the new addition of the image AR4, the correction amount becomes "g (where b < g ≤ f is satisfied)".

[0111] Also, at time t14, the image AR4 (the image that caused the correction amount change process) is deleted, and at time t15, which is delayed by a predetermined time TD from time t14, the correction amount of the image AR1 returns from the correction amount c to the original correction amount a. By providing a delay of the predetermined time TD, the discomfort felt by the viewer can be suppressed. Instead of providing a delay of the predetermined time TD, from the perspective of ensuring safety during driving, the correction amount return process may be performed when the vehicle speed of the host vehicle 1 is less than a predetermined value.

[0112] Seventh embodiment Next, reference is made to Fig. 11. Fig. 11 is a flowchart showing an example of a main procedure of display control.

[0113] In step S1, it is determined whether or not multiple images are to be displayed. If the answer is Y, the process proceeds to step S4, and if the answer is N, the process proceeds to step S2.

[0114] In step S2, the correction amount for the single display image (including no correction) is determined. In this determination, the correction amount may be larger for an image with a large depression angle than for an image with a small depression angle, and the correction amount may be larger for an image with a long display distance (virtual image display distance, projection distance) than for an image with a short display distance, and the correction amount may be larger for an image type for which the viewer has a high sensitivity to shaking than for an image type for which the viewer has a low sensitivity.

[0115] After step S2, the process proceeds to step S3 where correction processing is performed. Then, in step S10, it is determined whether display (vehicle display) is to be ended. If Y, the process ends, and if N, the process returns to step S1.

[0116] On the other hand, if the result in step S1 is Y and the process proceeds to step S4, the "image with the maximum correction amount" is identified from among the multiple images in step S4. Images with a correction amount smaller than the maximum correction amount are classified as "other images."

[0117] In step S5, a correction amount change process is performed. In other words, the correction amount of the other images is changed from the current correction amount (the correction amount of the single display when the correction amount change process has not been performed in the past, or the correction amount after the correction amount change process when the correction amount change process has been performed in the past) to the maximum correction amount or is set to the same.

[0118] In step S6, the correction process for each image is carried out using the correction amount set in step S5.

[0119] In step S7, it is determined whether or not the timing has come when the correction amount can be restored to its original state. If the answer is No, the process returns to step S6, and if the answer is Yes, the process proceeds to step S8.

[0120] In step S8, it is determined whether a predetermined time has elapsed since the timing when the correction amount can be restored or whether the vehicle speed of the vehicle is less than a predetermined threshold value. If the result is N, the process returns to step S6, and if the result is Y, the process proceeds to step S9.

[0121] In step S9, a correction amount return process is performed. Then, in step S10, it is determined whether or not to end the display (vehicle display). If Y, the process ends, and if N, the process returns to step S1.

[0122] As described above, according to the present invention, in shake correction that suppresses image shaking caused by vibration of the vehicle, shake correction is performed using a correction amount appropriate for the image to be displayed, and a display control device can be realized that can suppress the discomfort felt by the viewer when multiple images with different correction amounts are displayed.

[0123] The present invention is not limited to the above-described embodiment, and various modifications and applications are possible. In the above-described embodiment, pitching correction is used as an example of vibration correction, but vibration correction for rolling, for example, may be used.

[0124] The HUD device or display system also includes those used as simulators (for example, aircraft simulators, simulators as game devices, etc.).

[0125] In the above-described embodiment, the projection display device has been mainly described as an HUD device, but the present invention is not limited to this, and for example, an in-car projector or the like can also be used.

[0126] The present invention is not limited to the above-described exemplary embodiments, and those skilled in the art could easily modify the above-described exemplary embodiments to the extent that they fall within the scope of the claims. [Explanation of symbols]

[0127] 1 Vehicle (own vehicle), 2 Projected part (windshield, etc.), 3 Eye (viewpoint), 4 Steering wheel, 5 Viewer (passenger, driver), 9 Road surface, 41 Dashboard, 75 Various sensors (acceleration sensor, etc.), 77 ECU, 80 Display control device, 81 Control unit, 83 Display content determination unit, 84 Display distance setting unit, 85 Display display position determination unit, 86... depression angle calculation unit, 87... shake compensation unit, 88... compensation amount adjustment unit, 89... compensation amount setting unit, 90... compensation amount changing unit, 91... compensation amount restoration processing unit, 92... display state changing unit, 93... vibration information acquisition unit, 95... projection unit, 96... display unit (liquid crystal panel, screen, etc.), 98... display optical system, 100... HUD device, PS, PS1, PS2... display surface (virtual image display surface).

Claims

1. A display control device having a control unit that is mounted on a vehicle and controls image display of a display device that allows an image to be viewed by a viewer who is a passenger of the vehicle, The control unit is a vibration information acquisition unit that acquires vibration information related to vibrations of the vehicle while it is running; a shake correction unit that performs shake correction so as to suppress shaking of the image caused by the vibration and is capable of adjusting a correction amount in the shake correction; having The shake correction unit is When the number of images to be displayed is one, shake correction is performed with a correction amount for single display, including a correction amount of zero, suitable for the image; When there are a plurality of images to be displayed, the images are classified into a maximum correction amount image having a maximum correction amount in shake correction and other images having a correction amount smaller than the maximum correction amount; performing a correction amount change process for changing a current correction amount in the shake correction of the other images so as to approach the maximum correction amount or to become the same as the maximum correction amount; Display control device.

2. The control unit is The correction amount of the single display is set to be larger for an image with a large depression angle as seen by the viewer than for an image with a small depression angle. Or, For an image having a large display distance, which is the distance from the viewer's viewpoint or a predetermined reference point to the image to be displayed, the correction amount of the single display is set to be larger than that of an image having a small display distance. Or, The correction amount of the single display is set to be larger for a type of image for which the viewer has a high sensitivity to a change in the position of the displayed image than for a type of image for which the viewer has a low sensitivity. The display control device according to claim 1 .

3. The control unit is In a situation where the correction amount change process has been performed in the past, when an image to be displayed is newly added and a new correction amount change process is performed, If there is no change to the image with the maximum correction amount, the new correction amount change process is performed on the newly added image; When there is a change to the image with the maximum correction amount, the new correction amount change process is performed on each of the images other than the newly added image. The display control device according to claim 1 .

4. The control unit is In a first display state in which one or more images are displayed, a new image having a maximum correction amount is additionally displayed, and the one or more images correspond to the other images, so that the correction amount change process is performed on the one or more images, resulting in a second display state; Thereafter, when the new image is erased and accordingly a first timing arrives at which the correction amount after the correction amount change process for the one or more images can be returned to the correction amount in the first display state, At a second timing when a predetermined time has elapsed from the first timing, a correction amount restoration process is performed to restore the correction amount to the first display state. Or, A condition is that the vehicle speed is less than a predetermined threshold value, and when the condition is satisfied, a correction amount restoration process is performed to restore the correction amount in the first display state. The display control device according to claim 1 .

5. A display unit for displaying an image; A display control device according to any one of claims 1 to 4, A display device having the above configuration.

6. The display device is a projection type display device that projects display light of an image displayed on the display unit onto a projection target, thereby displaying the image as a virtual image on a virtual image display surface. The display device according to claim 5 .

7. A display control method for controlling image display of a display device mounted on a vehicle and allowing an image to be viewed by a viewer who is a passenger of the vehicle, comprising: A first step of acquiring vibration information related to vibrations of the vehicle while it is running; When performing shake correction to suppress shaking of the image caused by the vibration, When the image to be displayed is one, the shake correction is performed with a correction amount for single display, including a correction amount of zero, suitable for the image; When there are a plurality of images to be displayed, the images are classified into a maximum correction amount image having a maximum correction amount in shake correction and other images having a correction amount smaller than the maximum correction amount; performing a correction amount change process for changing the correction amount in the shake correction of the other images so as to approach the maximum correction amount or to be the same as the maximum correction amount; Display control method.

Images