Head-up display device and display control system
The head-up display device addresses the issue of driver annoyance by adjusting the display position of virtual images based on eye height, ensuring the depression angle remains above a threshold, thus reducing the intrusion of virtual images into the driver's line of sight and improving visibility.
Patent Information
- Application Number
- PCT/JP2024/018798
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-26
- Filing Date
- 2024-05-22
- Publication Date
- 2024-12-05
AI Technical Summary
Head-up display devices struggle to minimize annoyance to drivers by ensuring that virtual images are not intrusive when the driver's eye position changes, as the current systems often result in virtual images entering the driver's line of sight, leading to discomfort.
A head-up display device with a control unit that adjusts the display position of virtual images based on the viewer's eye height, ensuring the depression angle remains above a predetermined threshold, thereby reducing annoyance by moving the virtual image's position to avoid entering the driver's direct line of sight.
The solution effectively reduces driver annoyance by maintaining optimal visibility and minimizing the intrusion of virtual images into the driver's direct line of sight, even when the driver's eye position changes, enhancing the usability of perspective expressions in various driving scenarios.
Smart Images

Figure JP2024018798_05122024_PF_FP_ABST
Abstract
Description
Head-up display device and display control system
[0001] The present invention relates to a head-up display device or the like that is used in a moving body such as a vehicle and that allows a virtual display object to be visually superimposed on the foreground of the moving body.
[0002] For example, Patent Document 1 describes a head-up display (HUD) device that uses a virtual display object (hereinafter referred to as a virtual image) to enhance expressiveness in the depth direction (front-rear direction of the vehicle) and enables intuitive information presentation to a viewer such as a driver. This HUD device places a virtual image at a predetermined position in the depth, up-down, left-right direction within the space of the foreground (the actual view in the forward direction as seen by the vehicle occupant), and forms an augmented reality (AR) by adding and displaying the virtual image to a real landscape (foreground) such as a road surface.
[0003] Furthermore, for example, paragraph
[0070] of Patent Document 1 also describes a configuration in which the angle of a reflective member such as a concave mirror of the HUD device can be appropriately adjusted by operating a rotation mechanism equipped with an actuator. Therefore, by adjusting the angle by rotating the reflective member to align it with the line of sight of the driver when seated, viewers of various heights can view the virtual image displayed by the HUD device. In other words, by adjusting the angle of the reflective member, it is possible to adjust the "eyebox," which is the spatial region around the eyes where the target virtual image can be viewed.
[0004] Meanwhile, Patent Document 2 also describes a similar HUD device, which describes an example of an expression that creates a sense of perspective. For example, in the display example shown in FIG. 4 of Patent Document 2, a virtual image in the shape of an arrow indicating the direction ahead is displayed at a position slightly above the road surface on which the vehicle is traveling (a position above the road surface). In such a case, the driver, who is the viewer, can create a sense of perspective by gradually narrowing the width (width in the left-right direction) of the rear end of the arrow toward the tip, as in the display example shown in FIG. 5 of Patent Document 2. In other words, the rear end of the arrow is perceived as being close, and the tip of the arrow as being far away. Such an expression that creates a sense of perspective (an expression that uses perspective) typically uses a technique such as linear perspective (perspective projection).
[0005] The above-described perspective-applied expressions are considered to be effective for notifications in driving assistance technologies using ADAS (Advanced Driver-Assistance Systems), such as navigation displays, ACC (Adaptive Cruise Control Systems), and LKAS (Lane Keeping Assist Systems), and are expected to be used in a variety of situations.
[0006] JP 2021-51231 A JP 2010-156608 A
[0007] The above-mentioned "eyebox" refers to a predetermined rectangular area, for example, a predetermined area, which is set to be the same as or include a large portion (e.g., 80% or more) of the area (also called the iris) where the viewpoint of a viewer, such as a driver, is expected to be located in the vehicle cabin. Therefore, the eyebox can be defined as: (1) an area within which the entire virtual image is visible, and a portion of the virtual image is not visible outside the area; (2) an area within which at least a portion of the virtual image is visible, and no portion of the virtual image is visible outside the area; (3) an area within which at least a portion of the virtual image is visible at a predetermined brightness or higher (e.g., approximately 1 / 50 of the brightness of the virtual image of the image viewed at the center of the eyebox), and the entire virtual image is below the predetermined brightness outside the area; or (4) an area within which at least a portion of the virtual image can be viewed stereoscopically, and no portion of the virtual image is viewed stereoscopically outside the area, when the HUD device is capable of displaying a stereoscopically visible virtual image. That is, if the driver, who is the viewer, places both eyes outside the eyebox, the driver cannot see the entire virtual image, the visibility of the virtual image is very low and it is difficult to perceive, or the virtual image cannot be viewed stereoscopically.
[0008] Therefore, for example, to allow a short driver to view a virtual image, the position of the eyebox must be set relatively low. In this case, due to the optical properties of the HUD device, the height above the road surface at which the virtual image is formed increases. This reduces the angle (depression angle) between a line parallel to the road surface from the driver's eye position and a line connecting the driver's eye to the center of the virtual image. Under such circumstances, the virtual image appears to be directly in front of the driver's eyes, and when the driver turns their gaze forward (in a direction parallel to the road surface from the driver's eye position) to drive the vehicle, the virtual image is likely to enter their line of sight. HUD devices are expected to have the advantage that, when the driver faces forward, they can view information (images displayed by the HUD device) with minimal line of sight movement. However, if the virtual image is displayed in the same direction (very close) as the driver's likely line of sight to drive the vehicle, the driver's visual attention will be excessively focused on the virtual image rather than the real view outside the vehicle, and the driver may find the virtual image annoying.
[0009] In particular, since perspective-based representations are expected to be used in a variety of situations, it was desirable to make them easier to attract the driver's visual attention and free them from the annoyances mentioned above.
[0010] The present invention has been made to solve the above-mentioned problems, and aims to provide a head-up display device and a display control system that improves visibility by controlling the display position of a virtual image even if the viewer's eye position (height direction) changes, and that reduces the annoyance to the viewer, particularly when a perspective-enhanced representation is used.
[0011] Other objects of the present invention will become apparent to those skilled in the art by referring to the following exemplary aspects and best modes, as well as the accompanying drawings.
[0012] In order to facilitate an understanding of the outline of the present invention, the following examples are given of embodiments according to the present invention.
[0013] In a first aspect, a head-up display device allows a viewer to view a virtual image by superimposing it on a foreground in a display area virtually set in front of a vehicle, and includes a drive unit that can adjust the angle formed by a line parallel to the road surface based on the eye height of the viewer and a line connecting the viewer's eyes and the display area, and a control unit that controls the display of the virtual image in the display area, and if the adjusted angle differs from a predetermined angle reference value, the control unit controls to change the display position of the virtual image in the display area in accordance with the adjusted angle.
[0014] Here, the "predetermined angle reference value" is a value determined by the specifications of the HUD device through verification (experiments or simulations). For example, Figure 9 shows verification results for a HUD with a 10° × 3° field of view and a 10 m image plane distance. This graph shows the percentage of subjects who felt the annoyance was tolerable (acceptance rate) when the depression angle, which is the angle between a line parallel to the road surface and a line connecting the viewer's eyes and the display area, is changed based on the viewer's eye height. From this graph, it can be seen that the acceptance rate drops sharply when the depression angle falls below 1°. Therefore, for example, when performing a process to change the display position of a virtual image downward, it can be inferred that the depression angle of the upper end of the virtual image should be set deeper (higher) than 1° (predetermined angle reference value).
[0015] According to the first aspect, the control unit adjusts the angle (depression angle) between a line parallel to the road surface and a line connecting the viewer's eyes and the display area, based on the viewer's eye height, by using a drive unit that can adjust the angle, and compares the adjusted angle with a predetermined reference angle. If the adjusted angle differs from the predetermined reference value, the control unit changes the display position of the virtual image on the display area (virtual image formation surface) according to the adjusted angle. This reduces the annoyance felt by the viewer even when the viewer's eye position (height) changes.
[0016] In this way, the control unit controls the position of the upper end of the virtual image to move toward the lower end of the display area. In other words, by controlling the display position of the virtual image even if the viewer's eye position (in the vertical direction) changes due to height, the annoyance felt by the viewer, the driver, due to the virtual image easily entering his or her line of sight when looking straight ahead can be reduced.
[0017] In a second aspect dependent on the first aspect, the control unit may perform control to move the position of the upper end of the virtual image by a predetermined amount toward the lower end of the display area and display the virtual image.
[0018] Here, the "predetermined amount" refers to the amount of movement of the display position of the virtual image in the display area optimized for each angle (depression angle) formed by a line parallel to the road surface and a line connecting the viewer's eyes and the display area, based on the viewer's eye height. The display position of the virtual image (the display position of the virtual image in the display area whose angle has been adjusted by the drive unit) is a value determined by the above-mentioned verification. As a result of the verification, the control unit performs control to move the position of the upper end of the virtual image toward the lower end of the display area by the amount of movement of the optimized display position so that the depression angle is 1° or more.
[0019] According to the second aspect, the control unit displays the virtual image by moving the position of the upper end of the virtual image by a predetermined amount toward the lower end of the display area (virtual image formation surface). In other words, the control unit performs control to lower the position of the eyebox. Therefore, for example, assuming that a short viewer will be viewing the virtual image, the position of the virtual image displayed in the display area (virtual image formation surface) will be higher, thereby reducing the annoyance felt by the viewer due to the virtual image being more likely to be in their line of sight when looking straight ahead.
[0020] In a third aspect dependent on the first aspect, the control unit may perform control to move the position of the lower end of the virtual image by a predetermined amount toward the upper end of the display area and position and display the virtual image.
[0021] According to the third aspect, the control unit moves the position of the bottom edge of the virtual image by a predetermined amount toward the top edge of the display area (virtual image formation surface) and positions and displays the generated virtual image. In other words, the control unit performs control to raise the position of the eyebox. Therefore, for example, assuming that a tall viewer will be viewing the display, the position of the virtual image displayed in the display area (virtual image formation surface) is lowered, thereby reducing the annoyance felt by the viewer due to the virtual image being more likely to be in their line of sight when looking straight ahead.
[0022] In a fourth aspect dependent on the first aspect, the control unit may assign to the display area as a restricted area an area in which the angle formed by a line parallel to the road surface based on the eye height of the viewer and a line connecting the viewer's eyes and the display area is higher than a predetermined angle reference value, and perform control to place and display the virtual image in the restricted area.
[0023] Here, the "restricted area" is an area that is narrower than the display area (virtual image formation surface) that is virtually set in front of the vehicle, and is allocated by moving a predetermined amount toward the bottom or top of the display area.
[0024] According to the fourth aspect, the control unit moves the virtual image by a predetermined amount toward the lower end of the display area (virtual image formation surface) and displays it in an allocated restricted area, so that a viewer, for example, a driver, can view the virtual image displayed in the display area (virtual image formation surface) without moving his / her line of sight in the vertical direction. In other words, even if the viewer's eye position (in the vertical direction) changes due to height, the display position of the virtual image can be controlled to reduce the annoyance felt by the viewer, that is, the driver.
[0025] In a fifth aspect dependent on the first aspect, the control unit may change the degree of perspective provided in accordance with the adjusted angle, and perform control to position and display the virtual image in accordance with the degree of change.
[0026] Here, the "degree to which a sense of depth is imparted" refers to the level of perspective application, or in other words, the strength of image processing applied to emphasize the sense of depth. A perspective image generated by applying perspective is an image that has been subjected to image processing that emphasizes the sense of depth of the target image by, for example, changing the brightness, shade, color tone, etc. according to the display distance or the size of the target, or by applying image processing such as shading, overlapping, and blurring. The strength of this image processing is referred to as the "degree to which a sense of depth is imparted."
[0027] According to a fifth aspect, the control unit changes the degree of perspective according to the angle adjusted by the drive unit and displays the perspective image. For example, perspective images with an enhanced perspective according to the adjusted angle are prepared in advance, and the selected perspective image is displayed according to the adjusted angle. This ensures that the sense of depth provided by the perspective image overlaps with the road surface in the foreground, providing the viewer with a more natural and perspective-oriented view.
[0028] In a sixth aspect which is dependent on the first to fifth aspects, the control unit may control the display of the virtual image so that, when the end of the display area closer to the vehicle is the bottom end and the end farther from the vehicle is the top end, the virtual image is displayed at an angle to the viewer so that the top end is farther away than the bottom end.
[0029] According to the sixth aspect, a display area (virtual image formation surface) that is a virtual virtual image display surface is used, which is an uphill slope in which the upper end of the display area (virtual image formation surface) is farther away than the lower end, thereby enabling a display with enhanced expressiveness in the depth direction (front-to-rear direction of the vehicle) and realizing more natural visibility of the virtual image with a sense of perspective. In addition, the virtual image display distance (imaging distance) can be adjusted depending on the display position, and a display with a sense of depth can be realized over a fairly wide range at the far end.
[0030] In a seventh aspect dependent on the first to sixth aspects, the control unit may perform control to position and display the virtual image, which is a perspective image that can represent lanes along a road with a sense of perspective as seen by the viewer.
[0031] According to the seventh aspect, the control unit displays the width (width in the left-right direction) of the lane located at the lower end of the display area (virtual image formation surface) so that the width gradually narrows toward the upper end, thereby creating a sense of perspective. Furthermore, since a display with a sense of depth is realized over a fairly wide range at the upper end, this is considered to be particularly effective not only for lanes but also for notifications in driving assistance technologies using ADAS, such as when an ACC system is used to clearly notify the driver that a preceding vehicle has been captured, or when an LKAS system is used to clearly notify the driver whether or not the vehicle is deviating from its lane.
[0032] In an eighth aspect, there is provided a display control system having a head-up display device that displays a virtual image superimposed on a foreground in a display area virtually set in front of a vehicle, allowing a viewer to view the image, and a processing device that generates the virtual image, wherein the head-up display device has a display unit that has a virtual image display surface including an optical element, a drive unit that rotates the optical element within a predetermined angle range, and a control unit that controls the display of the virtual image in the display area, and the processing device takes in information regarding the angle range of the optical element that is set by a predetermined operation of the viewer, and enables adjustment of the angle between a line parallel to the road surface and a line connecting the viewer's eyes and the display area based on the eye height of the viewer according to the taken-in information, and if the adjusted angle differs from a predetermined angle reference value, controls to change the display position of the virtual image generated in accordance with the adjusted angle, and instructs the head-up display device to display the virtual image at the changed display position of the display area.
[0033] According to an eighth aspect, the processing device sets the rotation range of the optical element through a predetermined operation by the viewer, and compares the angle (depression angle) formed by a line parallel to the road surface and a line connecting the viewer's eyes and the display area, based on the viewer's eye height, adjusted by a drive unit that can adjust the angle, with a predetermined angle reference value. If the adjusted angle differs from the predetermined reference value, the processing device controls the display position of the virtual image on the display area (virtual image formation surface) according to the adjusted angle. Then, the processing device instructs the head-up display device to display the virtual image based on the changed display position. In response, the head-up display device (controller) controls the virtual image generated by the processing device and displays the virtual image on the display area (virtual image formation surface) according to the changed display position information. Therefore, even if the viewer's eye position (height direction) changes due to height or other reasons, a display control system can be provided that reduces the annoyance felt by the viewer, i.e., by controlling the display position of the virtual image through a predetermined operation by the viewer. In particular, expressions using perspective that are likely to attract visual attention are expected to be used in a variety of scenes, and the effect obtained is significant when there is a change in the eye position (height direction) of the driver who is the viewer.
[0034] Those skilled in the art will easily understand that the exemplified embodiments according to the present invention can be further modified without departing from the spirit of the present invention.
[0035] FIG. 1 is a diagram illustrating an example of application of a head-up display device according to an embodiment of the present invention to a vehicle. FIG. 2 is a block diagram illustrating the configuration of a vehicle control system to which the head-up display device according to an embodiment of the present invention is applied. FIG. 3 is a diagram illustrating a cross-sectional structure of a head-up display device according to an embodiment of the present invention, where FIG. 3(a) illustrates the rotation of an optical member (concave mirror) when the eye box is in a reference position and FIG. 3(b) illustrates the rotation of an optical member (concave mirror) when the eye box is lowered. FIG. 4 is a flowchart illustrating the operation of a vehicle control system to which the head-up display device according to an embodiment of the present invention is applied. FIG. 5 is a diagram illustrating an example of a display mode of the head-up display device according to an embodiment of the present invention. FIG. 6A is a diagram for explaining an example in which the present invention is not implemented, where the right diagram illustrates a state in which the eye box is set at a relatively high position and the left diagram illustrates a state in which the eye box is set at a relatively low position. FIG. 6B is a diagram illustrating an example of a display mode of the head-up display device according to an embodiment of the present invention, where the right diagram illustrates a state in which the eye box is set at a relatively high position and the left diagram illustrates a state in which the eye box is set at a relatively low position. FIG. 7 is a diagram illustrating an example of a display mode of the head-up display device according to an embodiment of the present invention. FIG. 8 is a diagram illustrating an example of a display mode of the head-up display device according to an embodiment of the present invention. Fig. 9 is a graph showing the visibility tolerance rate for each depression angle adjusted by the drive unit of the head-up display device according to the embodiment of the present invention. Fig. 10(a) shows an example of a notification form of a navigation device, Fig. 10(b) shows an example of a notification form of ACC of ADAS, and Fig. 10(c) shows an example of a notification form of LKAS of ADAS.
[0036] The best mode described below is used to facilitate understanding of the present invention, and therefore, those skilled in the art should be aware that the present invention is not unduly limited by the mode described below (hereinafter referred to as the present mode).
[0037] A vehicle control system to which the head-up display device of the present embodiment is applied will be described below with reference to the accompanying drawings.
[0038] (Configuration of the embodiment) FIG. 1 shows an example of application of a head-up display device (hereinafter referred to as an HUD device 100) of the present embodiment to a vehicle.
[0039] 1 , the projection display by the HUD device 100 is a display device provided inside an instrument panel IP of a vehicle CR, and the display light L projected by the HUD device 100 is reflected by a windshield WS of the vehicle CR toward a driver DR who is a viewer seated in the driver's seat inside the vehicle CR, thereby displaying a virtual image V. Here, the HUD device 100 can display the virtual image V anywhere within a specific display area VA (virtual image formation surface) that is virtually set in front of the vehicle CR.
[0040] In other words, the HUD device 100 emits (projects) display light (image) L emitted from a liquid crystal display unit 120 (FIG. 2), which will be described later, onto the windshield WS, which is the projection target, and allows the driver DR, who is the viewer, to view the display light L (virtual image V) obtained in response to this emission. This allows the driver DR to view the virtual image V superimposed on the scenery ahead of the vehicle CR.
[0041] In this case, the HUD device 100 projects display light L such that the virtual image V is tilted at an angle of 45 degrees or less (0 to 45 degrees) relative to the road surface on which the vehicle CR is traveling, and the upper part of the virtual image V appears to be tilted farther away from the viewer than the lower part. This allows the driver DR, who is the driver of the vehicle, to view the virtual image V displayed mainly superimposed on the road surface ahead of the vehicle CR, and obtain various information indicated by the content of the image.
[0042] An eye box IB is set in the HUD device 100. If a virtual range in space in which the HUD device 100 can form a virtual image V is defined as a display area VA, which is an imaging surface, the eye box IB is a field of view defined based on a virtual line connecting the viewer's eyes and the outer edge of the display area VA. The eye box IB is an angular range in which the driver DR, who is the viewer, can view the virtual image V. The angular range in which the driver DR can view the virtual image V can be adjusted by changing the angle (depression angle LD) formed between a line VL parallel to the road surface based on the viewer's eye height and a line (dashed line in FIG. 1 ) connecting the driver DR's eyes, who is the viewer, and the display area VA (virtual image imaging surface).
[0043] FIG. 2 is a block diagram showing an example of the configuration of a vehicle control system 1000 to which the HUD device 100 of this embodiment is applied.
[0044] 2, the vehicle control system 1000 includes a HUD device 100, an in-vehicle sensing device 200, a driving assistance device 300, a navigation device 400, and an operation input device 500. All of these devices enable bidirectional information transmission via a CAN (Control Area Network) 600, which is an in-vehicle LAN.
[0045] The on-vehicle sensing device 200 is a type of sensor necessary for recognizing the surrounding driving environment including the area ahead of the vehicle CR, and includes a camera, a LiDAR (Light Detection and Ranging), etc. It also includes an IMU (Inertial Measurement Unit) that detects the behavior of the vehicle CR, a vehicle speed sensor, etc. Information recognized or detected by the on-vehicle sensing device 200 is transferred via the CAN 600 to the driving assistance device 300, the navigation device 400, and the HUD device 100 of this embodiment.
[0046] The camera captures at least the forward field of view (actual view) of the vehicle CR, and the LiDAR uses near-infrared light, visible light, and ultraviolet light to, for example, illuminate an obstacle or the like in front of the vehicle CR captured by the camera, capture the reflected light with an optical sensor, and determine the distance to the obstacle based on the time difference. The IMU measures the vehicle's driving situation and attitude (acceleration [m / s 2 ] to detect translational movement in three axes, and angular velocity [deg / s] to detect rotational movement).
[0047] The driving assistance device 300 has at least one of a driving assistance function using ADAS such as LKAS and ACC that assists the driving operation of the driver DR, who is the viewer, and an automatic driving function that can act as a substitute for the driver. The driving assistance device 300 recognizes or detects the driving environment and behavior ahead of the vehicle CR based on information acquired from the on-board sensing device 200, and controls the vehicle CR (the drive system, braking system, and steering system, not shown) according to the analysis results of the information obtained here. In addition, the driving assistance device 300 can transfer the analysis results of this information to the HUD device 100, the navigation device 400, etc. via the CAN 600.
[0048] The navigation device 400 uses a global navigation satellite system (GNSS) such as a global positioning system (GPS) and a gyro sensor (GPS sensor 402), and an information processing unit 401 obtains map information from a map DB 404 or via wireless communication with the outside of the vehicle, and provides guidance to facilities in the vicinity of the vehicle CR, route guidance, etc. The navigation device 400 transfers a signal prompting display output based on this guidance information to the HUD device 100 via the CAN 600 at an appropriate timing.
[0049] The information processing unit 401 performs information processing such as searching for and providing guidance to destinations and facilities based on position information of the vehicle CR acquired from the in-vehicle sensing device 200, the driving assistance device 300, the operation input device 500, and built-in sensors such as a GPS and a gyro sensor (GPS sensor 402). Based on the results of this information processing, for example, when it is detected that the vehicle is approaching an intersection where it should turn, the information processing unit 401 generates a virtual image V such as an arrow indicating a right or left turn at the intersection and transfers it to the HUD device 100. The information processing unit 401 also outputs a drive signal via the CAN 600 to control the rotation of a concave mirror 141 (see FIGS. 3A and 3B ), which is an optical member (reflecting member) of the HUD device 100 (described later), in order to control the position of the eye box IB.
[0050] The information processing unit 401 further generates, for example, a lane Ln (perspective image VP) with perspective applied to evoke a road, generates a virtual image V superimposed with other parts (for example, speed limit signs, vehicle speed, etc.), and writes the virtual image V together with the calculated display position information in a predetermined area of the storage unit 403. As is well known, the perspective image VP is a virtual image V generated by modifying the brightness, shade, color tone, etc. according to the display distance and the size of the object, or by performing image processing such as shading, overlapping, and blurring to emphasize the perspective of the object image.
[0051] The storage unit 403 is a memory that is allocated with a program area and a work area, and that implements, for example, static RAM, dynamic RAM, flash memory, etc. Here, various programs necessary for navigation, such as destination and facility search and guidance, are written in the program area, and information generated in the execution process of the various programs described above and display information including virtual images to be displayed on the HUD device 100 are allocated and temporarily written in the work area.
[0052] The navigation device 400 has a map database (map DB 404). The map DB 404 can acquire and store the latest map information, for example, by communicating with an external center (not shown) via a V2X (Vehicle to X) type communication system (not shown). The map information stored in the map DB 404 is mapping data that has been digitized to represent the driving environment of the vehicle CR. The mapping data is preferably digital data of a particularly high-precision dynamic map. A "dynamic map" is a digital map that combines a huge amount of dynamic information that changes every moment, such as traffic regulations, construction information, accidents, congestion, pedestrians, and traffic lights, with static information such as high-precision three-dimensional position information (road surface information, slope information, three-dimensional structures).
[0053] The operation input device 500 is, for example, switches provided near a steering wheel (steering handle). By operating these switches, the driver, who is the viewer, can switch the HUD device 100 on / off, select an execution function of the driving assistance device 300 that executes ADAS, such as LKAS or ACC, set the air volume of the air conditioner, and further set setting information (the position of the eye box IB) that instructs the rotation operation of the optical member (concave mirror 141) by the drive unit 110 of the HUD device 100 (see FIGS. 3( a) and 3(b)).
[0054] The HUD device 100 includes a drive unit 110 , a liquid crystal display unit 120 , and a control unit 160 .
[0055] The driving unit 110 is connected to a concave mirror 141 (see FIGS. 3A and 3B ), which is an optical member (reflective member) provided inside the HUD device 100 (described later), and can rotate the concave mirror 141 to a predetermined angular position based on a control signal generated by the control unit 160. The rotational position of the concave mirror 141 in the HUD device 100 has several steps, and can be adjusted within this range. The concave mirror 141 uses a stepping motor for its rotational drive, and can be rotated a predetermined number of steps corresponding to the position (setting information) of the eye box IB set by operating the operation input device 500.
[0056] The liquid crystal display unit 120 is mainly composed of a light source 121 made up of a light emitting diode mounted on a wiring board, and a TFT (Thin Film Transistor Liquid Crystal) liquid crystal display element 122 located on the emission side (directly above) of the light source 121 so as to transmit illumination light from the light source 121 and form display light L. This means that the light source 121 is provided behind the liquid crystal display element 122, and the liquid crystal display element 122 displays a predetermined image using light emitted from the light source 121.
[0057] The liquid crystal display unit 120 has a virtual image display surface 120a (see FIGS. 3(a) and 3(b)) disposed at an angle to the optical axis, and outputs display light L consisting of light in the visible wavelength range. For example, a light source 121 that emits white light can be applied, and the light emitted from this light source 121 is transmitted through the liquid crystal display element 122 to output display light L containing a desired image. The liquid crystal display element 122 forms a desired image based on display image data (drive signals) generated by drawing calculations performed by the control unit 160.
[0058] The control unit 160 causes the liquid crystal display unit 120 (liquid crystal display element 122) to display images to be displayed based on vehicle information and the like transferred from the driving assistance device 300 or the navigation device 400, such as not only guide images for lane Ln and the like, but also the speed and remaining energy of the vehicle CR, time, route guide images, and the like, using image data such as numerical values and icons. Note that the images to be displayed are not limited to vehicle information such as the vehicle speed and remaining energy, and any display format can be adopted.
[0059] The control unit 160 outputs a drive signal to the liquid crystal display element 122 to prompt the display to be visible to the driver DR, who is the viewer, based on information transferred from the information processing unit 401 of the navigation device 400. The control unit 160 is also connected to a light source 121 that transmits light to the liquid crystal display element 122, and can turn the liquid crystal display element 122 on and off, adjust the brightness, etc., by outputting a drive signal.
[0060] The control unit 160 can control the display position of the virtual image V to be viewed, for example, in accordance with setting information regarding the position of the eye box IB acquired by the driver DR, who is the viewer, operating the operation input unit 500. Specifically, in accordance with the setting information regarding the position of the eye box IB, the control unit 160 rotates and adjusts the angle of the optical member (concave mirror 141) adjusted by the drive unit 110. If the angle differs from a "predetermined angle reference value," the control unit 160 controls the display position of the virtual image V in the display area VA (which is virtually set in front of the vehicle CR) in accordance with the adjusted angle.
[0061] FIG. 6A is a diagram illustrating an example in which the present invention is not implemented. The right diagram shows a state in which the eye box IB is set at a relatively high position IB1, while the left diagram shows a state in which the eye box IB is set at a relatively low position IB2. The position of the display area VA when the center of the eye box IB is aligned with position IB1 in the right diagram is designated VA1, and the center position VAo of the display area VA is designated VA1o. A virtual image VP is displayed at a predetermined position in the display area VA1, and the position of the upper end U10 of the virtual image V at this time is designated U11. The depression angle LD, which is the angle formed by a line VL1 parallel to the road surface based on position IB1 and a line connecting position IB1 and the center VAo of the display area VA, is designated LD1. In the right diagram, the depression angle LD1 is relatively large, and the position of the upper end U11 of the virtual image V displayed in the display area VA1 does not fall within the range of the depression angle threshold LDt. Therefore, the entire virtual image displayed in the display area VA1, including the upper end U11, is positioned at a position where the depression angle is greater than the depression angle threshold LDt, thereby reducing the annoyance felt by the viewer due to the virtual image entering their field of vision.
[0062] In the left diagram of FIG. 6A , the center position of the eyebox IB is aligned with position IB2 (the position of the eyebox IB is lowered compared to the right diagram). Lowering the position of the eyebox IB raises the display position within the display area VA. In FIG. 6A , lowering the center position of the eyebox IB from position IB1 to IB2 in the right diagram raises the center VAo of the display area VA from position VA1o to position VA2o in the right diagram. By lowering the eyebox IB and raising the display area VA accordingly, the depression angle LD2, which is the angle between a line VL2 parallel to the road surface based on position IB2 and a line connecting position IB2 and the center VA2o of the display area VA2, becomes smaller than the depression angle LD1 in the right diagram. Lowering the eyebox IB reduces the depression angle LD, so that, for example, as shown in the left diagram, the upper end U11 of the virtual image V displayed in the display area VA2 falls within the range of the depression angle threshold LDt. This makes it more likely that the viewer's field of vision will be partially obstructed by the virtual image, which is likely to be annoying.
[0063] 6B is a diagram showing an example of a display mode of the head-up display device of this embodiment. In the left diagram of FIG. 6B , the center position of the eye box IB is set to a relatively low position IB2 (the position of the eye box IB is lowered compared to the right diagram). In this embodiment, the center position of the eye box IB is set to a relatively low position IB2 (the position of the eye box IB is lowered compared to the right diagram). Therefore, in this embodiment, the control unit 160 can perform control to display the virtual image V at a position U12 obtained by moving a position U11 of the upper end U10 of the display area VA (virtual image formation surface) by a predetermined amount a toward the lower end of the display area VA (virtual image formation surface), as described below (see the left diagram of FIG. 6B ). In other words, even when the control unit 160 performs control to lower the position of the eye box IB (which in turn raises the display area VA), it can prevent the virtual image V from being displayed within a predetermined depression angle threshold LDt by lowering the relative position of the virtual image V (at least the upper end U10 of the virtual image V) displayed within the display area VA.
[0064] The control unit 160 may further perform control to arrange and display the virtual image V in a restricted area VA-S (see FIG. 7) allocated within the display area VA (virtual image formation surface).
[0065] The "predetermined angle reference value" is a value determined by verification (experiments and simulations) based on the specifications of the HUD device 100. According to verification by the inventors, the percentage of subjects who felt that the annoyance of changing the depression angle LD, which is the angle between a line VL parallel to the road surface based on the viewer's eye height and a line (the dashed line in FIG. 1 ) connecting the viewer's eyes and the display area VA (virtual image formation surface) was tolerable, dropped sharply when the depression angle LD fell below 1°. Therefore, for example, when performing a process to change the display position of the display area VA (virtual image formation surface) of the virtual image V downward, it can be inferred that the depression angle of the upper end of the virtual image V should be set deeper (higher) than 1° (threshold LDt: the predetermined angle reference value).
[0066] The control unit 160 controls the display of a virtual image V, which is a perspective image VP of a lane Ln or the like along a road Rd, with a sense of distance, as seen by the driver DR who is the viewer. The control unit 160 may control the display of the perspective image VP by changing the "degree of perspective" according to the adjusted angle (depression angle LD). Here, the "degree of perspective" refers to the execution level of image processing for applying perspective, in other words, the strength of image processing performed to enhance the perspective. For example, perspective images VP (VP1, VP2, VP3) with enhanced perspective corresponding to the adjusted angle (depression angle LD) may be prepared in advance, and the display may be switched to one of the perspective images VP (VP1, VP2, VP3) selected according to the adjusted angle (depression angle LD).
[0067] Furthermore, the control unit 160 sets the display area VA (virtual image formation surface) so that the lower end of the display area VA (virtual image formation surface) is viewed at an angle farther from the vehicle CR than the upper end, and can display the virtual image V, for example, at a position corresponding to the display position information generated by the information processing unit 401 of the navigation device 400.
[0068] 3A and 3B are schematic cross-sectional views showing the internal structure of the HUD device 100 of this embodiment. FIG. 3A shows the rotation of the concave mirror 141 when the eye box IB is in the reference position, and FIG. 3B shows the rotation of the concave mirror 141 when the position of the eye box IB is lowered.
[0069] 3( a ) and 3 ( b ), the HUD device 100 is mainly composed of a liquid crystal display unit 120 , a first reflector 130 , a second reflector 140 , and a housing 150 .
[0070] The liquid crystal display unit 120 has a virtual image display surface 120a and outputs display light L consisting of light in the visible wavelength range, and can be, for example, a light source 121 that emits white light. The liquid crystal display element 122 transmits the light emitted from the light source 121 to output display light L containing a desired image. The liquid crystal display element 122 forms the desired image based on display image data (drive signals) generated by drawing calculations performed by the control unit 160.
[0071] The first reflector 130 includes a cold mirror 131. The cold mirror 131 reflects light in the visible wavelength range (450 to 750 nm), which includes the emission wavelength range of the liquid crystal display unit 120, with a high reflectance of, for example, 80% or more, and reflects light outside the visible wavelength range with a low reflectance. In this case, the cold mirror 131 is designed to reflect light outside the visible wavelength range, particularly light in the infrared wavelength range (heat rays from external light such as sunlight), with a low reflectance of, for example, 15% or less. Light that is not reflected by the reflective layer of the cold mirror 131 is configured to pass through the cold mirror 131. The cold mirror 131 prevents the liquid crystal display element 122 from becoming too hot due to heat from sunlight or the like.
[0072] The second reflector 140 includes a concave mirror 141 that reflects the display light L from the cold mirror 131 (that is, the liquid crystal display element 122 ), and a mirror holder 142 that holds the concave mirror 141 .
[0073] The concave mirror 141 has a second reflective layer 141 a formed by vapor deposition on a resin substrate made of polycarbonate having a concave surface. The concave mirror 141 is provided at an angle so that the second reflective layer 141 a faces the cold mirror 131 and the light-transmitting cover 153 and can be seen from the light-transmitting cover 153.
[0074] The concave mirror 141 magnifies the display light L from the cold mirror 131 and reflects (projects) it toward the translucent cover 153 (windshield WS of the vehicle CR). This means that the concave mirror 141 magnifies the display light L reflected by the cold mirror 131 and projects the magnified display light L onto the windshield WS through the translucent cover 153. The concave mirror 141 is adhered to a mirror holder 142 with a double-sided adhesive member. The mirror holder 142 is made of a synthetic resin such as ABS (Acrylonitrile Butadiene Styrene) and is fixed to the housing 150.
[0075] The concave mirror 141 is connected to the drive unit 110 and can be rotated within a predetermined angle range depending on the situation. This allows the arrival point of the display light (L) within the windshield WS to be changed. In other words, the position of the eye box IB of the HUD device 100 changes. This allows the eye position at which the driver DR, who is the viewer, can view the virtual image V, and the display position within the display area VA (virtual image formation surface) where the virtual image V is displayed to be changed.
[0076] Here, the lower the position of the eye box IB, the higher the display position within the display area VA. This means that the depression angle LD, which is the angle between two straight lines: a straight line VL parallel to the road surface based on the eye height of the driver DR who is the viewer, and a straight line (dashed line in FIG. 1) connecting the driver DR's eyes and the display area VA (virtual image formation surface), changes in a decreasing direction. On the other hand, the higher the position of the eye box IB, the lower the display position of the virtual image V within the display area VA (virtual image formation surface). This means that the depression angle LD, which is the angle between two straight lines: a straight line VL parallel to the road Rd based on the eye height of the driver DR who is the viewer, and a straight line (dashed line in FIG. 1) connecting the driver DR's eyes and the display area VA (virtual image formation surface), changes in a increasing direction.
[0077] In other words, when the concave mirror 141, which is in the reference position shown in Fig. 3(a), rotates clockwise, the position of the eye box IB becomes lower as shown in Fig. 3(b), and conversely, when it rotates counterclockwise, the position of the eye box IB becomes higher. Note that, since the depression angle LD is uniquely determined by a predetermined formula depending on the position of the eye box IB, calculating the display position of the virtual image V at the position of the eye box IB is equivalent to calculating the display position of the virtual image V in the display area VA (virtual image formation plane) using the depression angle LD.
[0078] The housing 150 is made of, for example, a black light-blocking synthetic resin material, and is formed in an approximately box-like shape. The housing 150 holds and accommodates the projection unit consisting of the LCD display unit 120, the first reflector 130, and the second reflector 140 in its internal space, a space portion 151, and is provided with an opening window portion 152 through which the upper part (on the windshield WS side) of the concave mirror 141 in the second reflector 140 opens.
[0079] The housing 150 is also provided with a light-transmitting cover 153, which serves as an emission section, to cover the open window section 152. The light-transmitting cover 153 is made of a light-transmitting synthetic resin material (e.g., acrylic resin), is formed in a curved shape (curved surface shape), and functions as a light-transmitting member through which the display light L reflected by the concave mirror 141 passes (passes through). That is, the display light L reflected by the cold mirror 131 and the concave mirror 141, i.e., the display light L output by the projection section, is projected onto the windshield WS through the light-transmitting cover 153 formed on the housing 150, thereby displaying the virtual image V.
[0080] The HUD device 100 projects and outputs the image onto the lower side of the windshield WS. Therefore, the driver DR, who is the viewer, can see the virtual image V superimposed on the scenery ahead from the lower area of the windshield WS. Therefore, the driver DR, who is the viewer, mainly sees the virtual image V displayed on the road surface near the vehicle CR. Furthermore, due to the configuration of the projection unit of the HUD device 100, the virtual image V is outputted at an angle such that the upper part of the output image is farther from the vehicle CR than the lower part.
[0081] (Operation of the embodiment) Fig. 4 is a flowchart showing the operation of the vehicle control system 1000 to which the HUD device 100 of the present embodiment is applied. Fig. 5 is an example of a display form in a reference state (the position of the eye box IB is in a default state) of the foreground viewed by the driver DR who is the viewer and the virtual image V displayed superimposed on the foreground. Fig. 6B is a diagram showing an example of a display form of the HUD device 100 of the present embodiment, where the left diagram of Fig. 6B shows a case where the position of the eye box IB is lowered, and the right diagram of Fig. 6B shows a case where the position of the eye box IB is higher.
[0082] Hereinafter, the operation of the vehicle control system 1000 to which the HUD device 100 of the present embodiment shown in FIG. 2 is applied will be described with reference to FIGS. 4, 5, and 6B.
[0083] 4, the navigation device 400 (information processing unit 401) constantly monitors whether or not an operation input is made through the operation input unit 500 (step S101). When it detects that the driver DR, who is the viewer, has input setting information (the position of the eye box IB) to indicate his / her intention to change the position of the eye box IB from the reference state (to lower or raise it) (step S101 "YES"), the navigation device 400 acquires the setting information (step S102). The setting information of the eye box IB is information required to generate a drive signal that controls the rotation of the concave mirror 141 of the HUD device 100, which changes in response to a change in the position of the eye box IB.
[0084] The driving unit 110 of the HUD device 100 controls the angle fixing position of the concave mirror 141 by a rotation driving mechanism using a stepping motor, and therefore, the angle range set for the concave mirror 141 can be set by a predetermined number of steps in the clockwise direction and counterclockwise direction around a reference position. To simplify the explanation, the positions at which the angle of the concave mirror 141 can be fixed (angle fixing positions) are set to the reference position (setting information "0"), a position when rotated clockwise around the reference position (setting information "+1"), and a position when rotated counterclockwise around the reference position (setting information "-1"), and the explanation will be given assuming that rotation can be controlled over an angle range equivalent to each step.
[0085] For example, as shown in Fig. 3A, which illustrates the cross-sectional structure of the HUD device 100 when the eye box IB is at the reference position, when the concave mirror 141 rotates clockwise from the reference position, the position of the eye box IB moves down, and when the concave mirror 141 rotates counterclockwise, the position of the eye box IB moves up. Fig. 3B illustrates the rotation of the concave mirror 141 when the position of the eye box IB moves down.
[0086] Next, the information processing unit 401 calculates the depression angle LD, which is the angle between a line VL parallel to the road surface based on the viewer's eye height and a line (dashed line in FIG. 1 ) connecting the viewer's eyes and the display area (virtual image formation surface) (step S103). The depression angle LD is a value uniquely determined by the position of the eye box IB. Next, the information processing unit 401 compares the depression angle LD with a threshold value LDt, which is a predetermined angle reference value (step S104). Here, the "predetermined angle reference value" is a value determined by the specifications of the HUD device 100 through verification or the like. For example, the graph in FIG. 9 shows the results of verification conducted by the inventors using a HUD device 100 with a field angle of 10° × 3° and an image plane distance of 10 m.
[0087] 9 shows a graph of the percentage of subjects who felt the annoyance was tolerable (acceptance rate) when the depression angle LD, which is the angle between a line VL parallel to the road surface and a line connecting the viewer's eyes and the display area (the dashed line in FIG. 1 ) based on the viewer's eye height, is changed. From this graph, it can be seen that the tolerance rate drops sharply when the depression angle LD falls below 1°. Therefore, for example, when performing a process to change the display position of the virtual image V downward, it can be inferred that the depression angle of the upper end of the virtual image V should be set deeper (higher) than 1° (a predetermined reference angle).
[0088] If the comparison and determination process in step S104 determines that the depression angle LD is equal to the threshold value LDt (step S104 “=”), in other words, if the driver DR, who is the viewer, sets the position of the eye box IB as the reference position (“0”) or if no setting is made (default state), the information processing unit 401 generates, for example, a perspective image VP imitating a lane Ln, generates a virtual image V by superimposing this perspective image VP on other parts that are also generated (such as speed limit signs and vehicle speed), and writes the virtual image V together with calculated position information in a predetermined area of the storage unit 403. Then, at a predetermined timing, the information is read from the storage unit 403 and transferred to the control unit 160 of the HUD device 100 via the CAN 600 (step S105).
[0089] Then, in the HUD device 100, the control unit 160 performs a process of drawing (arranging) a virtual image, which is generated and transferred by the navigation device 400 (information processing unit 401), in a corresponding position in the display area VA in accordance with display position information based on the reference state calculated by the information processing unit 401 (step S108). Specifically, the control unit 160 performs a process of writing the virtual image into a display memory (VRAM: Video RAM) area allocated to the built-in RAM and having at least the same capacity as the display area VA (virtual image formation surface).
[0090] Next, the control unit 160 waits for the timing to display these virtual images (step S109 "YES"), reads the virtual image V including the drawn perspective image VP from the VRAM area of the built-in memory, and displays it on the LCD display unit 120 (step S110).
[0091] FIG. 5 shows an example of the display form of the virtual image V when the eye box IB is in the reference position. Here, it is assumed that the vehicle CR is traveling on a highway or a motorway. The control unit 160 can, for example, arrange and display a virtual image in the display area VA (virtual image formation surface) in which a speed limit sign V1, a vehicle speed V2, an ADAS function notification display V3, etc. are superimposed on a perspective image VP simulating a lane Ln generated by the information processing unit 401 of the navigation device 400. Furthermore, for example, a perspective image VP with a sense of perspective reminiscent of an actual lane Ln can also be displayed to clearly indicate that the ACC has captured a leading vehicle or to notify the driver DR, who is the viewer, that the LKAS has not deviated from the lane Ln.
[0092] On the other hand, if the depression angle LD is different from a predetermined reference angle value (1°) set as the threshold LDt in the “comparison determination process” of step S104, for example, if it is determined that the depression angle LD is equal to or greater than the threshold (step S104 “>”), in other words, if the driver DR, who is the viewer, sets the position of the eye box IB lower via the operation input device 500 (setting information “+1”), the information processing unit 401 generates, for example, a perspective image VP simulating a lane Ln, generates a virtual image by superimposing this perspective image with other generated features (such as speed limit signs and vehicle speed), and writes the virtual image together with calculated position information to a predetermined area of the storage unit 403. Then, at a predetermined timing, the information is read from the storage unit 403 and transferred to the control unit 160 of the HUD device 100 (step S106).
[0093] The display position information calculated here is, for example, a position U12 obtained by moving the upper end U10 of the virtual image V displayed in the display area VA (virtual image formation surface) by a "predetermined amount a" toward the lower end of the display area VA (virtual image formation surface) when the position of the eye box IB is lowered, as shown in the left diagram of FIG. 6B as an example of a display form. Here, the "predetermined amount a" refers to a movement amount optimized for each angle (depression angle LD) formed by a line VL parallel to the road Rd and a line (dashed line in FIG. 1) connecting the driver DR's eyes and the display area VA (virtual image formation surface) based on the eye height of the driver DR, who is the viewer, and is a value determined, for example, from the verification results shown in FIG. 9.
[0094] Next, in the HUD device 100, the control unit 160 performs a process of drawing (arranging) the virtual image V, which is generated and transferred by the navigation device 400 (information processing unit 401), at a corresponding position in the display area VA (virtual image formation surface) based on the display position information calculated by the information processing unit 401. Specifically, the control unit 160 performs a process of writing the virtual image V into a display memory (VRAM: Video RAM) area allocated to the built-in RAM and having at least the same capacity as the display area VA.
[0095] 6B shows an example of the display form of the virtual image V when the eye box IB is below the reference position. As described above, the position of the upper end U10 of the virtual image V in the display area VA (virtual image formation surface) is moved by a predetermined amount a from position U11 when the virtual image V is at the reference position to position U12.
[0096] In this manner, the control unit 160 executes a virtual image transition process in which the position U11 of the upper end U10 of the virtual image V in the display area VA (virtual image formation surface) is moved downward by a predetermined amount a to U12, thereby displaying the virtual image V. In other words, the control unit 160 lowers the relative position U11 of the upper end U10 of the virtual image V in the display area VA (virtual image formation surface) toward the lower end (position U12). As a result, even if the display area VA is raised, the display position of the virtual image V (at least the position of the upper end U10 of the virtual image V) is lowered (display position U12), thereby reducing the annoyance felt by the driver DR, who is the viewer, when looking straight ahead. In other words, even if the viewer's eye position (height direction) changes due to height, controlling the display position of the virtual image V can improve visibility and reduce annoyance. In particular, when a representation with perspective that attracts visual attention is used, the annoyance felt by the driver DR, who is the viewer, can be reduced, resulting in a significant effect.
[0097] It has been described that the HUD device 100 of this embodiment executes a virtual image transition process in which the position of the upper end of the virtual image V allocated to the display area VA (virtual image formation surface) is moved downward by a predetermined amount a (the upper end position at which the virtual image V is displayed is designated as U12). However, as shown in FIG. 7 as an example of the display form, for example, a restricted area VA-S that is narrower than the display area VA may be fixedly allocated within the display area VA (virtual image formation surface) allocated to a virtual space, and a virtual image V (with other parts superimposed thereon, such as a speed limit sign V1, vehicle speed V2, and ADAS function notification V3) including a perspective image VP may be arranged and displayed in this restricted area VA-S.
[0098] As described above, the position of the upper end of the display area VA (virtual image formation surface) assigned as the restricted area VA-S needs to be determined in accordance with the specifications of the HUD device 100. In this case, too, the same effect as when virtual image transition processing is performed in which the position of the upper end of the virtual image V in the display area VA (virtual image formation surface) is moved downward by a predetermined amount a is obtained.
[0099] Furthermore, if the “comparison determination process” in step S104 determines that the depression angle LD is different from the threshold value LDt of 1°, for example, that the depression angle LD is less than the threshold value LDt (step S104 “<”), in other words, if the driver DR, who is the viewer, has set the position of the eye box IB to be higher (setting information “−1”), the information processing unit 401 generates a virtual image in which the perspective image VP of the lane Ln, etc. is superimposed on other parts (speed limit signs, vehicle speed, etc.), and transfers this image together with the display position information calculated here to the control unit of the HUD device 100 (step S107).
[0100] Then, in the HUD device 100, the control unit 160 performs a process of drawing a virtual image in which the perspective image VP generated by the navigation device 400 (information processing unit 401) and other parts (such as a speed limit sign or vehicle speed) are superimposed at a corresponding position in the display area VA based on the display position information also calculated by the information processing unit 401 (step S108). Specifically, the control unit 160 executes a drawing process of writing the perspective image VP, generated by the navigation device 400 (information processing unit 401), in a VRAM area (virtual image formation surface) allocated in the built-in memory of the control unit 160 corresponding to the display area VA, superimposed with other parts (virtual images), such as a speed limit sign or vehicle speed.
[0101] Next, the control unit 160 waits for the timing to display these virtual images (step S109 "YES"), reads the virtual images including the drawn perspective images from the VRAM area of the built-in memory, and displays them on the LCD display unit 120 (step S110).
[0102] (Other Embodiments) It has been described that the HUD device 100 of the present embodiment executes a virtual image transition process in which the display information (U11-U12)×a near the upper end of the virtual image V is hidden by changing the display position of the virtual image V including the perspective image VP. However, in exchange for this, the sense of depth provided by the perspective image VP and the superimposition of the road Rd in the foreground of the vehicle CR are no longer guaranteed. For example, as shown in FIG. 7 , when the perspective image VP is placed in the restricted area VA-S, the vertical component of the display area VA becomes shorter, making it difficult for the driver DR, who is the viewer, to perceive a sense of depth.
[0103] For this reason, it is conceivable that the information processing unit 401 of the navigation device 400 changes the degree to which the sense of perspective is imparted (execution level) when generating the perspective image VP depending on the angle of the concave mirror 141 adjusted by the drive unit 110 of the HUD device 100.
[0104] Incidentally, the perspective image VP generated by image processing that imparts a sense of perspective is an image that has been subjected to image processing that emphasizes the sense of perspective of the target image by, for example, changing the brightness, shading, color tone, etc. according to the display distance, the size of the target, etc., or by applying image processing such as shading, overlapping, blurring, etc. Here, the strength of this image processing is referred to as the degree (execution level) of imparting a sense of perspective when generating the perspective image VP.
[0105] Therefore, when the information processing unit 401 of the navigation device 400 executes the "virtual image generation process" of steps S105, S106, and S107 of FIG. 4, it generates perspective images VP (VP1, VP2, and VP3) with an enhanced sense of perspective according to the respective execution levels set in accordance with the angle of the concave mirror 141, and stores the images together with their display position information in the storage unit 403 in advance. The perspective image VP (one of VP1, VP2, and VP3) selected in accordance with the adjusted angle is then read from the storage unit 403 and transferred to the HUD device 100. The control unit 160 of the HUD device 100 then arranges and displays a virtual image including the transferred perspective image PV (such as speed limit signs and other parts superimposed on the perspective image VP, such as vehicle speed) in the display area VA (virtual image formation surface). The pre-generated perspective images VP1, VP2, and VP3 are all omitted from the illustration.
[0106] This ensures a sense of depth created by the displayed perspective image VP and superimposition with the road Rd in the foreground, thereby providing the driver DR with a more natural and perspective-oriented visibility. For example, as shown in an example of a display form in Fig. 8, by switching the display so that the perspective image VP is tilted inward (narrowing the width of the front end side of the lane Ln and widening the width of the rear end side), it is possible to achieve a more natural and perspective-oriented visibility of the virtual image compared to the display forms shown in Figs.
[0107] 1 and 2 , the head-up display device of this embodiment is a HUD device 100 that displays a virtual image V superimposed on a foreground in a display area VA (virtual image formation surface) virtually set in front of a vehicle CR, allowing the viewer to view the image. The HUD device 100 includes a drive unit 110 that can adjust an angle (depression angle LD) formed by a line VL parallel to the road surface based on the eye height of the viewer, for example, a driver DR, and a line (broken line in FIG. 1 ) connecting the viewer's eyes and the display area (virtual image formation surface), and a control unit 160 that controls the display of the virtual image V in the display area VA (virtual image formation surface). Then, for example, as shown in Figure 4, if the adjusted angle (depression angle LD) is different from a predetermined angle reference value (e.g., 1°) (steps S104 ``>'', S104 ``<''), the control unit 160 performs control to change the display position of the virtual image V in accordance with the adjusted angle (steps S106, S107).
[0108] According to the HUD device 100 of this embodiment, when the angle (depression angle LD) formed by a line VL parallel to the road surface based on the eye height of the viewer and a line (dashed line in FIG. 1 ) connecting the viewer's eyes and the display area VA (virtual image projection surface) differs from a predetermined reference angle value (e.g., 1°) adjusted by the adjustable drive unit 110, the control unit 160 performs control to change the display position of the virtual image V in accordance with the adjusted angle. Therefore, when the eye position (in the height direction) of the viewer, for example, the driver DR, changes, the display position of the virtual image V can be controlled, thereby eliminating the annoyance felt by the viewer, i.e., the driver.
[0109] Furthermore, according to the HUD device 100 of this embodiment, for example, as shown in FIG. 6B as an example of a display form, when the position of the eye box IB is lowered to the extent that the position of the upper end U10 of the virtual image V falls within the depression angle threshold LDt, the control unit 160 displays the generated virtual image V at a position U12, which is obtained by moving the position U11 of the upper end U10 of the virtual image V displayed within the display area VA (virtual image formation surface) by a predetermined amount a toward the lower end of the display area VA. In other words, the control unit 160 performs control. Therefore, for example, assuming that a short viewer will view the virtual image, the display position of the virtual image V displayed in the display area (virtual image formation surface) is higher, thereby reducing the annoyance felt by the viewer due to the virtual image being more likely to be in the viewer's line of sight when looking straight ahead. In other words, even if the viewer's eye position (height direction) changes depending on the viewer's height, controlling the display position of the virtual image V can reduce the annoyance felt by the viewer.
[0110] Furthermore, according to the HUD device 100 of this embodiment, a restricted area VA-S (see FIG. 7) is allocated that is smaller than the display area VA (virtual image formation surface) virtually set in front of the vehicle CR, and the generated virtual image V is arranged and displayed in that area. For example, assuming that a viewer of short stature will view the image, the position of the virtual image V displayed in the display area (virtual image formation surface) is higher, so the viewer, i.e., the driver DR, can view the virtual image displayed in the display area (virtual image formation surface) without feeling annoyed. In other words, even if the viewer's eye position (height direction) changes depending on the viewer's height, controlling the display position of the virtual image can reduce the annoyance felt by the viewer, i.e., the driver. In particular, the annoyance felt by the viewer when a representation using perspective processing that attracts visual attention can be reduced, resulting in significant benefits.
[0111] Furthermore, according to the HUD device 100 of this embodiment, the control unit 160 displays the perspective image VP by changing the degree to which a sense of perspective is imparted (the execution level of image processing for emphasizing the sense of perspective) in accordance with the angle adjusted by the drive unit 110. For example, perspective images VP (VP1, VP2, VP3) in which the sense of perspective is emphasized in accordance with the adjusted angle are prepared in advance, and the perspective image VP (one of VP1, VP2, VP3) selected in accordance with the adjusted angle is displayed. This ensures the sense of depth provided by the perspective image VP and the superimposition of the road surface in the foreground, thereby providing the viewer with more natural visibility with a sense of perspective.
[0112] Furthermore, according to the HUD device 100 of this embodiment, by using the display area VA (virtual image formation surface) which is an uphill slope where the upper end of the display area VA (virtual image formation surface) is farther away than the lower end, it is possible to perform a display with enhanced expressiveness in the depth direction (front-to-rear direction of the vehicle), thereby realizing a more natural visibility of the virtual image V with a sense of perspective. Furthermore, the virtual image display distance (imaging distance) can be adjusted depending on the display position, and a display with a sense of depth can be realized over a fairly wide range at the upper end, and in particular, it is possible to ensure the sense of depth due to the perspective expression and the superimposition with the road Rd in the foreground.
[0113] Furthermore, according to the HUD device 100 of this embodiment, the control unit 160 controls the display of a virtual image V, which is a perspective image VP capable of representing lanes along the road Rd as seen by the viewer, with a sense of perspective. For example, the lane width (width in the lateral direction) at the bottom of the lane Ln gradually narrows toward the top, thereby creating a sense of perspective. Furthermore, a sense of depth is realized over a fairly wide area at the top (see FIG. 8 ). Therefore, the HUD device 100 is particularly effective not only for displaying lanes Ln, but also for displaying navigation displays such as speed limit signs and arrows indicating vehicle speed or vehicle direction of travel as shown in FIG. 10( a), and for providing notifications using driving assistance technologies using ADAS as shown in FIG. 10( b), such as clearly indicating that a leading vehicle has been captured by ACC and clearly indicating whether the vehicle is deviating from the lane Ln using LKAS as shown in FIG. 10( c).
[0114] 1 and 2 , the display control system of this embodiment is a display control system (e.g., a vehicle control system 1000) having a head-up display device (HUD device 100) that displays a virtual image V superimposed on a foreground in a display area VA (virtual image formation surface) virtually set in front of the vehicle CR, allowing the viewer to view the image, and a processing device (e.g., an information processing unit 401 of a navigation device 400) that generates the virtual image V. As shown in FIGS. 3( a) and 3(b), the HUD device 100 has a display unit (liquid crystal display unit 120) that has a virtual image display surface 102a including an optical element (concave mirror 141), a drive unit 110 that rotates the optical element within a predetermined angle range, and a control unit 160 that controls the display of the virtual image V in the display area VA. Then, as shown in FIG. 4, for example, the information processing unit 401 acquires information regarding the angle range of the optical element, which is set by a predetermined operation by the viewer DR (steps S101 and S102), and, based on the acquired information, adjusts the angle (depression angle LD) between a straight line VL parallel to the road surface Rd based on the eye height of the viewer DR and a straight line (dashed line in FIG. 1) connecting the viewer DR's eyes and the display area VA (virtual image formation surface) (step S103: calculation of depression angle). If the adjusted angle differs from a predetermined angle reference value (threshold value LDt) (step S104 “>” or step S104 “<”), the information processing unit 401 performs control to change the display position of the virtual image V generated in accordance with the adjusted angle (step S106 or step S107), and instructs the head-up display device (control unit 160) to display the virtual image V at the changed display position.
[0115] According to the display control system (vehicle control system 1000) of this embodiment, even if the eye position (height direction) of the viewer DR changes due to height or the like, it is possible to provide a display control system that reduces annoyance to the viewer by controlling the display position of the virtual image V through a predetermined operation by the viewer DR. In particular, expressions with perspective that attract visual attention are expected to be used in various scenes, and are highly effective when the eye position (height direction) of the driver who is the viewer changes.
[0116] (Modification) According to the HUD device 100 of the above-described embodiment, the information processing unit 401 of the navigation device 400 receives setting information regarding the position of the eye box IB set by the driver DR, who is the viewer, via the operation input unit 500, and adjusts the angle of the optical member (concave mirror 141) by the drive unit 110 of the HUD device 100. However, the information processing unit 401 is not limited to the navigation device 400, and may be replaced by the driving assistance device 300, or the HUD device 100 may be independently replaced by itself. In this case, however, the control unit 160 of the HUD device 100 needs to have both a function of receiving setting input regarding the position of the eye box IB set by the driver DR, who is the viewer, and a function of not only displaying the virtual image V but also generating a perspective image VP and superimposing it on other parts to generate the virtual image.
[0117] Furthermore, according to the HUD device of this embodiment, the concave mirror 141, which is an optical element, has been described as having three fixed angular positions (steps). However, it is common for the concave mirror 141 to have three or more steps. For example, if there are 21 steps, corresponding to the rotation angle per step of the stepping motor used as the rotation drive mechanism for the concave mirror 141, the central position, which serves as the reference position, is set to the 11th step, and 10 steps are assigned in the clockwise direction and 10 steps in the counterclockwise direction, thereby enabling precise control of changes in the position of the eye box IB in accordance with the viewer's height.
[0118] In the display control system (vehicle control system 1000) of this embodiment, the functions executed by the information processing unit 401 of the navigation device 400 are realized by a program executed by a computer. In this case, the information processing unit 401 includes a computer including at least one control device (e.g., a processor) and at least one storage device (e.g., a RAM) as hardware for executing the above-mentioned program.
[0119] Furthermore, the above-mentioned program may be recorded on one or more computer-readable recording media, rather than being temporarily stored. The recording media may or may not be included in the information processing unit 401. In the latter case, the above-mentioned program may be supplied to the above-mentioned information processing unit 401 via any wired or wireless transmission medium.
[0120] At least some of the above-described functions can be realized by a logic circuit. For example, an integrated circuit formed with a logic circuit is also included in the scope of the present invention. Alternatively, the above-described functions can be realized by, for example, a quantum computer. Furthermore, each process described in this embodiment may be executed by AI (Artificial Intelligence). In this case, the AI may operate on the information processing unit 401, or may operate on another device (for example, a cloud server, etc.).
[0121] The present invention is not limited to the above-described exemplary embodiments, and those skilled in the art will be able to easily modify the above-described exemplary embodiments to the extent that they fall within the scope of the claims.
[0122] 100... Head-up display device (HUD), 110... Drive unit, 120... Liquid crystal display unit (display unit), 120a... Virtual image display surface, 121... Light source, 122... Liquid crystal display element, 141... Concave mirror, 160... Control unit, 200... In-vehicle sensing device, 300... Driving assistance device, 400... Navigation device, 500... Operation input device, 1000... Vehicle control system (display control system), IB... Eye box, VL... Straight line parallel to the road surface, LD... Depression angle, IP... Instrument panel, DR... Viewer (driver), CR... Vehicle, Rd... Road, Ln... Lane, WS... Windshield, VA... Display area (virtual image formation surface), VA-S... Restricted area, V (V1, V2, V3)... Virtual image, VP... Perspective image
Claims
1. A head-up display device that displays a virtual image superimposed on a foreground in a display area virtually set in front of a vehicle, allowing a viewer to view it, the head-up display device having: a drive unit that can adjust the angle between a line parallel to the road surface based on the eye height of the viewer and a line connecting the viewer's eyes and the display area; and a control unit that controls the display of the virtual image in the display area, wherein if the adjusted angle differs from a predetermined angle reference value, the control unit controls to change the display position of the virtual image in the display area in accordance with the adjusted angle.
2. A head-up display device as described in claim 1, wherein the control unit performs control to move the position of the upper end of the virtual image by a predetermined amount toward the lower end of the display area and display the virtual image.
3. A head-up display device as described in claim 1, wherein the control unit performs control to move the position of the bottom end of the virtual image by a predetermined amount toward the top end of the display area and position and display the virtual image.
4. The head-up display device of claim 1, wherein the control unit assigns to the display area a restricted area an area in which the angle between a straight line parallel to the road surface based on the eye height of the viewer and a straight line connecting the viewer's eyes and the display area is higher than a predetermined angle reference value, and controls the placement and display of the virtual image in the restricted area.
5. A head-up display device as described in claim 1, wherein the control unit changes the degree of perspective depending on the adjusted angle, and performs control to position and display the virtual image depending on the degree of change.
6. A head-up display device as described in claim 1, wherein the control unit performs control to position and display the virtual image so that, when the end of the display area closer to the vehicle is defined as the bottom end and the end farther from the vehicle is defined as the top end, the virtual image is perceived by the viewer as being tilted so that the top end is farther away than the bottom end.
7. A head-up display device as claimed in any one of claims 1 to 6, wherein the control unit performs control to arrange and display the virtual image, which is a perspective image capable of expressing lanes along a road with a sense of distance as seen by a viewer.
8. A display control system having a head-up display device that displays a virtual image superimposed on a foreground in a display area virtually set in front of a vehicle, allowing a viewer to view the image, and a processing device that generates the virtual image, wherein the head-up display device has: a display unit having a virtual image display surface including an optical member; a drive unit that rotates the optical member within a predetermined angle range; and a control unit that controls the display of the virtual image in the display area, and the processing device takes in information regarding an angle range of the optical member set by a predetermined operation of the viewer, and enables adjustment of an angle between a straight line parallel to the road surface and a straight line connecting the viewer's eyes and the display area based on the eye height of the viewer according to the taken-in information, and when the adjusted angle differs from a predetermined angle reference value, performs control to change the display position of the virtual image generated in accordance with the adjusted angle, and instructs the head-up display device to display the virtual image at the changed display position of the display area.
Citation Information
Patent Citations
Display device, display control method, and program
JP2020050162A
Display control device, display device, vehicle, display control method and display control program
JP2022173686A
