vehicle

Abstract

To provide a technique of a head-up display device, which can achieve more preferable usability than before, so as to contribute to "3. Good Health and Well-Being" of sustainable development objectives.SOLUTION: A vehicle has a head-up display device (HUD device) 1 including: a housing 102 storing a light source unit for generating image light, a main display unit, and an optical system; a sub display unit (sub LCD2) arranged outside the housing 102; and a control unit for controlling display of image information on the main display unit and display of the image information on the sub display unit.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to the technology of a head-up display device (which may be described as a Head Up Display: HUD).

Background Art

[0002] For example, International Publication No. 2018 / 229961 (Patent Document 1) describes a light source device that is small, lightweight, has a high light utilization rate, is modular, and can be easily used as a planar light source, and a head-up display device equipped with the same.

[0003] A head-up display device such as that in Patent Document 1 can display various information such as driving information like vehicle speed and engine speed and navigation information as virtual images on a vehicle's windshield (or, in other words, the front glass) when used in a vehicle. By using a HUD, a driver can obtain the information necessary for driving without having to move their line of sight to an instrument panel incorporated in the dashboard. Therefore, HUDs contribute to the safe driving of automobiles and the like.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] Conventional head-up display (HUD) devices project image light, generated based on an image displayed on an image display device (display panel) using light from a light source, onto a predetermined area (sometimes referred to as a display area) such as a windshield, thereby providing the driver with a virtual image. This virtual image enables functions such as augmented reality (AR), which superimposes various image information onto the real scene. Head-up display devices with such functions are also called AR-HUDs.

[0006] The virtual image in a conventional head-up display device, such as in Patent Document 1, is directional. That is, the driver can only suitably view the virtual image when viewing the display area of ​​the windshield from a predetermined viewpoint position (including the eye box). Because the virtual image in the display area of ​​an AR-HUD has these characteristics, it can be difficult for users such as drivers to see, and there is room for improvement in terms of usability.

[0007] The objective of the present invention is to provide a technology that enables more favorable usability and other improvements in head-up display devices. [Means for solving the problem]

[0008] A typical embodiment of this disclosure has the following configuration. The vehicle of the embodiment is a vehicle that displays a virtual image in a display area based on the projection of image light, and comprises a housing that houses a light source device for generating the image light, a main display device, and an optical system, a sub-display device provided on the outside of the housing, and a control device that controls the display of image information on the main display device and the display of image information on the sub-display device. [Effects of the Invention]

[0009] According to representative embodiments of this disclosure, more favorable usability and other aspects can be achieved with respect to the technology of head-up display devices. Other issues, configurations, and effects will be shown in the embodiments for carrying out the invention. [Brief explanation of the drawing]

[0010] [Figure 1] This image shows how a HUD device according to one embodiment is used by a driver inside a vehicle. [Figure 2] A vehicle equipped with a HUD device according to one embodiment, and an example of the configuration of the HUD device are shown. [Figure 3] This shows a vehicle equipped with a HUD device according to one embodiment, and an example of the internal configuration of the HUD device. [Figure 4] This shows an example of the external appearance configuration of a HUD device as hardware according to one embodiment. [Figure 5A] An example of the functional block configuration of a HUD device according to one embodiment is shown. [Figure 5B] This shows an example of the functional block configuration of the MCU411 of a HUD device according to one embodiment. [Figure 6A] This shows an example of a functional block configuration for controlling two types of displays in a HUD device according to one embodiment. [Figure 6B] This shows an example of a functional block configuration for controlling two types of displays in a modified HUD device. [Figure 6C] This shows an example of a functional block configuration for controlling two types of displays in a modified HUD device. [Figure 7] This shows the startup operation flow of a HUD device according to one embodiment. [Figure 8] This shows an example of the configuration of a video display device in a HUD device according to one embodiment. [Figure 9] This shows two display examples in a HUD device according to one embodiment. [Figure 10] This shows a processing flow example of display control for two types of displays in a HUD device according to one embodiment, taking into account their type and amount of information. [Figure 11] This document shows two display examples for a HUD device according to one embodiment, taking into account the type and amount of information displayed on each display. [Figure 12]Shows display examples of two types of displays when adjusting initial settings in the HUD device of an embodiment. [Figure 13] Shows the processing flow at the time of initial settings in the HUD device of an embodiment. [Figure 14] Shows other display examples when adjusting initial settings in the HUD device of an embodiment. [Figure 15] Shows a display example in an error state in the HUD device of an embodiment. [Figure 16] Shows the processing flow in an error state in the HUD device of an embodiment. [Figure 17] Shows display examples according to scenes in the HUD device of an embodiment. [Figure 18] Shows display examples related to user settings of display methods for two types of displays in the HUD device of an embodiment. [Figure 19] Shows other display examples related to user settings of display methods for two types of displays in the HUD device of an embodiment. [Figure 20] Shows display examples of luminance change control according to scenes in the HUD device of an embodiment. [Figure 21] Shows an explanatory diagram related to distortion correction in the HUD device of an embodiment. [Figure 22] Shows an explanatory diagram related to pitching correction in the HUD device of an embodiment. [Figure 23] Shows a display example in debug mode in the HUD device of an embodiment. [Figure 24] Shows an explanatory diagram related to double display in the HUD device of an embodiment. [Figure 25] Shows the processing flow related to double display in the HUD device of an embodiment. [Figure 26] Shows an explanatory diagram related to the first example of destination change control in the HUD device of an embodiment. [Figure 27] Shows an explanatory diagram related to the second example of destination change control in the HUD device of an embodiment. [Figure 28] This diagram illustrates a third example of destination change control in a HUD device according to one embodiment. [Figure 29] This shows the processing flow for destination change control in a HUD device according to one embodiment. [Modes for carrying out the invention]

[0011] The embodiments of this disclosure will be described in detail below with reference to the drawings. In the drawings, the same parts are generally denoted by the same reference numerals, and repeated descriptions are omitted. In the drawings, the representation of components may not show their actual location, size, shape, and extent in order to facilitate understanding of the invention.

[0012] In explanations, when describing program-based processing, the focus may sometimes be on the program, functions, or processing units. However, the core hardware component is the processor, or a controller, device, computer, or system composed of such a processor. A computer, using its processor, executes processing according to a program read into memory, utilizing resources such as memory and communication interfaces as appropriate. This realizes the specified functions and processing units. A processor is composed of semiconductor devices such as a CPU or GPU. A processor is composed of devices and circuits capable of performing specified calculations. Processing is not limited to software program processing; it can also be implemented using dedicated circuits. FPGAs, ASICs, CPLDs, etc., can be used as dedicated circuits.

[0013] The program may be pre-installed as data on the target computer, or it may be distributed as data to the target computer from the program source. The program source may be a program distribution server on a communication network, or it may be a non-transient computer-readable storage medium (e.g., a memory card). The program may consist of multiple modules. Various types of data and information may be, but are not limited to, structures such as tables and lists. Representations such as identification information, identifiers, IDs, names, and numbers are interchangeable.

[0014] <Embodiment 1> Using Figures 1 to 29, an embodiment of the head-up display device of Embodiment 1 will be described as one embodiment of the present disclosure. For the purposes of this description, symbols such as (X, Y, Z) in Figure 1 may be used as appropriate to represent directions and coordinate systems. The Z direction is the vertical direction, and the X and Y directions are two orthogonal directions that constitute the horizontal plane. The X direction is the horizontal direction in the display area 105 and sub-LCD 2, and the left-right direction as seen from the driver. The Z direction is the vertical direction in the display area 105 and sub-LCD 2, and the up-down direction as seen from the driver. The Y direction is the front-back direction as seen from the driver.

[0015] The HUD device 1 of Embodiment 1 shown in Figure 1, etc., is a so-called AR-HUD that projects video light from an image display device 403 inside a housing 102 (as shown in Figure 2, etc.) onto a display area 105 of the windshield 103 of a vehicle 100 through an optical system such as a mirror 3a, thereby forming and displaying a virtual image 106 (and its corresponding video) that corresponds to augmented reality (AR) to the driver's viewpoint 108.

[0016] In Embodiment 1, the HUD device 1 has a display area 105 where a virtual image 106 is formed based on an image display device 403 (particularly the display panel 4) as the main display, and a sub-LCD 2 is provided as a sub-display in the housing 102. In Embodiment 1, the main display is the main display screen, and the sub-display or sub-LCD 2 is a sub-display device. The HUD device 1 in Embodiment 1 assists the driver's driving by displaying a virtual image using AR-HUD and displaying video using the sub-display.

[0017] The HUD device 1 of Embodiment 1 has a structure and installation features that include a sub-LCD 2 having a physical display screen 20 as a sub-display in the housing 102. The HUD device 1 also has features such as a circuit that drives and controls both the AR-HUD and the sub-display. Furthermore, the HUD device 1 has features in the software that controls the switching between the two types of displays, the AR-HUD and the sub-display.

[0018] [Usage example] Figure 1 shows an example of the installation and usage image of the HUD device 1 of Embodiment 1 inside a vehicle. The vehicle in Figure 1 is an example of a vehicle with a steering wheel 104 on the left side, and most of the housing 102 of the HUD device 1 is housed within the dashboard 101. A portion of the housing 102, including an opening (the convex portion in Figure 2, housing portion 102B in Figure 4), protrudes upward from the dashboard 101, and a sub-LCD 2 (LCD: liquid crystal display), which is a sub-display (in other words, a sub-display device), is provided in this portion of the housing 102. The sub-LCD 2 is positioned on the dashboard 101, in front of the steering wheel 104, with the display screen 20 facing the driver. The display area 105 (in other words, the HUD area) is the area on the vehicle's windshield 103 where the virtual image 106 is formed and displayed. The virtual image 106 is displayed within the display area 105 so that it can be seen from the driver's perspective.

[0019] The sub-LCD2 is mounted on the housing 102 of the HUD device 1. In other words, the housing 102 of the HUD device 1 also serves as the housing for the sub-LCD2. Conventional HUD devices have their entire housing housed within the dashboard 102. In contrast, in this HUD device 1, part of the housing 102 is housed within the dashboard 102, while the other part extends outside the dashboard 102 to house the sub-LCD2 (Figures 2 to 4). The installation position of this HUD device 1 is almost the same as conventional models, satisfying the condition that the virtual image 106 of the display area 105 is correctly visible from the driver's viewpoint (including the eye box). The installation position of the sub-LCD2 is oriented directly towards the driver and is located near the bottom edge of the display area 105.

[0020] In the AR-HUD display area 105, AR images, such as virtual images 106, are displayed as needed, superimposed on the real-world scenery. On the sub-LCD2, information to assist driving is displayed as a real image, for example, at all times. Examples of AR images include various types such as icons for warnings such as collision avoidance, icons for paying attention to other vehicles or people, and arrow images for navigation (route guidance). Examples of information displayed on the sub-LCD2 include various types such as vehicle speed, right / left turn information, driving guidance, and character images.

[0021] Furthermore, the projection target for the image light used to form the display area 105 of the virtual image 106 is not limited to the windshield 103, but may be other objects, such as a combiner (projection plate).

[0022] [Vehicle and HUD device (1)] Figure 2 shows an example of mounting the HUD device 1 in a vehicle 100, and an example of the configuration of the HUD device 1. In Figure 2, the HUD device 1 has an image display device 403 (Figure 5A, described later) and optical systems such as mirrors 3a and 3b arranged and fixed in a predetermined positional relationship within a housing 102. As shown in Figure 2, the image display device 403 is mounted inside the housing 102, or it may be mounted on a part of the outer perimeter of the housing 102. In Figure 2, the housing 102 is convex as an example of its shape, with a part of the convex shape protruding above the dashboard 101. On the outside of the housing 102, a sub-LCD 2 is fixed on the front side (the side facing the driver) of the convex part that protrudes upward from the dashboard 101. The image light from the image display device 403 is reflected by the mirrors 3a and 3b and emitted outside the housing 102 through an opening provided in the convex part. The image light is reflected in the display area 105 and directed towards the driver's viewpoint 108, forming a virtual image 106 from the driver's perspective.

[0023] [Vehicle and HUD device (2)] Figure 3 shows an example of the HUD device 1 mounted on a vehicle 100, as well as an overview of the internal configuration of the HUD device 1, as a detailed configuration example of Figure 2. The HUD device 1 has a light source device 5, a display panel 4, and optical systems such as mirrors 3a and 3b arranged and fixed in a predetermined positional relationship within a housing 102. Mirror 3b is composed of, for example, a free-form surface mirror or a mirror with an asymmetric shape along the optical axis, and in this case, it is a reflective mirror. In the example of Figure 3, the housing 102 is roughly rectangular in shape (Figure 4, described later). On the outside of the housing 102, a sub-LCD 2 is fixed on the front side (facing the driver) in the Y direction of the housing portion 102B (Figure 4) that extends upward from the dashboard 101. The light source device 5 and the display panel 4 constitute the video display device 403 in Figure 5A. The video display device 403 (light source device 5 and display panel 4) and optical systems such as mirrors 3a and 3b constitute the video display unit 402 in Figure 5A. The image display device 403 is a projection-type image display device that projects an image formed on the display panel 4 using light emitted from the light source device 5 (in other words, light from the light source).

[0024] The light source device 5 is composed of a semiconductor light source element, typically an LED (Light Emitting Diode) light source. The display panel 4 is typically a liquid crystal display (LCD). The display panel 4 emits image light based on the light from the light source device 5. The display panel 4 creates an image based on image information from the control device (Figure 5A) and displays it on the display screen of the display panel 4. The display panel 4 modulates the transmittance of the light from the light source device 5 for each pixel according to the image information, thereby forming an image for projection onto the display area 105 and emitting it as image light.

[0025] The image light from the display panel 4 exits through an optical system including a mirror 3a and a reflective mirror 3b, and exits from the upper opening of the housing 102. Mirror 3a reflects the image light from the display panel 4 toward the reflective mirror 3b. The reflective mirror 3b is, for example, a concave mirror. The reflective mirror 3b magnifies and reflects the image light from mirror 3a toward the opening of the housing 102 at a set angle. The image light exiting from the opening of the housing 102 is reflected by the surface of the display area 105 of the windshield 103 and directed toward the driver's line of sight 108.

[0026] As a result, when the driver looks forward from their viewpoint 108 (the viewpoint 108 within a predetermined eye box), they can see in the display area 105 a virtual image 106 formed by video light superimposed on the real scene in front of the windshield 103 (e.g., road, vehicles, people, etc.). The virtual image 106 can be video information that is aligned and superimposed on real objects, or video information that is displayed independently of real objects. The video information that becomes the virtual image 106 can be various, such as vehicle speed information, warnings, cautions, and route guidance information. The virtual image 106 enables functions such as augmented reality (AR).

[0027] In the example configuration of the video display unit 402 shown in Figure 5A, a drive mechanism 404, such as a motor, is provided on the reflective mirror 3b. The drive mechanism 404 allows adjustment of the angle of the reflective mirror 3b (Figure 3). This allows adjustment of the direction in which the video light is projected from the reflective mirror 3b onto the display area 105 of the windshield 103. In other words, the position of the display area 105 where the virtual image 106 is formed can be adjusted. The drive mechanism 404 changes the angle of the reflective mirror 3b based on control from the control device 401 or based on manual operation by the user. This allows for appropriate adjustment of the height position of the display area 105 as seen by the driver during the initial setup described later.

[0028] [Cabinet] Figure 4 shows an example of the configuration of the sub-LCD2 in the housing 102 of the HUD device 1. Figure 4(A) shows the state when the HUD device 1 is in use, and (B) shows the state when it is not in use. The housing 102 has a housing section 102A that is housed inside the dashboard 101, and a housing section 102B that extends outside the dashboard 101 above it. The housing section 102B has an opening 107 on its upper surface. The opening 107 is the part through which video light is transmitted toward the display area 105, and a glare trap or the like is provided therein. The sub-LCD2 is mounted on the front surface of the housing section 102B via a mounting device or the like (not shown). As a modification, the sub-LCD2 may be mounted so that it is embedded in the housing section 102B.

[0029] In this example, the sub-LCD2 is a widescreen LCD larger than the width of the housing 102B, and has a widescreen display screen 20. The display screen 20 of this widescreen sub-LCD2 is designed to be positioned near the bottom edge of the AR-HUD display area 105, as shown in Figure 1. The sub-LCD2 is connected to the control device (Figure 5A, described later) inside the housing 102 via wiring. The top edge of the sub-LCD2 may be at approximately the same position as the top surface of the housing 102B, or it may extend above that top surface. The sub-LCD2 should be positioned such that, from the driver's perspective, it obscures the opening 107 of the housing 102B. When in use, the driver mainly sees the sub-LCD2, so the housing 102B protruding from the dashboard 101 is not conspicuous. As a variation, the width of the sub-LCD2 may be less than or equal to the width of the housing 102B.

[0030] This HUD device 1 is not configured to house the entire housing 102 inside the dashboard 101, but rather has a configuration in which a portion of the housing 102B extends outside the dashboard 101. Therefore, the volume of the housing 102B inside the dashboard 101 can be reduced, making it easier to install inside the dashboard 101 even when the space inside the dashboard 101 is relatively narrow. As a result, this HUD device 1 can be easily installed in various types of vehicles.

[0031] Furthermore, since the housing section 102B may extend outside the dashboard 101, the structural flexibility of the housing section 102, such as its dimensions and shape, is increased. HUD devices generally require that an optical distance for forming a virtual image be secured within the housing. For this reason, optical systems such as reflective mirrors are provided inside the housing. With this HUD device 1, there is no need to forcibly miniaturize the housing section 102A, so it may be possible to reduce the optical systems (optical components such as reflective mirrors) used to increase the distance of the optical path inside the housing section 102. Alternatively, even if the housing section 102A has the same volume as conventional devices, with this HUD device 1, the optical system inside the housing section 102 can secure a longer optical path due to the housing section 102B extending outside the dashboard 101, enabling virtual image display at a greater distance.

[0032] In Figure 4(B), the HUD device 1 can also be used as a shutter to conceal the opening 107 by changing the position of the sub-LCD 2 using a predetermined mechanism when not in use. In this example, a hinge portion 102C is provided between the vicinity of the top edge of the sub-LCD 2 and one front edge of the top surface of the housing portion 102B, and the sub-LCD 2 has a rotation mechanism that allows it to rotate around the hinge portion 102C as the axis of rotation. When not in use, as shown in the figure, the sub-LCD 2 is rotated so that the display screen 20 of the sub-LCD 2 faces the opening 107 of the housing portion 102B. As a result, the opening 107 is concealed by the sub-LCD 2, and the back side of the sub-LCD 2 is positioned as the top surface. For example, a film for suppressing external light reflection (in other words, a light-shielding film) may be formed on the back of the sub-LCD 2. As a result, when not in use, the sub-LCD 2 can be used as a shutter to prevent or reduce internal panel burning and dust accumulation caused by the incidence of external light such as sunlight into the opening 107.

[0033] Furthermore, when not in use, it is undesirable for the display screen 20 of the sub-LCD2 to come into contact with the opening 107, as this could potentially damage the components. Therefore, the rotation mechanism should either be configured to stop at a position just before contact occurs, or a spacer should be provided between the opening 107 and the display screen 20 of the sub-LCD2. For example, spacers can be provided on the upper surface of the housing 102B near the outside of the four corners of the opening 102. When not in use, a portion of the display screen 20 of the sub-LCD2 is supported by these spacers, preventing contact between the opening 107 and the display screen 20.

[0034] Furthermore, during use, the orientation of the display screen 20 of the sub-LCD2 may be adjusted to suit the driver by changing the rotation angle using a mechanism such as the hinge part 102C.

[0035] The movement of the sub-LCD2 as described above may be controlled manually by the user, or a drive mechanism such as a motor may be provided so that the HUD device 1 automatically moves the sub-LCD2 when it starts up or shuts down. For example, when the HUD device 1 starts up, the sub-LCD2 is moved from state (B) to state (A) by the drive mechanism. When the HUD device 1 shuts down, the sub-LCD2 is moved from state (A) to state (B) by the drive mechanism.

[0036] Furthermore, even if the entire opening 107 is not covered by the sub-LCD2 in (B), at least the portion that is covered will still have the effect of preventing external light from entering.

[0037] The configuration in which the opening 107 can be hidden by the sub-LCD2 is not limited to the mechanism described in (B) above. As a variation, the housing 102 may be provided with a mechanism that allows the sub-LCD2 to slide, for example, by translating it in the Y direction. Another variation is that the sub-LCD2 itself may be composed of a foldable display.

[0038] If a mechanism is provided that allows the opening 107 to be covered by the sub-LCD2 as described in (B) above, it is possible to reduce the number of mechanisms provided inside the housing 102 for preventing external light from entering, such as a mechanism for rotating a reflective mirror.

[0039] Figure 4(C) shows a modified example of the housing 102. A sliding mechanism 102D to which a sub-LCD 2 is attached is provided on the front surface of the housing 102B. The sliding mechanism 102D allows the sub-LCD 2 to be slid, for example, in the Z direction. This makes it possible to adjust the height position of the sub-LCD 2. As another modification, a rotation angle adjustment mechanism may be added to allow the orientation of the sub-LCD 2 (in the optical axis direction of the display screen 20) to be changed.

[0040] [Example of functional block configuration for HUD device (1)] Figure 5A shows an example of the functional block configuration of the HUD device 1. The HUD device 1 includes a control device 401, a video display unit 402, a sub-LCD 2, a speaker 405, a sensor 409, etc. The control device 401 is composed of, for example, an electronic control unit (ECU). The video display unit 402 has a configuration as shown in Figure 2 or Figure 3 above.

[0041] The control device 401 is equivalent to a controller that controls the entire HUD device 1 and its various parts, and mainly performs video display control and audio output control of the HUD device 1. The control device 401 is composed of, for example, a wiring board. The control device 401 is mounted in the housing 102, for example, as shown in Figure 2 or Figure 3. Note that the control device 401 is not limited to being mounted inside the housing 102, but may also be mounted outside the housing 102. The control device 401 includes a vehicle information acquisition unit 407, a communication unit 408, a sensor 409 (which may be located outside the control device 401 or outside the housing 102), a microcontroller (MCU) 411, a non-volatile memory 211, a volatile memory 212, a display driver 223, an audio driver 324, etc.

[0042] The HUD device 1 acquires vehicle information 406 from various sensors (in other words, information acquisition devices) installed in various parts of the vehicle 100 (Figure 2) via the vehicle information acquisition unit 407. The various sensors periodically detect parameters related to conditions such as driving conditions inside and outside the vehicle 100. An example of various sensors is the camera 109 (external camera and internal camera) installed on the vehicle 100 shown in Figure 3. The HUD device 1 acquires detection information from the various sensors via the vehicle information acquisition unit 407 and can detect and judge various events related to the vehicle 100 based on the detection information. The HUD device 1 may also acquire detection information from sensors 409 installed on the HUD device 1. The HUD device 1 may detect and judge the state of the HUD device 1 and the state of the vicinity of the HUD device 1 based on the detection information from sensors 409.

[0043] The vehicle information acquisition unit 407 acquires vehicle information 406 based on a communication protocol that supports, for example, a CAN (Controller Area Network) interface or a LIN (Local Interconnect Network) interface.

[0044] Vehicle information 406 is a general term for information related to the driving status of vehicle 100. Vehicle information 406 includes, for example, vehicle 100's speed information, gear information, steering angle information, lamp illumination information, ambient light information, distance information, infrared information, engine ON / OFF information, camera image information, acceleration gyro information, GPS (Global Positioning System) information, navigation information, vehicle-to-vehicle communication information, and vehicle-to-infrastructure communication information. Camera image information is image information from camera 109, etc., and includes in-vehicle camera image information and exterior camera image information. GPS information includes current time information in addition to latitude and longitude.

[0045] Examples of various sensors (information acquisition devices) installed in the vehicle 100 or HUD device 1 are shown below. Examples of various sensors (information acquisition devices) include a vehicle speed sensor, shift position sensor, steering angle sensor, headlight sensor, illuminance sensor, chromaticity sensor, distance sensor, infrared sensor, engine start sensor, acceleration sensor, gyro sensor, temperature sensor, in-vehicle camera, out-of-vehicle camera, wireless transceiver for vehicle-to-infrastructure communication, wireless transceiver for vehicle-to-vehicle communication, GPS receiver, VICS (Vehicle Information and Communication System, registered trademark) receiver, etc. The various sensors are not limited to these and can be added, deleted, or replaced.

[0046] The vehicle speed sensor detects the speed of the vehicle (also referred to as vehicle speed) and generates speed information as the detection result. The shift position sensor detects the current gear and generates gear information as the detection result. The steering angle sensor detects the current steering angle and generates steering angle information as the detection result. The headlight sensor detects whether the headlights are ON or OFF and generates lamp illumination information as the detection result. The illuminance sensor and chromaticity sensor detect ambient light and generate ambient light information as the detection result.

[0047] The distance sensor detects the distance between the vehicle 100 and an external object and generates distance information as the detection result. The infrared sensor detects the presence and distance of objects in the vicinity of the vehicle 100 and generates infrared information as the detection result. The engine start sensor detects whether the engine is ON or OFF and generates ON / OFF information as the detection result. The acceleration sensor and gyro sensor detect the acceleration and angular velocity of the vehicle 100 and generate acceleration-gyro information representing the attitude and behavior of the vehicle 100 as the detection result. The temperature sensor detects the temperature inside and outside the vehicle and generates temperature information as the detection result.

[0048] An in-vehicle camera generates in-vehicle camera video information by capturing images inside the vehicle 100. An external camera generates external camera video information by capturing images outside the vehicle 100. In a specific example, camera 109 in Figure 3 is an in-vehicle camera that captures, for example, the driver's posture, eye position, and movement, thus constituting a Driver Monitoring System (DMS). By analyzing the images captured by the in-vehicle camera, it is possible to understand the driver's fatigue level and gaze position. An external camera captures, for example, the surrounding environment of the vehicle 100, such as in front of and behind it. By analyzing the images captured by the external camera, it is possible to understand the presence or absence of other vehicles and people around the vehicle 100, buildings and terrain, road surface conditions such as rain, snow, ice, and unevenness, and road signs. External cameras also include drive recorders that record driving conditions in video.

[0049] A vehicle-to-infrastructure (VICS) wireless transceiver generates vehicle-to-infrastructure communication information through vehicle-to-infrastructure communication between the vehicle 100 and roads, signs, signals, etc. A vehicle-to-vehicle (VV) wireless transceiver generates vehicle-to-vehicle communication information through vehicle-to-vehicle communication between the vehicle 100 and other surrounding vehicles. A GPS receiver generates GPS information by receiving GPS signals from GPS satellites. For example, the current time, latitude, and longitude can be obtained as GPS information. A VICS receiver generates VICS information obtained by receiving VICS signals. The GPS receiver and VICS receiver may be provided as part of a navigation system.

[0050] The control device 401 of the HUD device 1 controls the display of the video display unit 402 based on vehicle information 406 and other information, and forms a virtual image 106 using video light in the display area 105. The control device 401 of the HUD device 1 also controls the display of the sub-LCD2 based on vehicle information 406 and other information.

[0051] The MCU411 includes a processor such as a CPU (Central Processing Unit), memory, and various peripheral functions. The MCU411 within the control device 401 may also have the configuration shown in Figure 5B. In Figure 5B, the functional blocks implemented by the MCU411 include a video data generation unit 412, a distortion correction unit 413, a pitching correction unit 414, a light source adjustment unit 415, a mirror adjustment unit 416, and an audio data generation unit 417. Each of these units is primarily implemented by the MCU411's CPU reading and executing programs stored in non-volatile memory 421 or volatile memory 422. The control device 401 is not limited to implementations using the MCU411; it may also be implemented using an ECU or other semiconductor devices.

[0052] The MCU411 receives and acquires vehicle information 406 via the vehicle information acquisition unit 407. Based on the vehicle information 406 and other information, the MCU411 generates video information (video data) for the video display device 403 and audio data for the speaker 405.

[0053] In the configuration shown in Figure 5B, the video data generation unit 412 generates video data (original image, described later) that determines the content of the video to be displayed on the display panel 4 in order to form a virtual image 106 in the display area 105, based on vehicle information 406 and the like. The distortion correction unit 413 performs distortion correction processing, described later, on the video data from the video data generation unit 412 to generate corrected video data. Distortion correction corrects the distortion of the video that occurs according to the curvature of the windshield 103 when the video light from the video display device 403 is projected onto the windshield 103. In addition, the pitching correction unit 414 performs pitching correction processing, described later, on the video data from the video data generation unit 412 to generate corrected video data. In Embodiment 1, both distortion correction and pitching correction are performed on the video data.

[0054] The display driver 423 drives the display elements of the display panel 4 of the video display device 403 based on the corrected video data after distortion correction and pitch correction. As a result, the display panel 4 creates and displays an image on the display screen for projection onto the display area 105.

[0055] The light source adjustment unit 415 controls the brightness of the LED elements of the light source device 5 of the video display device 403, as described later.

[0056] When adjusting the position of the display area 105, the mirror adjustment unit 416 controls the drive mechanism 404 provided on the reflective mirror 3b of the video display unit 403 to change the angle of the reflective mirror 3b.

[0057] The voice data generation unit 417 generates voice data based on vehicle information 406 and other information as needed. Voice data is generated, for example, when providing voice guidance for the navigation system or when issuing warnings to the driver using the AR function. The voice driver 424 drives the speaker 405 based on the voice data and causes the speaker 405 to output sound.

[0058] In Figure 5A, the non-volatile memory 421 primarily stores programs executed by the CPU within the MCU 411, setting parameters used in the processing of various parts within the MCU 411, and predefined video and audio data. The volatile memory 422 primarily stores acquired vehicle information 406 and various data used in the processing of various parts within the MCU 411 as needed.

[0059] The communication unit 408 is a device equipped with a communication interface and communicates with the outside of the HUD device 1 based on a communication protocol such as CAN or LIN. The communication unit 408 and the vehicle information acquisition unit 407 may be integrated into one unit. Each part of the control device 401 in Figure 5A may be implemented using dedicated circuits such as FPGA (Field Programmable Gate Array).

[0060] [Example of functional block configuration for HUD device (2)] Figure 6A shows an example of a functional block configuration for controlling two types of displays (AR-HUD and sub-LCD2) in the HUD device 1, based on Figure 5A. In the configuration example of Figure 6A of Embodiment 1, the control device 401 has one MCU 411, and this one MCU 411 controls both the display panel 4 and the sub-LCD2. The display panel 4 is a main display device composed of, for example, an LCD, and the sub-LCD2 is a sub-display device.

[0061] Figure 6B shows a modified configuration of Figure 6A, in which the control device 401 has one control microcontroller 503, and this single control microcontroller 503 controls both the display panel 4 and the sub-LCD2.

[0062] The MCU411 in Figure 6A (and similarly in the case of the control microcontroller 503 in Figure 6B) has, as functional blocks, a display control unit 500, a first video data generation unit 501, a second video data generation unit 502, etc. The video control unit 500 determines how to display the video data to be displayed, which is generated based on vehicle information 406, etc., including two types of display destinations (for example, Figure 10 described later). Based on this determination, the video control unit 500 controls the first video data generation unit 501 and the second video data generation unit 502, etc.

[0063] The first video data generation unit 501 generates first video data for display on the display panel 4 when the display destination is the AR-HUD, and drives and controls the first display driver 511. The second video data generation unit 502 generates second video data for display on the sub-LCD2 when the display destination is the sub-LCD2, and drives and controls the second display driver 512. The first display driver 511 drives the display panel 4 based on the first video data. The second display driver 512 drives the sub-LCD2 based on the second video data.

[0064] The MCU411 reads and writes video data (first video data and second video data), etc., to a memory 520 located inside or outside the MCU411. The memory 520 may be a non-volatile memory 421, as shown in Figure 5A. In the example in Figure 6A, a configuration is shown in which both the first video data 521 to be provided to the display panel 4 and the second video data 522 to be provided to the sub-LCD2 are stored in the same memory 520.

[0065] Furthermore, in the example shown in Figure 6A, the first video data 521 is stored and managed separately, specifically as image data (e.g., first image data d11) and character data (e.g., first character data d12). Similarly, the second video data 522 is stored and managed separately as image data (e.g., first image data d21) and character data (e.g., first character data d22). Here, image data refers to 2D or 3D image data (still images or videos) in formats such as bitmaps. Here, character data refers to data containing character information such as character codes, fonts, and scaling ratios that are the source of the character images.

[0066] Furthermore, the speech bubble illustrates an example of the detailed data structure of the video data. For example, the first image data d11 has an ID, event information, display destination, display conditions, attributes, and the video data itself. The ID is the identification information of the video data. The event information is information indicating the event that triggered the creation of the video data. The display destination is information indicating the display destination (two types of displays) when the display destination of the video data has been determined (for example, setting the main display panel 4 to 0 and the sub-LCD2 to 1). The display conditions are information indicating the conditions (for example, display time or display end conditions) or methods (for example, dual display or moving display as described later) for how the video data is displayed on the display destination. The attributes of the video data are various attribute information such as type, priority or importance, data amount, information amount, and size (display size). The type is information such as the classification of video information, such as warning, caution, route guidance, vehicle speed, and facility information.

[0067] Figure 6C shows an example of the functional block configuration of the control device 401 in a modified version of Figure 6A. In this modified version, the control device 401 has two control microcontrollers (601, 602) independently arranged in parallel, with control microcontroller 601 controlling the display panel 4 and control microcontroller 602 controlling the sub-LCD 2. The control device 401 has control microcontrollers 601 and 602, vehicle information acquisition unit 407, communication unit 408, etc. interconnected with the bus 630.

[0068] The control microcontroller 601 has a display control unit 610 and a first video data generation unit 501 as functional blocks. The display control unit 610 determines how to display the video data to be displayed, including two types of display destinations, and controls the first video data generation unit 501 when the display destination is the display panel 4. When the display control unit 610 determines that the display destination for the video data to be displayed is the sub-LCD2, it communicates with the control microcontroller 602 and gives instructions to the display control unit 620. When the display control unit 620 determines that the display destination for the video data to be displayed is the sub-LCD2, it controls the second video data generation unit 502.

[0069] Furthermore, in the configuration example shown in Figure 6C, memory 641 used by the control microcontroller 601 and memory 642 used by the control microcontroller 602 are provided independently and in parallel. The control microcontroller 601 accesses memory 641 to read and write the first video data 521. The control microcontroller 602 accesses memory 642 to read and write the second video data 522.

[0070] As an example of another memory configuration, the memory used by the control microcontroller 601 and the memory used by the control microcontroller 602 may be made into a single common memory, and the first video data 521 and the second video data 522 may be stored in this common memory, similar to the memory 520 in Figure 6A.

[0071] Figure 6B shows an example configuration in which a single control microcontroller 503 controls two video outputs (two different display destinations). This configuration is space-saving and low-cost because it uses only one control microcontroller. However, this single control microcontroller is required to control the two video outputs and synchronize their output without delay. This single-control microcontroller configuration easily accommodates situations such as synchronizing video displays on two different displays.

[0072] The configuration example in Figure 6C shows a configuration in which two control microcontrollers (601, 602) control the video output of each system. Each control microcontroller basically only needs to control one type of corresponding video output. However, when synchronizing two video outputs, the two control microcontrollers need to communicate and coordinate to synchronize them.

[0073] The memory configuration example in Figure 6B shows a configuration with one control microcontroller 503 and one memory 520. In this configuration, the data is arranged and stored in memory 520 at separate addresses for the first video data 521 for the display panel 4 and the second video data 522 for the sub-LCD 2. While it is also possible to store all the data in one block, there are significant advantages to storing the data separately according to the display destination, and also by dividing it into image data and character data. For example, if you want to update only the second video data 522 for the sub-LCD 2, you can efficiently update only that second video data 522. If the block of all the mixed data is large, data that does not need to be updated will also be rewritten, which takes time to update.

[0074] The memory configuration example in Figure 6C shows a configuration with two control microcontrollers (601, 602) each having their own memory (641, 642). In this configuration, the first video data 521 for the display panel 4 is stored in memory 641, and the second video data 522 for the sub-LCD 2 is stored in memory 642. Storing data separately according to the display destination, and further separating image data from text data, offers significant advantages.

[0075] [When the HUD device is activated] Figure 7 shows an example of the operation flow when the HUD device 1 is started up. In step S1, the control device 401 of the HUD device 1 first starts preparing the sub-LCD 2 for display. Next, in step S2, the control device 401 of the HUD device 1 starts preparing the video display unit 402, which includes the display panel 4 compatible with the AR-HUD. In step S3, the HUD device 1 waits until the sub-LCD 2 is ready for display. In step S4, the HUD device 1 starts displaying on the sub-LCD 2. In step S5, the HUD device 1 waits until the AR-HUD is ready for display. In step S6, the HUD device 1 starts displaying on the display panel 4 compatible with the AR-HUD.

[0076] In step S7, the HUD device 1 checks if the conditions for transitioning to the off state or sleep state have been met. These conditions include, for example, when the user presses the power off button or when a certain period of time has elapsed without any change in the display. If the conditions are met, in step S8, the HUD device 1 first stops the display on the sub-LCD2. Next, in step S9, the HUD device 1 stops the display on the display panel 4 corresponding to the AR-HUD. In step S10, the HUD device 1 starts the AR-HUD display termination process. In step S11, the HUD device 1 starts the sub-LCD2 display termination process. In step S12, the HUD device 1 waits until the sub-LCD2 display termination process is complete. In step S13, the HUD device 1 waits until the AR-HUD display termination process is complete. After these display termination processes are complete, in step S14, the HUD device 1 transitions to the off state or sleep state.

[0077] Generally, the preparations related to the AR-HUD take more time than those for the sub-LCD2, hence the workflow described above.

[0078] [Example of internal configuration of a HUD device] Figure 8 shows an example configuration of the video display device 403 (Figure 5A) of the HUD device 1, specifically the display panel 4 and the light source device 5. Figure 8 shows a schematic cross-sectional view in the YZ plane. The light source device 5 (the part other than the display panel 4 in Figure 8) and the display panel 4 are arranged within the housing 102A (Figure 4).

[0079] The light source device 5 comprises, in order from the light source side, an LED substrate 801, a collimator 802, a polarization conversion element 803, a light guide 804, and a diffuser plate 805. A display panel 4 is positioned behind the diffuser plate 805 (on the output side). Behind the display panel 4 is the reflective mirror 3a shown in Figure 3. In this example configuration, as shown in the figure, the LED substrate 801, collimator 802, polarization conversion element 803, and light guide 804 are arranged along the Y direction, and the diffuser plate 805 and display panel 4 are arranged along the Z direction from the light guide 804. Note that this light source device 5 is just one example and is not limited to this configuration.

[0080] The LED substrate 801 is a substrate having multiple LED elements 801A as semiconductor light source elements. Although Figure 8 is a YZ cross-section and therefore only one LED element 801A and one collimator element are shown, multiple LED elements 801A are similarly arranged in the X direction on the main surface (XZ plane) of the LED substrate 801, and correspondingly multiple collimator elements are similarly arranged in the X direction on the collimator 802.

[0081] A collimator 802 is provided on the light-emitting side (Y direction) of the LED element 801A on the LED substrate 801. The collimator 802 is an element that controls the direction of light propagation, converting the light from the LED element 801A into approximately parallel light before emission. A polarization conversion element 803 is provided on the light-emitting side (Y direction) of the collimator 802. The polarization conversion element 803 is an element that aligns the polarization characteristics, converting the light from the collimator 802, which has random polarization as approximately parallel light, into light with linear polarization. The polarization conversion element 803 is composed of a combination of a polarization conversion prism and a waveplate.

[0082] A light guide 804 is provided on the light emission side (Y direction) of the polarization conversion element 803. The light guide 804 receives linearly polarized light in the Y direction from the polarization conversion element 803 from the incident part, and controls the light distribution by reflecting it in the Z direction, which is different from the Y direction, i.e., towards the direction of the display panel 4, and then emits it from the emission part. The light guide 804 is equipped with a reflecting part that performs reflection and light distribution control. The reflecting part is formed by alternating reflective surfaces and connecting surfaces. The emission surface of the emission part has, for example, a free-form surface shape for light distribution control.

[0083] The light emitted from the light guide 804 is generally directed upward in the Z direction, and in this example, slightly diagonally to the upper right. A diffuser plate 805 is provided on the light emission side of the light guide 804. The light from the light guide 804 is diffused by the diffuser plate 805 and incident on the back side of the display panel 4. The display panel 4 uses this incident light as a backlight to generate image light. This image light from the display panel 4 is directional.

[0084] [Two types of displays] In the HUD device 1 of Embodiment 1, the basic characteristics of the two types of displays, the AR-HUD display area 105 and the sub-LCD2, are as follows.

[0085] The virtual image 106 displayed in the AR-HUD display area 105 based on the display panel 4 is a directional image, and therefore can only be accurately viewed from the viewpoint 108 within the set eye box in the driving posture of a driver seated in the driver's seat of the vehicle 100 in Figure 3. The virtual image 106 in the display area 105 cannot be easily seen by passengers inside the vehicle.

[0086] The virtual image 106 is inferior to the real image on the sub-LCD2 in terms of image clarity and resolution. Conversely, the image on the sub-LCD2 is clearer than the virtual image 106. Also, the visibility of the virtual image 106 is easily affected by the ambient brightness and the content of the real scene. Furthermore, because the windshield 103 has a curved surface, the image projected onto the windshield 103 is distorted according to its curvature. This distortion requires some kind of correction (for example, the distortion correction mentioned above), and is eliminated by such correction. However, this distortion is not always completely eliminated, and fine details such as text images on the virtual image 106 may be difficult to see.

[0087] The display screen 20 of the sub-LCD2 is positioned, for example, directly facing the driver in the driver's seat, as shown in Figure 1. Because the image on the sub-LCD2 is a diffused image, it remains visible even if the driver's viewpoint moves within the space. The image is visible even when the driver is not in a predetermined driving posture. Passengers in the vehicle can also see the image on the sub-LCD2. Naturally, the image on the sub-LCD2 cannot achieve effects such as augmented reality (AR) by superimposing it onto the real scene. The image on the sub-LCD2 becomes displayable faster than the virtual image 106. The preparation time before display begins is shorter for the image on the sub-LCD2 compared to the virtual image 106. The visibility of the image on the sub-LCD2 does not decrease easily even when displaying fine details such as text.

[0088] The HUD device 1 of Embodiment 1 has a function to control the use of the two types of displays, taking into account the characteristics of the two types of displays described above. This control is performed, for example, by the display control unit 500 in Figure 6B or the display control unit 610 in Figure 6C.

[0089] [How to use them differently (1)] The following describes an example of controlling the use of the two types of displays described above. The display of the virtual image 106 in the display area 105 based on the display panel 4 is used for AR-dedicated or AR-emphasized displays. This virtual image 106 is effective when used for three-dimensional stereoscopic image representation. When displaying image information other than AR, a two-dimensional image representation is used. For displaying two-dimensional image information, the display area 105 or sub-LCD2 is used. For image information that is needed temporarily, the virtual image 106 in the display area 105 is mainly used. For image information that needs to be displayed at all times, sub-LCD2 is mainly used.

[0090] Sub-LCD2 displays 2D video information as non-AR video information. Sub-LCD2 displays video information that should always be displayed to the driver. For example, on a black background, Sub-LCD2 displays 2D video information (e.g., text images) in vivid color.

[0091] [Example Display (1)] Figure 9 shows an example of how two types of displays are used in the HUD device 1 of Embodiment 1. Figure 9(A) shows the image including the real-world view seen through the windshield 103, including the display area 105 as seen by the driver, and the image on the display screen 20 of the sub-LCD2. In the display area 105, an AR virtual image 901 is displayed superimposed on the real-world view, such as a road. This AR virtual image 901 is, for example, route guidance information (for example, an image consisting of multiple triangles) that navigates a right turn route. On the other hand, the sub-LCD2 mainly displays video information 902, including text images such as the current vehicle speed ("40 km / h") and the distance to the right turn point (right turn guidance). The video information 902 is displayed on the display screen 20 in a specified color, such as white on a black background, but in this drawing, the text is shown in black. The content displayed on the display screen 20 is not limited to this and can be varied in various ways.

[0092] Figure 9(B) shows another display example. In the display area 105, when a preceding vehicle is detected in the vehicle's lane, an AR virtual image 903 (for example, a ring-shaped image) is displayed to alert the driver to the preceding vehicle. Meanwhile, the sub-LCD2 displays video information 904 including text images such as vehicle speed, road name, and "Vehicle Ahead".

[0093] In Figure 9(A), etc., a gap area 900 in the vertical direction (Z direction) is illustrated between the display area 105 and the sub-LCD2. Such a gap area 900 may exist when the driver views the front from the driver's perspective. This gap area 900 arises depending on the implementation configuration of the HUD device 1 itself, including the sub-LCD2, and the mounting configuration of the HUD device 1 in the vehicle, as shown in Figure 1, etc. It is not limited to a configuration with a gap area 900; a configuration without a gap area 900 is also possible.

[0094] [How to use them differently (2)] When displaying video information, the HUD device 1 determines which display to use depending on the type of video information. For example, if a given video information can be displayed as either a virtual image 106 in the AR-HUD's display area 105 or as a video on the sub-LCD 2, the HUD device 1 will choose between displaying it in the display area 105 or on the sub-LCD 2, taking into consideration the type and amount of information of the video information. Based on the driving conditions at the time and the amount of information in the driver's field of view of the display area 105 and the display screen 20, the HUD device 1 selects and decides whether to display the video information in the display area 105 or on the sub-LCD 2. An example of such control is described below.

[0095] In a conventional HUD device with only one display (AR-HUD), let's assume that 100 units of information are displayed on that display. In contrast, the HUD device of Embodiment 1 can distribute and display these 100 units of information across two types of displays: the display area 105 and the sub-LCD2. Here, "information quantity" is an abstract concept, but it is a quantity that corresponds to the amount of processing required for the driver, such as cognitive processing. In Embodiment 1, for example, an image with 100 units of information can be displayed as a virtual image 106 with 60 units of information on the display area 105, and as an image with 40 units of information on the sub-LCD2. With this control, the amount of information in the image displayed as a virtual image 106 on the display area 106 can be limited to below a certain amount of information.

[0096] This allows the driver's field of view, including the display area 105, to ensure the necessary field of view for safe driving while providing assistance from the virtual image 106 and sub-LCD2 to an extent that does not place too much burden on the driver, thereby reducing the effects of motion sickness, dizziness, fatigue, etc.

[0097] [Example of control that takes information quantity into consideration] Figure 10 shows the processing flow of the HUD device 1 in relation to an example of control that takes into account the amount of information mentioned above. In step S101, for example, the control device 401 in Figure 6B receives video information to be displayed at each point in time based on various triggers, such as vehicle information 406. The control device 401 can grasp the situation of the vehicle and its surroundings from the vehicle information 406. The control device 401 obtains information about the video information to be displayed, such as type, data amount, size, priority, or importance. This information may be information attached to the video information in the form of attribute information, as in the example in Figure 6B above, or information stored in association with it, or it may be information newly generated by the control device 401. The control device 401 may, for example, calculate the amount of information in the video information.

[0098] "Type" refers to information that indicates the type of information, such as whether the video information is AR or non-AR, whether it is a 3D (three-dimensional) or 2D (two-dimensional) image, whether it is text information or not, and whether it is a GUI (Graphical User Interface such as a menu) or not. "Type" can also refer to classifications such as warning, caution, route guidance, vehicle speed, facility information, or advertising information. "Type" can also refer to information that indicates whether the information should be displayed as a virtual image 106 in the display area 105, on the sub-LCD2, or on either of the two types of displays. One type of "type" is text information, which is represented by, for example, a character code. A text image (for example, a text image such as a bitmap with 2D pixel values) is created using the text information as the source data to be displayed on the AR-HUD or sub-LCD2.

[0099] "Data volume" refers to the amount of data stored in memory for that video information. "Size" refers to the display size, such as the vertical and horizontal dimensions, when the video is displayed on the display area 105 or the display screen 20 of the sub-LCD2. Priority or importance is not mandatory, but if set, it indicates the level of priority or importance of that video information compared to all other video information. For example, video information such as emergency warnings is given the highest priority value. Video information to draw attention to objects around the vehicle is given a lower priority value than warnings. Route guidance information is given a lower priority value. Facility information and advertising information are given a lower priority value.

[0100] In step S102, the control device 401 uses the various information from step S101 to determine the display destination (one of the two types of displays) for each type of video information to be displayed that it has received at a given time. First, for video information for which the display destination is predetermined to be one type, the control device 401 provisionally determines that as the display destination. That is, for predetermined types of video information such as AR, the control device 401 provisionally determines the display destination to be the display panel 4 (and its corresponding display area 105). For predetermined types of video information such as text information, the control device 401 provisionally determines the display destination to be the sub-LCD2.

[0101] In step S103, if there is multiple video information to be displayed, the control device 401 sequentially determines which video information to actually display from among them. In this process, the control device 401 determines the video information to be displayed and its display destination one by one in order of type or priority, for example. The control device 401 determines the display destination of the video information one by one in order of priority. Also, in this process, the control device 401 counts the amount of display information in the display area 105 based on the display panel 4 and in the sub-LCD 2. This amount of display information is, for example, the amount of display information in the display area 105, which is the sum of the information amounts for one or more virtual images 106 within the display area 105. Similarly, the amount of display information in the sub-LCD 2 is the sum of the information amounts for one or more video information within the display screen 20. Furthermore, for video information of a type that can be displayed on any of the displays, the control device 401 selects either the display panel 4 or the sub-LCD2 as the display destination so as to satisfy the restriction that the amount of display information on the display panel 4 and the sub-LCD2 is less than or equal to a predetermined amount.

[0102] Regarding step S103, when the control device 401 calculates the amount of information in the video information, it may take into account the amount of data and size of the video information. Alternatively, the amount of information may be set in advance for each piece of video information. For example, the larger the amount of data or size of the video information, the larger the amount of information may be. The above amount of information and the amount of displayed information represent the magnitude of the burden on the driver when viewing or recognizing the video information or screen, as well as the ease / difficulty of that burden, and can also be described as the amount of burden or an index value.

[0103] Figure 11 shows a display example according to the control example in Figure 10. At one point, the display area 105 displays a virtual image 1101 of AR indicating attention to the preceding vehicle and a virtual image 1102 of AR prompting a warning about the distance between vehicles. The control device 401 displays one or more such virtual images 106 within a range where the amount of display information in the display area 105 is less than or equal to a predetermined amount. At the same time, the control device 401 also receives right-turn route guidance information, similar to that in Figure 9 (A), as video information to be displayed. However, when the control device 401 displays the right-turn route guidance information as AR in the display area 105, the amount of display information in the display area 105 exceeds a predetermined amount, and because the priority order is warning, attention, and route guidance, the control device 401 decides to prioritize the display of virtual images 1101 and 1102 and keep them displayed, and not display the AR for the right-turn route guidance.

[0104] Meanwhile, the control device 401 displays image information 1103 and 1104 using character images on the display screen 20 of the sub-LCD2, within the limits of the amount of information that can be displayed on the sub-LCD2. Image information 1104 is the vehicle speed. The control device 401 has decided to display the right-turn route guidance information on the sub-LCD2 as non-AR image information 1103 (an image representing a right turn 50m ahead) and displays it. In this way, the HUD device 1 of Embodiment 1 can provide driving assistance with a suitable display by using two types of displays.

[0105] [Usage: Default setting] Figure 12 shows an example of the initial display of the display area 105 of the HUD device 1 as an example of controlling the use of two types of displays. When a driver uses the HUD device 1 for the first time, adjustments (also called calibration) are made to suitably set the height of the display area 105 to match the driver's driving posture and eye position (including the eye box). The HUD device 1 allows this adjustment when the vehicle is stationary and safe, but does not allow it while the vehicle is in motion. In AR-HUD, such adjustment of the display area 105 for each driver is essential. If this adjustment is not made, the virtual image 106 of the display area 105 cannot be properly seen by the driver. For example, it is not possible to properly position and superimpose the AR virtual image 106 onto real-world objects such as vehicles or people. In more serious cases, the virtual image 106 may not be visible to the driver at all.

[0106] Therefore, conventional HUD devices, when adjusting the AR-HUD display area, display video information within the display area to adjust the position of the display area (including the virtual image), and allow the user to adjust the position of the display area (including the virtual image) according to their input. However, since the video information for adjustment itself is displayed in the display area as a virtual image of AR, there are cases where the video information itself is not visible to the driver. In such cases, adjustment becomes time-consuming.

[0107] Therefore, in the HUD device 1 of Embodiment 1, during initial adjustment, not only is adjustment video information displayed in the display area 105, but adjustment guide and support video information is also displayed on the sub-LCD 2. In other words, the HUD device 1 uses two types of displays to show the driver, who is the user, adjustment and guide / support video information in the display area 105. The user is then asked to perform the adjustment operation according to this video information.

[0108] In the example shown in Figure 12, the display area 105 initially displays virtual images 1201 and 1202 for adjustment. Virtual image 1201 represents the four corners of a rectangular frame, and virtual image 1202 is a grid image. Conventionally, the display area 105 is adjusted so that, from the driver's perspective, the virtual image 1202 of the grid fits within the virtual image 1201 of the frame. The adjustment operation can be performed, for example, by operating the remote control attached to the HUD device 1.

[0109] The HUD device 1 of Embodiment 1 further displays video information 1203 for adjustment guidance and support on the display screen 20 of the sub-LCD2. The video information 1203 may be, for example, a text image of a guide message such as "Is the grid within the frame in front? Yes / No," or "Adjust the height so that the grid fits within the frame in front," or other GUI components. Even if the user cannot clearly see the adjustment images (virtual images 1201, 1202) in the display area 105, they can still see the video information 1203 on the sub-LCD2, making it easier to perform adjustments by following the guidance provided by the video information 1203. After adjusting to match virtual image 1202 with virtual image 1201, the driver completes the adjustment, for example, by operating the remote control. The HUD device 1 saves and sets the state of the display area 105 after the adjustment.

[0110] The sub-LCD2 may also be a touch panel, in which case the user can perform touch input operations on the display screen 20. For example, the user can make adjustment decisions by selecting "Yes / No" for the video information 1203 on the display screen 20 and performing a touch operation.

[0111] Furthermore, the user can operate the HUD device 1 via an operating interface (e.g., buttons, switches, keys, etc.) installed on the vehicle's steering wheel 104 (Figure 1). This operation can also be used for the above adjustment. In this case, the information from the operating interface is received by a higher-level controller (e.g., the vehicle's ECU), and the HUD device 1 operates by receiving the operation information from that higher-level controller via CAN communication or the like. The processing flow in this case is as follows: When adjustment is selected by user operation on a menu screen displayed on the vehicle's control panel, the HUD device 1 receives the operation information from the higher-level controller, switches to adjustment mode, and starts displaying the virtual image related to the adjustment. The user performs operations such as up, down, left, and right on the operating interface for adjustment. The HUD device 1 receives the operation information from the higher-level controller and adjusts the display of the display area 105 (virtual image 1201, etc.) according to the operation. When the adjustment is complete, the user performs a confirmation operation. The HUD device 1 receives operation information for the decision operation from the higher-level controller, reflects and saves the adjustment result, and exits the adjustment mode.

[0112] Figure 13 shows the processing flow during the initial setup adjustment described above. In step S201, the HUD device 1 determines whether or not to perform adjustments to the AR-HUD display area 105, for example, at startup. If so, it proceeds to steps S202 and S203; otherwise, it proceeds to steps S206 and S207. In steps S202 and S203, the HUD device 1 displays a virtual image 106 for adjusting the display area 102, as shown in Figure 12, in the AR-HUD display area 105, and displays video information for adjustment support on the sub-LCD 2. Next, in step S204, the HUD device 1 adjusts the display area 105 according to the adjustment operation by the driver. In step S205, the HUD device 1 checks whether to perform readjustment (Y) or whether the adjustment is complete (N). For example, if there is no input from the driver indicating that the adjustment is complete, it performs readjustment and returns to steps S202 and S203 to repeat the process. If the adjustment is complete (N), proceed to steps S206 and S207. In steps S206 and S207, the HUD device 1 starts normal display of the virtual image 106 in the display area 105 of the adjusted AR-HUD and starts normal display of the video on the sub-LCD2.

[0113] [Adjustment using sub-LCD] Figure 14 shows other control and display examples related to adjustment assistance using the sub-LCD2 during the initial setup described above. The HUD device 1 of Embodiment 1 uses the sub-LCD2 to assist in adjusting the height position of the AR-HUD display area 105 (in other words, the AR display position). Figure 14(A) shows an example of the display state during adjustment. When the initial setup is performed using this function, the HUD device 1 displays video information for adjusting the display area 105 (for example, a virtual image of the frame 1201 and a virtual image of the grid 1202) in the display area 105, as shown in (A), while displaying video information 1401 for assisting the adjustment of the display area 105 on the display screen 20 of the sub-LCD2. Note that the sub-LCD2 is shown enlarged in Figure 14.

[0114] This video information 1401 includes, for example, information indicating that the AR-HUD is in adjustment mode (a mode for initial adjustments), and information such as menus, guide messages, and cursors. An example of a guide message is a text image prompting the user to adjust the adjustment video (virtual images 1201, 1202) in the display area 105, such as "↑ Please adjust so that the grid fits within the frame." In this example, the HUD device 1 also displays video information 1402 on the sub-LCD 2, which represents the content currently displayed as the adjustment video (virtual images 1201, 1202) in the display area 105. The HUD device 1 also displays text images as guide messages, such as "← Adjustment video currently displayed on the AR-HUD" and "※ If you cannot see it, please adjust your posture, etc.," which represent the content currently displayed in the AR-HUD's display area 105, as well as text images prompting the user to take action if they cannot see it.

[0115] The control device 401 of the HUD device 1 updates the display state of the adjustment video information (virtual images 1201, 1202) of the display area 105 in response to the driver's adjustment input, reflecting the adjustment result, and also updates the corresponding video information 1401, 1402 on the sub-LCD 2. The driver adjusts the display area 105 according to the adjustment support video information 1401, 1402 on the sub-LCD 2, so that the virtual image 1202 of the grid fits within the virtual image 1201 of the frame of the display area 105, as viewed from the driver's perspective. After the video information (virtual images 1201, 1202) of the display area 105 is adjusted to a suitable state, the driver inputs an input indicating that the adjustment is complete. In response to this input, the HUD device 1 saves the adjustment state of the display area 105 at that time and exits the adjustment mode.

[0116] This makes adjustments easier even if the driver cannot clearly see the adjustment images (virtual images 1201, 1202) in the display area 105, by viewing the adjustment support video information 1401, 1402 on the sub-LCD2. Even if the driver cannot see the adjustment images in the display area 105 at all, the driver can recognize the discrepancy by viewing the video information 1402 on the sub-LCD2 and easily determine whether adjustments are necessary. Even if the driver cannot clearly see the video information in the display area 105, they can adjust their seat and posture according to the video information 1401, 1402 on the sub-LCD2 to make the video information in the display area 105 visible.

[0117] Figure 14(B) shows another example of the adjustment support display related to (A). The control device 401 of the HUD device 1 displays an example of the AR virtual image 106 (in this example, a virtual image 1403 corresponding to the route guidance AR object) in the display area 105, and also displays adjustment video information 1404 related to the AR virtual image 106 in the display area 105 on the sub-LCD 2. The driver can select the AR to be adjusted. Multiple AR virtual images may be displayed in the display area 105 and each can be adjusted.

[0118] Examples of what can be adjusted or changed using the video information 1404 on the sub-LCD2 include the color (or brightness), size, shape, tilt, and position of the displayed AR virtual image 1403. GUI components are displayed on the sub-LCD2 for each adjustable item, such as color. GUI components include, for example, bars or palettes for changing color (or brightness), buttons for changing size and ratio, list boxes for selecting shapes (e.g., triangle, arrow), buttons for changing tilt (tilt relative to the horizontal plane, etc.), and buttons for changing position. The driver can adjust each adjustable item of the virtual image 1403 using the remote control or other means. The control device 401 updates the display state of the AR virtual image 1403 in response to the adjustment operation input and saves the determined adjustment state.

[0119] When using the above function, compared to the conventional method where adjustment video information is displayed only in the AR-HUD display area, adjustment support video information is provided using the sub-LCD2. This allows for a wider adjustment screen area, making adjustments easier and more detailed.

[0120] [Usage distinction: In case of errors] Figure 15 shows an example of how to display different information in the event of an error, as an example of another way to differentiate between the two displays. The HUD device 1 uses the display on the other display to notify the user of the error when one of the two displays (display area 105 and sub-LCD 2) becomes unusable or malfunctions.

[0121] In the example in Figure 15(A), when the display area 105 of the AR-HUD enters an error state, the control device 401 displays video information 1501 on the sub-LCD2 indicating that the display area 105 has entered an error state. The video information 1501 is a text image such as, "The communication status is poor, and the HUD cannot be displayed. It will be displayed when the communication status is restored."

[0122] The control device 401 detects an error state in which the virtual image 106 in the display area 105 cannot be displayed correctly due to various circumstances or causes. Examples of such circumstances or causes include a malfunction of a component related to the AR-HUD display, the activation of the solar protection function and the resulting display stoppage, or the inability to acquire ADAS information (ADAS: advanced driving assistant system) due to a CAN communication failure and thus the inability to update the display.

[0123] When an error occurs, the control device 401 notifies the user via the sub-LCD2 that the AR-HUD cannot display the virtual image correctly due to the error in any of the above-mentioned circumstances / causes. In addition, if the HUD device 1 knows the details of the error, the likelihood of recovery from the error, or how to deal with it, it also notifies the user via the sub-LCD2.

[0124] The example in Figure 15(B) shows a case where, if the display screen 20 of the sub-LCD2 becomes in an error state due to a broken wire or other reason, a virtual image 1502 is displayed and notified in the display area 105 of the AR-HUD indicating that the sub-LCD2 is in an error state. The virtual image 1502 is an AR with text, for example, "The sub-LCD cannot display due to a broken wire. Please check the connection." When the control device 401 detects an error state in the sub-LCD2, it displays a virtual image 1501 in the display area 105 of the AR-HUD indicating the error state of the sub-LCD2, its cause, and how to deal with it. Note that such a display on the AR-HUD is performed only when the vehicle is stopped, for example.

[0125] The above features allow the system to communicate an error to the user using the other display even when one of the two displays is showing an error, thus improving user convenience.

[0126] Figure 16 shows the processing flow in the event of the above error. In step S301, the HUD device 1 determines and detects whether the display area 105 of the AR-HUD is in an error state. If it is in an error state, it proceeds to step S302; otherwise, it proceeds to step S305. In step S302, the HUD device 1 determines and detects whether the display of the sub-LCD2 is in an error state. If it is in an error state, it proceeds to step S303; otherwise, it proceeds to step S304. In step S305, the HUD device 1 determines and detects whether the display of the sub-LCD2 is in an error state. If it is in an error state, it proceeds to step S306; otherwise, it proceeds to the end of the flow.

[0127] If the process proceeds to step S303, it corresponds to the case where both types of displays are in an error state. In this case, the HUD device 1 cannot display anything on either display, so it notifies the higher-level controller for the HUD device 1, such as the vehicle's electronic control unit (ECU), via communication that both types of displays of the HUD device 1 are in an error state and therefore cannot display anything.

[0128] If the process proceeds to step S304, it corresponds to the case where only the AR-HUD is in an error state. In this case, the HUD device 1 displays and notifies the user on the sub-LCD2 that the AR-HUD is in an error state, as shown in the example in Figure 15 (A).

[0129] If the process proceeds to step S306, it corresponds to the case where only sub-LCD2 is in an error state. In this case, the HUD device 1 displays and notifies that sub-LCD2 is in an error state in the display area 105 of the AR-HUD, as shown in the example of (B) in Figure 15.

[0130] [Usage differences: in different situations] Figure 17 shows an example of how to use the display differently depending on the situation. The HUD device 1 controls the display destination (two types of displays) and content of the video information according to the vehicle's driving conditions and surrounding conditions (sometimes collectively referred to as "situations").

[0131] One example of a difference in driving conditions is the type of road, such as city streets or highways. For instance, in city streets, vehicles travel at relatively low speeds, and there are many points to pay attention to while driving, making AR displays particularly useful. On the other hand, on highways, vehicles travel at relatively high speeds, and a lack of driver concentration can lead to serious accidents. Therefore, AR displays are less effective on highways than on city streets.

[0132] Furthermore, displaying too many virtual AR images 106 in the AR-HUD display area 105 could obstruct the driver's view and potentially confuse their perception due to the sheer volume of information. Accidents caused by this must be avoided. Compared to ordinary roads, it is preferable to reduce the amount of AR displayed in the AR-HUD display area 105 on highways.

[0133] Therefore, in the HUD device 1 of Embodiment 1, for example, in a general road scenario, the display of the warning AR virtual image 106 in the AR-HUD display area 105 is given the highest priority, and AR such as route guidance, which has a lower priority, is not displayed, or is displayed within a predetermined limit on the amount of display information. In addition, if there is video information with a lower priority, such as route guidance or facility information, the HUD device 1 mainly displays it on the sub-LCD 2.

[0134] Furthermore, in situations such as on a highway, the HUD device 1 prioritizes and minimizes the display of warning AR virtual images 106 in the AR-HUD display area 105, either by not displaying other ARs with lower priority or by displaying them only within predetermined limits. The HUD device 1 primarily displays lower-priority video information on the sub-LCD 2. While route guidance information is useful on highways, limited display of AR information, such as when approaching an exit from the highway, is sufficient. The HUD device 1 normally displays route guidance information on the sub-LCD 2.

[0135] The HUD device 1 controls the display destination and display content according to the various scenarios described above, as well as the type, priority, and amount of video information mentioned earlier.

[0136] The HUD device 1 obtains registered destination information and information such as the vehicle's current location and speed based on the vehicle information 406. Based on this information, the HUD device 1 determines and calculates the distance between the vehicle's current position and the destination (e.g., a right turn point, a highway exit, etc.) and the estimated arrival time. For example, when the vehicle approaches the next destination (e.g., a highway exit) within a predetermined distance, the HUD device 1 starts displaying AR information such as route guidance information in the AR-HUD display area 105.

[0137] Furthermore, the AR display for alerting other vehicles only needs to be shown when other vehicles are approaching in front of or around the vehicle itself. For vehicles approaching from behind, notification should be given via the AR display or the display on the sub-LCD2.

[0138] The example in Figure 17 shows a scene on a highway, where (A) is an example of the display at the first time point, while driving straight, and (B) is an example of the display at the second time point, when approaching a highway exit. In (A), the HUD device 1 minimizes the display of virtual images 106 in the AR-HUD display area 105 in order to ensure the driver's field of view, displaying nothing under normal circumstances and displaying a warning AR in emergencies. The HUD device 1 displays various predetermined information, such as vehicle speed, on the sub-LCD 2.

[0139] (B) When the HUD device 1 approaches the next destination, which is a highway exit, for example, when the distance between the vehicle's current position and the destination falls below a predetermined distance, it starts displaying an AR virtual image 1701 for route guidance (for example, multiple triangular images prompting a left turn) in the display area 105. In addition, in correspondence with the display of the virtual image 1701, the HUD device 1 also starts displaying video information 1702 of text images for route guidance (for example, "Exit 200m Ahead") on the sub-LCD2. When the vehicle reaches the destination, the HUD device 1 stops displaying the virtual image 1701 and the video information 1702 on the sub-LCD2.

[0140] Another example of a scenario is during traffic congestion. During traffic congestion, vehicles are moving slowly or are stopped, making it easier to utilize the display on the sub-LCD2 in addition to the AR display on the AR-HUD. In traffic congestion scenarios, the HUD device 1 uses the display area 105 and the sub-LCD2 to display video information related to the congestion. For example, the HUD device 1 displays video information such as the congestion status, an estimate of the congestion length, suggested detours, and the distance and time to rest areas. The HUD device 1 prioritizes displaying this video information in the AR-HUD's display area 105, and displays any information that cannot be displayed in the display area 105 on the sub-LCD2. Regarding the AR display of warnings about vehicles ahead, it may be difficult to utilize if the vehicle ahead is too close in traffic congestion, so it is advisable to disable that AR display.

[0141] Another example of this scenario is weather. For instance, in snowy or foggy conditions, the visibility of the AR-HUD display area 105 decreases. Based on vehicle information 406, the HUD device 1 determines the weather and, if it determines that the visibility of the AR virtual image 106 in the display area 105 is insufficient, it does not display the AR and instead displays alternative video information only on the sub-LCD2.

[0142] Furthermore, the HUD device 1 may use two different display destinations depending on the destination. For example, in situations such as dropping off or picking up family members at the station (when the destination is a known station), the driver knows the route, so the AR-HUD display area 105 omits the route guidance AR and mainly displays the warning AR. The HUD device 1 displays information such as the scheduled arrival of the train or emails from family members on the sub-LCD 2. When the destination is unknown, the HUD device 1 displays route guidance AR in the AR-HUD display area 105.

[0143] [Select the user to display] The HUD device 1 of Embodiment 1 has a function that allows the user to select and set whether to turn on or off the display of the virtual image 106 in the display area 105 of the AR-HUD and whether to turn on or off the display of video on the sub-LCD 2. The display states of the two displays are controlled according to this selection and setting. In addition, the user may also be able to select and set what kind of information is displayed on the AR-HUD and the sub-LCD 2, respectively.

[0144] Figure 18 shows an example of the display related to this function. First, (A) shows an example where the on / off status of the AR-HUD display is displayed in the sub-LCD2 menu, allowing the user to select the option. The sub-LCD2 display screen 20 shows the video information 1801 of the function menu. In this menu, there are three options for the AR-HUD display: "Yes," "Yes (partial)," and "No," which the user can select from. The selection can be made by touch input as described above, or by using a remote control. For example, if "Yes" is selected, the virtual image 106 in the display area 105 will be turned on, and if "No" is selected, the virtual image 106 in the display area 105 will be turned off. If "Yes (partial)" is selected, only some of the set types of information will have the virtual image 106 displayed. The settings for some types of information can also be configured by the user in other menus that are accessed.

[0145] (B) shows an example where the AR-HUD menu displays options such as turning the sub-LCD2 display on or off, allowing the user to select the desired option. The display area 105 shows a menu of functions in the form of a virtual image 1802. This menu offers two options for the sub-LCD2 display: "On" and "Off," which the user can select. The selection can be made using a remote control or voice input. For example, if "On" is selected, the display on the sub-LCD2 will be turned on.

[0146] Furthermore, regarding the functions described above, recommended combinations may be defined in advance and presented in a menu, allowing the user to select from them. Figure 19 shows an example of the display for this function. In the example in Figure 19, the display method setting menu is displayed in display area 105, and the user can select from multiple combinations of settings. Similarly, the display method setting menu is also displayed on sub-LCD2. The user can operate and configure the settings on either the AR-HUD or sub-LCD2, whichever they prefer.

[0147] As an example of display method combinations, the first combination has both the AR display and the sub-LCD display turned on, with destination guidance displayed in AR and vehicle speed displayed in the sub-LCD. The second combination has both the AR display and the sub-LCD display turned off, with both destination guidance and vehicle speed displayed in AR. The third combination has both the AR display and the sub-LCD display turned on, with both destination guidance and vehicle speed displayed in the sub-LCD. Detailed settings are also possible for each combination. In the detailed settings, you can set the type and priority of the video information to be displayed at the display destination. For example, as types of video information to be displayed on the AR-HUD, you can turn destination guidance on / off, turn vehicle speed on / off, and set the priority order for each type.

[0148] Furthermore, the HUD device 1 may be provided with separate remote controls for AR-HUD input and sub-LCD2 input as input devices and operation input means. Alternatively, there may only be one remote control for common input between AR-HUD and sub-LCD2. In that case, the user may be able to appropriately select whether to use that one remote control for AR-HUD input operation or sub-LCD2 input operation. Alternatively, the HUD device 1 may automatically determine which display to use the one remote control for based on the situation. For example, the HUD device 1 may control the remote control to target AR-HUD when only the AR-HUD display is on, as in the second combination, and control the remote control to target sub-LCD2 when only the sub-LCD2 display is on, as in the third combination. The control of the two types of input described above may be applied similarly to the aforementioned operation interface, etc., not just to the remote control. The HUD device 1 may also be operated via an operation interface installed on the vehicle's steering wheel, etc.

[0149] [Usage: Brightness control] Another example of how to differentiate their use is that the HUD device 1 allows for individual control and setting of the display brightness on the AR-HUD side and the sub-LCD 2 side. This setting can be done automatically or manually. The virtual image 106 in the display area 105 of the AR-HUD is formed, as shown in Figure 3, corresponding to a predetermined position in front of the vehicle via the windshield 103 from the driver's perspective. The brightness of this virtual image 106 should be dynamically changed according to the brightness of the display position of the virtual image 106 (for example, 10m in front of the vehicle). This brightness information of the virtual image display position can be obtained, for example, as one of the vehicle information 406. Alternatively, the HUD device 1 may acquire the brightness information of the virtual image display position using the sensor 409.

[0150] Furthermore, the HUD device 1 may acquire headlight ON / OFF information from vehicle information 406 and increase the brightness of the virtual image 106 of the AR-HUD when the headlights are turned ON (illuminated) in situations such as at night or inside a tunnel.

[0151] On the other hand, while the sub-LCD2 has little need for such dynamic brightness changes, the HUD device 1 may perform such changes. If so, the HUD device 1 changes the brightness of the display screen 20 of the sub-LCD2 according to the ambient light surrounding the sub-LCD2 (e.g., inside the vehicle), which can be determined from vehicle information 406 or sensor 409.

[0152] Figure 20 shows an example of the display brightness change control described above. (A) is the case when the surroundings, including the front of the vehicle, are relatively bright, such as during the day. (B) is the case when the surroundings, including the front of the vehicle, are relatively dark, such as at night. In (A), the HUD device 1 controls the display of the virtual image 2001 in the AR-HUD display area 105 so that the brightness of the virtual image 2001 is relatively higher (for example, compared to nighttime). This improves the visibility of the virtual image 2001. In (B), the HUD device 1 controls the display of the virtual image 2002 in the AR-HUD display area 105 so that the brightness of the virtual image 2002 is relatively lower (for example, compared to daytime). This reduces glare while ensuring the visibility of the virtual image 2002.

[0153] Furthermore, in the example shown in Figure 20, the HUD device 1 also controls the sub-LCD 2 so that the brightness on the display screen 20 is higher during the daytime (A) compared to the nighttime (B) (it is represented as whiter in the diagram).

[0154] In the above function, the HUD device 1 controls the display brightness of the AR-HUD display area 105 and the display screen 20 of the sub-LCD2 in order to improve overall visibility as much as possible. The HUD device 1 controls the brightness of the display area 105 and the brightness of the display screen 20 so that they have a predetermined brightness relationship.

[0155] Furthermore, in relation to the above functions, the HUD device 1 should unify the display style between the AR-HUD virtual image 106 and the sub-LCD 2. Elements that constitute the style include, for example, color (e.g., warm / cool colors), font, GUI design, etc. When the driver shifts their gaze between the display area 105 and the display screen 20, a unified style reduces the sense of incongruity. The user may also be able to select and set the style of the AR-HUD display and the sub-LCD 2 display from multiple themes in the user settings.

[0156] [Distortion Correction] The HUD device 1 performs distortion correction on the display of the AR-HUD virtual image 106 using predetermined hardware and software, such as the distortion correction unit 413 shown in Figure 5B. Figure 21 shows an example of a display with distortion correction. Distortion correction is the process of correcting the distortion of the virtual image 106 that occurs according to the curvature of the vehicle's windshield 103 (Figure 1) so that the distortion is reduced. Since the HUD device 1 directly projects image light onto the display area 105 of the windshield 103, such distortion correction is necessary to present a suitable virtual image 106. The windshield 103 has different curvatures in the horizontal and vertical directions, for example. Therefore, if the image information to be displayed is projected directly onto the display area 105, distortion will occur in the image according to that curvature. For example, if the image information to be displayed is a source image 2100 (for example, a star image in a square area), and it is projected directly based on that source image 2100, distortion like that of the virtual image 2101 will occur.

[0157] The control device 401 of the HUD device 1 performs, for example, a correction process 2102 (a correction process to cancel out distortion according to curvature) on the original image 2100 of the video information to be displayed, taking into account the curvature of the windshield 103. The video light corresponding to the corrected image 2103 is projected onto the display area 105. As a result, from the driver's perspective, the virtual image 2104 corresponding to the corrected image 2103 can be seen as distortion-free and close to the original image 2100.

[0158] On the other hand, the sub-LCD2 does not require the distortion correction described above. In Embodiment 1, the display screen 20 of the sub-LCD2 is flat and has no curvature. Therefore, distortion correction is unnecessary. When displaying the original image 2100 on the sub-LCD2, if it is displayed as is, the resulting image 2105 will be close to the original image 2100.

[0159] As a variation, the sub-LCD2 may be a curved display whose display screen 20 has a predetermined curvature. In that case, the HUD device 1 may perform distortion correction on the video information displayed on the sub-LCD2, taking into account the curvature of the display screen 20.

[0160] [Pitching Correction] The HUD device 1 performs pitch correction on the display of the AR-HUD virtual image 106 using predetermined hardware and software, such as the pitch correction unit 414 shown in Figure 5B. Figure 22 shows an example of a display related to pitch correction. Pitch correction is the vertical correction of the display position of the virtual image 106 in the display area 105 according to the shaking of the vehicle while driving. When there is shaking, the virtual image 106 in the display area 105 is displayed shifted relative to the position of the target vehicle in front, for example, as shown by the dashed line image (ring-shaped image). In this example, the image shown by the dashed line shows a case where it is shifted downwards relative to the position of the target vehicle in front. Pitch correction can reduce the misalignment of the superposition between the real object and the virtual image 106. Based on the vehicle information 406, the HUD device 1 grasps the shaking of the vehicle and corrects the display position within the display area 105 in the video information of the display target according to that shaking.

[0161] The control device 401 of the HUD device 1 performs, for example, a pitch correction process 2202 on the original image 2200 of the video information to be displayed, taking into account the state of vehicle vibration. The original image 2200 has a display position (e.g., point p1) in the display area 205. In the pitch correction process 2202, the display position is corrected (e.g., to point p2), and further, the size, tilt, etc. may be corrected as needed. In this example, the corrected display position is corrected to point p2, which is above point p1, so that the AR image is closer to the position of the target vehicle in front, that is, higher up. Video light corresponding to the corrected image 2203 is projected onto the display area 105. As a result, from the driver's viewpoint, the virtual image 2201 corresponding to the corrected image 2203 can be seen as being superimposed in line with the position of the target vehicle in front.

[0162] On the other hand, the sub-LCD2 does not require the pitching correction described above. Because the driver and the sub-LCD2 also shake along with the vehicle's movement, and because it is not an AR system, the driver can still see the video information on the sub-LCD2 without pitching correction. In the example shown in Figure 22, the video information on the display screen 20 of the sub-LCD2 is positioned at a predetermined location.

[0163] [Usage: Debug mode] Another example of its use is the ability to display information corresponding to the debug mode of the HUD device 1. The HUD device 1 transitions from normal mode to debug mode according to a predetermined operation. The debug mode is a mode for the HUD device 1 operator, such as developers and maintenance personnel, to debug and maintain the HUD device 1. In debug mode, the HUD device 1 displays useful information for developers and maintenance personnel (debug information, maintenance information), such as logs of CAN information received from the vehicle via CAN communication as vehicle information 406, software operation logs, and error codes, on at least one of the two displays. In the case of conventional HUD devices, there is no sub-LCD2, so even if such information is to be displayed, it will be displayed on the AR-HUD or on a device externally connected to the HUD device.

[0164] HUD device 1 stores, for example, error codes in case of errors. HUD device 1 may store, for example, the error codes for the past few instances, associated with date and time information, in a non-volatile memory 421 or similar. In debug mode, HUD device 1 displays the information, including the error code, on the selected display destination from among two types of displays.

[0165] Figure 23 shows an example of the debug mode display. (A) shows the case where an error code is displayed in the AR-HUD display area 105 during an actual driving test. (B) shows the case where an error code is displayed on the sub-LCD2 during an actual driving test. In (A), a virtual image 2301 of the error code is displayed in the display area 105. Although it is possible to display the error code as a virtual image 2301 as in (A), this virtual image 2301 can only be seen from the driver's eyebox viewpoint. Therefore, this display method is not very suitable for detailed displays, especially for displaying text information. Therefore, the HUD device 1 displays video information 2302 of the error code on the display screen 20 of the sub-LCD2, as in (B). This allows developers and maintenance personnel to efficiently perform debugging and other tasks by viewing the detailed video information 2302 of the error code on the sub-LCD2, even if there is a problem with the AR-HUD display, thereby improving the efficiency of development and maintenance.

[0166] The debug mode display in Figure 23 uses a control similar to the error status notification control example in Figure 15 mentioned above.

[0167] [Double display] The HUD device 1 of Embodiment 1 also has a function to display similar information twice on two different displays. This function can be used, for example, as a demo mode for developers, or during normal use by general users. The HUD device 1 transitions to the dual display mode upon a predetermined trigger, such as a mode selection input by the user. Alternatively, the HUD device 1 automatically performs dual display depending on the type of video information to be displayed.

[0168] Figure 24 shows an example of dual display. When the HUD device 1 is in dual display mode, for example, it displays the same information on the display screen 20 of the sub-LCD2 as the AR virtual image 106 displayed on the AR-HUD display area 105. In the example in Figure 24, virtual images 2401 and 2402 for route guidance are displayed on the display area 105. Virtual image 2401 is a triangular image representing a right turn, and virtual image 2402 is a text image representing "Turn right in 100m". Meanwhile, the display screen 20 of the sub-LCD2 displays video information 2403 corresponding to virtual image 2401 and video information 2404 corresponding to virtual image 2402.

[0169] Furthermore, the characteristics of the display area 105, such as size and brightness, are fundamentally different from those of the display screen 20, making it impossible to display an image on the display screen 20 that is exactly the same as the virtual image 106. Therefore, the HUD device 1 creates the AR virtual image 106 based on the original image, and also creates an image that is roughly the same or similar for display on the sub-LCD 2. The control device 401 displays two images that are roughly the same, taking into account the differences in the characteristics of the two types of displays.

[0170] The control device 401, for example, uses the first video data generation unit 501 and the second video data generation unit 502 (as shown in Figure 6B) to create video information 2411 for the display area 105 and video information 2412 for the sub-LCD 2, based on the original image 2400. The video information 2411 is data containing the display position (e.g., point q1) and size within the display area 105. The video information 2412 is data containing the display position (e.g., point q2) and size within the display screen 20. Based on this data, the control device 401 displays the video information at each display destination in sync with the timing.

[0171] The virtual images 2401 on the AR-HUD are directional, so they can only be seen from the driver's perspective in the driver's seat, and not from the perspective of passengers. On the other hand, the video information 2403 on the sub-LCD2 can be seen from the perspective of people other than the driver, such as passengers.

[0172] For example, this dual display can be used in demo modes during development or maintenance. This allows people other than the driver to perceive an image on the sub-LCD2 that is roughly the same as the virtual image 106 displayed on the AR-HUD. For example, a developer or other passenger can easily see what virtual image 106 is currently displayed on the AR-HUD by looking at the image on the sub-LCD2. This can improve development efficiency.

[0173] Furthermore, this dual display function can be used in various ways even when general users are using it normally. Let's explain this using the example in Figure 24. For example, suppose the driver specifies a destination, and the HUD device 1 displays AR route guidance related to the route to the destination in the display area 105. Depending on the relationship between the vehicle's current position and the distance to the destination, the HUD device 1 displays, for example, virtual images 2401 and 2402 in the AR-HUD display area 105. The driver can see these virtual images. On the other hand, passengers cannot see these virtual images, but by looking at the video information 2403 and 2404 on the sub-LCD 2, they can recognize the content of the route guidance that the HUD device 1 is currently presenting to the driver.

[0174] In this scenario, a passenger might find the suggested route incorrect or inefficient, or they might want to change their destination. In these cases, the passenger can immediately inform the driver. This allows for quick route changes based on the passenger's needs, improving convenience.

[0175] When the HUD device 1 displays an image on the sub-LCD2 that is similar to the virtual image 106 of the AR-HUD, it may perform image processing such as modification based on the original image to create and display an image (e.g., image information 2412) that is suited to the characteristics of the display screen 20, such as its size. Examples of modification include enlarging / reducing the image, changing the aspect ratio, and cropping. Even if the images displayed on the two displays are not the same, it is effective as long as the meaning of the image is conveyed to the person. In addition, the sub-LCD2 may display a text image that represents the meaning of the virtual image 106 (e.g., "Route guidance is currently displayed on the AR-HUD").

[0176] Figure 25 shows the processing flow of the HUD device 1 related to the control of selecting the display destination of video information from dual display on the AR-HUD only, the sub-LCD only, or both, in relation to the dual display function described above. In step S401, the control device 401 of the HUD device 1 recognizes the occurrence of an event based on the vehicle information 406 and obtains video information to be displayed according to the event. In step S402, the HUD device 1 determines the event, scene, type of video information, etc., and decides the display destination of the video information. Step S403 is a branch according to the display destination, and proceeds to step S404 if only the AR-HUD is used, to step S405 if only the sub-LCD2 is used, and to step S406 if both the AR-HUD and sub-LCD2 are used (dual display).

[0177] In step S404, the HUD device 1 displays a virtual image 106 in the display area 105 of the AR-HUD based on the video information generated by the first video data generation unit 501. In step S405, the HUD device 1 displays video information on the sub-LCD 2 based on the video information generated by the second video data generation unit 502. In step S406, the HUD device 1 displays a virtual image 106 in the display area 105 of the AR-HUD based on the video information generated by the first video data generation unit 501, and also displays video information on the sub-LCD 2 based on the video information generated by the second video data generation unit 502.

[0178] In step S407, the HUD device 1 determines whether a predetermined display termination condition corresponding to the virtual image 106 (video information) displayed in the AR-HUD display area 105 has been met. For example, in the case of route guidance in Figure 24, this condition is whether the vehicle's current position has reached the destination. If the condition is met, the HUD device 1 terminates the display of the virtual image 106 in the AR-HUD display area 105 in step S408.

[0179] In step S409, the HUD device 1 determines whether a predetermined display termination condition corresponding to the video information displayed on the sub-LCD 2 has been met. This condition is, for example, the same as the virtual image condition in step S407. If the condition is met, the HUD device 1 terminates the display of the video information on the sub-LCD 2 in step S410.

[0180] In step S411, the HUD device 1 determines whether a predetermined display termination condition has been met for both the virtual image 106 of the AR-HUD and the video information of the sub-LCD 2. This condition is, for example, the same as the condition in step S407. If the condition is met, the HUD device 1 terminates the display of video information from both the AR-HUD and the sub-LCD 2 in step S412.

[0181] In the control example above, regarding dual display, the start and end timings of the display of the virtual image 106 on the AR-HUD and the image on the sub-LCD2 were set to be the same, and the display termination conditions were also set to be the same for both. However, the following modifications are also possible. That is, the start and end timings of the display of the virtual image 106 on the AR-HUD and the image on the sub-LCD2 may be set to be different depending on the type and scene of the video information to be displayed. For example, the HUD device 1 may start displaying video information with the corresponding content to two different display destinations at the same time for a given video information, and terminate the display of each video information at different times with different display termination conditions for each display destination. In another example, the HUD device 1 may start displaying video information with the corresponding content to two different display destinations at different timings with different display start conditions for a given video information, and terminate the display of each video information at the same time. It is sufficient that the video information with the corresponding content to the two display destinations remains in a dual display state for at least a certain period of time.

[0182] [Change display destination (1)] The HUD device 1 of Embodiment 1 also has a function to control the display destination of video information by using two types of displays to transition, change, or move from one to the other. Such displays are also referred to as display destination change control or moving display. The HUD device 1 controls such moving display according to the type of video information and the scene.

[0183] Figure 26 shows an example of display when the display destination of certain video information is changed and moved from AR-HUD to sub-LCD2. (A) shows the state at the first time point in time when the video information of the notification, for example, a right-turn guidance, is displayed as a virtual image 106 in the display area 105 of AR-HUD. (B) shows the state at the second time point in time when the video information of the notification, for example, a right-turn guidance, is displayed on sub-LCD2.

[0184] For example, suppose an event occurs that requires notification to the driver. One example is that the vehicle is approaching its destination, and the notification is a notification that the vehicle is approaching its destination. Suppose the video information of that notification is represented in a non-AR form (e.g., as text images). Even in that case, the HUD device 1 first displays the video information of the notification as an AR virtual image 106 in the AR-HUD display area 105, which is easily within the driver's field of view and can be viewed without shifting the driver's gaze.

[0185] At the first moment, the HUD device 1 displays the video information on the display panel 4. This displays the AR virtual image 106 in the AR-HUD display area 105. Subsequently, when a certain period of time has elapsed, or when predetermined conditions are met, such as when it is confirmed that the driver has seen the AR virtual image 106 based on a sensor (for example, the camera 109 in Figure 3), the HUD device 1 displays video information with the same content as the virtual image 106 on the sub-LCD 2 at the second moment. At this time, the HUD device 1 erases the AR virtual image 106 by erasing the video information on the display panel 4. This provides the driver with a visual effect where, for example, the display destination of notification video information changes and moves from the AR-HUD display area 105 to the sub-LCD 2, as shown in the example in Figure 26.

[0186] A detailed example is illustrated in Figure 26. In (A), within the display area 105 of the AR-HUD, video information (specifically a combination of text and arrow images) is displayed as a virtual image 2601 to guide the vehicle to turn right, such as "Turn right in 300m". This virtual image 2601 is generated when the vehicle's current position approaches within a predetermined first distance from the destination (e.g., the right turn point). At this point, the HUD device 1 first selects the AR-HUD as the display destination (first display destination) for this video information and displays the video information on the display panel 4. As a result, during the first period from the first time point, the virtual image 2601 is displayed in the display area 105 of the AR-HUD.

[0187] In this control example, the reason for setting the first display destination to the AR-HUD is that, fundamentally, displaying it on the AR-HUD makes it easier for the driver to see and notice the image, thus minimizing the driver's eye movement. By seeing this virtual image 2601 that appears within the display area 105, the driver can recognize the message "Turn right 300m ahead."

[0188] Next, in (B), at a second time point after a period of time has elapsed since (A), the display of the virtual image 2601 within the AR-HUD display area 105 is erased, and video information 2603 having content corresponding to the virtual image 2601 is displayed on the display screen 20 of the sub-LCD2. The HUD device 1 terminates the display of the virtual image 2601 of the AR-HUD and erases it from the display area 105, as shown in area 2602, at a predetermined trigger. The predetermined trigger is the elapsed time or the travel of a predetermined distance. For example, one condition is when the vehicle's current position approaches within a predetermined second distance (shorter than the first distance mentioned above) from the destination (right turn point).

[0189] Along with the above deletion, the HUD device 1 displays video information 2603 (specifically a combination of a character image and an arrow image) having content corresponding to the virtual image 2601 on the display screen 20 of the sub-LCD 2. In other words, the HUD device 1 changes the display destination of this right-turn guidance video information from the first display destination to the second display destination. In other words, the HUD device 1 moves this right-turn guidance video from the first display destination to the second display destination.

[0190] At the second point in (B), the vehicle is closer to its destination than at the first point in (A), so the importance of the right-turn guidance video information to the driver is reduced. In addition, when turning right or left, there are many points to pay attention to in the vicinity, so it is safer to maintain a clear field of view. If there is an AR display or similar in the display area 105 at this time, the field of view may be obstructed, making it difficult for the driver to see what they want to see or need to see. Therefore, in this example, it is effective to change the display destination of this video information to sub-LCD2 and move the video information to sub-LCD2, thereby reducing the number of videos displayed in the AR-HUD display area 105 and reducing the amount of information displayed.

[0191] During the second period from the second time point, the right-turn guidance via video information 2603 continues on the sub-LCD2. The driver can periodically shift their gaze to the sub-LCD2 to view and confirm the video information 2601. The HUD device 1 terminates the display of video information 2603 on the sub-LCD2 at a predetermined trigger. This predetermined trigger may be, for example, a predetermined time elapsed from the second time point, or a predetermined distance traveled. For example, the vehicle reaching its destination could be a condition.

[0192] In the example in Figure 26, the video information for the right-turn guidance (virtual image 2601 and video information 2603) is determined according to the relationship between the vehicle's current position and the distance to the destination. This video information may be displayed as a static image or as a video that changes over time. For example, this video information may change sequentially according to the distance, such as "300m ahead," "200m ahead," and "100m ahead." In the example in Figure 26, the video information 2603 is updated to "100m ahead" in accordance with the distance traveled, relative to the virtual image 2601.

[0193] Furthermore, in controlling the moving display as described above, since the AR-HUD and sub-LCD2 have fundamentally different characteristics, similar to the dual display control mentioned earlier, it is sufficient to display roughly the same or related video information on both.

[0194] As another variation, the HUD device 1 may be controlled to return the moving display, such as AR, to its original state after the completion of a turn or other action. For example, the HUD device 1 may display certain video information as AR in the display area 105 at a first point in time before a right turn, move that video information to the sub-LCD2 and display it at a second point in time while turning right, and then display that video information again as AR in the display area 105 at a third point in time after the right turn. In other words, the HUD device 1 may be controlled to temporarily change the video information from the normal first display destination to the second display destination only when and when a predetermined condition (e.g., a right turn) is met.

[0195] As in the example above, the triggers and conditions for when to move the video information displayed in the AR-HUD's display area 105 as the first display destination to the sub-LCD2 as the second display destination, or for how long to display the video information on the second display destination, can be controlled according to the type of video information and the scene. For example, it may be set to a certain period of time after the occurrence of an event. Alternatively, for example, in response to an event detecting that a seat belt is not fastened, a warning may be displayed with the AR-HUD as the first display destination and the sub-LCD2 as the second display destination, and then the warning display may be terminated when the seat belt is fastened. In other examples, the display of video information may basically continue until the AR-HUD or sub-LCD2 is turned off. Alternatively, when new video information is generated in the AR-HUD or sub-LCD2, the old video information may be terminated. Alternatively, multiple video information may be continuously displayed in the AR-HUD or sub-LCD2 as long as they satisfy the amount of information that can be displayed.

[0196] Furthermore, when controlling the moving display as described above, there may be periods when the corresponding video information is temporarily displayed on both of the two displays, or periods when it is temporarily not displayed on either. In either case, a visual effect can be achieved in which the display destination of a certain video information changes.

[0197] As a variation of the above-described mobile display, it is also possible to set the first display destination to sub-LCD2 and the second display destination to AR-HUD, depending on the type of video information.

[0198] Figure 27 shows an example of display when the display destination of certain video information is changed and moved from SubLCD2 to AR-HUD. (A) shows that at the first time point, the video information 2701 "Turn right 500m ahead" is displayed on SubLCD2, which is the first display destination. (B) shows that at the second time point, the virtual image 2702 "Turn right 300m ahead" is displayed on the display area 105 of AR-HUD, which is the second display destination. The display content at the second time point is the same as the example in (A) of Figure 26. The control example in Figure 27 is based on the idea that at the first time point, the destination is far away and the importance of the right-turn guidance is still low, so the video information is displayed on SubLCD2. At the second time point, the importance of the right-turn guidance has increased, so the video information is displayed on AR-HUD.

[0199] [Change display destination (2)] Figure 28 shows an example of a modified display of the display destination change control described in Figures 26 and 27. The HUD device 1 initially displays a virtual image 2801 of a certain display target video information (in this example, a star-shaped image) as the first display destination within the display area 105 of the AR-HUD at the first time point (A). The virtual image 2801 is an AR with a predetermined function, but in this example it is shown in an abstracted form. The right-turn guidance in Figure 27 can also be applied similarly as an example of target video information.

[0200] The HUD device 1, at a predetermined trigger, changes the display destination of this video information (virtual image 2801) to the sub-LCD 2 as the second display destination. At that time, as shown in the second time point in (B), the HUD device 1 continuously moves the video information (virtual image 2801) toward the sub-LCD 2 (downward) on the display. That is, on the time axis, the display position of the virtual image 2801 moves downward within the display area 105. (B) shows an example of the state in which the display position of the video information (virtual image 2801) has reached the bottom edge of the display area 105 and is in the process of going beyond the bottom edge and out of frame. Simultaneously, the HUD device 1 continuously moves this video information into the display screen 20 of the sub-LCD 2. In (B), this video information is framed in from the top edge of the rectangle of the display screen 20. Naturally, the portion 2802 of this video information displayed within the display screen 20 is a non-AR image.

[0201] (C) shows the state after the video information has moved to a predetermined position on the display screen 20 of the sub-LCD2 at the third time point. This video information is displayed as video information 2803 on the display screen 20 of the sub-LCD2.

[0202] As described above, the HUD device 1 controls the display so that the display destination of the video information moves continuously from the first display position of the first display destination to the second display position of the second display destination. During this movement, the form of the video information changes from a virtual image 106 to a non-virtual image 106. In this control example, from the driver's perspective, the target video information moves continuously, making it easy to recognize the change in the display destination of the video information. It is similarly possible to continuously move the display destination of the video information from the sub-LCD2 to the display area 105.

[0203] Figure 29 shows the processing flow of the HUD device 1 when controlling the moving display shown in Figure 26, etc. In step S501, the control device 401 of the HUD device 1 obtains event information and target video information based on vehicle information 406, etc. In step S502, the HUD device 1 determines the event, scene, type of video information, etc., and determines the first and second display destinations for the video information when performing a moving display. In step S503, the HUD device 1 uses the first video data generation unit 501 (Figure 6A) to create first video information to be displayed at the first display destination and displays the first video information (e.g., virtual image 106) at the first display destination (e.g., AR-HUD). In step S504, the HUD device 1 determines the conditions for moving the first video information at the first display destination to the second display destination. If the conditions are met, in step S505, the HUD device 1 uses the second video data generation unit 502 (Figure 6A) to create second video information for display on the second display destination, displays the second video information on the second display destination (e.g., sub-LCD2), and erases the first video information on the first display destination. In the case of continuous moving display as shown in Figure 28, the processing content in step S505 is display control for continuous movement from the first display destination to the second display destination.

[0204] Although the present invention has been specifically described above based on embodiments, the present invention is not limited to the embodiments described above and can be modified in various ways without departing from the spirit of the invention. In each embodiment, components can be added, deleted, or replaced, except for essential components. Unless otherwise specified, each component may be singular or plural. Combinations of each embodiment are also possible.

[0205] The technology according to this embodiment allows for the optimal viewing of images of driving-related information, such as navigation information including destination and speed projected onto the windshield, as well as alert information when oncoming vehicles or pedestrians are detected. By providing an information display device (head-up display device) that is user-friendly for drivers and other users, reduces the driver's eye movement, and contributes to supporting safe driving, thereby preventing traffic accidents. This contributes to the United Nations' Sustainable Development Goal (SDG) 3, "Good Health and Well-being." [Explanation of Symbols]

[0206] 1...HUD device, 2...Sub-LCD, 20...Display screen, 101...Dashboard, 102...Housing, 103...Windshield, 104...Steering wheel, 105...Display area, 106...Virtual image.

Claims

1. A vehicle that displays a virtual image in a display area based on the projection of video light, Windshield and, A housing containing a light source device for generating the aforementioned image light, a main display device, and an optical system, A sub-display device provided on the outside of the aforementioned housing, A control device that controls the display of video information on the main display device and the display of video information on the sub-display device, Equipped with, A portion of the aforementioned housing is housed within the vehicle's dashboard. The control device displays video information for user settings of the display method, including turning the display on / off in the display area of ​​the windshield based on the main display device, on the sub-display device, and displays video information for user settings of the display method, including turning the display on / off in the sub-display device, on the main display device. The type of video information to be displayed on the main display device is controlled by operating the vehicle or the sub-display device. vehicle.

2. In the vehicle according to claim 1, The control device determines, with respect to the video information to be displayed, at least one of the main display device and the sub-display device as the display destination, based on at least one of the following: the surrounding conditions of the vehicle, the type of video information, the amount of data, the amount of information, the display size, or the priority. vehicle.

3. In the vehicle according to claim 1, The control device determines, with respect to the video information to be displayed, at least one of the main display device and the sub-display device as the display destination, based on the amount of information displayed on the main display device and the amount of information displayed on the sub-display device. vehicle.

4. In the vehicle according to claim 1, The control device, when adjusting the position of the display area during initial setup, displays adjustment video information on the main display device and adjustment support video information on the sub-display device. vehicle.

5. In the vehicle according to claim 4, The control device displays an AR image as adjustment video information on the main display device, and displays video information for adjusting at least one of the color, size, shape, tilt, or position of the AR image as adjustment support video information on the sub-display device. vehicle.

6. In the vehicle according to claim 1, The control device, when the display of the display area based on the main display device is in an error state, displays video information on the sub-display device indicating that the display area is in an error state, and when the display on the sub-display device is in an error state, displays video information on the main display device indicating that the sub-display device is in an error state. vehicle.

7. In the vehicle according to claim 1, The control device determines, with respect to the video information to be displayed, at least one of the main display device and the sub-display device as the display destination, depending on the scene, including the surrounding conditions of the vehicle. vehicle.

8. In the vehicle according to claim 1, The control device adjusts the brightness of the display on the main display device and the brightness of the display on the sub-display device according to the situation including the surrounding conditions of the vehicle and the conditions inside the vehicle. vehicle.

9. In the vehicle according to claim 1, The control device determines, based on the surrounding conditions of the vehicle or the type of video information, to be displayed as a dual display on both the main display device and the sub-display device. vehicle.

10. In the vehicle according to claim 1, The control device determines, with respect to the video information to be displayed, to either the main display device or the sub-display device as the first display destination, and to the other of the main display device or the sub-display device as the second display destination, and in the time axis, first displays the first video information to the first display destination, and then displays the second video information having content corresponding to the first video information to the second display destination. vehicle.

11. In the vehicle according to claim 10, The control device displays the video information to be displayed at the first display position of the first display destination, and displays it so as to move continuously from the first display position to the second display position of the second display destination. vehicle.

12. In the vehicle according to claim 1, The housing comprises a first housing portion housed within the vehicle's dashboard and a second housing portion extending outside the dashboard. The sub-display device is provided in the second housing portion. vehicle.

13. In the vehicle according to claim 12, The second housing portion has an opening from which the image light is emitted, The housing has a mechanism to change the arrangement of the sub-display device such that when the vehicle is not in use, the sub-display device hides at least a portion of the opening in the second housing. vehicle.

14. In the vehicle according to claim 1, Equipped with a remote control as an input device, The aforementioned remote control can be used for both the input operations of the main display device and the input operations of the sub-display device. When the main display device is turned on, the remote control is controlled to direct input to the main display device, and when the sub display device is turned on, the remote control is controlled to direct input to the sub display device. The type of video information to be displayed on the main display device is controlled by operating the sub-display device with the remote control. vehicle.

Images