Display system

Abstract

This system provides a display that can suppress the misalignment of multiple virtual images while keeping the size of the display down. [Solution] The display system 10 comprises a plurality of display units (first display unit 100, second display unit 200) that project virtual images (first virtual image 101, second virtual image 201) in front of the user, a viewpoint estimation unit (control unit 500) that estimates the user's viewpoint position, and an adjustment unit (control unit 500) that adjusts the display position of at least one of the plurality of virtual images based on the estimation result of the viewpoint estimation unit. Each of the plurality of display units comprises a display element (display element 130, 230) that displays an image which is the basis of the virtual image and emits video light of the image, and a final reflecting unit (windshield 2, second reflector 260) that reflects the video light emitted from the display element toward the user, and the final reflecting units provided in the plurality of display units are each separate.

Description

Technical Field

[0001] The present disclosure relates to a display system.

Background Art

[0002] Patent Document 1 discloses a HUD (Head-Up Display) that irradiates display light forming a first virtual image and a second virtual image toward the front windshield of a vehicle. As a result, the first virtual image and the second virtual image are displayed in front of the viewer by the display light reflected by the front window. Here, when the viewer's viewpoint fluctuates, the positional relationship between the first virtual image and the second virtual image is shifted. In order to suppress this shift in the positional relationship, Patent Document 1 discloses an angle adjustment unit that adjusts the position of at least one of the first virtual image and the second virtual image by adjusting the traveling direction of at least one of the first display light forming the first virtual image and the second display light forming the second virtual image.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in Patent Document 1, since the first display light and the second display light are reflected by the front window, the optical system has to be partially shared, but as a drawback, the optical configuration becomes large.

[0005] Therefore, an object of the present disclosure is to provide a display system that can suppress the positional deviation of a plurality of virtual images while suppressing an increase in size.

Means for Solving the Problems

[0006] A display system according to one aspect of the present disclosure comprises a plurality of display units that project a virtual image in front of a user, a viewpoint estimation unit that estimates the user's viewpoint position, and an adjustment unit that adjusts the display position of at least one of the plurality of virtual images based on the estimation result of the viewpoint estimation unit, wherein each of the plurality of display units comprises a display element that displays an image which is the basis of the virtual image and emits video light of the image, and a final reflection unit that reflects the video light emitted from the display element toward the user, and the final reflection units provided in each of the plurality of display units are separate entities. [Effects of the Invention]

[0007] According to this disclosure, it is possible to provide a display system that can suppress the misalignment of multiple virtual images while also suppressing the need for larger display sizes. [Brief explanation of the drawing]

[0008] [Figure 1] Figure 1 is a schematic diagram showing the display system according to Embodiment 1 installed in a vehicle. [Figure 2] Figure 2 is a schematic diagram showing the first display unit, the second display unit, and the control unit, which are components of the display system according to Embodiment 1. [Figure 3] Figure 3 is an explanatory diagram showing the first and second virtual images before and after the adjustment according to Embodiment 1. [Figure 4] Figure 4 is a schematic diagram showing the display system according to Embodiment 2 installed in a vehicle. [Figure 5] Figure 5 is a schematic diagram showing the third display unit according to Embodiment 2. [Figure 6] Figure 6 is a schematic diagram showing the display system according to Embodiment 3 installed in a vehicle. [Figure 7] Figure 7 is an explanatory diagram showing the first and second virtual images before and after the adjustment according to Embodiment 4. [Figure 8] Figure 8 is a schematic diagram showing the first display unit, the second display unit, and the control unit, which are components of the display system according to Embodiment 5. [Figure 9]Figure 9 is a schematic diagram showing the display system according to Embodiment 6 installed in a vehicle. [Figure 10] Figure 10 is an explanatory diagram showing examples of the display of the first virtual image and the second virtual image according to Embodiment 9. [Figure 11] Figure 11 is an explanatory diagram showing examples of the display of the first virtual image and the second virtual image according to Embodiment 9. [Modes for carrying out the invention]

[0009] (1) A display system according to one aspect of the present disclosure comprises a plurality of display units that project a virtual image in front of a user, a viewpoint estimation unit that estimates the user's viewpoint position, and an adjustment unit that adjusts the display position of at least one of the plurality of virtual images based on the estimation result of the viewpoint estimation unit, wherein each of the plurality of display units comprises a display element that displays an image which is the basis of the virtual image and emits video light of the image, and a final reflection unit that reflects the video light emitted from the display element toward the user, and the final reflection units provided in each of the plurality of display units are separate.

[0010] According to the display system described in (1), the adjustment unit adjusts the display position of at least one of the multiple virtual images based on the user's viewpoint position estimated by the viewpoint estimation unit. This adjustment suppresses positional shifts of multiple virtual images caused by fluctuations in the user's viewpoint position. Furthermore, since the final reflection units in each of the multiple display units are separate, the degree of freedom in designing the optical configuration within each display unit can be increased compared to the case where the final reflection units are common, making it possible to make the optical configuration more compact. In other words, the overall size of the display system can be suppressed. As a result, a display system can be provided that suppresses positional shifts of multiple virtual images while suppressing an increase in size.

[0011] (2) In the display system described in (1) above, the adjustment unit may adjust the display position of each of the multiple virtual images based on the estimation result of the viewpoint estimation unit.

[0012] According to the display system described in (2), since the adjustment unit adjusts the display positions of each virtual image, the relative positional deviation of the display positions of each virtual image can be suppressed more precisely.

[0013] (3) In the display system according to the above (1) or (2), the adjustment unit may adjust the display position of at least one virtual image among the plurality of virtual images so that the interval between the plurality of virtual images falls within a predetermined interval.

[0014] According to the display system described in (3), since the adjustment unit adjusts the display position of at least one virtual image among the plurality of virtual images so that the interval between the plurality of virtual images falls within a predetermined interval, the interval between the plurality of virtual images can be made smaller than the predetermined interval. Thereby, a plurality of virtual images can be displayed compactly, and the visibility of the plurality of virtual images for the user can be enhanced.

[0015] (4) In the display system according to the above (3), the plurality of virtual images are arranged side by side in the vertical direction, and the adjustment unit may adjust the display position of at least one virtual image among the plurality of virtual images so that the interval between the lower end of the displayable range of the virtual image arranged above and the upper end of the displayable range of the virtual image arranged below in a pair of adjacent virtual images in the vertical direction falls within a predetermined range.

[0016] According to the display system described in (4), the adjustment unit adjusts the display position of at least one virtual image among the plurality of virtual images so that the interval between the lower end of the displayable range of the virtual image arranged above and the upper end of the displayable range of the virtual image arranged below falls within a predetermined range. Thereby, a plurality of virtual images can be displayed compactly in the vertical direction. Therefore, when the user views a plurality of virtual images, the movement of the line of sight in the vertical direction is suppressed, so the visibility in the horizontal direction can be enhanced.

[0017] (5) In the display system according to any one of the above (1) to (4), it may be provided with three or more of the above display units.

[0018] (5) The display system described above provides a display system that can suppress the misalignment of each virtual image displayed by three or more display units while also suppressing the need for larger displays.

[0019] (6) In the display system described in any one of (1) to (5) above, the display unit may include a plurality of optical members including the display element and the final reflection unit, and a moving unit for moving at least one of the plurality of optical members, and the adjustment unit controls the moving unit to adjust the display position.

[0020] According to the display system described in (6), the adjustment unit controls the movement unit to adjust the display position, so that the display position of the virtual image can be adjusted by the movement unit while keeping the position of the image within the display element constant.

[0021] (7) In the display system described in (6) above, the moving part may move at least two or more of the plurality of optical members.

[0022] According to the display system described in (7), one moving part moves at least two of the multiple optical members, so the device configuration can be simplified compared to the case where one moving part is provided for each optical member to be moved.

[0023] (8) In the display system described in (6) or (7) above, the adjustment unit may control the movement unit such that the viewing distance to the lower end of the virtual image is shorter than the viewing distance to the upper end of the virtual image.

[0024] According to the display system described in (8), the adjustment unit makes the viewing distance to the lower edge of the virtual image shorter than the viewing distance to the upper edge of the virtual image, so that a virtual image that gives the driver a more natural sense of depth can be displayed.

[0025] (9) In the display system described in any one of (1) to (5) above, the adjustment unit may adjust the display position by adjusting the coordinate position of the image within the display element.

[0026] According to the display system described in (9), the adjustment unit adjusts the display position by adjusting the coordinate position of the image within the display element, so the display position can be adjusted without providing a moving unit.

[0027] (10) In the display system described in any one of (1) to (9) above, an input unit is provided into which adjustment instructions from the user are input, the adjustment unit adjusts the display position of one virtual image based on the adjustment instructions from the user input into the input unit, the viewpoint estimation unit estimates the viewpoint position based on the adjustment amount for the one virtual image, and the adjustment unit adjusts the display position of other virtual images based on the estimated viewpoint position.

[0028] According to the display system described in (10), the viewpoint estimation unit estimates the viewpoint position based on the amount of adjustment for one virtual image that has been adjusted by adjustment instructions from the user. In other words, the viewpoint position can be estimated even without a sensor to detect the user's viewpoint position. Furthermore, the adjustment unit adjusts the display position of other virtual images based on the estimated viewpoint position, so the display position of other virtual images can be adjusted quickly.

[0029] (11) In the display system described in (10) above, the input unit may receive fine adjustment instructions from the user, and the adjustment unit may fine-tune the display position of the other virtual image based on the fine adjustment instructions received in the input unit.

[0030] According to the display system described in (11), the display position of other virtual images is fine-tuned based on the fine-tune instruction, so that the display position of other virtual images can be fine-tuned to the position intended by the user.

[0031] (12) In the display system described in (10) or (11) above, a detection unit for detecting the user's viewpoint position may be provided, and the viewpoint estimation unit corrects the estimated viewpoint position based on the detection result of the detection unit.

[0032] According to the display system described in (12), the viewpoint estimation unit corrects the estimated viewpoint position based on the detection result of the detection unit, so that the viewpoint position can be estimated more accurately.

[0033] (13) In the display system described in any one of (10) to (12) above, the input unit may be a voice sensor.

[0034] According to the display system described in (13), since the input unit is a voice sensor, the user can input adjustment instructions by voice. Therefore, manual input by the user is not required when inputting adjustment instructions, and it is possible to prevent the user's driving operations from being interrupted by the adjustment instructions.

[0035] (14) In the display system described in any one of (1) to (9) above, a camera for photographing the user may be provided, and the viewpoint estimation unit may estimate the user's viewpoint position based on the image captured by the camera.

[0036] According to the display system described in (14), the viewpoint estimation unit estimates the user's viewpoint position based on the captured image obtained by the camera, so the viewpoint position can be estimated with high accuracy using the captured image. Therefore, positional shifts of multiple virtual images can be suppressed more accurately.

[0037] (15) In the display system described in any one of (1) to (9) above, a plurality of detection units for detecting the user's viewpoint position may be provided, and the viewpoint estimation unit estimates the user's viewpoint position based on the detection results of the plurality of detection units.

[0038] According to the display system described in (15), the viewpoint estimation unit estimates the user's viewpoint position based on the detection results of multiple detection units, so the viewpoint position can be estimated with high accuracy. Therefore, positional misalignment of multiple virtual images can also be suppressed more accurately.

[0039] (16) In the display system described in (4) above, the reflectivity of the final emitting surface forming one of a pair of virtual images adjacent in the vertical direction, where the absolute value of the depression angle or elevation angle is larger, may be greater than the reflectivity of the final reflective part forming the other virtual image.

[0040] According to the display system described in (16), the reflectivity of the final reflecting element forming one virtual image with a large absolute value of the depression or elevation angle is greater than the reflectivity of the final reflecting element forming the other virtual image. In other words, the other virtual image with a small absolute value of the depression or elevation angle may be positioned directly in front of the user, but since the reflectivity of the final reflecting element forming the other virtual image is smaller than the reflectivity of the first final reflecting element, a bright forward field of view for the user can be ensured.

[0041] (17) In the display system described in any one of (1) to (16) above, each of the plurality of display units may be arranged spatially independently of the optical path from the display element to the final reflection unit.

[0042] According to the display system described in (17), each of the multiple display units has an optical path that is spatially independent from the display element to the final reflecting unit. In other words, since the optical path of each display unit is spatially independent, maintenance can be made easier.

[0043] (18) In the display system described in (17) above, the display unit may include a plurality of optical members including the display element and the final reflection unit, a housing that houses at least a portion of the plurality of optical members, and a housing movement unit that moves the housing, and the adjustment unit controls the housing movement unit to adjust the display position.

[0044] According to the display system described in (18), the adjustment unit controls the housing movement unit to adjust the display position, so the optical path remains constant within the housing before and after adjustment. Therefore, optical distortion in the virtual image can be suppressed.

[0045] (19) In the display system described in any one of (6) to (8) above, the adjustment unit may adjust the viewing distance of the virtual image by controlling the moving unit to adjust the display position.

[0046] According to the display system described in (19), the adjustment unit controls the movement unit to adjust the display position and thereby adjust the viewing distance of the virtual image. This allows the magnification of each virtual image to be changed, increasing the degree of freedom in display effects.

[0047] (20) In the display system described in any one of (1) to (19) above, the display element may impart a distortion to the image based on an adjustment amount for the display position.

[0048] According to the display system described in (20), the display element imparts a distortion to the image based on the amount of adjustment relative to the display position, so that the display element displays an image that takes into account the distortion of the virtual image that may occur after adjustment. Therefore, the virtual image based on this image can be displayed with distortion suppressed.

[0049] (21) In the display system described in any one of (1) to (20) above, a notification unit may be provided, and the notification unit may provide an over-notification if the amount of adjustment of the display position for the at least one virtual image determined based on the estimation result of the viewpoint estimation unit exceeds the adjustment range of the adjustment unit.

[0050] According to the display system described in (21), if the amount of adjustment of the display position for at least one virtual image determined based on the estimation result of the viewpoint estimation unit exceeds the adjustment range of the adjustment unit, the notification unit will provide an over-notification. As a result, the user can recognize the adjustment range of the adjustment unit by the over-notification and move the viewpoint position according to the adjustment range.

[0051] (22) In the display system described in any one of (1) to (21) above, if the amount of adjustment of the display position for the at least one virtual image determined based on the estimation result of the viewpoint estimation unit exceeds the adjustment range of the adjustment unit, the adjustment unit may adjust the display position of the at least one virtual image to the vicinity of the upper or lower limit of the adjustment range.

[0052] According to the display system described in (22), if the amount of adjustment for the display position of at least one virtual image determined based on the estimation result of the viewpoint estimation unit exceeds the adjustment range of the adjustment unit, the display position of at least one virtual image is adjusted to the vicinity of the upper or lower limit of the adjustment range. Therefore, even for users whose viewpoint position exceeds the adjustment range of the adjustment unit, they can recognize a virtual image positioned near the upper or lower limit of the adjustment range by moving their viewpoint position based on the least possible head movement of the user.

[0053] (23) In a display system described in any one of (1) to (22) above, the system may include a notification unit and a determination unit that determines whether the user's gaze is directed toward the warning object within a predetermined time after the warning object is displayed on at least one of the plurality of display units, wherein the notification unit provides a gaze guidance notification if the determination unit determines that the user's gaze is not directed toward the warning object within the predetermined time.

[0054] According to the display system described in (23), if the user's gaze is not directed toward the warning target within a predetermined time, the notification unit will issue a gaze guidance notification, which will guide the user's gaze toward the warning target.

[0055] (Embodiment) The embodiments will be described in detail below with reference to the drawings. Note that the embodiments described below are all specific examples of this disclosure. The numerical values, shapes, materials, components, arrangement and connection configurations of components, steps, and the order of steps shown in the following embodiments are examples only and are not intended to limit this disclosure. Furthermore, among the components in the following embodiments, those not described in the independent claim representing the highest-level concept will be described as optional components.

[0056] In the following embodiments, expressions indicating the relative orientation of two directions, such as parallel and orthogonal, may be used, but these expressions include cases where the orientation is not strictly accurate. For example, when two directions are said to be parallel, unless otherwise specified, this means not only that the two directions are perfectly parallel, but also that they are substantially parallel, i.e., that they include a difference of, for example, a few percent. The optical paths illustrated in each figure in the following embodiments are for illustrative purposes only and do not necessarily reflect actual optical paths.

[0057] [Embodiment 1] Figure 1 is a schematic diagram showing the display system 10 according to Embodiment 1 installed in a vehicle 1. In Figure 1, the vehicle 1 is shown in cross-section. Figure 2 is a schematic diagram showing the components of the display system 10 according to Embodiment 1: the first display unit 100, the second display unit 200, and the control unit 500.

[0058] As shown in Figures 1 and 2, the display system 10 comprises a first display unit 100, a second display unit 200, a camera 600, and a control unit 500. The first display unit 100 and the second display unit 200 display vehicle information relating to vehicle 1 as a first virtual image 101 and a second virtual image 201, respectively. Examples of vehicle information include the vehicle speed of vehicle 1, engine speed, detection results of objects approaching vehicle 1, navigation information from vehicle 1's current location to its destination, and image information captured by an external camera that captures images of the rear and surroundings of vehicle 1.

[0059] <First display section> The first display unit 100 includes a display device body 109 and a windshield (front glass) 2 provided on the vehicle 1. The display device body 109 is an AR-HUD (Augmented Reality Head-up Display). The display device body 109 projects image light onto the windshield 2, which is the display medium. The projected image light is reflected by the windshield 2. This reflected light is directed towards the eyes of the driver, who is the user, sitting in the driver's seat. In other words, the windshield 2 is an example of a final reflecting surface that reflects the image light towards the driver.

[0060] The driver perceives the reflected light that enters their eyes as a first virtual image 101 that appears on the opposite side of the windshield 2 (outside the vehicle), with actual objects visible through the windshield 2 as the background. Thus, the first display unit 100 is an example of a display unit that projects a virtual image in front of the driver.

[0061] As shown in Figure 2, the display device body 109 comprises a housing 110, a cover portion 120, a display element 130, a first optical element 140, a second optical element 150, and a first movable portion 170. The display element 130, the first optical element 140, the second optical element 150, and the windshield 2 are examples of multiple optical members for projecting the first virtual image 101. The multiple optical members may include a hologram element.

[0062] The housing 110 is a box-shaped body made of light-shielding resin or metal. Specifically, the housing 110 has a roughly rectangular parallelepiped shape, with an opening 111 formed at its top. The opening 111 is closed by a cover portion 120. The internal space of the housing 110 and the cover portion 120 houses the display element 130, the first optical element 140, and the second optical element 150.

[0063] The cover portion 120 is a curved plate made of, for example, a translucent resin or glass. Specifically, the cover portion 120 has a shape that is convex downwards as a whole.

[0064] The display element 130 is, for example, a liquid crystal panel. When light from a light source (not shown) is shone onto the display element 130, it displays an image that forms the basis of the first virtual image 101 and shines the image light of the said image onto the first optical element 140. The display element 130 may also be an organic EL panel. The display element 130 is formed in a rectangular shape in plan view and is arranged in a position tilted with respect to the horizontal plane.

[0065] The first optical element 140 is positioned on the optical path of the image light emitted from the display element 130 and is an optical element that reflects the image light toward the second optical element 150. The first optical element 140 is a convex mirror formed in a rectangular shape in plan view. The first optical element 140 is positioned at an angle to the vertical plane. The reflective surface of the first optical element 140 faces the display element 130 and the second optical element 150. In other words, the reflective surface, which is the mirror surface of the convex mirror, of the first optical element 140 faces inward towards the housing 110, and the concave surface faces outward towards the housing 110.

[0066] The second optical element 150 is positioned on the optical path of the image light that has passed through the first optical element 140, and reflects the image light reflected by the first optical element 140 toward the aperture 111. Specifically, the second optical element 150 is a concave mirror formed in a rectangular shape in plan view. The second optical element 150 is positioned opposite the reflective surface side of the first optical element 140 and tilted with respect to the vertical plane of the housing 110. The reflective surface of the second optical element 150 faces the first optical element 140 and the cover portion 120. In other words, the reflective surface, which is the mirror surface of the concave mirror, of the second optical element 150 faces inward towards the housing 110, and the convex surface faces outward towards the housing 110. The image light reflected by the second optical element 150 is projected onto the windshield 2 through the aperture 111. This reflection directs the image light toward the eyes of the driver sitting in the driver's seat, forming the first virtual image 101. Figure 1 illustrates the position of the first virtual image 101 as seen from the driver's viewpoint. This position can be set by adjusting the viewing distance of the image light emitted from the display element 130 of the first display unit 100. The viewing distance is the distance from the driver's viewpoint to the image formation position of the virtual image (e.g., the first virtual image 101). The driver's viewpoint is, for example, the reference eye point. The reference eye point is "a point that represents the position of the driver's eyes in normal driving conditions."

[0067] The first moving unit 170 is located inside the housing 110 and adjusts the first display position in the vertical direction of the first virtual image 101. The first moving unit 170 adjusts the first display position by moving the second optical element 150, thereby changing the optical path of the image light that forms the first virtual image 101. Specifically, the first moving unit 170 is equipped with a rotation mechanism and a drive motor for adjusting the orientation (tilt) of the second optical element 150. The second optical element 150 has, for example, the center of its reflective surface as the center of rotation and rotates clockwise or counterclockwise in a side view. This rotational movement is performed by the rotation mechanism and drive motor provided in the first moving unit 170. This changes the orientation of the second optical element 150. As the orientation of the second optical element 150 changes, the optical path of the image light that forms the first virtual image 101 is also changed, and the first display position of the first virtual image 101 is adjusted in the vertical direction. In this embodiment, the first moving unit 170 is shown as rotating the second optical element 150, but it may also be slid. Sliding movement is movement along at least one direction, such as the vertical direction, the front-back direction, or the optical axis direction. Rotational movement and sliding movement may be combined. Furthermore, the first moving unit 170 may also move optical members other than the second optical element 150 provided in the display device body 109.

[0068] <Second display> As shown in Figure 1, the second display unit 200 projects image light toward the driver. The driver perceives the image light that enters their eyes as a second virtual image 201 projected far away from the opening 221 (see Figure 2) of the second display unit. Thus, the second display unit 200 is an example of a display unit that projects a virtual image in front of the driver. Specifically, the second display unit 200 projects the second virtual image 201 in front of the driver and below the first virtual image 101. Therefore, the downward angle of the driver's line of sight relative to the second virtual image 201 is greater than the downward angle of the driver's line of sight relative to the first virtual image 101.

[0069] As shown in Figure 2, the second display unit 200 comprises a housing 220, a display element 230, a polarizing half-mirror 240, a first reflector 250, a second reflector 260, and a second moving unit 270. The display element 230, the polarizing half-mirror 240, the first reflector 250, and the second reflector 260 are examples of multiple optical elements for projecting the second virtual image 201. The multiple optical elements may include a hologram element.

[0070] The housing 220 is a box-shaped body formed from a light-shielding resin or metal. An opening 221 facing backward is formed at the upper end of the rear of the housing 220 (the right direction in Figure 2 is defined as the rear or rear). Image light, which becomes the second virtual image 201, is projected from the opening 221. The internal space of the housing 220 houses a display element 230, a polarizing half mirror 240, a first reflector 250, and a second reflector 260.

[0071] The display element 230 is, for example, a liquid crystal panel. When light from a light source (not shown) is shone onto the display element 230, it displays an image that forms the basis of the second virtual image 201 and shines the image light of the said image onto the polarizing half mirror 240. The display element 230 may also be an organic EL panel. The display element 230 is positioned with its display surface facing backward. Although not shown in the illustration, a λ / 4 phase difference plate (hereinafter abbreviated as λ / 4 plate) is laminated on the display surface of the display element 230. The λ / 4 plate is a λ / 4 phase difference plate that creates a phase difference of 1 / 4 of the wavelength λ in the light incident on the λ / 4 plate. For example, if the light emitted from the display surface is linearly polarized S-polarization, it is converted to circularly polarized light by passing through the λ / 4 plate.

[0072] The polarizing half-mirror 240 is configured to reflect P-polarized light and transmit S-polarized light, with a reflective polarizer plate arranged on a flat glass substrate. Furthermore, a λ / 4 plate is laminated on the surface of the polarizing half-mirror 240. The polarizing half-mirror 240 is positioned opposite the display element 230 and the first reflector 250. S-polarized image light emitted from the display element 230 is converted to circularly polarized light by the λ / 4 plate laminated on the display element 230 and directed towards the polarizing half-mirror 240. This circularly polarized image light is converted to P-polarized light by the λ / 4 plate laminated on the polarizing half-mirror 240 and reflected by the reflective polarizer plate of the polarizing half-mirror 240. The reflected P-polarized image light is converted back to circularly polarized light by passing through the λ / 4 plate again. Therefore, the polarizing half-mirror 240 is positioned such that the circularly polarized image light incident on the polarizing half-mirror 240 is reflected towards the first reflector 250 in a circularly polarized state by the λ / 4 plate and reflective polarizer laminated on the polarizing half-mirror 240.

[0073] The first reflector 250 is a concave mirror and is positioned below the polarizing half mirror 240 in Figure 2. The first reflector 250 is positioned with its concave reflective surface facing upwards. The circularly polarized image light reflected by the polarizing half mirror 240 is reflected again by the first reflector 250 while remaining circularly polarized, and then returns to the polarizing half mirror 240. The image light incident on the polarizing half mirror 240 is converted to S-polarization by the λ / 4 plate stacked on the polarizing half mirror 240, and then passes through the reflective polarizer of the polarizing half mirror 240 and proceeds upwards in Figure 2.

[0074] The second reflector 260 is a flat mirror and is positioned above the polarizing half mirror 240. Therefore, the image light that passes through the polarizing half mirror 240 and travels upward is reflected by the second reflector 260. The second reflector 260 is positioned to reflect the image light toward the aperture 221. The image light reflected by the second reflector 260 is directed toward the eyes of the driver sitting in the driver's seat through the aperture 221, becoming the second virtual image 201. In other words, the second reflector 260 is an example of a final reflecting part that reflects the image light toward the driver.

[0075] As mentioned above, the downward angle of the driver's line of sight relative to the second virtual image 201 is greater than the downward angle of the driver's line of sight relative to the first virtual image 101. In other words, the first virtual image 101 is positioned directly in front of the driver more than the second virtual image 201. As a result, if the first virtual image 101 is bright, it is likely to obstruct the driver's view. To suppress this, the reflectivity of the second reflector 260 is made greater than the reflectivity of the windshield 2. As a result, the first virtual image 101, formed by the image light reflected from the windshield 2, is displayed fainter than the second virtual image 201, formed by the image light reflected from the second reflector 260. This ensures that the driver's forward view is bright.

[0076] In the second display unit 200, the optical path of the image light from the display element 230 to the second reflector 260 is located within the housing 220. Therefore, this optical path is spatially independent of the optical path of the image light from the display element 130 to the windshield 2 in the first display unit 100.

[0077] Figure 1 illustrates the position of the second virtual image 201 as seen from the driver's perspective. This position can be set by adjusting the viewing distance of the image light emitted from the display element 230 of the second display unit 200.

[0078] The second moving unit 270 is provided in the housing 220 of the second display unit 200 and adjusts the second display position in the vertical direction of the second virtual image 201. The second moving unit 270 adjusts the second display position by moving the second reflector 260, thereby changing the optical path of the image light that forms the second virtual image 201. Specifically, the second moving unit 270 is equipped with a rotation mechanism and a drive motor for adjusting the attitude (tilt) of the second reflector 260. The second reflector 260 has, for example, the center of its reflective surface as the center of rotation and rotates clockwise or counterclockwise in a side view. This rotational movement is performed by the rotation mechanism and drive motor provided in the second moving unit 270. This changes the attitude of the second reflector 260. As the attitude of the second reflector 260 changes, the optical path of the image light that forms the second virtual image 201 is also changed, and the second display position of the second virtual image 201 is adjusted in the vertical direction. In this embodiment, the example illustrates the case where the second moving unit 270 rotates the second reflector 260, but it may also slide. Sliding movement is movement along at least one direction, such as the vertical direction, the front-back direction, or the optical axis direction. By sliding in the optical axis direction, it is also possible to adjust the viewing distance of the second virtual image 201. Rotational movement and sliding movement may be combined. Furthermore, the second moving unit 270 may also move optical members other than the second reflector 260 provided in the second display unit 200.

[0079] <camera> As shown in Figure 1, camera 600 is positioned inside vehicle 1 near the top of the windshield 2 and captures images of the driver's head and the interior of the vehicle. In other words, the images captured by camera 600 include characteristic points of the driver's head and characteristic points of the interior of the vehicle.

[0080] <Control Unit> As shown in Figure 1, the control unit 500 is electrically connected to the first display unit 100, the second display unit 200, and the camera 600, and controls them. Specifically, the control unit 500 is equipped with a CPU, RAM, ROM, etc., and the CPU executes each process by loading the program in ROM into RAM and executing it.

[0081] The control unit 500 acquires images captured by the camera 600 and estimates the driver's viewpoint position from the captured images. Specifically, the control unit 500 extracts feature points of the driver's head and feature points of the vehicle interior by applying predetermined image processing to the captured images. Based on the feature points of the head and feature points of the vehicle interior, the control unit 500 estimates the driver's viewpoint position (coordinate values ​​of the viewpoint position in at least one of the front-to-back direction, up-and-down direction, and vehicle width direction). Thus, the control unit 500 is an example of a viewpoint estimation unit that estimates the driver's viewpoint position.

[0082] Furthermore, the control unit 500 also functions as an adjustment unit that adjusts the first display position of the first virtual image 101 and the second display position of the second virtual image 201, respectively, based on the estimation results. Specifically, the control unit 500 adjusts at least one of the first display position and the second display position by controlling the first moving unit 170 and the second moving unit 270 based on the estimation results. This control allows at least one of the first display position and the second display position to be adjusted to a position corresponding to the driver's viewpoint.

[0083] Here, the control unit 500 may add a distortion based on the adjustment amount for the display position to the image displayed on the display element. For example, after determining the adjustment amount for the first display position, the control unit 500 adds a distortion based on that adjustment amount to the image displayed on the display element 130. Similarly, after determining the adjustment amount for the second display position, the control unit 500 adds a distortion based on that adjustment amount to the image displayed on the display element 230. Therefore, images that take into account the distortion of the first virtual image 101 and the second virtual image 201 that may occur after adjustment are displayed on the display elements 130 and 230. Consequently, after adjustment, the distortion of the first virtual image 101 and the second virtual image 201 is canceled out, and the appearance of the first virtual image 101 and the second virtual image 201 does not change easily before and after adjustment, which does not cause discomfort to the driver.

[0084] <Adjustment of the first and second display positions> The following will specifically explain the differences between the first virtual image 101 and the second virtual image 201 before and after the adjustment of the first and second display positions.

[0085] Figure 3 is an explanatory diagram showing the first virtual image 101 and the second virtual image 201 before and after adjustment according to Embodiment 1. Figure 3 shows the state in which the driver views the vehicle 1Z in front of them from their own vehicle 1. In Figure 3, "A" is shown as the first virtual image 101 and "B" is shown as the second virtual image 201. Furthermore, in Figure 3, the first displayable range R1 of the first virtual image 101 and the second displayable range R2 of the second virtual image 201 are shown by dashed lines. The displayable range is the area in which the virtual image can be displayed. For example, the first displayable range R1 depends on the display area of ​​the display element 130, and the second displayable range R2 depends on the display area of ​​the display element 230. The first displayable range R1 corresponds to the first display position, and the second displayable range R2 corresponds to the second display position.

[0086] Figure 3(a) shows the first virtual image 101 and the second virtual image 201 before adjustment. This Figure 3(a) shows the case where a small driver is viewing the front of vehicle 1. As shown in Figure 3(a), the first virtual image 101 and the second virtual image 201 are arranged side by side in the vertical direction, with the first virtual image 101 above and the second virtual image 201 below. The lower end of the first displayable range R1 of the first virtual image 101 and the upper end of the second displayable range R2 of the second virtual image 201 are spaced apart by a distance D. The first virtual image 101 and the second virtual image 201 are spaced apart by a distance i.

[0087] Figure 3(b) shows the state before adjustment when switching from a small driver to a large driver. As the driver becomes larger, the viewpoint position rises, so from the perspective of the larger driver, the first virtual image 101 and the second virtual image 201 appear to be lower than in the case of Figure 3(a). Furthermore, the distance D1 between the lower end of the first displayable range R1 and the upper end of the second displayable range R2 becomes larger than the distance D. As a result, the distance i1 between the first virtual image 101 and the second virtual image 201 also increases beyond the distance i.

[0088] The control unit 500 acquires images captured by the camera 600, estimates the driver's viewpoint position from the captured images, and adjusts at least one of the first display position and the second display position by controlling the first moving unit 170 and the second moving unit 270 based on the estimation result.

[0089] Figure 3(c) shows the case where only the second display position (second displayable range R2) is adjusted. In this case, the first displayable range R1 remains unchanged from Figure 3(b), while the second displayable range R2 is positioned higher than in Figure 3(b). Here, it is preferable that the interval D2 between the lower end of the adjusted first displayable range R1 and the upper end of the second displayable range R2 is within a predetermined range. The predetermined range is within 30% of the difference between interval D1 and interval D, and more preferably within 20% of that difference. In this way, since the interval D2 between the lower end of the first displayable range R1 and the upper end of the second displayable range R2 is within a predetermined range, the interval i2 between the first virtual image 101 and the second virtual image 201 can also be reduced. It is preferable that the interval i2 is within a predetermined interval. The predetermined interval is within 30% of the difference between interval i1 and interval i, and more preferably within 20% of that difference.

[0090] This allows the first virtual image 101 and the second virtual image 201 to be displayed compactly in the vertical direction, thereby improving the user's visibility of the first virtual image 101 and the second virtual image 201.

[0091] Figure 3(d) shows the case where only the first display position (first displayable range R1) is adjusted. In this case, the second displayable range R2 remains unchanged from Figure 3(b), and the first displayable range R1 is positioned lower than in Figure 3(b). Here, it is preferable that the interval D3 between the lower end of the adjusted first displayable range R1 and the upper end of the second displayable range R2 is within a predetermined range, similar to the case of interval D2. The interval i3 between the first virtual image 101 and the second virtual image 201 should be within a predetermined interval.

[0092] Figure 3(e) shows the case where both the first and second display positions are adjusted. In this case, the second displayable range R2 is positioned higher than in Figure 3(b), and the first displayable range R1 is also positioned higher than in Figure 3(b). In particular, in Figure 3(e), both the first displayable range R1 and the second displayable range R2 are positioned at the same position as in Figure 3(b). This allows even taller drivers to position the first and second displayable ranges R1 and R2 within the same field of view as shorter drivers.

[0093] <Effects, etc.> As described above, according to this embodiment, the control unit 500 adjusts the display position of at least one of the multiple virtual images based on the estimated driver's viewpoint position. This adjustment suppresses positional shifts of the multiple virtual images caused by fluctuations in the driver's viewpoint position. Furthermore, since the final reflecting part (windshield 2) of the first display unit 100 and the final reflecting part (second reflector 260) of the second display unit 200 are separate components, the degree of design freedom for the optical configuration within the first display unit 100 and the second display unit 200 can be increased compared to the case where the final reflecting parts are common, making it possible to make the optical configuration more compact. In other words, the overall size of the display system 10 can be suppressed. As a result, a display system 10 can be provided that suppresses positional shifts of multiple virtual images while suppressing an increase in size.

[0094] Furthermore, since the control unit 500 adjusts the first display position of the first virtual image 101 and the second display position of the second virtual image 201, the relative positional misalignment between the first virtual image 101 and the second virtual image 201 can be suppressed with greater precision.

[0095] Furthermore, the control unit 500 adjusts at least one of the first display position and the second display position so that the intervals i2 and i3 between the first virtual image 101 and the second virtual image 201 are within a predetermined interval, thereby making the intervals i2 and i3 smaller than the predetermined interval. This allows the first virtual image 101 and the second virtual image 201 to be displayed compactly, improving the driver's visibility of the first virtual image 101 and the second virtual image 201.

[0096] Furthermore, the control unit 500 adjusts at least one of the first display position and the second display position so that the intervals D2 and D3 between the lower end of the first displayable range R1 and the upper end of the second displayable range R2 fall within a predetermined range. This allows the first virtual image 101 and the second virtual image 201 to be displayed compactly in the vertical direction. Therefore, when the driver views the first virtual image 101 and the second virtual image 201, vertical eye movement is suppressed, thereby improving visibility in the horizontal direction.

[0097] Furthermore, the control unit 500 controls the first moving unit 170 and the second moving unit 270 to adjust the first and second display positions, so that the position of the image within the display elements 130 and 230 remains constant while the first and second display positions can be adjusted by the first moving unit 170 and the second moving unit 270.

[0098] Furthermore, since the control unit 500 estimates the driver's viewpoint position based on the captured image obtained by the camera 600, the viewpoint position can be estimated with high accuracy using the captured image. Therefore, positional misalignment of the first virtual image 101 and the second virtual image 201 can be suppressed more accurately.

[0099] Furthermore, the reflectivity of the final reflecting part (second reflector 260) that forms the second virtual image 201 with a large downward angle is greater than the reflectivity of the final reflecting part (windshield 2) that forms the first virtual image 101. In other words, although the first virtual image 101 with a small downward angle is positioned directly in front of the driver, the reflectivity of the final reflecting part that forms the first virtual image 101 is smaller than the reflectivity of the final reflecting part that forms the second virtual image 201, thus ensuring a bright forward view for the driver.

[0100] In the conventional method, if the final reflective portion forming each virtual image is the same, it is necessary to divide that final reflective portion into multiple sections and apply a process to each section to give it a different reflectivity. In this embodiment, since the final reflective portion forming each virtual image is made of a different material, it is sufficient to use a material with a suitable reflectivity for each section. In other words, the aforementioned process becomes unnecessary, and manufacturing efficiency can be increased.

[0101] Furthermore, the optical paths from the display elements 130 and 230 to the final reflecting section (windshield 2, second reflector 260) of the first display unit 100 and the second display unit 200 are spatially independent. In other words, since the optical paths of the first display unit 100 and the second display unit 200 are spatially independent, maintenance can be easily performed.

[0102] Furthermore, the control unit 500 controls the first moving unit 170 and the second moving unit 270 to adjust the viewing distance of the first virtual image 101 and the second virtual image 201 by adjusting the first display position and the second display position. This allows the magnification of the first virtual image 101 and the second virtual image 201 to be changed, increasing the flexibility of the display effects.

[0103] Since the display elements 130 and 230 impart distortion to the image based on the adjustment amount for the first and second display positions, the display elements 130 and 230 display an image that takes into account the distortion of the first virtual image 101 and the second virtual image 201 that may occur after adjustment. Therefore, the first virtual image 101 and the second virtual image 201 based on this image can be displayed with distortion suppressed.

[0104] [Embodiment 2] In the following description, parts identical to those in Embodiment 1 may be denoted by the same reference numerals and their descriptions may be omitted. In the above embodiment, an example was given of a case with two display units. However, there may be three or more display units. Figure 4 is a schematic diagram showing the display system 10A according to Embodiment 2 installed in a vehicle 1. Figure 4 is a diagram corresponding to Figure 1. As shown in Figure 4, the display system 10A according to Embodiment 2 differs from Embodiment 1 in that a third display unit 300a is added.

[0105] The third display unit 300a is an electronic mirror type display device and is positioned in the upper center of the windshield 2 inside the vehicle. The display system 10A is equipped with a rear camera (not shown) that captures images of the area behind the vehicle 1. The third display unit 300a projects a third virtual image 301 based on the rear image obtained by the rear camera. Specifically, the third display unit 300a projects image light toward the driver. The driver perceives the image light that enters their eyes as the third virtual image 301 projected in the distance through the opening 321 (see Figure 5) of the third display unit 300a. Thus, the third display unit 300a is an example of a display unit that projects a virtual image in front of the driver. Specifically, the third display unit 300a projects the third virtual image 301 in front of the driver and above the first virtual image 101. Therefore, from the driver's perspective, the third virtual image 301, the first virtual image 101, and the second virtual image 201 are arranged vertically in that order from top to bottom. The absolute value of the elevation angle of the driver's line of sight relative to the third virtual image 301 is greater than the absolute value of the depression angle of the driver's line of sight relative to the first virtual image 101.

[0106] Figure 5 is a schematic diagram showing the third display unit 300a according to Embodiment 2. As shown in Figure 5, the third display unit 300a comprises a housing 320, a display element 330, a polarizing half mirror 340, a concave mirror 350, and a third moving unit 370. The display element 330, the polarizing half mirror 340, and the concave mirror 350 are examples of multiple optical members for projecting the third virtual image 301. The multiple optical members may include a hologram element.

[0107] The housing 320 is a box-shaped body formed from a light-shielding resin or metal. An opening 321 facing backward is formed at the rear of the housing 320 (the right direction in Figure 5 is defined as the rear or rear). Image light, which becomes the third virtual image 301, is projected from the opening 321. The internal space of the housing 320 houses a display element 330, a polarizing half mirror 340, a concave mirror 350, and a third moving part 370. The display element 330 and the third moving part 370 are controlled by the control unit 500.

[0108] The display element 330 is, for example, a liquid crystal panel. When light from a light source (not shown) is shone onto the display element 330, it displays an image that forms the basis of the third virtual image 301 and shines the image light of the said image onto the polarizing half mirror 340. The display element 330 may also be an organic EL panel. The display element 330 is positioned with its display surface facing downwards. Although not shown, a λ / 4 plate is laminated on the display surface of the display element 330. The λ / 4 plate is a λ / 4 phase difference plate that creates a phase difference of 1 / 4 of the wavelength λ in the light incident on the λ / 4 plate. For example, if the light emitted from the display surface is linearly polarized S-polarization, it is converted to circularly polarized light by passing through the λ / 4 plate.

[0109] The polarizing half-mirror 340 is configured to reflect P-polarized light and transmit S-polarized light, with a reflective polarizer plate arranged on a flat glass substrate. Furthermore, a λ / 4 plate is laminated on the surface of the polarizing half-mirror 340. The polarizing half-mirror 340 is positioned facing the display element 330 and the concave mirror 350. S-polarized image light emitted from the display element 330 is converted to circularly polarized light by the λ / 4 plate laminated on the display element 330 and directed toward the polarizing half-mirror 340. This circularly polarized image light is converted to P-polarized light by the λ / 4 plate laminated on the polarizing half-mirror 340 and reflected by the reflective polarizer plate of the polarizing half-mirror 340. The reflected P-polarized image light is converted back to circularly polarized light by passing through the λ / 4 plate again. Therefore, the polarizing half-mirror 340 is positioned such that the circularly polarized image light incident on the polarizing half-mirror 340 is reflected toward the concave mirror 350 as circularly polarized light by the λ / 4 plate and reflective polarizer plate laminated on the polarizing half-mirror 340.

[0110] The concave mirror 350 is positioned in front of the polarizing half mirror 340. The concave mirror 350 is positioned so that its concave surface, which is the reflective surface, faces backward. The circularly polarized image light reflected by the polarizing half mirror 340 is reflected again by the concave mirror 350 while remaining circularly polarized, and returns to the polarizing half mirror 340. The image light incident on the polarizing half mirror 340 is converted to S-polarization by the λ / 4 plate stacked on the polarizing half mirror 340, passes through the reflective polarizer of the polarizing half mirror 340, and proceeds to the rear in Figure 5. This image light passes through the opening 321 and goes towards the eyes of the driver sitting in the driver's seat, becoming the third virtual image 301. In other words, the concave mirror 350 is an example of a final reflecting part that reflects the image light towards the driver.

[0111] As mentioned above, the absolute value of the elevation angle of the driver's line of sight relative to the third virtual image 301 is greater than the absolute value of the depression angle of the driver's line of sight relative to the first virtual image 101. In other words, the first virtual image 101 is positioned directly in front of the driver more than the third virtual image 301. As a result, if the first virtual image 101 is bright, it is likely to obstruct the driver's view. To suppress this, the reflectivity of the concave mirror 350 is made greater than the reflectivity of the windshield 2. As a result, the first virtual image 101, formed by the image light reflected from the windshield 2, is displayed fainter than the third virtual image 301, formed by the image light reflected from the concave mirror 350. This ensures that the driver's forward view is bright.

[0112] In the third display unit 300a, the optical path of the image light from the display element 330 to the concave mirror 350 is located within the housing 320. Therefore, this optical path is spatially independent of the optical path of the image light from the display element 130 to the windshield 2 in the first display unit 100.

[0113] Figure 4 illustrates the position of the third virtual image 301 as seen from the driver's perspective. This position can be set by adjusting the viewing distance of the image light emitted from the display element 330 of the third display unit 300a.

[0114] The third moving unit 370 shown in Figure 5 is provided on the housing 320 of the third display unit 300a and adjusts the vertical position of the third display in the third virtual image 301. The third moving unit 370 adjusts the third display position by moving the concave mirror 350, thereby changing the optical path of the image light that forms the third virtual image 301. Specifically, the third moving unit 370 is equipped with a rotation mechanism and a drive motor for adjusting the orientation (tilt) of the concave mirror 350. The concave mirror 350 has, for example, the center of its reflective surface as the center of rotation and rotates clockwise or counterclockwise in a side view. This rotational movement is performed by the rotation mechanism and drive motor provided in the third moving unit 370. This changes the orientation of the concave mirror 350. As the orientation of the concave mirror 350 changes, the optical path of the image light that forms the third virtual image 301 is also changed, and the third display position of the third virtual image 301 is adjusted vertically.

[0115] The control unit 500 adjusts at least one of the first display position, second display position, and third display position by controlling the first moving unit 170, the second moving unit 270, and the third moving unit 370 based on the estimated viewpoint position. This control allows at least one of the first display position, second display position, and third display position to be adjusted to a position corresponding to the driver's viewpoint position. Preferably, the control unit 500 adjusts all of the first display position, second display position, and third display position to a position corresponding to the driver's viewpoint position.

[0116] As described above, even when the system includes a first display unit 100, a second display unit 200, and a third display unit 300a, it is possible to provide a display system 10A that can suppress the size increase while suppressing positional misalignment of at least one of the first virtual image 101, the second virtual image 201, and the third virtual image 301. The number of display units installed may be four or more. In any case, the control unit 500 only needs to adjust the display position of at least one of the virtual images projected by each display unit to a position corresponding to the driver's viewpoint, but it is preferable to adjust the display position of all virtual images.

[0117] [Embodiment 3] Figure 6 is a schematic diagram showing the display system 10B according to Embodiment 3 installed in the vehicle 1. Figure 6 corresponds to Figure 1. In Figure 6, the viewing distance to the lower end of the first virtual image 101c is shorter than the viewing distance to the upper end of the first virtual image 101c. Here, the first movable unit 170 provided in the first display unit 100 is capable of changing the orientation of the display element 130, which is an example of an optical member. The control unit 500 controls this first movable unit 170 and changes the orientation of the display element 130, thereby also changing the orientation of the first virtual image 101c. The control unit 500 may change the orientation of the first virtual image 101c based on the estimated viewpoint position.

[0118] Thus, the control unit 500 makes the viewing distance to the lower end of the first virtual image 101c shorter than the viewing distance to the upper end of the first virtual image 101c, allowing for the display of the first virtual image 101c that provides the driver with a more natural sense of depth. A similar configuration may be added to other display units.

[0119] [Embodiment 4] Embodiment 4 describes a case in which the control unit 500 adjusts the display position of the virtual image by adjusting the coordinate position of the image within the display element.

[0120] Figure 7 is an explanatory diagram showing the first virtual image 101 and the second virtual image 201 before and after the adjustment according to Embodiment 4. Figure 7 corresponds to Figure 3.

[0121] Figure 7(a) shows the first virtual image 101 and the second virtual image 201 before adjustment. This Figure 7(a) shows the case where a small driver is viewing the front of vehicle 1. The lower end of the first displayable range R1 of the first virtual image 101 and the upper end of the second displayable range R2 of the second virtual image 201 are spaced apart by a distance D. The first virtual image 101 and the second virtual image 201 are spaced apart by a distance i.

[0122] Figure 7(b) shows the state before adjustment when switching from a small driver to a large driver. As the driver becomes larger, the viewpoint position rises, so from the perspective of the larger driver, the first virtual image 101 and the second virtual image 201 appear to be lower than in the case of Figure 7(a). Furthermore, the distance D1 between the lower end of the first displayable range R1 and the upper end of the second displayable range R2 becomes larger than the distance D. As a result, the distance i1 between the first virtual image 101 and the second virtual image 201 also increases beyond the distance i.

[0123] The control unit 500 acquires images captured by the camera 600 and estimates the driver's viewpoint position from the captured images. Based on the estimation result, the control unit 500 adjusts the first display position of the first virtual image 101 and the second display position of the second virtual image 201 by adjusting the coordinate positions of the images within the display elements 130 and 230. Figure 7(c) shows the state after adjustment.

[0124] Specifically, as shown in Figure 7, the control unit 500 adjusts the coordinate position of the image within the display element 130 so that the first virtual image 101 is displayed at the first display position in Figure 7(a) within the first displayable range R1 without changing the first displayable range R1. Similarly, the control unit 500 adjusts the coordinate position of the image within the display element 230 so that the second virtual image 201 is displayed at the second display position in Figure 7(a) within the second displayable range R2 without changing the second displayable range R2.

[0125] As described above, the control unit 500 adjusts the first and second display positions by adjusting the coordinate positions of the images within the display elements 130 and 230, thus eliminating the need for fluctuations in the first displayable range R1 and the second displayable range R2. In other words, the moving parts (first moving part 170 and second moving part 270) can be omitted. Note that display position adjustment by the moving parts and display position adjustment by coordinate positions may be combined.

[0126] [Embodiment 5] Figure 8 is a schematic diagram showing the components of the display system 10C according to Embodiment 5, namely the first display unit 100c, the second display unit 200c, and the control unit 500. Figure 8 corresponds to Figure 2.

[0127] As shown in Figure 8, the first display unit 100c is equipped with a first housing movement unit 170c that moves the housing 110. The first housing movement unit 170c adjusts the first display position by changing the optical path of the image light that forms the first virtual image 101 by moving the housing 110. Specifically, the first housing movement unit 170c is equipped with a rotation mechanism and a drive motor for adjusting the orientation (tilt) of the housing 110. The housing 110 rotates clockwise or counterclockwise in a side view. This rotational movement is performed by the rotation mechanism and drive motor provided in the first housing movement unit 170c. This changes the orientation of the housing 110. As the orientation of the housing 110 changes, the optical path of the image light that forms the first virtual image 101 is also changed, and the first display position of the first virtual image 101 is adjusted in the vertical direction.

[0128] The second display unit 200c is equipped with a second housing movement unit 270c that moves the housing 220. The second housing movement unit 270c adjusts the second display position by changing the optical path of the image light that forms the second virtual image 201 by moving the housing 220. Specifically, the second housing movement unit 270c is equipped with a rotation mechanism and a drive motor for adjusting the orientation (tilt) of the housing 220. The housing 220 rotates clockwise or counterclockwise when viewed from the side. This rotational movement is performed by the rotation mechanism and drive motor provided in the second housing movement unit 270c. This changes the orientation of the housing 220. As the orientation of the housing 220 changes, the optical path of the image light that forms the second virtual image 201 is also changed, and the second display position of the second virtual image 201 is adjusted in the vertical direction.

[0129] The control unit 500 controls the first housing movement unit 170c and the second housing movement unit 270c based on the estimated viewpoint position to adjust the first display position and the second display position. In this way, the control unit 500 controls the first housing movement unit 170c and the second housing movement unit 270c to adjust the first display position and the second display position, so the optical path remains constant within the housings 110 and 220 before and after the adjustment. Therefore, optical distortion in the first virtual image 101 and the second virtual image 201 can be suppressed.

[0130] In this embodiment, the example illustrates the case where the first housing movement unit 170c and the second housing movement unit 270c rotate the housings 110 and 220, but sliding movement may also be used. Sliding movement is movement along at least one direction, such as the vertical direction, the front-back direction, or the optical axis direction. Rotational movement and sliding movement may be combined.

[0131] Furthermore, the display position adjustment by the housing movement unit and the display position adjustment by the movement unit may be combined. For example, the display position adjustment by the housing movement unit and the display position adjustment by the movement unit may be combined within a single display unit. Alternatively, for example, the first display unit may employ only the display position adjustment by the housing movement unit, while the second display unit may employ only the display position adjustment by the movement unit.

[0132] [Embodiment 6] Figure 9 is a schematic diagram showing the display system 10D according to Embodiment 6 installed in the vehicle 1. Figure 9 corresponds to Figure 1. As shown in Figure 9, the display system 10D is provided with an input unit 190d into which various instructions are input from the driver. The input unit 190d is, for example, a manually inputtable operation unit such as a touch panel or operation buttons. The input unit 190d may be located on the dashboard or on the steering wheel. The input unit 190d receives adjustment instructions to adjust the first display position of one virtual image (for example, the first virtual image 101) from among a plurality of virtual images. The adjustment instructions include a start instruction to begin adjusting the first display position, a move instruction to move the first display position, and a completion instruction. When a start instruction is input to the input unit 190d, the control unit 500 starts adjusting the first display position. When a move instruction is input to the input unit 190d, the control unit 500 controls the first move unit 170 to move the first display position to the position corresponding to the move instruction. When a completion instruction is received, the control unit 500 completes the adjustment of the first display position and performs the adjustment of the second display position.

[0133] Specifically, when a start command is input to the input unit 190d, the control unit 500 controls the display element 130 of the first display unit 100 to project a test image for adjustment as the first virtual image 101. The driver inputs a movement command to the input unit 190d while viewing the first virtual image 101, which is the test image.

[0134] The control unit 500 controls the first moving unit 170 based on a movement instruction to adjust the first display position of the first virtual image 101. At this time, the control unit 500 estimates the driver's viewpoint position from the amount of adjustment of the first virtual image 101. Specifically, since the installation coordinates of each optical element of the first display unit 100 are known, the control unit 500 can estimate the direction of illumination of the adjusted image light based on the installation coordinates of each optical element and the angle (amount of adjustment) of the second optical element 150 which has been changed by the first moving unit 170. Next, the control unit 500 estimates the viewpoint position as the intersection point between the driver's standard front-to-rear position in the vehicle 1 (center line of the vehicle's eye lips) and the illumination direction.

[0135] Based on the completion instruction, the control unit 500 controls the second moving unit 270 to adjust the second display position of the second virtual image 201 to correspond to the viewpoint position.

[0136] The control unit 500 estimates the viewpoint position based on the amount of adjustment to the first virtual image 101, which has been adjusted according to the adjustment instructions from the driver. In other words, the viewpoint position can be estimated even without a sensor (such as a camera 600) to detect the driver's viewpoint position. Furthermore, the control unit 500 adjusts the display position of the second virtual image 201 based on the estimated viewpoint position, so the second display position of the second virtual image 201 can be adjusted quickly and automatically.

[0137] Here, the input unit 190d may receive a fine-tuning instruction from the driver. In this case, the control unit 500 fine-tunes the second display position of the second virtual image 201 based on the fine-tuning instruction received in the input unit 190d. In this way, since the second display position of the second virtual image 201 is fine-tuned based on the fine-tuning instruction, the second display position of the second virtual image 201, which was previously adjusted automatically, can be fine-tuned to the position intended by the driver.

[0138] Furthermore, the display system 10D may include a detection unit for detecting the driver's viewpoint position. The detection unit may be the aforementioned camera 600, but may also include a seat position sensor for detecting the driver's seat position, a distance measuring sensor, etc. The control unit 500 corrects the estimated viewpoint position based on the detection result of the detection unit. As a result, the estimated viewpoint position is corrected based on the detection result of the detection unit, allowing for a more accurate estimation of the viewpoint position.

[0139] The input unit 190d may also be a voice sensor such as a microphone. In this case, the control unit 500 performs voice recognition processing on the voice input to the input unit 190d, thereby moving the first display position of the first virtual image 101 in the direction intended by the driver. Thus, since the input unit 190d is a voice sensor, the driver can input adjustment instructions by voice. Therefore, manual input by the user is not required when inputting adjustment instructions, and it is possible to prevent the user's driving operations from being interrupted by adjustment instructions.

[0140] [Embodiment 7] In the above embodiment 1, the control unit 500 adjusts at least one of the first display position and the second display position by controlling the first moving unit 170 and the second moving unit 270 based on the estimation result. However, depending on the estimation result, the amount of adjustment of the first display position and the second display position may exceed the adjustment range of the first moving unit 170 and the second moving unit 270. Specifically, this may occur when the driver's viewpoint rises or falls beyond the adjustment range.

[0141] In other words, if the amount of adjustment for the display position (first display position and second display position) for at least one virtual image (first virtual image 101 and second virtual image 201) determined based on the estimation result exceeds the adjustment range of the control unit 500, the control unit 500 will provide an over-notification.

[0142] The adjustment range in the first display position refers to the adjustment range of the first moving part 170, and more specifically, it refers to the rotation range of the second optical element 150 rotated by the first moving part 170.

[0143] In the case of the second display position, the adjustment range refers to the adjustment range of the second moving part 270, and more specifically, it refers to the rotation range of the second reflector 260 rotated by the second moving part 270.

[0144] An over-notification is a notification that informs the driver that the adjustment amount of the display position exceeds the adjustment range. The control unit 500 controls at least one of the first display unit 100 and the second display unit 200 to display an over-notification on at least one of the first virtual image 101 and the second virtual image 201. In this case, the control unit 500 and at least one of the first display unit 100 and the second display unit 200 constitute an example of a notification unit according to this disclosure. Other examples that can be included in the notification unit include a speaker capable of delivering an over-notification by voice, a light-emitting unit capable of delivering an over-notification by light, and a vibration device capable of delivering an over-notification by vibration. The vibration device may be in a form that transmits vibration to the driver, or it may be in a form that vibrates the display content of at least one of the first display unit 100 and the second display unit 200. Furthermore, the configuration for vibrating at least one of the display contents of the first display unit 100 and the second display unit 200 may be achieved, for example, by the control unit 500 controlling at least one of the first moving unit 170 and the second moving unit 270 to repeatedly perform a minute rotational movement of at least one of the second optical element 150 and the second reflector 260. Alternatively, the control unit 500 may repeatedly move at least one of the display contents of the display element 130 and the display element 230, for example, in the vertical direction. Alternatively, the control unit 500 may control a vibration actuator (not shown) to vibrate at least one of the entire first display unit 100 and the entire second display unit 200.

[0145] Thus, the control unit 500 performs an over-notification if the amount of adjustment for the display position of at least one virtual image determined based on the estimation result exceeds the adjustment range. As a result, the driver can recognize the adjustment range through the over-notification and move their viewpoint position by moving their head or other means according to the adjustment range.

[0146] Furthermore, the control unit 500 may also provide an over-notification if, when acquiring images captured by the camera 600 and estimating the driver's viewpoint position from those images, it estimates that the driver's gaze has moved horizontally and away from the eye box. This is because the first display unit 100 and the second display unit 200 do not support adjustment of the horizontal display position. Here, the point that represents the position of the driver's eyes in a normal driving state (reference eye point) varies depending on the driver's physique and posture, but even if the reference eye point changes, the range that most reference eye points fall within is called the eye box of the vehicle 1. The eye box is a virtual three-dimensional area.

[0147] [Embodiment 8] In the above embodiment 7, the control unit 500 provided an example of an over-notification when the amount of adjustment for the display position of at least one virtual image determined based on the estimation result exceeds the adjustment range of the control unit 500. In this embodiment 8, if the amount of adjustment for the display position of at least one virtual image determined based on the estimation result exceeds the adjustment range of the control unit 500, the control unit 500 adjusts the display position of at least one virtual image to the vicinity of the upper or lower limit of the adjustment range.

[0148] Here, we will explain in detail using the first display unit 100 as an example, but the same applies to the second display unit 200. If the amount of adjustment of the first display position of the first virtual image 101 determined by the estimation result exceeds the adjustment range, the control unit 500 controls the first display unit 100 to adjust the display position of the first virtual image 101 to the vicinity of the upper or lower limit of the adjustment range. Specifically, if the estimated driver's viewpoint position rises and exceeds the upper limit of the adjustment range, the control unit 500 adjusts the first display position of the first virtual image 101 to the vicinity of the upper limit of the adjustment range. The vicinity of the upper limit is a range of 20% from the upper limit to the adjustment range, preferably a range of 10% from the upper limit to the adjustment range.

[0149] On the other hand, if the estimated driver's viewpoint position descends and exceeds the lower limit of the adjustment range, the control unit 500 controls the first display unit 100 to adjust the first display position of the first virtual image 101 to the vicinity of the lower limit of the adjustment range. The vicinity of the lower limit is a range of 20% of the adjustment range from the lower limit, preferably a range of about 10% from the lower limit.

[0150] Thus, if the amount of adjustment for the display position of at least one virtual image determined based on the estimation result exceeds the adjustment range of the adjustment unit, the control unit 500 adjusts the display position of at least one virtual image to the vicinity of the upper or lower limit of the adjustment range. Therefore, even if the driver's viewpoint position is outside the adjustment range, the driver can recognize the virtual image positioned near the upper or lower limit of the adjustment range by moving their viewpoint position with the least possible head movement. In addition, the above-described over-notification may also be performed in Embodiment 8.

[0151] [Embodiment 9] Figures 10 and 11 are explanatory diagrams showing examples of the display of the first virtual image 101 and the second virtual image 201 according to Embodiment 9. In Figure 10, the second virtual image 201 contains image information captured by the vehicle's external camera. The external camera includes a front camera that captures the area in front of the vehicle 1. In this Embodiment 9, the second virtual image 201 contains the surrounding image (image information) of the area in front of the vehicle 1 captured by the front camera.

[0152] The control unit 500 detects warning targets around vehicle 1 by performing image processing on the surrounding image captured by the front camera. Warning targets are moving objects present around vehicle 1 (pedestrians, animals, vehicles other than vehicle 1 (cars, motorcycles, kick scooters, etc.)).

[0153] When the control unit 500 detects a warning target around the vehicle 1, it includes the warning target P captured by the front camera in the second virtual image 201 and projects it onto the second display unit 200. In this embodiment, an emphasis frame F is displayed superimposed on the surrounding image to make the warning target P stand out. The emphasis frame F is not displayed before the warning target around the vehicle 1 is detected based on the image data.

[0154] The control unit 500 acquires images captured by the camera 600 and estimates the driver's gaze from the captured images. Specifically, the control unit 500 applies predetermined image processing to the captured images to extract the driver's pupils and estimates the gaze from the tilt of the pupils.

[0155] The control unit 500 determines whether the estimated driver's gaze was directed towards the warning target P within a predetermined time after the warning target P is displayed on the second virtual image 201. In other words, the control unit 500 is an example of a determination unit according to this disclosure. The predetermined time is shorter than the time from when the warning target P is displayed until it becomes impossible to avoid contact with the warning target P. The predetermined time may be a fixed value set in advance, or it may be a variable value that changes depending on the situation.

[0156] If the control unit 500 determines that the driver's gaze is not directed toward the warning target P within a predetermined time, it provides a gaze guidance notification. Specifically, as shown in Figure 11, the control unit 500 controls the first display unit 100 to include a gaze guidance notification mark M in the first virtual image 101 at the position closest to the estimated driver's gaze direction within the displayable range of the first virtual image 101. In the example in Figure 11, since the estimated driver's gaze direction was to the upper left, the control unit 500 displays the mark M in the upper left of the first virtual image 101. The control unit 500 and the first display unit 100 are an example of a notification unit according to this disclosure. Through this notification by the mark M, the driver can recognize the presence of the warning target P and move their gaze toward the warning target P. Note that the gaze guidance notification mark M is not limited to being displayed at the position closest to the driver's gaze direction in the first virtual image 101, but may also be displayed conspicuously in the center of the first virtual image 101, for example.

[0157] The control unit 500 cancels the display of the guidance mark M when it detects that the estimated driver's gaze is directed towards the warning target P. This cancellation may be performed by the driver's voice or by the driver touching a button.

[0158] In this way, if the driver's gaze is not directed towards the warning target P within a predetermined time, a gaze guidance notification is issued, and this gaze guidance notification can guide the driver's gaze towards the warning target P.

[0159] Furthermore, the form of the eye-tracking notification can be anything as long as it directs the driver's gaze towards the warning target P. Other examples include a speaker capable of providing eye-tracking notification by sound, a light-emitting unit capable of providing eye-tracking notification by light, and a vibration device capable of providing eye-tracking notification by vibration. The vibration device may be in a form that transmits vibration to the driver, or it may be in a form that vibrates at least one of the display contents of the first display unit 100 and the second display unit 200. In the case of visual eye-tracking notification by a light-emitting unit, it is preferable that the eye-tracking notification is provided in the direction of the driver's gaze, as this makes it easier for the driver to notice the warning target P.

[0160] Furthermore, while this example illustrates a case where the control unit 500 determines whether the driver's gaze is directed towards the warning target P in the second virtual image 201, the control unit 500 may also determine whether the driver's gaze is directed towards the actual warning target. In this case, the control unit 500 determines whether the estimated driver's gaze is directed towards the coordinate position of the warning target included in the image information of the external camera. In addition to this, the control unit 500 may also determine whether the driver's gaze is directed towards the warning target by checking whether a part of the warning target included in the image information of the external camera is included within the effective field of view of a driver in a normal posture (within 30 degrees horizontally and within 20 degrees vertically). Furthermore, other warning targets may include warning targets reflected in the side mirrors or warning targets reflected in other display units.

[0161] (others) Although a display system relating to one or more embodiments of this disclosure has been described above based on embodiments, this disclosure is not limited to these embodiments. Without departing from the spirit of this disclosure, various modifications to these embodiments that a person skilled in the art could conceive of, or forms constructed by combining components from different embodiments, may also be included within the scope of one or more embodiments of this disclosure.

[0162] In the above embodiment, an example was given in which the first display position of the first virtual image 101 and the second display position of the second virtual image 201 are adjusted in the vertical direction based on the estimated viewpoint position. However, the first display position and the second display position may also be adjusted in the front-to-back direction or the vehicle width direction based on the estimated viewpoint position.

[0163] In the above embodiment, an example was given in which one moving part moves one optical element. However, at least two optical elements may be moved by one moving part. In this case, the device configuration can be simplified compared to the case in which one moving part is provided for each optical element to be moved.

[0164] In the above embodiment, an example was given in which one camera 600 is provided as a detection unit for detecting the driver's viewpoint position. However, multiple detection units may be provided. The multiple detection units may all be of the same type or may be of different types. Examples of detection units other than the camera 600 include a seat position sensor and a distance measuring sensor. In this case, the control unit 500 estimates the driver's viewpoint position based on the detection results of the multiple detection units. Since the driver's viewpoint position is estimated based on the detection results of the multiple detection units, the viewpoint position can be estimated with high accuracy. Therefore, positional shifts of multiple virtual images can be suppressed more accurately. The camera 600 may be a stereo camera. In this case, the three-dimensional position of the viewpoint can be detected more accurately.

[0165] Furthermore, the detection unit may estimate and detect the driver's viewpoint position from the angle of at least one of the side mirrors and the electronic rearview mirror. In this case, the viewpoint position can be detected with a simple configuration. [Industrial applicability]

[0166] This disclosure can be used in a display system for displaying virtual images. [Explanation of Symbols]

[0167] 1. 1Z Vehicle 2. Windshield (final reflector, optical components) 10, 10A, 10B, 10C, 10D display system 100, 100c First display unit (display unit, notification unit) 101, 101c First virtual image (virtual image) 109 Display unit body 110 cabinets 111 Opening 120 Cover section 130 Display elements (optical components) 140 First Optical Element (Optical Component) 150 Second optical element (optical component) 170 First moving part (moving part) 170c First enclosure movement section (enclosure movement section) 190d Input Section 200, 200c Second display section (display section, notification section) 201 Second Illusion (Illusion) 220 enclosures 221 Opening 230 Display elements (optical components) 240 Polarizing Half Mirror (Optical Component) 250 First reflecting mirror (optical component) 260 Second reflecting mirror (final reflecting section, optical component) 270 Second moving part (moving part) 270c Second chassis movement section (chassis movement section) 300a Third display section (display section) 301 Third Illusion (Illusion) 320 cabinets 321 Opening 330 Display elements (optical components) 340 Polarizing Half Mirror (Optical Component) 350 Concave mirror (final reflection section, optical component) 370 Third moving part (moving part) 500 Control Unit (Viewpoint Estimation Unit, Adjustment Unit, Judgment Unit, Notification Unit) 600 Camera (Detection Unit) D, D1, D2, D3, i, i1, i2, i3 Interval F Emphasis Frame M mark P Warning Target R1 First displayable range R2 Second displayable range

Claims

1. Multiple display units that project a virtual image in front of the user, A viewpoint estimation unit that estimates the user's viewpoint position, The system includes an adjustment unit that adjusts the display position of at least one virtual image among a plurality of virtual images based on the estimation result of the viewpoint estimation unit, Each of the aforementioned plurality of display units includes a display element that displays an image which is the basis of a virtual image and emits video light from the said image, The system includes a final reflecting unit that reflects the image light emitted from the display element toward the user, The final reflection units provided in the aforementioned plurality of display units are each separate components. Display system.

2. The adjustment unit adjusts the display position of each of the multiple virtual images based on the estimation result of the viewpoint estimation unit. The display system according to claim 1.

3. The adjustment unit adjusts the display position of at least one of the multiple virtual images so that the intervals between the multiple virtual images fall within a predetermined interval. The display system according to claim 1 or 2.

4. Multiple virtual images are arranged in a vertical line. The adjustment unit adjusts the display position of at least one of a plurality of virtual images so that the distance between the lower end of the displayable range of the upper virtual image and the upper end of the displayable range of the lower virtual image, which are adjacent to each other in the vertical direction, falls within a predetermined range. The display system according to claim 3.

5. A system comprising three or more of the aforementioned display units, The display system according to claim 4.

6. The aforementioned display unit is A plurality of optical members including the display element and the final reflection portion, The system includes a moving part that moves at least one of the plurality of optical members, The adjustment unit controls the moving unit to adjust the display position. The display system according to claim 1 or 2.

7. The moving part moves at least two or more of the plurality of optical members. The display system according to claim 6.

8. The adjustment unit controls the movement unit so that the viewing distance to the lower end of the virtual image is shorter than the viewing distance to the upper end of the virtual image. The display system according to claim 6.

9. The adjustment unit adjusts the display position by adjusting the coordinate position of the image within the display element. The display system according to claim 1 or 2.

10. It includes an input unit into which adjustment instructions from the user are input, The adjustment unit adjusts the display position of one virtual image based on the adjustment instructions from the user input to the input unit. The viewpoint estimation unit estimates the viewpoint position based on the adjustment amount for the one virtual image, The adjustment unit adjusts the display position of other virtual images based on the estimated viewpoint position. The display system according to claim 1 or 2.

11. The input unit receives the fine-tuning instructions from the user. The adjustment unit fine-tunes the display position of the other virtual image based on the fine-tuning instruction input to the input unit. The display system according to claim 10.

12. The system includes a detection unit for detecting the user's viewpoint position, The viewpoint estimation unit corrects the estimated viewpoint position based on the detection result of the detection unit. The display system according to claim 10.

13. The input unit is a sound sensor. The display system according to claim 10.

14. Equipped with a camera to photograph the aforementioned user, The viewpoint estimation unit estimates the user's viewpoint position based on the captured image obtained by the camera. The display system according to claim 1 or 2.

15. The system includes multiple detection units for detecting the user's viewpoint position, The viewpoint estimation unit estimates the user's viewpoint position based on the detection results of the plurality of detection units. The display system according to claim 1 or 2.

16. Of a pair of adjacent virtual images in the vertical direction, the reflectivity of the final emitting surface forming the virtual image with the larger absolute value of the depression or elevation angle is greater than the reflectivity of the final reflective surface forming the other virtual image. The display system according to claim 4.

17. Each of the multiple display units has an optical path from the display element to the final reflection unit that is spatially independent. The display system according to claim 1 or 2.

18. The aforementioned display unit is A plurality of optical members including the display element and the final reflection portion, A housing that accommodates at least a portion of the plurality of optical elements, The system includes a housing movement unit for moving the housing, The adjustment unit controls the housing movement unit to adjust the display position. The display system according to claim 17.

19. The adjustment unit adjusts the viewing distance of the virtual image by controlling the moving unit and adjusting the display position. The display system according to claim 6.

20. The display element imparts a distortion to the image based on an adjustment amount relative to the display position. The display system according to claim 1 or 2.

21. Equipped with a news department, The notification unit shall provide an over-notification if the amount of adjustment for the display position of the at least one virtual image, determined based on the estimation result of the viewpoint estimation unit, exceeds the adjustment range of the adjustment unit. The display system according to claim 1 or 2.

22. If the amount of adjustment for the display position of the at least one virtual image determined based on the estimation result of the viewpoint estimation unit exceeds the adjustment range of the adjustment unit, the adjustment unit adjusts the display position of the at least one virtual image to the vicinity of the upper or lower limit of the adjustment range. The display system according to claim 1 or 2.

23. The news department and, The system includes a determination unit that determines whether the user's gaze was directed towards the warning target within a predetermined time after the warning target was displayed on at least one of the plurality of display units, The notification unit, if the determination unit determines that the user's gaze is not directed towards the warning target within a predetermined time, will provide a gaze guidance notification. The display system according to claim 1 or 2.

Images