Display control device, display system, and display control method

Abstract

The present invention provides a display in which it is easy to ascertain a planned travel path.　This display control device displays, in a first display region 710, a real-view-following navigation image 210 that follows a specific superimposition target in a real view so as to maintain the positional relation with the specific superimposition target. When it is inferred that visibility in a forward view from a vehicle will be impaired, the display control device displays, in a second display region 720 that has a vertical center 720A which is disposed lower than the vertical center 710A of the first display region 710, a bird's-eye-view navigation image 220 which includes, in a diagonal view from above, at least a map image 221 that is of the surrounding area of the vehicle and a path image 222 that overlaps with the map image 221 and that indicates a planned travel path.

Description

[Technical Field]

[0001] The present disclosure relates to a display control device, a display system, and a display control method that are used in a vehicle and allow an image to be visually recognized by superimposing the image on a foreground of the vehicle. [Background Art]

[0002] A head-up display (HUD: Head Up Display) device displays an image (virtual object) superimposed on a landscape in front of a host vehicle, thereby expressing Augmented Reality (AR) in which information is added or emphasized to an actual scene or an overlay target existing in the actual scene. This device can contribute to safe and comfortable vehicle operation by minimizing the movement of the line of sight of a user driving the vehicle and accurately providing desired information.

[0003] In particular, the head-up display device described in Patent Document 1 displays a plurality of image elements (virtual objects) that are arranged along a road surface and guide a planned travel route expressed with different perspectives, thereby making the image (virtual object) and the actual scene (a predetermined position of the road surface based on the vehicle) more harmonious, and transmitting the route to an observer in an easy-to-understand manner. [Prior Art Documents] [Patent Documents]

[0004] [Patent Document 1] International Publication No. 2020 / 009219 [Summary of the Invention] [Problem to be Solved by the Invention]

[0005] When displaying virtual objects (images) that correspond to the position of real-world scenery (superimposed objects), for example, if another vehicle is present ahead, the virtual objects will remain visible to the observer, but the real-world scenery (superimposed objects) that are obscured by the other vehicle will no longer be visible to the observer. It is anticipated that relying solely on virtual objects that guide the planned route by being visible together with the real-world scenery will make it difficult to intuitively grasp the route. Therefore, there was room for improvement in providing a display that is easy for occupants to understand. [Means for solving the problem]

[0006] A summary of specific embodiments disclosed herein is provided below. It should be understood that these embodiments are presented solely to provide the reader with an overview of these specific embodiments and do not limit the scope of this disclosure. In fact, this disclosure may encompass various embodiments not described below.

[0007] This disclosure outlines the provision of easily understandable displays for occupants. More specifically, it also concerns providing displays that make it easy to understand the planned route.

[0008] Accordingly, in the first embodiment described herein, a display control device for controlling one or more image display units that display an image showing a planned driving route comprises one or more processors, the one or more processors acquire navigation information related to the planned driving route, and using at least the navigation information to display a real-scene tracking navigation image in a first display area that follows a specific superimposed object so as to maintain a positional relationship with a specific superimposed object in the real-scene, and acquire at least one of other vehicle information related to the position of other vehicles, visibility information related to the distance that can be seen ahead, and weather information as forward visibility information, and if it is estimated based at least the forward visibility information that the visibility of the foreground of the own vehicle is obstructed, the device causes a second display area having a vertical center located below the vertical center of the first display area to display an overhead navigation image that includes at least a map image of the area around the own vehicle and a route image that overlaps with the map image and shows the planned driving route, and these are viewed at an oblique angle. The system detects situations where specific superimposed objects associated with the real-world navigation image are difficult to see due to other vehicles, and displays an overhead navigation image. This allows route guidance to continue even if the visibility of the difficult-to-see real-world navigation image decreases or it disappears. Furthermore, even when the real-world navigation image continues to be displayed, checking it together with the overhead navigation image makes it easier to grasp the sense of distance in the real-world navigation image.

[0009] Furthermore, in a second embodiment that may be dependent on the first embodiment, the first display area is an area that overlaps with the actual view seen through the front windshield of the vehicle 1, and the second display area is an area located below the first display area and that does not overlap with the actual view seen through the front windshield. The first display area is an area in the front windshield where visible light is transmitted and the actual view is visible, for example, within the display area of ​​the head-up display device (image display unit). On the other hand, the second display area is an area that does not overlap with the actual view seen through the front windshield, for example, an area that overlaps with the shielding part of the front windshield or the dashboard. In a broader sense, the second display area only needs to be mostly within an area that does not overlap with the actual view seen through the front windshield, and may also include a portion (for example, less than 30% of the second display area) that is outside the area that does not overlap with the actual view seen through the front windshield.

[0010] Furthermore, in a third embodiment that may be dependent on the second embodiment, the second display area is within the area that overlaps with the shielding portion of the front windshield that obstructs the view from inside the vehicle. The second head-up display device (an example of an image display unit) can display a virtual image behind the front windshield by projecting the light of the image displayed by the display onto the shielding portion.

[0011] In other embodiments that may be dependent on the second embodiment, the image display unit may be a display panel provided between the dashboard and the front windshield. The image display unit is a display panel provided on the dashboard on the observer side of the light-transmitting portion of the head-up display device.

[0012] Furthermore, the image display unit is a display panel located on the dashboard behind the light-transmitting section of the head-up display device (at a position further away from the observer). The second display area in which the image display unit displays the overhead navigation image is positioned in an area that does not overlap with the actual view seen through the front windshield, when viewed from the center of the head-up display device's eyebox.

[0013] Furthermore, in a fourth embodiment that may be dependent on the first embodiment, the first display area is an area that overlaps with the actual view seen through the vehicle's front windshield, and the second display area is an area that overlaps with the actual view seen through the front windshield, and is positioned so that its upper edge is above the lower edge of the first display area.

[0014] Furthermore, in a fifth embodiment which may be dependent on any one of the first to third embodiments, the processor further performs a first movement process to move the real-scene tracking navigation image in a first movement direction which includes at least the left or right direction in which other vehicles are moving relative to the state where the visibility of the foreground of the vehicle is obstructed, and a process to hide the real-scene tracking navigation image during or before the first movement process.

[0015] Furthermore, in a fifth embodiment which may be dependent on the fourth embodiment, the processor changes the movement speed of the real-scene tracking navigation image in a monotonically non-decreasing manner with respect to the relative movement speed of other vehicles during the first movement process.

[0016] Furthermore, in a sixth embodiment which may be dependent on the fifth embodiment, the processor sets the movement speed of the real-scene tracking navigation image linearly with respect to the relative movement speed of other vehicles in the first movement processing.

[0017] Furthermore, in a seventh embodiment which may be dependent on the fifth embodiment, the processor sets the movement speed of the real-scene tracking navigation image non-linearly with respect to the relative movement speed of other vehicles in the first movement processing.

[0018] Furthermore, in an eighth embodiment which may be dependent on any one of the fourth to seventh embodiments, the processor further performs a second movement process to move the overhead navigation image to a second display area in a second movement direction which includes either the left or right direction included in the first movement direction, if it is estimated that the visibility of the foreground of the vehicle is obstructed, and a visibility enhancement process to improve the visibility of the overhead navigation image during or before the second movement process.

[0019] Furthermore, in a ninth embodiment which may be dependent on any one of the fourth to seventh embodiments, the processor further performs a third movement process to move the overhead navigation image to a second display area in a third movement direction which includes either the left or right direction not included in the first movement direction, if it is estimated that the visibility of the foreground of the vehicle is obstructed, and a visibility enhancement process to improve the visibility of the overhead navigation image during or before the third movement process.

[0020] Furthermore, in a tenth embodiment which may be dependent on any one of the fourth to seventh embodiments, the processor further performs a fourth movement process to move the overhead navigation image to a second display area in a fourth movement direction which includes either the upward or downward direction, not the left or right direction, if it is estimated that the visibility of the foreground of the vehicle is obstructed, and a visibility enhancement process to improve the visibility of the overhead navigation image during or before the fourth movement process.

[0021] Furthermore, in an eleventh embodiment which may be dependent on any one of the first to ten embodiments, the processor hides the overhead navigation image and displays the real-world-following navigation image in the first display area when it is estimated that the visibility of the vehicle's foreground is not obstructed.

[0022] Furthermore, in a twelfth embodiment which may be dependent on any one of the first to ten embodiments, the processor further performs a fifth movement process to move the overhead navigation image from the second display area, and a visibility reduction process to reduce the visibility of the overhead navigation image during or before the fifth movement process, if it is estimated that the visibility of the foreground of the vehicle is not obstructed.

[0023] Furthermore, in a thirteenth embodiment which may be dependent on any one of the first to twelfth embodiments, the processor further performs a sixth movement process which moves the real-scene tracking navigation image in a second movement direction which includes at least the left or right direction in which other vehicles are moving relative to the state, when it is estimated that the visibility of the foreground of the vehicle is not obstructed, and a process which improves the visibility of the real-scene tracking navigation image during or before the sixth movement process.

[0024] Furthermore, in a fourteenth embodiment, which may be dependent on any one of the eleventh to thirteenth embodiments, the second determination area is wider than the first determination area.

[0025] Furthermore, in the display control device of a fifteenth embodiment, which may be dependent on any one of the first to fourteenth embodiments, the processor determines a forward visibility condition based on visibility information and weather information, and reduces the brightness of the bird's-eye view navigation image as the forward visibility condition deteriorates. As the forward visibility condition deteriorates, the foreground becomes harder to visually recognize. Accordingly, the visual attention of the observer is more likely to be directed to the image (the bird's-eye view navigation image) than to the foreground whose visibility is relatively reduced, and consequently, it is also expected that the observer may find the image bothersome. According to the fifteenth embodiment, by reducing the brightness of the bird's-eye view navigation image as the forward visibility condition deteriorates, it is possible to reduce an excessive shift of the observer's visual attention to the image (the bird's-eye view navigation image).

[0026] In the display control device of the fifteenth embodiment, the processor determines a forward visibility condition based on visibility information and weather information, displays a real scene following navigation image in a first display area when it is determined that the condition is a first forward visibility condition, displays the bird's-eye view navigation image in a second display area at a first brightness when it is determined that the condition is a second forward visibility condition in which the foreground is harder to visually recognize than in the first forward visibility condition, and displays the bird's-eye view navigation image in the second display area at a second brightness lower than the first brightness when it is determined that the condition is a third forward visibility condition in which the foreground is harder to visually recognize than in the second forward visibility condition.

[0027] Note that in the display control device of the fifteenth embodiment, the processor may continuously reduce the brightness of the bird's-eye view navigation image according to the degree of deterioration of the forward visibility condition. The processor may quantify the degree of deterioration of the forward visibility condition, and linearly reduce the brightness of the bird's-eye view navigation image with respect to the numerical value of the degree of deterioration of the forward visibility condition (provided that the brightness may be changed non-linearly). In another example, the processor may reduce the brightness of the bird's-eye view navigation image stepwise according to the degree of deterioration of the forward visibility condition.

[0028] Furthermore, in a display control device of a 16th embodiment which may be dependent on any one of the first to 15 embodiments, the processor determines the forward visibility state based on visibility information and weather information. If it is determined to be a first forward visibility state, it displays a real-scene tracking navigation image in a first display area. If it is determined to be a second forward visibility state in which the foreground is less visible than in the first forward visibility state, it displays an overhead navigation image in a second display area with a first brightness. If it is determined to be a third forward visibility state in which the foreground is less visible than in the second forward visibility state, it displays an overhead navigation image in a second display area with a second brightness lower than the first brightness. If it is determined to be a fourth forward visibility state in which the foreground is less visible than in the third forward visibility state, it displays an overhead navigation image in a third display area having a vertical center located below the vertical center of the second display area. As the forward visibility state deteriorates, the foreground becomes less visible. As a result, the observer's visual attention is more likely to be directed towards the image (overhead navigation image) than towards the foreground, which has relatively reduced visibility, and consequently, it is conceivable that the observer may find the image bothersome. According to the 16th embodiment, as the forward visibility deteriorates, the area in which the overhead navigation image is displayed is lowered, making it less likely for the image (overhead navigation image) to enter the field of view of an observer facing forward (in other words, it moves away from the central field of view), thereby reducing the tendency to focus too much on the image (overhead navigation image) when the forward visibility deteriorates.

[0029] Furthermore, in the 17th embodiment of the display control device, which may be dependent on any one of the first to 16 embodiments, the processor may adjust the ease of switching between various processes based on the operation information in the operation unit. The various processes include (1) a process to display the overhead navigation image in the second display area, (2) a process to hide the overhead navigation image, (3) a process to lower the brightness of the overhead navigation image, (4) a process to increase the brightness of the overhead navigation image, and (5) a process to display the overhead navigation image in the third display area. The ease of switching between various processes can be adjusted by relaxing or tightening the switching conditions (parameter conditions, time conditions). According to the 17th embodiment, the user can adjust the switching conditions as appropriate.

[0030] Furthermore, in the 18th embodiment of the display control device, which may be dependent on any one of the first to 17 embodiments, the processor may switch the display / hide of the overhead navigation image based on the operation information in the operation unit. The processor may also automatically switch the display / hide of the real-scene tracking navigation image in conjunction with the switching of the display / hide of the overhead navigation image. Specifically, the processor may automatically hide the real-scene tracking navigation image when displaying the overhead navigation image, and automatically display the real-scene tracking navigation image when hiding the overhead navigation image.

[0031] Furthermore, in a display control device of the 19th embodiment which may be dependent on any one of the first to 18 embodiments, the processor determines the forward visibility state based on visibility information and weather information, and if it is determined to be a first forward visibility state, it displays the overhead navigation image in the second display area with a first brightness, and if a fifth forward visibility state in which the foreground is easier to see than the first forward visibility state continues for a predetermined time or longer, it hides the overhead navigation image and displays the real-scene tracking navigation image in the first display area.

[0032] Furthermore, the display system of the 20th embodiment includes a display control device according to any one of the first to 19 embodiments, and a head-up display device that displays a virtual image of the planned driving route in a first display area and a second display area. [Brief explanation of the drawing]

[0033] [Figure 1] This figure shows examples of applications of a vehicle display system to a vehicle according to several embodiments. [Figure 2] This is a block diagram of a vehicle display system according to several embodiments. [Figure 3] This figure shows the area of ​​the map image surrounding the vehicle, as displayed as an overhead navigation image, according to several embodiments. [Figure 4] This diagram conceptually shows a first determination region and a second determination region when the surface is viewed from above, according to several embodiments. [Figure 5]This figure shows an example of displaying a real-scene tracking navigation image in a first display mode in several embodiments. [Figure 6A] This figure shows an example of displaying an overhead navigation image in a second display mode in some embodiments. [Figure 6B] This figure shows an example of displaying an overhead navigation image in a second display mode in some embodiments. [Figure 7] This figure shows examples of how the real-scene tracking navigation image is displayed when switching from the first display mode to the second display mode in some embodiments. [Figure 8] This figure shows examples of how the real-scene tracking navigation image is displayed when switching from the first display mode to the second display mode in some embodiments. [Figure 9] This diagram illustrates the arrangement of a first display area and a second display area according to several embodiments. [Figure 10] This figure shows an example of the arrangement of a second display area according to several embodiments. [Figure 11A] This figure shows the configuration of the image display unit in a display system according to several embodiments. [Figure 11B] This figure shows the configuration of the image display unit in a display system according to several embodiments. [Figure 11C] This figure shows the configuration of the image display unit in a display system according to several embodiments. [Figure 12] This figure illustrates the arrangement of a third display area according to several embodiments. [Figure 13] This figure illustrates the arrangement of a third display area according to several embodiments. [Modes for carrying out the invention]

[0034] Figures 1 to 13 below provide a description of the configuration of an exemplary display system. However, the present invention is not limited to the following embodiments (including those shown in the drawings). Modifications (including the deletion of components) can be made to the embodiments described below. Furthermore, in order to facilitate understanding of the present invention, explanations of known technical matters will be omitted as appropriate.

[0035] Refer to Figure 1. The vehicle display system 10 consists of an image display unit 20, a display control device 30 that controls the image display unit 20, a locator 410, a GNSS device 411, an IMU 412, a map information storage device 420, a navigation device 430, a surrounding monitoring sensor 440, a person detection device 450, an external communication connection device 460, a visibility detection unit 470, a weather database 480, a weather sensor 485, and an operation unit 490. In this description of the embodiment, the left-right direction when a driver 4 seated in the driver's seat of the vehicle 1 is facing forward of the vehicle 1 is defined as the X-axis (left is the positive X-axis direction), the up-down direction as the Y-axis (up is the positive Y-axis direction), and the front-back direction as the Z-axis (forward is the positive Z-axis direction).

[0036] In one embodiment of the vehicle display system 10, the image display unit 20 is a head-up display (HUD) device 20A installed in the dashboard 5 of the vehicle 1. The HUD device 20A emits display light 20a through a light-transmitting unit 26 toward the front windshield 2 (an example of a projection target member), and allows the image to be viewed in a display area 110 located in front of the front windshield 2 (an example of a projection target member) (positive Z-axis direction). When viewed from inside the eye box EB, the driver 4 can see the virtual image displayed by the HUD device 20A within the display area 110.

[0037] In the description of this embodiment, the term "eyebox" refers to: (1) an area within which the entire virtual image of the image can be seen, but outside the area, at least a portion of the virtual image of the image cannot be seen; (2) an area within which at least a portion of the virtual image of the image can be seen, but outside the area, no portion of the virtual image of the image can be seen; (3) an area within which at least a portion of the virtual image of the image can be seen with a brightness of a predetermined level or higher, but outside the area, the entire virtual image of the image is below the predetermined brightness level; or (4) if the HUD device 20A is capable of displaying a stereoscopic virtual image, an area within which at least a portion of the virtual image can be seen stereoscopically, but outside the area, no portion of the virtual image can be seen stereoscopically. In other words, if the observer places their eyes (both eyes) outside the eyebox EB, the observer will not be able to see the entire virtual image of the image, the visibility of the entire virtual image of the image will be very low and difficult to perceive, or the virtual image of the image will not be able to be seen stereoscopically. The predetermined brightness level is, for example, about 1 / 50 of the brightness level of the virtual image of the image seen at the center of the eyebox. The "eyebox" is set to be the same as, or to include most of (for example, 80% or more of) the area where the observer's viewpoint is expected to be located in a vehicle equipped with the HUD device 20A.

[0038] The display area 110 is a plane, curved surface, or partially curved surface area on which the image generated inside the HUD device forms a virtual image, and is also called the image-forming surface. The display area 110 itself has low visibility to the extent that it is not actually visible to the driver 4, or is difficult to see.

[0039] The image display unit 20 includes a display unit 22 having a display surface for displaying images, and a relay optical system 24. The display unit 22 may be a transmissive display that transmits light from a backlight such as an LCD, or a projection display that projects images onto a screen. In these cases, the display surface is the display surface in a transmissive display, or the screen in a projection display.

[0040] The relay optical system 24 is arranged in the optical path of the image light from the display 22 between the display 22 and the front windshield 2, and consists of one or more optical members that project the image light from the display 22 onto the front windshield 2 outside the image display unit 20. The relay optical system 24 includes at least one concave mirror, but may also include, for example, one or more refractive optical members such as lenses, diffractive optical members such as holograms, reflective optical members, or a combination thereof.

[0041] The image display unit 20 may also be a head-mounted display (HMD) device 20B. The driver 4 wears the HMD device 20B on their head and sits in the seat of their vehicle 1, thereby viewing the displayed image superimposed on the real-world scenery 300 through the front windshield 2 of their vehicle 1. The display area 110 in which the vehicle display system 10 displays a predetermined image is fixed (adjustably positioned) at a specific location relative to the coordinate system of the vehicle 1, and when the driver 4 faces that direction, they can view the image displayed within the display area 110 fixed at that specific location.

[0042] The image display unit 20, based on the control of the display control device 30, can also make the observer (typically the driver seated in the driver's seat of the vehicle 1) 4 perceive visual augmented reality (AR) by displaying an image near superimposed objects (an example of a specific positional relationship between the image and the superimposed object), at a position overlapping the superimposed object (an example of a specific positional relationship between the image and the superimposed object), or at a position set relative to the superimposed object (an example of a specific positional relationship between the image and the superimposed object), which exist in the real space (real scene) 300 that is visible through the front windshield 2 of the vehicle 1, such as obstacles (pedestrians, bicycles, motorcycles, other vehicles, etc.), driving lanes 310, road signs, and features (buildings, bridges, etc.), or at a position overlapping the superimposed object (an example of a specific positional relationship between the image and the superimposed object). The image display unit 20 can display images including AR images whose display position changes according to the position of the superimposed object, and / or non-AR images whose display position does not change according to the position of the superimposed object.

[0043] Figure 2 is a block diagram of a vehicle display system 10 according to several embodiments. The display control device 30 comprises one or more I / O interfaces 31, one or more processors 33, one or more image processing circuits 35, and one or more memories 37. The various functional blocks shown in Figure 2 may consist of hardware, software, or a combination of both. Figure 2 is only one embodiment, and the illustrated components may be combined with fewer components, or additional components may be included. For example, the image processing circuit 35 (e.g., a graphics processing unit) may be included in one or more processors 33.

[0044] As shown in the figure, the processor 33 and the image processing circuit 35 are operably connected to the memory 37. More specifically, the processor 33 and the image processing circuit 35 can operate the vehicle display system 10, for example, by executing a program stored in the memory 37, such as generating and / or transmitting image data. The processor 33 and / or the image processing circuit 35 may include at least one general-purpose microprocessor (e.g., a central processing unit (CPU)), at least one application-specific integrated circuit (ASIC), at least one field-programmable gate array (FPGA), or any combination thereof. The memory 37 includes any type of magnetic medium such as a hard disk, any type of optical medium such as CDs and DVDs, any type of semiconductor memory such as volatile memory, and non-volatile memory. The volatile memory may include DRAM and SRAM, and the non-volatile memory may include ROM and NVROM.

[0045] As shown in the figure, the processor 33 is operablely connected to the I / O interface 31. The I / O interface 31 communicates (also referred to as CAN communication) with the vehicle ECU (described later) and other electronic devices (codes 401 to 420 described later) installed in the vehicle, for example, in accordance with the CAN (Controller Area Network) standard. The communication standard adopted by the I / O interface 31 is not limited to CAN, but includes wired communication interfaces such as CANFD (CAN with Flexible Data Rate), LIN (Local Interconnect Network), Ethernet (registered trademark), MOST (Media Oriented Systems Transport: MOST is a registered trademark), UART, or USB, or in-vehicle communication (internal communication) interfaces, which are short-range wireless communication interfaces within tens of meters, such as personal area networks (PANs) such as Bluetooth (registered trademark) networks and local area networks (LANs) such as 802.11x Wi-Fi (registered trademark) networks. Furthermore, the I / O interface 31 may also include an external communication interface for outside vehicles, such as a wide-area communication network (e.g., an internet communication network) using cellular communication standards such as wireless wide-area network (WAN0, IEEE802.16-2004 (WiMAX: Worldwide Interoperability for Microwave Access)), IEEE802.16e-based (Mobile WiMAX), 4G, 4G-LTE, LTE Advanced, and 5G.

[0046] As shown in the figure, the processor 33 is interconnected with the I / O interface 31 so as to be able to exchange information with various other electronic devices connected to the vehicle display system 10 (I / O interface 31). For example, a navigation device, a vehicle ECU, and external communication devices installed in the vehicle 1 are interconnected with the I / O interface 31 so as to be able

[0047] The locator 410 shown in Figure 2 mainly consists of a microcomputer equipped with a processor, RAM, memory, input / output interface, and bus connecting these components. The locator 410 sequentially determines the vehicle's position and direction of travel by combining positioning data received by the GNSS device 411, measurement results from the IMU 412, and vehicle speed information output to the I / O interface 31. The locator 410 provides the vehicle's position and direction information based on the positioning results to the display control device 30, etc.

[0048] The GNSS device 411 shown in Figure 2 receives positioning signals transmitted from artificial satellites (positioning satellites), detects the current position of the vehicle, and outputs the detected position data to the locator 410. The GNSS device 411 is capable of receiving positioning signals from each positioning satellite of at least one of the following satellite positioning systems: GPS, GLONASS, Galileo, IRNSS, QZSS, Beidou, etc.

[0049] The IMU412 is an Inertial Measurement Unit, which includes, for example, a gyro sensor and an accelerometer, and detects angular velocity in the yaw, pitch, and roll directions, outputting the detected angular velocity data to the locator 410. The locator 410 detects changes in the vehicle's attitude by detecting the angular velocity in the yaw, pitch, and roll directions.

[0050] The map information storage device 420 is a storage medium that stores map data including the area around the vehicle. Based on the location and direction information of the vehicle 1 determined by the locator 410, it outputs map data of the area around the vehicle's current location to the display control device 30. The map information storage device 420 also outputs the relative position (distance and direction) of superimposed objects (feature points such as buildings and road junctions) to the display control device 30 based on the location and direction information of the vehicle 1 determined by the locator 410. Figure 1 shows an example where the map information storage device 420 is externally attached to the display control device 30, but the map information storage device 420 may be built into the display control device 30. The map information storage device 420 may also be a server that distributes map information to the display control device 30 via communication from outside the vehicle. The map information storage device 420 can acquire the latest map data from a server located in the cloud.

[0051] The navigation device 430 is an in-vehicle device that provides route guidance to a destination set by the driver or other user. The navigation device 430 is equipped with a navigation map database (not shown). The navigation map database is mainly composed of non-volatile memory and stores map data used for route guidance (hereinafter referred to as navigation map data). The navigation map data contains link data and node data for roads, etc. The navigation device 430 provides navigation information indicating the content of the route guidance to the display control device 30. The navigation information includes, for example, location information and road shape information for intersections or junctions, as well as direction information indicating the direction the vehicle should proceed at intersections or junctions.

[0052] The surrounding monitoring sensor 440 detects superimposed objects present in the vicinity (front, side, and rear) of the vehicle 1. The superimposed objects detected by the surrounding monitoring sensor 440 may include, for example, obstacles (pedestrians, bicycles, motorcycles, other vehicles, etc.), the road surface of the driving lane, lane markings, roadside objects, and / or ground objects (buildings, etc.), as described later. The external sensor consists of, for example, a detection unit consisting of a radar sensor such as a millimeter-wave radar, ultrasonic radar, or laser radar, a camera, or any combination thereof, and a processing unit that processes (data fuses) the detection data from one or more of these detection units. Conventional and well-known methods are applied to object detection using these radar sensors and camera sensors. These sensors can detect objects, and if superimposed objects exist, they can detect the presence or absence of superimposed objects in three-dimensional space, and if superimposed objects exist, their position (relative distance from vehicle 1, position in the left-right direction when vehicle 1's direction of travel is the front-back direction, position in the up-down direction, etc.), size (size in the lateral direction (left-right direction), height direction (up-down direction), etc.), direction of movement (lateral direction (left-right direction), depth direction (front-back direction)), speed of change (lateral direction (left-right direction), depth direction (front-back direction)), and / or type of superimposed object. One or more surrounding monitoring sensors 440 can detect superimposed objects in front of vehicle 1 at each detection cycle of the sensor and output superimposed object information (information such as the presence or absence of superimposed objects, and if superimposed objects exist, the position, size, and / or type of each superimposed object) to the processor 33. This superimposed object information may also be transmitted to the processor 33 via other equipment (for example, vehicle ECU 401). Furthermore, when using a camera as a sensor, an infrared camera or near-infrared camera is preferable so that the superimposed object can be detected even in dark surroundings such as at night. Also, when using a camera as a sensor, a stereo camera that can acquire distance and other parameters through parallax is preferable.

[0053] The observer detection device 450 may include a camera, such as an infrared camera, that detects the eye position of an observer seated in the driver's seat of vehicle 1, and may output the captured image to the processor 33. The processor 33 may acquire the captured image (an example of information that can estimate the eye position) from the observer detection device 450, and by analyzing this captured image using a method such as pattern matching, detect the coordinates of the observer's eye position, and output a signal indicating the detected eye position coordinates to the processor 33.

[0054] Furthermore, the observer detection device 450 may output to the processor 33 the analysis results obtained by analyzing the camera's captured image (for example, a signal indicating where the observer's eye position belongs in the spatial region corresponding to a set of multiple display parameters). Note that the method for obtaining the observer's eye position in vehicle 1, or information that can estimate the observer's eye position, is not limited to these, and may be obtained using known eye position detection (estimation) techniques.

[0055] Furthermore, the observer detection device 450 may detect the rate of change of the observer's eye position and / or direction of movement, and output a signal indicating the rate of change of the observer's eye position and / or direction of movement to the processor 33.

[0056] The external communication device 460 is a communication device that exchanges information with the vehicle 1, and is, for example, a communication device that connects with the vehicle 1 via vehicle-to-vehicle communication (V2V: Vehicle To Vehicle), a communication device that connects with a pedestrian via vehicle-to-pedestrian communication (V2P: Vehicle To Pedestrian) (a portable information terminal carried by a pedestrian), or a communication device that connects via vehicle-to-roadside infrastructure (V2I: Vehicle To Roadside Infrastructure). In a broader sense, it includes everything that is connected via communication with the vehicle 1 (V2X: Vehicle To Everything). The external communication device 460 may, for example, acquire the location of pedestrians, bicycles, motorcycles, other vehicles (such as preceding vehicles), the road surface, lane markings, roadside objects, and / or other local features (such as buildings), and transmit them to the processor 33. Furthermore, the external communication device 460 has the same functions as the locator 410 described above, and may transmit the location information and direction information of the vehicle 1 to the display control device 30, etc. It also has the functions of the road information database 403 described above, and may acquire the road information (an example of surrounding vehicle-related information) and transmit it to the processor 33. Note that the information acquired from the external communication device 460 is not limited to what is described above.

[0057] The visibility detection unit 470 detects the visibility distance, which is the distance that can be seen in front of the vehicle 1, and outputs this visibility information to the display control device 30 (I / O interface 31). The visibility detection unit 470 calculates the visibility distance (the distance that can be seen) from the foreground image captured by a camera unit (not shown) installed on the vehicle 1 (part or all of this function may be included in the visibility information processing module 508 described later).

[0058] The visibility detection unit 470 may output the determination result of whether the visibility is in the long range (for example, visibility of 200m or more but less than 300m) (for example, first forward visibility state), the medium range (for example, visibility of 100m or more but less than 200m) (for example, second forward visibility state), or the short range (for example, visibility of less than 100m) (for example, third forward visibility state) as visibility information to the display control device 30 (I / O interface 31) (Note that some or all of this function may be included in the visibility information processing module 508 described later).

[0059] The control unit 490 is a hardware switch located on the steering wheel 6 of the vehicle 1, and outputs a signal to the display control device 30 (I / O interface 31) in response to an operation by the occupant of the vehicle 1 (generally the driver). The control unit 490 may also be a software switch displayed on a display (not shown) inside the vehicle 1.

[0060] The software components stored in memory 37 include a real-scene tracking navigation image generation module 502, an overhead navigation image generation module 504, a surrounding vehicle-related information detection module 506, a visibility information processing module 508, a weather information processing module 510, a foreground visibility determination module 512, and an image switching module 514.

[0061] The real-scene-following navigation image generation module 502 controls the position, shape, and size of the real-scene-following navigation image 210 so that it is visible in a desired positional relationship with a specific superimposed object (for example, a specific branch road, a specific position in a driving lane, a specific driving lane among multiple driving lanes, etc.) 310 of the real-scene 300. That is, the position and shape of the real-scene-following navigation image 210 are continuously updated at a predetermined interval to match the relative position and shape of the superimposed object 310. Based on map data and navigation information, the real-scene-following navigation image generation module 502 places virtual objects for route guidance on a virtual road model around the vehicle 1 (at least in the direction of the planned driving route), and sets a virtual viewpoint position based on locator information (position and angle of the vehicle 1). Then, the real-scene-following navigation image generation module 502 displays the virtual objects as seen from the virtual viewpoint position as the real-scene-following navigation image 210. In addition, the real-scene tracking navigation image generation module 502 may adjust the position of the real-scene tracking navigation image 210 based on the position of the driver's eyes detected by the viewer detection device 450. For example, the real-scene tracking navigation image generation module 502 determines the left-right and up-down positions of the real-scene tracking navigation image 210 so that its center aligns with the center of the superimposed object 310 for visual confirmation. The "specific positional relationship" can be adjusted depending on the situation of the superimposed object 310 or the vehicle 1, the type of superimposed object 310, the type of image displayed, etc. In addition, the real-scene tracking navigation image generation module 502 may adjust the position of the real-scene tracking navigation image 210 based on the change in the vehicle's attitude detected by the IMU 412. Furthermore, the real-scene tracking navigation image generation module 502 may reduce the size of the real-scene tracking navigation image 210 as the distance set to the real-scene tracking navigation image 210 or the superimposed object 310 increases. The scene-following navigation image generation module 502 may be omitted, and the display control device 30 may acquire image data of the scene-following navigation image 210 from the I / O interface 31.

[0062] The overhead navigation image generation module 504 places virtual objects for route guidance on a virtual road model around the vehicle 1 based on map data and navigation information, and sets a virtual viewpoint position above and behind the vehicle 1 based on locator information (position and angle of the vehicle 1). The overhead navigation image generation module 504 then displays the virtual objects as seen from the virtual viewpoint position as an overhead navigation image 220. The overhead navigation image generation module 504 may be omitted, and the display control device 30 may acquire image data of the overhead navigation image 220 from a navigation device 430 or the like via the I / O interface 31.

[0063] Figure 3 shows the area of ​​the map image surrounding the vehicle, which is displayed as the overhead navigation image 220. Here, the map is assumed to be the αγ plane, and a virtual viewpoint position MM for displaying the bird's-eye view is located at a height Mβ on the β axis perpendicular to the αγ plane, that is, above and behind the vehicle position icon PP. In the example in Figure 3, the virtual viewpoint position MM is set at a distance of Mγ1 behind (γ negative direction) and Mβ1 above (β positive direction) the vehicle position icon PP, and the downward angle θ is set to θ1. As shown in Figure 3, the area of ​​the map enclosed by the rectangular virtual field of view FGHJ with a downward angle θ from the virtual viewpoint position MM is enclosed by the symbols F1'G1'H1'J1', and can be represented as a trapezoid with a wide range in the left-right direction (α direction) on the far side (γ positive direction) and a narrow range in the left-right direction (α direction) on the near side (γ negative direction). In some display modes of this embodiment, the virtual field of view E has a narrower width in the left-right direction (α direction) of the area that overlaps with the far side of the map than the rectangular virtual field of view FGHJ. In other words, the area of ​​the map enclosed by the virtual field of view E (FKLJ) observed from the virtual viewpoint position MM has a narrower left-right direction (α direction) on the far side (γ positive direction) than the area enclosed by the symbols F1'K1'L1'J1' and F1'G1'H1'J1'.

[0064] The overhead navigation image generation module 504 sets the position Mβ(Mγ) and angle θ of the virtual viewpoint position MM relative to the map image. By changing the position Mβ(Mγ) and angle θ of the virtual viewpoint, the overhead navigation image generation module 504 changes the area of ​​the map image around the vehicle that is displayed as the overhead navigation image 220. Typically, the overhead navigation image generation module 504 sets the position of the "virtual viewpoint" to the area around the vehicle (usually a suitable location behind the vehicle). The angle θ of the "virtual viewpoint" is the angle between the direction passing through the vertical midpoint of the virtual field of view K (described later) from the virtual viewpoint position MM and the αγ plane, and will be referred to as the depression angle below.

[0065] The overhead navigation image generation module 504 sets a virtual viewpoint position MM at a distance of Mγ behind (γ negative direction) and Mβ above (β positive direction) the vehicle position icon PP. Therefore, when the vehicle is traveling straight on a straight road, viewing the vehicle position icon PP from the virtual viewpoint position MM will result in an overhead view where the vehicle position icon PP is facing forward. When the virtual viewpoint position MM moves, the map image around the vehicle rotates (the angle changes) and is displayed. For example, if the vehicle position icon PP is steered and rotates 90 degrees counterclockwise when viewed from directly above, the virtual viewpoint position MM will also rotate 90 degrees counterclockwise in conjunction with this.

[0066] The overhead navigation image generation module 504 shortens the distance between the vehicle position icon PP and the virtual viewpoint position MM when increasing the scale of the overhead navigation image 220. Specifically, it shortens at least one of the distance Mγ in the forward / backward direction (γ direction) and the distance Mβ in the up / down direction (β direction) so that the distance between the vehicle position icon PP and the virtual viewpoint position MM is shortened. The overhead navigation image generation module 504 may also increase the scale of the overhead navigation image 220 (in stages) when the distance to a route change point (for example, a branching road) becomes shorter.

[0067] The surrounding vehicle-related information detection module 506 acquires information (also called surrounding vehicle-related information) about surrounding vehicles (also called interrupting vehicles) W located in front of or to the side of the vehicle 1, which the foreground visibility determination module 512 (described later) uses to determine the visibility of the actual scenery. The surrounding vehicle-related information detection module 506 may, for example, acquire information from the surrounding monitoring sensor 440 and / or the external communication connection device 460 indicating the position, direction of movement, and / or speed of the surrounding vehicles W located in front of and / or to the side of the vehicle 1. The surrounding vehicle-related information detection module 506 may also acquire information from the surrounding vehicles W via the external communication connection device 460 indicating the turn signals, steering angle, and / or the planned route and schedule of the surrounding vehicles W according to the driver assistance system.

[0068] The visibility information processing module 508 determines, based on the visibility information (visibility) input from the visibility detection unit 470, whether the forward visibility state is a first forward visibility state (e.g., visibility of 200m or more and less than 300m), a second forward visibility state with lower visibility of the foreground than the first forward visibility state (e.g., visibility of 100m or more and less than 200m), a third forward visibility state with lower visibility of the foreground than the second forward visibility state (e.g., visibility of 50m or more and less than 100m), a fourth forward visibility state with lower visibility of the foreground than the third forward visibility state (e.g., visibility of less than 50m), or a fifth forward visibility state with higher visibility of the foreground than the first forward visibility state (e.g., visibility of 300m or more). In other words, the visibility information processing module 508 may include table data for determining the forward visibility state based on the visibility information (visibility).

[0069] Furthermore, the visibility information processing module 508 may include some or all of the functions of the visibility detection unit 470. That is, the visibility information processing module 508 may calculate the visibility (the distance that can be seen) from the foreground image captured by a camera unit (not shown) provided on the vehicle 1.

[0070] The weather information processing module 510, via the I / O interface 31, refers to the weather database 480 and obtains weather information indicating that the estimated weather around the vehicle 1 is bad weather (e.g., fog, mist, rain, snow, and their levels). The visibility information processing module 508 may determine, based on the weather information (e.g., fog, mist, rain, snow, and their levels) input from the weather database 480, whether the forward visibility state is a first forward visibility state, a second forward visibility state, a third forward visibility state, or a fourth forward visibility state. In other words, the weather information processing module 510 may include table data for determining the forward visibility state based on weather information (e.g., fog, mist, rain, snow, and their levels).

[0071] Furthermore, the weather information processing module 510 may obtain weather information from weather sensors 485, such as a rain sensor, via the I / O interface 31, indicating that the estimated weather around the vehicle 1 is bad weather (for example, the level of rain), and determine the forward visibility condition according to the level of rain.

[0072] The foreground visibility determination module 512 determines whether a surrounding vehicle W is present in front of the vehicle 1 and / or to the side, based on the position of the surrounding vehicle W obtained from the surrounding vehicle-related information detection module 506, and whether it is obstructing the view of the actual scenery. If the foreground visibility determination module 512 detects that the position of the surrounding vehicle W has entered the first determination area E1, it determines that the visibility of the foreground has decreased. Also, if the foreground visibility determination module 512 detects that the position of the surrounding vehicle W has left the second determination area E2, it determines that the visibility of the foreground has increased.

[0073] Figure 4 is a conceptual diagram showing the first determination region E1 and the second determination region E2 when the road surface is viewed from above. The first determination region E1 is, for example, the area within the driving lane 310 of the vehicle 1, and the second determination region E2 is a wider area than the first determination region E1, for example, the area including the area outside the left lane marking 312 and the area outside the right lane marking 313 of the vehicle 1's driving lane 310. By making the ranges of the first determination region E1, which determines entry, and the second determination region E2, which determines exit, different, hysteresis control becomes possible to prevent frequent switching between entry and exit determinations.

[0074] Furthermore, the foreground visibility determination module 512 calculates the possibility (also called interruption probability) that a surrounding vehicle W will interrupt the driving lane of the vehicle 1, based on the position, direction of movement, speed of movement, turn signals, steering angle, and / or planned route and schedule of the surrounding vehicle W located in front of and / or to the side of the vehicle 1, obtained from the surrounding vehicle-related information detection module 506. If the calculated interruption probability exceeds a predetermined threshold stored in the memory 37, the module may determine that there is a high probability of interruption (it may also be predicted that the vehicle will enter the first determination area E1 in front of the vehicle 1). The foreground visibility determination module 512 may also predict that the surrounding vehicle W (in this case, the vehicle W that has entered the first determination area E1) will exit the first determination area E1 (or second determination area E2) in front of the vehicle 1, based on the surrounding vehicle-related information of the surrounding vehicle W described above.

[0075] Furthermore, the foreground visibility determination module 512 may estimate the position (also called the interruption point) where a surrounding vehicle W will interrupt the driving lane 310 of the vehicle 1, based on the position, direction of movement, and speed of surrounding vehicles W located in front of and / or to the side of the vehicle 1, the turn signals and steering angles of the surrounding vehicles W, and / or the planned route and schedule of the driving support system. For example, the foreground visibility determination module 512 may estimate the interruption path of the surrounding vehicle W and identify the vicinity of the intersection of this interruption path and the left lane marking 312 or the right lane marking 313 of the driving lane 310 in which the vehicle 1 is traveling as the interruption point.

[0076] The foreground visibility determination module 512 does not necessarily have a function to calculate the possibility of interruption and / or a function to determine the possibility of interruption. Some or all of these functions may be provided separately from the display control device 30 of the vehicle display system 10, for example, in another ECU (not shown) within the vehicle 1.

[0077] The image switching module 514 switches the display mode of the navigation image based on the determination result of the foreground visibility determination module 512. Specifically, the image switching module 514 sets itself to a first display mode that displays only the real-scenery tracking navigation image 210 when (1) it is estimated that the foreground of the vehicle 1 is not obstructed by another vehicle W (for example, when it is detected that another vehicle W is leaving or is expected to leave a predetermined second determination area E2 in front of the vehicle), and sets itself to a second display mode that displays an overhead navigation image 220 instead of (or in addition to) the real-scenery tracking navigation image 210 when (2) it is estimated that the foreground of the vehicle 1 is obstructed by another vehicle W (for example, when it is detected that another vehicle W is entering or is expected to enter a predetermined first determination area E1 in front of the vehicle),

[0078] Figure 5 shows an example of the display of the real-world tracking navigation image 210 in the first display mode. The real-world tracking navigation image 210 shown in Figure 5 is an AR image that is superimposed on the driving lane 311 (an example of a specific superimposed object 310), which is the area between the left lane marking 312 and the right lane marking 313, extends from near to far along the road surface of the driving lane 311, and is bent to match the position and shape of the branching point 315 (an example of a specific superimposed object 310). The real-world tracking navigation image 210 is displayed within the first display area 710 of the HUD display area 610. The vertical (Y-axis) center of the first display area 710 is denoted by the symbol 710A.

[0079] Figure 6A shows an example of the display of the overhead navigation image 220 in a second display mode in some embodiments. The image switching module 514 displays the overhead navigation image 220 instead of the real-world tracking navigation image 210 when it is estimated that the foreground of vehicle 1 is obstructed by another vehicle W (second display mode). That is, in the second display mode in some embodiments, the image switching module 514 hides the real-world tracking navigation image 210. The overhead navigation image 220 includes a map image 221 of the area around the vehicle 1, a route image 222 placed on the map image 221, and a vehicle position icon 223 (which may be omitted). The overhead navigation image 220 is displayed in the second display area 721 (720) of the HUD display area 610. The vertical (Y-axis direction) center of the second display area 721 (720) is denoted by reference numeral 721A (720A). The vertical center 721A (720A) of the second display area 721 (720) is positioned below the vertical center 710A of the first display area 710. In other words, the overhead navigation image 220 is positioned below (negative Y-axis direction) the real-world tracking navigation image 210.

[0080] Figure 6B shows an example of the display of the overhead navigation image 220 in a second display mode in some embodiments. The image switching module 514 displays the overhead navigation image 220 in addition to the real-scene tracking navigation image 210 when it is estimated that the foreground of vehicle 1 is obstructed by another vehicle W (second display mode). That is, in the second display mode in some embodiments, the image switching module 514 does not hide the real-scene tracking navigation image 210. A portion of the upper part of the second display area 721 (720) where the overhead navigation image 220 is displayed may overlap with a portion of the lower part of the first display area 710 where the real-scene tracking navigation image 210 is displayed. In such a case, as shown in Figure 6B, the image switching module 514 displays the overhead navigation image 220 on a layer above the real-scene tracking navigation image 210. Here, at least a portion of the overhead navigation image 220 that overlaps with the real-scene tracking navigation image 210 may be made semi-transparent. As a result, the overhead navigation image 220 is recognized as being in front of the real-scene tracking navigation image 210, and the clarity of information can be improved while displaying both the real-scene tracking navigation image 210 and the overhead navigation image 220.

[0081] Figure 7 shows an example of the display of the real-scene tracking navigation image 210 when switching from the first display mode to the second display mode in several embodiments. When the image switching module 514 switches to the second display mode, it reduces the brightness of the real-scene tracking navigation image 210 and then hides it. This prevents the observer from being confused by the sudden disappearance of the real-scene tracking navigation image 210. More specifically, when the image switching module 514 switches to the second display mode, it may gradually reduce the brightness of the real-scene tracking navigation image 210 before hiding it. If the real-scene tracking navigation image 210 is not hidden in the second display mode, the image switching module 514 may display both a non-low-brightness overhead navigation image 220 and a low-brightness real-scene tracking navigation image 210. This allows the viewer to be drawn to the overhead navigation image 220 while still viewing the real-scene tracking navigation image 210.

[0082] Conversely, when switching from the second display mode to the first display mode, the image switching module 514 may gradually increase the brightness of the real-scene tracking navigation image 210 from a low brightness (or hidden) state. Also, when starting to display the overhead navigation image 220, the image switching module 514 may gradually increase the brightness from a hidden state, and conversely, when hiding the overhead navigation image 220, it may gradually decrease the brightness before hiding it.

[0083] Figure 8 shows examples of the display of the real-scene tracking navigation image 210 when switching from the first display mode to the second display mode in several embodiments. When switching to the second display mode, the image switching module 514 moves the position of the real-scene tracking navigation image 210. For example, if another vehicle W approaches from the right in front of the vehicle 1, the image switching module 514 moves the position of the real-scene tracking navigation image 210 to the left (in a predetermined direction of movement Mo). At this time, the image switching module 514 gradually lowers the brightness of the real-scene tracking navigation image 210 while moving it, and then hides it.

[0084] As described above, in the display system 10 of the first embodiment, the display control device 30 that controls one or more image display units 20 that display an image showing the planned driving route is equipped with one or more processors 33, and the one or more processors 33 acquire navigation information related to the planned driving route, and use the navigation information to display a real-scene tracking navigation image 210 in the first display area 710 that follows a specific superimposed object so as to maintain the positional relationship with a specific superimposed object in the real-scene, and acquire other vehicle information related to the position of other vehicles W, and if it is estimated that the visibility of the foreground of the own vehicle is obstructed based on at least the other vehicle information, it causes a second display area 720 having a vertical center 720A located below the vertical center 710A of the first display area 710 to display an overhead navigation image 220 that includes at least a map image 221 of the area around the own vehicle and a route image 222 that overlaps with the map image 221 and shows the planned driving route, and is viewed at an oblique angle from above. The system detects when a specific superimposed object associated with the real-world tracking navigation image 210 is difficult to see due to another vehicle W, and displays the overhead navigation image 220. This allows the system to continue providing route guidance even if the visibility of the difficult-to-see real-world tracking navigation image 210 decreases or it becomes invisible. Furthermore, even when the real-world tracking navigation image 210 continues to be displayed, checking it together with the overhead navigation image 220 makes it easier to grasp the sense of distance in the real-world tracking navigation image 210.

[0085] Furthermore, in the display system 10 of the second embodiment which may be dependent on the first embodiment, the first display area 710 is an area that overlaps with the actual view seen through the front windshield 2 of the vehicle 1, and the second display area 720 is located below the first display area 710 and is an area that does not overlap with the actual view seen through the front windshield. The first display area 710 is an area 110 in the front windshield 2 that allows visible light to pass through and the actual view to be seen, as shown in Figure 9, for example, within the display area 110 of the head-up display device 20A (image display unit 20). On the other hand, the second display area 720 is an area 120 that does not overlap with the actual view seen through the front windshield 2, as shown in Figure 9, for example, an area that overlaps with the shielding part 2C or the dashboard 5 of the front windshield 2. In a broader sense, the second display area 720 only needs to be within the area 120 that does not largely overlap with the actual view seen through the front windshield 2, and may also include a portion (for example, less than 30% of the second display area 720) that is outside the area 120 that does not overlap with the actual view seen through the front windshield 2.

[0086] Furthermore, in the display system 10 of a third embodiment which may be dependent on the second embodiment, as shown in Figure 10, the second display area 720 is within the area 120 that overlaps with the shielding portion 2C that obstructs the view from inside the vehicle in the front windshield 2. As shown in Figure 11A, the second head-up display device 20C (an example of the image display unit 20) can display a virtual image behind the front windshield 2 by projecting the light of the image displayed by the display 22C onto the shielding portion 2C.

[0087] Furthermore, in the display system 10 of other embodiments that may be dependent on the second embodiment, the image display unit 20 may be a display panel provided between the dashboard 5 and the front windshield 2. The image display unit 20D in Figure 11B is a display panel provided on the dashboard 5 on the driver 4 side of the light-transmitting unit 26 of the head-up display device 20A. The second display area 720 in which the image display unit 20D in Figure 11B displays the overhead navigation image 220 is located in an area 120 that does not overlap with the actual view seen through the front windshield 2 when viewed from the center of the eye box EB of the head-up display device 20A.

[0088] Furthermore, as shown in Figure 11C, the image display unit 20E is a display panel located on the dashboard 5 further back (away from the driver 4) than the light-transmitting unit 26 of the head-up display device 20A. The second display area 720 in Figure 11C, where the image display unit 20D displays the overhead navigation image 220, is positioned in an area 120 that does not overlap with the actual view seen through the front windshield 2 when viewed from the center of the eye box EB of the head-up display device 20A.

[0089] Furthermore, in the display system 10 of the fourth embodiment which may be dependent on the first embodiment, as shown in Figures 6A and 6B, the first display area 710 is an area that overlaps with the actual view seen through the vehicle's front windshield, and the second display area 720 is an area that overlaps with the actual view seen through the front windshield, and is positioned so that its upper end is above the lower end of the first display area 710.

[0090] Furthermore, in a fifth embodiment of the display system 10 that may be dependent on any one of the first to third embodiments, the processor 33 further performs a first movement process to move the real-scene tracking navigation image 210 in a first movement direction that includes at least the left or right direction in which another vehicle W is moving relative to the current vehicle, when it is estimated that the visibility of the foreground of the current vehicle is obstructed, and a process to hide the real-scene tracking navigation image 210 during or before the first movement process.

[0091] Furthermore, in the display system 10 of the fifth embodiment which may be dependent on the fourth embodiment, the processor changes the movement speed of the real-scene tracking navigation image 210 in a monotonically non-decreasing manner with respect to the relative movement speed of other vehicles W during the first movement process.

[0092] Furthermore, in the display system 10 of the sixth embodiment which may be dependent on the fifth embodiment, the processor sets the movement speed of the real-scene tracking navigation image 210 linearly with respect to the relative movement speed of other vehicles W in the first movement processing.

[0093] Furthermore, in the display system 10 of the seventh embodiment which may be dependent on the fifth embodiment, the processor sets the movement speed of the real-scene tracking navigation image 210 non-linearly with respect to the relative movement speed of other vehicles W in the first movement processing.

[0094] Furthermore, in the eighth embodiment of the display system 10, which may be dependent on any one of the fourth to seventh embodiments, the processor further performs a second movement process to move the overhead navigation image 220 to a second display area 720 in a second movement direction that includes either the left or right direction included in the first movement direction, if it is estimated that the visibility of the foreground of the vehicle is obstructed, and a visibility enhancement process to improve the visibility of the overhead navigation image 220 during or before the second movement process.

[0095] Furthermore, in the display system 10 of the ninth embodiment, which may be dependent on any one of the fourth to seventh embodiments, the processor further performs a third movement process to move the overhead navigation image 220 to the second display area 720 in a third movement direction that includes either the left or right direction, which is not included in the first movement direction, if it is estimated that the visibility of the foreground of the vehicle is obstructed, and a visibility enhancement process to improve the visibility of the overhead navigation image 220 during or before the third movement process.

[0096] Furthermore, in a display system 10 of a tenth embodiment which may be dependent on any one of the fourth to seventh embodiments, the processor further performs a fourth movement process to move the overhead navigation image 220 to the second display area 720 in a fourth movement direction which includes either the upward or downward direction, not the left or right direction, if it is estimated that the visibility of the foreground of the vehicle is obstructed, and a visibility enhancement process to improve the visibility of the overhead navigation image 220 during or before the fourth movement process.

[0097] Furthermore, in the display system 10 of the 11th embodiment, which may be dependent on any one of the first to ten embodiments, the processor, when it is estimated that the visibility of the vehicle's foreground is not obstructed, hides the overhead navigation image 220 and displays the real-scene tracking navigation image 210 in the first display area 710.

[0098] Furthermore, in a display system 10 of a twelfth embodiment which may be dependent on any one of the first to ten embodiments, the processor further performs a fifth movement process to move the overhead navigation image 220 from the second display area 720, and a visibility reduction process to reduce the visibility of the overhead navigation image 220 during or before the fifth movement process, if it is estimated that the visibility of the foreground of the vehicle is not obstructed.

[0099] Furthermore, in a display system 10 of a thirteenth embodiment which may be dependent on any one of the first to twelfth embodiments, the processor further performs a sixth movement process which moves the real-scene tracking navigation image 210 in a second movement direction which includes at least the left or right direction in which another vehicle W is moving relative to the current vehicle, when it is estimated that the visibility of the foreground of the current vehicle is not obstructed, and a process which improves the visibility of the real-scene tracking navigation image 210 during or before the sixth movement process.

[0100] Furthermore, in the display system 10 of the 14th embodiment, which may be dependent on any one of the 11th to 13th embodiments, the second determination area E2 is wider than the first determination area E1.

[0101] Furthermore, in the display control device 30 of the 15th embodiment, which may be dependent on any one of the first to 14 embodiments, the processor 33 determines the forward visibility state based on visibility information and weather information, and lowers the brightness of the overhead navigation image 220 as the forward visibility state deteriorates. As the forward visibility state deteriorates, the foreground becomes harder to see. As a result, the observer's visual attention is more likely to be directed towards the image (overhead navigation image) than towards the foreground, which has relatively reduced visibility, and consequently, it is conceivable that the observer may find the image bothersome. According to the 15th embodiment, by lowering the brightness of the overhead navigation image as the forward visibility state deteriorates, it is possible to mitigate the observer's excessive focus on the image (overhead navigation image).

[0102] In the display control device of the 15th embodiment, the processor 33 determines the forward visibility state based on visibility information and weather information. If it is determined to be a first forward visibility state, it displays the real-scene tracking navigation image 210 in the first display area 710. If it is determined to be a second forward visibility state in which the foreground is less visible than in the first forward visibility state, it displays the overhead navigation image 220 in the second display area 720 with a first brightness. If it is determined to be a third forward visibility state in which the foreground is less visible than in the second forward visibility state, it displays the overhead navigation image 220 in the second display area 720 with a second brightness lower than the first brightness.

[0103] In the display control device 30 of the 15th embodiment, the processor 33 may continuously lower the brightness of the overhead navigation image in accordance with the degree of deterioration of the forward visibility. The processor 33 may quantify the degree of deterioration of the forward visibility and linearly lower the brightness of the overhead navigation image 220 in relation to the numerical value (level) of the degree of deterioration of the forward visibility (however, it may also be changed non-linearly). In another example, the processor may gradually lower the brightness of the overhead navigation image in accordance with the degree of deterioration of the forward visibility.

[0104] Furthermore, in the 16th embodiment of the display control device, which may be dependent on any one of the first to 15 embodiments, the processor 33 determines the forward visibility state based on visibility information and weather information. If it is determined to be a first forward visibility state, it displays the actual scene tracking navigation image 210 in the first display area 710. If it is determined to be a second forward visibility state, where the foreground is less visible than in the first forward visibility state, it displays the overhead navigation image 220 in the second display area 720 with a first brightness. If it is determined that the foreground is less visible than in the second forward view state, the overhead navigation image 220 is displayed in the second display area 720 at a second brightness lower than the first brightness. If it is determined that the foreground is less visible than in the third forward view state, the overhead navigation image 220 is displayed in the third display area 730, which has a vertical center 730A located below the vertical center 720A of the second display area 720. As the forward view condition deteriorates, the foreground becomes less visible. As a result, the observer's visual attention is more likely to be directed towards the image (overhead navigation image) than towards the foreground, which has relatively reduced visibility, and it is conceivable that the observer may find the image bothersome. According to the 16th embodiment, as the forward visibility deteriorates, the area in which the overhead navigation image 220 is displayed is lowered, making it less likely for the image (overhead navigation image 220) to enter the field of view of an observer facing forward (in other words, it moves away from the central field of view), thereby reducing the tendency to look too much at the image (overhead navigation image) when the forward visibility deteriorates.

[0105] Figure 12 is a diagram illustrating the third display area. In the sixteenth embodiment, the first display area 710 and the second display area 720 are located in the area 110 where the actual scene is visible, and the third display area 730 may be located in the area 120 which is below the area 110 where the actual scene is visible and does not overlap with the actual scene.

[0106] Figure 13 is a diagram illustrating the third display area. In the 16th embodiment (modified version), the first display area 710, the second display area 720, and the third display area 730 may be arranged in the area 110 where the actual scenery is visible. The processor 33 reduces the size of the third display area 730, lowers the position of the third display area 730, or a combination thereof, so that the vertical center 730A of the third display area 730 is lower than the vertical center 720A of the second display area 720.

[0107] Furthermore, in the 17th embodiment of the display control device 30, which may be dependent on any one of the first to 16 embodiments, the processor 33 may adjust the ease of switching between various processes based on the operation information in the operation unit 490. The various processes include (1) a process to display the overhead navigation image 220 in the second display area 720, (2) a process to hide the overhead navigation image 220, (3) a process to lower the brightness of the overhead navigation image 220, (4) a process to increase the brightness of the overhead navigation image 220, and (5) a process to display the overhead navigation image 220 in the third display area 730. The ease of switching between various processes can be adjusted by relaxing or tightening the switching conditions (parameter conditions, time conditions). According to the 17th embodiment, the user can adjust the switching conditions as appropriate.

[0108] Furthermore, in the 18th embodiment of the display control device 30, which may be dependent on any one of the first to 17 embodiments, the processor 33 may switch the display / hide status of the overhead navigation image 220 based on the operation information in the operation unit 490. The processor 33 may also automatically switch the display / hide status of the real-scene tracking navigation image 210 in conjunction with the switching of the display / hide status of the overhead navigation image 220. Specifically, the processor 33 may automatically hide the real-scene tracking navigation image 210 when displaying the overhead navigation image 220, and automatically display the real-scene tracking navigation image 210 when hiding the overhead navigation image 220.

[0109] Furthermore, in the display control device 30 of the 19th embodiment, which may be dependent on any one of the first to 18 embodiments, the processor 33 determines the forward visibility state based on visibility information and weather information. If it is determined that the first forward visibility state is present, it displays the overhead navigation image 220 in the second display area 720 with a first brightness. If the fifth forward visibility state, in which the foreground is easier to see than the first forward visibility state, continues for a predetermined time or longer, it hides the overhead navigation image 220 and displays the real-scene tracking navigation image 210 in the first display area.

[0110] Furthermore, the display system 10 of the 20th embodiment includes a display control device 30 of any one of the first to 19 embodiments, and a head-up display device 20A that displays a virtual image of the planned driving route in a first display area 710 and a second display area 720. [Explanation of symbols]

[0111] 1: Own vehicle 2: Front windshield 2C: Shielding part 4: Driver 5: Dashboard 6: Road surface 10: Vehicle display system 20: Image display section 20A:HUD device 20B:HMD device 20C: Second head-up display device 20D: Image display section 20E: Image display unit 20a:Display light 22:Display unit 22C: Display 24: Relay Optics 26: Light transmission part 30: Display control device 31: I / O Interface 33: Processor 35: Image processing circuit 37: Memory 210: Real-world scenery tracking navigation image 220: Overhead navigation image 221: Map image 222: Route image 223: Vehicle position icon 300: Actual Scenery 310: Superimposed object 315: Branching point 401: Vehicle ECU 403: Road Information Database 410: Locator 411 :GNSS device 420: Map Information Storage Device 430: Navigation device 440: Surroundings monitoring sensor 450: Person detection device 460: External communication device 502: Real-world scenery tracking navigation image generation module 504: Overhead Navigation Image Generation Module 506: Surrounding Vehicle Related Information Detection Module 508: Visibility Information Processing Module 510: Weather information processing module 512: Foreground visibility determination module 514: Image switching module 610:HUD display area 700: Eye position 710: First display area 720: Second display area 730: Third display area EB: ibox Mo: Direction of movement W: Other vehicles

Claims

1. In a display control device (30) that controls one or more image display units (20) that display images showing the planned route, One or more processors (33) Obtain navigation information related to the aforementioned planned route, Using at least the aforementioned navigation information, a real-scene tracking navigation image (210) that follows the specific superimposed object in order to maintain the positional relationship with the specific superimposed object in the real-scene is displayed in the first display area (710). The system acquires forward visibility information, which includes information about other vehicles (W) related to their position, visibility information related to the distance that can be seen ahead, and at least one of weather information. If it is estimated, based at least on the forward view information, that the visibility of the foreground of the vehicle is obstructed, then an overhead navigation image (220) is displayed in a second display area (720) having a vertical center (720A) positioned below the vertical center (710A) of the first display area (710), which includes at least a map image (221) of the area around the vehicle and a route image (222) that overlaps with the map image (221) and shows the planned route, and which is viewed at an oblique angle from above. A display control device characterized by the following:

2. The first display area (710) is an area that overlaps with the actual view seen through the front windshield of the vehicle, The second display area (720) is located below the first display area (710) and is an area that does not overlap with the actual view seen through the front windshield. The display control device according to feature 1.

3. The second display area (720) is an area that overlaps with the shielding portion (2C) of the front windshield that obstructs the view from inside the vehicle. The display control device according to claim 2.

4. The first display area (710) is an area that overlaps with the actual view seen through the front windshield of the vehicle, The second display area (720) is an area that overlaps with the actual view seen through the front windshield, and is positioned such that its upper end is above the lower end of the first display area (710). The display control device according to feature 1.

5. The aforementioned processor (33) If it is presumed that the visibility of the area in front of the vehicle is obstructed, A first movement process that moves the real-scene tracking navigation image (210) in a first movement direction that includes at least the left or right direction in which the other vehicle (W) is moving relative to it, and During or before the first movement process, a process is further executed to hide the actual scenery tracking navigation image (210). The display control device according to feature 1.

6. In the first move process, the processor The movement speed of the aforementioned real-scene tracking navigation image (210) is changed monotonically and non-decreasingly with respect to the relative movement speed of the other vehicle (W). The display control device according to claim 5.

7. In the first move process, the processor The movement speed of the aforementioned real-scene tracking navigation image (210) is set linearly with respect to the relative movement speed of the other vehicle (W). The display control device according to feature 6.

8. In the first move process, the processor The movement speed of the aforementioned real-scene tracking navigation image (210) is set non-linearly with respect to the relative movement speed of the other vehicle (W). The display control device according to feature 6.

9. The aforementioned processor, If it is presumed that the visibility of the area in front of the vehicle is obstructed, A second movement process that moves the overhead navigation image (220) to the second display area (720) in a second movement direction that includes either the left or right direction included in the first movement direction, and During or before the second movement process, a visibility enhancement process is further performed to improve the visibility of the overhead navigation image (220). The display control device according to claim 5.

10. The aforementioned processor, If it is presumed that the visibility of the area in front of the vehicle is obstructed, A third movement process that moves the overhead navigation image (220) to the second display area (720) in a third movement direction that includes either the left or right direction, which is not included in the first movement direction, and During or before the third movement process, a visibility enhancement process is further performed to improve the visibility of the overhead navigation image (220). The display control device according to claim 5.

11. The aforementioned processor, If it is presumed that the visibility of the area in front of the vehicle is obstructed, A fourth movement process that moves the overhead navigation image (220) to the second display area (720) in a fourth movement direction that includes either the upward or downward direction, excluding the left-right direction, and During or before the fourth movement process, a visibility enhancement process is further performed to improve the visibility of the overhead navigation image (220). The display control device according to claim 5.

12. The aforementioned processor, If it is presumed that the visibility of the area in front of the vehicle is not obstructed, A fifth movement process to move the overhead navigation image (220) out of the second display area (720), and During or before the fifth movement process, a visibility reduction process is further performed to reduce the visibility of the overhead navigation image (220). The display control device according to feature 1.

13. The aforementioned processor, Based on the aforementioned visibility information and weather information, the forward visibility condition is determined, If it is determined that the first forward view state is present, the real-scene tracking navigation image (210) is displayed in the first display area (710). If it is determined that the foreground is less visible in the second forward view state than in the first forward view state, the overhead navigation image (220) is displayed in the second display area (720) with the first brightness. If it is determined that the foreground is less visible than in the second forward view state, the overhead navigation image (220) is displayed in the second display area (720) at a second brightness lower than the first brightness. The display control device according to feature 1.

14. The aforementioned processor, Based on the aforementioned visibility information and weather information, the forward visibility condition is determined, If it is determined that the first forward view state is present, the real-scene tracking navigation image (210) is displayed in the first display area (710). If it is determined that the foreground is less visible in the second forward view state than in the first forward view state, the overhead navigation image (220) is displayed in the second display area (720) with the first brightness. If it is determined that the foreground is less visible than in the second forward view state, the overhead navigation image (220) is displayed in the second display area (720) with a second brightness lower than the first brightness. If it is determined that the foreground is less visible than in the third forward view state, the overhead navigation image (220) is displayed in a third display area (730) having a vertical center (730A) located below the vertical center (720A) of the second display area (720). The display control device according to feature 1.

15. A display control device according to any one of claims 1 to 14, The system includes a head-up display device that displays a virtual image of the planned driving route in the first display area (710) and the second display area (720), A display system characterized by the following features.

16. In a display control method for controlling one or more image display units (20) that display images showing a planned route, To obtain navigation information related to the aforementioned planned route, Using at least the aforementioned navigation information, a real-scene tracking navigation image (210) that follows the specific superimposed object in order to maintain the positional relationship with the specific superimposed object in the real-scene is displayed in the first display area (710), The forward visibility information includes obtaining information on other vehicles (W) related to the position of other vehicles, visibility information related to the distance that can be seen ahead, and at least one of weather information. If, at least based on the forward visibility information, it is estimated or predicted that the visibility of the vehicle's foreground is obstructed by the other vehicle (W), This includes displaying a map image (221) of the area surrounding the vehicle, and an overhead navigation image (220) that overlaps with the map image (221) and shows the planned route, in a second display area (720) having a vertical center positioned below the vertical center of the first display area (710), such that these are viewed from an oblique angle. A display control method characterized by the following:

Images