Information display method, information display device, and program for vehicle

Abstract

An information display device of a vehicle acquires the coordinates of an object (20) in a real space, and displays a virtual object (20A) at a position corresponding to the coordinates of the object (20) by superimposing the object (20) on a forward view seen through a windshield (2) via an AR-HUD (1). A controller (3) detects the pitch angle (α) of the vehicle and applies a pitch angle correction of "− Gα" to the display position of the virtual object (20A), which is shifted by "θ + α", in order to offset the effect of the pitch angle (α). A gain (G) is set to a smaller value as a distance (L) becomes smaller so that the virtual object (20A) does not extend below the display area.

Description

Vehicle information display method, device, and program

[0001] The present invention relates to a vehicle information display that uses an AR-HUD (Augmented Reality Head-Up Display) to display a virtual image superimposed on a real scene.

[0002] In recent years, HUDs (head-up displays), which project and display various information necessary while driving a vehicle onto the windshield in front of the driver's seat, have become increasingly popular. Furthermore, AR-HUDs (augmented reality head-up displays) are known as a more advanced version of HUD technology. AR-HUDs can superimpose virtual information (also called virtual objects) made up of virtual images onto the real information visible through the windshield, reducing the need for the driver to move their viewpoint or focus. This reduces the need for the driver to move their focus, compared to typical HUDs, which display information on the glass surface of the windshield.

[0003] Virtual information can include a variety of things, one of which is virtual information that should be displayed at specific coordinates in real space. For example, as guidance information for a car navigation system, virtual information such as arrows indicating intersections where to turn right or left is displayed at a position that overlaps with the intersection in the real scene. In addition, a destination point display made up of a virtual object of an appropriate shape is also displayed so that it overlaps with the real coordinate position.

[0004] When such a display is performed, if the vehicle tilts forward or backward, there is a problem in that the position of the virtual information displayed by the AR-HUD shifts up or down from the appropriate position corresponding to the coordinates in real space. Patent Document 1 describes a car navigation system that is not an AR-HUD but displays guidance information such as an arrow superimposed on an image from a camera facing forward of the vehicle, in which a similar problem is addressed by detecting the tilt of the vehicle in the forward or backward direction and correcting the display position of the guidance information up or down to offset the effect of this tilt.

[0005] When a virtual object, i.e., a virtual image, is displayed by the AR-HUD, the driver recognizes the distance to the object based on the downward angle, which is the angle downward relative to the horizontal plane. Therefore, the closer the vehicle is to the destination, intersection location, or other location that should be indicated by the object, the lower the object is displayed. Here, the display area of ​​the AR-HUD is limited in size compared to the width of the field of view through the windshield, and therefore, as the distance to the location (coordinates) to be displayed becomes smaller, the object approaches the lower edge of the display area.

[0006] Therefore, if correction is made according to the tilt in the fore-and-aft direction of the vehicle (i.e., the pitch angle), when the display position of the object is corrected downward, the virtual object may move out of the display area and disappear. In particular, when pitch vibration occurs, which changes the pitch angle due to unevenness in the road surface, the virtual object may repeatedly appear and disappear, making it difficult to see.

[0007] JP 2009-250827 A

[0008] This invention is an information display method for a vehicle, which acquires the coordinates of an object in real space, and displays a virtual image at a position corresponding to the coordinates of the object via the vehicle's AR-HUD, superimposing it on the view ahead seen through the windshield; the method acquires the distance between the vehicle and the object, detects the pitch angle of the vehicle, and, in order to offset the influence of this pitch angle, performs display position correction according to the magnitude of the pitch angle, such that the virtual image is displaced downward relative to the display area of ​​the AR-HUD when the pitch angle is negative, and in the opposite direction when the pitch angle is positive; and when the distance is small, the amount of display position correction relative to the pitch angle is relatively small compared to when the distance is large.

[0009] When the distance to the object is small, that is, when the user approaches the object, the amount of display position correction for the pitch angle becomes small, so this display position correction makes it less likely that the virtual image (virtual object) will move below the display area.

[0010] 1 is an explanatory diagram of the configuration of an information display device according to an embodiment; an explanatory diagram showing the relationship between the distance to an object and the look-down angle; an explanatory diagram showing the effect of the vehicle's tilt (pitch angle); an explanatory diagram of the disappearance of a virtual object due to pitch angle correction; an explanatory diagram of the gain of pitch angle correction; an explanatory diagram of an example in which the gain is set to 0 when the distance is closer than a predetermined distance; an explanatory diagram showing an example of a route guidance display seen through the windshield; an explanatory diagram showing an example of a destination display seen through the windshield; an explanatory diagram showing an example of a white line detection display without gain correction; an explanatory diagram showing an example of a route guidance display without gain correction; an explanatory diagram showing an example of a vehicle width display without gain correction; an explanatory diagram showing an example of a right / left turn hit prevention display without gain correction.

[0011] An embodiment of the present invention will now be described in detail with reference to the accompanying drawings.

[0012] FIG. 1 is an explanatory diagram showing the configuration of an information display device for a vehicle according to one embodiment. The vehicle according to this embodiment is a typical automobile equipped with an AR-HUD (Augmented Reality Head-Up Display) 1 in front of the driver's seat. The AR-HUD 1 displays virtual information (virtual objects) superimposed on real information visible through a windshield 2. In the present invention, the display may take any form; for example, a projector may be provided below the windshield 2, and light reflected by the surface of the windshield 2 and reaching the driver's eyes generates a virtual image of the virtual information on a virtual focal plane located an appropriate distance (e.g., several meters) in front of the driver. The virtual information may include a virtual image indicating a right or left turn for route guidance to be displayed superimposed on an intersection, as described below, an icon indicating a destination, or appropriate information such as the speed limit or actual vehicle speed.

[0013] The AR-HUD 1 is controlled by a controller 3. That is, the controller 3 generates a virtual image and controls its display position. A car navigation system 4 using a GPS, which provides route guidance for the vehicle, is connected to the controller 3. The car navigation system 4 includes map information, which includes coordinate data for nodes such as intersections. The car navigation system 4 also identifies the vehicle's position. The illustrated system also includes information acquisition devices such as a radar 5 and a camera 6 for detecting the positions of preceding vehicles and pedestrians. Note that a node is a component that, together with links, constitutes a road network on a digitized road map, and indicates a point such as an intersection or other nodal point on a road network representation. Each node and link has a unique number, and nodes are connected by links to constitute the road network. Hereinafter, a node where a right or left turn should be made is also referred to as an intersection.

[0014] An information display device using the AR-HUD 1 according to one embodiment includes a viewpoint detection camera 7 that detects the driver's viewpoint position in order to optimize the position of the virtual image of the AR-HUD 1. The viewpoint detection camera 7 constitutes, for example, part of a driver monitoring system (DMS) that monitors the driver's condition. The viewpoint detection camera 7 is disposed, for example, near the upper edge of the windshield, facing the driver's head. The controller 3 processes images acquired by the viewpoint detection camera 7 to detect the driver's viewpoint position (i.e., the position of the driver's eyes). Note that the driver's viewpoint position may be detected without using a camera, and may be detected indirectly, for example, from the driver's seating position, driver height, or the like. The controller 3 corrects the position at which the virtual image, i.e., the object, is generated by the AR-HUD 1 according to the viewpoint position thus detected, so that the actual scene seen by the driver through the windshield 2 and the object of the AR-HUD 1 are properly superimposed.

[0015] The controller 3 is configured as a part of the functions of an in-vehicle computer system that performs various controls, and in a mode in which route guidance is provided by the car navigation system 4, it displays virtual images indicating right and left turns required for route guidance, virtual images indicating the destination, etc. via the AR-HUD 1. The controller 3 performs functions such as obtaining the coordinates of nodes (intersections) via the car navigation system 4, obtaining the coordinates of the destination, creating display data for the virtual images, calculating the distance between the vehicle and the intersection or destination, and changing the gain of the pitch angle correction (described later) according to this distance.

[0016] In addition, a signal indicating a pitch angle α, which is the inclination of the vehicle in the longitudinal direction, is input to the controller 3 from a pitch angle detection unit 8, which is configured by another vehicle control controller or the like. The pitch angle detection unit 8 detects the pitch angle α based on a detection signal from, for example, an acceleration sensor. Note that the pitch angle α takes a positive value when the front end of the vehicle is relatively downward (i.e., facing downward), and takes a negative value when the front end of the vehicle is relatively upward (i.e., facing upward). When the AR-HUD 1 displays a virtual image showing the position of an object in real space (e.g., a virtual image showing a right / left turn at an intersection or a virtual image showing a destination), the display position of the virtual image is corrected according to the detected pitch angle α, as described below. Hereinafter, this correction will be referred to as pitch angle correction.

[0017] 7 is a simplified explanatory diagram showing the view seen through the windshield 2 when the vehicle approaches an intersection (corresponding to an object) that is a node where a left turn is to be made, for example. A rectangular display area 21 of the AR-HUD 1 is present in a portion of the view through the windshield 2. The display area 21 is basically positioned directly in front of the driver. Note that the frame around the periphery of the display area 21 is not actually displayed, but is shown in the figure for illustrative purposes. Various virtual information required while driving is displayed in this display area 21 via the AR-HUD 1.

[0018] The display area 21 in the illustrated example is divided into an upper area 21A and a lower area 21B by a boundary line 22 defined by regulations. Note that the boundary line 22 is illustrated for illustrative purposes only and is not actually displayed. Only specific information permitted by regulations, including driving direction guidance, can be displayed in the upper area 21A. Information other than the specific information must be displayed in the lower area 21B below the boundary line 22. In the illustrated example, as information other than the specific information, multiple virtual images 23, such as a guidance display icon 23a of the car navigation system 4, an icon 23b indicating the status of ACC (adaptive cruise control), an icon 23c indicating whether hands-off driving is possible, an icon 23d indicating a speed setting, a set speed display 23e, and a current speed display 23f, are displayed in a horizontal row in the lower area 21B. The virtual images 23 of information other than the specific information are each displayed at a fixed position within the lower area 21B. These virtual images 23 are hereinafter referred to as fixed virtual images 23. These fixed virtual images 23 are not subjected to pitch angle correction. Furthermore, in order to prevent overlap with the fixed virtual image 23, the specific information is displayed only in the upper region 21A and is not displayed in the lower region 21B.

[0019] A virtual image 24 (hereinafter referred to as an arrow icon 24) indicating a right or left turn required for route guidance when approaching an intersection where a right or left turn is required is basically displayed in the upper area 21A as one piece of specific information. This is displayed superimposed on the real scene so as to appear at the intersection point (node) indicated by coordinates. The arrow icon 24 in the illustrated example is an example indicating a left turn, and has a form in which three arrowhead-shaped segments are arranged at intervals. For example, the center segment corresponds to the position of the intersection node. Therefore, the road in real space located to the left of this arrow icon 24 is the road on which you should turn left and proceed.

[0020] The arrow icon 24 guiding right or left turns is displayed in an appropriate color and brightness. The size of the arrow icon 24 is calculated so that it has a constant size (for example, a constant width) at the intersection in 3D space, so that in the display area 21, the longer the distance to the intersection, the smaller the icon will appear, and the shorter the distance, the larger the icon will appear.

[0021] In the AR-HUD 1, when a virtual object, i.e., a virtual image, is displayed by the AR-HUD 1, the driver perceives the distance to the object based on a look-down angle, which is a downward angle relative to a horizontal plane. FIG. 2 is an explanatory diagram of this look-down angle. The driver looks down on a real object (e.g., an intersection) 20 from a viewpoint 15, and the look-down angle θ at this time is geometrically determined by the height H of the viewpoint 15 from the road surface and the distance L from the vehicle to the object 20. The plane labeled 16 in the figure indicates a virtual focal plane on which a virtual image is generated by the AR-HUD 1. This focal plane 16 is located outside the windshield 2, i.e., in front of the vehicle. A virtual object (e.g., the arrow icon 24 described above) is displayed on this focal plane 16 at a position that corresponds to the look-down angle θ of the real object, allowing the driver to perceive the virtual object as being located on the coordinates of the real object. The outline of this focal plane 16 is the display area 21 of the AR-HUD 1, but the focal plane 16 in Figure 2 specifically shows only the upper area 21A. The triangle labeled 17 indicates the visual angle at which the driver views the upper area 21A of the AR-HUD 1, in other words, the display angle of the upper area 21A, and is determined by the line of sight 17a relative to the upper edge of the upper area 21A and the line of sight 17b relative to the lower edge (i.e., the boundary line 22 described above). Virtual images relating to specific information, such as the arrow icon 24, are not displayed below the downward line of sight 17b.

[0022] Next, pitch angle correction will be described with reference to FIG. 3. While traveling, a vehicle may tilt forward or backward due to, for example, unevenness in the road surface, i.e., the pitch angle α may oscillate and change. FIG. 3(a) shows an ideal state in which a vehicle 31 is traveling horizontally (pitch angle α = 0) on a horizontal road surface 30, and a virtual object 20A is virtually displayed on the coordinates of the target object 20 within a display angle 17 corresponding to an upper region 21A of the AR-HUD 1. In contrast, FIG. 3(b) shows a state in which the vehicle 31 is tilted upward (the pitch angle α is negative). In this state, the display angle 17 of the AR-HUD 1 is tilted upward with respect to the road surface 30, so that the virtual object 20A is virtually displayed at a relatively high position. Therefore, the coordinate point in real space on which the virtual object 20A is superimposed is farther away than the coordinates of the actual target object 20, causing the driver to mistakenly perceive the point indicated by the virtual object 20A as being relatively far away.

[0023] To counteract this phenomenon, as shown in FIG. 3C, a pitch angle correction of the display position is performed corresponding to the magnitude of the pitch angle α. That is, the display position is corrected in the opposite direction by an angle equal to the pitch angle α. Specifically, when the pitch angle α is negative, the display position is corrected in a direction in which the virtual object 20A is displaced downward relative to the display area 21 of the AR-HUD 1, and when the pitch angle α is positive, the display position is corrected in a direction in which the virtual object 20A is displaced upward. As a result, the virtual object 20A is virtually displayed at the corrected position indicated by the dashed line, for example, and the coordinates of the original target object 20 and the coordinate position indicated by the virtual object 20A coincide with each other.

[0024] On the other hand, when pitch angle correction is performed in this manner, if the original display position of virtual object 20A in upper region 21A is at a lower position close to boundary line 22, when the display position of virtual object 20A is corrected downward, virtual object 20A may move downward out of upper region 21A and disappear. Figures 4 are explanatory diagrams of this phenomenon, and Figure 4(a) shows an example of the display position of virtual object 20A when pitch angle α is 0, that is, when the vehicle is horizontal. When the coordinate point that virtual object 20A is attempting to indicate is relatively close, that is, when distance L shown in Figure 2 is relatively small, the display position of virtual object 20A in display region 21 (upper region 21A) will be shifted downward. In such a case, when the display position is corrected downward by pitch angle correction, the display position of virtual object 20A moves out of upper region 21A and will no longer be displayed, as shown in Figure 4(b). In particular, if the pitch angle α repeatedly changes between positive and negative as a pitch vibration, the virtual object 20A will repeatedly appear and disappear, making it difficult to see.

[0025] Therefore, in one embodiment, when the distance L to the object 20 is small, the display position correction amount for the pitch angle α is relatively small compared to when the distance L is large. For example, in one embodiment in which the display position correction amount is calculated by multiplying the pitch angle α by a gain G, the gain G is set smaller as the distance L is smaller. Note that a gain G of 1 means that the display position is corrected by an angle equal to the pitch angle α.

[0026] In one embodiment, the gain G can be set as a function of the distance L using an appropriate arithmetic expression, table, or map. When the distance L is sufficiently large, the gain G can be set to 1. The gain G set according to the distance L may have a characteristic that changes continuously or in steps according to the distance L.

[0027] In one embodiment, the gain G is calculated by the following calculation using the distance L. Fig. 5 is an explanatory diagram relating to the calculation of this gain G, and Fig. 5(a) shows the relationship between the virtual object 20A and the downward angle θ when the pitch angle α is 0. As shown in Fig. 5(a), "tan θ = H / L", and when θ is small, "θ = H / L...Equation 1" holds.

[0028] FIG. 1B shows a case where the pitch angle α [rad] is other than 0, indicating that the direction of the virtual object 20A is shifted by the angle α due to the inclination of the vehicle (pitch angle α), and a pitch angle correction of "-Gα" is performed to offset this. With this pitch angle correction, the display position of the virtual object 20A becomes "θ+α-Gα" as shown in the figure. Here, as shown in FIG. 1B, a maximum allowable deviation amount Δx [m] in the forward / backward direction is set with respect to the distance L. As shown in FIG. 1B, "tan{θ+(1-G)α}=H / (L-Δx)" holds, and when θ is small, "θ+(1-G)α=H / (L-Δx)...Equation 2" holds.

[0029] Solving Equation 1 and Equation 2 gives us "G = 1 - HΔx / {αL(L - Δx)}". Using this equation, we can find the gain G according to the distance L, and obtain the characteristic that the gain G decreases as the distance L decreases.

[0030] By reducing the gain of the pitch angle correction when the distance L is small in this way, the disappearance of the virtual object 20A (moving downward from the upper region 21A and disappearing) described with reference to FIG. 4 is less likely to occur.

[0031] Furthermore, when the distance L is equal to or less than a certain distance, the gain G may be set to 0 so that no pitch angle correction is performed. In Fig. 6, it is assumed that the maximum amplitude of the pitch angle α is αmax [rad]. If the downward angle with respect to the lower edge of the upper region 21A, i.e., the boundary line 22, is β [rad], then "β = θ + αmax...Equation 3" holds.

[0032] By solving Equation 1 and Equation 3, the relationship L = H / (β - αmax) is obtained. If the distance L that satisfies this relationship is set to distance L0, and gain G is controlled to be 0 when distance L to target object 20 becomes equal to or less than distance L0, virtual object 20A will not be displaced below boundary line 22 due to pitch angle correction.

[0033] In either case, the gain G may be limited only when the pitch angle α is negative, and the gain G may be kept at 1 when the pitch angle α is positive.

[0034] 8 shows a pin-shaped destination icon 25 as another example of a virtual object 20A displayed at a position corresponding to the coordinates of an object. The virtual object 20A for indicating such a specific coordinate position is subjected to pitch angle correction as described above, and the gain G becomes small when the distance L is small.

[0035] 9 to 12 show examples of virtual images (virtual objects) in which the pitch angle correction is continued with the gain G set to "G = 1" without limiting the gain G according to the distance L. In all of these virtual objects, the front-to-rear length (length along the vehicle's fore-and-aft direction) is equal to or greater than a certain value, and the virtual objects do not appear unnatural even if the lower part of the virtual object (in other words, the front part) is partially obscured by the boundary line 22 due to the pitch angle correction.

[0036] FIG. 9 shows an example in which a white line (e.g., a center line) that separates lanes is recognized and a strip-shaped virtual object (white line detection display) 31 is displayed. FIG. 10 shows a so-called carpet display in which a large arrow 32 is displayed on the road surface for route guidance. FIG. 11 shows an example in which a vehicle width display 33 consisting of a pair of straight lines indicating the vehicle's width is displayed by the AR-HUD 1. FIG. 12 shows an example of an L-shaped display 34 along a corner to prevent vehicles from getting caught in the vehicle when turning right or left. For these cases, pitch angle correction with G=1 is performed even when the distance L becomes small. This suppresses positional deviation of the virtual object due to changes in the pitch angle.

[0037] When the gain G is reduced to perform pitch angle correction when the distance L is short, the degree to which the gain G is reduced may be changed depending on the vehicle speed or acceleration.

[0038] In one example, when the vehicle speed is high, the gain G is relatively increased. Similarly, when the vehicle acceleration is high, the gain G is relatively increased. When the vehicle speed and acceleration are high, the influence of the positional displacement of the virtual object due to the change in the pitch angle α is large, so by relatively increasing the gain G, it is possible to suppress the positional displacement of the virtual object.

[0039] In another example, when the vehicle speed or acceleration is high, pitch angle correction makes it easier for the virtual object to extend beyond the lower edge of the upper region 21A, so in order to prevent this, gain G is corrected to be relatively small when the vehicle speed or acceleration is high.

[0040] Although one embodiment of the present invention has been described above, the present invention is not limited to the above embodiment and various modifications are possible. For example, the form of the virtual image or virtual object is not limited to the configuration shown in the drawings, and the present invention can be applied to virtual images of any other shape.

[0041] In addition, in the above embodiment, the lower part of the display area 21 is the lower area 21B for the fixed virtual image 23, but the present invention can also be applied in the same way when the entire display area 21 is used to display an arrow icon 24 or the like.

[0042] Furthermore, in the above embodiment, the amount of display position correction is relatively reduced when the distance L is small by changing the gain G, but the present invention is not limited to this, and the amount of display position correction may be reduced by other methods.

Claims

1. A vehicle information display method that acquires the coordinates of an object in real space, and displays a virtual image at a position corresponding to the coordinates of the object via the vehicle's AR-HUD, superimposing it on the view ahead seen through the windshield; acquires the distance between the vehicle and the object, detects the vehicle's pitch angle, and, in order to offset the effect of this pitch angle, corrects the display position according to the magnitude of the pitch angle, displacing the virtual image downward relative to the AR-HUD display area when the pitch angle is negative, and in the opposite direction when the pitch angle is positive; and when the distance is small, the amount of display position correction relative to the pitch angle is relatively smaller than when the distance is large.

2. The vehicle information display method according to claim 1, wherein the display position correction amount is calculated by multiplying the pitch angle by a gain, and the gain is set to be smaller as the distance is smaller.

3. The vehicle information display method according to claim 2, wherein the gain is continuously changed according to the distance.

4. The vehicle information display method according to claim 2, wherein the gain is set to 1 when the pitch angle is positive, and the gain is made smaller as the distance becomes smaller when the pitch angle is negative.

5. The vehicle information display method according to claim 1, wherein when the distance is smaller than a predetermined distance, the display position correction for the pitch angle is not performed.

6. The vehicle information display method according to claim 1, wherein the virtual image is a virtual image displayed on a road surface visible through a windshield.

7. The vehicle information display method according to claim 1, wherein the virtual image is a virtual image displayed for vehicle route guidance.

8. A vehicle information display method as described in claim 1, which determines whether the virtual image to be displayed is a type of virtual image that has a length in the fore-and-aft direction of the vehicle, and if it is a type of virtual image that has a length in the fore-and-aft direction of the vehicle, does not perform a decreasing correction of the display position correction amount when the above distance is small.

9. The vehicle information display method according to claim 2, wherein the gain, which is set according to the distance, is set to be relatively larger as the vehicle speed increases.

10. The vehicle information display method according to claim 2, wherein the gain, which is set according to the distance, is set to be relatively larger as the acceleration of the vehicle increases.

11. The vehicle information display method according to claim 2, wherein the gain, which is set according to the distance, is set to be relatively smaller as the vehicle speed increases.

12. The vehicle information display method according to claim 2, wherein the gain, which is set according to the distance, is set to be relatively smaller as the acceleration of the vehicle increases.

13. A vehicle information display program that causes an on-board computer to execute the vehicle information display method according to claim 1.

14. An information display device for a vehicle comprising: a display coordinate acquisition unit that acquires the coordinates of an object in real space; an AR-HUD that displays a virtual object superimposed on the view ahead seen through the windshield; and a control unit that controls the AR-HUD to display a virtual image showing the object as the object at a position corresponding to the coordinates of the object, wherein the control unit acquires the distance between the vehicle and the object, detects the pitch angle of the vehicle, and, in order to offset the effect of the pitch angle, corrects the display position in a direction that displaces the virtual image downward relative to the display area of ​​the AR-HUD when the pitch angle is negative, and in the opposite direction when the pitch angle is positive, according to the magnitude of the pitch angle, and when the distance is small, makes the amount of display position correction relative to the pitch angle relatively smaller compared to when the distance is large.

Images