Information display method, device, and program for vehicle

Abstract

An information display device of a vehicle acquires coordinates of an object (20) in a real space, and displays a virtual object (20A) in a position corresponding to the coordinates of the object (20) in superimposition with a front view seen through a windshield (2) via an AR-HUD (1). A controller (3) detects a pitch angle (α) of the vehicle, and applies a pitch angle correction of "-Gα" to a display position of the virtual object (20A) deviated by "θ+α" in order to offset the influence of the pitch angle (α). A gain (G) is set to a value that decreases as a distance (L) decreases, so that the virtual object (20A) does not go beyond a display region downward.

Description

Technical Field

[0001] This invention relates to vehicle information display using an AR-HUD (Augmented Reality Head-Up Display) to display virtual images overlaid with real-world scenery. Background Technology

[0002] In recent years, head-up displays (HUDs) that project various information needed for driving onto the windshield in front of the driver's seat have become increasingly popular. Furthermore, AR-HUDs (augmented reality head-up displays) are known as a technology that further develops HUDs. In an AR-HUD, virtual information (also known as virtual objects) composed of virtual images is overlaid on the real information seen through the windshield, resulting in less movement of the driver's viewpoint or focus compared to a conventional HUD that displays information on the windshield surface.

[0003] Virtual information can encompass various types of information, one of which is virtual information that should be displayed at specific coordinates in real space. For example, as guidance information for a car navigation system, virtual information such as arrows indicating intersections where left or right turns should be displayed in the real scene at a position overlapping with the intersection. Additionally, the location display of a destination, composed of appropriately shaped virtual objects, is also displayed in a way that overlaps with its real-world coordinates.

[0004] In such a display scenario, if the vehicle tilts forward or backward, the position of the virtual information displayed by the AR-HUD may shift vertically from its proper position corresponding to the coordinates in real space. In Patent Document 1, although not an AR-HUD, a car navigation system that overlays arrow-like guidance information with an image from a camera facing forward of the vehicle addresses the same issue by detecting the vehicle's forward or backward tilt and vertically correcting the display position of the guidance information to counteract the effect of the tilt.

[0005] When displaying virtual objects, or virtual images, via an AR-HUD, the driver perceives the object's distance using a downward-facing angle relative to the horizontal plane—a top-down view. Therefore, the closer the vehicle is to the location represented by the object, such as a destination or intersection, the lower the object is displayed. Since the AR-HUD's display area is finite compared to the field of vision through the windshield, the object appears closer to the lower edge of the display area as the distance to the intended location (coordinates) decreases.

[0006] Therefore, if a correction is made corresponding to the vehicle's tilt (i.e., pitch angle), the virtual object may deviate from the display area and disappear when its display position is corrected downwards. In particular, when pitch vibrations occur due to changes in the pitch angle caused by road surface unevenness, the virtual object may repeatedly appear or disappear and become difficult to see.

[0007] Existing technical documents

[0008] Patent documents

[0009] Patent Document 1: Japanese Patent Application Publication No. 2009-250827 Summary of the Invention

[0010] This invention is a method for displaying vehicle information. It obtains the coordinates of an object in real space and displays a virtual image at a position corresponding to the object's coordinates by overlaying it with the view visible through the windshield via the vehicle's AR-HUD. The method involves obtaining the distance between the vehicle and the object, detecting the vehicle's pitch angle, and, to counteract the effect of the pitch angle, performing a display position correction corresponding to the pitch angle in the direction the virtual image is displaced downwards relative to the AR-HUD's display area when the pitch angle is negative, and performing a display position correction corresponding to the pitch angle in the opposite direction when the pitch angle is positive. When the distance is small, the amount of display position correction for the pitch angle is relatively reduced compared to when the distance is large.

[0011] When the distance to the object is small, that is, when approaching the object, the amount of display position correction relative to the pitch angle becomes smaller. Therefore, it is less likely that the virtual image (virtual object) will deviate downwards from the display area due to the display position correction. Attached Figure Description

[0012] Figure 1 This is a structural illustration of an information display device according to an embodiment.

[0013] Figure 2 It is an explanatory diagram showing the relationship between the distance to the object and the top-down view.

[0014] Figure 3 This is an illustration of the effects of vehicle tilt (pitch angle).

[0015] Figure 4 This is an illustration of how to make virtual objects disappear through pitch angle correction.

[0016] Figure 5 This is an explanatory diagram regarding the gain of pitch angle correction.

[0017] Figure 6 This is an illustration of an example where the gain is set to 0 when the distance is closer than the specified distance.

[0018] Figure 7 This is an explanatory diagram showing an example of a path guidance display as seen through the windshield.

[0019] Figure 8 This is an illustrative diagram showing an example of the destination displayed as seen through the windshield.

[0020] Figure 9 This is an explanatory diagram showing an example of white line detection display without gain correction.

[0021] Figure 10 This is an illustrative diagram showing an example of a path guide display without gain correction.

[0022] Figure 11 This is an explanatory diagram showing an example of vehicle width display without gain correction.

[0023] Figure 12 This is an explanatory diagram illustrating an example of preventing entanglement during left and right turns without gain correction. Detailed Implementation

[0024] Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

[0025] Figure 1 This is an explanatory diagram illustrating the configuration of an information display device for a vehicle according to one embodiment. The vehicle in this embodiment is a typical automobile, equipped with an AR-HUD (Augmented Reality Head-Up Display) 1 relative to the driver's seat. This AR-HUD 1 overlays virtual information (virtual objects) onto real information seen through the windshield 2. In this invention, its form can be any, for example, a projector is provided at the lower part of the windshield 2, and a virtual image of the virtual information is generated at an imaginary focal plane at an appropriate distance (e.g., several meters) in front of the driver by light reflected from the surface of the windshield 2 and reaching the driver's eyes. As described later, the virtual information can include appropriate information such as virtual images indicating left and right turns for route guidance to be overlaid at intersections, displays of icons indicating destinations, speed limits, or actual vehicle speed.

[0026] AR-HUD1 is controlled by controller 3. That is, virtual images are generated in controller 3, and their display position is controlled. A car navigation system 4 using GPS is connected to controller 3 for vehicle route guidance. Car navigation system 4 possesses map information, including coordinate data of nodes such as intersections. Furthermore, the vehicle's position is determined through car navigation system 4. The illustrated system also includes information acquisition devices such as radar 5 and cameras 6 for detecting the positions of vehicles and pedestrians ahead. In addition, a node refers to a component of the road network in the digitized road map, along with road segments, representing points such as intersections and other nodes in the road network. Nodes and links each have unique numbers, and nodes are connected by links to form the road network. Hereinafter, nodes where left or right turns are indicated will also be referred to as intersections.

[0027] An information display device using an embodiment of AR-HUD1 includes a viewpoint detection camera 7 for detecting the driver's viewpoint position in order to ensure the proper positioning of the virtual image of AR-HUD1. The viewpoint detection camera 7 is, for example, part of a driver monitoring system (DMS) that monitors the driver's state. The viewpoint detection camera 7 is, for example, positioned near the upper edge of the windshield, facing the driver's head. The controller 3 processes the image acquired by the viewpoint detection camera 7 to detect the driver's viewpoint position (i.e., the position of the pupil). It should be noted that the driver's viewpoint position can be detected indirectly, without a camera, for example, based on the driver's seating position, height, etc. The controller 3 corrects the generation position of the virtual image of AR-HUD1, i.e., the object, based on the detected viewpoint position, so that the real scene seen by the driver through the windshield 2 correctly overlaps with the object in AR-HUD1.

[0028] The controller 3 functions as part of the vehicle's computer system, performing various controls. In route guidance mode via the car navigation system 4, it displays virtual images of left and right turns required for route guidance and virtual images of the destination via the AR-HUD1. The controller 3 also acquires the coordinates of nodes (intersections) via the car navigation system 4, acquires the coordinates of the destination, creates display data for virtual images, calculates the distance between the vehicle and the intersection / destination, and adjusts the gain for pitch angle correction (described later) corresponding to that distance.

[0029] Additionally, a pitch angle detection unit 8, which is composed of other vehicle control controllers, inputs a signal representing the vehicle's tilt in the forward / backward direction, i.e., the pitch angle α, to the controller 3. The pitch angle detection unit 8 detects the pitch angle α, for example, based on the detection signal from an acceleration sensor. Furthermore, the pitch angle α is positive when the front end of the vehicle is relatively downward (so-called downward), and negative when the front end of the vehicle is relatively upward (so-called upward). When the AR-HUD1 displays a virtual image representing the position of an object in real space (e.g., a virtual image representing a left or right turn at an intersection, or a virtual image representing a destination), as described later, the display position of the virtual image is corrected based on the detected pitch angle α. Hereinafter, this correction will be referred to as pitch angle correction.

[0030] Figure 7 This is a simplified illustration showing the view seen through the windshield 2 when a vehicle approaches an intersection (equivalent to an object) where a left turn is required. A rectangular display area 21 of AR-HUD 1 exists within a portion of the area seen through the windshield 2. The display area 21 is essentially positioned in front of the driver. The outer perimeter of the display area 21 is not actually shown but is illustrated for illustrative purposes. Various virtual information required for driving is displayed within this display area 21 via AR-HUD 1.

[0031] The illustrated display area 21 is divided into an upper area 21A and a lower area 21B by a boundary line 22 defined by regulations. The boundary line 22 is illustrated for illustrative purposes and is not actually displayed. Only specific information permitted by regulations, including guidance on the direction of travel, can be displayed in the upper area 21A. Information other than this specific information must be displayed in the lower area 21B, which is below the boundary line 22. As information other than this specific information, in the illustrated example, multiple virtual images 23, such as the guidance display icon 23a of the car navigation system 4, the icon 23b indicating the status of ACC (Adaptive Cruise Control), the icon 23c indicating whether to let go, the icon 23d indicating the speed setting, and the display of the set speed 23e and the current speed 23f, are displayed horizontally in a row in the lower area 21B. The respective display positions of these virtual images 23 for information other than this specific information within the lower area 21B are fixed. Hereinafter, they will be referred to as fixed virtual images 23. These fixed virtual images 23 are not subject to pitch angle correction. In addition, to prevent overlap with the fixed virtual image 23, specific information is displayed only in the upper region 21A and not in the lower region 21B.

[0032] A virtual image 24 (hereinafter referred to as arrow icon 24) indicating the left or right turn required when approaching an intersection where a turn is necessary is displayed as one of the specific pieces of information in the upper area 21A. This image is displayed overlaid on the real scene in a manner that appears at the location (node) of the intersection as shown in the coordinates. The arrow icon 24 in the example diagram is an example of a left turn, arranged with three arrow-shaped segments spaced apart. For example, the central segment corresponds to the location of the intersection node. Therefore, the road in the real space to the left of this arrow icon 24 is the road where a left turn should be made.

[0033] The arrow icons 24 that guide left and right turns are represented with appropriate colors and brightness. In addition, the size of the arrow icons 24 is calculated to have a certain size (e.g., a certain width) at the intersection point in 3D space. Therefore, within the display area 21, the longer the distance to the intersection point, the smaller it is displayed, and the shorter the distance, the larger it is displayed.

[0034] In AR-HUD1, when displaying virtual objects or virtual images, the driver identifies the distance to the object by looking down at an angle relative to the horizontal plane, i.e., a top-down view. Figure 2 This is an explanatory diagram regarding the top-down perspective. The driver observes a real object (e.g., an intersection) 20 from viewpoint 15. The top-down perspective θ is geometrically determined by the height H from the road surface to viewpoint 15 and the distance L from the vehicle to the object 20. Additionally, the plane labeled 16 in the diagram represents the imaginary focal plane of the virtual image generated by AR-HUD1. This focal plane 16 is located outside the windshield 2, i.e., in front of the vehicle. By displaying the virtual object (e.g., the arrow icon 24 mentioned above) at a position corresponding to the top-down perspective θ within this focal plane 16, the driver perceives the virtual object as being located on the coordinates of the real object. The outline of this focal plane 16 becomes the display area 21 of AR-HUD1, but... Figure 2 The focal plane 16 specifically refers only to the upper region 21A. The triangle marked with reference numeral 17 indicates the driver's viewing angle of the upper region 21A of the AR-HUD1, in other words, the display angle of the upper region 21A, determined by the line of sight 17a towards the upper edge of the upper region 21A and the line of sight 17b towards the lower edge (i.e., the aforementioned boundary line 22). Virtual images related to specific information, such as arrow icon 24, are not displayed below the lower line of sight 17b.

[0035] Next, based on Figure 3 The pitch angle correction is explained. During driving, the vehicle may tilt forward or backward due to road surface irregularities, meaning the pitch angle α may sometimes fluctuate. Figure 3(a) represents the ideal state where vehicle 31 is traveling horizontally (pitch angle α = 0) on a level road surface 30. Virtual object 20A is virtually displayed on the coordinates of object 20 within a display angle 17 corresponding to the upper region 21A of AR-HUD1. In contrast, Figure 3 (b) indicates the state where vehicle 31 is tilted upwards (pitch angle α is negative). In this state, the display angle 17 of AR-HUD1 is tilted upwards relative to the road surface 30, so the virtual object 20A is virtually displayed at a relatively high position. Therefore, the coordinates of the real space overlaid by the virtual object 20A are farther away than the coordinates of the original object 20, and the driver mistakenly believes that the location shown by the virtual object 20A is relatively far away.

[0036] To counteract this phenomenon, such as Figure 3 As shown in (c), a pitch angle correction is performed on the display position corresponding to the magnitude of the pitch angle α. That is, the display position is corrected in the opposite direction by an angle equal to the pitch angle α. Specifically, when the pitch angle α is negative, the display position is corrected in the direction that the virtual object 20A is shifted downward relative to the display area 21 of AR-HUD1; when the pitch angle α is positive, the display position is corrected in the direction that the virtual object 20A is shifted upward. Thus, for example, the virtual object 20A is virtually displayed at the corrected position shown by the dashed line, and the coordinates of the original object 20 are consistent with the coordinates shown by the virtual object 20A.

[0037] On the other hand, when pitch angle correction is performed in this manner, if the original display position of the virtual object 20A in the upper region 21A is located near the lower side of the boundary line 22, when the display position of the virtual object 20A is corrected downwards, the virtual object 20A may move downwards from the upper region 21A and disappear. Figure 4 This is an illustration of the phenomenon. Figure 4 (a) shows an example of the display position of virtual object 20A when the pitch angle α is 0, i.e., when the vehicle is level. When the coordinate point that virtual object 20A wants to represent is relatively close, i.e. Figure 2 When the distance L shown is relatively small, the virtual object 20A within display area 21 (upper area 21A) is displayed lower. In this case, when the display position is corrected downwards using tilt angle correction, as... Figure 4 As shown in (b), the virtual object 20A is displayed off-center from the upper region 21A, and the virtual object 20A is not displayed. In particular, when the pitch angle α changes repeatedly in the positive and negative directions as pitch vibration, the appearance and disappearance of the virtual object 20A occur repeatedly, making it difficult to see the virtual object 20A.

[0038] Therefore, in one embodiment, when the distance L to the object 20 is small, the amount of display position correction relative to the pitch angle α is relatively smaller compared to when the distance L is large. For example, in an embodiment where the amount of display position correction is calculated by multiplying the pitch angle α by the gain G, the smaller the distance L, the smaller the gain G is set. Furthermore, if the gain G is 1, it means that the display position is corrected to an angle equal to the pitch angle α.

[0039] In one embodiment, the gain G can be set as a function of distance L using appropriate formulas, tables, or mappings. When the distance L is sufficiently large, the gain G can be set to 1. The gain G set according to distance L can be a characteristic that changes continuously with respect to distance L, or it can be a characteristic that changes in stages.

[0040] In one embodiment, the gain G is calculated by using a distance L operation as described below. Figure 5 This is an explanatory diagram about the calculation of the gain G. Figure (a) shows the relationship between the virtual object 20A and the angle of view θ when the pitch angle α is 0. As shown in Figure (a), when "tanθ=H / L" and θ is small, "θ=H / L…Equation 1".

[0041] Figure (b) illustrates the case where the pitch angle α [rad] is not 0. This indicates that the direction of the virtual object 20A is offset by an angle α due to the vehicle's tilt (pitch angle α), and a pitch angle correction of "-Gα" is performed to offset this offset. Through this pitch angle correction, as shown in the figure, the display position of the virtual object 20A becomes "θ+α-Gα". Here, as shown in Figure (b), the maximum allowable offset Δx [m] in the forward and backward directions is set with respect to distance L. As shown in Figure (b), "tan{θ+(1-G)α}=H / (L-Δx)", when θ is small, becomes "θ+(1-G)α=H / (L-Δx)…Equation 2".

[0042] Solving equations 1 and 2 yields "G = 1 - HΔx / {αL(L - Δx)}". This equation can be used to calculate the gain G corresponding to the distance L, revealing that the smaller the distance L, the smaller the gain G.

[0043] Thus, by reducing the gain of pitch angle correction at a distance of L, it is difficult to generate a result based on... Figure 4 The virtual object 20A is described as disappearing (it goes down from the upper area 21A and disappears).

[0044] Additionally, when the distance L is below a certain distance, the gain G can be set to 0 without pitch angle correction. Figure 6 In this context, we assume the maximum amplitude of the pitch angle α is αmax [rad]. If we set the angle of attack relative to the lower edge of the upper region 21A, i.e., the boundary line 22, as β [rad], then it is "β=θ+αmax…Equation 3".

[0045] Solving equations 1 and 3 yields the relationship L = H / (β - αmax). If the distance L satisfying this relationship is set as distance L0, and the gain G is controlled to 0 when the distance L to object 20 is below distance L0, then through pitch angle correction, the virtual object 20A will not shift downwards towards the boundary line 22.

[0046] Furthermore, in either case, the gain G can be limited only when the pitch angle α is negative, and kept at 1 when the pitch angle α is positive.

[0047] exist Figure 8 In the example shown, a pin-shaped destination icon 25 is represented as another virtual object 20A displayed at a position corresponding to the coordinates of the target object. The virtual object 20A used to represent such a specific coordinate position is subjected to pitch angle correction as described above, and its gain G decreases as the distance L decreases.

[0048] on the other hand, Figures 9-12 Examples of virtual images (virtual objects) that continue to undergo pitch angle correction set to "G=1" without limiting the gain G corresponding to the distance L. These are virtual objects with a front-to-back length (length along the vehicle's front-to-back direction) of a certain value, and are virtual objects that do not feel unnatural even if the lower part (in other words, the near-front part) of the virtual object disappears partially due to the boundary line 22 through pitch angle correction.

[0049] Figure 9 This example shows the identification of white lines (e.g., center lines) that divide lanes and the display of strip-shaped virtual objects (white line detection display) 31. Figure 10 This refers to the so-called carpet display, where large arrows (32) are shown on the road surface for route guidance. Figure 11 This is an example of a vehicle width display 33, which is shown by AR-HUD1 and consists of a pair of straight lines representing the width of the vehicle. Figure 12 This is an example of an L-shaped display 34 along the corner used to prevent entanglement during left and right turns. For this purpose, a pitch angle correction of G=1 is also performed when the distance L decreases. This suppresses the positional shift of the virtual object caused by changes in the pitch angle.

[0050] When performing pitch angle correction with reduced gain G at a distance of L hours, the degree of reduction in gain G can also be adjusted according to vehicle speed and vehicle acceleration.

[0051] In one example, the gain G is relatively increased when the vehicle speed is high. Similarly, the gain G is relatively increased when the vehicle's acceleration is high. At high vehicle speeds and accelerations, the change in pitch angle α has a significant impact on the positional offset of the virtual object. Therefore, by relatively increasing the gain G, the positional offset of the virtual object can be suppressed.

[0052] In another example, when the vehicle speed or acceleration is high, the virtual object tends to protrude from the lower edge of the upper region 21A through pitch angle correction. Therefore, in order to suppress this situation, the gain G is corrected to be relatively small when the vehicle speed or acceleration is high.

[0053] The present invention has been described above as an embodiment, but the present invention is not limited to the above embodiment and various modifications are possible. For example, the form of the virtual image or virtual object is not limited to the structure shown in the figure, and the present invention can be applied to virtual images of any other shape.

[0054] Furthermore, in the above embodiment, the lower part of the display area 21 is designated as the lower area 21B for fixing the virtual image 23, but the present invention is also applicable when the entire display area 21 is used to display arrow icons 24, etc.

[0055] In addition, in the above embodiment, the display position correction amount is relatively reduced at a distance L by changing the gain G. However, the present invention is not limited to this, and the display position correction amount can also be reduced by other methods.

Claims

1. A method for displaying vehicle information, comprising acquiring the coordinates of an object in real space, and displaying a virtual image at the position corresponding to the coordinates of the object by overlaying it with the scene visible through the windshield via the vehicle's AR-HUD, wherein the method is characterized in that... Obtain the distance between the vehicle and the object. Detect the vehicle's pitch angle. To counteract the effect of the pitch angle, when the pitch angle is negative, a display position correction corresponding to the magnitude of the pitch angle is performed in the direction in which the virtual image is shifted downwards relative to the AR-HUD display area; when the pitch angle is positive, a display position correction corresponding to the magnitude of the pitch angle is performed in the opposite direction. When the distance is small, the amount of display position correction for the pitch angle is relatively reduced compared to when the distance is large.

2. The vehicle information display method according to claim 1, characterized in that, The above-mentioned display position correction amount is calculated by multiplying the above-mentioned pitch angle by the gain. The smaller the distance mentioned above, the smaller the gain should be set.

3. The vehicle information display method according to claim 2, characterized in that, The aforementioned gain varies continuously depending on the aforementioned distance.

4. The vehicle information display method according to claim 2, characterized in that, When the pitch angle is positive, set the above gain to 1. When the pitch angle is negative, the smaller the distance mentioned above, the greater the reduction in the aforementioned gain.

5. The vehicle information display method according to claim 1, characterized in that, When the distance is less than the specified distance, no display position correction will be performed for the above pitch angle.

6. The vehicle information display method according to claim 1, characterized in that, The aforementioned virtual image is a virtual image displayed on the road surface as seen through the windshield.

7. The vehicle information display method according to claim 1, characterized in that, The aforementioned virtual images are displayed for vehicle route guidance.

8. The vehicle information display method according to claim 1, characterized in that, Determine whether the displayed virtual image is a type of virtual image with length in the longitudinal direction of the vehicle. If the virtual image is of a type that has length in the forward and backward direction of the vehicle, the aforementioned reduction correction for the display position correction amount when the distance is small is not performed.

9. The vehicle information display method according to claim 2, characterized in that, The higher the vehicle speed, the greater the gain will be based on the distance set above.

10. The vehicle information display method according to claim 2, characterized in that, The higher the vehicle's acceleration, the greater the gain will be based on the distance specified above.

11. The vehicle information display method according to claim 2, characterized in that, The higher the vehicle speed, the smaller the gain will be based on the distance specified above.

12. The vehicle information display method according to claim 2, characterized in that, The higher the vehicle's acceleration, the smaller the gain set according to the distance will be.

13. A vehicle information display program, characterized in that, The onboard computer executes the vehicle information display method as described in claim 1.

14. An information display device for a vehicle, comprising: a coordinate acquisition unit that acquires the coordinates of an object in real space; an AR-HUD that displays a virtual object by overlapping it with a view seen through a windshield; and a control unit that controls the AR-HUD to display a virtual image representing the object at a position corresponding to the coordinates of the object, characterized in that... The aforementioned control unit obtains the distance between the vehicle and the object. Detect the vehicle's pitch angle. To counteract the effect of the pitch angle, when the pitch angle is negative, a display position correction corresponding to the magnitude of the pitch angle is performed in the direction in which the virtual image is shifted downwards relative to the AR-HUD display area; when the pitch angle is positive, a display position correction corresponding to the magnitude of the pitch angle is performed in the opposite direction. When the distance is small, the amount of display position correction for the pitch angle is relatively reduced compared to when the distance is large.

Images