Information display method and device for vehicle

Abstract

According to the present invention, a vehicle is provided with an AR-HUD (1), which can display information superimposed on a real scene visible through a windshield (2). During operation of an ACC system (8), a virtual image indicating a target vehicle (11) is displayed via the AR-HUD (1). When the distance to the target vehicle (11) is far, one horizontal rod-shaped first mark (M1) extending to the left and right is displayed below the target vehicle (11) visible through the windshield (2). When the distance to the target vehicle (11) is close, a pair of left and right second marks (M2) for indicating the target vehicle is displayed on both the left and right sides of the target vehicle (11).

Description

Vehicle information display method and device

[0001] The present invention relates to a vehicle information display that uses an AR-HUD (Augmented Reality Head-Up Display) to display a mark made of a virtual image near a target vehicle traveling ahead of the vehicle.

[0002] In recent years, HUDs (head-up displays), which project and display various information necessary while driving a vehicle onto the windshield in front of the driver's seat, have become popular, and AR-HUDs (augmented reality head-up displays) are known as a more advanced HUD technology. AR-HUDs can display virtual information consisting of virtual images superimposed on real information visible through the windshield, reducing the need for the driver to move their viewpoint or focus as compared to general HUDs, which display information on the glass surface of the windshield.

[0003] Patent Document 1 discloses that when a vehicle distance control device, known as ACC (adaptive cruise control), which drives by following a preceding vehicle, is activated, a horizontal bar-shaped mark (which is a virtual image) extending to the left and right is displayed below the preceding vehicle (which is real information) that is the target of the vehicle distance control, to indicate that it is a target vehicle recognized by the system.

[0004] This horizontal bar-shaped mark is basically displayed at a height position below the actual target vehicle, specifically, near the road surface below the rear end of the vehicle. However, when the inter-vehicle distance becomes shorter, for example, due to deceleration of the target vehicle, the height position near the road surface below the rear end of the vehicle where the virtual mark should be displayed becomes lower than the finitely sized display area of ​​the AR-HUD. Therefore, if the horizontal bar-shaped mark indicating the target vehicle is to be continuously displayed within the display area of ​​the AR-HUD, the virtual mark will be displayed overlapping the height position of the bumper or brake lights at the rear of the actual target vehicle, making it difficult for the driver to see.

[0005] JP 2023-018498 A

[0006] The vehicle information display method of the present invention acquires information on the position and external shape of a target vehicle traveling ahead of the vehicle, and if the distance to the target vehicle is far, displays a first mark in the shape of a single horizontal bar extending to the left and right below the target vehicle visible through the windshield via the vehicle's AR-HUD to indicate that it is the target vehicle, and if the distance to the target vehicle is close, displays a pair of second marks on the left and right sides of the target vehicle visible through the windshield via the AR-HUD to indicate that it is the target vehicle.

[0007] In other words, the display of the first mark and the second mark are switched depending on the distance from the vehicle to the target vehicle. With the first mark, which is a single horizontal bar extending to the left and right, even if the positional relationship between the virtual display mark and the real vehicle, i.e., the target vehicle, shifts slightly to the left or right due to a shift in viewpoint position or the like, the shift is less noticeable.

[0008] On the other hand, the second marks arranged on both the left and right sides of the target vehicle can be displayed without overlapping with the actual target vehicle even if the inter-vehicle distance becomes short.

[0009] FIG. 10 is an explanatory diagram of the configuration of an information display device according to an embodiment. FIG. 11 is an explanatory diagram of (a) a first mark and (b) a second mark displayed for a target vehicle. FIG. 12 is an explanatory diagram showing the relationship between the distance to the target vehicle and the downward angle. FIG. 13 is a flowchart showing the processing flow of an embodiment. FIG. 14 is an explanatory diagram of an embodiment including an intermediate mark. FIG. 15 is an explanatory diagram of an example of (a) a display path and (b) a lighting pattern. FIG. 16 is an explanatory diagram of a mark obtained by the display path and lighting pattern of FIG. 6. FIG. 17 is an explanatory diagram of another example of (a) a display path and (b) a lighting pattern. FIG. 18 is an explanatory diagram of a mark obtained by the display path and lighting pattern of FIG.

[0010] An embodiment of the present invention will now be described in detail with reference to the accompanying drawings.

[0011] FIG. 1 is an explanatory diagram showing the configuration of an information display device for a vehicle according to one embodiment. The vehicle according to this embodiment is a typical automobile equipped with an AR-HUD (Augmented Reality Head-Up Display) 1 in front of the driver's seat. The AR-HUD 1 displays virtual information superimposed on real information visible through a windshield 2. While any format is acceptable in the present invention, a projector may be provided below the windshield 2. Light reflected from the surface of the windshield 2 reaches the driver's eyes, generating a virtual image of the virtual information on a virtual focal plane located an appropriate distance (e.g., several meters) ahead of the driver. Examples of virtual information include virtual images superimposed on intersections indicating right and left turns for route guidance, as well as appropriate information such as speed limits and actual vehicle speeds. In this embodiment, however, the AR-HUD 1 also includes a display of a target vehicle traveling ahead, as described below. The AR-HUD 1 has a finitely sized display area, e.g., a rectangular display area elongated from side to side in the direction ahead of the driver.

[0012] The AR-HUD 1 is controlled by a controller 3. That is, the controller 3 generates a virtual image and controls its display position. The controller 3 is connected to an adaptive cruise control (ACC) system 8 of the vehicle. The ACC system 8 has a function of following a target vehicle traveling ahead while maintaining a constant inter-vehicle distance, and performs functions such as capturing and recognizing the target vehicle to be followed, controlling the vehicle's powertrain, and controlling the vehicle's braking system. Furthermore, under conditions where there is no preceding vehicle to be followed, the ACC system 8 drives the vehicle at a constant vehicle speed set in advance by the driver. Note that the ACC system 8 may automatically perform steering to follow the target vehicle, or may be configured to only control the inter-vehicle distance.

[0013] Information acquisition devices such as radar 5 and cameras 6 are connected to the controller 3 to detect the positions and external shapes of vehicles, pedestrians, etc. ahead or around the vehicle. The information acquired by these information acquisition devices is also used in the ACC system 8. In addition, although not essential for the present invention, a car navigation system 4 using so-called GPS for providing route guidance for the vehicle is connected to the controller 3. The car navigation system 4 has map information, and this map information includes coordinate data of nodes such as intersections. The car navigation system 4 also identifies the vehicle's position.

[0014] Furthermore, an information display device using the AR-HUD 1 according to one embodiment includes a viewpoint detection camera 7 that detects the driver's viewpoint position in order to optimize the position of the virtual image of the AR-HUD 1. The viewpoint detection camera 7 constitutes, for example, part of a driver monitoring system (DMS) that monitors the driver's condition. The viewpoint detection camera 7 is disposed, for example, near the upper edge of the windshield, facing the driver's head. The controller 3 processes images acquired by the viewpoint detection camera 7 to detect the driver's viewpoint position (i.e., the position of the driver's eyes). Note that the driver's viewpoint position may be detected without using a camera, and may be detected indirectly, for example, from the driver's seating position, driver height, or the like. The controller 3 corrects the position at which the virtual image, i.e., the object, is generated by the AR-HUD 1 according to the viewpoint position thus detected, so that the real scene seen by the driver through the windshield 2 and the object of the AR-HUD 1 are properly superimposed.

[0015] The controller 3 is configured as a part of the on-board computer system that performs various controls, and while the ACC system 8 is operating, it displays a target vehicle mark consisting of a virtual image near the target vehicle in the real scene via the AR-HUD 1 to indicate to the driver that the ACC system 8 has captured and recognized the target vehicle to be followed and which vehicle is the target vehicle.

[0016] That is, the controller 3 acquires information about the position and external shape of a target vehicle recognized by the ACC system 8 as a tracking target based on information from information acquisition devices such as the radar 5 and the camera 6. For example, the distance between the vehicle and the target vehicle is calculated using the radar 5, and image processing of the camera image can be used to obtain information such as whether the target vehicle is a passenger car or a large truck, as well as information such as the center of gravity position and external contour of the vehicle image. Then, based on the distance between the vehicle and the target vehicle, a first mark is displayed via the AR-HUD 1 in the long distance area, and a second mark is displayed via the AR-HUD 1 in the close distance area.

[0017] 2A is an explanatory diagram showing a first mark M1, which is a virtual image, superimposed on a target vehicle 11 in the real environment. The first mark M1 has the shape of a horizontal bar extending to the left and right below the target vehicle 11 visible through the windshield 2. For example, the first mark M1 is displayed in bright green. In a preferred example, the first mark M1 has a spindle shape that gradually tapers at both left and right ends.

[0018] The first mark M1 generated as a virtual image by the AR-HUD1 is displayed in 3D space as if it were located near the road surface below the rear end of the target vehicle 11, and is displayed, for example, at a position with an appropriate distance L1 from the lower edge of the vehicle (for example, the lower end of the tire). The length of the first mark M1 in the left-right direction (vehicle width direction) is set to be slightly longer than the vehicle width of the target vehicle 11.

[0019] In such a first mark M1, even if the display position of the first mark M1 is slightly shifted to the left or right with respect to the actual target vehicle 11 due to a shift in the viewpoint position or an error in detecting the target vehicle position at a long distance, the shift is not noticeable and does not cause discomfort to the driver.

[0020] 2(b) is an explanatory diagram showing a second mark M2, which is a virtual image used in the close-range area, superimposed on the target vehicle 11 in the real environment. The second mark M2 consists of a pair of marks displayed as a single bar extending in the vertical direction on both the left and right sides of the target vehicle 11 seen through the windshield 2. For example, the second mark M2 is displayed in bright green, like the first mark M1. In a preferred example, the second mark M2 is spindle-shaped, gradually tapering at both the top and bottom ends.

[0021] The second mark M2 generated as a virtual image by the AR-HUD 1 is displayed in 3D space as if it were on both the left and right sides of the rear end of the target vehicle 11, for example, near the side edge of the vehicle but not overlapping the side edge, with an appropriate distance L2 in the left-right direction. The length of the second mark M2 in the up-down direction (vehicle height direction) is set to approximately correspond to the range of the center of the body of the target vehicle 11.

[0022] Such a second mark M2 can be displayed within the display area of ​​the AR-HUD 1 even when the inter-vehicle distance from the subject vehicle to the target vehicle 11 is small. In other words, as the inter-vehicle distance becomes small, the angle (angle with respect to the horizontal plane) of the view point looking down on the rear end of the target vehicle 11 from the viewpoint position becomes large, and the position where the first mark M1 should be displayed may be below the lower edge of the display area of ​​the AR-HUD 1. With such an inter-vehicle distance, the first mark M1 cannot be displayed in the desired positional relationship shown in FIG. 2( a). On the other hand, since the second mark M2 is positioned on both the left and right sides of the target vehicle 11, it is possible to continue displaying the second mark M2 in the desired positional relationship shown in FIG. 2( b) as long as the actual target vehicle 11 is within the display area of ​​the AR-HUD 1.

[0023] FIG. 3 is an explanatory diagram of the above-mentioned look-down angle. The driver looks down at the rear end of the real target vehicle 11 from his / her viewpoint 15, and the look-down angle θ1 at that time is geometrically determined by the height Hd of the viewpoint 15 from the road surface and the distance x from the host vehicle to the target vehicle 11. This look-down angle θ1 relative to the rear end of the target vehicle 11 is referred to as the "first look-down angle." The plane labeled 16 in the figure indicates a virtual focal plane on which a virtual image is generated by the AR-HUD 1. This focal plane 16 is located outside the windshield 2, i.e., in front of the vehicle. The outline of this focal plane 16 is the display area of ​​the AR-HUD 1. The triangle labeled 17 indicates the visual angle at which the driver views the display area of ​​the AR-HUD 1. The look-down angle θ2 is determined by the driver's line of sight relative to the lower edge of the display area. This look-down angle θ2 relative to the lower edge of the display area is referred to as the "second look-down angle."

[0024] At the distance x illustrated in FIG. 3, the first look-down angle θ1 is smaller than the second look-down angle θ2, and the rear end of the target vehicle 11 is visible within the display area of ​​the AR-HUD 1. Therefore, the first mark M1 can be displayed. As the inter-vehicle distance x decreases, the first look-down angle θ1 becomes larger than the second look-down angle θ2. In this state, the rear end of the target vehicle 11 extends downward beyond the display area of ​​the AR-HUD 1, and the first mark M1 cannot be displayed. Therefore, at such an inter-vehicle distance x, the display switches to the second mark M2, and a virtual image of the target vehicle 11 is displayed.

[0025] FIG. 4 is a flowchart showing the process flow for displaying the target vehicle, which is repeatedly executed upon activation of the ACC system 8. First, the distance x from the vehicle to the target vehicle 11 is detected (step 1), and a first look-down angle θ1 is calculated based on this distance x (step 2). The second look-down angle θ2 relative to the display area is constant once the position of the viewpoint 15 is determined. The first look-down angle θ1 and the second look-down angle θ2 are compared (step 3). If the first look-down angle θ1 is smaller than the second look-down angle θ2, the first mark M1 is selected (step 5). If the first look-down angle θ1 is equal to or greater than the second look-down angle θ2, the second mark M2 is selected (step 4). Either of the selected marks is displayed on the AR-HUD 1 in step 6. Steps 1 to 6 are repeated while the ACC system 8 is ON, and the display ends when the ACC system 8 is turned OFF (step 7).

[0026] Furthermore, since the first mark M1 is displayed below the height position of the rear end of the target vehicle 11, more preferably, the first downward look-down angle θ1 may be a downward look-down angle relative to a position slightly below the lower edge of the rear end of the target vehicle 11 (a position corresponding to the first mark M1).

[0027] In other words, it is desirable to switch the display from the first mark M1 to the second mark M2 at a distance such that the display position of the first mark M1 to be displayed below the target vehicle 11 visible through the windshield 2 is below the lower edge of the display area of ​​the AR-HUD 1.

[0028] Next, in the example of Figure 2 described above, the display is simply switched between the first mark M1 and the second mark M2. However, in order to avoid giving the driver a sense of discomfort, it is also possible to include the display of an intermediate mark that transitions from the first mark M1 to the second mark M2 as the inter-vehicle distance becomes shorter.

[0029] FIG. 5 shows an embodiment in which two intermediate marks (first intermediate mark MM1 and second intermediate mark MM2) are provided between the shape of the first mark M1 and the shape of the second mark M2. For example, as the target vehicle 11 approaches the host vehicle, the display changes in the order of "(a) → (b) → (c) → (d)." Note that a rectangular frame 18 in the figure indicates the display area of ​​the AR-HUD 1. This frame 18 is not actually displayed. Diagram (a) shows a state in which the inter-vehicle distance is sufficiently large, and the aforementioned single horizontal bar-shaped first mark M1 is displayed below the rear end of the target vehicle 11. Diagram (d) shows a state in which the inter-vehicle distance is the smallest, and the aforementioned pair of left and right second marks M2 are displayed on the left and right sides of the target vehicle 11.

[0030] 1B shows the first intermediate mark MM1 displayed at the beginning of the transition from the first mark M1 to the second mark M2. The first intermediate mark MM1 has a shape that looks like a single horizontal bar of the first mark M1 that separates in the center and moves left and right.

[0031] Figure (c) shows the second intermediate mark MM2 that is displayed after the first intermediate mark MM1 when transitioning from the first mark M1 to the second mark M2. This second intermediate mark MM2 has a shape that looks like the first intermediate mark MM1 moves further away to the left and right, and then extends upward near the tire.

[0032] Therefore, as the shape changes continuously from the first mark M1 to the first intermediate mark MM1 and the second intermediate mark MM2 to the second mark M2, it becomes easier for the driver to understand that the first mark M1 and the second mark M2 are indications having the same function, that is, both indicate the target vehicle 11 that is to be followed.

[0033] Next, FIGS. 6 and 7 show examples of setting a continuous virtual display path to realize continuous display changes including intermediate marks. FIG. 6( a) shows an example of a virtual display path 21. The virtual display path 21 is generated as a single virtual line extending continuously from below the target vehicle 11 visible through the windshield 2, surrounding both the left and right sides. Then, as shown in FIG. 6( b), by making a partial section of the display path visible, as shown in FIG. 7, (a) a first mark M1, (b) an intermediate mark MM, and (c) a second mark M2 are displayed. For explanatory purposes, the display path in FIG. 6( a) is numbered from 0 to 100 to indicate the position in the path. Correspondingly, FIG. 6( b) shows the illumination section within the length from 0 to 100. The illumination patterns in columns a, b, and c in FIG. 6( b) correspond to the virtual images in FIGS. 7( a), 7( b), and 7( c), respectively.

[0034] For example, by making the central portion of the display path visible as shown in section a of Fig. 6(b), a virtual display like Fig. 7(a) is obtained, and by making both intermediate portions of the display path visible as shown in section b of Fig. 6(b), a virtual display like Fig. 7(b) is obtained. Furthermore, by making both end portions of the display path visible as shown in section c of Fig. 6(b), a virtual display like Fig. 7(c) is obtained. The display path is set so that when a partial section is made visible in this way, this virtual image does not overlap with part of the actual target vehicle 11.

[0035] Although the figure shows three stages of display, the lighting period can be controlled more finely between the states (a) and (c) in Figure 7, in other words, so that it changes continuously.

[0036] 8 and 9 show similar examples of setting a virtual display path. In this example, the shape of the display path is slightly different from the examples in FIGS. 6 and 7, with the vertically extending sections on both sides of the target vehicle 11 curved in a generally S-shape. As a result, particularly when a pair of left and right second marks M2 are set as shown in FIG. 9(c), each of the left and right marks has a shape similar to an arrow, making it easier for the driver to recognize the vehicle sandwiched between the two arrow shapes as the target vehicle 11.

[0037] In this configuration where display paths are set and the display transitions continuously, it is easy to see that even if the display changes, it is a display having the same function.

[0038] While one embodiment of the present invention has been described above, the present invention is not limited to the above embodiment and various modifications are possible. For example, in the flowchart of FIG. 4, switching between the first mark M1 and the second mark M2 is performed based on the magnitude relationship between the first look-down angle θ1 and the second look-down angle θ2. However, switching between the first mark M1 and the second mark M2 may be performed by comparing the distance x with an appropriate threshold value. Alternatively, the first look-down angle may be compared with an appropriate threshold value.

[0039] In the above embodiment, the target vehicle is described as the vehicle to be followed when the ACC system is used for following. However, the target vehicle to be displayed in the present invention is not limited to this, and the present invention can be widely applied as an information display for vehicles that require some kind of display, without involving following or vehicle distance control.

[0040] Furthermore, in the above explanation, we have mainly described a situation in which the inter-vehicle distance becomes shorter, but conversely, as the inter-vehicle distance becomes longer from the short distance range in which the second mark M2 is displayed, the display will switch from the second mark M2 to the first mark M1. The same applies when an intermediate mark is provided. Note that when the display is switched discontinuously, it is desirable to provide an appropriate hysteresis to the distance or downward angle at which the switching occurs in order to prevent frequent switching.

Claims

1. Obtain information on the position and outer shape of a target vehicle traveling ahead of the host vehicle. When the distance to the target vehicle is far, via the AR-HUD of the vehicle, below the target vehicle visible through the windshield, display a single horizontally extending bar-shaped first mark to indicate that it is the target vehicle. When the distance to the target vehicle is close, via the AR-HUD, on both the left and right sides of the target vehicle visible through the windshield, display a pair of left and right second marks to indicate that it is the target vehicle. A method for displaying vehicle information.

2. The second mark is displayed in a bar shape extending vertically in the vicinity of the side edge of the target vehicle and not overlapping the side edge. The method for displaying vehicle information according to claim 1.

3. Perform a display switch from the first mark to the second mark at a distance where the display position of the first mark to be displayed below the target vehicle visible through the windshield is below the lower edge of the display area of the AR-HUD. The method for displaying vehicle information according to claim 1.

4. Determine the driver's viewpoint position, determine a first downward viewing angle from this viewpoint position to the lower rear edge of the target vehicle, determine a second downward viewing angle from this viewpoint position to the lower edge of the display area of the AR-HUD, and display the second mark when the first downward viewing angle is larger than the second downward viewing angle. The method for displaying vehicle information according to claim 1.

5. At an intermediate distance between the long-distance area where the first mark is displayed and the short-distance area where the second mark is displayed, display an intermediate mark having an intermediate shape that transitions from the first mark to the second mark. The method for displaying vehicle information according to claim 1.

6. Generate a virtual display path that continuously extends to surround both the left and right sides from below the target vehicle visible through the windshield, and make a partial section of this display path visible to display the first mark, the intermediate mark, and the second mark. The method for displaying vehicle information according to claim 5.

7. Have hysteresis between a first switching distance for switching from the first mark to the second mark when approaching the target vehicle and a second switching distance for switching from the second mark to the first mark when leaving the target vehicle. The method for displaying vehicle information according to claim 1.

8. The vehicle information display method according to claim 1, wherein the target vehicle is a vehicle recognized as a following target during the operation of the inter-vehicle distance control device of the vehicle.

9. An information acquisition unit that acquires information on the position and outer shape of a target vehicle traveling ahead of the host vehicle, an AR-HUD that displays a virtual object superimposed on the forward scene visible through the windshield, and a control unit that controls the AR-HUD to display a mark indicating that the vehicle is the target vehicle as a virtual object in the vicinity of the target vehicle visible through the windshield. The control unit displays a single horizontally extending bar-shaped first mark indicating that the vehicle is the target vehicle below the target vehicle when the distance to the target vehicle is far, and a pair of left and right second marks indicating that the vehicle is the target vehicle on both the left and right sides of the target vehicle when the distance to the target vehicle is near. A vehicle information display method.

Images