Virtual Image Display Device

Abstract

To provide a virtual image display device capable of distinguishing and recognizing presence of each target even when a plurality of targets are densely present in a range where a relative position from an own vehicle is close.SOLUTION: A virtual image display device includes: a surrounding state recognition unit 6 configured to recognize a traffic participant present around a vehicle as a target to acquire target position information including information on a relative position between an own vehicle and the target; and a display control unit 8 configured to display target icon images I1 and I2 at a display position in a display screen 51 determined based on the target position information. The display control unit 8 can switch between an image display under a normal display mode in which a display position is determined based on the target position information so that a driver can recognize a relative position between the own vehicle and the target and an image display under an enlarged display mode for determining the display position so that an interval between two target icon images on the display screen 51 becomes larger than the normal display mode.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to a virtual image display device, and more particularly to a virtual image display device that displays target images of targets present around a moving object on a display screen that is visible to a driver. [Background technology]

[0002] In recent years, in order to improve traffic safety, virtual image display devices have been proposed that display useful virtual images to support safe driving on a monitor that can be viewed by a driver while driving. For example, in the device shown in Patent Document 1, an image acquired by an external camera facing the rear of the vehicle is displayed on a monitor. This allows the driver to recognize the presence of an object behind the vehicle by viewing the monitor when backing up the vehicle, thereby improving traffic safety. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2009-226978 [Patent Document 2] Japanese Patent Publication No. 2020-013393 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in general virtual image display devices, in order to enable the driver to quickly compare the scenery that can be seen directly with his or her own eyes with the virtual image displayed on the monitor, the virtual image is often displayed on the monitor in a manner that gives the viewer a sense of perspective.

[0005] However, in a virtual image drawn based on perspective, the size of the target image decreases as the distance from the vehicle in the depth direction (i.e., the direction of travel) increases. Furthermore, the display position of the target image gradually approaches the vanishing point as the distance from the vehicle in the depth direction increases. Therefore, in a virtual image drawn based on perspective, for example, when multiple targets are located far from the vehicle, it is difficult for the driver to recognize the distance between these distant targets (especially the distance in the depth direction).

[0006] In contrast, in the invention described in Patent Document 1, when the size of a predetermined notification target on the display screen is equal to or smaller than a threshold and the driver has difficulty recognizing it, an enlarged image of the notification target on the display screen is additionally displayed. According to the invention described in Patent Document 1, it is possible to make it easier to recognize targets that are far away from the vehicle.

[0007] However, when multiple targets are closely spaced, they are displayed overlapping on the display screen, and simply enlarging the size of the display screen can make it difficult to distinguish and recognize individual targets. Also, when multiple targets are present within the display screen, if an enlarged image is additionally displayed on the display screen as in the invention described in Patent Document 1, there is a risk that the enlarged image will obscure other targets.

[0008] The present invention aims to provide a virtual image display device that allows the user to distinguish and recognize the presence of individual targets even when multiple targets are densely located within a range close in relative position to the user's vehicle. [Means for solving the problem]

[0009] (1) A virtual image display device according to the present invention (for example, a virtual image display device 1 described later) recognizes traffic participants present around a moving body (for example, a vehicle V described later) as targets, and displays target images (for example, target icon images I1 and I2 described later) relating to the targets on a display screen (for example, a display screen 51 described later) that is visible to a driver of the moving body. The virtual image display device includes a surrounding condition recognition unit (for example, a virtual image display device 1 described later) that acquires surrounding condition information relating to the conditions around the moving body and acquires target position information including information relating to the relative position between the moving body and the target based on the surrounding condition information. a surroundings state recognition unit 6) and a display control unit (for example, a display control unit 8 described later) that displays the target image at a display position within the display screen determined based on the target position information, wherein the display control unit is capable of switching between image display under a normal display mode in which the display position is determined based on the target position information so that the driver can recognize the relative position between the moving body and the target, and image display under an enlarged display mode in which the display position is determined so that the distance between the two target images on the display screen is larger than in the normal display mode.

[0010] (2) In this case, when the display control unit displays multiple target images on the display screen in the enlarged display mode, it is preferable that the display control unit displays target-related information associated with each target near the target image.

[0011] (3) In this case, it is preferable that the target-related information includes at least one of position information, moving direction information, moving speed information, and type information of the target.

[0012] (4) In this case, it is preferable that the display control unit displays the target image in a manner that creates a sense of perspective on the display screen, and when displaying an image in the enlarged display mode, determine the display position so that both or either the vertical spacing between the two target images along the vertical direction of the screen and the horizontal spacing along the horizontal direction of the screen are larger than in the normal display mode.

[0013] (5) In this case, the display screen is divided into a short-distance display area (e.g., short-distance display area R1 described below) in which the target image of a short-distance target whose travel distance along the travel direction of the moving body to the target is less than a distance threshold is displayed, and a long-distance display area (e.g., long-distance display area R2 described below) in which the target image of a long-distance target whose travel distance along the travel direction of the moving body is equal to or greater than the distance threshold is displayed, and when the display control unit displays multiple target images on the display screen in the enlarged display mode, it is preferable that the display position be determined so that the distance between two target images displayed in the long-distance display area is larger than in the normal display mode.

[0014] (6) In this case, it is preferable that the display control unit displays an image in the enlarged display mode when a first target and a second target are recognized by the surrounding condition recognition unit and the distance between the first target and the second target along the direction of travel is less than a distance threshold.

[0015] (7) In this case, the virtual image display device preferably further includes a gaze direction acquisition unit (e.g., gaze direction acquisition unit 7 described later) that acquires the gaze direction, which is the direction of the driver's line of sight or face, and the display control unit preferably displays the image in the enlarged display mode when the target is present in the gaze direction.

[0016] (8) In this case, it is preferable that the display control unit switches between image display in the normal display mode and image display in the enlarged display mode based on the surrounding state information.

[0017] (9) In this case, it is preferable that the surrounding condition recognition unit recognizes traffic participants existing in the direction of travel from the moving body as the targets based on the surrounding condition information, and the display control unit displays images in the enlarged display mode when, based on the surrounding condition information, it recognizes more than a predetermined number of the targets within a predetermined time period, recognizes an area where the density of the targets is greater than a predetermined value, recognizes the presence of a specific facility or equipment in the direction of travel from the moving body, or recognizes the presence of a specific type of road in the direction of travel from the moving body. [Effects of the Invention]

[0018] (1) In the present invention, the surrounding condition recognition unit recognizes traffic participants present around the moving body, i.e., the host vehicle, as targets and acquires target position information including information on the relative position between the host vehicle and the targets. The display control unit displays target images at display positions within the display screen determined based on the target position information. The display control unit is also capable of switching between image display in a normal display mode, in which the display position is determined based on the target position information so that the driver can recognize the relative position between the moving body and the targets, and image display in an enlarged display mode, in which the display positions are determined so that the distance between two target images on the display screen is greater than in the normal display mode. Therefore, according to the present invention, by displaying images in such an enlarged display mode, even if two or more targets are closely spaced within a range of close relative positions from the host vehicle, these target images are displayed with a gap between them on the display screen, allowing the driver to distinguish and recognize the presence of each target, thereby improving traffic safety.

[0019] (2) In the present invention, when a plurality of target images are displayed on the display screen in the enlarged display mode, the display control unit displays target-related information associated with each target near the target image. That is, in the enlarged display mode, the target images of two targets are displayed on the display screen at a greater distance than in the normal display mode, and the target-related information is displayed near the target images. This makes it possible to display a large amount of information without causing a viewer to feel cluttered, thereby improving traffic safety.

[0020] (3) In the present invention, the target-related information includes at least one of target position information, moving direction information, and type information. Therefore, according to the present invention, information that is difficult to grasp from the target image alone, such as the target position, moving direction, moving speed, and type, can be quickly recognized.

[0021] (4) In the present invention, the display control unit displays target images on the display screen in a manner that creates a sense of perspective. When displaying images in the enlarged display mode, the display control unit determines the display positions so that the vertical spacing between two target images along the vertical direction of the screen (i.e., the distance between the two targets along the traveling direction) and / or the horizontal spacing between two target images along the horizontal direction of the screen (i.e., the distance between the two targets along the width direction) are larger than in the normal display mode. Therefore, according to the present invention, the spacing between target images on the display screen can be made larger than in the normal display mode without impairing the sense of perspective.

[0022] (5) In the present invention, the display control unit displays target images on the display screen in a manner that creates a sense of perspective, and divides the display screen into a short-distance display area in which target images of short-distance targets are displayed, and a long-distance display area that is an area vertically above the short-distance display area and in which target images of long-distance targets are displayed. In this case, the display positions of long-distance target images displayed in the long-distance display area converge to a virtual horizon line within the display screen as the distance from the vehicle along the traveling direction increases, making it easier for two target images to overlap. Therefore, in the present invention, when multiple target images are displayed on the display screen in the enlarged display mode, the display control unit determines the display positions so that the distance between two target images displayed in the long-distance display area is greater than in the normal display mode. Therefore, according to the present invention, it is possible to enable the driver to distinguish and recognize the presence of multiple long-distance targets that are located in positions where the target images would overlap in the normal display mode.

[0023] (6) In the present invention, when the first and second targets are recognized and the distance between the first and second targets along the traveling direction is equal to or less than the distance threshold, the display control unit displays images in the enlarged display mode. As a result, when two target images would overlap in the normal display mode, the display control unit displays images in the enlarged display mode, so that the two target images do not overlap, allowing the driver to distinguish and recognize the presence of each target.

[0024] (7) In the present invention, the gaze direction acquisition unit acquires the gaze direction, which is the direction of the driver's line of sight or face, and the display control unit displays an image in an enlarged display mode when a target exists in the gaze direction, i.e., when the driver is gazing at a target, thereby enabling the driver to distinguish and recognize multiple targets existing in the direction of gaze.

[0025] (8) In the present invention, the display control unit switches between image display in the normal display mode and image display in the enlarged display mode based on surrounding condition information about the surroundings of the vehicle. Therefore, according to the present invention, it is possible to switch between the normal display mode and the enlarged display mode at an appropriate timing in response to changes in the surroundings of the vehicle.

[0026] (9) In the present invention, the display control unit, based on the surrounding condition information, displays images in an enlarged display mode when it recognizes a predetermined number of targets or more within a predetermined time, recognizes an area where the target density is equal to or greater than a predetermined value, recognizes the presence of a specific facility or equipment on the traveling direction side of the vehicle, or recognizes the presence of a specific road shape on the traveling direction side of the vehicle. Therefore, according to the present invention, when there are already many targets on the traveling direction side or when it is predicted that many targets will appear on the traveling direction side, it displays images in an enlarged display mode, thereby enabling the driver to distinguish and recognize these multiple targets. [Brief explanation of the drawings]

[0027] [Figure 1] 1 is a diagram schematically illustrating the configuration of a virtual image display device according to an embodiment of the present invention and a vehicle equipped with the virtual image display device; [Figure 2] FIG. 1 is a plan view of a vehicle traveling on a road with three lanes on each side, viewed from above. [Figure 3] 3 is a diagram showing an example of a virtual image displayed on the display screen when a display control unit performs image display in a normal display mode in the situation shown in FIG. 2. FIG. [Figure 4] 3 is a diagram showing an example of a virtual image displayed on the display screen when a display control unit displays an image in an enlarged display mode in the situation shown in FIG. 2. FIG. [Figure 5] 10 is a flowchart showing a procedure for displaying a virtual image in a display control unit. DETAILED DESCRIPTION OF THE INVENTION

[0028] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A virtual image display device according to an embodiment of the present invention will be described below with reference to the drawings.

[0029] FIG. 1 is a diagram schematically showing the configuration of a virtual image display device 1 according to this embodiment and a vehicle V on which this virtual image display device 1 is mounted.

[0030] The virtual image display device 1 recognizes traffic participants around the moving body, particularly those traffic participants ahead of the moving body in the direction of travel from a viewpoint determined based on the moving body, as targets, and displays target images associated with these targets on a display screen 51 of a display 55 provided in a position visible to the driver of the moving body, thereby assisting the driver in recognizing targets around the moving body. Note that the following description will be given of a case in which the virtual image display device 1 is mounted on a vehicle V, which is a four-wheeled automobile, but the present invention is not limited to this. The present invention may also be mounted on moving bodies other than four-wheeled automobiles, such as trucks, saddle-type vehicles, and bicycles. Furthermore, when mounted on a moving body, particularly a saddle-type vehicle, the virtual image display device may be mounted on a helmet worn by the rider, who is the user.

[0031] The vehicle V is equipped with an external sensor unit 2 that acquires information about the area ahead in the direction of travel, a navigation device 3, an interior camera unit 4, and a virtual image display device 1 that displays virtual images on a display screen 51 to assist the driver in safe driving.

[0032] The external sensor unit 2 is composed of a front camera unit 21, a front radar unit 22, and the like.

[0033] The front camera unit 21 includes a digital camera using a solid-state imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The front camera unit 21 is attached, for example, to a position on the roof of the vehicle V, on the inside of the cabin, near the windshield, facing forward in the direction of travel. The front radar unit 22 includes a millimeter-wave radar that detects an object by measuring a wave reflected from the object in response to millimeter-wave irradiation. The front radar unit 22 is attached, for example, to the front bumper of the vehicle V, facing forward in the direction of travel.

[0034] The navigation device 3 includes, for example, a GNSS (Global Navigation Satellite System) receiver that identifies the current position of the vehicle based on signals received from GNSS satellites, a storage device that stores map information, etc. Here, the map information also includes information about road signs. The navigation device 3 transmits information about the current position of the vehicle to the virtual image display device 1 together with the map information of the current position.

[0035] The interior camera unit 4 includes, for example, a digital camera using a solid-state image sensor as described above. The interior camera unit 4 is installed inside the vehicle cabin facing the face of the driver sitting in the driver's seat. The image of the driver captured by the interior camera unit 4 is transmitted to the virtual image display device 1.

[0036] The virtual image display device 1 comprises a display 5 provided in a position visible to the driver while driving, a surrounding condition recognition unit 6 that acquires surrounding condition information regarding the conditions around the vehicle V acquired by an external sensor unit 2 and a navigation device 3 and recognizes the conditions around the vehicle V, a gaze direction acquisition unit 7 that acquires the gaze direction, which is the direction of the driver's line of sight or face, based on an image transmitted from the interior camera unit 4, and a display control unit 8 that displays a virtual image of the vehicle V ahead in the direction of travel, generated based on the recognition results by the surrounding condition recognition unit 6, on the display screen 51 of the display 5.

[0037] In this embodiment, the display control unit 8 displays a virtual image on the display screen 51 of the display 5 provided in front of the driver's seat, but the present invention is not limited to this. The display control unit 8 may also display a virtual image on a part of the windshield as a display screen by using a head-up display.

[0038] The surrounding condition recognition unit 6 recognizes one or more traffic participants located ahead in the traveling direction from a viewpoint determined for the vehicle V as targets based on surrounding condition information acquired from the external sensor unit 2 and the navigation device 3, and acquires target position information and target-related information for each target. The target position information includes information on the relative position between the viewpoint and each target. More specifically, the target position information includes information on the traveling direction distance from the viewpoint to each target along the traveling direction and information on the width direction distance from the viewpoint to each target along the width direction perpendicular to the traveling direction. The target-related information includes at least one of the target position information, moving direction information, moving speed information, and type information.

[0039] Here, the viewpoint set for vehicle V is set at any position, for example, inside the cabin of vehicle V (more specifically, at the driver's seat) or outside the cabin of vehicle V (more specifically, at a position a predetermined distance behind the rear end of vehicle V in the direction of travel). If the viewpoint is set at the driver's seat, a virtual image from the same viewpoint as the driver can be displayed. Also, if the viewpoint is set at a position away from the rear end of vehicle V, a bird's-eye view virtual image that includes the vehicle in the field of view can be displayed.

[0040] The surrounding condition recognition unit 6 acquires target position information and target-related information as described above, and also acquires traffic environment information of traffic participants including the vehicle V and the recognized targets, based on the surrounding condition information acquired from the external sensor unit 2 and the navigation device 3. Here, the traffic environment information includes information on the number of lanes on the road being traveled, the presence or absence of sidewalks, the types of facilities and equipment present ahead in the direction of travel, the shape of the road present ahead in the direction of travel, etc.

[0041] In this embodiment, the surrounding condition recognition unit 6 acquires surrounding condition information from the external sensor unit 2 and the navigation device 3 as in-vehicle devices that move together with the vehicle V, but the present invention is not limited to this. The surrounding condition recognition unit 6 may acquire surrounding condition information through vehicle-to-vehicle communication between the vehicle and another vehicle traveling nearby, or through communication between the vehicle and a server device that monitors the traffic conditions of the road on which the vehicle is traveling.

[0042] The display control unit 8 generates a virtual image that reproduces the view ahead in the traveling direction from the viewpoint based on the recognition result by the surrounding condition recognition unit 6, and displays this virtual image on the display screen 51 of the display 5. More specifically, the display control unit 8 draws target images associated with each target recognized by the surrounding condition recognition unit 6 superimposed at a predetermined position and with a predetermined size on a background image according to the traffic environment recognized by the surrounding condition recognition unit 6, thereby generating a virtual image that gives the viewer a sense of perspective.

[0043] The display control unit 8 is capable of generating a virtual image in a normal display mode or an enlarged display mode and displaying the virtual image. The procedure for generating a virtual image in each display mode will be described below using the situation shown in FIG. 2 as an example.

[0044] FIG. 2 is a plan view of a vehicle V traveling on a road 9 with three lanes on each side, viewed from above. FIG. 2 shows a case in which the host vehicle, vehicle V, is traveling in a center lane 91, and another vehicle 93 and a pedestrian 94 are present ahead of the host vehicle in the direction of travel. FIG. 2 also shows a case in which the other vehicle 93 is stopped in lane 92 next to the sidewalk, and the pedestrian 94 is about to enter lane 92 from the sidewalk ahead of the other vehicle 93 in the direction of travel. In the following, a case will be described in which a viewpoint 95 is set behind the vehicle V in the direction of travel, and the surrounding condition recognition unit 6 recognizes the other vehicle 93 as a first target and the pedestrian 94 as a second target. Therefore, in FIG. 2, "y1" indicates the distance in the direction of travel of the first target, "y2" indicates the distance in the direction of travel of the second target, "x1" indicates the distance in the width direction of the first target, and "x2" indicates the distance in the width direction of the second target.

[0045] <Normal display mode> Fig. 3 is a diagram showing an example of a virtual image displayed on the rectangular display screen 51 when the display control unit 8 performs image display in the normal display mode in the situation shown in Fig. 2. As shown in Fig. 3, the virtual image is composed of a background image Ib that resembles, for example, the lane markings on the road on which the vehicle is traveling, a host vehicle icon image Iv associated with the vehicle V, and target icon images I1 and I2 associated with the respective targets 93 and 94. In the following, the host vehicle icon image Iv and the target icon image I1 associated with the target vehicle are simplified by square marks, and the target icon image I2 associated with the target pedestrian is simplified by a circle.

[0046] When displaying images in the normal display mode, the display control unit 8 displays the vehicle icon image Iv and the target icon images I1 and I2 on a background image Ib generated according to the traffic environment recognized by the surrounding condition recognition unit 6, at positions and sizes that allow the viewer to recognize the relative positions between the vehicle and each target and create a sense of perspective. Note that the following describes a case in which only road dividing lines are displayed as the background image, but the present invention is not limited to this. In addition to road dividing lines, structures such as facilities and equipment present on the road may also be displayed as the background image, or a lattice-like grid may also be displayed as the background image.

[0047] When displaying images in normal display mode, the display control unit 8 reduces the size of each icon image Iv, I1, I2 on the display screen 51 as the travel direction distance, which is the distance along the travel direction from the viewpoint to the target, becomes longer.

[0048] When displaying images in the normal display mode, the display control unit 8 moves the display positions of each of the icon images Iv, I1, and I2 on the display screen 51 farther from a horizontal reference line L1 virtually defined within the display screen 51 in the horizontal direction of the screen as the widthwise distance from the viewpoint to the target increases, and moves closer to the horizontal reference line L1 in the horizontal direction of the screen as the distance in the traveling direction increases. Furthermore, the display control unit 8 displays targets that are on the right hand side as viewed from the viewpoint along the traveling direction on the right hand side of the horizontal reference line L1 on the display screen 51, and displays targets that are on the left hand side as viewed from the viewpoint along the traveling direction on the left hand side of the horizontal reference line L1 on the display screen 51. Note that the following description will be given of a case in which the horizontal reference line L1 extending along the vertical direction of the screen is set to the center of the display screen 51, but the present invention is not limited to this.

[0049] Furthermore, when displaying images in the normal display mode, the display control unit 8 moves the display position of each icon image Iv, I1, I2 on the display screen 51 farther away from the bottom edge line L2 of the display screen 51 along the vertical direction of the screen, which is perpendicular to the horizontal direction of the screen, as the distance from the viewpoint to the target in the travel direction becomes longer.

[0050] When displaying an image in the normal display mode, the display control unit 8 sets the display position and size of each icon image Iv, I1, and I2 on the display screen 51 according to the above-described procedure, thereby displaying a virtual image that allows the viewer to recognize the relative position between the vehicle and each target and that creates a sense of perspective. However, in the virtual image displayed on the display screen 51 in the normal display mode, the target icon images I1 and I2 associated with two targets 93 and 94 that are located far from the viewpoint along the traveling direction often overlap as shown in Fig. 3, making it difficult for the driver to distinguish between the multiple targets 93 and 94 that are located far from the viewpoint.

[0051] <Enlarged display mode> FIG. 4 is a diagram showing an example of a virtual image displayed on the display screen 51 when the display control unit 8 performs image display in the enlarged display mode in the situation shown in FIG.

[0052] As shown in Figure 4, when displaying an image in the enlarged display mode, the display control unit 8 displays the vehicle icon image Iv and target icon images I1 and I2 on a background image Ib generated according to the traffic environment recognized by the surrounding condition recognition unit 6 at a position and size that allows the viewer to recognize the relative positions between the vehicle and each target and creates a sense of perspective.

[0053] When displaying images in the enlarged display mode, the display control unit 8 changes the display positions of at least the target icon images I1 and I2 on the display screen 51 from those in the normal display mode to allow the viewer to easily distinguish and recognize multiple targets that are far from the viewpoint. More specifically, when displaying images in the enlarged display mode, the display control unit 8 determines the display positions of the target icon images I1 and I2 so that the distance between the two target icon images I1 and I2 on the display screen 51 is larger than the distance between the two target icon images I1 and I2 displayed on the display screen 51 in the normal display mode.

[0054] More specifically, the display control unit 8 extracts, from among the multiple targets recognized by the surrounding condition recognition unit 6, those whose target icon images are spaced apart by a predetermined distance or less on the display screen in the normal display mode as targets to be enlarged, and changes the display positions of both or either of the two target icon images associated with the two extracted targets to move away from each other from their display positions in the normal display mode. In this case, the display control unit 8 moves the display position of the extracted target object to be enlarged, which is closer to the viewing point in the traveling direction, by a predetermined distance along the vertical direction of the screen toward the bottom line L2 of the screen from its display position determined in the normal display mode, and moves the display position of the extracted target object to be enlarged, which is farther from the viewing point in the traveling direction, by a predetermined distance along the vertical direction of the screen from its display position determined in the normal display mode. Furthermore, the display control unit 8 moves the display position of the extracted enlarged target object that is closer to the horizontal reference line L1 by a predetermined distance along the horizontal direction of the screen from the display position determined in the normal display mode, and moves the display position of the extracted enlarged target object that is farther from the horizontal reference line L1 by a predetermined distance along the horizontal direction of the screen from the display position determined in the normal display mode. As a result, when displaying an image in the enlarged display mode, the display control unit 8 determines the display positions so that the vertical spacing along the vertical direction of the screen and / or the horizontal spacing along the horizontal direction of the screen between the two target icon images is larger than in the normal display mode.

[0055] When changing the display position of the target to be enlarged from the display position in the normal display mode as described above, the display control unit 8 may make the size of the target icon image that moves closer to the bottom line L2 of the screen along the vertical direction of the screen slightly larger than the size in the normal display mode, and may make the size of the target icon image that moves farther away from the bottom line L2 of the screen along the vertical direction of the screen slightly smaller than the size in the normal display mode. This allows the distance between two target icon images to be increased without impairing the sense of perspective.

[0056] By performing the above-described procedure to display an image in the enlarged display mode, the display control unit 8 may be able to eliminate the overlap of two target icon images I1 and I2 in the virtual image generated in the normal display mode (see FIG. 3) in the enlarged display mode, as shown in FIG. 4. This allows the driver to easily distinguish and recognize two targets 93 and 94 that are far away from the viewpoint.

[0057] As described above, the display control unit 8 increases the distance between two target icon images in the enlarged display mode compared to the normal display mode, making it easier to recognize multiple long-distance targets located far from the viewpoint. However, this may make it difficult to grasp the sense of distance to short-distance targets located close to the viewpoint. Therefore, as shown by the imaginary dividing line L3 in FIG. 4, the display control unit 8 divides the display screen 51 into a short-distance display area R1, in which target icon images of short-distance targets whose travel-direction distance from the viewpoint to the target is less than the distance threshold (see FIG. 2), and a long-distance display area R2, which is located vertically above the short-distance display area R1 and displays target icon images of long-distance targets whose travel-direction distance is equal to or greater than the distance threshold. Furthermore, when displaying multiple target icon images in the enlarged display mode, the display control unit 8 determines the display positions of two target icon images displayed in the long-distance display area R2 so that the distance between them is greater than in the normal display mode, and sets the display positions of target icon images displayed in the short-distance display area R1 to be the same as those in the normal display mode. The distance threshold may be a fixed value or may be varied depending on the vehicle speed, i.e., the faster the vehicle speed, the longer the distance threshold may be.

[0058] As described above, in the enlarged display mode, the distance between the two target icon images I1 and I2 is greater than in the normal display mode. Therefore, when displaying multiple target icon images I1 and I2 in the enlarged display mode, the display control unit 8 may utilize the space between the target icon images I1 and I2 to display target-related information associated with each target near each target icon image I1 and I2, as shown in Fig. 4. Note that Fig. 4 shows a case where the target-related information is displayed by an arrow 99 indicating the direction of movement of a second target 94 that is about to enter the roadway from the sidewalk.

[0059] In this embodiment, the display control unit 8 sets the form of the background image Ib to be the same in both the normal display mode and the enlarged display mode, but the present invention is not limited to this. The form of the background image may be different between the normal display mode and the enlarged display mode as long as the sense of perspective is not impaired.

[0060] FIG. 5 is a flowchart showing a procedure for displaying a virtual image on the display screen 51 in the virtual image display device 1.

[0061] First, in step ST1, the surrounding condition recognition unit 6 acquires surrounding condition information from the external sensor unit 2 and the navigation device 3, and then the process proceeds to step ST2.

[0062] Next, in step ST2, the surrounding condition recognition unit 6 acquires target position information, target-related information, and traffic environment information based on the surrounding condition information acquired in step ST1, and then proceeds to step ST3.

[0063] Next, in step ST3, the line-of-sight direction acquisition unit 7 acquires the line-of-sight direction of the driver based on the image transmitted from the interior camera unit 4, and the process proceeds to step ST4.

[0064] Next, in step ST4, the display control unit 8 determines whether any one or combination of six enlarged display conditions (A) to (F) defined for setting the display mode of the virtual image to the enlarged display mode is satisfied, based on the surrounding state information acquired in step ST1 and the line of sight direction acquired in step ST3. If the determination result in step ST4 is NO, the display control unit 8 proceeds to step ST5, where it displays the virtual image generated in the normal display mode on the display screen 51, and then returns to step ST1. If the determination result in step ST4 is YES, the display control unit 8 displays the virtual image generated in the enlarged display mode on the display screen 51, and then returns to step ST1. As described above, the display control unit 8 switches between image display in the normal display mode and image display in the enlarged display mode, based on the surrounding state information.

[0065] Enlarged display condition (A): Based on the surrounding condition information, a predetermined number of targets or more must be recognized within a predetermined time. Enlarged display condition (B): Based on the surrounding condition information, the presence of an area where the density of targets is equal to or greater than a predetermined value is recognized. Enlarged display condition (C): Based on the surrounding condition information, the presence of specific facilities (e.g., stations and commercial facilities with a lot of people and moving objects coming and going) or specific equipment (e.g., crosswalks with a lot of pedestrian movement, traffic lights, etc.) must be recognized in the direction of travel from the viewpoint. Enlarged display condition (D): Based on the surrounding state information, the presence of a specific type of road (for example, an intersection, a point where two roads meet, etc.) in the direction of travel from the viewpoint must be recognized. Enlarged display condition (E): The target is recognized as being in the driver's line of sight while driving. Enlarged display condition (F): When the first target and the second target are recognized in the direction of pointing from the viewpoint, and the distance between the first target and the second target along the direction of travel is equal to or less than a predetermined distance threshold.

[0066] The virtual image display device 1 according to this embodiment has the following advantages. (1) The surrounding condition recognition unit 6 recognizes traffic participants present around the host vehicle V as targets and acquires target position information including information on the relative position between the host vehicle and the targets. The display control unit 8 displays target icon images at display positions on the display screen 51 determined based on the target position information. The display control unit 8 can also switch between image display in a normal display mode, in which the display position is determined based on the target position information so that the driver can recognize the relative position between the vehicle V and the targets, and image display in an enlarged display mode, in which the display positions are determined so that the distance between two target icon images on the display screen 51 is greater than in the normal display mode. Therefore, according to the virtual image display device 1, by displaying images in such an enlarged display mode, even if two or more targets are closely spaced within a range of close relative positions from the host vehicle, these target images are displayed with a distance between them on the display screen 51, allowing the driver to distinguish and recognize the presence of each target, thereby improving traffic safety.

[0067] (2) In the present invention, when a plurality of target images are displayed on the display screen in the enlarged display mode, the display control unit displays target-related information associated with each target near the target image. That is, in the enlarged display mode, the target images of two targets are displayed on the display screen at a greater distance than in the normal display mode, and the target-related information is displayed near the target images. This makes it possible to display a large amount of information without causing a viewer to feel cluttered, thereby improving traffic safety.

[0068] (3) The target-related information includes at least one of the target's position information, moving direction information, and type information. Therefore, the virtual image display device 1 allows users to quickly recognize information that is difficult to grasp from the target image alone, such as the target's position, moving direction, moving speed, and type.

[0069] (4) The display control unit 8 displays the target icon images on the display screen 51 in a manner that creates a sense of perspective. When displaying images in the enlarged display mode, the display control unit 8 determines the display positions so that the vertical spacing between two target icon images along the vertical direction of the screen (i.e., the distance between the two targets along the traveling direction) and / or the horizontal spacing between two target icon images along the horizontal direction of the screen (i.e., the distance between the two targets along the width direction) are larger than in the normal display mode. Therefore, the virtual image display device 1 can widen the spacing between target icon images on the display screen 51 compared to the normal display mode without impairing the sense of perspective.

[0070] (5) In the virtual image display device 1, the display control unit 8 displays target icon images on the display screen 51 in a manner that creates a sense of perspective, and divides the display screen 51 into a short-distance display region R1 in which target icon images of short-distance targets are displayed, and a long-distance display region R2, which is an area vertically above the short-distance display region R1 and in which target icon images of long-distance targets are displayed. In this case, the display positions of the target icon images of long-distance targets displayed in the long-distance display region R2 converge to a virtual horizon line defined within the display screen 51 as the distance from the host vehicle along the traveling direction increases, making it easier for two target icon images to overlap. Therefore, in the virtual image display device 1, when multiple target icon images are displayed on the display screen 51 in the enlarged display mode, the display control unit 8 determines the display positions so that the distance between two target icon images displayed in the long-distance display region R2 is greater than in the normal display mode. Therefore, the virtual image display device 1 allows the driver to distinguish and recognize the presence of multiple long-distance targets that are located in positions where target icon images would overlap in the normal display mode.

[0071] (6) In the virtual image display device 1, when the first and second targets are recognized and the distance between the first and second targets along the traveling direction is equal to or less than the distance threshold, the display control unit 8 displays the image in the enlarged display mode. As a result, when two target icon images would overlap in the normal display mode, the image is displayed in the enlarged display mode so that the two target icon images do not overlap, allowing the driver to distinguish and recognize the presence of each target.

[0072] (7) In the virtual image display device 1, the gaze direction acquisition unit 7 acquires the gaze direction, which is the direction of the driver's line of sight or face, and the display control unit 8 displays an image in an enlarged display mode when a target exists in the gaze direction, i.e., when the driver is gazing at a target. This allows the driver to distinguish and recognize multiple targets existing in the direction of gaze.

[0073] (8) In the virtual image display device 1, the display control unit 8 switches between image display in the normal display mode and image display in the enlarged display mode based on surrounding state information about the surroundings of the vehicle. Therefore, the virtual image display device 1 can switch between the normal display mode and the enlarged display mode at an appropriate timing in response to changes in the surroundings of the vehicle.

[0074] (9) In the virtual image display device 1, the display control unit 8 displays images in an enlarged display mode when, based on the surrounding condition information, it recognizes a predetermined number of targets within a predetermined time, recognizes an area where the target density is equal to or greater than a predetermined value, recognizes the presence of a specific facility or equipment on the traveling direction side of the vehicle, or recognizes the presence of a specific road shape on the traveling direction side of the vehicle. Thus, according to the virtual image display device 1, when there are already many targets on the traveling direction side or when it is predicted that many targets will appear on the traveling direction side, it displays images in an enlarged display mode, allowing the driver to distinguish and recognize these multiple targets.

[0075] Although one embodiment of the present invention has been described above, the present invention is not limited to this, and the detailed configuration may be modified as appropriate within the scope of the spirit of the present invention.

[0076] For example, in the above embodiment, the display control unit 8 has been described as displaying a virtual image on the display screen 51 in a manner that gives the viewer a sense of perspective, but the present invention is not limited to this. The display control unit 8 may also display a virtual image in a manner that does not give the viewer a sense of perspective, such as a bird's-eye view image or a planar image, but that allows the viewer to recognize the relative position between the vehicle and a target. [Explanation of symbols]

[0077] V...Vehicle 1...Virtual image display device 2...External sensor unit 21...Front camera unit 22...Forward radar unit 3. Navigation device 4...Indoor camera unit 5. Display 51…Display screen 6...Surrounding condition recognition unit 7… Gaze direction acquisition unit 8...Display control unit L1: Horizontal reference line L2…Screen bottom line I1, I2...Target icon image (target image)

Claims

1. A virtual image display device that recognizes traffic participants present around a moving body as targets and displays target images relating to the targets on a display screen that is visible to a driver of the moving body, a surrounding condition recognition unit that acquires surrounding condition information relating to the conditions around the moving body, and acquires target position information including information relating to the relative position between the moving body and the target based on the surrounding condition information; a display control unit that displays a moving object image at a moving object display position within the display screen and also displays the target image at a target display position within the display screen determined based on the target position information, The display control unit an image display in a normal display mode in which the target display position is determined at a position away from the moving object display position based on the target position information so that the driver can recognize the relative position between the moving object and the target; and an image display under an enlarged display mode in which the target display position is determined so that the distance between the two target images on the display screen is larger than in the normal display mode.

2. The virtual image display device according to claim 1, characterized in that, when a plurality of target images are displayed on the display screen in the enlarged display mode, the display control unit displays target-related information associated with each of the targets near the target image.

3. 3. The virtual image display device according to claim 2, wherein the target-related information includes at least one of position information, moving direction information, moving speed information, and type information of the target.

4. The virtual image display device according to claim 3, characterized in that the display control unit displays the target image in a manner that creates a sense of perspective on the display screen, and when displaying an image in the enlarged display mode, determines the target display position so that both or either the vertical spacing along the vertical direction of the screen and the horizontal spacing along the horizontal direction of the screen between the two target images is larger than in the normal display mode.

5. the display screen is divided into a short-distance display area in which the target image of a short-distance target whose travel direction distance along the travel direction of the moving body to the target is less than a distance threshold is displayed, and a long-distance display area in which the target image of a long-distance target whose travel direction distance is equal to or greater than the distance threshold is displayed, the long-distance display area being an area above the short-distance display area along the vertical direction of the screen; The virtual image display device according to claim 4, characterized in that when multiple target images are displayed on the display screen in the enlarged display mode, the display control unit determines the target display position so that the distance between two target images displayed in the long-distance display area is larger than in the normal display mode.

6. The virtual image display device according to claim 4, characterized in that the display control unit displays an image in the enlarged display mode when a first target and a second target are recognized by the surrounding condition recognition unit and the distance between the first target and the second target along the direction of travel is less than a distance threshold.

7. A gaze direction acquisition unit is further provided to acquire a gaze direction, which is a direction of the driver's line of sight or face, 5. The virtual image display device according to claim 4, wherein the display control unit displays the image in the enlarged display mode when the target exists in the line of sight direction.

8. The virtual image display device according to claim 4 , wherein the display control unit switches between image display in the normal display mode and image display in the enlarged display mode based on the surrounding state information.

9. the surrounding condition recognition unit recognizes, based on the surrounding condition information, a traffic participant present in a traveling direction of the moving object as the target; The virtual image display device of claim 8, characterized in that the display control unit displays an image in the enlarged display mode when, based on the surrounding condition information, it recognizes more than a predetermined number of targets within a predetermined time, when it recognizes an area where the density of targets is more than a predetermined value, when it recognizes the presence of a specific facility or specific equipment in the direction of travel from the moving body, or when it recognizes the presence of a specific type of road in the direction of travel from the moving body.

Images