Virtual Image Display Device

Abstract

To provide a virtual image display device capable of displaying a virtual image by which a sense of distance of a distant target can be easily grasped.SOLUTION: A virtual image display device includes a surrounding state recognition unit that acquires travel direction distance and widthwise distances of a target, and a display control unit that displays target icon images I1 and I2 in a display screen 51. The display control unit can switch between image display under a normal display mode and image display under an enlarged display mode. In the normal display mode, a display position of the target icon images I1 and I2 is kept away from a horizontal reference line L1 in the display screen 51 along the screen horizontal direction as the widthwise direction distance increases, and the display position is moved closer to the horizontal reference line L1 along the screen horizontal direction as distance in a travel direction increases, and the display position is moved away from a screen bottom edge line L2 along a screen vertical direction as the travel direction distance increases. In the enlarged display mode, a vertical scale that is a ratio of distance along a screen vertical direction between the screen bottom line L2 and the display position relative to the travel direction distance is made larger than the normal display mode.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to a virtual image display device, and more particularly to a virtual image display device that displays a target image of a target that exists in a direction of a point of view from a viewpoint determined based on a human user or a mobile object traveling with the user, on a display screen visible to the user in a manner that creates a sense of perspective. [Background technology]

[0002] In recent years, in order to improve traffic safety, virtual image display devices have been proposed that display useful virtual images to support safe driving on a monitor that can be viewed by a driver while driving. For example, the device shown in Patent Document 1 displays virtual images of blind spots on the left and right ahead of the vehicle on a monitor. This allows a driver who is driving to recognize the presence of a vehicle that may appear in front of the vehicle from the blind spot by viewing the monitor, thereby improving traffic safety. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2009-226978 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in general virtual image display devices, in order to enable the driver to quickly compare the scenery that can be seen directly with his or her own eyes with the virtual image displayed on the monitor, the virtual image is often displayed on the monitor in a manner that gives the viewer a sense of perspective.

[0005] However, in a virtual image drawn based on perspective, the size of the target image decreases as the distance from the vehicle in the depth direction (i.e., the direction of travel) increases. Furthermore, the display position of the target image gradually approaches the vanishing point as the distance from the vehicle in the depth direction increases. Therefore, in a virtual image drawn based on perspective, for example, when multiple targets are located far from the vehicle, it is difficult for the driver to recognize the distance between these distant targets (especially the distance in the depth direction).

[0006] An object of the present invention is to provide a virtual image display device that can display an image that makes it easy to grasp the sense of distance to a target that is located far away. [Means for solving the problem]

[0007] (1) A virtual image display device according to the present invention (for example, a virtual image display device 1 described later) recognizes a traffic participant present in a direction of orientation from a viewpoint (for example, a viewpoint 95 described later) determined based on a human user or a moving object traveling with the user, as a target, and displays a target image (for example, target icon images I1 and I2 described later) relating to the target on a display screen visible to the user (for example, a display screen 51 described later) in a manner that creates a sense of perspective. The virtual image display device according to the present invention includes a surrounding state recognition unit (for example, a surrounding state recognition unit described later) that acquires surrounding state information relating to the state around the viewpoint and acquires target position information including information on a direction distance from the viewpoint to the target along the direction of orientation and information on a width direction distance from the viewpoint to the target along a width direction perpendicular to the direction of orientation based on the surrounding state information. a display control unit (e.g., a display control unit 8 described later) that displays the target image at a display position within the display screen determined based on the target position information, and the display control unit is capable of switching between an image display under a normal display mode in which the display position is moved farther in the horizontal direction from a horizontal reference (e.g., a horizontal reference line L1 described later) within the display screen as the width direction distance increases and moved closer to the horizontal reference along the horizontal direction as the pointing direction distance increases, and the display position is moved farther in the vertical direction from a bottom edge of the display screen (e.g., a screen bottom edge line L2 described later) as the pointing direction distance increases, and an image display under an enlarged display mode in which a vertical scale, which is the ratio of the distance along the vertical direction between the bottom edge and the display position to the pointing direction distance, is larger than that in the normal display mode.

[0008] (2) In this case, it is preferable that the vertical scale for a short-distance target whose pointing direction distance is equal to or less than the distance threshold is the same in the normal display mode and the enlarged display mode.

[0009] (3) In this case, it is preferable that the display control unit switches between image display in the normal display mode and image display in the enlarged display mode based on the surrounding state information.

[0010] (4) In this case, it is preferable that the display control unit displays the image in the enlarged display mode when, based on the surrounding condition information, it recognizes more than a predetermined number of targets within a predetermined time, when it recognizes an area where the density of targets is greater than a predetermined value, when it recognizes that a specific facility or equipment exists in the direction of direction from the viewpoint, or when it recognizes that a road of a specific shape exists in the direction of direction from the viewpoint.

[0011] (5) In this case, the virtual image display device preferably further includes a gaze direction acquisition unit that acquires the gaze direction, which is the line of sight or facial direction of the user who is the driver of the moving body, and the display control unit preferably displays the image in the enlarged display mode when the target is present in the gaze direction.

[0012] (6) In this case, it is preferable that the display control unit displays the image in the enlarged display mode when a first target and a second target are recognized in the pointing direction from the viewpoint and when the distance between the first target and the second target along the pointing direction is equal to or less than a distance threshold.

[0013] (7) In this case, when the display control unit displays multiple target images on the display screen in the enlarged display mode, it is preferable that the display control unit displays target-related information associated with each target near the target image.

[0014] (8) In this case, it is preferable that the target-related information includes at least one of position information, movement direction information, movement speed information, and type information of the target. [Effects of the Invention]

[0015] (1) In the present invention, the surroundings recognition unit recognizes traffic participants present in the direction of orientation from the viewpoint as targets and acquires target position information including information on the orientation distance from the viewpoint to the target and the widthwise distance from the viewpoint to the target. The display control unit displays a target image on the display screen at a display position within the display screen determined based on the target position information, thereby displaying an image on the display screen in a manner that gives the viewer a sense of perspective. The display control unit is also capable of switching between image display in a normal display mode (i.e., image display based on perspective), in which the display position is moved farther horizontally from a horizontal reference within the display screen as the widthwise distance increases and closer to the horizontal reference as the orientation direction distance increases, and the display position is moved farther vertically from the bottom edge of the display screen as the orientation direction distance increases, and image display in an enlarged display mode, in which the vertical scale, which is the ratio of the distance on the display screen between the bottom edge and the display position in the vertical direction to the orientation direction distance, is larger than that in the normal display mode. In this way, in the enlarged display mode, by making the vertical scale larger than in the normal display mode, for example, when two targets are present in the distance, the target images of these two targets are displayed at a greater vertical distance on the display screen than in the normal display mode. Therefore, according to the present invention, by displaying images in this enlarged display mode, the user can easily grasp the sense of distance to targets present in the distance.

[0016] (2) By increasing the vertical scale as described above, the user can easily grasp the sense of distance to targets located far from the viewpoint, but it is difficult to grasp the sense of distance to targets located close to the viewpoint. In contrast, in the present invention, the vertical scale for close-range targets whose pointing direction distance is equal to or less than the distance threshold is made the same in the normal display mode and the enlarged display mode, so that the user can easily grasp the sense of distance to both targets located far from the viewpoint and targets located close to the viewpoint.

[0017] (3) In the present invention, the display control unit switches between image display in the normal display mode and image display in the enlarged display mode based on surrounding state information about the surrounding state of the viewpoint. Therefore, according to the present invention, it is possible to switch between the normal display mode and the enlarged display mode at an appropriate timing in response to changes in the surrounding state of the viewpoint.

[0018] (4) In the present invention, the display control unit, based on the surrounding condition information, displays an image in an enlarged display mode when it recognizes a predetermined number or more of targets within a predetermined time, recognizes an area where the target density is equal to or greater than a predetermined value, recognizes the presence of a specific facility or equipment in the direction of direction from the viewpoint, or recognizes the presence of a specific type of road in the direction of direction from the viewpoint. Therefore, according to the present invention, when many targets already exist in the direction of direction or when many targets are predicted to appear in the direction of direction, the display control unit displays an image in an enlarged display mode, allowing the user to easily grasp the distance between these multiple targets.

[0019] (5) In the present invention, the gaze direction acquisition unit acquires the gaze direction, which is the line of sight or the direction of the face of the user who is the driver of the mobile vehicle, and the display control unit displays an image in an enlarged display mode when a target exists in the gaze direction, i.e., when the user is gazing at the target. This allows the user to easily grasp the sense of distance to the target they are gazing at.

[0020] (6) In the present invention, when a first and a second target are recognized in the pointing direction from the viewpoint and the distance between the first and second targets along the pointing direction is equal to or less than a distance threshold, the display control unit displays images in the enlarged display mode. As a result, when two target images would overlap in the normal display mode, the display control unit displays images in the enlarged display mode to prevent the two target images from overlapping.

[0021] (7) In the present invention, when multiple target images are displayed on the display screen in enlarged display mode, the display control unit displays target-related information associated with each target near the target image, thereby allowing a large amount of information to be displayed on the display screen without causing the viewer to feel cluttered.

[0022] (8) In the present invention, the target-related information includes at least one of target position information, moving direction information, moving speed information, and type information. Therefore, according to the present invention, information that is difficult to grasp from the target image alone, such as the target position, moving direction, moving speed, and type, can be quickly recognized. [Brief explanation of the drawings]

[0023] [Figure 1] 1 is a diagram schematically illustrating the configuration of a virtual image display device according to an embodiment of the present invention and a vehicle equipped with the virtual image display device; [Figure 2] FIG. 1 is a plan view of a vehicle traveling on a road with three lanes on each side, viewed from above. [Figure 3] 3 is a diagram showing an example of a virtual image displayed on the display screen when a display control unit performs image display in a normal display mode in the situation shown in FIG. 2. FIG. [Figure 4] 3 is a diagram showing an example of a virtual image displayed on the display screen when a display control unit displays an image in an enlarged display mode in the situation shown in FIG. 2. FIG. [Figure 5] FIG. 10 is a diagram comparing the vertical scale between the normal display mode and the enlarged display mode. [Figure 6] 10 is a flowchart showing a procedure for displaying a virtual image in a display control unit. DETAILED DESCRIPTION OF THE INVENTION

[0024] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A virtual image display device according to an embodiment of the present invention will be described below with reference to the drawings.

[0025] FIG. 1 is a diagram schematically showing the configuration of a virtual image display device 1 according to this embodiment and a vehicle V on which this virtual image display device 1 is mounted.

[0026] The virtual image display device 1 recognizes traffic participants (pedestrians, moving objects, etc.) present in the direction of travel (the direction of the user's line of sight or face when the viewpoint is based on the user, or the direction of travel of the moving object when the viewpoint is based on the moving object) as targets from a viewpoint determined based on a human user or a moving object traveling with the user, and displays target images associated with the targets on a display screen 51 of a display 5 provided at a position visible to the user in a manner that creates a sense of perspective, thereby assisting the user in recognizing the targets. Note that the following description will be given assuming that the user is a driver of a car, and that the virtual image display device 1 is mounted on a vehicle V, which is a four-wheeled automobile; however, the present invention is not limited to this. The present invention may also be applied to a driver of a moving object other than a four-wheeled automobile, such as a truck, a saddle-type vehicle, or a bicycle, and the virtual image display device may be mounted on these moving objects. Furthermore, when mounted on a moving object, particularly a saddle-type vehicle, the virtual image display device may be mounted on a helmet worn by the rider, the user. Furthermore, the user may be a pedestrian, and the virtual image display device may be a wearable device (for example, glasses-type, watch-type, etc.) that can be worn by the pedestrian.

[0027] The vehicle V is equipped with an external sensor unit 2 that acquires information about the area ahead in the direction of travel, a navigation device 3, an interior camera unit 4, and a virtual image display device 1 that displays virtual images on a display screen 51 to assist the driver in safe driving.

[0028] The external sensor unit 2 is composed of a front camera unit 21, a front radar unit 22, and the like.

[0029] The front camera unit 21 includes a digital camera using a solid-state imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The front camera unit 21 is attached, for example, to a position on the roof of the vehicle V, on the inside of the cabin, near the windshield, facing forward in the direction of travel. The front radar unit 22 includes a millimeter-wave radar that detects an object by measuring a wave reflected from the object in response to millimeter-wave irradiation. The front radar unit 22 is attached, for example, to the front bumper of the vehicle V, facing forward in the direction of travel.

[0030] The navigation device 3 includes, for example, a GNSS (Global Navigation Satellite System) receiver that identifies the current position of the vehicle based on signals received from GNSS satellites, a storage device that stores map information, etc. Here, the map information also includes information about road signs. The navigation device 5 transmits information about the current position of the vehicle to the virtual image display device 1 together with the map information of the current position.

[0031] The interior camera unit 4 includes, for example, a digital camera using a solid-state image sensor as described above. The interior camera unit 4 is installed inside the vehicle cabin facing the face of the driver sitting in the driver's seat. The image of the driver captured by the interior camera unit 4 is transmitted to the virtual image display device 1.

[0032] The virtual image display device 1 comprises a display 5 provided in a position visible to the driver while driving, a surrounding condition recognition unit 6 that acquires surrounding condition information regarding the conditions around the vehicle V acquired by an external sensor unit 2 and a navigation device 3 and recognizes the conditions around the vehicle V, a gaze direction acquisition unit 7 that acquires the gaze direction, which is the direction of the driver's line of sight or face, based on an image transmitted from the interior camera unit 4, and a display control unit 8 that displays a virtual image of the vehicle V ahead in the direction of travel, generated based on the recognition results by the surrounding condition recognition unit 6, on the display screen 51 of the display 5.

[0033] In this embodiment, the display control unit 8 displays a virtual image on the display screen 51 of the display 5 provided in front of the driver's seat, but the present invention is not limited to this. The display control unit 8 may also display a virtual image on a part of the windshield as a display screen by using a head-up display.

[0034] The surrounding condition recognition unit 6 recognizes one or more traffic participants present ahead in the traveling direction from a viewpoint determined for the vehicle V as targets based on surrounding condition information acquired from the external sensor unit 2 and the navigation device 3, and acquires target position information and target-related information for each target. The target position information includes information on the traveling direction distance from the viewpoint to the target along the traveling direction and information on the width direction distance from the viewpoint to the target along the width direction perpendicular to the traveling direction. The target-related information includes at least one of the target position information, moving direction information, moving speed information, and type information.

[0035] Here, the viewpoint set for vehicle V is set at any position, for example, inside the cabin of vehicle V (more specifically, at the driver's seat) or outside the cabin of vehicle V (more specifically, at a position a predetermined distance behind the rear end of vehicle V in the direction of travel). If the viewpoint is set at the driver's seat, a virtual image from the same viewpoint as the driver can be displayed. Also, if the viewpoint is set at a position away from the rear end of vehicle V, a bird's-eye view virtual image that includes the vehicle in the field of view can be displayed.

[0036] The surrounding condition recognition unit 6 acquires target position information and target-related information as described above, and also acquires traffic environment information of traffic participants including the vehicle V and the recognized targets, based on the surrounding condition information acquired from the external sensor unit 2 and the navigation device 3. Here, the traffic environment information includes information on the number of lanes on the road being traveled, the presence or absence of sidewalks, the types of facilities and equipment present ahead in the direction of travel, the shape of the road present ahead in the direction of travel, etc.

[0037] In this embodiment, the surrounding condition recognition unit 6 acquires surrounding condition information from the external sensor unit 2 and the navigation device 3 as in-vehicle devices that move together with the vehicle V, but the present invention is not limited to this. The surrounding condition recognition unit 6 may acquire surrounding condition information through vehicle-to-vehicle communication between the vehicle and another vehicle traveling nearby, or through communication between the vehicle and a server device that monitors the traffic conditions of the road on which the vehicle is traveling.

[0038] The display control unit 8 generates a virtual image that reproduces the view ahead in the traveling direction from the viewpoint based on the recognition result by the surrounding condition recognition unit 6, and displays this virtual image on the display screen 51 of the display 5. More specifically, the display control unit 8 draws target images associated with each target recognized by the surrounding condition recognition unit 6 superimposed at a predetermined position and with a predetermined size on a background image according to the traffic environment recognized by the surrounding condition recognition unit 6, thereby generating a virtual image that gives the viewer a sense of perspective.

[0039] The display control unit 8 is capable of generating a virtual image in a normal display mode or an enlarged display mode and displaying the virtual image. The procedure for generating a virtual image in each display mode will be described below using the situation shown in FIG. 2 as an example.

[0040] FIG. 2 is a plan view of a vehicle V traveling on a road 9 with three lanes on each side, as seen from above. FIG. 2 shows a case in which the host vehicle, vehicle V, is traveling in a center lane 91, and another vehicle 93 and a pedestrian 94 are present ahead of the host vehicle in the traveling direction. FIG. 2 also shows a case in which the other vehicle 93 is stopped in lane 92 on the sidewalk side, and the pedestrian 94 is about to enter lane 92 from the sidewalk ahead of the other vehicle 93 in the traveling direction. In the following, a case will be described in which a viewpoint 95 is set behind the vehicle V in the traveling direction, and the surrounding state recognition unit 6 recognizes the other vehicle 93 as a first target and the pedestrian 94 as a second target.

[0041] <Normal display mode> Fig. 3 is a diagram showing an example of a virtual image displayed on the rectangular display screen 51 when the display control unit 8 performs image display in the normal display mode in the situation shown in Fig. 2. As shown in Fig. 3, the virtual image is composed of a background image Ib that resembles, for example, the lane markings on the road on which the vehicle is traveling, a host vehicle icon image Iv associated with the vehicle V, and target icon images I1 and I2 associated with the respective targets 93 and 94. In the following, the host vehicle icon image Iv and the target icon image I1 associated with the target vehicle are simplified by square marks, and the target icon image I2 associated with the target pedestrian is simplified by a circle.

[0042] When displaying images in the normal display mode, the display control unit 8 displays the vehicle icon image Iv and the target icon images I1 and I2 at positions and sizes that give the viewer a sense of perspective on a background image Ib generated according to the traffic environment recognized by the surrounding condition recognition unit 6. Note that the following describes a case in which only road dividing lines are displayed as the background image, but the present invention is not limited to this. In addition to road dividing lines, structures such as facilities and equipment present on the road may also be displayed as the background image, or a lattice-like grid may also be displayed as the background image.

[0043] When displaying images in normal display mode, the display control unit 8 reduces the size of each icon image Iv, I1, I2 on the display screen 51 as the travel direction distance, which is the distance along the travel direction from the viewpoint to the target, becomes longer.

[0044] When displaying images in the normal display mode, the display control unit 8 moves the display positions of each of the icon images Iv, I1, and I2 on the display screen 51 farther from a horizontal reference line L1 virtually defined within the display screen 51 in the horizontal direction of the screen as the widthwise distance from the viewpoint to the target increases, and moves closer to the horizontal reference line L1 in the horizontal direction of the screen as the distance in the traveling direction increases. Furthermore, the display control unit 8 displays targets that are on the right hand side as viewed from the viewpoint along the traveling direction on the right hand side of the horizontal reference line L1 on the display screen 51, and displays targets that are on the left hand side as viewed from the viewpoint along the traveling direction on the left hand side of the horizontal reference line L1 on the display screen 51. Note that the following description will be given of a case in which the horizontal reference line L1 extending along the vertical direction of the screen is set to the center of the display screen 51, but the present invention is not limited to this.

[0045] Furthermore, when displaying images in the normal display mode, the display control unit 8 moves the display position of each icon image Iv, I1, I2 on the display screen 51 farther away from the bottom edge line L2 of the display screen 51 along the vertical direction of the screen, which is perpendicular to the horizontal direction of the screen, as the distance from the viewpoint to the target in the travel direction becomes longer.

[0046] In the following description, the horizontal scale is defined as the ratio of the distance along the horizontal direction of the screen between the horizontal reference line L1 and the display position on the display screen 51 to the actual widthwise distance. That is, in the examples of Figures 2 and 3, the horizontal scale for the first target object 93 is xv1 / x1, and the horizontal scale for the second target object 94 is xv2 / x2. Therefore, when displaying an image in the normal display mode, the display control unit 8 asymptotically causes the horizontal scale to approach 0 as the distance in the traveling direction becomes longer.

[0047] In the following description, the vertical scale is defined as the ratio of the distance along the vertical direction of the screen between the bottom edge line L2 of the screen and the display position on the display screen 51 to the actual distance in the traveling direction. That is, in the examples of Figures 2 and 3, the vertical scale for the first target object 93 is yv1 / y1, and the vertical scale for the second target object 94 is yv2 / y2. Therefore, when displaying an image in the normal display mode, the display control unit 8 asymptotically reduces the vertical scale to 0 as the distance in the traveling direction increases.

[0048] When displaying an image in the normal display mode, the display control unit 8 can display a virtual image that gives the viewer a sense of perspective by using the above-described procedure to set the display position and size of each icon image Iv, I1, and I2 on the display screen 51. However, in the virtual image displayed on the display screen 51 in the normal display mode, the target icon images I1 and I2 associated with two targets 93 and 94 that are located far away from the viewpoint along the traveling direction often overlap as shown in Fig. 3, making it difficult for the driver to grasp the sense of distance between the targets 93 and 94 that are located far away from the viewpoint.

[0049] <Enlarged display mode> FIG. 4 is a diagram showing an example of a virtual image displayed on the display screen 51 when the display control unit 8 performs image display in the enlarged display mode in the situation shown in FIG.

[0050] As shown in Figure 4, when displaying an image in the enlarged display mode, the display control unit 8 displays the vehicle icon image Iv and target icon images I1 and I2 on a background image Ib generated according to the traffic environment recognized by the surrounding condition recognition unit 6 at a position and size that gives the viewer a sense of perspective.

[0051] When displaying an image in the enlarged display mode, the display control unit 8 changes the display positions of at least the target icon images I1 and I2 on the display screen 51 from the normal display mode to allow the viewer to easily grasp the sense of distance to targets that are far from the viewpoint. More specifically, when displaying an image in the enlarged display mode, the display control unit 8 makes the vertical scale larger than in the normal display mode.

[0052] 3 and 4, the display distance between the two target icon images I1 and I2 on the display screen 51 along the vertical direction of the screen is larger in the enlarged display mode than in the normal display mode. Therefore, the overlap of the two target icon images I1 and I2 in the virtual image generated in the normal display mode (see FIG. 3) may be eliminated in the enlarged display mode, as shown in FIG. 4. This makes it easier for the driver to grasp the distance of targets 93 and 94 that are far away from the viewpoint.

[0053] As described above, by increasing the vertical scale in the enlarged display mode compared to the normal display mode, the display control unit 8 can make it easier to grasp the sense of distance to long-distance targets located far from the viewpoint. However, in this case, it may become difficult to grasp the sense of distance to short-distance targets located close to the viewpoint. Therefore, it is preferable that the display control unit 8 sets the vertical scale for short-distance targets whose distance in the traveling direction is equal to or less than the distance threshold (see FIG. 2) in the normal display mode and the enlarged display mode, and sets the vertical scale for long-distance targets whose distance in the traveling direction is equal to or greater than the distance threshold in the enlarged display mode. Note that this distance threshold may be a fixed value or may be varied according to the vehicle speed. In other words, the distance threshold may be increased as the vehicle speed increases.

[0054] 5 is a diagram comparing the vertical scale between the normal display mode (see dashed line) and the enlarged display mode (see solid line and dashed dotted line). As shown by the dashed line in FIG. 5, when displaying an image in the normal display mode, the display control unit 5 asymptotically brings the vertical scale closer to 0 as the distance in the traveling direction increases. This means that the display position of the target icon image set in the normal display mode converges to a horizontal line virtually defined within the display screen 51 as the distance in the traveling direction increases.

[0055] In contrast, as illustrated by the solid and dashed lines in FIG. 5 , when displaying an image in the enlarged display mode, the display control unit 5 sets the vertical scale for a short-distance target whose traveling direction distance is equal to or less than the distance threshold to be equal to that in the normal display mode, and sets the vertical scale for a long-distance target whose traveling direction distance is equal to or greater than the distance threshold to be larger than that in the normal display mode. In this case, the display control unit 5 may keep the vertical scale for a long-distance target constant at the same value as that for a short-distance target whose traveling direction distance is equal to the distance threshold (see the solid line in FIG. 5 ), or may decrease the vertical scale as the traveling direction distance increases (see the dashed-dotted line in FIG. 5 ). However, if the vertical scale for a long-distance target is made larger than that for a short-distance target whose traveling direction distance is equal to the distance threshold, the sense of perspective will be impaired. For this reason, when displaying an image in the enlarged display mode, it is preferable that the display control unit 8 sets the vertical scale for a long-distance target to be larger than that in the normal display mode, within a range equal to or less than that for a short-distance target whose traveling direction distance is equal to the distance threshold.

[0056] When displaying an image in the enlarged display mode, the display control unit 8 may set the horizontal scale to be the same as that in the normal display mode, or may fine-tune the positional relationship with the background image Ib so that it does not appear unnatural.

[0057] As described above, in the enlarged display mode, the distance between the two target icon images I1 and I2 is greater than in the normal display mode. Therefore, when displaying multiple target icon images I1 and I2 in the enlarged display mode, the display control unit 8 may utilize the space between the target icon images I1 and I2 to display target-related information associated with each target near each target icon image I1 and I2, as shown in Fig. 4. Note that Fig. 4 shows a case where the target-related information is displayed by an arrow 99 indicating the direction of movement of a second target 94 that is about to enter the roadway from the sidewalk.

[0058] In the present embodiment, the display control unit 8 sets the shape of the background image Ib and the sizes of the icon images Iv, I1, and I2 on the display screen 51 to be the same in both the normal display mode and the enlarged display mode, but the present invention is not limited to this. The shape of the background image and the sizes of the icon images may be changed between the normal display mode and the enlarged display mode as long as the sense of perspective is not impaired.

[0059] FIG. 6 is a flowchart showing a procedure for displaying a virtual image on the display screen 51 in the virtual image display device 1.

[0060] First, in step ST1, the surrounding condition recognition unit 6 acquires surrounding condition information from the external sensor unit 2 and the navigation device 3, and then the process proceeds to step ST2.

[0061] Next, in step ST2, the surrounding condition recognition unit 6 acquires target position information, target-related information, and traffic environment information based on the surrounding condition information acquired in step ST1, and then proceeds to step ST3.

[0062] Next, in step ST3, the line-of-sight direction acquisition unit 7 acquires the line-of-sight direction of the driver based on the image transmitted from the interior camera unit 4, and the process proceeds to step ST4.

[0063] Next, in step ST4, the display control unit 8 determines whether any one or combination of six enlarged display conditions (A) to (F) defined for setting the display mode of the virtual image to the enlarged display mode is satisfied, based on the surrounding state information acquired in step ST1 and the line of sight direction acquired in step ST3. If the determination result in step ST4 is NO, the display control unit 8 proceeds to step ST5, where it displays the virtual image generated in the normal display mode on the display screen 51, and then returns to step ST1. If the determination result in step ST4 is YES, the display control unit 8 displays the virtual image generated in the enlarged display mode on the display screen 51, and then returns to step ST1. As described above, the display control unit 8 switches between image display in the normal display mode and image display in the enlarged display mode, based on the surrounding state information.

[0064] Enlarged display condition (A): Based on the surrounding condition information, a predetermined number of targets or more must be recognized within a predetermined time. Enlarged display condition (B): Based on the surrounding condition information, the presence of an area where the density of targets is equal to or greater than a predetermined value is recognized. Enlarged display condition (C): Based on the surrounding condition information, the presence of specific facilities (e.g., stations and commercial facilities with a lot of people and moving objects coming and going) or specific equipment (e.g., crosswalks with a lot of pedestrian movement, traffic lights, etc.) must be recognized in the direction of travel from the viewpoint. Enlarged display condition (D): Based on the surrounding state information, the presence of a specific type of road (for example, an intersection, a point where two roads meet, etc.) in the direction of travel from the viewpoint must be recognized. Enlarged display condition (E): The target is recognized as being in the driver's line of sight while driving. Enlarged display condition (F): When the first target and the second target are recognized in the direction of pointing from the viewpoint, and the distance between the first target and the second target along the direction of travel is equal to or less than a predetermined distance threshold.

[0065] The virtual image display device 1 according to this embodiment has the following advantages. (1) The surrounding condition recognition unit 6 recognizes traffic participants present in the traveling direction from the viewpoint as targets, and acquires target position information including information on the traveling distance from the viewpoint to the target and the widthwise distance from the viewpoint to the target. The display control unit 8 displays a target icon image on the display screen 51 at a display position determined based on the target position information, thereby displaying a virtual image on the display screen 51 in a manner that gives the viewer a sense of perspective. The display control unit 8 can also switch between image display in a normal display mode (i.e., image display based on perspective), in which the display position of the target icon image is moved farther from the horizontal reference line L1 in the display screen 51 in the horizontal direction of the screen as the widthwise distance increases and closer to the horizontal reference line L1 in the horizontal direction of the screen as the traveling direction distance increases, and the display position is moved farther from the bottom edge line L2 of the display screen 51 in the vertical direction of the screen as the traveling direction distance increases, and image display in an enlarged display mode, in which the vertical scale, which is the ratio of the distance on the display screen 51 between the bottom edge line L2 and the display position in the vertical direction of the screen to the traveling direction distance, is larger than in the normal display mode. In this way, by making the vertical scale larger in the enlarged display mode than in the normal display mode, for example, if two targets are present in the distance, the target icon images of these two targets are displayed farther apart in the vertical direction of the screen on the display screen 51 than in the normal display mode. Therefore, according to the virtual image display device 1, by displaying images in such an enlarged display mode, the user can easily grasp the sense of distance to targets present in the distance.

[0066] (2) By increasing the vertical scale as described above, the driver can easily grasp the sense of distance to targets located far from the viewpoint, but it is difficult to grasp the sense of distance to targets located close to the viewpoint. In contrast, in the virtual image display device 1, the vertical scale for close-range targets whose distance in the traveling direction is equal to or less than the distance threshold is made the same in the normal display mode and the enlarged display mode, so that the driver can easily grasp the sense of distance to both targets located far from the viewpoint and targets located close to the viewpoint.

[0067] (3) The display control unit 8 switches between image display in the normal display mode and image display in the enlarged display mode based on surrounding state information about the surrounding state of the viewpoint. Therefore, the virtual image display device 1 can switch between the normal display mode and the enlarged display mode at an appropriate timing in response to changes in the surrounding state of the viewpoint.

[0068] (4) When the display control unit 8 recognizes a predetermined number of targets within a predetermined time, recognizes an area where the target density is equal to or greater than a predetermined value, recognizes the presence of a specific facility or equipment on the traveling direction side from the viewpoint, or recognizes the presence of a specific road shape on the traveling direction side from the viewpoint, based on the surrounding condition information, the display control unit 8 displays an image in an enlarged display mode. Therefore, according to the virtual image display device 1, when there are already many targets on the traveling direction side or when it is predicted that many targets will appear on the traveling direction side, the image is displayed in an enlarged display mode, allowing the driver to easily grasp the distance between these multiple targets.

[0069] (5) The gaze direction acquisition unit 7 acquires the gaze direction, which is the direction of the driver's line of sight or face, and the display control unit 8 displays an image in an enlarged display mode when a target exists in the gaze direction, i.e., when the driver is gazing at the target. This allows the driver to easily grasp the sense of distance to the target they are gazing at.

[0070] (6) When the first and second targets are recognized in the traveling direction from the viewpoint and the distance between the first and second targets along the traveling direction is equal to or less than the distance threshold, the display control unit 8 displays the image in the enlarged display mode. As a result, when two target icon images would overlap in the normal display mode, the image is displayed in the enlarged display mode, so that the two target icon images do not overlap.

[0071] (7) When displaying multiple target icon images on the display screen 51 in the enlarged display mode, the display control unit 8 displays target-related information associated with each target near the target icon image, thereby allowing a large amount of information to be displayed on the display screen 51 without causing the viewer to feel cluttered.

[0072] (8) The target-related information includes at least one of the target's position information, moving direction information, moving speed information, and type information. Therefore, the virtual image display device 1 allows users to quickly recognize information that is difficult to grasp from the target icon image alone, such as the target's position, moving direction, moving speed, and type.

[0073] Although one embodiment of the present invention has been described above, the present invention is not limited to this, and the detailed configuration may be modified as appropriate within the scope of the spirit of the present invention. [Explanation of symbols]

[0074] V...Vehicle 1...Virtual image display device 2...External sensor unit 21...Front camera unit 22...Forward radar unit 3. Navigation device 4...Indoor camera unit 5. Display 51…Display screen 6...Surrounding condition recognition unit 7... Gaze direction acquisition unit 8...Display control unit L1...Horizontal reference line (horizontal reference) L2…Screen bottom line (bottom edge) I1, I2...Target icon image (target image)

Claims

1. A virtual image display device that recognizes a traffic participant present in a direction of a direction from a viewpoint determined based on a human user or a moving object traveling with the user as a reference, and displays a target image of the target on a display screen visible to the user in a manner that creates a sense of perspective, a surrounding state recognition unit that acquires surrounding state information related to the state around the viewpoint, and acquires target position information based on the surrounding state information, the target position information including information about a pointing direction distance from the viewpoint to the target along the pointing direction and information about a width direction distance from the viewpoint to the target along a width direction perpendicular to the pointing direction; a display control unit that displays the target image at a display position within the display screen that is determined based on the target position information, The display control unit an image display in a normal display mode in which the display position is moved farther from a horizontal reference in the display screen along the horizontal direction as the width direction distance increases and is moved closer to the horizontal reference along the horizontal direction as the orientation direction distance increases, and the display position is moved farther from a bottom edge of the display screen along the vertical direction as the orientation direction distance increases; and an image display under an enlarged display mode in which a vertical scale, which is the ratio of the pointing direction distance, which is the distance along the pointing direction from the viewpoint to the target, to the distance along the vertical direction from the bottom end of the display screen to the target image, is larger than that in the normal display mode.

2. 2. The virtual image display device according to claim 1, wherein the vertical scale for a short-distance target whose pointing direction distance is equal to or less than a distance threshold is the same in the normal display mode and the enlarged display mode.

3. 3. The virtual image display device according to claim 1, wherein the display control unit switches between image display in the normal display mode and image display in the enlarged display mode based on the surrounding state information.

4. The virtual image display device described in claim 3, characterized in that the display control unit displays an image in the enlarged display mode when, based on the surrounding condition information, it recognizes more than a predetermined number of targets within a predetermined time, when it recognizes an area where the density of targets is more than a predetermined value, when it recognizes the presence of a specific facility or specific equipment in the direction of direction from the viewpoint, or when it recognizes the presence of a specific type of road in the direction of direction from the viewpoint.

5. a gaze direction acquisition unit that acquires a gaze direction, which is a line of sight or a facial direction of the user who is a driver of the moving body; 3. The virtual image display device according to claim 1, wherein the display control unit displays the image in the enlarged display mode when the target exists in the line of sight direction.

6. The virtual image display device according to claim 1 or 2, characterized in that the display control unit displays an image in the enlarged display mode when a first target and a second target are recognized in the pointing direction from the viewpoint and when the distance between the first target and the second target along the pointing direction is equal to or less than a distance threshold.

7. The virtual image display device according to claim 1 or 2, characterized in that, when a plurality of target images are displayed on the display screen in the enlarged display mode, the display control unit displays target-related information associated with each of the targets near the target image.

8. 8. The virtual image display device according to claim 7, wherein the target-related information includes at least one of position information, moving direction information, moving speed information, and type information of the target.

Images