Superimposed image display device

Abstract

To provide a superimposed image display device that can suggest to an occupant a risk of movement on a lane caused by approach of the other vehicle and also suggest to the occupant a direction in which the other vehicle is approaching.SOLUTION: A superimposed image display device is configured to obtain a situation around an own vehicle and display a guiding object along a compartment line between the own vehicle and the other vehicle when it is determined that the other vehicle is approaching in a direction of the own vehicle running on a lane different from a lane on which the own vehicle is running on the basis of the situation, and to display an animation in a mode of showing the direction in which the other vehicle is approaching, when displaying the guiding object.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] The present invention relates to a superimposed image display device that performs driving support for a vehicle. [Background technology]

[0002] Conventionally, various means have been used as information providing means for providing various information for vehicle driving support such as route guidance and warning of obstacles to vehicle occupants. For example, the information is displayed on a liquid crystal display installed in the vehicle, or sound is output from a speaker. In recent years, one such information providing means is a device that provides information by displaying an image superimposed on the surrounding environment (landscape, real scene) of the occupant. For example, a head-up display, a windshield display, or a method of displaying an image superimposed on a captured image of the vehicle surroundings displayed on a liquid crystal display corresponds to the information providing means.

[0003] As one example of driving assistance provided by displaying an image superimposed on the surrounding environment, for example, JP 2015-210644 A discloses a technology for displaying a wall-like object along a dividing line to alert the occupant to the risk of changing lanes when a vehicle is approaching from behind. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2015-210644 A (paragraphs 0024-0026, Figure 2) Summary of the Invention [Problem to be solved by the invention]

[0005] Here, the technology of the above-mentioned Patent Document 1 indicates to the occupant that a rear vehicle is approaching by displaying a wall-like object, but the other vehicle approaching the host vehicle is not limited to the rear vehicle, and may be, for example, a vehicle in front that is decelerating or a vehicle on the side that is changing lanes toward the host vehicle. Also, the contents that the occupant should pay attention to vary greatly depending on the direction from which the other vehicle approaches. However, the above-mentioned Patent Document 1 does not assume the approach of other vehicles other than the rear vehicle, and is not able to indicate to the occupant the direction from which the other vehicle is approaching.

[0006] The present invention has been made to solve the above-mentioned problems in the conventional art, and aims to provide a superimposed image display device that, when there is another vehicle approaching the vehicle, can notify occupants of the risk of changing lanes due to the approach of the other vehicle, and also indicate the direction in which the other vehicle is approaching. [Means for solving the problem]

[0007] In order to achieve the above-mentioned object, the superimposed image display device of the present invention is an superimposed image display device that allows an occupant of the host vehicle to view a guide object that provides information to the occupants of the host vehicle by superimposing it on the scenery around the host vehicle, and has a surrounding condition acquisition means for acquiring the surrounding condition of the host vehicle, and an object display means for displaying a guide object along the dividing line between the host vehicle and the other vehicle when it is determined based on the surrounding condition that another vehicle traveling in a different lane to the host vehicle is approaching in the direction of the host vehicle, and the object display means displays the guide object in an animation in a manner indicating the direction in which the other vehicle is approaching. It should be noted that the "scenery" includes not only the scenery actually viewed from the vehicle (actual scenery), but also an image of the scenery captured, an image of the scenery reproduced, and the like. Effect of the Invention

[0008] According to the superimposed image display device of the present invention having the above-mentioned configuration, when there is another vehicle approaching the own vehicle, it is possible to inform the occupant of the risk of lane shifting due to the approach of the other vehicle by displaying a guide object along the dividing line between the own vehicle and the other vehicle. Furthermore, it is possible to also suggest the direction from which the other vehicle is approaching by displaying an animation of the guide object in a manner indicating the direction from which the other vehicle is approaching. [Brief description of the drawings]

[0009] [Figure 1] 1 is a schematic configuration diagram of a superimposed image display device according to an embodiment of the present invention. [Diagram 2] 1 is a diagram showing an example of a landscape viewed by a vehicle occupant and a virtual image superimposed on the landscape. [Diagram 3] 1 is a block diagram showing a navigation device according to an embodiment of the present invention; [Figure 4] 5 is a flowchart of a driving support processing program according to the present embodiment. [Diagram 5] 11 is a diagram showing an example of a guide object that prompts a driver to move to a recommended lane. FIG. [Figure 6] 11A and 11B are diagrams showing examples of guide objects visually recognized by a vehicle occupant in a situation where another vehicle is approaching; [Figure 7] 13A and 13B are diagrams showing changes in the relative position of a guide object when another vehicle approaches the host vehicle from behind; [Figure 8] 13A and 13B are diagrams showing changes in the relative position of a guide object when another vehicle approaches the host vehicle from ahead. [Figure 9] 13A and 13B are diagrams showing changes in the relative position of a guide object when another vehicle approaches the host vehicle from the side; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, a detailed description will be given of an embodiment of a superimposed image display device 1 according to the present invention with reference to the drawings. First, a schematic configuration of the superimposed image display device 1 according to this embodiment will be described with reference to Fig. 1. Fig. 1 is a schematic configuration diagram of the superimposed image display device 1 according to this embodiment.

[0011] 1, the superimposed image display device 1 basically includes a navigation device 3 mounted on a vehicle 2, and a front display 4 also mounted on the vehicle 2 and connected to the navigation device 3. The front display 4 functions as a head-up display together with a windshield 5 of the vehicle 2 as described below, and serves as an information providing means for providing various information to an occupant 6 of the vehicle 2.

[0012] Here, the navigation device 3 has functions of searching for a recommended route to the destination, displaying a map image of the area around the current position of the vehicle 2 based on map data acquired from a server or stored in memory, and providing driving guidance along a set guide route together with the front display 4. Note that the navigation device 3 does not need to have all of the above functions, and the present invention can be constituted as long as it has at least the function of providing driving guidance along a guide route. Note that the structure of the navigation device 3 will be described in detail later.

[0013] Meanwhile, the front display 4 is a liquid crystal display that is installed inside the dashboard 7 of the vehicle 2 and has a function of displaying images on an image display surface provided on the front. For example, a CCFL (cold cathode fluorescent lamp) or a white LED is used as a backlight. Note that, as the front display 4, in addition to a liquid crystal display, an organic EL display or a combination of a liquid crystal projector and a screen may also be used.

[0014] The front display 4 functions as a head-up display together with the windshield 5 of the vehicle 2, and is configured to reflect the image output from the front display 4 onto the windshield 5 in front of the driver's seat so that it can be viewed by an occupant 6 of the vehicle 2.

[0015] In this embodiment, images displayed on the front display 4 and visually recognized by the occupant 6 of the vehicle 2 include information related to the vehicle 2 and various information used to assist the occupant 6 in driving. For example, there are warnings for objects (other vehicles, pedestrians, guide signs) that are the subject of warning to the occupant 6, guidance information based on the guidance route set by the navigation device 3 and the guidance route (arrows indicating the direction of right or left turns, icons indicating markers of guidance branch points, distance to guidance branch points, position and direction of the recommended lane in which the vehicle should travel, guidance encouraging the driver to change lanes to the recommended lane, etc.), warnings displayed on the road surface (watch out for rear-end collisions, speed limits, etc.), lane markings on which the vehicle is traveling, current vehicle speed, shift position, remaining energy, advertising images, facility information, map images, traffic information, news, weather forecasts, time, the screen of a connected smartphone, television programs, etc. In the embodiment described below, the guidance images include guidance images that indicate the recommended lane in which the vehicle is recommended to travel on the road on which the vehicle is currently traveling in order to travel along the guidance route, guidance images that encourage the vehicle to move into the recommended lane, and wall-shaped images that are displayed to indicate the risk of changing lanes due to the approach of another vehicle when another vehicle is approaching.

[0016] In addition, when the occupant 6 sees the image displayed on the front display 4 reflected by the windshield 5, the image displayed on the front display 4 at a distant position beyond the windshield 5 is perceived by the occupant 6 as a virtual image 10, not at the position of the windshield 5. The virtual image 10 is displayed superimposed on the surrounding environment (landscape, real scene) in front of the vehicle, and can be displayed superimposed on any object (road surface, dividing line, intersection, building, object to be warned, etc.) located in front of the vehicle.

[0017] For example, FIG. 2 shows an example in which the virtual image 10 is displayed superimposed on the surrounding environment (scenery, real scene) in front of the vehicle. As shown in FIG. 2, for example, an image 9A of a line segment extending in the traveling direction of the vehicle and an image 9B of a plurality of wall shapes are displayed on the front display 4. As a result, the image 9A of the line segment and the image 9B of the wall shape displayed on the front display 4 are reflected on the windshield 5 and viewed by the vehicle occupant 6, so that the virtual image 10A of the image 9A of the line segment is superimposed on the scenery through the windshield 5, for example, on the road surface. In addition, the virtual image 10B of the image 9B of the wall shape is superimposed on the position of the division line. However, the virtual image 10 cannot be superimposed anywhere on the scenery through the windshield 5, and the range in which the virtual image 10 can be superimposed and viewed by the vehicle occupant 6 is limited. Specifically, the range is determined by the size of the display screen of the front display 4 and the installation position with respect to the dashboard 7.

[0018] On the other hand, the position in the depth direction where the virtual image 10 is generated, more specifically, the distance L from the occupant 6 to the virtual image 10 (hereinafter referred to as the imaging distance), is determined by the position of the front display 4. For example, the imaging distance L is determined by the distance (optical path length) along the optical path from the position where the image is displayed on the front display 4 to the windshield 5. For example, the optical path length is set so that the imaging distance L is 1.5 m.

[0019] In addition, in this embodiment, the front display 4 installed inside the dashboard 7 is used as a means for displaying an image to be superimposed on the scenery around the vehicle, but other means may be used. For example, a head-up display (HUD) device may be provided as an in-vehicle device, or the windshield 5 may be a transparent liquid crystal display and an image may be displayed directly on the windshield 5. Furthermore, it is also possible to display the scenery around the vehicle captured by a front camera 11 (described later) on a liquid crystal display inside the vehicle, and to display an image to be superimposed on the displayed scenery on the same liquid crystal display. In this case, too, the image displayed on the liquid crystal display becomes an image to be superimposed on the scenery around the vehicle, similar to the HUD.

[0020] A front camera 11 is installed above the front bumper of the vehicle or on the back side of the rearview mirror. The front camera 11 is an imaging device having a camera using a solid-state imaging element such as a CCD, and is installed with its optical axis direction facing forward in the traveling direction of the vehicle. Then, image processing is performed on the image captured by the front camera 11, so that the situation of the front environment (i.e., the environment on which the virtual image 10 is superimposed) visually recognized by the occupant 6 through the windshield is detected. Note that a sensor such as a millimeter wave radar may be used instead of the front camera 11. The image captured by the front camera 11 is also used to detect dividing lines on the road on which the vehicle is traveling, to detect obstacles, and to identify the lane on which the vehicle is currently traveling, as described later.

[0021] An in-vehicle camera 12 is installed on the top surface of the vehicle's instrument panel. The in-vehicle camera 12 is an imaging device having a camera using a solid-state imaging element such as a CCD, and is installed with its optical axis direction facing the driver's seat. A range in the vehicle where the face of an occupant is generally expected to be located is set as a detection range (imaging range of the in-vehicle camera 12), and an image of the face of the occupant 6 sitting in the driver's seat is captured. Then, image processing is performed on the captured image captured by the in-vehicle camera 12, thereby detecting the eye position (starting point of gaze) and gaze direction of the occupant 6.

[0022] Next, a schematic configuration of the navigation device 3 constituting the above-mentioned superimposed image display device 1 will be described with reference to Fig. 3. Fig. 3 is a block diagram showing the navigation device 3 according to this embodiment.

[0023] As shown in Fig. 3, the navigation device 3 according to this embodiment includes a current position detection unit 13 for detecting the current position of the vehicle 2 on which the navigation device 3 is mounted, a data recording unit 14 on which various data are recorded, a navigation ECU 15 for performing various calculation processes based on input information, an operation unit 16 for receiving operations from a user, a liquid crystal display 17 for displaying a map of the vehicle's surroundings and facility information on facilities to the user, a speaker 18 for outputting voice guidance on route guidance, a DVD drive 19 for reading a DVD as a storage medium, and a communication module 20 for communicating with an information center such as a probe center or a VICS (registered trademark: Vehicle Information and Communication System) center. The navigation device 3 is also connected to the front display 4, the front camera 11, and the in-vehicle camera 12 described above via an in-vehicle network such as a CAN. In addition, the navigation device 3 is also connected to an outside camera 25 installed in addition to the front camera 11 so as to be able to capture the surrounding environment of the vehicle, and a vehicle control ECU 26 for performing various controls on the vehicle on which the navigation device 3 is mounted, so as to be able to communicate with each other.

[0024] Each of the components of the navigation device 3 will be described below in order. The current position detection unit 13 is composed of a GPS 21, a vehicle speed sensor 22, a steering sensor 23, a gyro sensor 24, etc., and is capable of detecting the current vehicle position, direction, vehicle running speed, current time, etc. Here, the vehicle speed sensor 22 in particular is a sensor for detecting the travel distance and vehicle speed of the vehicle, and generates pulses in response to the rotation of the drive wheels of the vehicle and outputs the pulse signal to the navigation ECU 15. The navigation ECU 15 then calculates the rotation speed and travel distance of the drive wheels by counting the generated pulses. Note that the navigation device 3 does not need to be equipped with all of the above four types of sensors, and the navigation device 3 may be configured to be equipped with only one or more types of sensors among these.

[0025] The data recording unit 14 also includes a hard disk (not shown) as an external storage device and recording medium, and a recording head (not shown) which is a driver for reading out the map information DB 31 and predetermined programs recorded on the hard disk and writing predetermined data to the hard disk. The data recording unit 14 may have a flash memory, a memory card, or an optical disk such as a CD or a DVD instead of the hard disk. The map information DB 31 may also be stored in an external server and the navigation device 3 may acquire it through communication.

[0026] Here, the map information DB 31 is a storage means that stores, for example, link data 32 relating to roads (links), node data 33 relating to node points, branch point data 34 relating to branch points, point data relating to points such as facilities, map display data for displaying a map, search data for searching for routes, search data for searching for points, etc.

[0027] The link data 32 includes, for each link constituting the road network, data representing the width, gradient, cant, bank, road surface condition, and shape complement point data for identifying the link shape between nodes (for example, the shape of the curve in the case of a curved road), the number of lanes on the road, the lane width of each lane, the traffic division in the direction of travel for each lane, the presence or absence of an oncoming lane (whether there is a two-way traffic section or not), the presence or absence of a central divider, points where the road width narrows, and railroad crossings; for corners, data representing the radius of curvature, intersections, T-junctions, corner entrances and exits, etc.; for road attributes, data representing downhill roads, uphill roads, etc.; and for road types, data representing expressways and general roads (national highways, prefectural roads, narrow streets, etc.).

[0028] The node data 33 also includes data regarding the coordinates (positions) of actual road branching points (including intersections, T-junctions, etc.) and node points set at predetermined distances on each road depending on the radius of curvature, etc., node attributes indicating whether the node corresponds to an intersection, etc., a connecting link number list which is a list of link numbers of links connecting to the node, an adjacent node number list which is a list of node numbers of nodes adjacent to the node via links, and the height (altitude) of each node point, etc.

[0029] In addition, the branch point data 34 stores the intersection name of the branch point, corresponding node information that identifies the node that forms the branch point, connecting link information that identifies the link connected to the branch point, the direction name corresponding to the link connected to the branch point, information that identifies the shape of the branch point, etc.

[0030] On the other hand, the navigation ECU (electronic control unit) 15 is an electronic control unit that controls the entire navigation device 3, and includes a CPU 41 as an arithmetic device and a control device, a RAM 42 that is used as a working memory when the CPU 41 performs various arithmetic processing and stores route data when a route is searched, a ROM 43 that stores a driving support processing program (FIG. 4) described later as well as a control program, and an internal storage device such as a flash memory 44 that stores a program read from the ROM 43. The navigation ECU 15 has various means as processing algorithms. For example, the surrounding situation acquisition means acquires the surrounding situation of the host vehicle. The object display means displays a guide object along a dividing line between the host vehicle and the other vehicle when it is determined based on the surrounding situation that another vehicle traveling in a lane different from the host vehicle is approaching in the direction of the host vehicle.

[0031] The operation unit 16 is operated when inputting a departure point as a travel start point and a destination point as a travel end point, and has a plurality of operation switches (not shown) such as various keys and buttons. The navigation ECU 15 performs control to execute various corresponding operations based on switch signals output by pressing each switch. The operation unit 16 may have a touch panel provided on the front surface of the liquid crystal display 17. It may also have a microphone and a voice recognition device.

[0032] The liquid crystal display 17 also displays map images including roads, traffic information, operation guidance, operation menus, key guidance, guided route from the departure point to the destination, guidance information along the guided route, news, weather forecasts, time, e-mail, television programs, etc. In this embodiment, the front display 4 is provided as information display means, and therefore the liquid crystal display 17 may be omitted if the map images and the like are displayed on the front display 4.

[0033] The speaker 18 also outputs voice guidance for guiding the vehicle along a guide route based on instructions from the navigation ECU 15, and traffic information guidance.

[0034] The DVD drive 19 is a drive capable of reading data recorded on a recording medium such as a DVD or a CD. Based on the read data, music and video are played, and the map information DB 31 is updated. Instead of the DVD drive 19, a card slot for reading and writing data into a memory card may be provided.

[0035] The communication module 20 is a communication device for receiving traffic information consisting of various information such as congestion information, regulation information, and traffic accident information transmitted from a traffic information center, such as a VICS center or a probe center, and is, for example, a mobile phone or a DCM.

[0036] On the other hand, the exterior camera 25 is composed of a camera using a solid-state image sensor such as a CCD, and is attached to the bumper, door mirror, etc. of the vehicle, and the optical axis direction is set so that the surrounding environment of the vehicle can be captured. The number of exterior cameras 25 may be one or more. The exterior camera 25, together with the front camera 11, captures images of the front, side, and rear of the vehicle in the traveling direction while the vehicle is traveling, and in particular detects other vehicles traveling around the vehicle. However, it is not necessarily necessary to set the imaging range to the entire front, side, and rear of the vehicle in the traveling direction. In addition to the camera, sensors such as millimeter wave radar and laser sensors, vehicle-to-vehicle communication, and road-to-vehicle communication may be used to detect other vehicles.

[0037] The vehicle control ECU 26 is an electronic control unit that controls the vehicle equipped with the navigation device 3. The vehicle control ECU 26 is connected to each driving part of the vehicle such as the steering, brake, accelerator, etc., as well as a shift lever sensor, direction indicators, etc., and in this embodiment, the navigation device 3 is capable of acquiring driving behavior (vehicle behavior) of the driver who drives the vehicle from the vehicle control ECU 26 via the CAN. For example, steering operation, accelerator and brake operation, shift position operation, turn on of direction indicators, etc.

[0038] Next, a driving support processing program executed by the navigation ECU 15 in the navigation device 3 having the above configuration will be described with reference to Fig. 4. Fig. 4 is a flowchart of the driving support processing program according to this embodiment. Here, the driving support processing program is executed after the ACC power supply (accessory power supply) of the vehicle is turned on, and is a program that performs driving support for the vehicle by allowing the image displayed on the front display 4 to be visually recognized by superimposing it on the scenery (actual scene) around the vehicle. The program shown in the flowchart in Fig. 4 below is stored in the RAM 42 and ROM 43 of the navigation device 3, and is executed by the CPU 41.

[0039] In the following description, an example of providing vehicle driving guidance along a guide route set by the navigation device 3 as vehicle driving support using a guide object will be described. In addition, the guide object to be displayed may be a guide object indicating a recommended lane in which driving is recommended on the road on which the vehicle is currently traveling in order to travel along the guide route, or a guide object that encourages the vehicle to move to the recommended lane, while in addition, when another vehicle approaches, a wall-shaped guide object that suggests the risk of lane changing due to the approach of another vehicle is displayed. However, the superimposed image display device 1 can also provide guidance and information other than the above-mentioned driving support using a guide object. In addition, the guide object to be displayed at that time can be other than the guide object that provides the above-mentioned recommended lane guidance or the guide object that suggests the risk of lane changing. For example, guidance objects can be used to display warnings to occupants about objects that should be warned about (other vehicles, pedestrians, guide signs), warnings to be displayed on the road surface (watch out for rear-end collisions, speed limits, etc.), the direction of travel to the next guidance junction, current vehicle speed, shift position, remaining energy, advertising images, facility information, guide signs, map images, traffic information, news, weather forecasts, time, the screen of a connected smartphone, etc.

[0040] First, in the driving support processing program, in step (hereinafter abbreviated as S) 1, the CPU 41 judges whether the vehicle is in a situation where it should change lanes. Here, the situation where the vehicle is in a situation where it should change lanes includes, for example, a situation where the vehicle is not currently traveling in a recommended lane that is recommended for traveling on the road on which the vehicle is currently traveling in order to travel along a guide route, and a guide object 51 that encourages the vehicle to move to the recommended lane is displayed as shown in Fig. 5. Alternatively, even if a guide route is not set or the guide object 51 that encourages the vehicle to move to the lane is not displayed, if it is predicted that the vehicle will change lanes based on the vehicle behavior acquired via the vehicle control ECU 26 (for example, the turn signal is turned on even though there is no intersection nearby, or a steering operation is started, etc.), it is judged that the vehicle is in a situation where it should change lanes.

[0041] If it is determined that the vehicle is in a situation where it needs to change lanes (S1: YES), the process proceeds to S2. On the other hand, if it is determined that the vehicle is not in a situation where it needs to change lanes (S1: NO), the current guidance is continued, and the driving support processing program is terminated.

[0042] Next, in S2, the CPU 41 detects the relative position and relative speed of other vehicles located around the vehicle based on the captured images of the front camera 11 and the exterior camera 25. Sensors such as millimeter wave radar and laser sensors, vehicle-to-vehicle communication, and road-to-vehicle communication may be used. Furthermore, position information and speed information of other vehicles may be obtained from an external server (probe server) that collects information on each vehicle traveling on the road, and the relative position and relative speed of other vehicles located around the vehicle may be detected based on the information obtained from the external server. The detection of the relative speed based on the captured images can be calculated based on, for example, the amount of change in the position of the other vehicles between frames. Furthermore, when there are multiple other vehicles around the vehicle, the relative position and relative speed are obtained for each of the other vehicles.

[0043] Next, in S3, the CPU 41 determines whether or not there is another vehicle approaching the host vehicle, more specifically, whether or not there is another vehicle that may be a risk factor for the host vehicle about to change lanes. Specifically, the determination is made as follows.

[0044] First, the CPU 41 calculates the travel trajectory of the host vehicle from the current time until a predetermined time has elapsed (for example, 3 seconds) assuming that the host vehicle has changed lanes following the guidance. Similarly, the CPU 41 calculates the travel trajectory of the other vehicle from the current time until a predetermined time has elapsed based on the relative position and relative speed of the other vehicle acquired in S2. Then, the CPU 41 compares the calculated travel trajectory of the host vehicle with the travel trajectory of the other vehicle, and if it is determined that the two vehicles intersect, or if they do not intersect but approach within a predetermined distance (for example, 5 m), it determines that there is another vehicle that is a risk factor for the lane change that the host vehicle is about to make (S3: YES). Note that other vehicles traveling in the same lane as the host vehicle are basically excluded because they do not pose a risk for lane change, but for example, other vehicles that are expected to change lanes afterwards even if they are traveling in the same lane as the host vehicle may be included as targets. In addition, other vehicles traveling in a different lane from the host vehicle (other vehicles traveling in the oncoming lane) are excluded because they do not pose a risk for lane change.

[0045] Then, if it is determined that there is another vehicle approaching the vehicle, more specifically, another vehicle that is a risk factor for the lane change that the vehicle is about to make (S3: YES), the process proceeds to S4. On the other hand, if it is determined that there is no other vehicle approaching the vehicle, more specifically, another vehicle that is a risk factor for the lane change that the vehicle is about to make (S3: NO), the current guidance is continued and the driving support processing program is terminated. In addition, if no other vehicle was detected around the vehicle, the process also returns to S3: NO.

[0046] Next, in S4, the CPU 41 identifies the lane in which the host vehicle is traveling (hereinafter referred to as the host vehicle lane) and the lane in which the other vehicle determined to be approaching the host vehicle in S3 is traveling (hereinafter referred to as the other vehicle lane). Basically, the "host vehicle lane" and the "other vehicle lane" are identified by performing image processing on the images captured by the front camera 11 and the exterior camera 25. Specifically, the "host vehicle lane" and the "other vehicle lane" are identified by the following processing.

[0047] First, the CPU 41 recognizes (detects) features located around the vehicle by performing image processing on the images captured by the front camera 11 and the outside camera 25. Specifically, the recognition (detection) targets are the dividing lines drawn on the road surface (including the road surface of the lane in which the vehicle is traveling, as well as the road surface of lanes other than the lane in which the vehicle is traveling) and the road edge (specifically, the edge of the roadway, which is the boundary between the roadway and the sidewalk if there is a sidewalk). It is desirable to detect the color and type (solid line, dashed line, etc.) of the dividing lines. Regarding the road edge, basically, structures such as blocks, guard rails, and median strips provided at the road edge are detected as the road edge, but for roads without such structures at the road edge, a break in the asphalt or the outermost dividing line (if the lane is divided by a center line from the oncoming lane), the center line may also be detected as the road edge. Then, in addition to the detection results of the dividing lines and road edges, map information is also referred to to identify the lane in which the vehicle is traveling. The lane of other vehicles is identified by using the detected dividing line, the identified lane of the own vehicle, and the relative position of the other vehicle with respect to the own vehicle. For example, it is possible to identify that the road on which the own vehicle is traveling has a total of three lanes (excluding the oncoming lane), the own vehicle is traveling in the leftmost lane, and the other vehicle is traveling in the center lane. Note that the map information contains information for each link that identifies the presence or absence of an oncoming lane and the number of lanes.

[0048] However, the method of identifying the vehicle's own lane and other vehicle's lane is not limited to the above method, and it is also possible to identify the lane by detecting features painted on the road surface other than dividing lines (e.g., traffic divisions based on the direction of travel).

[0049] Next, in S5, the CPU 41 identifies a dividing line between the host vehicle and the other vehicle based on the host vehicle lane and the other vehicle lane identified in S4. The dividing lines included in the road on which the vehicle is traveling are also identified in the process of identifying the host vehicle lane and the other vehicle lane in S4. If the host vehicle lane and the other vehicle lane are adjacent to each other, there is only one dividing line between the host vehicle lane and the other vehicle lane, and if there is another lane between the host vehicle lane and the other vehicle lane, there may be two or more dividing lines between the host vehicle and the other vehicle.

[0050] Thereafter, in S5, the CPU 41 determines, based on the relative position and relative speed of the other vehicle acquired in S2, whether or not the other vehicle determined in S3 to be approaching the host vehicle is approaching, particularly from behind, the host vehicle.

[0051] If the vehicle determined to be approaching the host vehicle in S3 is determined to be approaching from behind the host vehicle (S6: YES), the process proceeds to S7. On the other hand, if the vehicle determined to be approaching the host vehicle in S3 is determined to be approaching from a position other than the rear of the host vehicle (S6: NO), the process proceeds to S9.

[0052] In S7, the CPU 41 determines the position (range) for displaying the guide object on the front display 4. The guide object to be displayed in S7 and thereafter is a plurality of (e.g., three) wall-shaped guide objects superimposed on the road surface at the center of the lane in which the vehicle is currently traveling, superimposed on a line segment extending in the direction of travel of the vehicle and a dividing line between the vehicle and another vehicle, and arranged along the dividing line. Specifically, the following process is performed in S7.

[0053] First, the CPU 41 detects the eye positions of the vehicle occupants based on the captured image captured by the in-vehicle camera 12. Next, the position of the windshield 5 onto which the image is to be projected by the front display 4 is obtained. The eye positions of the vehicle occupants and the position of the windshield 5 are specified by three-dimensional position coordinates. Next, the CPU 41 obtains the position at which the guidance object to be displayed on the front display 4 is to be superimposed on the scenery. As described above, in S7, the road surface along the center of the planned travel lane and the dividing line between the vehicle and another vehicle (if there are multiple dividing lines, the dividing line closest to the vehicle is used, but this does not apply when another vehicle approaches from the side as described later) are used. Next, the CPU 41 determines a projection range of the guidance object on the windshield 5 based on the identified eye positions of the vehicle occupants, the position of the windshield 5, and the position where the guidance object is to be superimposed. Furthermore, the CPU 41 also determines a display range of the guidance object on the front display 4 from the determined projection range. Note that the shape of the guidance object to be displayed on the front display 4 can be specified, for example, by arranging the guidance object in a three-dimensional space that reproduces the periphery of the vehicle, and calculating the shape when viewed from a position corresponding to the current position of the vehicle in the three-dimensional space.

[0054] Next, in S8, the CPU 41 transmits a control signal to the front display 4, and displays an image of the guide object on the front display 4 in the display range determined in S7. As a result, it becomes possible for the vehicle occupant to visually recognize the guide object (more precisely, a virtual image of the guide object, the same applies below) superimposed on the scenery. Specifically, the virtual image of the guide object visually recognized by the occupant in S8 is superimposed on the road surface near the center of the host vehicle driving lane 52 in the scenery as shown in FIG. 6, and is a line segment object 53 extending in the traveling direction of the host vehicle, and a plurality of wall-shaped objects 55 superimposed on a dividing line 54 on the right side of the host vehicle driving lane. Note that the example shown in FIG. 6 is particularly a case where the other vehicle driving lane is adjacent to the right side of the host vehicle driving lane, and conversely, when the other vehicle driving lane is adjacent to the left side of the host vehicle driving lane, the wall-shaped object 55 is superimposed on the dividing line on the left side of the host vehicle driving lane. In addition, the thickness (length in the road width direction) of the line segment object 53 corresponds to the vehicle width of the host vehicle. On the other hand, the height and width of the wall-shaped object 55 are adjusted (for example, 1 m wide and 1.5 m high) so as not to significantly obstruct the occupant's field of vision, and furthermore, the walls are arranged with a predetermined distance (for example, 1 m) between them along the traveling direction of the vehicle. Furthermore, the transmittance when displaying the wall-shaped object 55 is set to a semi-transparent state so that the part hidden by the virtual image of the object 55 can be seen by the occupant. For example, the transmittance is set to 50%. Then, the process proceeds to S12.

[0055] Here, the line object 53 displayed in S8 does not necessarily indicate the recommended lane on which the vehicle is currently traveling so that the vehicle travels along the guided route. That is, even if the vehicle is not currently traveling in the recommended lane and the guide object 51 urging the vehicle to move to the recommended lane is displayed as shown in Fig. 5, if it is determined that another vehicle will approach the vehicle before the vehicle follows the guide to change lanes (S3: YES), there is a risk of the vehicle changing lanes, so the guide urging the vehicle to move to the recommended lane is stopped and the guide is temporarily changed to one to continue traveling in the current lane. In that case, after the vehicle has avoided the other vehicle, the display of the guide object 51 urging the vehicle to move to the recommended lane as shown in Fig. 5 is restored.

[0056] In this embodiment, the wall-shaped object 55 is not displayed with a fixed relative position from the vehicle itself, but is animated to indicate the direction in which the other vehicle approaches. More specifically, an animation is displayed so that the object 55 moves from the direction in which the other vehicle approaches toward the vehicle itself, so that the object 55 can be seen by the occupant. That is, when it is determined that the other vehicle approaches from the rear of the vehicle itself (S6: YES), an animation is displayed in which the wall-shaped object 55 moves (relatively) from the rear to the front (toward the vehicle itself) of the vehicle itself, as shown in FIG. 7. After the wall-shaped object 55 moves to a position on the right front of the vehicle (a position that is most easily visible to the occupant), the relative position is fixed (animation display is terminated). The moving speed when the relative position of the wall-shaped object 55 is moved is set to be proportional to the relative speed of the other vehicle determined to approach. That is, the faster the relative speed of the other vehicle is, the faster the wall-shaped object 55 moves.

[0057] On the other hand, in S9, the CPU 41 determines whether or not the other vehicle determined to approach the host vehicle in S3 is approaching the host vehicle, particularly from the front, based on the relative position and relative speed of the other vehicle acquired in S2.

[0058] If the vehicle determined to be approaching the host vehicle in S3 is determined to be approaching from the front of the host vehicle (S9: YES), the process proceeds to S10. On the other hand, if the vehicle determined to be approaching the host vehicle in S3 is determined to be approaching from the side of the host vehicle, not from the front or rear of the host vehicle (S9: NO), the process proceeds to S11.

[0059] In S10, the CPU 41 determines a position (range) for displaying the guide object on the front display 4. Then, the process proceeds to S8, in which the CPU 41 transmits a control signal to the front display 4, and displays an image of the guide object on the front display 4 in the display range determined in S10. As a result, it becomes possible for the vehicle occupants to visually recognize the guide object (more precisely, a virtual image of the guide object) superimposed on the scenery.

[0060] The guide objects to be displayed in S10 and after are the object 53 of a line segment that is superimposed on the road surface at the center of the lane on which the vehicle is currently traveling and extends in the direction of travel of the vehicle, as in S7, and a plurality of (e.g., three) wall-shaped objects 55 that are superimposed on the dividing line between the vehicle and the other vehicle and arranged along the dividing line (FIG. 6). However, since the other vehicle approaches from the front, not the rear, an animation of the wall-shaped object 55 moving (relatively) from the front to the rear (toward the vehicle) of the vehicle is displayed in S10 and after, as shown in FIG. 8. After the wall-shaped object 55 moves to a position on the right front of the vehicle (a position that is most easily seen by the occupants), the relative position is fixed (animation display is terminated). The moving speed when moving the relative position of the wall-shaped object 55 is set to be proportional to the relative speed of the other vehicle that is determined to approach. That is, the faster the relative speed of the other vehicle is, the faster the wall-shaped object 55 moves.

[0061] On the other hand, in S11, the CPU 41 determines a position (range) for displaying the guide object on the front display 4. Then, the process proceeds to S8, where the CPU 41 transmits a control signal to the front display 4, and displays an image of the guide object on the front display 4 in the display range determined in S11. As a result, it becomes possible for the vehicle occupants to visually recognize the guide object (more precisely, a virtual image of the guide object) superimposed on the scenery.

[0062] The guide objects to be displayed from S11 onwards are, as in S7, a line object 53 superimposed on the road surface at the centre of the lane in which the vehicle is currently travelling and extending in the direction of travel of the vehicle, and a plurality of (e.g. three) wall-shaped objects 55 superimposed on the dividing line between the vehicle and the other vehicle and arranged along the dividing line (Fig. 6). However, since the other vehicle approaches from the side, not from behind, from S11 onwards, an animation is displayed in which the wall-shaped object 55 moves (relatively) towards the vehicle in the road width direction, as shown in Fig. 9. In the example shown in FIG. 9, the other vehicle lane is the lane two lanes to the right of the host vehicle lane (with one lane in between) and approaches the host vehicle by moving to the left lane, i.e., the other vehicle approaches from the right side. The wall-shaped object 55 is initially superimposed on the left dividing line of the other vehicle lane, and the relative position is fixed after the other vehicle finally moves to the position of the right dividing line of the host vehicle lane (animation display ends). In addition, if the relative position of the other vehicle to the host vehicle is reversed, i.e., if the other vehicle approaches from the left side, the arrangement of the wall-shaped object 55 is also reversed from the example shown in FIG. 9. The moving speed when moving the relative position of the wall-shaped object 55 is set to be proportional to the relative speed of the other vehicle determined to approach. In other words, the faster the relative speed of the other vehicle is, the faster the wall-shaped object 55 moves.

[0063] After that, the processes of S6 to S11 are repeated until the approaching other vehicle is avoided (S12: YES). If the position or direction of the vehicle changes, the position and shape of the guide object will change accordingly. In addition, in S12, the determination as to whether the approaching other vehicle has been avoided is made in the opposite manner to S3, and the vehicle's travel path from the present time until a predetermined time has elapsed is compared with the travel path of the other vehicle. If it is determined that the two do not intersect and will not approach within a predetermined distance (for example, 5 m), it is determined that the other vehicle has been avoided.

[0064] Then, when it is determined that the approaching other vehicle has been avoided (S12: YES), the displayed guide object is hidden. When the wall-shaped object 55 is hidden, the transparency may be gradually increased to finally hide it, or the size may be gradually reduced to finally hide it. In addition, if the guide object 51 encouraging the driver to move to the recommended lane as shown in FIG. 5 was displayed before the other vehicle approached, the display of the guide object 51 encouraging the driver to move to the recommended lane will be restored after the other vehicle has been avoided.

[0065] As a result of the above processing, in the superimposed image display device 1 of the present embodiment, when another vehicle approaches while the vehicle is changing lanes, a wall-shaped object 55 is displayed along the dividing line to indicate the risk of changing lanes due to the approach of the other vehicle (FIG. 5). This allows the occupant to visually and easily understand that it is better not to change lanes in the direction of the wall-shaped object, and it is possible to indicate to the occupant the risk of lane movement due to the approach of the other vehicle. Furthermore, the wall-shaped object 55 does not have a fixed relative position with respect to the vehicle, but can be changed relative to the vehicle to visually indicate the direction in which the other vehicle is approaching in an easily understandable manner. In addition, the wall-shaped object 55 is divided into a plurality of wall shapes and has a high transmittance, so that the object 55 can be prevented from blocking the occupant's view.

[0066] As described above in detail, the superimposed image display device 1 according to the present embodiment and the computer program executed by the superimposed image display device 1 acquire the surrounding conditions of the vehicle (S2, S3), and when it is determined based on the surrounding conditions that another vehicle traveling in a lane different from the vehicle approaches the direction of the vehicle (S3: YES), a guide object is displayed along the dividing line between the vehicle and the other vehicle (S7, S8, S10, S11). Furthermore, when the guide object is displayed, it is animated to indicate the direction in which the other vehicle approaches, so that the displayed guide object can suggest to the occupant the risk of lane shifting due to the approach of the other vehicle. Furthermore, by displaying the guide object as an animation to indicate the direction in which the other vehicle approaches, it is also possible to suggest the direction in which the other vehicle approaches. In addition, an animation is displayed in which the guide object moves from the direction in which the other vehicle is approaching toward the vehicle itself, so that a passenger who sees the guide object can easily visually grasp the direction in which the other vehicle is approaching. In addition, the relative speed of the other vehicle to the vehicle itself is obtained (S2), and the faster the relative speed of the other vehicle is, the faster the animation of the guide object is displayed, so that a passenger who sees the guide object can easily visually grasp the speed at which the other vehicle is approaching. In addition, the guide object is an object having a plurality of wall shapes arranged at predetermined intervals along the traveling direction of the vehicle, so that the occupant can easily visually understand that it is better not to change lanes in the direction of the wall-shaped object, while the occupant's view is prevented from being obstructed by the object.

[0067] Incidentally, the present invention is not limited to the above-described embodiment, and it goes without saying that various improvements and modifications are possible without departing from the spirit and scope of the present invention. For example, in this embodiment, a head-up display system is used as a means for displaying an image to be superimposed on the scenery around the vehicle, but the windshield 5 may be a transparent liquid crystal display and an image may be displayed directly on the windshield 5. Furthermore, it is also possible to display the scenery around the vehicle captured by a front camera 11 (described later) on a liquid crystal display inside the vehicle, and to display an image to be superimposed on the displayed scenery on the same liquid crystal display. In this case, too, the image displayed on the liquid crystal display is an image to be superimposed on the scenery around the vehicle, similar to the HUD.

[0068] In addition, in this embodiment, the front display 4 is configured to generate a virtual image in front of the windshield 5 of the vehicle 2, but the virtual image may be generated in front of a window other than the windshield 5. Furthermore, the target onto which the image is reflected by the front display 4 may not be the windshield 5 itself, but a visor (combiner) installed around the windshield 5.

[0069] In addition, in this embodiment, a wall-shaped guide object is displayed when it is determined that the vehicle is in a situation where it needs to change lanes, for example, when guidance is being given to encourage the vehicle to move lanes. However, regardless of whether the vehicle is in a situation where it needs to change lanes, a wall-shaped guide object may also be displayed if there is another vehicle approaching the vehicle.

[0070] In this embodiment, the relative position of the wall-shaped guide object with respect to the vehicle is changed to indicate the direction in which the other vehicle is approaching, but instead of changing the relative position, the direction in which the other vehicle is approaching may be indicated by, for example, changing the shape or display color of the guide object.Similarly, the relative speed of the other vehicle may be indicated by, for example, changing the shape or display color of the guide object instead of changing the moving speed of the guide object.

[0071] It is also possible to indicate the distance from the vehicle to another vehicle by changing the display mode of the wall-shaped guide object. For example, when the other vehicle is far away from the vehicle, the guide object can be displayed in blue, and then as the other vehicle approaches the vehicle, the display color can be changed in the order of blue, yellow, and red (changing to colors that call for greater caution).

[0072] In addition, in this embodiment, the wall-shaped guide objects are three wall-shaped objects arranged along the traveling direction of the vehicle, but the number of wall-shaped objects may be four or more. Also, the shape of the wall-shaped objects may be a shape other than a rectangular shape.

[0073] In addition, in this embodiment, the guide objects displayed are a line object 53 extending in the direction of travel of the vehicle and a wall-shaped object 55 superimposed on the lane markings, but it is also possible to display only the wall-shaped object 55.

[0074] In this embodiment, the navigation ECU 15 of the navigation device 3 executes the driving support processing program (FIG. 4), but the executing entity can be changed as appropriate. For example, the control unit of the front display 4, the vehicle control ECU, or other in-vehicle device may execute the program. [Explanation of symbols]

[0075] Reference Signs List 1: superimposed image display device, 2: vehicle, 3: navigation device, 4: front display, 5: windshield, 6: occupant, 11: front camera, 25: exterior camera, 15: navigation ECU, 41: CPU, 42: RAM, 43: ROM, 51, 53, 55: guidance object

Claims

1. A superimposed image display device for displaying information to a vehicle occupant in a vehicle by superimposing a guide object on a landscape around the vehicle, the superimposed image display device comprising: A surrounding condition acquisition means for acquiring a surrounding condition of the host vehicle; and an object display means for displaying a guide object along a lane line between the host vehicle and the other vehicle when it is determined based on the surrounding conditions that the other vehicle traveling in a different lane from the host vehicle is approaching in the direction of the host vehicle, The object display means is a superimposed image display device that displays the guidance object in an animation in a manner indicating the direction in which the other vehicle is approaching.

2. The superimposed image display device according to claim 1 , wherein the object display means displays an animation in which the guide object moves from a direction in which the other vehicle is approaching toward the host vehicle.

3. A speed acquisition means for acquiring a relative speed of another vehicle with respect to the own vehicle, 3. The superimposed image display device according to claim 2, wherein the object display means performs animation display in which the guide object moves faster as the relative speed of the other vehicle increases.

4. 4. The superimposed image display device according to claim 1, wherein the guide object is a plurality of wall-shaped objects arranged at predetermined intervals along the traveling direction of the host vehicle.

Images